Image processing method, image processing apparatus, and computer program

Abstract

To provide a technology capable of acquiring a desired image processing sequence optimal for training a machine learning model.SOLUTION: An image processing method includes a step in which a maximum output evaluation value specifies a machine learning model of a calculation source, and an image processing sequence used to generate an intermediate image group inputted to the machine learning model of the calculation source, and a step (a) includes a step for generating a new sequence set by replacing one or more image processing sequences having a low performance evaluation value with an image processing sequence composed of a new combination of sequence elements in the case that the maximum output evaluation value is less than a threshold.SELECTED DRAWING: Figure 10

Description

Technical Field

[0001] The present disclosure relates to the technologies of an image processing method, an image processing apparatus, and a computer program.

Background Art

[0002] Conventionally, there is known a technique of generating a plurality of image processing sequences by combining a plurality of image processing filters and determining an image processing sequence that can obtain a desired image by image processing among the plurality of image processing sequences (Patent Document 1). Further, conventionally, regarding the recognition of an object using a neural network type machine learning model, after performing preprocessing such as alignment using feature points in an image, learning of the machine learning model is performed using the preprocessed image (Patent Document 2).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the technique of Patent Document 1, by using genetic programming, a plurality of image processing sequences are generated and evaluation is repeatedly performed for each generation, thereby obtaining an image processing sequence with high evaluation. However, in the technique of Patent Document 1, since evaluation is performed only on an image processing sequence obtained by combining a plurality of image processing filters, in a complex process in which image processing by an image processing sequence and other processing different from the image processing are combined, it may be difficult to accurately obtain a desired processing result.

[0005] In the technology described in Patent Document 2, multiple machine learning models are trained and integrated for each of the multiple types of preprocessing. Thus, in the technology described in Patent Document 2, it is necessary to train and integrate multiple machine learning models corresponding to multiple types of preprocessing, which may make it difficult to obtain the optimal preprocessing for obtaining the desired processing result.

[0006] Therefore, there has been a long-standing need for a technology that combines image processing using image processing sequences with post-processing using machine learning models, in order to obtain an image processing sequence that is optimal for training a machine learning model. [Means for solving the problem]

[0007] According to a first embodiment of this disclosure, an image processing method is provided. This image processing method comprises: (a) generating a sequence set comprising a plurality of image processing sequences, each having a different combination of (i) an image input layer for inputting an image, (ii) at least one of a plurality of image processing layers, and (iii) an image output layer for outputting the image after image processing; (b) processing a plurality of training images with each of the plurality of image processing sequences, and generating a group of intermediate images, which is a set of intermediate images, for each of the plurality of image processing sequences; (c) inputting the group of intermediate images for each of the plurality of image processing sequences into the input layer of a corresponding plurality of neural network type machine learning models having an input layer, an intermediate layer, and an output layer, and training the machine learning model so that the layer output, which is the output of the output layer, approaches a target output that is a predetermined target for each of the plurality of training images; and (d) the corresponding plurality The process includes: (a) For each machine learning model, a step of comparing the layer output with the target output to calculate an output evaluation value indicating the degree of similarity between the layer output and the target output; (e) If the maximum output evaluation value, which is the highest of the multiple output evaluation values ​​calculated for each of the multiple machine learning models, is higher than a predetermined threshold, a step of identifying the machine learning model from which the maximum output evaluation value was calculated and the image processing sequence used to generate the intermediate image group input to the machine learning model from which the calculation was made; and if the maximum output evaluation value is less than the threshold, a step of generating a new sequence set by replacing one or more of the generated multiple image processing sequences that have a low performance evaluation value regarding the prediction accuracy of the machine learning model with an image processing sequence composed of a new combination of sequence elements; and steps (b) to (e) are executed again on the new sequence set.

[0008] According to a second embodiment of this disclosure, an image processing apparatus is provided. This image processing apparatus comprises: a sequence generation unit that generates a sequence set consisting of a plurality of image processing sequences, each having a different combination of (i) an image input layer for inputting an image, (ii) at least one of a plurality of image processing layers, and (iii) an image output layer for outputting the image after image processing; an image processing unit that processes a plurality of training images using each of the plurality of image processing sequences and generates an intermediate image group for each of the plurality of image processing sequences, which is a set of a plurality of intermediate images that are the images after image processing; and inputs the intermediate image group for each of the plurality of image processing sequences into the input layer of a plurality of machine learning models of the neural network type, which has an input layer, an intermediate layer, and an output layer, so that the layer output, which is the output of the output layer, approaches a predetermined target output for each of the plurality of training images. The system includes a learning unit that performs learning of the machine learning model, and an evaluation unit that, for each of the multiple machine learning models, compares the layer output with the target output and calculates an output evaluation value indicating the degree of similarity between the layer output and the target output. The evaluation unit identifies the machine learning model from which the maximum output evaluation value, which is the highest of the multiple output evaluation values ​​calculated for each of the multiple machine learning models, is higher than a predetermined threshold, and identifies the machine learning model from which the maximum output evaluation value was calculated and the image processing sequence used to generate the intermediate image group input to the machine learning model from which the maximum output evaluation value was calculated. The sequence generation unit generates a new sequence set by replacing one or more image processing sequences from the generated multiple image processing sequences that have a low performance evaluation value regarding the prediction accuracy of the machine learning model with an image processing sequence composed of a new combination of sequence elements.

[0009] According to a third embodiment of this disclosure, a computer program is provided. This computer program has the function of generating a sequence set consisting of a plurality of image processing sequences, each having a different combination of (a) an image input layer for inputting an image, (ii) at least one of a plurality of image processing layers, and (iii) an image output layer for outputting the image after image processing, where the sequence elements are: (a) a function for image processing a plurality of training images using each of the plurality of image processing sequences, and for each of the plurality of image processing sequences, a group of intermediate images which is a set of intermediate images which are the images after image processing; (c) a function for inputting the group of intermediate images for each of the plurality of image processing sequences into the input layer of a corresponding plurality of neural network type machine learning models having an input layer, an intermediate layer and an output layer, and for training the machine learning model such that the layer output, which is the output of the output layer, approaches a predetermined target output for each of the plurality of training images; and (d) the corresponding plurality of machines Function (a) includes a function to calculate an output evaluation value indicating the degree of similarity between the layer output and the target output by comparing the layer output and the target output for each machine learning model, and a function to identify the machine learning model from which the maximum output evaluation value, which is the highest of the multiple output evaluation values ​​calculated for each of the multiple machine learning models, is calculated, and the computer is made to execute the following functions: (a) If the maximum output evaluation value is less than the threshold, the computer is made to execute the following functions: (b) If the maximum output evaluation value is less than the threshold, the computer is made to execute the following functions: (a) If the maximum output evaluation value is less than the threshold, the computer is made to execute the following functions: (b) If the computer is made to execute the maximum output evaluation value, which is the highest of the multiple output evaluation values ​​calculated for each of the multiple machine learning models, the computer is made to execute the maximum output evaluation value and the computer is made to execute the image processing sequence used to generate the intermediate image group input to the machine learning model from which the maximum output evaluation value is calculated, and (b) If the computer is made to execute the maximum output evaluation value, it is made to execute the maximum output evaluation value. [Brief explanation of the drawing]

[0010] [Figure 1] A diagram illustrating the generation system of an embodiment. [Figure 2] A diagram illustrating memory storage devices. [Figure 3] A diagram showing an example of training data. [Figure 4] A diagram to explain the gene table. [Figure 5] A diagram used to explain a population of a single generation. [Figure 6] A diagram showing the image processing sequence of the tree structure generated by the gene translation region. [Figure 7] A diagram used to explain machine learning models. [Figure 8] A flowchart showing the processes performed by the generation device. [Figure 9] The first flowchart showing the pre-processing steps of step S10. [Figure 10] A second flowchart showing the pre-processing steps of step S10. [Figure 11] A diagram illustrating the pre-treatment process. [Figure 12] A diagram illustrating the Type 1 evaluation process. [Figure 13] A schematic diagram illustrating the process of generating an individual. [Figure 14] A flowchart showing the utilization process in step S50 of Figure 8. [Modes for carrying out the invention]

[0011] A. Embodiments: Figure 1 is a diagram illustrating a generation system 10 of an embodiment. The generation system 10 comprises a generation device 20 as an image processing device and an imaging device 200. The generation device 20 and the imaging device 200 can transmit and receive data via wired or wireless means. The imaging device 200 is a camera that captures an image of an object and acquires an image. The image acquired by the imaging device 200 is transmitted to the generation device 20.

[0012] The generation device 20 has a preprocessing sequence generation function, a machine learning model generation function, and a result output function. The preprocessing sequence function generates a specific image processing sequence DSq to obtain intermediate images from images to be input to the machine learning model in order to improve the prediction accuracy of the machine learning model. The machine learning model generation function trains a neural network type machine learning model by inputting multiple intermediate images and generates a trained machine learning model. The result output function obtains intermediate images by processing the image using the specific image processing sequence DSq, inputs the intermediate images into the trained machine learning model generated by the machine learning model generation function, and obtains the output of the output layer. In this embodiment, the generation device 20 automatically generates a machine learning model to achieve a desired objective based on the training image LM. For example, if the desired objective is to inspect whether an object is good or defective from an image of the object, the specific image processing sequence DSq is used to process the image of the object to generate intermediate images, and the generated intermediate images are input into the trained machine learning model to determine whether the object is good or defective. Furthermore, the generation device 20 of this embodiment automatically generates an optimal specific image processing sequence DSq for generating intermediate images to be input for training from training images, in order to generate a machine learning model with high prediction accuracy.

[0013] The preprocessing sequence generation function generates multiple image processing sequences Sq using genetic programming, and identifies a specific image processing sequence DSq from among the generated multiple image processing sequences Sq to obtain an intermediate image after image processing that is optimal for training. In this embodiment, the preprocessing sequence generation function, the machine learning model generation function, and the result output function are installed in a single generation device 20, but this is not limited to the present. In other embodiments, the preprocessing sequence generation function, the machine learning model generation function, and the result output function may be installed in separate devices or may reside in the cloud.

[0014] The generation device 20 is an electronic computer such as a personal computer. The generation device 20 comprises a processor 25, a storage device 70, an input unit 150, and a display unit 160. The processor 25 controls the operation of the generation device 20 by executing various programs stored in the storage device 70. The detailed functions of the processor 25 will be described later. The storage device 70 is composed of memory such as RAM and ROM. The storage device 70 stores various programs for realizing each function of the processor 25. Details of the storage device 70 will be described later. The input unit 150 is an interface that receives information from the outside. For example, the input unit 150 receives input of captured images from the imaging device 200, or input of images generated by the user using other devices. The display unit 160 displays various information. The display unit 160 is, for example, a liquid crystal monitor.

[0015] The processor 25 functions as a sequence generation unit 30, an image processing unit 40, a learning and evaluation unit 50, and a display control unit 60 by executing various programs in the storage device 70. Some of the functions performed by the processor 25 may be implemented by hardware circuits. Hereinafter, the term "processor" encompasses CPUs and GPUs.

[0016] The sequence generation unit 30 generates a sequence set consisting of multiple image processing sequences Sq with different combinations of sequence elements by structurally representing the gene sequence shown by individual IV in a tree structure. The image processing sequence Sq is represented by a combination of multiple sequence elements. The sequence elements include (i) an image input layer, which is a node that inputs an image; (ii) an image processing layer, which is an intermediate node between the image input layer and the image output layer and acts as an image processing filter; and (iii) an image output layer, which is a node that outputs an intermediate image MM, which is the final image after image processing.

[0017] The sequence generation unit 30 comprises an individual generation unit 32 and a gene translation unit 34. The individual generation unit 32 generates individual IVs, which are composed of multiple genes arranged in one or two dimensions. Each gene constituting individual IV is a numerical value representing an image input layer in, an image processing layer type (in this embodiment, an image processing filter Ft), and so on. Different image input layers in and different image processing filters Ft are assigned to each numerical range. In other words, individual IVs are generated by arranging genes corresponding to the image input layer in and the image processing layer Ft, respectively. The individual generation unit 32 generates a predetermined number of individual IVs for each generation. In this embodiment, the individual generation unit 32 generates individual IVs with a predetermined gene length by arranging genes in one dimension. In each generation, the combination of sequence elements of multiple image processing sequences Sq differs. For the very first generation, the individual generation unit 32 generates a population 84 composed of multiple individual IVs by randomly placing numerical values ​​from 0 to 1 (up to three decimal places) as genes at each gene locus of individual IVs. For the second generation and beyond, the individual generation unit 32 generates a new individual IV from the individual IV of the previous parent generation using at least one method, such as crossover and mutation, thereby generating a population 84 which is a set of the newly generated individual IV and the individual IV maintained in the parent generation, i.e., an individual IV that is a replica of the individual IV of the parent generation.

[0018] The gene translation unit 34 translates each gene of individual IV by referring to the gene table 82, which will be described later, stored in the memory device 70, and generates an image processing sequence Sq represented as a tree structure. The gene table 82 is a table that defines the corresponding image processing filters for each numerical range of the numerical value that is a gene, and the connection relationships of each sequence element.

[0019] The image processing unit 40 processes the image input to the image input layer using the image processing sequence Sq generated by the gene translation unit 34 and the specific image processing sequence DSq stored in the memory device 70, and outputs the processed intermediate image from the image output layer.

[0020] The learning and evaluation unit 50 learns and evaluates the neural network type machine learning model 88, which will be described later, stored in the memory device 70. The learning and evaluation unit 50 has a learning unit 52 and an evaluation unit 54. The learning unit 52 performs learning of the machine learning model 88 using intermediate images MM generated by image processing the learning images LM of the learning data 74, which will be described later, using the image processing sequence Sq. The evaluation unit 54 performs evaluation processing on the learned machine learning model 88. The evaluation unit 54 can perform a first type of evaluation processing using the intermediate output output from the intermediate layer of the machine learning model 88, and a second type of evaluation processing using the layer output output from the output layer. In addition, the evaluation unit 54 generates prediction results based on the layer output output by inputting an image into the machine learning model 88. Details of the evaluation processing performed by the evaluation unit 54 will be described later.

[0021] The display control unit 60 causes various information to be displayed on the display unit 160. For example, the display control unit 60 displays on the display unit 160 the processed image obtained by a specific image processing sequence DSq or an image processing sequence Sq represented by individual IV.

[0022] Figure 2 is a diagram illustrating the memory device 70. The memory device 70 stores training data 74, a group of image filters 80, a gene table 82, a population 84, a group of intermediate images 86, a machine learning model 88, specific data 90, and various programs 95. The specific data 90 has a specific image processing sequence DSq and model parameters 94. The model parameters 94 are parameters of a trained machine learning model 88 that has been determined by the evaluation unit 54 to satisfy predetermined criteria. The model parameters 94 are parameters that are updated by the training of the machine learning model 88, such as weights and biases determined by training. The machine learning model 88 having these model parameters 94 is also called a trained specific model 88. The specific image processing sequence DSq is the image processing sequence Sq used to generate the training data 74 used to train the trained specific model 88. The specific image processing sequence DSq is represented, for example, by layer identifiers that identify multiple image processing filters, which are multiple image processing layers, and the order of the layer identifiers, i.e., the connection relationship. Various programs 95 are programs executed by the processor 25.

[0023] Figure 3 shows an example of training data 74. Training data 74 consists of training images LM and target outputs GT, which are ground truth data associated with the training images. Target outputs GT are predetermined target outputs for each of the multiple training images LM. A training image LM is, for example, an image of a circuit board, which is an industrial product. Target outputs GT are, for example, data showing the inspection results of a manufactured industrial product. In this embodiment, this is either label 0, which is a label indicating a good product, or label 1, which is a label indicating a defective product. In other words, in this embodiment, target outputs GT are either "0" or "1" and are discrete values. Labels are also called classes. Multiple training images LM are prepared. Training image LM1 is an image of a good circuit board captured by the imaging device 200, and is associated with label 0, which indicates a good product. Training image LM2 is an image of a circuit board with a defective portion DA present, captured by the imaging device 200, and is associated with label 1, which indicates a defective product.

[0024] Each training image LM of the training data 74 is processed by an image processing sequence Sq represented by individual IV generated by the sequence generation unit 30. This generates an intermediate image MM from the training image LM. For example, intermediate image MM1 is generated from training image LM1, and intermediate image MM2 is generated from training image LM2. The intermediate image MM and the label associated with the intermediate image MM are input as a set to the machine learning model 88 for training. In order to improve the prediction accuracy of the machine learning model 88, the intermediate image MM processed by the image processing sequence Sq is used for training the machine learning model 88. For example, the image processing sequence Sq may be used to perform image processing such that the difference between the defective part DA and the normal part other than the defective part DA is more emphasized. In other embodiments, the target output GT does not have to be a discrete value. For example, the target output GT may be an image. For example, the target output GT may be a binarized image in which the defective part DA is a black image and the part other than the defective part DA is a white image.

[0025] The image filter group 80 shown in Figure 2 is a collection of multiple image filters that are candidates for sequence elements in the image processing sequence Sq. Examples of image filters include mean filters, maximum value filters, minimum value filters, binarization filters, moving average filters, Laplacian filters, Gaussian filters, Sobel filters, gamma correction filters, and filters that combine two images into one. Note that even if image processing filters are of the same type, if their kernel size or coefficients differ, they are stored in the image filter group 80 as different image processing filters.

[0026] Gene table 82 is a table that shows the relationship between the connection of sequence elements that are candidates for image processing sequence Sq and the type of image processing filter that corresponds to the numerical value indicated by the gene.

[0027] Figure 4 is a diagram illustrating the gene table 82. For ease of understanding, it is assumed below that the image filter group 80 consists of five image processing filters FtA to FtE. In reality, more than five image processing filters are stored in the image processing filter group. The gene table 82 defines the number of inputs, the number of outputs, and the occurrence range for each filter type, which indicates the type of image processing filter. The filter type as a sequence element type is an identifier that identifies each image processing filter in the image filter group 80 and the image input layer. In Figure 4, the image input layer is represented by the filter type "in". The number of inputs indicates the number of sequence elements connected to the input side of the image processing filter or image input layer. If the number of inputs is "2", the data output from two sequence elements is input to the image processing filter. The number of outputs indicates the number of sequence elements connected to the output side of the image processing filter. In this embodiment, the number of outputs is always "1", and the image processing layer and image input layer are connected to one output-side sequence element.

[0028] The appearance range of gene table 82 defines the range of the gene's numerical value VG. In this embodiment, the numerical values ​​from 0 to 1 are divided into six equal parts so that each image processing filter appears randomly with equal probability in the image processing sequence Sq. For example, if the numerical value VG of a gene is between 0 and 0.167, the image processing filter FtA is assigned to this gene.

[0029] The population 84 shown in Figure 2 is a collection of multiple individuals IV in one generation generated by the individual generation unit 32. Figure 5 is a diagram illustrating the population 84 of one generation G. In this embodiment, we describe an example in which five individuals IV1 to IV5 are generated in one generation G. Each individual IV1 to IV5 has multiple gene loci arranged one-dimensionally. The gene lengths of each individual IV1 to IV5 are the same. Each individual IV1 to IV5 is constructed by sequentially arranging genes in gene loci No. 1 to No. 10. The individual generation unit 32 places genes represented by numerical values ​​from 0 to 1 to three decimal places in each gene locus. In other words, in this embodiment, the values ​​that a gene can take at each gene locus are numerical values ​​from 0 to 1 to three decimal places.

[0030] Before further explaining Figure 2, we will use Figure 6 to explain the process of generating the image processing sequence Sq by the gene translation unit 34. Figure 6 shows the image processing sequence Sq of the tree structure generated by the gene translation unit 34. The image processing sequence Sq shown in Figure 6 is image processing sequence Sq1, which was generated based on individual IV1 shown in Figure 5.

[0031] The gene translation unit 34 uses individual IV, the gene table 82, and the image filter group 80 to generate a tree-structured image processing sequence Sq starting from the image output layer out. First, the gene translation unit 34 sets the image output layer out as the terminal node and connects the sequence elements in the order of the genes of individual IV. The connection of the sequence elements is performed according to predetermined connection rules. In this embodiment, the image processing sequence Sq is generated according to the rules that prioritize placement from the terminal node side in the order of the gene sequence, and that the connection destination on the left side, as shown in Figure 6, is prioritized over the connection destination on the right side. In other words, the image processing layers or image input layers in are placed from the image output layer out side in the order of the gene sequence, and if the input side is divided into multiple parts, the left side is prioritized until the sequence on the left becomes the image input layer in. In this embodiment, the number of inputs to the image output layer out is predetermined to be "1".

[0032] The gene translation unit 34 refers to the numerical value VG of gene No. 1 of individual VI1, "0.534," and the gene table 82 to determine that "0.534" is within the range of occurrence of image processing filter FtD. This connects image processing filter FtD to the input side of the image output layer out. Next, the gene translation unit 34 refers to the gene table 82 to determine that the number of inputs to image processing filter FtD is "2," thereby placing two sequence elements on the input side of image processing filter FtD. Specifically, the gene translation unit 34 places the sequence element corresponding to gene No. 2 of individual VI1 at the left input connection point shown in Figure 6. That is, the gene translation unit 34 refers to the numerical value VG of gene No. 2, "0.292," and the gene table 82 to determine that "0.292" is within the range of occurrence of image processing filter FtB. This places image processing filter FtB at the left input connection point of image processing filter FtD.

[0033] Next, the gene translation unit 34 refers to the image processing filter FtB represented by gene No. 2 and the gene table 82 to determine that the number of inputs to image processing filter FtB is "1", and places one sequence element on the input side of image processing filter FtB. Specifically, the gene translation unit 34 refers to the numerical value VG of gene No. 3, "0.462", and the gene table 82 to determine that "0.462" is within the range of appearance of image processing filter FtC. As a result, image processing filter FtC is placed at the input connection destination of image processing filter FtB indicated by gene No. 3.

[0034] Next, the gene translation unit 34 refers to the image processing filter FtC represented by gene No. 3 and the gene table 82 to determine that the number of inputs to the image processing filter FtC is "1," and places one sequence element on the input side of the image processing filter FtC. Specifically, the gene translation unit 34 refers to the numerical value VG of gene No. 4, "0.856," and the gene table 82 to determine that "0.856" is within the range of appearance of the image input layer in. This places the image input layer in at the input connection destination of the image processing filter FtC indicated by gene No. 3. This completes the generation of the sequence on the left side of the image processing sequence Sq in Figure 6.

[0035] Next, the gene translation unit 34 places the sequence element corresponding to gene No. 5 of individual VI1 at the input connection destination on the right side as shown in Figure 6. In other words, the gene translation unit 34 refers to the numerical value VG of gene No. 5, "0.138," and the gene table 82 to determine that "0.138" is within the range of appearance of image processing filter FtA. As a result, image processing filter FtA is placed at the input connection destination on the right side of image processing filter FtD.

[0036] Furthermore, the gene translation unit 34 refers to the image processing filter FtA represented by gene No. 5 and the gene table 82 to determine that the number of inputs to the image processing filter FtA is "1," and places one sequence element on the input side of the image processing filter FtA. Specifically, the gene translation unit 34 refers to the numerical value VG of gene No. 6, "0.932," and the gene table 82 to determine that "0.932" is within the range of appearance of the image input layer in. As a result, the image input layer in is placed at the input connection destination of the image processing filter FtA indicated by gene No. 5. With this, the generation of the sequence on the right side of the image processing sequence Sq in Figure 6 is completed. As described above, the gene translation unit 34 terminates the generation process of the image processing sequence Sq even if not all genes of individual IV1 are used, when all input sequence elements, which are input connection destinations connected to output sequence elements, become image input layers in. Furthermore, if, when using all the genes of individual IV1, the gene translation unit 34 does not have all of the input sequence elements that are the input connection destinations set to the image input layer in, it will terminate the image processing sequence Sq generation process by setting the remaining unconnected input sequence elements to the image input layer in.

[0037] The gene translation unit 34 terminates the image processing sequence Sq generation process if all input sequence elements become image input layer in, even if not all genes of individual IV1 are used.

[0038] The image processing sequence Sq1, represented by individual VI1, has two image input layers in. An image input to one image input layer in is processed by image processing filter FtC, then by image processing filter FtB, to generate a first processed image. An image input to the other image input layer in is processed by image processing filter FtA to generate a second processed image. The first and second processed images are then processed by image processing filter FtD, and an intermediate image MM is output by the image output layer out.

[0039] As shown in Figure 2, the intermediate image group 86 is a collection of intermediate images MM generated by processing each training image LM of the training data 74 with multiple image processing sequences Sq represented by each individual IV. Each intermediate image MM is associated with the image processing sequence Sq used for image processing and the label of the original training image LM.

[0040] As described above, the machine learning model 88 is a neural network type machine learning model. In this embodiment, the machine learning model 88 is, for example, a convolutional neural network type. Figure 7 is a diagram illustrating the machine learning model 88. The machine learning model 88 comprises an input layer L1, an intermediate layer L2, and an output layer L3. The input layer L1 is a layer into which an image to be predicted, such as an intermediate image MM, is input. The intermediate layer L2 has at least one layer L22 to L2X. Each layer L22 to L2X of the intermediate layer L2 is composed of multiple nodes Nd, and the first node output Cx, which is output from each node Nd constituting the intermediate layer L2 on the input layer L1 side, is input to the next node Nd of the intermediate layer L2 to output a second node output Cx. The node output Cx is a value that indicates the feature quantity extracted from the image input to the input layer L1. In Figure 7, for ease of understanding, the node outputs Cx output from each node Nd of layer L22 are shown using codes Cxa to Cxx. Furthermore, the set of node outputs Cx output from multiple nodes Nd in a specific layer of the hidden layer L2, or the set of node outputs Cx output from all nodes Nd in all layers L22 to L2X that constitute the hidden layer L2, is also called the intermediate output C. The intermediate output C is an X-dimensional vector formed by the array of node outputs Cx output from each of the X nodes Nd, and represents the image features input to the machine learning model 88.

[0041] The output layer L3 has multiple nodes FNd. The output layer L3 uses the output from the hidden layer L2 to output a layer output FC that indicates the prediction result. The layer output FC is a vector composed of judgment values ​​FCa and FCb output from each node FNd. The judgment values ​​FCa and FCb are usually values ​​normalized by the softmax function. It can also be said that the judgment values ​​FCa and FCb represent probabilities. The evaluation unit 54 shown in Figure 1 generates a class as a prediction result corresponding to the node FNd1 and FNd2 that output the maximum value among the judgment values ​​FCa and FCb output from each node FNd1 and FNd2 of the output layer L3. For example, if the judgment value FCa output from node FNd1 is the maximum value, the evaluation unit 54 outputs result information indicating "good product" as the prediction result.

[0042] Figure 8 is a flowchart showing the process performed by the generation device 20. This process includes the steps of generating a specific image processing sequence DSq and training a machine learning model 88 in step S10, and the utilization process using the specific image processing sequence DSq in step S50, which is performed after step S10. The process in step S10 is performed by the preprocessing sequence generation function and the machine learning model generation function of the generation device 20. The process in step S250 is performed by the result output function of the generation device 20. The process in step S10 is also called the preprocessing process.

[0043] Figure 9 is a first flowchart showing the pre-treatment process in step S10. Figure 10 is a second flowchart showing the pre-treatment process in step S10. Figure 11 is a diagram illustrating the pre-treatment process.

[0044] As shown in Figure 9, first, in step S101, the storage device 70 stores a plurality of image processing filters as an image filter group 80. Step S101 is executed when the user inputs a plurality of candidate image processing filters to be used in a specific image processing sequence DSq to the generation device 20. In step S102, the storage device 70 stores training data 74, which consists of a plurality of training images LM associated with the target output GT, which is the correct answer data. Step S102 is executed when the user inputs a plurality of training images LM associated with the target output GT to the generation device 20. Note that the order of steps S101 and S102 is not limited to this.

[0045] Next, in step S103, the individual generation unit 32 generates the current generation G n Multiple individuals IV n A population 84 is generated, consisting of multiple individuals IV. n 1-IV n It is composed of 5. The generated population 84 is stored in the memory device 70. Next, in step S104, the gene translation unit 34 processes each individual IV n 1-IV n Select one of the five and refer to gene table 82 to identify one individual IV. n Translate the text to generate the image processing sequence Sq.

[0046] Next, in step S105, the image processing unit 40 processes the training image LM using the image processing sequence Sq generated in step S104 to generate an intermediate image MM. Next, in step S106, the image processing unit 40 outputs the generated intermediate image MM to the storage device 70. As a result, the storage device 70 stores the intermediate image MM as an intermediate image group 86. The intermediate image MM is associated with a sequence identifier for identifying the image processing sequence Sq used in the image processing.

[0047] Next, in step S107, the image processing unit 40 processes the current generation G nDetermine whether the processes of step S104 to step S106 have been executed for all individuals IV. If the determination in step S107 is "No", the processes after step S104 are executed again. In this case, in step S104, the gene translation unit 34 selects one from the remaining individuals IV for which the processes of step S104 to step S106 have not been completed to generate an image processing sequence Sq. On the other hand, if the determination in step S107 is "Yes", the processing flow shown in FIG. 10 is executed. As shown in FIG. 11, for each individual IV in the current generation G n from 1 to IV n in 5, the learning image LM is image-processed by each corresponding image processing sequence Sq to generate an intermediate image MM. In FIG. 11, the intermediate images MM generated by the respective image processing sequences Sq of individuals IV n from 1 to IV n in 5 are illustrated as intermediate images MM_1 to MM_5. For each individual IV in the current generation G n from 1 to IV n in 5, when the processes of step S104 to step S107 are executed, the intermediate images MM_1 to MM_5 are stored in the storage device 70 as the population 84. The intermediate images MM_1 to MM_5 are each generated in plurality corresponding to a plurality of learning images LM. A set of a plurality of intermediate images MM generated by one image processing sequence Sq is also called an intermediate image group GMM. As described above, the steps of step S104 to step S107 are steps of image-processing a plurality of learning images LM by a plurality of image processing sequences Sq and generating an intermediate image group GMM, which is a set of a plurality of intermediate images MM that are the images after image processing, for each of the plurality of image processing sequences Sq.

[0048] As shown in FIG. 10, in step S108, the learning evaluation unit 50 selects one individual IV n from the individuals IV n from 1 to IV n in 5 of the current generation G n ​​​​​​The intermediate image group GMM generated using the image processing sequence Sq represented by 1 is input to the input layer L1 of the machine learning model 88 before training. In step S108, the learning evaluation unit 50 extracts the intermediate output C output from each node Nd of the intermediate layer L2 of the machine learning model 88. The extracted intermediate output C of each node Nd is stored in the memory device 70. In step S108, the learning evaluation unit 50 extracts the layer output FC output from each node FNd of the output layer L3 of the machine learning model 88. The extracted layer output FC of each node FNd is stored in the memory device 70. The parameters of the machine learning model 88 before training, such as weights and biases, can use either predetermined initial values, random values, or values ​​generated by training in the past.

[0049] In step S109, the evaluation unit 54 performs a first-type evaluation process using the intermediate output C. Figure 12 is a diagram illustrating the first-type evaluation process. The evaluation unit 54 performs the first-type evaluation process using the intermediate output C for each training image LM obtained by inputting multiple intermediate images MM generated by the image processing sequence Sq represented by one individual IV into the machine learning model 88. Each intermediate output C is associated with the target output GT of the training image LM that generated it. The evaluation unit 54 groups the multiple intermediate output C obtained in step S108 according to the corresponding target output GT. In this embodiment, as shown in Figure 12, the evaluation unit 54 groups the multiple intermediate output C into group G0 of label 0, which indicates a "good product" target output GT, and group G1 of label 1, which indicates a "defective product" target output GT. Note that in Figure 12, for ease of understanding, the grouping is shown assuming that the intermediate output C is a two-dimensional vector composed of two node outputs Cx, but in reality, the intermediate output C is represented by an X-dimensional vector with more than 2 dimensions. The evaluation unit 54 calculates an intermediate evaluation value S, which is represented by the degree of variation σa and σb of the intermediate output C within groups G0 and G1, and the degree of variation β between groups G0 and G1. FThe representative values ​​of the intermediate output C for each group G0 and G1 are the representative values ​​Gr0 and Gr1. In this embodiment, the representative value is the average value. Specifically, the evaluation unit 54 uses the following formula (1) to calculate the intermediate evaluation value S F Calculate.

[0050]

number

[0051] In equation (1) above, the numerator on the right side is the sum of the variances representing the degree of variation within each group of intermediate output C. Also, in equation (1) above, the denominator on the right side is the variance representing the degree of variation β between the mean values ​​Gr0 and Gr1 of each group of intermediate output C. In other words, the numerator on the right side of equation (1) above can also be said to be the sum of the degrees of variation σa and σb in Figure 12. Furthermore, the numerator on the right side of equation (1) above can also be said to be the degree of variation relative to the mean values ​​Gr0 and Gr1. The groups G0 and G1 of intermediate output C, which are features of the intermediate image MM, are separated from each other, and the smaller the variance within groups G0 and G1, the more accurately the target output GT corresponding to the intermediate image MM is output. In other words, the intermediate evaluation value S in equation (1) above... F The smaller the value, the more favorable the intermediate image MM is for the machine learning model 88 to use for training to improve prediction accuracy. In other words, the intermediate evaluation value S F This is an index value related to the prediction accuracy of the machine learning model 88, and the intermediate evaluation value S F The smaller the value, the higher the prediction accuracy tends to be.

[0052] As shown in Figure 10, after step S109, the evaluation unit 54 performs a second type of evaluation process using the layer output FC. The evaluation unit 54 performs a first type of evaluation process using the layer output FC for each training image LM obtained by inputting multiple intermediate images MM generated by the image processing sequence Sq represented by a single individual IV into the machine learning model 88. The evaluation unit 54 compares the layer output FC output in step S108 with the target output GT associated with the training image LM from which the layer output FC was derived, and calculates an output evaluation value S0 that indicates the degree of similarity between the layer output FC and the target output GT. The output evaluation value S0 is an index value related to the prediction accuracy of the machine learning model 88, and the larger the output evaluation value S0, the higher the prediction accuracy tends to be. In this embodiment, the layer output FC and the target output GT are each represented by two-dimensional vectors. Specifically, the layer output FC and the target output GT are two-dimensional vectors composed of a value indicating the probability of label 0 indicating a good product and a value indicating the probability of label 1 indicating a defective product, respectively. The layer output FC is a vector whose elements are the judgment values ​​FCa and FCb shown in Figure 7. The target output GT is a vector where the probability of the class indicated by the target output GT is "1", and the probability of all other classes is "0". The layer output FC and the target output GT are vectors that sequentially arrange the probabilities for each class.

[0053] The evaluation unit 54 calculates the output evaluation value S0 using the following equation (2). In this embodiment, the output evaluation value S0 is a value relating to the Euclidean distance, and specifically, it is the average value of the Euclidean distance between each target output GT and the corresponding layer output FC. The degree of similarity between the layer output FC and the target output GT can be easily evaluated using the output evaluation value S0 relating to the Euclidean distance.

number

[0054] The output evaluation value S0 is not limited to Euclidean distance or values ​​related to Euclidean distance; other index values ​​that represent the degree of similarity can be used. For example, the output evaluation value S0 may be cosine similarity.

[0055] Intermediate evaluation value S calculated in step S109 F The output evaluation value S0 calculated in step S110 is stored in the memory device 70 in association with the corresponding individual IV. Note that the order of processing between step S109 and step S110 is not limited to the above.

[0056] As shown in Figure 10, following steps S109 and S110, in step S111, the evaluation unit 54 determines whether or not the evaluation processes of steps S109 and S110 have been performed for all individuals IV in the current generation Gn. If the determination in step S111 is "No", the learning evaluation unit 50 determines whether or not the evaluation processes of steps S109 and S110 have been performed for all individuals IV in the current generation Gn. n Of the individual IVs, the remaining individual IVs that have not undergone the processing in steps S109 and S110. n One is selected from the selected individual IV, and the intermediate image group GMM generated by the image processing sequence Sq represented by the selected individual IV is extracted from the storage device 70. Then, the processing in steps S108 to S110 is executed again on the extracted intermediate image group GMM. The machine learning model 88 to which the intermediate image group GMM is input in step S108, which is executed again, is the machine learning model 88 before training. The machine learning model 88 is composed of multiple individuals IV n 1-IV n Multiple versions are available depending on the case of 5, and they are all the same neural network type algorithm. Figure 11 shows multiple individuals IV n 1-IV n The codes "88_1" to "88_5" are used to distinguish between the machine learning models 88 corresponding to 5.

[0057] On the other hand, if a "Yes" determination is made in step S111, the evaluation unit 54 determines in step S112 whether the termination condition has been met. The termination condition is that the maximum output evaluation value S0max, which is the highest output evaluation value S0 among the multiple output evaluation values ​​S0 calculated for each of the multiple machine learning models 88 corresponding to multiple individuals IV1 to IV5, is higher than a predetermined threshold. If the output evaluation value S0max is higher than the threshold, the learning evaluation unit 50 identifies the image processing sequence Sq used to generate the intermediate image group GMM input to the machine learning model 88 from which the maximum output evaluation value S0max was calculated as a specific image processing sequence DSq in step S115. The identified specific image processing sequence DSq is stored in the storage device 70. If the output evaluation value S0max is higher than the threshold, the learning evaluation unit 50 identifies the machine learning model 88 from which the maximum output evaluation value S0max was calculated in step S116. The identified machine learning model 88 is stored in the storage device 70. Specifically, the learning evaluation unit 50 stores the parameters of the machine learning model 88 used for calculation as model parameters 94 in the storage device 70.

[0058] If it is determined in step S112 that the termination condition is not met, the learning unit 52 performs a learning process in step S113, which involves updating the parameters of multiple machine learning models 88_1 to 88_5. The learning process in step S113 is performed by inputting the intermediate image group GMM for each of the multiple image processing sequences Sq into the input layer L1 of the corresponding multiple machine learning models 88_1 to 88_5, and updating the parameters of the machine learning model 88 so that the layer output FC of the output layer L3 approaches the target output GT defined for each of the multiple learning images LM. The learning unit 52 performs the above learning using, for example, backpropagation. The intermediate image MM input in a single learning process may be the entire intermediate image group GMM or a part of the intermediate image group GMM. For example, the learning unit 52 may perform the learning process by mini-batch learning.

[0059] Next, in step S114, the learning unit 52 determines whether the learning process in step S113 has been repeated the specified number of times. If the determination in step S114 is "No", the processes from step S108 to step S113 are repeatedly executed until the specified number of learning times is reached, unless the termination condition is met. In other words, the learning unit 52 completes the learning process by repeating the learning of the machine learning model 88 a predetermined specified number of times so that the layer output FC approaches the target output GT. The specified number of learning times is N, where "N" is an integer of 2 or more. In this embodiment, "N" is set to 500. This makes it possible to further improve prediction accuracy by repeatedly performing learning on the machine learning model N times using the intermediate image group GMM generated by the same image processing sequence Sq.

[0060] On the other hand, if the determination of "Yes" is made in step S114, the individual generation unit 32 executes step S103 again, as shown in Figure 9. Step S103, which is executed again, is the parent generation G generated in step S103 of the previous processing routine. m Individual IV m 1~IM m Out of 5, the performance evaluation value EV is among the lowest J-th individual IV m new individual IV n This is executed by replacing it with "J", where "J" is an integer greater than or equal to 1. In other words, the individual generation unit 32 duplicates individual IVs whose performance evaluation value EV does not fall within the lower Jth position, and generates new individual IVs by performing at least one of crossover or mutation on individual IVs whose performance evaluation value EV does not fall within the lower Jth position. This makes it easy to generate a new image processing sequence Sq with a different combination of sequence elements from the image processing sequence Sq generated by the previous routine.

[0061] The performance evaluation value EV is an index value that shows a positive correlation with the prediction accuracy of the machine learning model 88. In this embodiment, the performance evaluation value EV is calculated by combining the output evaluation value S0 and the intermediate evaluation value S FIt is calculated by a function that includes . For example, the performance evaluation value EV is calculated by the following formula (3). EV = A × S₀ + B × S₀ F +C (3) Here, A is a number greater than or equal to 0, B is a number less than or equal to 0, and C is the constant term.

[0062] In equation (3) above, if "A" is set to a number greater than 0, the performance evaluation value EV shows a positive correlation with the output evaluation value S0. Also, in equation (3) above, by setting at least one of "A" and "B" to a value other than "0", the performance evaluation value EV shows a positive correlation with the output evaluation value S0 and the intermediate evaluation value S F It is expressed using at least one of the above. Furthermore, in equation (3) above, if both "A" and "B" are set to "0", the constant term "C" is set to a value that shows a positive correlation with the prediction accuracy of the machine learning model 88.

[0063] Furthermore, in equation (3) above, by setting "B" and "C" to "0", the performance evaluation value EV may be the output evaluation value S0, or may be represented by the output evaluation value S0.

[0064] Figure 13 shows that in step S103, the individual generation unit 32 generates parent generation G m Individual IV m 1~IM m From 5 to the current generation G n Individual IV n 1~IM n This diagram schematically shows the process for generating 5. Figure 13 shows the current generation G n Individual IV n 1-IV n 2 is the parent generation G generated in step S103 of the previous routine. m Individual IV m 1-IV m Of the 5, individual IV m 1-IV m It is generated by duplicating 2. Also, the current generation G shown in Figure 13 n Individual IV n 3 is the parent generation G m Individual IVm 1 and individual IV m This individual was produced by crossing it with 2. Also, the current generation G shown in Figure 13. n Individual IV n 4 is parent generation G m Individual IV m 2 and individual IV m This individual was produced by crossing it with 3. Also, the current generation G shown in Figure 13. n Individual IV n 5 is the parent generation G m Individual IV m This is individual IV, which was generated by mutating 2. In this way, the individual generation unit 32 generates an image processing sequence Sq composed of a different combination of sequence elements than the previous generation G by newly generating individual IV using at least one of crossover and mutation. In this way, if the maximum output evaluation value S0max of the image processing sequence Sq, which has been repeatedly trained for a specified number of training times, is less than the threshold, then one or more image processing sequences Sq with a low performance evaluation value EV among the multiple generated image processing sequences Sq are replaced with an image processing sequence composed of a new combination of sequence elements, thereby generating the next generation G n A new sequence set is generated. Next generation G n The new sequence set is the current generation sequence set, and the following routines from step S104 onwards are executed.

[0065] In the processing from step S104 onward, which targets a new sequence set, the same image processing sequence Sq that constitutes the sequence set used in the previous routine may be processed as follows: That is, the intermediate evaluation value S F The output evaluation value S0 is an intermediate evaluation value S calculated for the image processing sequence Sq that constitutes the previous sequence set, without performing any further processing to calculate it. F Alternatively, the output evaluation value S0 or the performance evaluation value EV may be used. This will result in the intermediate evaluation value S FSince there is no need to recalculate the output evaluation value S0 or the performance evaluation value EV, the processing efficiency of the pre-processing steps performed in the image processing method can be improved.

[0066] Figure 14 is a flowchart showing the utilization processing step S50 in Figure 8. In the utilization processing step, first, in step S501, the generation device 20 stores the specific image processing sequence DSq received by the input unit 150 in the storage device 70. The specific image processing sequence DSq is represented by the image processing filters that constitute the sequence elements and the order in which the sequence elements are arranged. If the utilization processing step is performed by the same generation device 20 that performed the pre-processing step, the specific image processing sequence DSq is already stored in the storage device 70, so step S501 can be omitted. Also, in step S502, the generation device 20 stores the trained specific model 88, which is the machine learning model 88 identified in step S116 of Figure 10, in the storage device 70. Specifically, in step S502, the trained specific model 88, which is a machine learning model 88 having model parameters 94, is stored in the storage device 70. Furthermore, if the utilization processing step is performed by the same generation device 20 that performed the preprocessing step, the machine learning model 88 having the model parameters 94 is already stored in the storage device 70, so step S502 can be omitted.

[0067] Next, in step S503, the image processing unit 40 acquires an image that is the subject of the utilization processing step by reading it from the storage device 70 via the input unit 150. The image acquired in step S503 is, for example, an image of the same type of object as the learning image LM. If there are multiple target images, the image processing unit 40 may acquire an image by reading one of the multiple images that have been pre-stored in the storage device 70, or it may acquire the target image from the imaging device 200 each time it is time to perform the image processing described later.

[0068] Next, in step S504, the image processing unit 40 obtains an intermediate image MM from the image acquired in step S503 by performing image processing using a specific image processing sequence DSq. That is, it applies image processing filters stored in the image filter group 80 of the storage device 70 to the image in the order indicated by the specific image processing sequence DSq and performs image processing. The intermediate image MM is generated when step S504 is executed.

[0069] Next, in step S505, the evaluation unit 54 inputs the intermediate image MM to the trained specific model 88. In step S506, the evaluation unit 54 outputs a prediction result based on the layer output FC output from the output layer L3 of the trained specific model 88. In this embodiment, the evaluation unit 54 outputs the class corresponding to the node FNd1 and FNd2 that output the maximum value among the judgment values ​​FCa and FCb output from each node FNd1 and FNd2 of the layer output FC as the prediction result.

[0070] Next, in step S507, the evaluation unit 54 determines whether to continue the usage processing process. For example, the evaluation unit 54 determines to continue the usage processing process if information indicating the continuation of the usage processing process is input from the user, or if there are still unprocessed images that are the target of the usage processing process remaining in the storage device 70. If the usage processing process is to be continued, the processing from step S503 onwards is executed again. On the other hand, the evaluation unit 54 terminates the usage processing process if information indicating the termination of the usage processing process is input from the user, or if there are no unprocessed images that are the target of the usage processing process remaining in the storage device 70.

[0071] According to the above embodiment, if the maximum output evaluation value S0max among the multiple output evaluation values ​​S0 calculated for each machine learning model 88_1 to 88_5 is less than a threshold, as shown in Figure 13, one or more image processing sequences with low performance evaluation values ​​EV among the multiple image processing sequences are replaced with new image processing sequences to generate a new sequence set. This makes it easy to identify the optimal image processing sequence Sq for improving the prediction performance of the machine learning model 88. Furthermore, according to the above embodiment, the intermediate image MM processed by the image processing sequence Sq is input to the machine learning model 88 for training and image prediction. In other words, even in the complex process of inputting the intermediate image MM after image processing of an image to the machine learning model 88, the prediction accuracy of the machine learning model 88 can be improved. Furthermore, according to the above embodiment, as shown in Figures 9 and 10, the generation of candidate image processing sequences Sq to obtain the optimal intermediate image MM for input to the machine learning model 88 and the evaluation using the output evaluation value obtained by inputting the intermediate image MM generated by the generated image processing sequence Sq to the machine learning model 88 are executed alternately. Therefore, compared to generating all candidate image processing sequences Sq and then evaluating the machine learning model 88 for each image processing sequence Sq, the processing time and computational cost required to identify the trained specific model 88 and the specific image processing sequence DSq can be reduced. Furthermore, according to the above embodiment, even if the number of image processing layers used in the image processing sequence Sq is increased, the generation of the image processing sequence Sq and the evaluation of the machine learning model 88 are performed alternately. This makes it possible to generate the optimal specific image processing sequence DSq for improving prediction accuracy while suppressing an increase in the training time of the machine learning model 88.

[0072] Furthermore, according to the above embodiment, as shown in Figure 13, the individual generation unit 32 generates individual IV up to the Jth individual whose performance evaluation value EV is as follows. m new individual IV nBy replacing it, a new image processing sequence Sq is generated. In other words, using the performance evaluation value EV, it is possible to identify the lowest J image processing sequences Sq that are unsuitable for improving the predictive performance of the machine learning model 88. This allows unsuitable image processing sequences Sq to be replaced with new image processing sequences Sq.

[0073] B. Other embodiments: B-1. Other Embodiments 1: In the above embodiment, the image processing of the training image LM by multiple image processing sequences Sq shown by multiple individuals IV belonging to one generation G, the training process of multiple machine learning models 88, and the intermediate evaluation value S are performed. F The calculation processes for the output evaluation value S0 were performed sequentially using a single generation device 20, but they may also be performed in parallel using multiple generation devices 20.

[0074] B-2. Other Embodiments 2: In the above embodiment, the image processing sequence Sq was represented by individual IV and gene table 82, but is not limited to this. For example, the image processing sequence Sq may be represented by a plurality of sequence elements and a table defining the connection relationships of each sequence element. Also, in the above embodiment, the sequence generation unit 30 generated a plurality of image processing sequences Sq using a genetic algorithm, but the generation method is not limited to this. A plurality of image processing sequences Sq may be generated by randomly arranging a plurality of sequence elements.

[0075] B-3. ​​Other Embodiments 3: In the above embodiment, various algorithms can be used as the neural network type machine learning model 88. For example, the machine learning model 88 may be a vector neural network type model that uses vector neurons, and may be a model called a capsule network. A vector neuron is a neuron whose input and output are vectors. A capsule network is a machine learning model in which vector neurons called capsules are used as nodes in the network.

[0076] C. Other forms: This disclosure is not limited to the embodiments described above, and can be implemented in various forms without departing from its spirit. For example, this disclosure can also be implemented in the following forms (aspects). The technical features in the embodiments described above that correspond to the technical features in each of the forms described below can be replaced or combined as appropriate in order to solve some or all of the problems of this disclosure, or to achieve some or all of the effects of this disclosure. Furthermore, if such technical features are not described as essential in this specification, they can be deleted as appropriate.

[0077] (1) According to a first embodiment of the present disclosure, an image processing method is provided. This image processing method comprises: (a) generating a sequence set comprising a plurality of image processing sequences, each having a different combination of (i) an image input layer for inputting an image, (ii) at least one of a plurality of image processing layers, and (iii) an image output layer for outputting the image after image processing; (b) processing a plurality of training images with each of the plurality of image processing sequences, and generating a group of intermediate images, which is a set of intermediate images, for each of the plurality of image processing sequences; (c) inputting the group of intermediate images for each of the plurality of image processing sequences into the input layer of a corresponding plurality of neural network type machine learning models having an input layer, an intermediate layer, and an output layer, and training the machine learning model so that the layer output, which is the output of the output layer, approaches a predetermined target output for each of the plurality of training images; and (d) the corresponding plurality The process includes: (a) For each machine learning model, a step of comparing the layer output with the target output to calculate an output evaluation value indicating the degree of similarity between the layer output and the target output; (e) If the maximum output evaluation value, which is the highest of the multiple output evaluation values ​​calculated for each of the multiple machine learning models, is higher than a predetermined threshold, a step of identifying the machine learning model from which the maximum output evaluation value was calculated and the image processing sequence used to generate the intermediate image group input to the machine learning model from which the calculation was made; and if the maximum output evaluation value is less than the threshold, a step of generating a new sequence set by replacing one or more of the generated multiple image processing sequences that have a low performance evaluation value regarding the prediction accuracy of the machine learning model with an image processing sequence composed of a new combination of sequence elements; and steps (b) to (e) are executed again on the new sequence set.In this configuration, if the maximum output evaluation value among the multiple output evaluation values ​​calculated for each machine learning model is below a threshold, one or more image processing sequences with low performance evaluation values ​​are replaced with new image processing sequences to generate a new sequence set. This makes it easy to identify the optimal image processing sequence for improving the predictive performance of the machine learning model.

[0078] (2) In the above embodiment, in step (d), the layer output and the target output are each represented by vectors, and the output evaluation value calculated in step (d) may be a value relating to the Euclidean distance. According to this embodiment, the degree of similarity between the layer output and the target output can be evaluated by a value relating to the Euclidean distance.

[0079] (3) In the above embodiment, step (c) may be performed by inputting the intermediate image group into the input layer and performing training of the machine learning model so that the layer output approaches the target output, repeating this process a predetermined N times (where N is an integer of 2 or more). According to this embodiment, the prediction accuracy can be further improved by performing training on the machine learning model N times.

[0080] (4) In the above embodiment, the target output is represented by discrete values, and the image processing method further includes (f) in step (c) a step of obtaining a plurality of intermediate outputs that represent feature quantities output from the intermediate layer corresponding to each of the plurality of training images, grouping the plurality of intermediate outputs according to the corresponding target output, and calculating an intermediate evaluation value represented by the degree of variation of the intermediate outputs within the group and the degree of variation between the groups, wherein the performance evaluation value may be calculated by a function that includes the output evaluation value and the intermediate evaluation value. According to this embodiment, an image processing sequence unsuitable for improving the predictive performance of the machine learning model can be identified using the performance evaluation value calculated by a function that includes the output evaluation value and the intermediate evaluation value. As a result, the unsuitable image processing sequence can be replaced with a new image processing sequence.

[0081] (5) In the above embodiment, in step (a), the image processing sequence is represented by an individual having a plurality of genes corresponding to the image input layer and the image processing layer, respectively, and a gene table defining the connection relationships of the sequence elements in the image processing sequence, and step (a) may generate the image processing sequence composed of a new combination of the sequence elements by newly generating the individual using at least one of crossover and mutation. According to this embodiment, an image processing sequence composed of a new combination of sequence elements can be easily generated using at least one of crossover and mutation.

[0082] (6) In the above configuration, in step (a) or step (e), for the same image processing sequence that constitutes the sequence set used in the previous routine, the output evaluation value or performance evaluation value for the image processing sequence that constitutes the sequence set used in the previous routine may be used without executing steps (b) to (d). This configuration eliminates the need to recalculate the output evaluation value or performance evaluation value, thereby improving the processing efficiency of the image processing method.

[0083] (7) According to a second embodiment of the present disclosure, an image processing apparatus is provided. The image processing apparatus comprises: a sequence generation unit that generates a sequence set comprising a plurality of image processing sequences, each having a different combination of (i) an image input layer for inputting an image, (ii) at least one of a plurality of image processing layers, and (iii) an image output layer for outputting the image after image processing; an image processing unit that processes a plurality of training images using each of the plurality of image processing sequences and generates an intermediate image group for each of the plurality of image processing sequences, which is a set of a plurality of intermediate images that are images after image processing; and inputs the intermediate image group for each of the plurality of image processing sequences into the input layer of a plurality of machine learning models of the neural network type, which has an input layer, an intermediate layer, and an output layer, so that the layer output, which is the output of the output layer, approaches a target output that is a predetermined target for each of the plurality of training images. The system comprises a learning unit that performs learning of the machine learning model, and an evaluation unit that, for each of the multiple machine learning models, compares the layer output with the target output and calculates an output evaluation value indicating the degree of similarity between the layer output and the target output. The evaluation unit identifies the machine learning model from which the maximum output evaluation value, which is the highest of the multiple output evaluation values ​​calculated for each of the multiple machine learning models, is higher than a predetermined threshold, and identifies the machine learning model from which the maximum output evaluation value was calculated and the image processing sequence used to generate the intermediate image group input to the machine learning model from which the maximum output evaluation value was calculated. The sequence generation unit generates a new sequence set by replacing one or more image processing sequences with low performance evaluation values ​​regarding the prediction accuracy of the machine learning model with an image processing sequence composed of a new combination of sequence elements. In this configuration, if the maximum output evaluation value among the multiple output evaluation values ​​calculated for each machine learning model is lower than a threshold, a new sequence set is generated by replacing one or more image processing sequences with low performance evaluation values ​​with a new image processing sequence.This makes it easy to identify the optimal image processing sequence for improving the predictive performance of machine learning models.

[0084] (8) According to a third embodiment of the present disclosure, a computer program is provided. The computer program has the function of generating a sequence set consisting of a plurality of image processing sequences, each having a different combination of (a) an image input layer for inputting an image, (ii) at least one of a plurality of image processing layers, and (iii) an image output layer for outputting the image after image processing, wherein the sequence elements are: (b) a function for image processing a plurality of training images using each of the plurality of image processing sequences, and for each of the plurality of image processing sequences, a group of intermediate images which is a set of intermediate images which are images after image processing; (c) a function for inputting the group of intermediate images for each of the plurality of image processing sequences into the input layer of a plurality of machine learning models of the neural network type having an input layer, an intermediate layer and an output layer, and for training the machine learning model such that the layer output which is the output of the output layer approaches a target output which is a predetermined target for each of the plurality of training images; and (d) the function of training the corresponding plurality of machines Function (a) includes a function to calculate an output evaluation value indicating the degree of similarity between the layer output and the target output by comparing the layer output and the target output for each machine learning model, and a function to identify the machine learning model from which the maximum output evaluation value, which is the highest of the multiple output evaluation values ​​calculated for each of the multiple machine learning models, is calculated, and the computer is made to execute the following functions: (a) If the maximum output evaluation value is less than the threshold, the computer is made to execute the following functions: (b) If the maximum output evaluation value is less than the threshold, the computer is made to execute the following functions: (a) If the maximum output evaluation value is less than the threshold, the computer is made to execute the following functions: (b) If the computer is made to execute the maximum output evaluation value, which is the highest of the multiple output evaluation values ​​calculated for each of the multiple machine learning models, the computer is made to execute the maximum output evaluation value and the computer is made to execute the image processing sequence used to generate the intermediate image group input to the machine learning model from which the maximum output evaluation value is calculated, and (b) If the computer is made to execute the maximum output evaluation value, it is made to execute the maximum output evaluation value.In this configuration, if the maximum output evaluation value among the multiple output evaluation values ​​calculated for each machine learning model is below a threshold, one or more image processing sequences with low performance evaluation values ​​are replaced with new image processing sequences to generate a new sequence set. This makes it easy to identify the optimal image processing sequence for improving the predictive performance of the machine learning model.

[0085] In the above configuration, step (a) may terminate the generation process of the image processing sequence corresponding to the individual even if not all of the multiple genes constituting the individual are used as the image processing sequence, provided that there are no input-side sequence elements unconnected to the output-side sequence elements. This configuration allows for the generation of an image processing sequence even if not all of the multiple genes are used. In the above configuration, if all of the multiple genes constituting the individual are used as the image processing sequence, and there are input-side sequence elements unconnected to the output-side sequence elements, step (a) may terminate the generation process of the image processing sequence corresponding to the individual by setting the image input layer to the unconnected input-side sequence elements. This ensures reliable generation of an image processing sequence using an individual.

[0086] This disclosure can also be implemented in various forms other than those described above. For example, it can be implemented in the form of a non-transitory storage medium on which a computer program is recorded. [Explanation of symbols]

[0087] 10...Generation system, 20...Generation device, 25...Processor, 30...Sequence generation unit, 32...Individual generation unit, 34...Gene translation unit, 40...Image processing unit, 50...Learning and evaluation unit, 52...Learning unit, 54...Evaluation unit, 60...Display control unit, 70...Storage device, 74...Learning data, 80...Image filter group, 82...Gene table, 84...Individual population, 86...Intermediate image group, 88,88_1~88_5...Machine learning model, 90...Specific data, 94...Model parameters, 95...Various programs, 15 0...Input unit, 160...Display unit, 200...Imaging device, DA...Defective area, Ft...Image processing layer, FtA~FtE...Image processing filter, G...Generation, GT...Target output, LM...Training image, MM...Intermediate image, Sq, Sq1...Image processing sequence, VI1~VI5...Individual, out...Image output layer, Cx...Node output, DSq...Specific image processing sequence, FC...Layer output, FCa, FCb...Judgment value, FNd, FNd1, FNd2...Node, Ft, FtA~FtE...Image processing filter, IV, IV m IV m 1-IV m 5, IV n IV n 1-IV n 5...Individual, L1...Input layer, L2...Hidden layer, L22, L2X...Layers, L3...Output layer, LM...Training image, Nd...Node, in...Image input layer

Claims

1. An image processing method, (a) A step of generating a sequence set comprising a plurality of image processing sequences, each having a different combination of (i) an image input layer for inputting an image, (ii) at least one of a plurality of image processing layers, and (iii) an image output layer for outputting the image after image processing, wherein (a) the sequence elements include: (b) A step of processing multiple training images by each of the multiple image processing sequences, and for each of the multiple image processing sequences, generating a set of intermediate images which is a collection of multiple intermediate images which are images after image processing, (c) A step of inputting the group of intermediate images for each of the plurality of image processing sequences into the input layer of a plurality of machine learning models of the neural network type, which has an input layer, an intermediate layer and an output layer, and training the machine learning model so that the layer output, which is the output of the output layer, approaches a predetermined target output for each of the plurality of training images, (d) For each of the corresponding machine learning models, a step of comparing the layer output with the target output to calculate an output evaluation value that indicates the degree of similarity between the layer output and the target output, (e) If the maximum output evaluation value, which is the highest of the multiple output evaluation values ​​calculated for each of the multiple machine learning models, is higher than a predetermined threshold, the process includes identifying the machine learning model from which the maximum output evaluation value was calculated and the image processing sequence used to generate the intermediate image group input to the machine learning model from which the calculation was made. Step (a) includes, if the maximum output evaluation value is less than the threshold, a step of generating a new sequence set by replacing one or more of the generated image processing sequences that have a low performance evaluation value regarding the prediction accuracy of the machine learning model with an image processing sequence composed of a new combination of sequence elements, Steps (b) through (e) are performed again for the new sequence set, In step (d) above, the layer output and the target output are each represented by vectors, An image processing method wherein the output evaluation value calculated in step (d) is a value relating to Euclidean distance.

2. An image processing method, (a) A step of generating a sequence set comprising a plurality of image processing sequences, each having a different combination of (i) an image input layer for inputting an image, (ii) at least one of a plurality of image processing layers, and (iii) an image output layer for outputting the image after image processing, wherein (a) the sequence elements include: (b) A step of processing multiple training images by each of the multiple image processing sequences, and for each of the multiple image processing sequences, generating a set of intermediate images which is a collection of multiple intermediate images which are images after image processing, (c) A step of inputting the group of intermediate images for each of the plurality of image processing sequences into the input layer of a plurality of machine learning models of the neural network type, which has an input layer, an intermediate layer and an output layer, and training the machine learning model so that the layer output, which is the output of the output layer, approaches a predetermined target output for each of the plurality of training images, (d) For each of the corresponding machine learning models, a step of comparing the layer output with the target output to calculate an output evaluation value that indicates the degree of similarity between the layer output and the target output, (e) If the maximum output evaluation value, which is the highest of the multiple output evaluation values ​​calculated for each of the multiple machine learning models, is higher than a predetermined threshold, the process includes identifying the machine learning model from which the maximum output evaluation value was calculated and the image processing sequence used to generate the intermediate image group input to the machine learning model from which the calculation was made. Step (a) includes, if the maximum output evaluation value is less than the threshold, a step of generating a new sequence set by replacing one or more of the generated image processing sequences that have a low performance evaluation value regarding the prediction accuracy of the machine learning model with an image processing sequence composed of a new combination of sequence elements, Steps (b) through (e) are performed again for the new sequence set, Step (c) is performed by inputting the intermediate image group into the input layer and repeating the training of the machine learning model so that the layer output approaches the target output a predetermined N times (where N is an integer of 2 or more). The aforementioned target output is expressed as a discrete value, The aforementioned image processing method further includes: (f) The process includes a step in step (c) in which a plurality of intermediate outputs representing feature quantities output from the intermediate layer corresponding to each of the plurality of training images are obtained, the plurality of intermediate outputs are grouped according to the corresponding target output, and an intermediate evaluation value is calculated represented by the degree of variation of the intermediate outputs within the group and the degree of variation between the groups, An image processing method wherein the performance evaluation value is calculated by a function that includes the output evaluation value and the intermediate evaluation value.

3. An image processing method according to claim 1 or claim 2, In step (a), the image processing sequence is represented by an individual having a plurality of genes corresponding to the image input layer and the image processing layer, respectively, and a gene table defining the connection relationships of the sequence elements in the image processing sequence. The image processing method comprises step (a) generating the image processing sequence composed of a new combination of the sequence elements by generating a new individual using at least one of crossover and mutation.

4. An image processing method, (a) A step of generating a sequence set comprising a plurality of image processing sequences, each having a different combination of (i) an image input layer for inputting an image, (ii) at least one of a plurality of image processing layers, and (iii) an image output layer for outputting the image after image processing, wherein (a) the sequence elements include: (b) A step of processing multiple training images by each of the multiple image processing sequences, and for each of the multiple image processing sequences, generating a set of intermediate images which is a collection of multiple intermediate images which are images after image processing, (c) A step of inputting the group of intermediate images for each of the plurality of image processing sequences into the input layer of a plurality of machine learning models of the neural network type, which has an input layer, an intermediate layer and an output layer, and training the machine learning model so that the layer output, which is the output of the output layer, approaches a predetermined target output for each of the plurality of training images, (d) For each of the corresponding machine learning models, a step of comparing the layer output with the target output to calculate an output evaluation value that indicates the degree of similarity between the layer output and the target output, (e) If the maximum output evaluation value, which is the highest of the multiple output evaluation values ​​calculated for each of the multiple machine learning models, is higher than a predetermined threshold, the process includes identifying the machine learning model from which the maximum output evaluation value was calculated and the image processing sequence used to generate the intermediate image group input to the machine learning model from which the calculation was made. Step (a) includes, if the maximum output evaluation value is less than the threshold, a step of generating a new sequence set by replacing one or more of the generated image processing sequences that have a low performance evaluation value regarding the prediction accuracy of the machine learning model with an image processing sequence composed of a new combination of sequence elements, Steps (b) through (e) are performed again for the new sequence set, An image processing method in which, in step (a) or step (e), for the same image processing sequence that constitutes the sequence set used in the routine up to the previous time, the output evaluation value or the performance evaluation value for the image processing sequence that constitutes the sequence set used in the routine up to the previous time is used without executing steps (b) to (d).

5. An image processing device, A sequence generation unit generates a sequence set comprising a plurality of image processing sequences, each having a different combination of the following sequence elements: (i) an image input layer for inputting an image, (ii) at least one of a plurality of image processing layers, and (iii) an image output layer for outputting the image after image processing; An image processing unit that processes multiple training images using each of the multiple image processing sequences and generates an intermediate image group, which is a collection of multiple intermediate images that are images after image processing, for each of the multiple image processing sequences, A learning unit inputs the group of intermediate images for each of the multiple image processing sequences into the input layer of a group of corresponding neural network-type machine learning models having an input layer, an intermediate layer, and an output layer, and performs training on the machine learning model so that the layer output, which is the output of the output layer, approaches a predetermined target output for each of the multiple learning images. For each of the corresponding machine learning models, the system includes an evaluation unit that compares the layer output with the target output and calculates an output evaluation value indicating the degree of similarity between the layer output and the target output. The evaluation unit, if the maximum output evaluation value, which is the highest of the multiple output evaluation values ​​calculated for each of the multiple machine learning models, is higher than a predetermined threshold, identifies the machine learning model from which the maximum output evaluation value was calculated and the image processing sequence used to generate the intermediate image group input to the machine learning model from which the calculation was made. If the maximum output evaluation value is less than the threshold, the sequence generation unit generates a new sequence set by replacing one or more of the generated image processing sequences that have a low performance evaluation value regarding the prediction accuracy of the machine learning model with an image processing sequence composed of a new combination of sequence elements. The layer output and the target output are each represented by vectors. The output evaluation value calculated by the evaluation unit is a value relating to the Euclidean distance. Image processing device.

6. An image processing apparatus, A sequence generation unit generates a sequence set comprising a plurality of image processing sequences, each having a different combination of the following sequence elements: (i) an image input layer for inputting an image, (ii) at least one of a plurality of image processing layers, and (iii) an image output layer for outputting the image after image processing; An image processing unit that processes multiple training images using each of the multiple image processing sequences and generates an intermediate image group, which is a collection of multiple intermediate images that are images after image processing, for each of the multiple image processing sequences, A learning unit inputs the group of intermediate images for each of the multiple image processing sequences into the input layer of a group of corresponding neural network-type machine learning models having an input layer, an intermediate layer, and an output layer, and performs training on the machine learning model so that the layer output, which is the output of the output layer, approaches a predetermined target output for each of the multiple learning images. For each of the corresponding machine learning models, the system includes an evaluation unit that compares the layer output with the target output and calculates an output evaluation value indicating the degree of similarity between the layer output and the target output. The evaluation unit, if the maximum output evaluation value, which is the highest of the multiple output evaluation values ​​calculated for each of the multiple machine learning models, is higher than a predetermined threshold, identifies the machine learning model from which the maximum output evaluation value was calculated and the image processing sequence used to generate the intermediate image group input to the machine learning model from which the calculation was made. If the maximum output evaluation value is less than the threshold, the sequence generation unit generates a new sequence set by replacing one or more of the generated image processing sequences that have a low performance evaluation value regarding the prediction accuracy of the machine learning model with an image processing sequence composed of a new combination of sequence elements. The learning unit inputs the intermediate image group into the input layer and performs training of the machine learning model for a predetermined N times (where N is an integer of 2 or more) so that the layer output approaches the target output. The aforementioned target output is expressed as a discrete value, The evaluation unit acquires a plurality of intermediate outputs representing the feature quantities output from the intermediate layer corresponding to each of the plurality of training images, groups the plurality of intermediate outputs according to the corresponding target output, and calculates an intermediate evaluation value represented by the degree of variation of the intermediate outputs within the group and the degree of variation between the groups. The image processing apparatus wherein the performance evaluation value is calculated by a function that includes the output evaluation value and the intermediate evaluation value.

7. An image processing apparatus, A sequence generation unit generates a sequence set comprising a plurality of image processing sequences, each having a different combination of the following sequence elements: (i) an image input layer for inputting an image, (ii) at least one of a plurality of image processing layers, and (iii) an image output layer for outputting the image after image processing; An image processing unit that processes multiple training images using each of the multiple image processing sequences and generates an intermediate image group, which is a collection of multiple intermediate images that are images after image processing, for each of the multiple image processing sequences, A learning unit inputs the group of intermediate images for each of the multiple image processing sequences into the input layer of a group of corresponding neural network-type machine learning models having an input layer, an intermediate layer, and an output layer, and performs training on the machine learning model so that the layer output, which is the output of the output layer, approaches a predetermined target output for each of the multiple learning images. For each of the corresponding machine learning models, the system includes an evaluation unit that compares the layer output with the target output and calculates an output evaluation value indicating the degree of similarity between the layer output and the target output. The evaluation unit, if the maximum output evaluation value, which is the highest of the multiple output evaluation values ​​calculated for each of the multiple machine learning models, is higher than a predetermined threshold, identifies the machine learning model from which the maximum output evaluation value was calculated and the image processing sequence used to generate the intermediate image group input to the machine learning model from which the calculation was made. If the maximum output evaluation value is less than the threshold, the sequence generation unit generates a new sequence set by replacing one or more of the generated image processing sequences that have a low performance evaluation value regarding the prediction accuracy of the machine learning model with an image processing sequence composed of a new combination of sequence elements. An image processing apparatus in which, for the same image processing sequence that constitutes the sequence set used in the previous routine, the output evaluation value or the performance evaluation value for the image processing sequence that constitutes the sequence set used in the previous routine is used.

8. It is a computer program, (a) A function to generate a sequence set comprising multiple image processing sequences, each having a combination of (i) an image input layer for inputting an image, (ii) at least one of a plurality of image processing layers, and (iii) an image output layer for outputting the image after image processing, wherein the sequence elements are: (b) A function that processes multiple training images using each of the multiple image processing sequences, and generates a set of intermediate images, which is a collection of multiple intermediate images, for each of the multiple image processing sequences, (c) A function that inputs the group of intermediate images for each of the multiple image processing sequences into the input layer of a corresponding group of neural network-type machine learning models having an input layer, an intermediate layer and an output layer, and performs training of the machine learning model so that the layer output, which is the output of the output layer, approaches a predetermined target output for each of the multiple training images, (d) A function to compare the layer output with the target output for each of the corresponding machine learning models and calculate an output evaluation value indicating the degree of similarity between the layer output and the target output, (e) If the maximum output evaluation value, which is the highest of the multiple output evaluation values ​​calculated for each of the multiple machine learning models, is higher than a predetermined threshold, the computer is made to execute a function to identify the machine learning model from which the maximum output evaluation value was calculated and the image processing sequence used to generate the intermediate image group input to the machine learning model from which the calculation was made. Function (a) includes, if the maximum output evaluation value is less than the threshold, a function that generates a new sequence set by replacing one or more of the generated image processing sequences that have a low performance evaluation value regarding the prediction accuracy of the machine learning model with an image processing sequence composed of a new combination of sequence elements, Functions (b) through (e) are executed again for the new sequence set, In the function (d) described above, the layer output and the target output are each represented by vectors, A computer program in which the output evaluation value calculated in the function (d) is a value relating to Euclidean distance.

9. A computer program, (a) A function to generate a sequence set comprising multiple image processing sequences, each having a combination of (i) an image input layer for inputting an image, (ii) at least one of a plurality of image processing layers, and (iii) an image output layer for outputting the image after image processing, wherein the sequence elements are: (b) A function that processes multiple training images using each of the multiple image processing sequences, and generates a set of intermediate images, which is a collection of multiple intermediate images, for each of the multiple image processing sequences, (c) A function that inputs the group of intermediate images for each of the multiple image processing sequences into the input layer of a corresponding group of neural network-type machine learning models having an input layer, an intermediate layer and an output layer, and performs training of the machine learning model so that the layer output, which is the output of the output layer, approaches a predetermined target output for each of the multiple training images, (d) A function to compare the layer output with the target output for each of the corresponding machine learning models and calculate an output evaluation value indicating the degree of similarity between the layer output and the target output, (e) If the maximum output evaluation value, which is the highest of the multiple output evaluation values ​​calculated for each of the multiple machine learning models, is higher than a predetermined threshold, the computer is made to execute a function to identify the machine learning model from which the maximum output evaluation value was calculated and the image processing sequence used to generate the intermediate image group input to the machine learning model from which the calculation was made. Function (a) includes, if the maximum output evaluation value is less than the threshold, a function that generates a new sequence set by replacing one or more of the generated image processing sequences that have a low performance evaluation value regarding the prediction accuracy of the machine learning model with an image processing sequence composed of a new combination of sequence elements, Functions (b) through (e) are executed again for the new sequence set, The function (c) is performed by inputting the intermediate image group into the input layer and repeating the training of the machine learning model so that the layer output approaches the target output a predetermined N times (where N is an integer of 2 or more). The aforementioned target output is expressed as a discrete value, The aforementioned computer program, further, (f) Function (c) includes a function that acquires a plurality of intermediate outputs representing feature quantities output from the intermediate layer corresponding to each of the plurality of training images, groups the plurality of intermediate outputs according to the corresponding target output, and calculates an intermediate evaluation value represented by the degree of variation of the intermediate outputs within the group and the degree of variation between the groups. The performance evaluation value is calculated by a computer program using a function that includes the output evaluation value and the intermediate evaluation value.

10. A computer program, (a) A function to generate a sequence set comprising multiple image processing sequences, each having a combination of (i) an image input layer for inputting an image, (ii) at least one of a plurality of image processing layers, and (iii) an image output layer for outputting the image after image processing, wherein the sequence elements are: (b) A function that processes multiple training images using each of the multiple image processing sequences, and generates a set of intermediate images, which is a collection of multiple intermediate images, for each of the multiple image processing sequences, (c) A function that inputs the group of intermediate images for each of the multiple image processing sequences into the input layer of a corresponding group of neural network-type machine learning models having an input layer, an intermediate layer and an output layer, and performs training of the machine learning model so that the layer output, which is the output of the output layer, approaches a predetermined target output for each of the multiple training images, (d) A function to compare the layer output with the target output for each of the corresponding machine learning models and calculate an output evaluation value indicating the degree of similarity between the layer output and the target output, (e) If the maximum output evaluation value, which is the highest of the multiple output evaluation values ​​calculated for each of the multiple machine learning models, is higher than a predetermined threshold, the computer is made to execute a function to identify the machine learning model from which the maximum output evaluation value was calculated and the image processing sequence used to generate the intermediate image group input to the machine learning model from which the calculation was made. Function (a) includes, if the maximum output evaluation value is less than the threshold, a function that generates a new sequence set by replacing one or more of the generated image processing sequences that have a low performance evaluation value regarding the prediction accuracy of the machine learning model with an image processing sequence composed of a new combination of sequence elements, Functions (b) through (e) are executed again for the new sequence set, A computer program that, in function (a) or function (e), uses the output evaluation value or performance evaluation value for the image processing sequence that constitutes the sequence set used in the previous routine, without executing function (b) to function (d).

Images