Method for generating image processing sequences, apparatus for generating them, and computer program

Abstract

To reduce the possibility that processing time required for specifying an image processing sequence satisfying a reference condition becomes longer.SOLUTION: An image processing sequence generation method includes using a reference table to generate a plurality of image processing sequences so as to have a probability relationship where an input-side sequence element having a higher connection reference value with respect to an output-side sequence element has a higher connection probability of being connected to the output-side sequence element.SELECTED DRAWING: Figure 13

Description

[Technical Field]

[0001] This disclosure relates to a technique for generating image processing sequences. [Background technology]

[0002] Conventionally, there is a known technique for structurally representing individuals in a tree-like structure using genetic programming (Patent Document 1). In this conventional technique, genetic programming is used to generate an image processing sequence that combines multiple image processing filters in a tree structure. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2007-87055 [Overview of the project] [Problems that the invention aims to solve]

[0004] The technology described in Patent Document 1 uses genetic programming to generate multiple image processing sequences and repeatedly evaluates them generation by generation to identify an image processing sequence that satisfies the criteria. However, with conventional technology, it may be necessary to generate individuals over many generations to identify the desired image processing sequence, which can result in a long processing time to identify an image processing sequence that satisfies the criteria. In particular, the processing time to identify an image processing sequence that satisfies the criteria increases when the number of image filters used in the image processing sequence increases or when the gene length increases. [Means for solving the problem]

[0005] According to a first embodiment of this disclosure, a method for generating image processing sequences is provided. This image generation method includes the steps of: (a) generating a sequence set comprising a plurality of image processing sequences, each having a different combination of sequence elements: (i) an image input layer for inputting an image, (ii) at least one of a plurality of image processing layers, and (iii) an image output layer for outputting the image after image processing; (b) processing a training image using each of the plurality of image processing sequences to generate an output image, which is the image after image processing for each of the plurality of image processing sequences; (c) for each of the plurality of image processing sequences, comparing the output image with a target image associated with the training image to calculate an evaluation value indicating the degree of similarity between the output image and the target image; and (d) using a plurality of evaluation values ​​corresponding to each of the plurality of image processing sequences to determine the degree of similarity indicated by the plurality of degrees of similarity. The process comprises a step of identifying an image processing sequence that satisfies predetermined criteria, wherein each of the plurality of image processing sequences is represented by an individual having a plurality of genes arranged corresponding to the image input layer and the image processing layer, respectively, and a reference table defining the connection relationships of the sequence elements in the image processing sequence, the reference table being a table that defines connection reference values ​​for each element set which is a pair of connected input and output sequence elements, indicating the compatibility of connections for processing the image to output a desired image which is the target image, and the process of generating the plurality of image processing sequences using the reference table is performed such that, with respect to the output sequence element, the input sequence element with a higher connection reference value has a higher probability of being connected to the output sequence element.

[0006] According to a second embodiment of this disclosure, an image processing sequence generation apparatus is provided. This generation apparatus comprises: a sequence generation unit that generates a sequence set consisting of a plurality of image processing sequences, each having a different combination of (i) an image input layer for inputting an image, (ii) at least one of a plurality of image processing layers, and (iii) an image output layer for outputting the image after image processing; an image processing unit that processes a learning image using each of the plurality of image processing sequences and generates an output image, which is the image after image processing, for each of the plurality of image processing sequences; and an output evaluation unit that, for each of the plurality of image processing sequences, compares the output image with a target image associated with the learning image and calculates an evaluation value indicating the degree of similarity between the output image and the target image, wherein the output evaluation unit calculates a plurality of evaluation values ​​corresponding to each of the plurality of image processing sequences. Using the aforementioned degree of similarity of multiple elements, the image processing sequence that satisfies predetermined criteria is identified, and each of the multiple image processing sequences is represented by an individual having a plurality of genes arranged corresponding to the image input layer and the image processing layer, and a reference table that defines the connection relationships of the sequence elements in the image processing sequence, the reference table being a table that defines connection reference values ​​for each element set which is a pair of connected input and output sequence elements, indicating the compatibility of the connection for processing the image to output a desired image which is the target image, and the sequence generation unit uses the reference table to generate the multiple image processing sequences such that, for output sequence elements, the input sequence element with a higher connection reference value has a higher probability of being connected to the output sequence element.

[0007] A third embodiment of this disclosure provides a computer program for causing a computer to generate image processing sequences. This computer program has the function of generating a sequence set consisting of a plurality of image processing sequences, each having a different combination of sequence elements: (a) an image input layer for inputting an image, (ii) at least one of a plurality of image processing layers, and (iii) an image output layer for outputting the image after image processing; (b) a function for processing a training image using each of the plurality of image processing sequences and generating an output image which is the image after image processing for each of the plurality of image processing sequences; (c) a function for comparing the output image with a target image associated with the training image for each of the plurality of image processing sequences and calculating an evaluation value indicating the degree of similarity between the output image and the target image; and (d) using a plurality of degrees of similarity indicated by a plurality of evaluation values ​​corresponding to each of the plurality of image processing sequences Function (a) is performed by a computer to identify an image processing sequence that satisfies predetermined criteria, and in function (a), each of the plurality of image processing sequences is represented by an individual having a plurality of genes arranged corresponding to the image input layer and the image processing layer, and a reference table that defines the connection relationships of the sequence elements in the image processing sequence, the reference table being a table that defines connection reference values ​​for each element set which is a pair of connected input and output sequence elements that process the image and output a desired image which is the target image, and function (a) uses the reference table to generate the plurality of image processing sequences such that, with respect to the output sequence element, the input sequence element with a higher connection reference value has a higher probability of being connected to the output sequence element. [Brief explanation of the drawing]

[0008] [Figure 1] This is a diagram illustrating the generation system of an embodiment. [Figure 2]Figure for explaining a memory device. [Figure 3] Figure showing an example of learning data. [Figure 4] Figure for explaining a gene table. [Figure 5] Figure for explaining an individual population of one generation. [Figure 6] Figure showing an image processing sequence of a tree structure generated by a gene translation unit. [Figure 7] Figure for explaining a reference table. [Figure 8] Flowchart showing the processing executed by a generation device. [Figure 9] Detailed flowchart of the generation process of the reference table in step S10. [Figure 10] First figure for explaining the generation process of the reference table. [Figure 11] Second figure for explaining the generation process of the reference table. [Figure 12] Detailed flowchart of the generation process of a specific image processing sequence. [Figure 13] Figure showing the detailed flowchart of step S308. [Figure 14] First figure for explaining the processing from step S316 to S322. [Figure 15] Second figure for explaining the processing from step S316 to S322. [Figure 16] Third figure for explaining the processing from step S316 to S322. [Figure 17] Fourth figure for explaining the processing from step S316 to S322. [Figure 18] Fifth figure for explaining the processing from step S316 to S322. [Figure 19] Flowchart of utilization processing.

Mode for Carrying Out the Invention

[0009] A. Embodiment: Figure 1 is a diagram illustrating a generation system 10 of an embodiment. The generation system 10 comprises a generation device 20 and an imaging device 200. The generation device 20 and the imaging device 200 can transmit and receive data via wired or wireless means. The imaging device 200 is a camera that captures an image of an object and acquires an image. The image acquired by the imaging device 200 is transmitted to the generation device 20.

[0010] The generation device 20 has a sequence generation function that generates a specific image processing sequence DSq to obtain a desired image after image processing, a reference table generation function that is executed when the sequence generation function is performed, and an image processing function. The reference table generation function generates a reference table 88 that is referenced when the sequence generation function is performed. The image processing function is a function that processes an image using the image processing sequence Sq. In this embodiment, the generation device 20 automatically generates a specific image processing sequence DSq to obtain a desired image to achieve a desired purpose based on the captured image. For example, the generation device 20 processes an image of a circuit board, which is an industrial product, captured by the imaging device 200, using the specific image processing sequence DSq, and outputs a binarized image as the desired image in which defective parts are represented as a black image and the rest as a white image. Based on this binarized image, the presence and location of defective parts are identified by the user or the device, and the industrial product is inspected.

[0011] The sequence generation function generates multiple image processing sequences Sq using genetic programming and identifies a specific image processing sequence DSq from among the generated image processing sequences Sq to obtain the desired image after processing. The reference table generation function generates a reference table 88 to determine which sequence element corresponds to the numerical value of a gene located at a gene locus. In this embodiment, the sequence generation function, the reference table generation function, and the image processing function are installed in a single generation device 20, but this is not limited to the present. In other embodiments, the sequence generation function, the reference table generation function, and the image processing function may each be installed in separate devices or may reside in the cloud.

[0012] The generation device 20 is an electronic computer such as a personal computer. The generation device 20 comprises a processor 25, a storage device 70, an input unit 150, and a display unit 160. The processor 25 controls the operation of the generation device 20 by executing various programs stored in the storage device 70. The detailed functions of the processor 25 will be described later. The storage device 70 is composed of memory such as RAM or ROM. The storage device 70 stores various programs for realizing each function of the processor 25 and various data used for generating the image processing sequence Sq. Details of the storage device 70 will be described later. The input unit 150 is an interface that receives information from the outside. For example, the input unit 150 receives input of captured images from the imaging device 200, or input of images generated by the user using other devices. The display unit 160 displays various information. The display unit 160 is, for example, a liquid crystal monitor.

[0013] The processor 25 functions as a sequence generation unit 30, an image processing unit 40, an output evaluation unit 50, a reference table generation unit 55, and a display control unit 60 by executing various programs in the storage device 70. Some of the functions performed by the processor 25 may be implemented by hardware circuits. Hereinafter, the term "processor" encompasses CPUs and GPUs.

[0014] The sequence generation unit 30 generates a sequence set consisting of multiple image processing sequences Sq with different combinations of sequence elements by structurally representing the gene sequence shown by individual IV in a tree structure. The image processing sequence Sq is represented by a combination of multiple sequence elements. The sequence elements include (i) an image input layer, which is a node that receives an image; (ii) an image processing layer, which is an intermediate node between the image input layer and the image output layer, acting as an image processing filter; and (iii) an image output layer, which is a node that outputs the final image after image processing, which is the output image PM. The image processing sequence Sq includes at least one image processing layer.

[0015] The sequence generation unit 30 comprises an individual generation unit 32 and a gene translation unit 34. The individual generation unit 32 generates individual IVs, which are composed of multiple genes arranged in one or two dimensions. Each gene constituting individual IV is a numerical value representing the type of image input layer and image processing layer, and in this embodiment, the image processing filter. Different image input layers and image processing filters are assigned to each numerical range indicating the range of occurrence. In other words, individual IVs are generated by arranging genes corresponding to the image input layer and the image processing layer, respectively. The individual generation unit 32 generates a predetermined number of individual IVs for each generation. In this embodiment, the individual generation unit 32 generates individual IVs with a predetermined gene length by arranging genes in one dimension. In each generation, the combination of sequence elements of multiple image processing sequences Sq is different. For the very first generation, the individual generation unit 32 generates a population 84 composed of multiple individual IVs by randomly placing numerical values ​​from 0 to 1 (up to the third decimal place) as genes at each gene locus of individual IVs. For the second generation and beyond, the individual generation unit 32 generates new individuals IV from among the individuals IV of the previous parent generation that have high evaluation values ​​(described later) using at least one method, such as crossover and mutation, thereby generating a population 84 which is a set of newly generated individuals IV and individuals IV maintained in the parent generation, i.e., individuals IV that have been replicated from the parent generation's individuals IV.

[0016] The gene translation unit 34 translates each gene of individual IV by referring to the gene table 82 and reference table 88, which are stored in the memory device 70, and generates an image processing sequence Sq represented as a tree structure. The gene table 82 and reference table 88 are tables that define the corresponding image processing filters for each numerical range of the numerical value that is a gene, and the connection relationships of each sequence element. The image processing sequence Sq is represented by individual IV and the gene table 82, or by individual IV and the reference table 88.

[0017] The image processing unit 40 processes the image input to the image input layer using the image processing sequence Sq generated by the gene translation unit 34 and the specific image processing sequence DSq stored in the memory device 70, and outputs the processed output image PM from the image output layer.

[0018] The output evaluation unit 50 compares the output image PM, which is the image after image processing obtained by processing the learning image LM with the image processing sequence Sq indicated by individual IV, with the target image TM associated with the learning image LM, and calculates an evaluation value EV that indicates the degree of similarity between the output image PM and the target image TM. The output evaluation unit 50 calculates the evaluation value EV for each of the multiple image processing sequences Sq indicated by multiple individual IVs in one generation G. The calculated evaluation value EV is stored in the storage device 70 in association with the individual IV. The output evaluation unit 50 also uses the calculated evaluation value EV to determine whether the image processing sequence Sq satisfies predetermined criteria, and identifies the image processing sequence Sq that satisfies the criteria as a specific image processing sequence DSq.

[0019] As an evaluation value (EV), one of the following can be used: Mean Squared Error (MSE), Peak Signal-to-Noise Ratio (PSNR), or Structural Similarity (SSIM). When MSE is used as the evaluation value, the smaller the MSE value, the higher the SD (Similarity Standard) between the output image PM and the target image TM. In other words, the reciprocal of MSE indicates the SD. When PSNR is used as the evaluation value, the larger the PSNR, the higher the SD (Similarity Standard) between the output image PM and the target image TM. In other words, PSNR indicates the SD. When SSIM is used, for example, the image is divided into small regions, and the average value of the SSIM calculated for each small region, which is the MSSIM (Mean SSIM), is used as the evaluation value (EV). When MSSIM is used as the evaluation value (EV), the larger the MSSIM, the higher the SD (Similarity Standard). In other words, MSSIM indicates the SD. Note that "SD (Similarity Standard)" is also called "Similarity SD."

[0020] The reference table generation unit 55 generates a reference table 88 using the evaluation value EV calculated by the output evaluation unit 50 and the individual IV associated with the evaluation value EV. The reference table 88 is a table that defines a connection reference value RV for each element set, which is a pair of sequence elements on the connected input and output sides, indicating the compatibility of the connection for processing an image to output a desired image targeting a target image TM. The desired image is an image similar to the target image TM. The method for generating the reference table 88 and the details of the reference table 88 will be described later.

[0021] The display control unit 60 displays various information on the display unit 160. For example, the display control unit 60 displays a specific image processing sequence DSq, an output image PM, a target image TM, and an evaluation value EV on the display unit 160.

[0022] Figure 2 is a diagram illustrating the memory device 70. The memory device 70 stores training data 74, a group of image filters 80, a gene table 82, a population 84, a reference table 88, a specific image processing sequence DSq, and various programs 94. The specific image processing sequence DSq is an image processing sequence Sq that has been determined by the output evaluation unit 50 to satisfy predetermined criteria. The specific image processing sequence DSq is represented, for example, by layer identifiers that identify multiple image processing filters, which are multiple image processing layers, and the order of the layer identifiers, i.e., the connection relationship. The various programs 94 are programs executed by the processor 25.

[0023] Figure 3 shows an example of training data 74. Training data 74 has multiple training sets SM, each consisting of a training image LM, which is the target of image processing by the image processing sequence Sq, and a target image TM associated with the training image LM. The training image LM is, for example, an image of a circuit board, which is an industrial product. The target image TM is a binarized image in which the defective area DA is represented as a black image and the rest as a white image. The target image TM is generated, for example, by the user setting the defective area DA as a black image and the rest as a white image, using the location of the defective area LM, which has been identified in advance, as a reference. Each of the multiple training image LMs is an image of the same type of circuit board with a different manufacturing number, captured by the imaging device 200. Each training image LM and each target image TM corresponding to each training image LM are assigned an identification number.

[0024] The image filter group 80 in Figure 2 is a collection of multiple image filters that are candidates for sequence elements in the image processing sequence Sq. Examples of image filters include mean filters, maximum value filters, minimum value filters, binarization filters, moving average filters, Laplacian filters, Gaussian filters, Sobel filters, gamma correction filters, and filters that combine two images into one. Note that even if image processing filters are of the same type, if their kernel size or coefficients differ, they are stored in the image filter group 80 as different image processing filters.

[0025] Gene Table 82 is a table that shows the relationship between the connection of sequence elements that are candidates for the image processing sequence Sq and the type of image processing filter that corresponds to the numerical value indicated by the gene.

[0026] Figure 4 is a diagram illustrating the gene table 82. For ease of understanding, it is assumed below that the image filter group 80 consists of five image processing filters FtA to FtE. In reality, more than five image processing filters are stored in the image processing filter group. The gene table 82 defines the number of inputs, the number of outputs, and the occurrence range for each filter type, which indicates the type of image processing filter. The filter type is an identifier that identifies each image processing filter in the image filter group 80 and the image input layer. In Figure 4, the image input layer is represented by the filter type "in". The number of inputs indicates the number of sequence elements connected to the input side of the image processing filter or image input layer. If the number of inputs is "2", the data output from two sequence elements is input to the image processing filter. The number of outputs indicates the number of sequence elements connected to the output side of the image processing filter. In this embodiment, the number of outputs is always "1", and the image processing layer and image input layer are connected to one output-side sequence element.

[0027] The appearance range of gene table 82 defines the range of the gene's numerical value VG. In this embodiment, the numerical values ​​from 0 to 1 are divided into six equal parts to set the appearance range, so that each image processing filter appears randomly with equal probability in the image processing sequence Sq. For example, if the numerical value VG of a gene is between 0 and 0.167, the image processing filter FtA is assigned to this gene.

[0028] The population 84 shown in Figure 2 is a collection of multiple individuals IV in one generation generated by the individual generation unit 32. Figure 5 is a diagram illustrating the population 84 of one generation G. In this embodiment, we describe an example in which five individuals IV1 to IV5 are generated in one generation G. Each individual IV1 to IV5 has multiple gene loci arranged one-dimensionally. The gene lengths of each individual IV1 to IV5 are the same, and each individual IV1 to IV5 is constructed by sequentially arranging genes in gene loci No. 1 to No. 10. The individual generation unit 32 places genes represented by numerical values ​​from 0 to 1 to three decimal places in each gene locus. In other words, in this embodiment, the values ​​that a gene can take at each gene locus are numerical values ​​from 0 to 1 to three decimal places.

[0029] Before further explaining Figure 2, we will use Figure 6 to explain the process of generating the image processing sequence Sq by the gene translation unit 34. Figure 6 shows the image processing sequence Sq of the tree structure generated by the gene translation unit 34. The image processing sequence Sq shown in Figure 6 is image processing sequence Sq1, which was generated based on individual IV1 shown in Figure 5.

[0030] In this embodiment, the gene translation unit 34 generates image processing sequences Sq in two ways: pre-generation and specific generation. The pre-generation process is performed when the reference table generation function is executed and the reference table 88 is generated. The specific generation process is performed when the sequence generation function using the reference table 88 is executed. The difference lies in the table referenced when generating image processing sequences Sq: the pre-generation process uses the gene table 82, while the specific generation process uses the reference table. Since the algorithm for generating image processing sequences Sq from individual IV is the same in both the pre-generation and specific generation processes, the generation of image processing sequences Sq in the pre-generation process will be described first below.

[0031] In the pre-generation process, the gene translation unit 34 uses individual IV, the gene table 82, and the image filter group 80 to generate a tree-structured image processing sequence Sq starting from the image output layer out. As shown in Figure 6, the gene translation unit 34 first sets the image output layer out as the terminal node, which is the end of the output side, and generates the image processing sequence Sq by connecting sequence elements to the input side in the order of the gene sequence of individual IV. The connection of sequence elements is performed according to predetermined connection rules. In this embodiment, the image processing sequence Sq is generated according to the rules that prioritize placement from the terminal node side in the order of the gene sequence, and that the connection destination on the left side shown in Figure 6 is prioritized over the connection destination on the right side. In other words, the image processing layers or image input layers in are placed from the image output layer out side in the order of the gene sequence, and if the input side is divided into multiple parts, the left side is prioritized until the sequence on the left becomes the image input layer in. In this embodiment, the number of inputs to the image output layer out is set to "1" in advance.

[0032] The gene translation unit 34 refers to the numerical value VG of gene No. 1 of individual VI1, "0.534," and the gene table 82 to determine that "0.534" is within the range of occurrence of image processing filter FtD. This connects image processing filter FtD to the input side of the image output layer out. Next, the gene translation unit 34 refers to the gene table 82 to determine that the number of inputs to image processing filter FtD is "2," thereby placing two sequence elements on the input side of image processing filter FtD. Specifically, the gene translation unit 34 places the sequence element corresponding to gene No. 2 of individual VI1 at the left input connection point shown in Figure 6. That is, the gene translation unit 34 refers to the numerical value VG of gene No. 2, "0.292," and the gene table 82 to determine that "0.292" is within the range of occurrence of image processing filter FtB. This places image processing filter FtB at the left input connection point of image processing filter FtD.

[0033] Next, the gene translation unit 34 refers to the image processing filter FtB represented by gene No. 2 and the gene table 82 to determine that the number of inputs to image processing filter FtB is "1", and places one sequence element on the input side of image processing filter FtB. Specifically, the gene translation unit 34 refers to the numerical value VG of gene No. 3, "0.462", and the gene table 82 to determine that "0.462" is within the range of appearance of image processing filter FtC. As a result, image processing filter FtC is placed at the input connection destination of image processing filter FtB indicated by gene No. 3.

[0034] Next, the gene translation unit 34 refers to the image processing filter FtC represented by gene No. 3 and the gene table 82 to determine that the number of inputs to the image processing filter FtC is "1," and places one sequence element on the input side of the image processing filter FtC. Specifically, the gene translation unit 34 refers to the numerical value VG of gene No. 4, "0.856," and the gene table 82 to determine that "0.856" is within the range of appearance of the image input layer in. This places the image input layer in at the input connection destination of the image processing filter FtC indicated by gene No. 3. This completes the generation of the sequence on the left side of the image processing sequence Sq in Figure 6.

[0035] Next, the gene translation unit 34 places the sequence element corresponding to gene No. 5 of individual VI1 at the input connection destination on the right side as shown in Figure 6. In other words, the gene translation unit 34 refers to the numerical value VG of gene No. 5, "0.138," and the gene table 82 to determine that "0.138" is within the range of appearance of image processing filter FtA. As a result, image processing filter FtA is placed at the input connection destination on the right side of image processing filter FtD.

[0036] Furthermore, the gene translation unit 34 refers to the image processing filter FtA represented by gene No. 5 and the gene table 82 to determine that the number of inputs to the image processing filter FtA is "1," and places one sequence element on the input side of the image processing filter FtA. Specifically, the gene translation unit 34 refers to the numerical value VG of gene No. 6, "0.932," and the gene table 82 to determine that "0.932" is within the range of appearance of the image input layer in. As a result, the image input layer in is placed at the input connection destination of the image processing filter FtA indicated by gene No. 5. With this, the generation of the sequence on the right side of the image processing sequence Sq in Figure 6 is completed.

[0037] The gene translation unit 34 terminates the image processing sequence Sq generation process using the pre-generation process if all input sequence elements become image input layer in, even if not all genes of individual IV1 are used. Furthermore, if all genes of individual IV1 are used and not all input sequence elements become image input layer in, the gene translation unit 34 terminates the image processing sequence Sq generation process by setting the remaining unconnected input sequence elements to image input layer in.

[0038] The image processing sequence Sq1, represented by individual VI1, has two image input layers in. An image input to one image input layer in is processed by image processing filter FtC, then by image processing filter FtB, to generate a first processed image. An image input to the other image input layer in is processed by image processing filter FtA to generate a second processed image. The first and second processed images are then processed by image processing filter FtD, and a single output image PM is output by the image output layer out.

[0039] The reference table 88 shown in Figure 2 is referenced by the gene translation unit 34 when it generates an image processing sequence Sq from individual IV through a specific generation process. Figure 7 is a diagram illustrating the reference table 88. The reference table 88 defines an input-side identifier indicating the type of candidate input-side sequence element to be placed on the input side, an output-side identifier indicating the type of candidate output-side sequence element to be connected to the input-side sequence element, the number of inputs, the number of outputs, and a connection reference value RV. The input-side sequence element consists of an image processing filter FtA~FtE and an image input layer in. The output-side sequence element consists of an image processing filter FtA~FtE and an image output layer out. In other words, the reference table 88 defines an element set which is a pair of connected input and output sequence elements. The number of inputs indicates the number of sequence elements connected to the input side of the input-side sequence element. The number of outputs indicates the number of sequence elements connected to the output side of the input-side sequence element. The average value ARV as the connection reference value RV is a value defined for each element set and indicates the compatibility of the connection for processing an image to output a desired image targeting the target image TM. For example, the average ARV of an element set with filter FtA connected to the input side and filter FtC connected to the output side is "0.478". The average ARV is determined by the reference table generation function. In this embodiment, the desired image is a binarized image in which defective parts are represented as black images and the rest as white images, and is similar to the target image TM shown in Figure 3.

[0040] Figure 8 is a flowchart showing the processes performed by the generation device 20. First, in step S10, the reference table generation function of the generation device 20 performs the process of generating the reference table 88. Next, in step S30, the sequence generation function of the generation device 20 performs the process of generating a specific image processing sequence DSq using the reference table 88. Then, in step S50, the image processing function of the generation device 20 performs the process of utilizing the specific image processing sequence DSq.

[0041] Figure 9 is a detailed flowchart of the process for generating the reference table 88 in step S10. Figure 10 is the first diagram illustrating the process for generating the reference table 88. Figure 11 is the second diagram illustrating the process for generating the reference table 88.

[0042] As shown in Figure 9, first, in step S100, the storage device 70 stores a plurality of image processing filters as an image filter group 80. Step S100 is executed when the user inputs a plurality of candidate image processing filters to be used in a specific image processing sequence DSq to the generation device 20. In step S101, the storage device 70 stores training data 74 consisting of a plurality of training sets SM. Step S101 is executed when the user inputs the training data 74 to the generation device 20. In step S102, the reference table generation unit 55 initializes the reference table 88. Specifically, as shown in Figure 10, the reference table generation unit 55 sets all the average values ​​ARV, which are the values ​​of each element set in the reference table 88, to zero. Note that the order of steps S100, S101, and S102 is not limited to this.

[0043] Next, as shown in Figure 9, in step S103, the individual generation unit 32 generates a population 84 consisting of multiple individuals IV for the current generation G. A population 84 in one generation G consists of multiple individuals IV1 to IV5. The generated population 84 is stored in the memory device 70. Next, in step S104, the gene translation unit 34 selects one of the individuals IV1 to IV5 and translates the selected individual IV by referring to the gene table 82 to generate an image processing sequence Sq.

[0044] Next, in step S105, the image processing unit 40 processes multiple training images LM using the image processing sequence Sq generated in step S104 to generate multiple output images PM. Next, in step S109, the output evaluation unit 50 compares the multiple output images PM generated in step S105 with the target image TM associated with the training image LM that each of the multiple output images PM was generated from, and calculates multiple evaluation values ​​EV to evaluate the processing result. The output evaluation unit 50 uses the average of the multiple evaluation values ​​EV as the final evaluation value EV that indicates a similar degree SD. Also, in step S109, the output evaluation unit 50 calculates a connection reference value RV from the final evaluation value EV. In this embodiment, the output evaluation unit 50 uses a similar degree SD as the connection reference value RV. That is, if the final evaluation value EV is the mean squared error, the connection reference value RV is the reciprocal of the evaluation value EV. Also, if the final evaluation value EV is PNSR, MSE, or SSIM, the connection reference value RV is the value of the evaluation value EV.

[0045] Next, in step S110, the reference table generation unit 55 updates the reference table 88. For example, if the connection reference value RV calculated from the final evaluation value EV of the image processing sequence Sq1 shown in Figure 6 is "0.783", then all element sets of the pair of input-side sequence elements and output-side sequence elements constituting the image processing sequence Sq1 are extracted, and the connection reference value RV, "0.783", is added to the column of the reference table 110 representing the extracted element set. Here, in the column of the reference table 110 representing one element set, the average value ARV obtained by dividing by the number of additions is defined as the final connection reference value RV. That is, the connection reference value RV after the nth time n ARV is the average value of the element set calculated by adding the values ​​of the elements. n This is calculated using the following formula (1).

[0046]

number

[0047] If the same element set exists in the image processing sequence Sq1, the connection reference value RV may be added for each identical element set, or the connection reference value RV may be added only once. In this embodiment, even if there are multiple identical element sets in a single image processing sequence Sq1, the connection reference value RV is added only once. When the element sets of the input-side sequence elements and output-side sequence elements are represented as [input-side sequence elements, output-side sequence elements], the image processing sequence Sq1 shown in Figure 6 has the following element sets. The reference table generation unit 55 calculates the average value ARV by adding the connection reference value "0.783" to the corresponding element set column in the reference table 88 for each of the following element sets, as shown in Figure 10. [Image processing filter FtD, Image output layer out] [Image processing filter FtB, Image processing filter FtD] [Image processing filter FtC, Image processing filter FtB] [Image input layer in, Image processing filter FtC] [Image processing filter FtA, Image processing filter FtD] [Image input layer in, Image processing filter FtA]

[0048] As shown in Figure 9, in step S111, the reference table generation unit 55 determines whether the processes in steps S104 to S110 have been executed for all individual IVs of a single generation G. If the determination in step S111 is "No", the processor 25 selects one individual IV from among the individual IVs of a single generation G that has not yet undergone steps S104 to S110 and executes the processes in steps S104 to S111 again.

[0049] On the other hand, when the determination in step S111 is "Yes", the reference table generation unit 55 determines whether the end condition is satisfied in step S113. The end condition is the condition that the processing routine from step S103 to step S111 has been executed a predetermined number of times in advance. The predetermined number of times in advance is set to, for example, the number of times such that the connection reference value RV is added at least once, preferably two or more times, to all columns of the element set of the reference table 88. For example, the number of times in advance is set such that the processing routine from step S104 to step S111 is executed for at least twice the number of element sets defined in the reference table 88 of the individuals IV.

[0050] As described above, the reference table 88 is generated by using each of the plurality of image processing sequences Sq generated by the process of step S104 so as to have a relationship in which the input-side sequence element is connected to the output-side sequence element with a random probability in a pair of sequence elements. Thus, the degree of similarity SD generated thereby is used as the connection reference value RV, whereby the possibility that an element set with high connection compatibility is included in the image processing sequence Sq can be increased.

[0051] When the end condition is not satisfied, the individual generation unit 32 re-executes step S103 in the next processing routine. Step S103 to be re-executed is the individuals IV of the parent generation G m generated in step S103 in the previous processing routine m 1 to IM m 5 among which the degree of similarity SD is up to the lower J-th m as the new individuals IV nThis is executed by replacing it with "J", where "J" is an integer greater than or equal to 1. In other words, the individual generation unit 32 duplicates individual IVs whose degree of similarity SD does not fall within the lower Jth position, and generates new individual IVs by performing at least one of crossover or mutation on individual IVs that do not fall within the lower Jth position. This makes it easy to generate a new image processing sequence Sq with a different combination of sequence elements from the image processing sequence Sq generated previously. Individual IV generated by step S103, which is executed again. n The process from step S104 onwards is executed again for the target.

[0052] On the other hand, if the termination condition is met, the reference table generation unit 55 outputs the reference table 88 at the time the termination condition was met in step S115 and stores it in the storage device 70.

[0053] Figure 12 is a detailed flowchart of the process for generating a specific image processing sequence DSq. First, in step S300, multiple image processing filters are stored as an image filter group 80. Step S300 is executed when the user inputs multiple candidate image processing filters to be used in the specific image processing sequence DSq into the generation device 20. Note that if the process for generating the specific image processing sequence DSq is executed on the same generation device 20 that performed the process for generating the reference table 88, the image filter group 80 is already stored in the storage device 70. Therefore, in this case, step S300 can be omitted.

[0054] Furthermore, in step S301, the storage device 70 stores the learning data 74, which is composed of multiple learning sets SM. Step S301 is executed when the user inputs the learning data 74 to the generation device 20. Note that if the generation process of a specific image processing sequence DSq is performed on the same generation device 20 in which the generation process of the reference table 88 was performed, and the same learning data 74 is used, step S301 can be omitted. On the other hand, for example, if the learning data 74 is generated from a circuit board that is the source of the learning data 74 used in the generation process of the reference table 88, and from a circuit board of the same type but manufactured in a different factory, or if the purpose of checking for the presence or absence of defective parts DA is the same but the product from which the learning data 74 is based is different, then step S301 is executed.

[0055] In step S304, the storage device 70 stores the reference table 88 generated during the reference table 88 generation process. Note that if the generation process for the specific image processing sequence DSq is performed on the same device 20 that performed the reference table 88 generation process, the reference table 88 is already stored in the storage device 70. Therefore, in this case, step S304 can be omitted. Note that the order of steps S300 to S304 is not limited to the order described above.

[0056] Next, in step S306, the individual generation unit 32 generates multiple individuals IV for the current generation G. n A population 84 consisting of the above is generated. Step S306 is the same process as step S103 in Figure 9. That is, in step S306, the population 84 in one generation G is composed of multiple individuals IV1 to IV5. The generated population 84 is stored in the memory device 70.

[0057] Next, in step S308, the gene translation unit 34 selects one of the individuals IV1 to IV5 and translates the selected individual IV by referring to the reference table 88 to generate the image processing sequence Sq. The generation of the image processing sequence Sq in step S308 is performed by the specific generation process described above. Details of step S308 will be described later.

[0058] Next, in step S350, the image processing unit 40 uses the image processing sequence Sq generated in step S308 to process multiple training images LM and generate multiple output images PM.

[0059] Next, in step S352, the output evaluation unit 50 evaluates the processing result by comparing the multiple output images PM generated in step S350 with the target image TM associated with the training image LM that is the source of each of the multiple output images PM, and calculating multiple evaluation values ​​EV. The output evaluation unit 50 uses the average of the multiple evaluation values ​​EV as the final evaluation value EV, which indicates the degree of similarity SD between the output image PM and the target image TM.

[0060] Next, in step S354, the output evaluation unit 50 determines whether the processing in steps S308 to S352 has been performed on all individuals IV of a single generation G. If the determination in step S354 is "No", the processor 25 selects one individual IV from among the individuals IV of a single generation G that has not yet performed steps S308 to S352 and performs the processing in steps S308 to S352 again. In other words, the image processing unit 40 processes the learning image LM using multiple image processing sequences Sq corresponding to multiple individuals IV constituting a single generation G, and generates an output image PM for each of the multiple image processing sequences Sq. Then, the output evaluation unit 50 compares the multiple output images PM generated for each of the multiple image processing sequences Sq with the corresponding target image TM and calculates an evaluation value EV.

[0061] On the other hand, if a "Yes" determination is made in step S354, the output evaluation unit 50 determines in step S356 whether there is an image processing sequence Sq that satisfies predetermined criteria using multiple evaluation values ​​EV calculated in the current generation. Specifically, the evaluation determination unit 46 determines that there is an image processing sequence Sq that satisfies the criteria if the highest degree of similarity SDmax among the multiple degree of similarity SDs indicated by the multiple evaluation values ​​EV calculated based on multiple individual IVs of the current generation is equal to or greater than a predetermined termination threshold. In other words, the process in step S356 is a process of identifying an image processing sequence Sq that satisfies predetermined criteria using the degree of similarity SDs indicated by the multiple evaluation values ​​EV corresponding to each of the multiple image processing sequences Sq.

[0062] If there is an image processing sequence Sq that satisfies the criteria, the output evaluation unit 50 outputs the image processing sequence Sq that satisfies the criteria as a specific image processing sequence DSq in step S358. The output specific image processing sequence DSq is stored in the storage device 70.

[0063] On the other hand, if there is no image processing sequence Sq that satisfies the criteria, the processing in steps S306 to S356 is executed as the next routine. In other words, in step S306 of the next routine, the individual generation unit 32 generates a sequence set by replacing one or more target image processing sequences Sq, which are among the multiple image processing sequences Sq generated in the previous routine that have a low degree of similarity SD indicated by the evaluation value EV, with a new image processing sequence Sq with different sequence elements. In this embodiment, the individual generation unit 32 generates new individual IVs for the current generation in the current routine using at least one of crossover and mutation, based on individual IVs whose degree of similarity SD does not fall within the lower Jth position among the multiple individual IVs of the parent generation used in the previous routine. The individual generation unit 32 also duplicates individual IVs whose degree of similarity SD does not fall within the lower Jth position among the multiple individual IVs of the parent generation. A new sequence set is formed by the duplicated individual IVs and the individual IVs newly generated using crossover and mutation. Then, the processing in steps S308 to S356 is executed again for the new sequence set. In other embodiments, among the multiple individual IVs of the parent generation, for individual IVs whose degree of similarity SD is less than or equal to a predetermined replication criterion value, new individual IVs may be generated using crossover or sudden displacement based on individual IVs whose degree of similarity SD is greater than the replication criterion value. As described above, by replacing one or more image processing sequences Sq with a low degree of similarity SD with new image processing sequences with different combinations of sequence elements generated based on image processing sequences Sq with a high degree of similarity SD, image processing sequences Sq with a low degree of similarity SD can be excluded from processing. This makes it possible to identify image processing sequences Sq that satisfy the criterion conditions more efficiently.

[0064] In the processing steps S308 to S356 targeting a new sequence set, for image processing sequences Sq that are the same as those used in the previous routine, the evaluation value EV calculated in the previous routine is used. This eliminates the need to recalculate the evaluation value EV, thereby improving the processing efficiency of the specific image processing sequence generation process.

[0065] Figure 13 is a diagram showing a detailed flowchart of step S308. First, in step S311, the gene translation unit 34 selects one of the IV1 to IV5 that constitute the current generation G that has not been processed. Then, in step S314, the gene translation unit 34 reads the reference table 88 stored in the memory device 70.

[0066] Next, in step S316, the gene translation unit 34 selects one connection destination, which is an unconnected output sequence element to which no input sequence elements are connected. In this embodiment, the image output layer out is set as the terminal node, and the sequence elements are connected sequentially from the image output layer out side. Therefore, in step S316 of the first routine, the gene translation unit 34 selects the image output layer out as the connection destination. The connection of the sequence elements is performed according to the predetermined connection rules as described above. As described above, the gene translation unit 34 first sets the image output layer out as the terminal node, which is the output end, and generates the image processing sequence Sq by connecting the sequence elements to the input side in the order of the gene sequence of individual IV. Furthermore, as described above, the gene translation unit 34 arranges the elements in the order of the gene sequence, prioritizing the terminal node side, and also prioritizes the connection destination on the left side shown in Figure 6 over the connection destination on the right side.

[0067] Following step S316, in step S318, the gene translation unit 34 selects an input sequence element to be connected to the output sequence element by referring to the reference table 88. Specifically, the gene translation unit 34 uses the reference table 88 to generate an image processing sequence Sq such that the higher the average value ARV as a connection reference value RV for the output sequence element, the higher the probability of connection to the output sequence element. The image processing sequence Sq having the above probability relationship is generated when an input sequence element is selected according to the following input sequence element selection algorithm.

[0068] <Selection algorithm for input sequence elements> The prerequisites for the selective algorithm are as follows: • Genetic value: VG (VG is a number between 0 and 1, with up to three decimal places) • Output-side sequence element identifier ···j Here, j is represented by a non-negative integer, and a different value is assigned to each output sequence element. In this embodiment, the identifier j is assigned values ​​from 0 to 5 for each input sequence element, FtA to out. • Input sequence element identifier to be connected ···i Here, i is represented by an integer greater than or equal to 0, from 0 to N. f It is represented by identifiers up to . Nf is the number of candidate input sequence elements minus 1, and in this embodiment, as shown in Figure 7, the number of input sequence elements is 6, so N f It is 5. Also, in the order of the input sequence elements FtA~in, the identifier i is assigned a value from 0 to 5. • Values ​​of the element set in reference table 88...t ij t ij This is the average ARV shown in Figure 7, and the subscript "ij" is an identifier used to identify the element set, corresponding to the identifiers of the input and output sequence elements. For example, t 12In Figure 7, this represents 0.086, which is the average ARV of the element set where the input sequence element is image processing filter FtB and the output sequence element is image processing filter FtC.

[0069] In the selection algorithm, the probability P is that input sequence element i is connected to the input side of output sequence element j. ij It is calculated by the following formula (2).

number

[0070] Furthermore, in the selection algorithm, the upper threshold T of the numerical range of the gene to which the input sequence element i is connected to the output sequence element j. ij This is calculated using the following formula (3). The lower threshold is set to the upper threshold of the input sequence element corresponding to the previous identifier. If there is no previous identifier, the lower threshold is set to "0".

number

[0071] By determining the numerical range indicating the gene's appearance range using the selection algorithm described above, an image processing sequence Sq is generated such that the ratio of the average ARV values ​​(each connection reference value) for each input sequence element, which are connection candidates for the same output sequence element, becomes the ratio of the connection probabilities for connecting to the output sequence element.

[0072] The gene translation unit 34 uses the upper threshold calculated by equation (3) above to set different numerical ranges as occurrence ranges for each of the input sequence elements, namely the image input layer in and the multiple image processing filters FtA to FtE, which are multiple image processing layers. Then, it selects the image input layer in or the image processing filter as the input sequence element whose gene value corresponds to the set numerical range.

[0073] Following step S318, in step S320, the gene translation unit 34 determines whether the generation of the image processing sequence Sq has been completed. Specifically, the gene translation unit 34 determines that the generation of the image processing sequence Sq has been completed if all upstream nodes are in the image input layer and there are no unconnected nodes. In this case, the gene translation unit 34 outputs the image processing sequence Sq generated in step S322 to the image processing unit 40. On the other hand, if the gene translation unit 34 determines that the generation of the image processing sequence Sq has not been completed, it executes the processing from step S316 onwards again.

[0074] The selection of input sequence elements using the above selection algorithm will be explained below using a specific example of generating an image processing sequence Sq from individual IVa. Figure 14 is the first diagram illustrating the process from step S316 to step S322 in Figure 13. Figure 15 is the second diagram illustrating the process from step S316 to step S322 in Figure 13. Figure 16 is the third diagram illustrating the process from step S316 to step S322 in Figure 13. Figure 17 is the fourth diagram illustrating the process from step S316 to step S322 in Figure 13. Figure 18 is the fifth diagram illustrating the process from step S316 to step S322 in Figure 13.

[0075] As shown in Figure 14, in the first routine Rt1, the gene translation unit 34 selects the image output layer out as the connection destination in step S316. Next, the gene translation unit 34 uses the average value ARV of the element set whose output-side sequence element in the reference table 88 shown in Figure 7 is the image output layer out, and equations (2) and (3) above to calculate the upper threshold T ijBy calculating this, the numerical range in which the input sequence element appears is determined. As shown in Figure 15, the gene translation unit 34 executes the selection algorithm. As a result, for each input sequence element in the element set whose output sequence element is the image output layer out, the gene translation unit 34 sets a wider numerical range indicating the appearance range for input sequence elements with a higher average value ARV, which is the connection reference value RV. This generates an image processing sequence Sq having the above connection relationship. The gene translation unit 34 compares the numerical value of gene No. 1, 0.742, shown in Figure 14, with the numerical range shown in Figure 15, and selects the image processing filter FtE that falls within the numerical range as the input sequence element for the image output layer out. The gene translation unit 34 also refers to the reference table 88 and, since the number of inputs for image processing filter FtE is "2", sets two search branches on the input side of image processing filter FtB.

[0076] As shown in Figure 14, in the next routine Rt2, the gene translation unit 34 selects the image processing filter FtE as the connection destination in step S316. Next, the gene translation unit 34 uses the average value ARV of the element set whose output sequence element in the reference table 88 shown in Figure 7 is the image processing filter FtE, and the above-mentioned equations (2) and (3) to calculate the upper threshold T ij By calculating this, the numerical range of the input sequence element is determined. As shown in Figure 16, the gene translation unit 34 executes the selection algorithm. As a result, for each input sequence element in the element set that the output sequence element is the image processing filter FtE, the gene translation unit 34 sets a wider numerical range indicating the occurrence range for input sequence elements with a higher average ARV. The gene translation unit 34 compares the numerical value of gene No. 2, 0.942, shown in Figure 14, with the numerical range shown in Figure 16, and selects the image input layer in, which falls within the numerical range, as the input sequence element on the left side of the image processing filter FtB.

[0077] As shown in Figure 14, in the next routine Rt3, the gene translation unit 34 selects the image processing filter FtE as the connection destination in step S316. Next, the gene translation unit 34 uses the average value ARV of the element set whose output sequence element in the reference table 88 shown in Figure 7 is the image processing filter FtE, and the above-mentioned equations (2) and (3) to calculate the upper threshold T ij By calculating this, the numerical range of the input sequence element is determined. Since this numerical range is the same as that in Figure 16, the gene translation unit 34 uses the numerical range used in the previous routine Rt2. The gene translation unit 34 compares the numerical value of gene No. 3, 0.075, shown in Figure 14, with the numerical range shown in Figure 16, and selects the image processing filter FtA, which falls within the numerical range, as the input sequence element to the right of image processing filter FtB. The gene translation unit 34 also refers to the reference table 88 and, since the number of inputs for image processing filter FtA is "1", sets one search branch on the input side of image processing filter FtA.

[0078] As shown in Figure 14, in the next routine Rt4, the gene translation unit 34 selects the image processing filter FtA as the connection destination in step S316. Next, the gene translation unit 34 uses the average value ARV of the element set whose output sequence element in the reference table 88 shown in Figure 7 is the image processing filter FtA, and the above-mentioned equations (2) and (3) to calculate the upper threshold T ijBy calculating this, the numerical range of the input sequence element is determined. As shown in Figure 17, the gene translation unit 34 executes the selection algorithm. As a result, for each input sequence element of the element set that the output sequence element is the image processing filter FtA, the gene translation unit 34 sets a wider numerical range indicating the occurrence range for input sequence elements with a higher average ARV. The gene translation unit 34 compares the numerical value of gene No. 4, 0.412, shown in Figure 14, with the numerical range shown in Figure 17, and selects the image processing filter FtB that falls within the numerical range as the input sequence element of the image processing filter FtA. The gene translation unit 34 also refers to the reference table 88 and, since the number of inputs for image processing filter FtB is "1", sets one search branch on the input side of image processing filter FtA.

[0079] As shown in Figure 14, in the next routine Rt5, the gene translation unit 34 selects the image processing filter FtB as the connection destination in step S316. Next, the gene translation unit 34 uses the average value ARV of the element set whose output sequence element in the reference table 88 shown in Figure 7 is the image processing filter FtB, and the above-mentioned equations (2) and (3) to calculate the upper threshold T ijBy calculating this, the numerical range of the input sequence element is determined. As shown in Figure 18, the gene translation unit 34 executes the selection algorithm. As a result, for each input sequence element of the element set that the output sequence element is the image processing filter FtB, the gene translation unit 34 sets a wider numerical range indicating the occurrence range for input sequence elements with a higher average ARV. The gene translation unit 34 compares the numerical value of gene No. 5, 0.995, shown in Figure 14, with the numerical range shown in Figure 18, and selects the image input layer in that falls within the numerical range as the input sequence element of the image processing filter FtB. As a result, all upstream nodes have become image input layers in, and the generation of the image processing sequence Sq is completed. In other words, if there are no input sequence elements that are not connected to the output sequence element, the gene translation unit 34 terminates the generation process of the image processing sequence Sq corresponding to individual IV, even if not all of the multiple genes constituting individual IV are used as the image processing sequence Sq. As a result, the gene translation unit 34 can generate the image processing sequence Sq even if not all of the multiple genes are used.

[0080] Figure 19 is a flowchart of the utilization process. The utilization process is executed after the specific image processing sequence DSq is identified by the generation process in Figure 12. First, in step S500, the storage device 70 stores the multiple image processing filters received by the input unit 150 as an image filter group 80. The multiple image processing filters in the image filter group 80 stored in step S500 may be only the image processing filters used in the specific image processing sequence DSq. Note that if the utilization process and the generation process in Figure 12 are executed by the same generation device 20, step S500 can be omitted, and the image filter group 80 stored in the storage device 70 in step S300 can be used.

[0081] Next, in step S501, the storage device 70 stores the specific image processing sequence DSq output in step S358 of Figure 12. Note that the order of steps S500 and S501 is not limited to that above.

[0082] Next, in step S503, the image processing unit 40 acquires an image that is the subject of image processing for the specific image processing sequence DSq by reading it from the storage device 70 via the input unit 150. The image acquired in step S503 is, for example, an image of the same type of object as the training image LM. If there are multiple images to be processed, the image processing unit 40 may acquire an image by reading one of the multiple images pre-stored in the storage device 70, or it may acquire the target image from the imaging device 200 each time it is time to execute the image processing described later.

[0083] Next, in step S504, the image processing unit 40 performs image processing on the image acquired in step S503 using a specific image processing sequence DSq. That is, it applies image processing filters stored in the image filter group 80 of the storage device 70 to the image in the order indicated by the specific image processing sequence DSq.

[0084] Next, in step S505, an output image PM, which is the image after image processing using the specific image processing sequence DSq, is output. Step S505 is executed, for example, by storing the output image PM in the storage device 70 or by displaying it on the display unit 160.

[0085] Next, in step S506, the image processing unit 40 determines whether to continue image processing using the specific image processing sequence DSq. For example, the image processing unit 40 determines to continue image processing if information indicating the continuation of image processing is input from the user, or if there are still unprocessed images to be processed remaining in the storage device 70. If image processing is to be continued, the processing from step S503 onwards is executed again. On the other hand, the image processing unit 40 terminates the usage process if information indicating the termination of image processing is input from the user, or if there are no unprocessed images to be processed remaining in the storage device 70.

[0086] According to the above embodiment, using the reference table 88, multiple image processing sequences Sq can be generated such that, for output sequence elements, the higher the connection reference value RV of the input sequence element, the higher the probability of connection to the output sequence element. This makes it possible to efficiently identify image processing sequences Sq that satisfy the criteria without increasing the number of image processing sequences Sq to be generated. In other words, since image processing sequences Sq that satisfy the criteria can be identified with fewer image processing sequences Sq, the processing time required to identify image processing sequences Sq that satisfy the criteria can be reduced. In particular, when generating more complex specific image processing sequences DSq by increasing the types of image processing layers that are candidates for sequence elements or by increasing the gene length, the increase in processing time due to the expansion of the image processing sequence search space can be suppressed by using the reference table 88. Furthermore, according to the above embodiment, the common reference table 88 can also be used to identify specific image processing sequences DSq for achieving similar objectives. For example, this applies when the objective of determining whether or not there is a defective part DA is common, but the target products are different, or when the objective of distinguishing between good and defective printed materials is common, but the types of printed materials being distinguished are different.

[0087] Furthermore, according to the above embodiment, as shown in equations (2) and (3) and Figures 15 to 18, multiple image processing sequences Sq are generated such that the ratio of the average ARV as the connection reference value RV is equal to the ratio of connection probabilities, i.e., the ratio of numerical ranges. This increases the likelihood that an element set with high connection compatibility is included in the image processing sequence Sq, thus enabling the identification of an image processing sequence Sq that satisfies the criterion conditions more efficiently. In addition, according to the above embodiment, as shown in equations (2) and (3) and Figures 15 to 18, by setting a wider numerical range for input sequences with higher average ARV as the connection reference value RV, the probability that an input sequence summary with a high connection reference value RV is set as a sequence element connected to an output sequence element can be increased. This enables the identification of an image processing sequence that satisfies the criterion conditions more efficiently. Furthermore, according to the above embodiment, the reference table generation unit 55 generates a reference table 88 using similar SD values ​​for the connection reference value RV, thereby generating a connection reference value RV that better reflects the connection compatibility, i.e., the compatibility of the element set for obtaining the desired image.

[0088] B. Other embodiments: B-1. Other Embodiments 1: In the above embodiment, the processing results of each of the multiple image processing sequences Sq represented by multiple individuals IV belonging to a single generation G were evaluated one by one in sequence. However, it is also possible to evaluate the processing results of multiple image processing sequences Sq in parallel using multiple generation devices 20.

[0089] B-2. Other Embodiments 2: In the above embodiment, the image processing sequence Sq was represented by individual IV and gene table 82 in the reference table generation process, but is not limited to this. For example, in the reference table generation process, the image processing sequence Sq may be represented by a table that defines multiple sequence elements and the connection relationships of each sequence element. Also, in the reference table generation process, the sequence generation unit 30 generated multiple image processing sequences Sq using a genetic algorithm, but the generation method is not limited to this. Multiple image processing sequences Sq may be generated by randomly arranging multiple sequence elements.

[0090] B-3. ​​Other Embodiments 3: In the above embodiment, the reference table 88 is generated in the reference table generation process shown in Figure 9, and the reference table is not updated during the generation process of a specific image processing sequence, but this is not limited to this. For example, the reference table 88 may be updated during the generation process of a specific image processing sequence based on the evaluation value EV calculated in step S352 of Figure 12.

[0091] B-4. Other Embodiments 4: In the above embodiment, the gene translation unit 34 generated multiple image processing sequences using the reference table 88 and equations (2) and (3) to have a probability relationship with respect to the output sequence element such that the higher the connection reference value RV of the input sequence element, the higher the probability of connection to the output sequence element. However, it is not limited to this. For example, the gene translation unit 34 does not need to use equations (2) and (3) if it can generate multiple image processing sequences that have the above probability relationship. For example, the gene translation unit 34 may generate multiple image processing sequences that have the above probability relationship by gradually increasing the numerical range in which the input sequence element appears as the connection reference value RV increases.

[0092] C. Other forms: This disclosure is not limited to the embodiments described above, and can be implemented in various forms without departing from its spirit. For example, this disclosure can also be implemented in the following forms (aspects). The technical features in the embodiments described above that correspond to the technical features in each of the forms described below can be replaced or combined as appropriate in order to solve some or all of the problems of this disclosure, or to achieve some or all of the effects of this disclosure. Furthermore, if such technical features are not described as essential in this specification, they can be deleted as appropriate.

[0093] (1) According to a first embodiment of the present disclosure, a method for generating an image processing sequence is provided. This image generation method includes the steps of: (a) generating a sequence set comprising a plurality of image processing sequences, each having a different combination of sequence elements: (i) an image input layer for inputting an image, (ii) at least one of a plurality of image processing layers, and (iii) an image output layer for outputting the image after image processing; (b) processing a training image with each of the plurality of image processing sequences to generate an output image, which is the image after image processing for each of the plurality of image processing sequences; (c) for each of the plurality of image processing sequences, comparing the output image with a target image associated with the training image to calculate an evaluation value indicating the degree of similarity between the output image and the target image; and (d) using a plurality of evaluation values ​​corresponding to each of the plurality of image processing sequences to determine the degree of similarity indicated by the plurality of degrees of similarity. The method comprises a step of identifying an image processing sequence that satisfies predetermined criteria, wherein each of the plurality of image processing sequences is represented by an individual having a plurality of genes arranged corresponding to the image input layer and the image processing layer, respectively, and a reference table defining the connection relationships of the sequence elements in the image processing sequence, the reference table being a table that defines connection reference values ​​for each element set, which are a pair of connected input and output sequence elements, indicating the compatibility of connections for processing the image to output a desired image targeting the target image, and step (a) generates the plurality of image processing sequences using the reference table such that, with respect to the output sequence element, the higher the connection reference value of the input sequence element, the higher the probability of connection to the output sequence element. With this configuration, since multiple image processing sequences can be generated using the reference table such that, with respect to the output sequence element, the higher the connection reference value of the input sequence element, the higher the probability of connection to the output sequence element, an image processing sequence that satisfies the criteria can be identified efficiently without increasing the number of image processing sequences to be generated.This reduces the processing time required to identify an image processing sequence that meets the criteria.

[0094] (2) In the above embodiment, step (a) may generate a plurality of image processing sequences such that the ratio of the connection reference values ​​of each input-side sequence element, which is a candidate for connection of the same output-side sequence element, is equal to the ratio of the connection probabilities. According to this embodiment, by generating a plurality of image processing sequences such that the ratio of the connection reference values ​​is equal to the ratio of the connection probabilities, the likelihood of an element set with high connection compatibility being included in the image processing sequence can be increased. This makes it possible to identify an image processing sequence that satisfies the criteria more efficiently.

[0095] (3) In the above configuration, the plurality of genes constituting the individual are represented by numerical values, and step (a) generates the image processing sequence by arranging the image output layer at the output end and connecting the sequence elements from the output side to the input side, and step (a) selects the corresponding image input layer or image processing layer as the sequence element when the numerical value of the gene falls within one of the numerical ranges by setting different numerical ranges for each of the image input layer and the plurality of image processing layers, and step (a) generates the plurality of image processing sequences such that the probability relationship exists by setting a wider numerical range for input sequence elements with higher connection reference values ​​relative to the output sequence element. According to this configuration, by setting a wider numerical range for input sequences with higher connection reference values, the probability of input sequence elements with high connection reference values ​​being set as sequence elements can be increased. This makes it possible to identify image processing sequences that satisfy the criteria more efficiently.

[0096] (4) In the above embodiment, if there are no input-side sequence elements that are not connected to the output-side sequence elements, step (a) may terminate the generation process of the image processing sequence corresponding to the individual even if not all of the multiple genes constituting the individual are used as the image processing sequence. According to this embodiment, an image processing sequence can be generated even if not all of the multiple genes are used. In addition, in the above embodiment, if all of the multiple genes constituting the individual are used as the image processing sequence, and there are input-side sequence elements that are not connected to the output-side sequence elements, step (a) may terminate the generation process of the image processing sequence corresponding to the individual by setting the image input layer to the unconnected input-side sequence elements. In this way, an image processing sequence can be reliably generated using an individual.

[0097] (5) In the above embodiment, the predetermined criterion condition in step (d) is that the degree of similarity with the highest value among the multiple degrees of similarity indicated by each of the multiple evaluation values ​​is equal to or greater than a predetermined termination threshold, and step (a) may include a step of generating the sequence set by replacing one or more of the multiple image processing sequences generated by the previous routine that have a low degree of similarity with a new image processing sequence with a different combination of sequence elements if the predetermined criterion condition is not met. According to this embodiment, by replacing one or more image processing sequences with a low degree of similarity with a new image processing sequence with a different combination of sequence elements, the image processing sequences with a low degree of similarity can be excluded from processing. This makes it possible to identify image processing sequences that satisfy the criterion condition more efficiently.

[0098] (6) In the above embodiment, in step (e), for the same image processing sequence that constitutes the sequence set up to the previous time, the evaluation value calculated by step (c) for the image processing sequence that constitutes the sequence set up to the previous time may be used without performing steps (b) and (c). In this embodiment, there is no need to recalculate the evaluation value, so the processing efficiency of the generation method can be improved.

[0099] (7) In the above embodiment, the reference table may be generated by using the degree of similarity between the output image generated using each of the plurality of image processing sequences, which are generated such that in the pair of sequence elements, the input sequence element is connected to the output sequence element with a random probability, and the target image as the connection reference value. In this embodiment, by using the degree of similarity as the connection reference value, the possibility that a set of elements with high connection compatibility is included in the image processing sequence can be increased.

[0100] (8) According to a second embodiment of the present disclosure, an image processing sequence generation apparatus is provided. The generation apparatus comprises: a sequence generation unit that generates a sequence set consisting of a plurality of image processing sequences, each having a different combination of (i) an image input layer for inputting an image, (ii) at least one of a plurality of image processing layers, and (iii) an image output layer for outputting the image after image processing; an image processing unit that processes a learning image using each of the plurality of image processing sequences and generates an output image, which is the image after image processing, for each of the plurality of image processing sequences; and an output evaluation unit that, for each of the plurality of image processing sequences, compares the output image with a target image associated with the learning image and calculates an evaluation value indicating the degree of similarity between the output image and the target image, wherein the output evaluation unit calculates a plurality of evaluation values ​​corresponding to each of the plurality of image processing sequences. Using the multiple degrees of similarity shown, an image processing sequence that satisfies predetermined criteria is identified. Each of the multiple image processing sequences is represented by an individual having a sequence of genes corresponding to the image input layer and the image processing layer, respectively, and a reference table defining the connection relationships of the sequence elements in the image processing sequence. The reference table is a table that defines connection reference values ​​for each set of elements, which are a pair of connected input and output sequence elements, indicating the compatibility of connections for processing the image to output a desired image targeting the target image. The sequence generation unit uses the reference table to generate the multiple image processing sequences such that, for output sequence elements, the higher the connection reference value of the input sequence element, the higher the probability of connection to the output sequence element. With this configuration, since multiple image processing sequences can be generated using the reference table such that, for output sequence elements, the higher the connection reference value of the input sequence element, the higher the probability of connection to the output sequence element, an image processing sequence that satisfies the criteria can be identified efficiently without increasing the number of image processing sequences to be generated.This reduces the processing time required to identify an image processing sequence that meets the criteria.

[0101] (9) According to a third embodiment of the present disclosure, a computer program is provided for causing a computer to perform the generation of an image processing sequence. The computer program has the function of generating a sequence set consisting of a plurality of image processing sequences, each having a different combination of the following sequence elements: (a) an image input layer for inputting an image, (ii) at least one of a plurality of image processing layers, and (iii) an image output layer for outputting the image after image processing; (b) a function for processing a training image with each of the plurality of image processing sequences and generating an output image which is the image after image processing for each of the plurality of image processing sequences; (c) a function for comparing the output image with a target image associated with the training image for each of the plurality of image processing sequences and calculating an evaluation value indicating the degree of similarity between the output image and the target image; and (d) using a plurality of degrees of similarity indicated by a plurality of evaluation values ​​corresponding to each of the plurality of image processing sequences Function (a) is performed by a computer to identify an image processing sequence that satisfies predetermined criteria, and in function (a), each of the plurality of image processing sequences is represented by an individual having a plurality of genes arranged corresponding to the image input layer and the image processing layer, and a reference table that defines the connection relationships of the sequence elements in the image processing sequence, the reference table being a table that defines connection reference values ​​for each element set which is a pair of connected input and output sequence elements that process the image and output a desired image which is the target image, and function (a) uses the reference table to generate the plurality of image processing sequences such that, with respect to the output sequence element, the input sequence element with a higher connection reference value has a higher probability of being connected to the output sequence element.In this configuration, using a reference table, multiple image processing sequences can be generated such that, for output sequence elements, the higher the connection reference value of the input sequence element, the higher the probability of connection to the output sequence element. This allows for efficient identification of image processing sequences that satisfy the criteria without increasing the number of image processing sequences generated. As a result, the processing time required to identify image processing sequences that satisfy the criteria can be reduced.

[0102] This disclosure can also be implemented in various forms other than those described above. For example, it can be implemented in the form of a non-transitory storage medium on which a computer program is recorded. [Explanation of Symbols]

[0103] 10…Generation system, 20…Generation device, 25…Processor, 30…Sequence generation unit, 32…Individual generation unit, 34…Gene translation unit, 40…Image processing unit, 46…Evaluation and judgment unit, 50…Output evaluation unit, 55…Reference table generation unit, 60…Display control unit, 70…Storage device, 74…Training data, 80…Image filter group, 82…Gene table, 84…Individual population, 88…Reference table, 94…Various programs, 110…Reference table, 150…Input unit, 160…Display unit, 200…Imaging device, ARV…Average value, DA…Defective part, DSq…Specific image processing sequence, EV…Evaluation value, FtA~FtE…Image processing filter, IV,IV1~IV5,IVa…Individual, LM…Training image, PM…Output image, RV…Connection reference value, Rt1~Rt5…Processing routine, SM…Training set, Sq,Sq1…Image processing sequence, TM…Target image

Claims

1. A method for generating an image processing sequence, (a) A step of generating a sequence set comprising a plurality of image processing sequences, each having a combination of (i) an image input layer for inputting an image, (ii) at least one of a plurality of image processing layers, and (iii) an image output layer for outputting the image after image processing, wherein the sequence elements are: (b) A step of processing a training image using each of the plurality of image processing sequences, and generating an output image which is the image after the image processing for each of the plurality of image processing sequences, (c) For each of the plurality of image processing sequences, a step of comparing the output image with a target image associated with the training image and calculating an evaluation value indicating the degree of similarity between the output image and the target image, (d) A step of identifying an image processing sequence that satisfies predetermined criteria using a plurality of evaluation values ​​corresponding to each of the plurality of image processing sequences, In step (a) above, each of the plurality of image processing sequences is represented by an individual having a plurality of genes corresponding to the image input layer and the image processing layer, respectively, and a reference table defining the connection relationships of the sequence elements in the image processing sequence, The aforementioned reference table is a table that defines connection reference values ​​for each set of elements which are a pair of sequence elements on the connected input and output sides, indicating the compatibility of the connection for processing the image and outputting the target image as the target image. Step (a) involves generating a plurality of image processing sequences using the reference table such that, for each output sequence element, the higher the connection reference value of the input sequence element, the higher the probability of connection to the output sequence element. The multiple genes constituting the individual are represented by numerical values, Step (a) involves arranging the image output layer at the output end and connecting the sequence elements from the output side to the input side to generate the image processing sequence. Step (a) involves setting different numerical ranges for each of the image input layer and the multiple image processing layers, so that when the numerical value of the gene falls within one of the numerical ranges, the corresponding image input layer or image processing layer is selected as the sequence element. Step (a) is a generation method that generates a plurality of image processing sequences having the probability relationship by setting a wider numerical range for input sequence elements that have a higher connection reference value than the output sequence element.

2. A method for generating according to claim 1, Step (a) is a generation method that generates a plurality of image processing sequences such that the ratio of the connection reference values ​​of each input sequence element, which is a candidate for connection of the same output sequence element, is equal to the ratio of the connection probabilities.

3. A method for generating according to claim 1, The generation method wherein step (a) terminates the generation process of the image processing sequence corresponding to the individual if there are no input-side sequence elements that are not connected to the output-side sequence elements, even if not all of the multiple genes constituting the individual are used as the image processing sequence.

4. In the generation method according to any one of claims 1 to 3, The predetermined standard condition in step (d) is that the degree of similarity with the highest value among the multiple degrees of similarity indicated by each of the multiple evaluation values ​​is equal to or greater than a predetermined termination threshold. The generation method includes, if the predetermined criteria conditions are not met, a step of generating a sequence set by replacing one or more of the image processing sequences that are less similar among the plurality of image processing sequences generated by the previous routine with a new image processing sequence with a different combination of sequence elements.

5. A method for generating according to claim 4, A generation method in which, in step (e), for the same image processing sequence that constitutes the sequence set up to the previous time, the evaluation value calculated by step (c) for the image processing sequence that constitutes the sequence set up to the previous time is used without performing steps (b) and (c).

6. A method for generating according to any one of claims 1 to 5, The method for generating the reference table involves using the degree of similarity between the output image, generated using each of the plurality of image processing sequences, which are generated such that in the pair of sequence elements, the input sequence element is connected to the output sequence element with a random probability, and the target image, as the connection reference value.

7. An image processing sequence generation device, A sequence generation unit generates a sequence set comprising multiple image processing sequences, each representing a combination of (i) an image input layer for inputting an image, (ii) at least one of multiple image processing layers, and (iii) an image output layer for outputting the image after image processing, wherein the sequence elements are: An image processing unit that processes a training image using each of the multiple image processing sequences and generates an output image which is the image after the image processing for each of the multiple image processing sequences, The system includes an output evaluation unit that, for each of the plurality of image processing sequences, compares the output image with a target image associated with the training image and calculates an evaluation value indicating the degree of similarity between the output image and the target image. The output evaluation unit identifies the image processing sequence that satisfies predetermined criteria using the multiple degrees of similarity indicated by the multiple evaluation values ​​corresponding to each of the multiple image processing sequences, Each of the aforementioned plurality of image processing sequences is represented by an individual having a plurality of genes corresponding to the image input layer and the image processing layer, respectively, and a reference table defining the connection relationships of the sequence elements in the image processing sequence. The aforementioned reference table is a table that defines connection reference values ​​for each set of elements which are a pair of sequence elements on the connected input and output sides, indicating the compatibility of the connection for processing the image and outputting the target image as the target image. The sequence generation unit generates the plurality of image processing sequences using the reference table, such that the input sequence element with a higher connection reference value has a higher connection probability to the output sequence element. The multiple genes constituting the individual are represented by numerical values, The sequence generation unit generates the image processing sequence by arranging the image output layer at the output end and connecting the sequence elements from the output side to the input side. The sequence generation unit sets different numerical ranges for each of the image input layer and the multiple image processing layers, and when the numerical value of the gene falls within one of the numerical ranges, it selects the corresponding image input layer or image processing layer as the sequence element. The sequence generation unit generates a plurality of image processing sequences having the probability relationship by setting a wider numerical range for input sequence elements that have a higher connection reference value than the output sequence elements.

8. A computer program that causes a computer to perform the generation of an image processing sequence, (a) A function to generate a sequence set comprising multiple image processing sequences, each having a combination of (i) an image input layer for inputting an image, (ii) at least one of multiple image processing layers, and (iii) an image output layer for outputting the image after image processing, wherein the sequence elements are: (b) A function that processes a training image using each of the multiple image processing sequences and generates an output image which is the image after the image processing for each of the multiple image processing sequences, (c) A function to compare the output image with a target image associated with the training image for each of the plurality of image processing sequences and to calculate an evaluation value indicating the degree of similarity between the output image and the target image, (d) A function to identify an image processing sequence that satisfies predetermined criteria using a plurality of evaluation values ​​corresponding to each of the plurality of image processing sequences, and to cause the computer to perform this function, In the function (a) described above, each of the multiple image processing sequences is represented by an individual having a sequence of genes corresponding to the image input layer and the image processing layer, and a reference table defining the connection relationships of the sequence elements in the image processing sequence. The aforementioned reference table is a table that defines connection reference values ​​for each set of elements which are a pair of sequence elements on the connected input and output sides, indicating the compatibility of the connection for processing the image and outputting the target image as the target image. Function (a) generates the multiple image processing sequences using the reference table such that, for output sequence elements, the higher the connection reference value of the input sequence element, the higher the probability of connection to the output sequence element. The multiple genes constituting the individual are represented by numerical values, Function (a) generates the image processing sequence by arranging the image output layer at the output end and connecting the sequence elements from the output side to the input side. Function (a) selects the corresponding image input layer or image processing layer as the sequence element when the numerical value of the gene falls within one of the numerical ranges, by setting different numerical ranges for each of the image input layer and the multiple image processing layers. Function (a) is a computer program that generates a plurality of image processing sequences having the probability relationship by setting a wider numerical range for input sequence elements that have a higher connection reference value than the output sequence element.

Images