Image processing system

Abstract

The purpose of the present invention is to improve the reliability of an image processing system by taking into account the distribution of positive solution data corresponding to the output of a learning model and determining the output of the learning model while containing a deviation. For example, the present invention is provided with: a model application unit for generating an estimated high-quality image by applying a machine learning model to a low-quality image; an evaluation value calculation unit that calculates an evaluation value for the estimated high-quality image; and a determination unit that determines the estimated high-quality image on the basis of the evaluation value of the estimated high-quality image, the determination unit determining the estimated high-quality image using a deviation range of the evaluation value of a high-quality image obtained by capturing the low-quality image with high quality, the deviation range being acquired in advance and corresponding to the evaluation value of the estimated high-quality image. It is determined whether the speculated high quality image is uncertain data.

Description

Technical Field

[0001] This invention relates to image processing systems. Background Technology

[0002] As an image processing system used in inspection and measurement, there is the technology described in Patent Document 1. In this document, paragraph 0012 describes that "the scoring output unit 30 reads the inspection image extracted by the inspection image extraction unit 10 and the inspection information from the storage unit 70, inputs the read inspection image into the learning model constructed by the learning unit 20, and calculates a judgment score for each inspection image… The scoring output unit 30 assigns the calculated defect score to the inspection information and saves it in the storage unit 70." Paragraph 0013 describes that "the judgment unit 40 reads the inspection information assigned a defect score by the scoring output unit 30 from the storage unit 70, compares the threshold used to determine whether the inspection image is good or bad (i.e., the judgment threshold) with the defect score, and determines whether the inspection image is good or bad, that is, determines whether the inspection image is 'good,' 'bad,' or 'unclear whether it is good or bad.'"

[0003] Furthermore, paragraph 0031 of this document describes that "the threshold update unit 60 determines whether the threshold update condition, that is, the predetermined condition for updating the judgment threshold, is met. If the threshold update condition is met, the process is transferred to step S60 to update the judgment threshold. If the threshold update condition is not met, the good or bad judgment process ends. In this embodiment, the threshold update condition is determined by the judgment unit 40 as 'good or bad is unclear' and the number of inspection images in the inspection images that have been given predetermined determination information by the visual inspection unit 50, and whose defect scores are within a predetermined range, reaches a predetermined number."

[0004] Existing technical documents

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 2022-169974 Summary of the Invention

[0007] The problem that the invention aims to solve

[0008] The technique described in this document uses a learning model to calculate the decision score, which may be subject to errors and other biases. However, the technique described in this document does not consider the bias of the decision score, which may lead to misjudgments.

[0009] Therefore, the objective of this invention is to improve the reliability of the image processing system by considering the distribution of correct data corresponding to the output of the learning model, thereby incorporating biases into the determination of the learning model's output.

[0010] Methods for solving problems

[0011] To address the aforementioned issues, the image processing system of the present invention includes, for example, a model application unit that applies a machine learning model to a low-quality image to generate a hypothesized high-quality image; an evaluation value calculation unit that calculates an evaluation value for the hypothesized high-quality image; and a determination unit that determines the hypothesized high-quality image based on its evaluation value, wherein the determination unit uses a pre-obtained deviation range of the evaluation value of a high-quality image obtained by capturing the low-quality image at high quality, corresponding to the evaluation value of the hypothesized high-quality image, to determine whether the hypothesized high-quality image is uncertain data.

[0012] Invention Effects

[0013] According to the present invention, by considering the distribution of positive solution data corresponding to the output of the learning model, the bias can also be included in the determination of the output of the learning model, thereby improving the reliability of the image processing system. Attached Figure Description

[0014] Figure 1 This is a block diagram of the image processing system in Embodiment 1.

[0015] Figure 2 This is a block diagram illustrating the structure used to obtain the evaluation value.

[0016] Figure 3A This is a graph representing an example of the deviation in evaluation values.

[0017] Figure 3B This is a diagram illustrating an example of how the range of deviations is determined.

[0018] Figure 3C This is another example of a method for determining the range of deviation.

[0019] Figure 4A This is a diagram illustrating an example of determining a high-quality inferred image as definitive data.

[0020] Figure 4B This is another example of determining a speculative high-quality image as definitive data.

[0021] Figure 4C This is a diagram illustrating an example of classifying a speculative high-quality image as uncertain data.

[0022] Figure 4D This is another example of judging speculative high-quality images as uncertain data.

[0023] Figure 5 This is a flowchart for handling deviations in obtaining evaluation values.

[0024] Figure 6This is a graph showing an example of the deviation in the evaluation values.

[0025] Figure 7 This is a flowchart of image evaluation processing.

[0026] Figure 8 This is an example of a display showing the acquisition of a high-quality image.

[0027] Figure 9 This is a block diagram of the image processing system in Embodiment 2.

[0028] Figure 10 This is a block diagram illustrating the structure used to calculate the allowed range of inputs.

[0029] Figure 11 This is a diagram illustrating how the allowed input range is calculated.

[0030] Figure 12A This is a graph representing an example where the evaluation value of a low-quality image is outside the allowed range of the input.

[0031] Figure 12B This is another example of a graph showing that the evaluation value of a low-quality image is outside the allowed range of the input.

[0032] Figure 12C This is a graph representing an example where the evaluation value of a low-quality image is within the allowed range of the input.

[0033] Figure 13 This is a flowchart from input judgment processing to image evaluation processing. Detailed Implementation

[0034] The embodiments are described below using the accompanying drawings.

[0035] Example 1

[0036] Figure 1 This is a block diagram of the image processing system according to Embodiment 1 of the present invention. In this embodiment, the inspection and measurement of semiconductors using images captured by an optical microscope, scanning electron microscope, etc., will be described.

[0037] The image processing system 1 of this embodiment includes a model application unit 11, an evaluation value calculation unit 12, a determination unit 13, and an image acquisition unit 14.

[0038] <Model Application Department 11>

[0039] The model application unit 11 is input with low-quality images captured by optical microscopes, scanning electron microscopes, etc. Then, a machine learning model is applied to the low-quality images to generate inferred high-quality images.

[0040] Low-quality images refer to images with a low cumulative number of pixels and a lot of noise, images with unsuitable shooting parameters and a blurry image, images with low contrast of objects within the image, and images with low resolution.

[0041] On the other hand, high-quality images refer to images of higher quality than low-quality images. Examples include images with a high cumulative number of images and low noise, images with appropriate shooting parameters and clear objects within the image, images with suitable contrast of objects within the image, and images with high resolution. Such high-quality images have high data reliability. However, increasing the cumulative number of images captured, or increasing the shooting interval, increases the shooting time, thus reducing the throughput from image acquisition to image processing. Therefore, this invention uses inferred high-quality images generated by applying a machine learning model to low-quality images.

[0042] It is hypothesized that a high-quality image is an image with the same quality as a high-quality image. Therefore, while ensuring data reliability, the reduction in throughput from image acquisition to image processing is suppressed. Machine learning models include, for example, models that remove noise and blur to improve image quality, and models that correct contrast to emphasize objects within an image.

[0043] <Evaluation Value Calculation Section 12>

[0044] The evaluation value calculation unit 12 calculates the inferred evaluation value of the high-quality image. Here, the evaluation value is a measure of the image features obtained by applying machine learning models such as deep learning to the image, the image features obtained by using t-SNE as a dimensionality compression method, the probability of defects in objects within the image, and the width of objects within the image.

[0045] <Judgment Section 13>

[0046] The determination unit 13 determines the predicted high-quality image based on its evaluation value. Specifically, the determination unit 13 determines the deviation range of the evaluation value of the predicted high-quality image based on the deviation of each evaluation value. If the entire deviation range is within a predetermined allowable range, or if the entire deviation range is not within the allowable range, the predicted high-quality image is determined as certain data. If a part of the deviation range is not within the allowable range, the predicted high-quality image is determined as uncertain data. Furthermore, the allowable range is set based on the results of semiconductor inspection and measurement, etc. Additionally, the deviation of the evaluation value is obtained before the determination unit 13 determines the evaluation value of the predicted high-quality image.

[0047] <<Bias in Evaluation Values>>

[0048] Here, use Figure 2 as well as Figures 3A-3C Explain the deviation of the evaluation values. Figure 2 This is a block diagram illustrating the structure used to obtain the evaluation value. For example... Figure 2 As shown, image processing system 1, in addition to Figure 1 In addition to the model application unit 11 and the evaluation value calculation unit 12 shown, a deviation acquisition unit 15 is also provided. Furthermore, in this embodiment, the image processing system 1 is assumed to include... Figure 1 as well as Figure 2 The structure is used for explanation, but it has Figure 1 The structure of the device and having Figure 2 The structure of the device can also be separated.

[0049] The model application unit 11 applies a machine learning model to multiple low-quality images to generate multiple inferred high-quality images. The low-quality images include learning data used to train the machine learning model, validation data used to determine the hyperparameters of the machine learning model, and test data used to verify the accuracy of the machine learning model. The inferred high-quality images are images inferred from the low-quality images by applying the machine learning model; they are images comparable to the high-quality images.

[0050] The evaluation value calculation unit 12 calculates the evaluation value of the inferred high-quality image and the evaluation value of the high-quality image, which is obtained by taking a low-quality image with higher quality as the inferred source of the inferred high-quality image and including the object.

[0051] Figure 3A This is a diagram illustrating an example of how the range of deviations is determined. Figure 3A This is a graph that obtains the evaluation values ​​of multiple high-quality images and their corresponding evaluation values. The horizontal axis represents the inferred evaluation value of the high-quality image, and the vertical axis represents the evaluation value of the high-quality image. Figure 3A In this study, the relationship between the predicted evaluation value of a high-quality image and the evaluation value of a high-quality image is recorded as multiple plotted points P1.

[0052] The deviation acquisition unit 15 calculates the distribution range of a predetermined number of plotting points among multiple plotting points P1. Figure 3A In this example, the range from the lower limit LL1 to the upper limit UL1 is illustrated as the distribution range of 90% of the plotted points P1 (90% range). The determination unit 13 uses the evaluation value of the high-quality image corresponding to the predicted high-quality image evaluation value as the deviation from the evaluation value. Specifically, the determination unit 13... Figure 3A In the chart shown, the horizontal axis is considered as the inferred high-quality image rating. For example, when the inferred high-quality image rating is x1, the inferred high-quality image rating x1 is used as... Figure 3AThe value of the horizontal axis. Then, the determination unit 13 extracts the lower limit y1 and the upper limit y2 of the evaluation value of the high-quality image within 90% of the predicted high-quality image evaluation value x1, and sets the lower limit of the deviation range of the predicted high-quality image evaluation value x1 as y1 and the upper limit of the deviation range as y2. That is, Figure 3A The range DR1 shown represents the deviation range of the inferred evaluation value x1 for a high-quality image.

[0053] Figure 3B This is a diagram illustrating an example of how the range of deviations is determined. Figure 3B plotting point P1 and Figure 3A Same. Figure 3B In the diagram, the range from the lower limit LL2 to the upper limit UL2 is shown as the distribution range (99% range) of plotted points, encompassing 99% of plotted point P1. (Comparison) Figure 3A and Figure 3B The 99% range is wider along the vertical axis than the 90% range. Figure 3B In the case shown, the lower limit of the deviation range for the evaluation value x1 of the high-quality image is y3, and the upper limit of the deviation range is y4. That is, Figure 3B The range DR2 shown represents the deviation range of the inferred evaluation value x1 for a high-quality image.

[0054] Figure 3C This is another example of a method for determining the range of deviation. Figure 3C plotting point P1 and Figure 3A Same. Figure 3C In the first chart shown, two plotted points near the evaluation value of the inferred high-quality image calculated by the evaluation value calculation unit 12 are extracted. The lower of the evaluation values ​​of the high-quality images of the two extracted plotted points is set as the lower limit of the deviation range, and the higher of the evaluation values ​​is set as the upper limit of the deviation range. For example, for the evaluation value x2 of the inferred high-quality image, the evaluation value y5 of the high-quality images of the two nearby plotted points is set as the lower limit, and the evaluation value y6 is set as the upper limit to determine the deviation range DR3 of the evaluation value. Moreover, for the evaluation value x3 of the inferred high-quality image, the evaluation value y7 of the high-quality images of the two nearby (e.g., two adjacent) plotted points is set as the lower limit, and the evaluation value y8 is set as the upper limit to determine the deviation range DR4 of the evaluation value. By determining the deviation range of the evaluation value based on the evaluation values ​​of the high-quality images of the two nearby plotted points in this way, it is not necessary to... Figure 3A , Figure 3BSetting upper and lower limits for the entire interval allows for a suitable range of deviations in the evaluation values ​​used to predict high-quality images. Alternatively, minimum and maximum values ​​can be extracted from the evaluation values ​​of high-quality images at multiple plotted points within a predetermined interval. The maximum value can be used as the upper limit, and the minimum value as the lower limit to determine the range of deviations in the evaluation values.

[0055] use Figures 3A-3C The deviation of the evaluation value shown is used to determine the range of deviation of the evaluation value based on the evaluation value of the high-quality image calculated by the evaluation value calculation unit 12.

[0056] <<Judgment of Judgment Section 13>>

[0057] use Figures 4A to 4D Specifically, the determination of the determination unit 13 is explained. Furthermore, in Figures 4A to 4D In the diagram, the horizontal axis represents the evaluation value. Values ​​less than m1 are outside the allowable range, values ​​greater than m1 and less than m2 are within the allowable range, and values ​​greater than m2 are outside the allowable range. The boundary values ​​between the allowable range and the outside of the allowable range, such as m1 and m2, can be set based on prior knowledge of the electrical characteristics of previously manufactured semiconductors, which may result in qualified or unqualified products. Alternatively, they can be set based on the average and standard deviation of the number of inspection and measurement samples.

[0058] Figure 4A This is a diagram illustrating an example where a hypothetical high-quality image is determined to be definitive data. In Figure 4A In this context, the evaluation value y9 of the high-quality image corresponding to the predicted high-quality image evaluation value x4 is the lower limit, the evaluation value y10 of the high-quality image is the upper limit, and the entire deviation range DR5 of the predicted high-quality image evaluation value x4 is contained within the allowable range. Therefore, in this case, the determination unit 13 determines the predicted high-quality image as the definitive data.

[0059] Figure 4B This is another example of how a hypothetical high-quality image is determined to be definitive data. In Figure 4B In this context, the evaluation value y11 of the high-quality image corresponding to the predicted high-quality image evaluation value x5 is the lower limit, and the evaluation value y12 of the high-quality image is the upper limit. The entire deviation range DR6 of the predicted high-quality image evaluation value x5 is not included in the allowable range. Therefore, in this case, the determination unit 13 determines the predicted high-quality image as the definitive data.

[0060] Figure 4C This is a diagram illustrating an example where a speculatively high-quality image is judged as uncertain data. In Figure 4CIn this case, the evaluation value y13 of the high-quality image corresponding to the predicted high-quality image evaluation value x6 is the lower limit, and the evaluation value y14 of the high-quality image is the upper limit. A portion of the deviation range DR7 of the predicted high-quality image evaluation value x6 is not included in the allowable range. In this situation, the predicted high-quality image evaluation value x6 is within the allowable range, but it may deviate within the deviation range DR7. Therefore, it is impossible to determine whether the predicted high-quality image evaluation value is within or outside the allowable range. Therefore, the determination unit 13 determines the predicted high-quality image as uncertain data.

[0061] Figure 4D This is another example of a speculatively high-quality image being classified as uncertain data. In Figure 4D In this case, the evaluation value x7 of the inferred high-quality image is defined as the lower limit of the evaluation value y15 of the high-quality image, and the evaluation value y16 of the high-quality image is defined as the upper limit. A portion of the deviation range DR8 of the inferred high-quality image evaluation value x7 is not included in the allowable range. In this situation, the evaluation value x7 of the inferred high-quality image is outside the allowable range, but it may deviate within the deviation range DR8. Therefore, it is impossible to determine whether the evaluation value of the inferred high-quality image is within or outside the allowable range. Therefore, the determination unit 13 determines the inferred high-quality image as uncertain data.

[0062] In this way, the judgment is made not only based on the inferred evaluation value of the high-quality image, but also considering the range of deviation, thereby suppressing false judgments.

[0063] Then, if the determination unit 13 determines that the predicted high-quality image is uncertain data, the image acquisition unit 14 (see reference) acquires the image. Figure 1 A high-quality image obtained by capturing objects in a low-quality image at a higher quality is used instead of a speculative high-quality image that is considered uncertain data.

[0064] <Image Acquisition Section 14>

[0065] The image acquisition unit 14 captures high-quality images of objects contained in low-quality images from the inference source of high-quality learning, which is determined to be uncertain data, thus obtaining high-quality images. High-quality capture is achieved by changing the image capture conditions or image processing methods. Compared to generating inferred high-quality images, the accuracy and quality of high-quality images are improved, but this is equivalent to methods that require more processing time, have lower throughput, or cannot be executed within a limited time. For example, in the image acquisition unit 14, the cumulative count is increased to capture high-quality images with high cumulative counts and less noise. In addition, there are cases where different methods are applied in the model application unit 11 and the image acquisition unit 14. For example, when a low-quality image with low cumulative counts and a lot of noise is input, the model application unit 11 uses a machine learning model to infer an image with a high cumulative count. In contrast, the image acquisition unit 14 uses a machine learning model that emphasizes objects within the image and corrects for contrast that is easy to detect defects, or a classical image processing method.

[0066] <Handling of Deviations in Evaluation Values>

[0067] Figure 5 This is a flowchart for handling deviations in obtaining evaluation values. Additionally, Figure 5 Steps S11~S13 of the flowchart are passed through Figure 2 The structure shown is used to achieve this.

[0068] In step S11, the model application unit 11 applies a machine learning model to the low-quality image to generate a predicted high-quality image. In step S12, the evaluation value calculation unit 12 calculates the evaluation value of the predicted high-quality image and the evaluation value of the high-quality image, which is obtained by capturing objects contained in the low-quality image, which serves as the prediction source for the predicted high-quality image, at a higher quality. In step S13, as described above, the deviation acquisition unit 15 uses the evaluation value of the predicted high-quality image and the evaluation value of the high-quality image to obtain the deviation between the evaluation value of the predicted high-quality image and the evaluation value of the high-quality image.

[0069] In addition, image processing system 1 Figure 2 In addition to the structure shown, it also has a display section (not shown), as... Figure 5 Step S14 can also be prompted to the user by displaying the deviation of the evaluation value obtained in step S13 on the display unit. Figure 6 This is a graph showing an example of the deviation from the evaluation values. For example... Figure 6 As shown, the deviation of the evaluation value and the method for determining the currently used deviation range can also be displayed on the display unit. At this time, based on the display of the method for determining the deviation range, such as... Figure 3A , Figure 3B In this way, the upper and lower limits are displayed on the first graph representing the deviation of the evaluation values. For example, as... Figure 6 As shown, if the deviation range is determined by a method "based on the deviation range of 90%", then the upper limit UL1 and lower limit LL1 are displayed on the chart. On the other hand, for example, if the deviation range is determined by a method "based on the deviation range closest to the plotted point", then the upper and lower limits are not displayed on the chart.

[0070] Additionally, the image processing system 1 may also have an instruction input unit (not shown) that accepts instructions from the user. Furthermore, as... Figure 5 Step S15 may also involve receiving instructions from the user who has confirmed the deviation of the evaluation value indicated in step S14, regarding a change in the method for determining the deviation range. If no instructions are received from the user, the process ends. Figure 5 The flowchart is shown. On the other hand, if there is an instruction from the user, the process returns to step S14, changing the display of the method for determining the deviation range and the display of the upper and lower limits of the graph representing the deviation range. Furthermore, the display unit and the instruction input unit can also be implemented using a device such as a touch screen.

[0071] <Image Evaluation Processing>

[0072] Figure 7 This is a flowchart of image evaluation processing. Figure 7 The flowchart is through Figure 1 The structure shown is used to implement this. In step S21, a low-quality image is input to the model application unit 11. In step S22, the model application unit 11 applies a machine learning model to generate a predicted high-quality image. In step S23, the evaluation value calculation unit 12 calculates the evaluation value of the predicted high-quality image generated in step S22. In step S24, the determination unit 13 determines the evaluation value based on the... Figure 5 The deviation of the evaluation value calculated in step S13 determines the deviation range of the high-quality image corresponding to the predicted high-quality image evaluation value. In step S25, the determination unit 13 determines whether the entire deviation range determined in step S24 is included within the allowable range. If the entire deviation range is included within the allowable range, or if the entire deviation range is not included within the allowable range, the flowchart ends. On the other hand, if a part of the deviation range is not included within the allowable range, the process proceeds to step S26. In step S26, the user is prompted to obtain a high-quality image instead of the predicted high-quality image. For example, the image processing system 1 uses... Figure 1 The display section, not shown in the image, prompts the user to obtain a high-quality image. Figure 8 This is an example of a display showing the acquisition of a high-quality image. For example... Figure 8 As shown, the display unit can also display the estimated high-quality image's evaluation value, deviation range, and allowable range while simultaneously displaying the acquired high-quality image. This is achieved through... Figure 8As shown in the display, the user can understand the reasons for acquiring high-quality images and determining them as uncertain data. Then, in step S27, the image acquisition unit 14 changes the image capturing conditions or image processing method to capture a high-quality image.

[0073] According to the present invention, an image processing system is provided that can ensure the accuracy of image evaluation regardless of deviations in the evaluation values ​​of the image.

[0074] Example 2

[0075] Figure 9 This is a block diagram of the image processing system 1 of Embodiment 2. The image processing system 1 of this embodiment verifies low-quality images input to the evaluation value calculation unit 12 in Embodiment 1 and excludes low-quality images that are unsuitable for input to the evaluation value calculation unit 12. Hereinafter, the description will focus on the differences from Embodiment 1.

[0076] <Input Validation Section 16>

[0077] The image processing system 1 of this embodiment includes an input verification unit 16. The input verification unit 16 calculates the evaluation value of a low-quality image input to the model application unit 11, and determines whether to input the low-quality image to the model application unit 11 based on the evaluation value. Specifically, the input verification unit 16 inputs the low-quality image to the model application unit 11 if the evaluation value of the low-quality image is included in a pre-obtained input allowable range, and does not input the low-quality image to the model application unit 11 if the evaluation value of the low-quality image is not included in the pre-determined input allowable range.

[0078] <<Calculation of allowed input range>>

[0079] use Figure 10 as well as Figure 11 The calculation of the allowed input range is explained. Figure 10 This is a block diagram illustrating the structure used to calculate the allowed range of input. For example... Figure 10 As shown, image processing system 1, in addition to Figure 9 In addition to the model application unit 11 and the evaluation value calculation unit 12 shown, an input allowable range calculation unit 17 is also provided. Furthermore, in this embodiment, the image processing system 1 is assumed to have... Figure 9 as well as Figure 10 It can be explained using the structure, but it can also have... Figure 9 The structure of the device and having Figure 10 The structural devices are separate, and they cooperate to achieve what will be described later. Figure 13 The flowchart.

[0080] The Model Application Department 11 applies machine learning models to low-quality images to generate inferred high-quality images.

[0081] The evaluation value calculation unit 12 calculates the evaluation value of low-quality images and infers the evaluation value of high-quality images.

[0082] Figure 11 This is a diagram illustrating the calculation method for the input allowable range. The input allowable range calculation unit 17 obtains evaluation values ​​for multiple low-quality images and corresponding predicted evaluation values ​​for high-quality images, presented as a graph where the horizontal axis represents the evaluation values ​​of low-quality images and the vertical axis represents the predicted evaluation values ​​of high-quality images. Figure 11 In the graph, the relationship between the evaluation values ​​of multiple low-quality images and the predicted evaluation values ​​of high-quality images is recorded as plotting point P2. Then, the graph is divided into multiple intervals using the values ​​of the horizontal axis. In this embodiment, the values ​​of the horizontal axis are divided into four intervals: less than x8, x8 and above but less than x9, x9 and above but less than x10, and x10 and above.

[0083] The input allowable range calculation unit 17 extracts the maximum value from the minimum value of the evaluation value of the predicted high-quality image of the plotted point located in the same interval from each of the multiple segmented intervals, and uses the range of the low-quality image of the extracted interval as the input allowable range.

[0084] exist Figure 11 In the dataset, no plotted point P2 exists in the intervals less than x8 and greater than x10. That is, low-quality images with evaluation values ​​less than x8 or greater than x10 are not included in the learning, validation, or test data, indicating that inferences about high-quality images based on low-quality images are unreliable. Therefore, the intervals of low-quality images with evaluation values ​​less than x8 and greater than x10 are outside the allowed input range.

[0085] On the other hand, Figure 11 In this embodiment, plotted points P2 exist in the intervals of x8 and above but less than x9 and x9 and above but less than x10, respectively. In this embodiment, the cases where there are 12 plotted points P2 in each of the intervals of x8 and above but less than x9 and x9 and above but less than x10 are described, but this is not a limitation. Furthermore, the maximum value of the inferred high-quality image evaluation value for plotted points P2 in the interval of x8 and above but less than x9 is set to MAX1, and the minimum value is set to MIN1. Similarly, the maximum value of the inferred high-quality image evaluation value for plotted points P2 in the interval of x9 and above but less than x10 is set to MAX2, and the minimum value is set to MIN2.

[0086] exist Figure 11In this context, the range of x8 and above but less than x9 is wider than the range from the minimum value MIN1 to the maximum value MAX1. This means the deviation of the evaluation value calculated based on the inferred high-quality image is greater than the evaluation value calculated based on the low-quality image. In the deviation acquisition unit 15, the deviation of the evaluation value of the high-quality image is obtained based on the evaluation value calculated in the inferred high-quality image. Therefore, if the evaluation value calculated based on the input image falls within the range of x8 and above but less than x9, the deviation of the evaluation value calculated based on the inferred high-quality image may become larger, and the reliability of the evaluation value deviation decreases. Therefore, if the evaluation value calculated based on the input image is x8 and above but less than x9, it falls outside the allowable range. Similarly, if the evaluation value of the low-quality image falls within the range of x8 and above but less than x9, the deviation of the evaluation value of the inferred high-quality image may be larger, and the reliability of the evaluation value of the inferred high-quality image decreases. Therefore, the range of x8 and above but less than x9 is set to be outside the input allowable range. However, the allowable range of the deviation of the inferred high-quality image's evaluation value can also be provided by the user in advance.

[0087] In contrast, the range of x9 and below x10 is smaller than the range from the minimum value MIN2 to the maximum value MAX2. When the evaluation value of a low-quality image falls within this range, the deviation in predicting the evaluation value of a high-quality image is likely to be smaller, thus increasing the reliability of the predicted evaluation value. Therefore, the range of x9 and below x10 is set as the allowed input range. Alternatively, the user can be prompted with the allowed input range, and the calculation method for the allowed input range can be changed if necessary.

[0088] <<Input Decision>>

[0089] use Figures 12A-12C This section provides a detailed explanation of the input verification unit 16's decision-making process. Additionally, in... Figures 12A-12C In the image, the horizontal axis represents the evaluation value for low-quality images. Additionally, in... Figures 12A-12C In China, use Figure 11 The allowed input range is defined as follows: x9 is outside the allowed input range, x9 and above but less than x10 is within the allowed input range, and x10 and above is outside the allowed input range.

[0090] Figure 12A This is a graph representing an example where the evaluation value of a low-quality image falls outside the allowed range of the input. Figure 12A In the process, the evaluation value (plotted point P3) of the low-quality image is less than x8, which is outside the allowed input range. Therefore, since the evaluation value (plotted point P3) of the low-quality image is not included in the allowed input range, the input verification unit 16 does not input the low-quality image to the model application unit 11, and it becomes unsuitable data for input. The image acquisition unit 14 then captures a high-quality image to obtain a high-quality image.

[0091] Figure 12B This is another example of a low-quality image having an evaluation value that falls outside the allowed input range. Figure 12B In the process, the evaluation value (plotted point P4) of the low-quality image is x8 or higher but less than x9, which is outside the allowed input range. Therefore, since the evaluation value (plotted point P4) of the low-quality image is not included in the allowed input range, the input verification unit 16 does not input the low-quality image to the model application unit 11, and it becomes unsuitable data for input. The image acquisition unit 14 then captures a high-quality image to obtain a high-quality image.

[0092] Figure 12C This is an example of a low-quality image whose evaluation value falls within the allowed range of the input. Figure 12C In the input verification unit 16, the evaluation value (plotted point P5) of the low-quality image is x9 or higher and less than x10, which is within the input allowable range. Therefore, since the evaluation value (plotted point P4) of the low-quality image is within the input allowable range, the low-quality image is input to the model application unit 11 by the input verification unit 16.

[0093] <Display Section>

[0094] The image processing system 1 of this embodiment may also include a display unit (not shown). Furthermore, the display unit of this embodiment displays... Figure 11 The chart shown.

[0095] Additionally, the display unit shows the evaluation value and input allowable range of low-quality images that were not input to the model application unit 11. For example, Figure 12A or Figure 12B That's how it's displayed. Additionally, at this time, the reason why a low-quality image is deemed unsuitable for input can also be displayed. For example, in... Figure 12A In the case shown, the display unit shows reasons such as the low-quality image not being included in the learning data, validation data, and test data, and therefore the inference of a high-quality image's evaluation value based on the low-quality image cannot be trusted. Additionally, in Figure 12B In the case shown, the display unit may show reasons such as the deviation of the evaluation value of the inferred high-quality image based on the low-quality image may increase, and the evaluation value of the inferred high-quality image based on the low-quality image is unreliable.

[0096] <Input Decision Processing>

[0097] Figure 13This is a flowchart from input determination processing to image evaluation processing. In step S31, the input verification unit 16 obtains the input allowed range. In step S32, a low-quality image is input to the input verification unit 16. In step S33, the input verification unit 16 calculates the evaluation value of the low-quality image. In step S34, the input verification unit 16 determines whether the evaluation value of the low-quality image is within the input allowed range. In step S34, if it is determined that the evaluation value of the low-quality image is within the input allowed range, the process proceeds to step S21. Steps S21 to S27 are the same as in Embodiment 1. On the other hand, in step S34, if it is determined that the evaluation value of the low-quality image is not within the input allowed range, the process proceeds to step S35. In step S35, the display unit displays the evaluation value of the low-quality image and the input allowed range, and displays the reason why the low-quality image is determined to be unsuitable for input and the acquisition of a high-quality image, thereby prompting the user with this information.

[0098] When the deviation of the predicted evaluation value of a high-quality image is large, the accuracy of estimating the deviation between the predicted evaluation value of the high-quality image and the actual evaluation value of the high-quality image decreases. However, according to this embodiment, when the deviation of the predicted evaluation value of the high-quality image is large, it is not input to the model application unit 11, thus preventing judgment errors in the determination unit 13. In addition, since the processing of the model application unit 11 is omitted because it is not input to the model application unit 11, the processing throughput can be improved.

[0099] Furthermore, the present invention is not limited to the embodiments described above, and includes various modifications. For example, the embodiments described above are examples that have been explained in detail for ease of understanding of the present invention, and are not necessarily limited to having all the described structures. Additionally, a part of the structure of one embodiment can be replaced with the structure of another embodiment, and a structure of another embodiment can be added to the structure of one embodiment. Furthermore, regarding a part of the structure of each embodiment, other structures can be added, deleted, or replaced. Furthermore, the various structures, functions, processing units, etc., described above can be partially or entirely implemented in hardware, for example, by designing with integrated circuits. Furthermore, the various structures, functions, etc., described above can also be implemented in software by a processor interpreting and executing programs that implement each function. The programs, tables, files, and other information implementing each function can be placed in recording devices such as memory, hard disks, SSDs (Solid State Drives), or recording media such as IC cards, SD cards, and DVDs.

[0100] Symbol Explanation

[0101] 11…Model application unit, 12…Evaluation value calculation unit, 13…Judgment unit, 14…Image acquisition unit, 15…Deviance acquisition unit, 16…Input verification unit, 17…Input allowable range calculation unit.

Claims

1. An image processing system, characterized by, Possessing: a model application section that applies a machine learning model to a low-quality image to generate a presumed high-quality image; an evaluation value calculation section that calculates an evaluation value of the presumed high-quality image; and a determination section that determines the presumed high-quality image based on the evaluation value of the presumed high-quality image, the determination section determines whether the presumed high-quality image is uncertain data using a deviation range of evaluation values of high-quality images that are obtained by photographing the low-quality image in high quality, which is previously acquired in correspondence with the evaluation value of the presumed high-quality image.

2. The image processing system according to claim 1, wherein the determination section determines the presumed high-quality image as certain data when the entire deviation range is included in a predetermined allowable range, or determines the presumed high-quality image as uncertain data when a part of the deviation range is not included in the allowable range.

3. The image processing system according to claim 2, wherein the image processing system possesses a deviation acquisition section that acquires the deviation range from a distribution of evaluation values of the high-quality images that correspond to the evaluation value of the presumed high-quality image.

4. The image processing system according to claim 3, wherein the determination section causes an image acquisition section that acquires images to acquire the high-quality image in place of the presumed high-quality image determined as the uncertain data when the presumed high-quality image is determined as the uncertain data.

5. The image processing system according to claim 3, wherein the evaluation value includes any one of an image feature quantity calculated from an image, an inspection result of an object within an image, and a measurement value of an object within an image.

6. The image processing system according to claim 3, wherein the image processing system possesses: a display section that displays a determination method of the deviation range; and an instruction input section that accepts an instruction for changing the determination method of the deviation range.

7. The image processing system according to claim 3, wherein the image processing system possesses a display section that displays, for the presumed high-quality image determined as the uncertain data, the evaluation value of the presumed high-quality image, the deviation range, and the allowable range.

8. The image processing system according to claim 3, wherein the image processing system further possesses an input verification section that calculates an evaluation value of the low-quality image input to the model application section, and determines whether to input the low-quality image to the model application section based on the evaluation value of the low-quality image.

9. The image processing system according to claim 8, wherein The input verification section inputs the low-quality image to the model application section when the evaluation value of the low-quality image is included in a pre-acquired input allowable range, and does not input the low-quality image to the model application section when the evaluation value of the low-quality image is not included in the pre-determined input allowable range.

10. The image processing system according to claim 9, wherein The image processing system includes an input allowable range calculation section that acquires a distribution of evaluation values of the estimated high-quality image corresponding to the evaluation value of the low-quality image, and sets a width between a minimum value and a maximum value of the distribution as the input allowable range.

11. The image processing system according to claim 10, wherein The image processing system includes a display section that displays the evaluation value of the low-quality image that is not input to the model application section and the input allowable range.

Images