Information processing device, information display device, information processing method, information display method, and program

Abstract

In the present invention, a good-product model for highly accurate abnormality detection can be obtained even under a condition with an insufficient number of good-product image samples.　An information processing device 1 comprises: a feature amount extraction unit 110 that extracts a feature amount from a plurality of good-product images 50 of an inspection target; a division method determination unit 120 that divides a good-product image into a plurality of small regions and groups the plurality of small regions into a plurality of groups on the basis of the similarity in the feature amount distribution of the respective small regions; and a model building unit 140 that uses the feature amounts of the small regions belonging to each group to build a good-product model for each group.

Description

Information processing device, information display device, information processing method, information display method, and program CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is based on Japanese Patent Application No. 2024-128235, filed on August 2, 2024, the contents of which are incorporated herein by reference.

[0002] The present disclosure relates to an information processing device, an information display device, an information processing method, an information display method, and a program.

[0003] Conventionally, there has been known an inspection system that detects abnormalities in an inspection object based on an image of the inspection object. For example, Patent Literature 1 discloses an inspection system that subdivides an image of a known inspection object to generate multiple partial images, and performs an inspection on each partial image using a machine learning model created for each partial image.

[0004] Japanese Patent Application Laid-Open No. 2022-131559

[0005] To improve the accuracy of a model for anomaly detection, it is better to use a larger number of good product image samples to build the model. However, in reality, only a small number of good product image samples are often obtained, and building a model with a small number of samples can lead to overfitting, resulting in a problem of low anomaly detection accuracy. The system of Patent Document 1 creates a different machine learning model for each partial image, but further ingenuity is required to build a highly accurate model with a small number of good product image samples. The present disclosure has been made in consideration of this problem, and one of its objectives is to enable the acquisition of a good product model for highly accurate anomaly detection even in situations where the number of good product image samples is insufficient.

[0006] The present disclosure employs the following configuration to solve the above-described problems: An information processing device according to one aspect of the present disclosure includes a feature extraction unit that extracts feature values ​​from a plurality of good product images of an inspection object, a division method determination unit that divides the good product images into a plurality of small regions and groups the plurality of small regions based on the similarity in distribution of feature values ​​of each small region, and a model construction unit that constructs a good product model for each group using the feature values ​​of the small regions belonging to each group.

[0007] With the above configuration, a non-defective product model can be constructed for each group having similar distributions of feature quantities, so that a non-defective product model with high accuracy in detecting anomalies can be constructed even in a situation where there are insufficient samples of non-defective product images.

[0008] The system may also be configured to include a feature extraction unit that extracts features from an inspection image of the inspection object, a region division unit that divides the inspection image into a plurality of regions corresponding to groups, and an abnormality calculation unit that calculates the degree of abnormality for each small region using a good product model of the corresponding group.

[0009] With the above configuration, it is possible to calculate the degree of abnormality for each small region using a non-defective product model that matches the characteristics of the region.

[0010] The image processing device may also be configured to include an abnormality degree integration unit that calculates the abnormality degree of the entire inspection image based on the abnormality degree calculated for each small region.

[0011] With the above configuration, the abnormality degrees calculated for each small region can be integrated to perform a highly accurate pass / fail judgment.

[0012] An information display device according to one aspect of the present disclosure includes an image display unit that displays an image of an object to be inspected, a mask display unit that displays a mask indicating boundaries that divide the image into multiple regions superimposed on the image, and a change operation receiving unit that receives an operation to change the position of the boundaries, and constructs a good product model for each region divided by the boundaries using the features of multiple small regions contained in each region.

[0013] With the above configuration, the user can check the appropriateness of the divided regions, and can change the division method to a more appropriate one as needed.

[0014] The apparatus may also be configured to include an inspection accuracy display unit that displays the inspection accuracy of the entire image when the degree of abnormality is calculated using the corresponding non-defective product model for each region divided by the boundaries.

[0015] With the above configuration, the user can check the inspection accuracy of the divided regions, and can change to a more appropriate division method based on the inspection accuracy.

[0016] The inspection accuracy display unit may be configured to display the inspection accuracy of the entire image corresponding to the boundary positions before and after the change operation so that the inspection accuracy can be compared.

[0017] With the above configuration, the user can check the division method while comparing the inspection accuracy results corresponding to the boundary positions before and after the change operation.

[0018] An information processing method according to one aspect of the present disclosure includes a feature extraction step of extracting features from a plurality of good product images of an inspection object; a division method determination step of dividing the good product images into a plurality of small regions and grouping the small regions into a plurality of groups based on the similarity in the distribution of the features of each small region; and a model construction step of constructing a good product model for each group using the features of the small regions belonging to each group.

[0019] With the above configuration, a non-defective product model can be constructed for each group having similar distributions of feature quantities, so that a non-defective product model with high accuracy in detecting anomalies can be constructed even in a situation where there are insufficient samples of non-defective product images.

[0020] An information display method according to one aspect of the present disclosure includes an image display step of displaying an image of an object to be inspected, a mask display step of displaying a mask indicating boundaries dividing the image into multiple regions superimposed on the image, a change operation receiving step of receiving an operation to change the position of the boundaries, and a step of constructing a good product model for each region divided by the boundaries using feature quantities of multiple small regions contained in each region.

[0021] With the above configuration, the user can check the appropriateness of the divided regions, and can change the division method to a more appropriate one as needed.

[0022] A program according to one aspect of the present disclosure causes a computer to execute a feature extraction step of extracting features from a plurality of good product images of an inspection object; a division method determination step of dividing the good product images into a plurality of small regions and grouping the small regions into a plurality of groups based on the similarity in the distribution of the features of each small region; and a model construction step of constructing a good product model for each group using the features of the small regions belonging to each group.

[0023] With the above configuration, a non-defective product model can be constructed for each group having similar distributions of feature quantities, so that a non-defective product model with high accuracy in detecting anomalies can be constructed even in a situation where there are insufficient samples of non-defective product images.

[0024] A program according to one aspect of the present disclosure causes a computer to execute an image display step of displaying an image of an object to be inspected, a mask display step of displaying a mask indicating boundaries that divide the image into multiple regions superimposed on the image, a change operation reception step of receiving an operation to change the position of the boundaries, and a step of constructing a good product model for each region divided by the boundaries using features of multiple small regions contained in each region.

[0025] With the above configuration, the user can check the appropriateness of the divided regions, and can change the division method to a more appropriate one as needed.

[0026] According to the present disclosure, even in a situation where the number of samples of non-defective product images is insufficient, a non-defective product model for highly accurate anomaly detection can be obtained.

[0027] FIG. 1 is a diagram showing an example of a non-defective product image (A) and an inspection image (B) according to an embodiment of the present disclosure. FIG. 2 is a diagram showing an example of the hardware configuration of an information processing device (information display device) and an information processing terminal according to an embodiment of the present disclosure. FIG. 3 is a diagram showing an example of the functional configuration of an information processing device (information display device) according to an embodiment of the present disclosure. FIG. 4 is a diagram showing an example of a screen displayed on a display according to an embodiment of the present disclosure. FIG. 5 is a diagram showing another example of a screen displayed on a display according to an embodiment of the present disclosure. A flowchart of a procedure for constructing a non-defective product model according to an embodiment of the present disclosure. A flowchart of a procedure for calculating the degree of abnormality of an inspection image according to an embodiment of the present disclosure. A flowchart of a procedure for a user to check and correct a divided region according to an embodiment of the present disclosure.

[0028] An embodiment according to one aspect of the present disclosure (hereinafter also referred to as "the present embodiment") will be described below with reference to the drawings. However, the embodiment described below is merely an example of the present disclosure in all respects. Needless to say, various improvements and modifications can be made without departing from the scope of the present disclosure. In other words, when implementing the present disclosure, specific configurations according to the embodiment may be appropriately adopted. Note that, although data appearing in the present embodiment is described in natural language, more specifically, it may be specified in any of computer-recognizable pseudo-language, commands, parameters, or machine language, but is not limited to these.

[0029] <Overview> The present disclosure is applicable to, for example, an apparatus that uses a machine learning model to perform visual inspection of images of inspected workpieces, such as industrial products, and detect anomalies. The apparatus according to the present disclosure has a function for constructing a non-defective product model for calculating the degree of anomaly of an inspection image 60, and a function for calculating the degree of anomaly of the inspection image 60 using the constructed non-defective product model. The non-defective product model is constructed using features extracted from a non-defective product image 50 of the inspected workpiece. Existing image processing techniques can be used to extract the features from the non-defective product image 50. Examples of techniques that can be used include Scale-Invariant Feature Transform (SIFT), Histograms of Oriented Gradients (HOG), Speeded Up Robust Features (SURF), and Local Binary Pattern (LBP). Other examples include a trained convolutional neural network (CNN) and Haar.

[0030] FIG. 1 illustrates an example of a non-defective image (A) and an inspection image (B) of a workpiece to be inspected. It is desirable that the non-defective image 50 and the inspection image 60 be as consistent as possible in terms of image size, alignment, lighting, and other conditions during image capture. Alternatively, these conditions may be corrected after capture to be identical. As shown in FIG. 1, both images contain background regions such as small regions 51 and 52 and workpiece edge regions (the boundary between the workpiece and the background) such as small regions 53, 54, and 55, due to factors such as alignment during image capture. The features extracted from each image are position-dependent, and feature values ​​at corresponding positions tend to be similar even in different non-defective images. Furthermore, within a single image, there are small regions with similar feature values. For example, in FIG. 1, small regions 51 and 52 are located in the background region, and the feature values ​​of each small region are similar. Similarly, small regions 54 and 55 are both located in the workpiece edge region, and the feature values ​​of each small region are similar. On the other hand, although the small region 53 is located in the workpiece edge region, the similarity between the feature amount of the small region 53 and the feature amounts of the small regions 51, 52, 54, and 55 is low.

[0031] In the present disclosure, a non-defective product image 50 is divided into multiple small regions, and small regions with similar features, such as small regions 51 and 52 and small regions 54 and 55, are grouped together to construct a non-defective product model for each group. For example, if there are 10 samples of non-defective product images, 10 samples of features are extracted from each small region. Small regions with similar distributions of features for the 10 samples are grouped together, and the features of the small regions belonging to the same group are used as the features for the entire group to construct a non-defective product model for each group. In other words, if a group contains five small regions, five times as many features can be obtained, or 50 samples. In this way, even if the original number of samples is small, by grouping similar small regions, a non-defective product model can be constructed using a pseudo-large number of samples, thereby improving the accuracy of the non-defective product model. Note that the distribution of features may be expressed as a range in which a sample set of features plotted in a feature space exists, or may be expressed as a probability distribution by calculating statistics of multiple features. In other words, the distribution of features may be either a discrete distribution or a continuous distribution.

[0032] When calculating the degree of abnormality of an inspection image using the constructed non-defective product model, the inspection image is divided into small regions, and the degree of abnormality is calculated using the non-defective product model of the group to which each small region belongs.

[0033] <Configuration> FIG. 2 is a diagram illustrating an example of the hardware configuration of an information processing device (information display device) 1 according to this embodiment. The information processing device 1 is a computer including a processor 11, a main memory 12, an input / output interface 13, a communication interface 14, and a storage device 15. The storage device 15 is a computer-readable recording medium such as a semiconductor memory (e.g., a volatile memory or a non-volatile memory, but is not limited to these) or a disk medium (e.g., a magnetic recording medium or a magneto-optical recording medium, but is not limited to these). The storage device 15 stores a program to be executed by the processor 11. The program is loaded from the storage device 15 into the main memory 12 and interpreted and executed by the processor 11. The storage device 15 also includes a database 16. The database 16 may be implemented in an external storage device. The database 16 stores, for example, constructed non-defective product models.

[0034] The information processing device 1 is connected to an information processing terminal 2 via a communication network. There may be multiple information processing terminals 2. The information processing device 1 and the information processing terminal 2 may be implemented as the same device. The information processing device 1 is, for example, a server device, but is not limited to a server device and may also be a computer such as a desktop, laptop, or tablet. The information processing terminal 2 may also be a computer such as a smartphone, mobile phone, desktop, laptop, or tablet. The information processing terminal 2 has a control unit 24 including a CPU and memory, a communication unit 25 for connecting to the network N, an input unit 21 such as a touch panel or keyboard for accepting operations from a user, a storage unit 26 in which various programs and data necessary for executing processing in the control unit 24 are stored, and a display 23 for displaying a screen.

[0035] <Functional Configuration> Fig. 3 is a diagram showing an example of the functional configuration of the information processing device 1. As shown in Fig. 3, the information processing device 1 includes an image input unit 100, a feature extraction unit 110, a division method determination unit 120, a region division unit 130, a model construction unit 140, an anomaly degree calculation unit 150, an anomaly degree integrating unit 151, an image display unit 170, a mask display unit 171, a change operation acceptance unit 180, and an inspection accuracy display unit 181, which are functional modules executed by the processor 11. In addition, the division method determination unit 120 includes a tile extraction unit 121, a similarity measurement unit 122, and a grouping unit 123.

[0036] The image input unit 100 has a function of accepting input of a non-defective product image 50 used to construct a non-defective product model and an inspection image 60 for calculating an abnormality degree. The non-defective product image 50 and the inspection image 60 may be images captured by a known camera.

[0037] The feature extraction unit 110 extracts feature amounts from the input non-defective product image 50 or inspection image 60. Any known method can be used to extract the feature amounts. For example, the feature extraction unit 110 may extract the feature amounts using a feature extractor that retains spatial information, such as CNN or Haar.

[0038] The division method determination unit 120 divides the non-defective image 50 into a plurality of small regions (hereinafter also referred to as tiles), and groups the plurality of tiles into a plurality of groups based on the similarity of the distribution of the feature amounts of each tile. The division method determination unit 120 includes a tile extraction unit 121, a similarity measurement unit 122, and a grouping unit 123.

[0039] The tile extraction unit 121 divides the non-defective image 50 into a plurality of tiles with a predetermined tile size and stride, and extracts the feature amount of each tile. The tile size and stride may be set to arbitrary values ​​by the user.

[0040] The smaller each tile, the more likely it is that grouping will occur. On the other hand, if there are spots on the workpiece surface, a moderate size may make it easier to distinguish them from noise. Therefore, the tile size and stride settings can be changed to appropriate sizes depending on the properties of the workpiece being inspected.

[0041] The similarity measurement unit 122 calculates the similarity of the distribution of feature values ​​of each tile. The calculation of the similarity may be performed using a method that uses statistical similarity, or may be performed using an approximation method based on clustering. For example, the similarity between all tiles may be calculated using statistical similarity such as KL distance, PE distance, optimal transport, or independence test.

[0042] The grouping unit 123 classifies tiles having a certain degree of similarity or more into the same group based on the calculated similarity. Alternatively, an approximation method using clustering may be used to treat multiple feature points of each tile as one feature vector, and classify tiles having a certain degree of similarity or more into the same group by clustering.

[0043] For example, for vectors of the same size obtained from each tile, an approximation method using clustering such as K-means can be used to treat multiple feature points of each tile as one feature vector, and those with a certain level of similarity or higher can be classified into the same group through clustering.

[0044] The similarity measurement unit 122 and the grouping unit 123 may be integrated into one clustering processing unit.

[0045] The region dividing unit 130 divides the non-defective image 50 based on the division method determined by the division method determining unit 120 .

[0046] The model construction unit 140 constructs a good product model based on the feature quantities of the good product images 50. The model construction unit 140 constructs a single good product model for the group by using the feature quantities for the good product images extracted for each small region belonging to the same group as the feature quantities of the group. For example, if there are n good product samples and m small regions are grouped together, a good product model is constructed for each small region using n good product samples. However, a good product model can be constructed for each group using n × m good product samples, i.e., m times the number of good product samples per small region. This makes it possible to achieve higher performance than when a good product model is constructed for each small region, even when the number of good product samples is insufficient. Furthermore, by grouping small regions with similar feature distributions together, it is possible to achieve higher performance than conventional methods, even when the distribution of good product features is position-dependent.

[0047] The good product model is a model constructed based on the distribution of the feature quantities of good products, and is used, for example, to estimate the degree of abnormality. The good product model is expressed, for example, by statistics or a sample set of the feature quantities of good products. The model construction unit 140 may construct the good product model using, for example, a neural network.

[0048] Furthermore, by increasing the feature amount per good product model while taking into consideration the position dependency of the distribution of the feature amount of the good product image 50, the good product distribution expressed by the model can be made closer to the true good product distribution, leading to accurate pass / fail judgment.

[0049] The image input unit 100 receives input of an inspection image 60. The inspection image 60 is an image captured by a known camera.

[0050] The feature extraction unit 110 extracts features from the inspection image 60 to reduce redundancy in the image data. Any known method can be used to extract the features. For example, the feature extraction unit 110 extracts features from the input image using a feature extractor that retains spatial information, such as CNN or Haar.

[0051] As one example, features established by existing image processing techniques such as Haar-like features and Hog features may be extracted, or features may be extracted using a trained deep learning feature extractor such as a CNN. In this case, a model trained on general object recognition images or a model trained on defect images may be used.

[0052] The region dividing unit 130 divides the inspection image 60 based on the division method determined by the division method determining unit 120 .

[0053] The anomaly degree calculation unit 150 constructs a good product model in the learning stage using the feature amounts of the good product image 50, and uses the good product model to calculate the degree of anomaly for the feature amounts of the inspection image 60. The anomaly degree calculation unit 150 calculates the degree of anomaly for the feature amounts of the target image based on the distance or similarity to the good product model constructed in the learning stage.

[0054] The anomaly degree calculation unit 150 calculates the degree of anomaly by extracting feature values ​​from each small region of the inspection image 60 and calculating the distance from the non-defective product distribution. The smaller the distance, the more similar the feature values ​​are to the non-defective product image, and the more likely the inspected workpiece is a non-defective product. The degree of anomaly may also be calculated using statistics (mean and covariance) of a non-defective product model. In this case, the feature values ​​of the non-defective product image may be assumed to be distributed according to a multivariate normal distribution, and the inverse of the probability density for the feature value may be used as the degree of anomaly.

[0055] If the degree of abnormality calculated by the abnormality degree calculation unit 150 is equal to or greater than a predetermined value, the abnormality degree calculation unit 150 determines the test result as pass, and if the degree of abnormality is less than the predetermined value, the abnormality degree calculation unit 150 determines the test result as fail.

[0056] The abnormality degree calculation unit 150 may calculate the inspection performance. Based on the degree of abnormality calculated by the abnormality degree calculation unit 150, the determined inspection results are tallied and the inspection performance can be calculated by reflecting results such as a result that was determined to be pass but actually turned out to be fail.

[0057] The inspection performance may be calculated using, for example, a receiver operating characteristic (ROC) curve. More specifically, the ROC may be calculated using a plurality of images of non-defective and defective products prepared in advance, and then the area under the curve (AUC) may be calculated and used as an index of inspection performance. Alternatively, the false positive rate and false negative rate may be calculated and used as an index of inspection performance. This can improve the reliability of the inspection performance calculated by the anomaly degree calculation unit 150.

[0058] The anomaly degree calculation unit 150 calculates the degree of anomaly for the feature quantities of the inspection image 60 for each divided region using a good product model constructed in the learning stage using the feature quantities of the good product image 50. For example, the degree of anomaly may be calculated based on the distance between each feature quantity in each tile and the nearest point of the set of stored feature quantities of good products corresponding to each feature quantity in each tile during inference, and the degree of anomaly for each tile may be integrated to calculate the degree of anomaly for the entire image.

[0059] The anomaly degree integrating unit 151 may integrate the calculated anomaly degrees for each divided small region and output the integrated anomaly degree. The anomaly degree integrating unit 151 may also perform processing to calculate the maximum anomaly degree, the average anomaly degree, or the like, and output the integrated anomaly degree.

[0060] For example, an improved method with high noise resistance may involve clustering small regions based on the degree of anomaly, extracting clusters with areas larger than a predetermined threshold, and calculating the average of the degrees of anomaly for each small region included in the cluster to obtain an integrated degree of anomaly. This method not only calculates the degree of anomaly, but also identifies the location of an anomaly if the inspection result is a failure. Additionally, by displaying the clusters on the display 23 so that they overlap with the inspection image 60, the user can identify the location of an anomaly in the workpiece being inspected, thereby assisting in analyzing the cause of the anomaly.

[0061] The anomaly level integrating unit 151 may integrate the inspection results for each divided small region and output the integrated inspection result. For example, the anomaly level integrating unit 151 may determine the integrated inspection result as failing when there are a predetermined number or more (for example, one) of failures among the inspection results for each divided small region. If there are no failures, the integrated inspection result may be determined as passing.

[0062] The small regions may be pixels of an image, or may be square or rectangular regions. The anomaly degree calculation unit 150 calculates the anomaly degree for each pixel, for example, based on the distance to the non-defective product feature distribution. However, the anomaly degree may be calculated using other methods, or multiple calculation methods may be employed. In this case, the anomaly degree integration unit 151 may integrate the multiple calculation methods and output the anomaly degree.

[0063] The image display unit 170 displays the image showing the divided areas on the screen, superimposed on any of the non-defective product images 50 or inspection images 60. The area division status and inspection results can be confirmed from the display screen controlled by the image display unit 170.

[0064] The image display unit 170 can switch between the non-defective image 50 and the inspection image 60, for example, in response to an instruction from the user.

[0065] The image display unit 170 can display, for example, the following items as checkable items on the screen: Multiple region division patterns Abnormality level and judgment for each image Inspection performance

[0066] The image display unit 170 may display an editing screen, a display selection screen, etc., which will be described later. For example, the image display unit 170 may display a plurality of area division patterns, allowing the user to select a preferred pattern, and the result of the selection may be received from the change operation receiving unit 180, which will be described later.

[0067] The mask display unit 171 receives the change in the division method instructed by the user via the change operation receiving unit 180, which will be described later, and displays a mask M corresponding to the changed division method on the display 23. As a result, the mask M indicating the boundaries dividing the image into a plurality of divided regions can be superimposed on the image displayed by the image display unit 170.

[0068] The mask display unit 171 may receive the division method of the area received by the division method determination unit 120 and display a mask M corresponding to the division method on the display 23, or may display a mask M corresponding to an area divided by any method.

[0069] The mask M displayed by the mask display unit 171 may be filled with different colors so that the divided areas of the same group are the same color. It may also have a predetermined transparency so that the underlying image can be seen. In addition to color coding, the mask M may also be filled with hatching.

[0070] The change operation receiving unit 180 receives an instruction to change the divided areas from the user. For example, the user issues an instruction to change the area division method on an editing screen on a display screen controlled by the image display unit 170, and the change operation receiving unit 180 outputs the received instruction as division method data to the area dividing unit 130.

[0071] The change operation receiving unit 180 may receive an instruction to reevaluate the changed region division. For example, when the user operates a reevaluation button displayed on the image display unit 170 to instruct calculation of the degree of abnormality, the inspection accuracy display unit 181 (described later) can display, for each changed region division, the inspection accuracy of the entire image when the degree of abnormality is calculated using the corresponding non-defective product model.

[0072] As a result, the region division state can be visualized on the image, allowing the user to check and change the division state.By changing the region division method, the user can interpret and intervene in the degree of abnormality and judgment results.

[0073] For example, a user can view the division status overlaid on the image and compare it with the inspection performance to see how the division of the region has affected the model performance.

[0074] The user can also determine whether the inspection area is correctly divided into subareas to which different inspection methods should be applied.

[0075] The inspection accuracy display unit 181 may display on the screen the inspection accuracy of the entire image when the degree of abnormality is calculated using the corresponding non-defective product model for each region divided by the boundaries.

[0076] The inspection accuracy display unit 181 displays the inspection accuracy of the entire image when the degree of abnormality is calculated using the corresponding non-defective product model for each area divided by boundaries, allowing the user to easily try and error by checking the relationship between the division method and the judgment result.

[0077] As described above, the user can check the inspection accuracy of the divided regions, and can change to a more appropriate division method based on the inspection accuracy.

[0078] Furthermore, the inspection accuracy display unit 181 may display the inspection accuracy of the entire image corresponding to the boundary positions before and after the change operation in a manner that allows comparison.

[0079] For a plurality of images divided by different division methods, the inspection accuracy of the entire plurality of images can be calculated, and the calculation results can be displayed on the screen for each image by the inspection accuracy display unit 181.

[0080] As described above, the user can check the division method while comparing the inspection accuracy results corresponding to the boundary positions before and after the change operation.

[0081] <Screen Example> FIG. 4 is a diagram illustrating an example of a screen displayed on the display 23. Using FIG. 4, the user's operation for checking and correcting the divided areas will be specifically described. First, as shown in FIG. 4(A), the display 23 displays an image (left side of the screen) in which a mask M (defining areas R1, R2, and R3) is superimposed on the work P, and an image (right side of the screen) in which an abnormality map is superimposed on the work P. The abnormality map is, for example, a mask M in which each small area is filled with a color that increases in brightness as the degree of abnormality increases. Below the image on the right, the abnormality degree calculated based on the divided area shown on the left, the judgment result (judgment based on the abnormality degree), and the inspection performance are displayed. Operating the "Edit Area" button displayed at the top left of the screen transitions to an editing screen such as that shown in FIG. 4(B).

[0082] On the edit screen, the range of each of the regions R1 to R3 can be changed using a mouse, etc. When the region confirmation button on the upper right of the screen is operated, the screen transitions to the screen shown in FIG.

[0083] The screen in Fig. 4(C) displays the mask M after the area change on the left side. As shown in Fig. 4(C), in the changed mask M, part of the original area R2 has been changed to area R1. When the "Reevaluate" button at the top right of the screen is pressed, the screen transitions to the screen shown in Fig. 4(D).

[0084] The screen in FIG. 4(D) displays the recalculated abnormality degree and inspection performance corresponding to the changed divided area.

[0085] FIG. 5 is a diagram showing another example of a screen displayed on the display 23. In the example of FIG. 5, the user can select a division method from a plurality of options. As shown in FIG. 5, an image with a mask M overlaid, each with a different division method, is displayed on the left side of the screen, and the abnormality level and inspection performance calculated for each division method are displayed on the right side. The user can select a division method from the options. Note that the images on the right side of FIGS. 4A, 4C, 4D, and 5 are not limited to the mode in which an abnormality level map is overlaid; the work P may be displayed without an abnormality level map overlaid, or an abnormality may be indicated in another display mode.

[0086] <Operation> Next, the operation of the information processing device 1 according to this embodiment will be described.

[0087] First, the procedure for constructing a non-defective model will be described with reference to the flowchart of FIG.

[0088] First, in step S101, the image input unit 100 receives an input of a non-defective product image 50 of the workpiece to be inspected.

[0089] Next, in step S102, the feature extraction unit 110 extracts feature amounts from the input non-defective product image 50.

[0090] Next, in step S103, the tile extraction unit 121 divides the non-defective image 50 into a plurality of tiles with a predetermined tile size and stride, and extracts the feature amount of each tile.

[0091] Next, in step S104, the similarity measurement unit 122 calculates the similarity between each tile based on the distribution of the feature amount of each tile.

[0092] Next, in step S105, the grouping unit 123 divides the tiles into a plurality of groups based on the calculated similarities between the tiles.

[0093] Next, in step S106, the model construction unit 140 constructs a non-defective model for each group using the feature amounts of the tiles belonging to each group.

[0094] Next, a procedure for calculating the degree of abnormality of the inspection image 60 using the non-defective product model will be described with reference to the flowchart of FIG.

[0095] First, in step S201, the image input unit 100 receives input of an inspection image 60 of a workpiece to be inspected.

[0096] Next, in step S202, the feature extraction unit 110 extracts features from the input inspection image 60.

[0097] Next, in step S203 , the region dividing unit 130 divides the inspection image 60 into a plurality of regions corresponding to the groups generated by the grouping unit 123 .

[0098] Next, in step S204, the abnormality degree calculation unit 150 calculates the abnormality degree for each tile in the inspection image 60 using the non-defective product model of the corresponding group constructed by the model construction unit 140.

[0099] Next, in step S205, the abnormality degree integrating unit 151 calculates the overall abnormality degree of the inspection image 60 based on the abnormality degrees calculated for each divided region. For example, the abnormality degree integrating unit 151 calculates the overall abnormality degree of the inspection image 60 by calculating the average value of the abnormality degrees calculated by each abnormality degree calculating unit 150.

[0100] Next, the procedure for the user to check and correct the divided areas will be described with reference to the flowchart of FIG.

[0101] First, in step S301, the image display unit 170 displays an image of the workpiece to be inspected on the display 23 of the information processing terminal 2. The image to be displayed may be a randomly selected non-defective product image 50 or an inspection image 60.

[0102] Next, in step S302, the mask display unit 171 displays a mask M indicating the boundaries dividing the image into a plurality of divided regions, superimposed on the image displayed in step S301. The displayed mask M may be a mask M indicating the boundaries dividing the image into a plurality of divided regions using a method determined by the division method determination unit 120, or may be a mask M indicating the boundaries dividing the image into a plurality of divided regions using any division method designated by the user. The mask M may be displayed in a different color for each divided region.

[0103] As shown in Figure 4, a mask M is displayed superimposed on the workpiece P. The mask M indicates the boundaries of three divided regions R1, R2, and R3. Region R2 is divided into two, with region R3 in between.

[0104] Next, in step S303, the inspection accuracy display unit 181 calculates and displays the degree of abnormality and inspection accuracy of the entire image when the degree of abnormality is calculated using the corresponding non-defective model for each divided area divided by the boundaries of the mask M.

[0105] The good product model used here is a good product model for each divided region constructed by the model construction unit 140. In other words, it is a model constructed using the feature amounts of tiles belonging to each divided region. Note that the model construction unit 140 may construct one good product model for a divided region by using the feature amounts for the good product image extracted for each tile belonging to the same divided region as the feature amounts for the entire divided region.

[0106] Next, in step S304, the change operation receiving unit 180 determines whether or not an instruction to change the divided area has been received from the user. For example, when the user presses an “Edit Area” button displayed on the screen, the change operation receiving unit 180 determines that an instruction to change the divided area has been received from the user.

[0107] If the change operation receiving unit 180 determines that the user has pressed the "Edit Area" button (step S304: YES), the process proceeds to step S305. If the change operation receiving unit 180 determines that the user has not pressed the "Edit Area" button (step S304: NO), the process remains at step S304.

[0108] Next, in step S305, the change operation receiving unit 180 acquires operation information instructed by a user through an operation on the screen, and changes the division method. The user may instruct a change in the division method by operating the mouse on the editing screen, for example.

[0109] Next, in step S306, the mask display unit 171 receives the change in the division method instructed by the user, and displays a mask M corresponding to the changed division method on the display 23. Furthermore, the inspection accuracy display unit 181 displays on the screen the degree of abnormality and the inspection accuracy calculated corresponding to the changed division area. Note that the degree of abnormality and the inspection accuracy may be recalculated by the user operating a re-evaluation button displayed on the display 23. The inspection accuracy display unit 181 may display the degree of abnormality and the inspection accuracy before and after the division area change operation so that they can be compared.

[0110] Next, in step S307, the change operation receiving unit 180 determines whether or not it has received an instruction from the user to end the change of the divided area. For example, the change operation receiving unit 180 determines whether or not it has received an instruction from the user to end the change of the divided area by the user operating a "Decide Area" button or a "Reevaluate" button displayed on the screen.

[0111] If it is determined that the user has pressed the "Confirm Area" button, the change operation receiving unit 180 determines that it has received an instruction from the user to end the change to the divided area (step S307: YES), the divided area is confirmed, and the processing ends.

[0112] If it is determined that the user has pressed the "Reevaluate" button, the change operation receiving unit 180 determines that it has not received an instruction from the user to end the change of the divided area (step S307: NO), and proceeds to step S305, where the change operation receiving unit 180 performs the above-mentioned processing, such as acquiring operation information instructed by the user again operating on the screen.

[0113] As described above, according to the present disclosure, regions having a distribution of feature quantities are grouped, and a good product model is constructed for each group. In this case, the feature quantities of small regions belonging to the same group are used as the feature quantities of the group, so that it is possible to substantially increase the number of good product samples, and a high-precision good product model can be constructed. Furthermore, by constructing a high-precision good product model, the model construction unit 140 can increase the reliability of the good product model.

[0114] Furthermore, when determining whether an inspection image is good or bad, the inspection image is divided into regions corresponding to each group, and the degree of abnormality is calculated using the good product model of the corresponding group for each region, thereby enabling accurate determination of whether the image is good or bad.

[0115] Furthermore, a mask M indicating the region division method is superimposed on the image of the inspection image 60 on the display 23, and the user can change the division method on the screen, so that the user can change to a more appropriate division method as needed. At this time, the abnormality level values ​​and inspection accuracy calculated using the division methods before and after the change can be displayed to assist the user in changing the divided regions.

[0116] The present embodiment is intended to facilitate understanding of the present disclosure and is not intended to limit the present disclosure. The present disclosure may be modified or improved without departing from the spirit thereof, and the present disclosure also includes equivalents thereof.

[0117] Furthermore, in this disclosure, the term "unit" does not simply mean a physical means, but also includes cases where the functions of the "unit" are realized by software. Furthermore, the functions of one "unit" or device may be realized by two or more physical means, devices, or software, or the functions of two or more "units" or devices may be realized by one physical means, device, or software. Note that some or all of the above embodiments may also be described as in the following supplementary notes, but are not limited to the following.

[0118] (Supplementary Note 1) An information processing device comprising: a feature extraction unit that extracts feature amounts from a plurality of good-quality images of an inspection object; a division method determination unit that divides the good-quality images into a plurality of small regions and groups the plurality of small regions based on the similarity in distribution of feature amounts of each small region; and a model construction unit that constructs a good-quality model for each group using feature amounts of the small regions belonging to each group. (Supplementary Note 2) The information processing device according to Supplementary Note 1, comprising: a feature extraction unit that extracts feature amounts from an inspection image of the inspection object; a region division unit that divides the inspection image into a plurality of regions corresponding to the groups; and an abnormality degree calculation unit that calculates a degree of abnormality for each small region using the good-quality model of the corresponding group. (Supplementary Note 3) The information processing device according to Supplementary Note 2, comprising: an abnormality degree integrating unit that calculates a degree of abnormality for the inspection image as a whole based on the degree of abnormality calculated for each small region. (Supplementary Note 4) An information display device comprising: an image display unit that displays an image of an inspection object; a mask display unit that displays a mask indicating boundaries that divide the image into a plurality of regions by superimposing it on the image; and a change operation receiving unit that receives an operation to change the position of the boundaries, wherein, for each region divided by the boundaries, a good product model is constructed using feature amounts of a plurality of small regions included in each of the regions. (Supplementary Note 5) The information display device according to Supplementary Note 4, further comprising: an inspection accuracy display unit that displays, for each region divided by the boundaries, the inspection accuracy of the entire image when a degree of abnormality is calculated using a corresponding good product model. (Supplementary Note 6) The information display device according to Supplementary Note 5, wherein the inspection accuracy display unit displays the inspection accuracy of the entire image corresponding to the position of the boundary before and after the change operation in a comparable manner. (Supplementary Note 7) An information processing method executed by an information processing device, comprising: a feature extraction step of extracting features from a plurality of good product images of an inspection object; a division method determination step of dividing the good product images into a plurality of small regions and grouping the plurality of small regions into a plurality of groups based on the similarity of the distribution of the feature values ​​of each small region; and a model construction step of constructing a good product model for each group using the feature values ​​of the small regions belonging to each group.(Supplementary Note 8) An information display method executed by an information display device, comprising: an image display step of displaying an image of an inspection object, a mask display step of displaying a mask indicating boundaries that divide the image into a plurality of regions by superimposing it on the image, a change operation receiving step of receiving an operation to change the position of the boundaries, and a step of constructing a good product model for each region divided by the boundaries using feature amounts of a plurality of small regions included in the region. (Supplementary Note 9) A program for causing a computer to execute: a feature extraction step of extracting feature amounts from a plurality of good product images of the inspection object, a division method determination step of dividing the good product images into a plurality of small regions and grouping the plurality of small regions into a plurality of groups based on the similarity in distribution of feature amounts of each small region, and a model construction step of constructing a good product model for each group using feature amounts of the small regions belonging to each group. (Supplementary Note 10) A program for causing a computer to execute the following steps: an image display step for displaying an image of an object to be inspected; a mask display step for displaying a mask indicating boundaries that divide the image into a plurality of regions superimposed on the image; a change operation receiving step for receiving an operation to change the position of the boundaries; and a step for constructing a non-defective product model for each region divided by the boundaries using feature quantities of a plurality of small regions included in each of the regions.

Claims

1. An information processing device comprising: a feature extraction unit that extracts features from a plurality of good product images of an inspection object; a division method determination unit that divides the good product images into a plurality of small regions and groups the plurality of small regions based on the similarity of the distribution of the feature values ​​of each small region; and a model construction unit that constructs a good product model for each group using the feature values ​​of the small regions belonging to each group.

2. The information processing device according to claim 1, comprising: a feature extraction unit that extracts features from an inspection image of the inspection object; a region division unit that divides the inspection image into a plurality of regions corresponding to the groups; and an abnormality calculation unit that calculates the degree of abnormality for each small region using the non-defective product model of the corresponding group.

3. The information processing device according to claim 2, further comprising: an abnormality degree integration unit that calculates the abnormality degree of the entire inspection image based on the abnormality degree calculated for each small region.

4. An information display device comprising: an image display unit that displays an image of an object to be inspected; a mask display unit that displays a mask indicating boundaries that divide the image into a plurality of regions superimposed on the image; and a change operation receiving unit that receives an operation to change the position of the boundaries, wherein an information display device constructs a non-defective product model for each region divided by the boundaries using feature quantities of a plurality of small regions included in each of the regions.

5. The information display device according to claim 4, further comprising an inspection accuracy display unit that displays the inspection accuracy of the entire image when the degree of abnormality is calculated using the corresponding non-defective product model for each area divided by the boundary.

6. An information display device according to claim 5, wherein the inspection accuracy display unit displays the inspection accuracy of the entire image corresponding to the boundary positions before and after the change operation in a comparative manner.

7. An information processing method executed by an information processing device, comprising: a feature extraction step of extracting features from a plurality of good product images of an inspection object; a division method determination step of dividing the good product images into a plurality of small regions and grouping the plurality of small regions into a plurality of groups based on the similarity of the distribution of the feature values ​​of each small region; and a model construction step of constructing a good product model for each group using the feature values ​​of the small regions belonging to each group.

8. An information display method executed by an information display device, comprising: an image display step of displaying an image of an object to be inspected; a mask display step of displaying a mask indicating boundaries dividing the image into a plurality of regions superimposed on the image; a change operation receiving step of receiving an operation to change the position of the boundaries; and a step of constructing a non-defective product model for each region divided by the boundaries using feature quantities of a plurality of small regions included in each of the regions.

9. A program for causing a computer to execute the following steps: a feature extraction step of extracting features from a plurality of good product images of an inspection object; a division method determination step of dividing the good product images into a plurality of small regions and grouping the plurality of small regions into a plurality of groups based on the similarity of the distribution of the feature values ​​of each small region; and a model construction step of constructing a good product model for each group using the feature values ​​of the small regions belonging to each group.

10. A program for causing a computer to execute the following steps: an image display step for displaying an image of an object to be inspected; a mask display step for displaying a mask indicating boundaries that divide the image into multiple regions superimposed on the image; a change operation receiving step for receiving an operation to change the position of the boundaries; and a step for constructing a non-defective product model for each region divided by the boundaries using feature quantities of multiple small regions included in each of the regions.

Images