Image inspection method and image inspection system for objects

Abstract

Even in situations where sufficient training data or source images for training data cannot be prepared, the image recognition AI can learn efficiently and effectively. [Solution] This image inspection method comprises a model generation step S1 in which a model to be used as a reference for recognizing an object W from an image Im is generated by machine learning using training data relating to an image Im of a predetermined object W; a recognition step S2 in which information about the object W is recognized and output from the image Im using the generated model; and a determination step S3 in which it is determined whether or not the object W is in the intended normal state based on the information about the object W that has been recognized and output. Furthermore, it comprises a classification step S4 in which the information about the image Im and the object W corresponding to the result is classified into a normal determination list and an abnormal determination list based on the determination result, and a re-determination step S5 in which a person in charge looks at the image Im classified into the abnormal determination list and re-determines the state of the object W, and training data is created from at least the image Im that is deemed to be misdetermined among the re-determined image Im.

Description

Technical Field

[0001] The present invention relates to an object image inspection method and an image inspection system.

Background Art

[0002] Conventionally, in production sites such as factories, inspections using images have been performed for the purpose of reducing the burden of various inspections by human visual inspection. In particular, recently, with the development of artificial intelligence (AI) for various purposes, attempts have been made to apply image inspections using image recognition AI (here, an algorithm capable of improving recognition accuracy through learning).

[0003] Here, image recognition AI is mainly classified into three types: object detection type, image classification type, and image segmentation (element segmentation) type. Each image recognition AI has relatively advantageous points. For example, the AI of the image classification type is superior to other types of AI in terms of execution speed, the AI of the image segmentation type is superior to other types of image recognition AI in terms of recognition accuracy, and the AI of the object detection type is superior to other types of image recognition AI in terms of high versatility with a balance in comprehensive aspects such as execution speed, recognition accuracy, annotation, and post-processing. From the above viewpoints, for example, when it is desired to determine (inspect) the quality of a state in which a plurality of objects are in a predetermined positional relationship with a predetermined purpose such as the state related to the assembly of a predetermined product by image inspection in a short time, the object detection type of AI is suitable.

[0004] For example, Patent Document 1 discloses an image inspection system using an object detection type of AI, which acquires good product image data and defective product image data including defective parts of an inspection object to create teacher data, learns a defect detection engine for determining the quality of the appearance of the inspection object by the above AI, and inspects the quality of the appearance of the inspection object reflected in the input image based on the learned defect detection engine.

Prior Art Documents

Patent Documents

[0005] [Patent Document 1] Japanese Patent Publication No. 2023-172508 [Overview of the Initiative] [Problems that the invention aims to solve]

[0006] Incidentally, in image inspection using this type of image recognition AI, it is crucial to prepare sufficient training data. However, preparing training data requires operators to perform specific processing on images of objects that have actually been acquired, which limits the number of training data that can be prepared. Moreover, introducing this type of image inspection system using image recognition AI into a production site often happens during the production preparation phase, after considering the significance (benefits) of the introduction. As a result, engineers have to spend a lot of time on production preparation, and there is a problem that they cannot allocate enough time to create training data. Furthermore, since the production preparation phase is before the production line is actually running, it is difficult to acquire images of objects in a manner that is close to actual operation, and there is a problem that the number of images available for training data is inevitably limited.

[0007] For example, it might seem possible to supplement the amount of training data by adding image data that has been subjected to a predetermined object detection process to images that are ultimately judged to be in the intended state (no problem) to pre-prepared training data. However, since the detection results of object detection in image recognition AI tend to exhibit certain tendencies (biases), using image data that has been judged to be in the intended state one after another as training data may amplify the tendencies of the detection results. Therefore, supplementing training data in this manner is not desirable.

[0008] The problems described above are not limited to image inspection using object detection type AI; they can also occur when using other types of image recognition AI, such as image classification type AI or image segmentation type AI.

[0009] In light of the above circumstances, this specification aims to address the technical challenge of enabling efficient and effective learning by image recognition AI even when sufficient training data or images to serve as the basis for training data cannot be provided. [Means for solving the problem]

[0010] The aforementioned problem is solved by the image inspection method according to the present invention. Specifically, this inspection method is an image inspection method for an object comprising: a model generation step of generating a model that serves as a criterion for recognizing an object from an image using machine learning with training data relating to an image of a predetermined object; a recognition step of recognizing and outputting information about an object from an image using the generated model; and a determination step of determining whether or not the object is in the intended normal state based on the information about the object that has been recognized and output. The method further comprises a classification step of classifying the image and object information corresponding to the determination result into a normal determination list and an abnormal determination list, and a re-determination step of having a person in charge look at the images classified into the abnormal determination list and re-determine the state of the object, and is characterized in that training data is created from at least the images that are deemed to be misdetermined among the re-determined images.

[0011] The accuracy of image recognition using this type of image recognition AI tends to decrease as the circumstances (conditions and environment) in which the image is acquired change. On the other hand, for example, the defect rate of a manufacturing line tends to improve (decrease) as the operating period of the line increases. From these points, the inventors considered that images in which an object is determined not to be in the intended normal state include a considerable number of images in which the image recognition AI incorrectly determined to be abnormal. Therefore, inspected images (images in which the judgment step has been performed) are classified into a normal judgment list and an abnormal judgment list. Images classified into the abnormal judgment list are reviewed by a person (person in charge) to re-determine the state of the object, and training data is created from at least the images that were found to be misjudged among the re-judged images.

[0012] By creating training data using images that have already been inspected in this way, it is possible to supplement the number of training data and efficiently improve the accuracy of image recognition, even in situations where a sufficient number of training data images cannot be prepared. Furthermore, since only images that are highly likely to have been misrecognized by the image recognition AI can be used as training data from among the inspected images, they are suitable as training data, thereby effectively improving the accuracy of image recognition. Alternatively, it is possible to prevent as much as possible the decline in image recognition accuracy that occurs with the continuous operation of the image inspection system, and to maintain high recognition accuracy. In addition, according to the present invention, it is possible to know the percentage of images that the image recognition AI has misjudged (misrecognized), so it is possible to understand the trends of the image recognition AI and predict the future decline in recognition accuracy, and based on this prediction, it is possible to more efficiently readjust the timing (period) of accuracy verification by re-evaluation each time.

[0013] Furthermore, in the image inspection method according to the present invention, training data may be created from images classified into a normal judgment list.

[0014] Images classified as "normal" are presumed to be the type of images that the image recognition AI being used can accurately recognize objects within. Therefore, by using images separated into the "normal" list as training data, the accuracy of image recognition of the same type can be further improved. However, if training continues using only images that are easy to recognize as training data, a bias may develop in the images that can be accurately recognized, and this bias may widen. However, as mentioned above, by correcting images that are easily misclassified and using them as training data, it is possible to avoid as much as possible the occurrence of bias in image recognition.

[0015] Furthermore, in the image inspection method according to the present invention, machine learning may be performed using training data created from images deemed to be misclassified as priority training data, and then machine learning may be performed using all training data, including the priority training data, as total training data.

[0016] In this way, by first performing machine learning using priority training data that includes training data created from images identified as misclassified, and then performing machine learning again using all training data including the priority training data, it becomes possible to improve the recognition level of images that the system struggles to recognize, while maintaining an overall balance.

[0017] Furthermore, in the image inspection method according to the present invention, whenever a predetermined number of images classified as abnormal in the abnormality judgment list are accumulated, the person in charge may be automatically notified that a re-evaluation step should be performed, and an interface may be provided on which the results of the re-evaluation step can be entered, and the images classified as abnormal in the abnormality judgment list may be displayed on the interface. Note that the automatic notification to the person in charge referred to here includes not only means of directly transmitting information to the person in charge's sight or hearing (lighting, sound, etc.), but also means of indirectly transmitting information to the person in charge (display on a predetermined interface, notification by email, etc.).

[0018] This approach allows the person in charge to be notified of the timing when the re-evaluation steps can be performed in batches. Therefore, the person in charge can reliably perform the re-evaluation steps at appropriate times without having to check the number of images classified as abnormal in the list one by one. Furthermore, by configuring the interface as described above, the person in charge can re-evaluate images by looking at the interface display and immediately input the re-evaluation results as data, thus enabling the series of tasks to be performed efficiently. As a result, even a considerable number of misclassified images can be re-evaluated within a predetermined time and added to the training data as needed.

[0019] Furthermore, the aforementioned problems can also be solved by the image inspection system according to the present invention. Specifically, this inspection system is an image inspection program that causes a computer to perform the following steps: a model generation step of generating a model that serves as a criterion for recognizing an object from an image using machine learning with training data relating to an image of a predetermined object; a recognition step of recognizing and outputting information about an object from an image using the generated model; and a determination step of determining whether or not an object is in the intended normal state based on the information about the object that has been recognized and outputted; and an interface that enables communication of predetermined data between the computer and the database. The database stores the image and object information corresponding to the determination result in a normal determination list and an abnormal determination list based on the determination result, and the interface is configured to display images classified in the abnormal determination list, to allow a person in charge to input the result of re-determining the state of the object after viewing the displayed image, and to allow the person in charge to create training data from at least the images that are deemed to be misdetermined among the re-determined images.

[0020] By constructing an image inspection system in this manner, personnel can easily re-evaluate the state of an object, and only images suitable for training data from the re-evaluated images can be reused as training data. Therefore, even in situations where sufficient images for training data cannot be prepared, the number of training data can be supplemented, and the accuracy of image recognition can be efficiently improved. Furthermore, since only images that are highly likely to have been misrecognized by the image recognition AI from the inspected images can be used as training data, they are suitable as training data, thereby effectively improving the accuracy of image recognition. In addition, the image inspection system according to the present invention also allows for knowing the percentage of images that the image recognition AI has mis-evaluated (misrecognized), so it is possible to understand the trends of the image recognition AI and predict future declines in recognition accuracy. Based on this prediction, it is possible to more efficiently readjust the timing (period) of accuracy verification through re-evaluation each time. [Effects of the Invention]

[0021] As described above, according to the image inspection method and the image inspection system according to the present invention, even in a situation where teacher data or an image that is the source of teacher data cannot be sufficiently prepared, learning by the image recognition AI can be efficiently and effectively performed. Therefore, it is possible to stably perform highly accurate image recognition and thus image inspection.

Brief Description of Drawings

[0022] [Figure 1] It is a flowchart of an image inspection method according to an embodiment of the present invention. [Figure 2] It is a diagram conceptually showing the overall configuration of an image inspection system for implementing the image inspection method shown in FIG. 1. [Figure 3] It is a detailed flowchart of the model generation step shown in FIG. 1. [Figure 4] It is a detailed flowchart of the determination step shown in FIG. 1. [Figure 5] It is a detailed flowchart of the classification step shown in FIG. 1. [Figure 6] It is a detailed flowchart of the re-determination step shown in FIG. 1. [Figure 7] It is a detailed flowchart of the machine learning step shown in FIG. 3. [Figure 8] It is a diagram schematically depicting the flow of the image inspection method shown in FIG. 1.

Embodiments for Carrying Out the Invention

[0023] Hereinafter, the content of an image inspection method and an image inspection system according to an embodiment of the present invention will be described based on the drawings.

[0024] Figure 1 is a flowchart showing the flow of an image inspection method according to one embodiment of the present invention. Figure 2 is a conceptual diagram showing the overall configuration of an image inspection system 10 for implementing the image inspection method shown in Figure 1. As shown in these figures, the image inspection method according to this embodiment comprises a model generation step S1, a detection step S2, a determination step S3, a classification step S4, and a re-determination step S5. The image inspection system 10 according to this embodiment also comprises an image acquisition unit 11, an image inspection program 12, a computer 13, an interface 14, and a database 15. First, the details of the image inspection system 10 will be described, and then the details of the image inspection method will be described.

[0025] The image acquisition unit 11 is, for example, one or more cameras, configured to capture images of the object W to be inspected and, if necessary, its surroundings. The image Im (see Figure 8, described later) obtained by the image acquisition unit is made available for transmission to the computer 13 in a digitized state. In this case, the image acquisition unit 11 may directly acquire a digital image, or it may digitize an analog image acquired by the image acquisition unit 11.

[0026] The image inspection program 12 is configured to obtain information about object W from the image Im acquired by the image acquisition unit 11 using an image recognition AI (image recognition algorithm), and to cause the computer 13 to execute a process to determine whether or not object W is in the intended normal state based on the acquired information about object W.

[0027] In this embodiment, the image inspection program 12 uses an object detection type image recognition AI to recognize (detect) information about object W from image Im, and enables inspection of object W based on the detected information. The program structure allows the computer 13 to execute the model generation step S1, detection step S2, determination step S3, and classification step S4 of the steps S1 to S5 shown in Figure 1. Details of each step S1 to S5 will be described later.

[0028] Interface 14 is configured to display images classified into the abnormality judgment list in classification step S4 and to allow input of the results of re-judgment step S5. Therefore, this interface 14 is located in a position where a person engaged in a predetermined task at the installation site of the image inspection system 10 according to this embodiment, such as a predetermined production site, can perform the predetermined operation in the person in charge of re-judgment step S5.

[0029] The data that the person in charge can input through interface 14 includes data related to the results of the re-evaluation step S5 described above (details will be described later), as well as annotation data performed on images related to the predetermined re-evaluation result. Details of each data will be described later. The data input through interface 14 is stored in database 15.

[0030] In addition to the data described above, database 15 stores and accumulates the image Im acquired by the image acquisition unit 11 for actual image inspection, information on the object W detected from this image Im, and data related to the results obtained in the judgment step S3. These data are stored and accumulated in database 15 in a linked state. Database 15 also stores and accumulates training data as training data, which is image Im with predetermined annotations. It is preferable that the data related to the image inspection and the data related to machine learning be stored and accumulated in different data areas within database 15. Of course, these data may also be stored and accumulated in multiple databases 15. Details of each data will be described later.

[0031] Next, the details of each step S1 to S5 of the image inspection method according to this embodiment will be described.

[0032] (S1) Model generation step As shown in Figure 3, the model generation step S1 includes a training data creation step S11 and a machine learning step S12. These training data creation step S11 and machine learning step S12 are performed by having the computer 13 execute the image inspection program 12.

[0033] (S11) Steps to create training data In step S11, an image Im is prepared in advance, which can be determined to represent the object W in a normal state as intended. Training data is then created by performing predetermined annotations on the prepared image Im. The image Im used in step S11 is, for example, an image acquired by the image acquisition unit 11, and images that have undergone processing related to the detection step S2 and determination step S3 described later are excluded in principle.

[0034] For example, when performing machine learning (machine learning step S12) using training data related to the image Im of object W on the object detection type image recognition AI (an object detection type deep neural network represented by YOLO), as shown in Figure 8, training data is created by identifying a rectangular region R surrounding object W (here, two objects W1 and W2) in the image Im as annotation, and tagging the identified region R with information such as a predetermined class (type).

[0035] The annotation described above is primarily performed by the image inspector. This annotation may be performed using interface 14 shown in Figure 2, or using other interfaces not shown.

[0036] The created training data is stored in database 15. In this embodiment, it is stored in database 15 as general training data, for example, in a state distinct from the priority training data described later.

[0037] (S12) Machine Learning Steps In step S12, the object detection type image recognition AI described above is made to perform machine learning using the training data created in step S12 and stored in database 15, thereby generating a reference model (object detection model) for detecting information about object W from image Im. Here, as shown in Figure 8, a rectangular region R surrounding object W (W1, W2) in image Im is identified, and the image recognition AI is trained using training data in which a predetermined class (type) is tagged in region R, thereby generating the object detection model described above.

[0038] The training data, which serves as the training data, may consist only of training data based on image data of objects W that are able to pass the image inspection, as described above, or it may include training data based on image data of objects W that are unable to pass the image inspection, as needed. The above training data is stored as training data in a database 15 that can communicate with the computer 13, for example. When the image inspection program 12 is executed by the computer 13, it is used by reading (recalling) it from the database 15 as needed.

[0039] (S2) Detection step In step S2, the model generated in step S1 is used to detect information about object W from the image Im of object W acquired by the image acquisition unit 11 using the image recognition AI described above. In this embodiment, as shown in Figure 8, the image recognition AI individually identifies objects W (W1, W2…) contained in the image Im input to the image recognition AI, identifies a rectangular region R surrounding objects W (W1, W2), and outputs information about the region R (position, size, shape, type, etc.) and a confidence score, which is a score related to the reliability of the detection. It should be noted that the information output in step S2 is not limited to the above example, and an image recognition AI capable of acquiring other information may be used as needed.

[0040] (S3) Decision step In step S3, it is determined whether object W is in the intended normal state based on the information of object W obtained in step S2. In this embodiment, the determination step S3 includes filtering steps S31a to S31c and an overall determination step S32 (see Figure 4).

[0041] (S31a)~(S31c) Filtering steps In steps S31a to S31c, multiple filtering processes a to c (steps S31a to S31c) are sequentially performed on the information of object W acquired in step S2 to make a preliminary determination of whether object W is in the intended normal state. The multiple filtering processes a to c that can be performed here include confidence rate, position, size, and shape (aspect ratio) of region R. For example, if the first filtering process a is a filtering process related to the confidence rate, a lower threshold for the confidence rate (e.g., 60%) is set, and the information related to the confidence rate among the information acquired in step S2 is compared with the threshold data of the corresponding filtering process a. Then, it is determined whether the data related to the confidence rate is less than the threshold data. This makes a preliminary determination of the state of object W in terms of confidence rate (step S31a). The types, number, order, and conditions of the multiple filtering processes a to c described above are arbitrary and are set appropriately according to the object W to be inspected. That is, in Figure 4, the number of filtering processes performed is three, but it may be two or fewer or four or more.

[0042] After performing one or more filtering processes a to c as described above, a decision is made on whether or not to perform the comprehensive judgment process S32 based on the results of filtering processes a to c. For example, if any of the filtered steps S31a to S31c performed result below the lower threshold (or above the upper threshold), the judgment step S3 is terminated without performing the comprehensive judgment process S32. In this case, all inspection data, including the results of all filtered processes a to c (corresponding image Im, information on the detected object W, and the results of all filtered processes a to c), is stored in the database 15 (steps S33, S34).

[0043] (S32) Overall Judgment Step On the other hand, if the results obtained in all of the multiple filtering steps S31a to S31c are above the lower threshold (or below the upper threshold), then the comprehensive judgment process is performed. Possible comprehensive judgment processes performed here include false / missing item detection and relative position detection. For example, if the comprehensive judgment step S32 performs a comprehensive judgment process related to relative position detection (see Figure 8), the information about the name (type), size, and position of object W(W1,W2) obtained in step S2 is compared with the corresponding information, name (type), size, and position of object W(W1,W2). Then, it is determined whether or not the information matches perfectly. This performs a secondary determination of the state of object W in terms of relative position. Note that the comprehensive judgment process performed is not limited to one type. For example, two or more comprehensive judgment processes may be performed in a predetermined order depending on the state of object W to be inspected.

[0044] When the comprehensive judgment step S32 is performed as described above, all inspection data, including the results of the comprehensive judgment process performed (corresponding image Im, information on the detected object W, the results of all filtering processes a to c, and the results of the comprehensive judgment process), are stored in the database 15 (steps S33, S35).

[0045] (S4) Classification step In step S4, based on the result of the determination step S3, the information of the image Im and object W corresponding to the result is classified into a normal determination list and an abnormal determination list. Step S4 is performed by the computer 13 executing the image inspection program 12. Figure 5 shows a flowchart of one embodiment of the classification step S4. As shown in Figure 5, in the classification step S4 according to this embodiment, first all inspection data including the inspection result for each image Im is retrieved from the database 15 (step S41), and the following classification process is performed for each retrieved data (image Im).

[0046] First, the system checks whether data related to the judgment result exists for the retrieved image Im (Step S42). Then, it performs classification processing on the image Im and object W for which data related to the judgment result exists (Step S43). Information on the image Im and object W for which data related to the judgment result does not exist is discarded (Step S44).

[0047] In step S43, which relates to the classification process, it is determined whether the judgment result is determined to be the intended normal state. If it is determined to be a normal state, the information of the image Im and object W is classified into a normal judgment list and stored in the database 15 (step S45). In this embodiment, training data is created based on the image Im classified into these normal judgment lists (step S46). Since the information of the object W detected in step S2 is already present in the image Im, this information of object W is used as training data and stored in the database 15.

[0048] On the other hand, if the judgment process result is determined to be abnormal, the information of the image Im and object W is classified into an abnormality judgment list and stored in the database 15 (step S47).

[0049] The series of classification processes described above (steps S42 to S47) are repeated until there are no more unclassified images among the multiple images Im called from the database 15 (step S48). In this way, classification processing is performed on the multiple images Im called from the database 15. When the number of images Im classified into the anomaly detection list and stored in the database 15 exceeds a predetermined number, the re-determination step S5 is performed. That is, each time step S4 is performed, it is determined whether or not the number of images Im classified into the anomaly detection list has reached a predetermined number (step S49), and the re-determination step S5 is performed only if the predetermined number has been reached. If the predetermined number has not been reached, the classification step S4 is terminated without performing the re-determination step S5.

[0050] (S5) Re-evaluation step In step S5, the person in charge examines the image Im classified in the abnormality judgment list and re-evaluates the state of object W. Figure 6 shows a detailed flowchart of the re-evaluation step S5. As shown in Figure 6, in step S5, the computer 13 first executes the image inspection program 12 to automatically notify the person in charge that a re-evaluation should be performed (step S51). As an example of automatic notification, the computer 13 sends an email to the person in charge's email address via a predetermined interface 14 informing them that a re-evaluation of the image inspection should be performed. This ensures that the person in charge knows that a re-evaluation should be performed at a predetermined time (when they check the email).

[0051] Next, the computer 13 executes the image inspection program 12 to retrieve the data necessary for re-evaluation from the database 15 (step S52). Here, it retrieves from the database 15 multiple images Im that have been classified as abnormal in the abnormality judgment list but have not been re-evaluated, information on the object W detected from each image Im, and data related to the results of the judgment processing performed based on that information. Then, the computer 13 executes the image inspection program 12 to display the retrieved images Im on the display unit of the interface 14 (step S53). In addition to the images Im, the computer 13 also displays the judgment results obtained in judgment step S3 (the results of each filter processing a to c, and the result of the overall judgment processing if any) on the display unit of the interface 14.

[0052] In this case, the computer 13 may display not only the image Im but also information about the object W detected from the image Im (for example, information about the identified region R, such as the region R shown in Figure 8, tagging information such as class, and the size and shape of region R) together with the image Im. In addition, in this case, region R may be highlighted and displayed according to the content of the filtering process that resulted in an NG judgment, for example.

[0053] As described above, with the necessary preparations for re-evaluation completed, the person in charge looks at the image Im displayed on the interface 14 and performs the re-evaluation (step S54). In this case, the re-evaluation may be performed only on the items determined to be abnormal (filter processing a to c, overall judgment processing), or on all items. At this time, the interface 14 is made available for the person in charge to input the results of the re-evaluation. If the display unit of the interface 14 is a touch panel, the display unit is shown with options for the re-evaluation items and their results (for example, judgment is correct or incorrect) that can be selected by touch.

[0054] As described above, after the person in charge inputs the re-evaluation result into the interface 14, the computer 13 performs different processing depending on the input. That is, if the person in charge inputs that a misjudgment is found as a result of the re-evaluation (step S55), the computer 13 changes the display on the display unit of the interface 14 so that the person in charge can correct the information detected from the image Im through the interface 14 and create training data. The person in charge corrects the information detected from the image Im, such as the content and class of region R, and creates training data from the image Im classified in the anomaly judgment list according to the changed display (step S56). In this embodiment, the training data created in step S56 is transmitted to the database 15 by the computer 13 as priority learning data, distinguished from general learning data, and is newly stored in the priority learning data area within the learning data area already stored in the database 15 (step S57).

[0055] On the other hand, if the re-evaluation results in the automatic evaluation performed according to the image inspection program 12 being correct (the re-evaluation result is also NG), the data used for the re-evaluation is discarded (step S58).

[0056] The series of re-evaluation processes described above (steps S53 to S58) are repeated until there are no more unevaluated images among the images Im retrieved from the database 15 (step S59). With this, the re-evaluation step S5 is completed.

[0057] In the re-evaluation step S5, if priority training data is created and stored in the database 15, machine learning is performed using the priority training data in the model generation step S1. Figure 7 shows a detailed flowchart of the machine learning step S12 in that case. In this case, first, untrained training data is retrieved from the database 15 (step S121). Here, it is determined whether or not the retrieved training data contains the priority training data created in step S5 (step S122). If it is determined that the priority training data is included, the image recognition AI is first made to perform machine learning using only the priority training data (step S123). After that, the image recognition AI is made to perform machine learning using all the training data, including the priority training data used in step S123, to generate an object detection model.

[0058] As described above, in the image inspection method according to this embodiment, the person in charge looks at the image Im classified in the abnormality judgment list and re-determines the state of the object W (re-determines), and at the same time, training data is created from at least the image Im that is deemed to have been mis-determined among the re-determined image Im. By creating training data using the inspected image Im in this way, even in situations where a sufficient amount of training data images cannot be prepared, the number of training data can be supplemented and the accuracy of image recognition can be efficiently improved. Furthermore, according to the inspection method according to this embodiment, only the image Im that the image recognition AI is likely to have mis-recognized among the inspected image Im can be used as training data, so it is possible to effectively improve the accuracy of image recognition. Alternatively, it is possible to prevent as much as possible the image recognition accuracy will decrease with the continuous operation of the image inspection system 10 and maintain high recognition accuracy. In addition, according to the invention according to this embodiment, the proportion of images that the image recognition AI has mis-determined (mis-recognized) can be known, so it is possible to grasp the trend of the image recognition AI and predict the future decline in recognition accuracy, and based on this prediction it is possible to more efficiently set the timing (period) of accuracy verification by re-determining each time.

[0059] In particular, in this embodiment, in machine learning step S12, after performing machine learning using training data created from images Im that were deemed to be misclassified as priority training data, machine learning is performed using all training data, including the priority training data, as all training data. This makes it possible to improve the detection level by the image recognition AI even for images Im of types (categories) that are not good at accurately detecting objects W, while maintaining an overall balance.

[0060] Furthermore, in this embodiment, whenever a predetermined number of images Im classified as abnormal in the abnormal judgment list are accumulated, the person in charge is automatically notified that the re-judgment step S5 should be performed (step S51). An interface 14 is provided on which the results of the re-judgment step S5 can be input, and the images Im classified as abnormal in the abnormal judgment list are displayed on the interface 14 (step S53). In this way, the person in charge can be notified of the timing at which the re-judgment step S5 can be performed in batches. Therefore, the person in charge can reliably perform the re-judgment step S5 (step S54) at an appropriate time without having to check the number of images Im classified as abnormal in the abnormal judgment list one by one. In addition, by configuring the interface 14 as described above, the person in charge can re-judgment the images Im by looking at the display of the interface 14 and immediately input the re-judgment results as data, so that the series of operations can be performed efficiently. Therefore, even if there is a considerable number of misjudgmented images Im, it is possible to re-judgment them within a predetermined time and add them to the training data as necessary.

[0061] Although one embodiment of the present invention has been described above, the image inspection method and image inspection system according to the present invention may also adopt configurations other than those described above, without departing from the spirit of the invention.

[0062] For example, in the above embodiment, we have illustrated a case where the image recognition AI is made to perform machine learning using the priority training data, and then the image recognition AI is made to perform machine learning using all training data, including the priority training data. However, of course, other learning methods are also possible. For example, the machine learning step S12 after acquiring the priority training data may be constructed by performing machine learning using only the priority training data (step S123) without using any training data other than the priority training data. Alternatively, the machine learning step S12 after acquiring the priority training data may be constructed by performing machine learning using all training data, including the priority training data (step S124), without performing machine learning using only the priority training data.

[0063] Furthermore, in the above embodiment, an example was given in which an object detection type AI is applied as the image recognition AI incorporated into the image inspection program 12 (state determination program), but of course, other types of AI may also be applied to the present invention. [Explanation of symbols]

[0064] 10. Image Inspection System 11 Image acquisition unit 12 Image Examination Program 13 Computer 14 Interfaces 15 Databases Im Image R region W,W1,W2 Object

Claims

1. A model generation step of generating a model that serves as a criterion for recognizing an object from an image using machine learning, using training data relating to an image of a predetermined object. A recognition step of recognizing and outputting information about the object from the image using the generated model, and An image inspection method for an object, comprising a determination step of determining whether or not the object is in the intended normal state based on the information of the recognized and outputted object, A classification step in which, based on the result of the determination, the image and the information of the object corresponding to the result are classified into a normal determination list and an abnormal determination list, and The system further includes a re-determination step in which a person in charge looks at the images classified in the aforementioned abnormality determination list and re-determines the state of the object, An image inspection method for objects, comprising creating training data from at least one of the re-evaluated images that was found to be misclassified.

2. An image inspection program that causes a computer to perform the following steps: a model generation step of generating a model that serves as a criterion for recognizing an object from an image using machine learning with training data relating to an image of a predetermined object; a recognition step of recognizing and outputting information about the object from the image using the generated model; and a determination step of determining whether the object is in the intended normal state based on the information of the object that has been recognized and outputted. The aforementioned computer, A database for storing the aforementioned training data, and An object image inspection system comprising an interface that enables communication of predetermined data with the database, The database, based on the result of the determination, classifies and stores the image and object information corresponding to the result into a normal determination list and an abnormal determination list. An object image inspection system wherein the interface is configured to display images classified in the abnormality judgment list, to allow the person in charge to input the result of re-judging the state of the object after viewing the displayed images, and to allow the person in charge to create the training data from at least the images that are deemed to be misjudged among the re-judged images.

Images