Image inspection device

Abstract

To improve learning accuracy and learning efficiency by facilitating the annotation of a plurality of portions.SOLUTION: A pre-trained model includes a feature amount extraction section and a classification section. A learning data setting section includes a setting screen display section, a candidate area estimation section, and a learning data generation section. When a re-estimation instruction for instructing the execution of the estimation of a candidate area is received in a state where a position of a first candidate area, which is a candidate area superimposed on a setting image by the setting screen display section and estimated by the candidate area estimation section, is displayed, the candidate area estimation section estimates a second candidate area so that an area other than the first candidate area is set as the second candidate area. The learning data generation section generates learning data on the basis of the setting image, the first candidate area, and the second candidate area. A learning execution section updates the pre-trained model so as to classify the candidate area into a class of an object on the basis of the learning data.SELECTED DRAWING: Figure 3A

Description

[Technical Field]

[0001] The present disclosure relates to an image inspection device capable of executing a trained model. [Background technology]

[0002] For example, in the factory automation industry, image inspection devices are widely known to replace the process of visually inspecting workpieces by humans. Image inspection devices capture images of workpieces and judge the captured images based on certain criteria. If the image inspection device is an image sensor, the judgment results are output via I / O to an external device such as a programmable logic controller (PLC).

[0003] When the criteria used for judgment by an image inspection device are based on certain image features, even changes in the image features that would not affect the results of a visual inspection by a human may affect the judgment results of the image inspection device. Therefore, an image inspection device that uses an image recognition model obtained by machine learning to obtain judgment results closer to those of a visual inspection is known, such as the image inspection device disclosed in Patent Document 1. The image inspection device of Patent Document 1 is configured to generate a learning device that distinguishes between good and defective product images by learning good product images that have been assigned attributes as good products and defective product images that have been assigned attributes as defective products, and to input newly acquired images into the learning device during operation to make a pass / fail judgment. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2020-187072 Summary of the Invention [Problem to be solved by the invention]

[0005] However, creating an image recognition model obtained through machine learning requires a large amount of image data, and it is difficult for each user to collect the necessary image data, or the burden on the user increases.

[0006] Therefore, the image inspection device of Patent Document 1 has a function to present candidate learning images as an aid in creating learning data. In order to improve the learning of the classifier (learning accuracy and learning efficiency), it is preferable to instruct which class data close to the boundary between classes (good / bad) belong to. To assist in the generation of learning data that allows such instruction, the auxiliary function of Patent Document 1 presents image data close to the class boundary.

[0007] When there is one inspection area per imaging area, the difficulty in generating appropriate learning data is selecting appropriate image data. Annotating that image data is relatively easy, as it only requires presenting the captured image, or a captured image cropped according to the inspection area, to the user and asking whether it is a "good" or "bad" product.

[0008] On the other hand, when multiple annotations are required for a captured image, the annotation process is relatively difficult. In particular, when learning from areas that the user did not actively annotate, all areas that should be annotated must be annotated. This type of annotation work is time-consuming for the user.

[0009] The present disclosure has been made in consideration of the above points, and its purpose is to improve learning accuracy and learning efficiency by making it easy to perform annotation at multiple locations. [Means for solving the problem]

[0010] To achieve the above object, an aspect of the present disclosure may be based on an image inspection device that executes a pre-trained model that detects objects in an image of input image data. The image inspection device includes a control unit that executes the pre-trained model and functions as a training data setting unit that sets training data for on-site training of the pre-trained model, and a training execution unit that updates the pre-trained model based on the training data.

[0011] The pre-trained model includes a feature extraction unit that extracts features that indicate the characteristics of the input image data from the image data, and a classification unit that outputs a class to which an image region of the image data belongs based on the features extracted by the feature extraction unit, and detects the object in the image of the image data.

[0012] The learning data setting unit includes a setting screen display unit that causes a display device to display a setting screen including a setting image display area that displays a setting image, a candidate area estimation unit that executes a pre-trained model to estimate a candidate area for the setting image, and a learning data generation unit that generates the learning data based on the setting image and the estimated candidate area.

[0013] When a re-estimation instruction is received to execute estimation of the candidate area while the setting screen display unit displays the position of a first candidate area, which is the candidate area estimated by the candidate area estimation unit, superimposed on the setting image, the candidate area estimation unit estimates the second candidate area so as to set an area other than the first candidate area as the second candidate area, and the training data generation unit generates the training data based on the setting image, the first candidate area, and the second candidate area. The learning execution unit updates the pre-trained model based on the training data so as to classify the candidate area into the object class.

[0014] According to this configuration, after the candidate area estimation unit estimates a first candidate area, the image inspection device, upon receiving an instruction to estimate a learning candidate area, estimates an area other than the first candidate area as a second candidate area. As a result, when multiple annotations are required for the set image data, the candidate area estimation unit estimates multiple learning candidate areas, and the required multiple annotations are automatically performed without the user having to perform multiple annotations. In other words, learning data that improves learning accuracy and learning efficiency can be easily obtained. The image inspection device then trains a trained model to classify the features of pixels corresponding to multiple learning candidate areas into object classes, thereby improving learning accuracy and learning efficiency without requiring the user's effort.

[0015] After estimating the learning candidate area as the second candidate area, the setting screen display unit can display the second candidate area in a manner different from the first candidate area. That is, the area estimated as the first candidate area and the second candidate area are displayed in a distinguishable manner. In particular, when the image inspection device starts estimating the learning candidate area as the second candidate area and determines the first candidate area as the learning area, the user can distinguish the second candidate area estimated thereafter from the first candidate area, making it easy for the user to understand that the second candidate area has been automatically estimated.

[0016] The learning data setting unit can accept a user-specified learning area by a user specifying the setting image data displayed in the setting image display area. The setting screen display unit can display the user-specified learning area and the learning candidate area in different forms, so that the user can distinguish between the learning candidate area and the user-specified learning area.

[0017] The learning data setting unit can accept a deletion instruction for the learning candidate area displayed by the setting screen display unit and hold the learning candidate area as a deletion set position. In this case, the candidate area estimation unit estimates the learning candidate area so as not to set the same position as the deletion set position as the learning candidate area. Therefore, a learning candidate area that has been deleted once will not be presented again as a learning candidate area, and there is no need to delete it again.

[0018] The candidate area estimation unit estimates, as a first candidate area, an area having a feature amount similar to a reference feature amount, and estimates, as a second candidate area, an area having a feature amount similar to a feature amount corresponding to the first candidate area displayed when an instruction to execute estimation of the learning candidate area is received. This makes it easier for the estimation of the learning candidate area to be in line with the user's intention.

[0019] After estimating the second candidate area, the candidate area estimation unit can also estimate, based on the feature values ​​for each pixel, a position corresponding to a class feature that is similar to a reference class feature within a certain range as a third learning candidate area. In this case, the candidate area estimation unit can estimate the third learning candidate area using the class feature values ​​of the second candidate area as the reference class feature. The candidate area estimation unit can also estimate the third learning candidate area by expanding the certain range.

[0020] The learning execution unit can also train the trained model to classify the first candidate area into the object class and the area other than the first candidate area into the background class. In this case, the learning execution unit can train a boundary separating the object class and the background class. This allows the trained model to detect the object in the inference image data image based on the feature amount for each pixel and the boundary. [Effects of the Invention]

[0021] As described above, the candidate area estimation unit can estimate the first candidate area and the second candidate area, and the estimated first candidate area and second candidate area can be reflected in the learning data, making it easy to annotate multiple locations and improving learning accuracy and efficiency. [Brief explanation of the drawings]

[0022] [Figure 1] 1 is a diagram illustrating an image inspection device according to an embodiment of the present invention during operation. FIG. [Figure 2] FIG. 2 is a hardware configuration diagram of the image inspection device. [Figure 3A] FIG. 2 is a functional block diagram of a DSP. [Figure 3B] This is a conceptual diagram of when the learning tool execution unit executes the AI ​​object detection tool. [Figure 4] 10 is a flowchart showing an example of a procedure for setting up an image inspection device. [Figure 5] 10 is a flowchart illustrating an example of a procedure for setting an inspection tool. [Figure 6] 10 is a flowchart illustrating an example of a procedure for estimating a learning candidate region. [Figure 7] FIG. 10 is a diagram illustrating a display example of a setting screen. [Figure 8] FIG. 10 is a diagram illustrating an example of display of tool setting items. [Figure 9] FIG. 10 is a diagram showing a display example of a first candidate region. [Figure 10] FIG. 10 is a diagram showing a display example of a setting area determination window. [Figure 11] FIG. 10 is a diagram illustrating an outline of a method for estimating a learning candidate area. [Figure 12] FIG. 10 is a diagram showing an example of a display of an image used in learning execution. [Figure 13] 10 is a flowchart showing an example of a procedure for performing an inspection. [Figure 14] FIG. 10 is a diagram illustrating an example of a tool setting screen. [Figure 15] FIG. 10 is a diagram illustrating an example of a tool selection window. [Figure 16] FIG. 10 is a diagram showing an example of a screen displayed when learning settings are started. [Figure 17] FIG. 10 is a diagram illustrating an example of a screen displayed when a box is set. [Figure 18] FIG. 10 is a diagram illustrating an example of a screen displayed when a count target is set. [Figure 19] FIG. 10 is a diagram illustrating an example of a screen displayed when a count target is automatically detected. [Figure 20] FIG. 10 is a diagram showing an example of a rotated training image. [Figure 21] FIG. 10 is a diagram illustrating an example of a warning display window. [Figure 22] FIG. 10 is a diagram showing an example of a window for setting upper and lower limits of the count number. [Figure 23] FIG. 10 is a diagram illustrating an example of a test screen. [Figure 24] FIG. 10 is a diagram illustrating an example of an extended function setting window. [Figure 25] FIG. 10 is a diagram illustrating an example of a screen displayed during operation. [Figure 26] FIG. 10 is a diagram showing an example of a display of a determination result by a sum tool. [Figure 27] FIG. 10 is a diagram showing a case where a tool is set in the middle stage. [Figure 28] FIG. 10 is a diagram showing a case where a search range is set in the middle section by the user. [Figure 29] FIG. 10 is a diagram showing an example of a screen in which counting targets are automatically detected when the middle row is the search range. [Figure 30] FIG. 10 is a diagram showing an example of a warning display. [Figure 31] FIG. 10 is a diagram showing a case where a plurality of learning count tools are set. [Figure 32] FIG. 10 is a diagram showing a case where a learning count tool that is the target of a total tool is selected. [Figure 33] 10 is an example of a screen that is displayed after the settings for the total tool are completed. [Figure 34] FIG. 10 is a diagram showing an example of a screen displayed when a total tool is in operation. [Figure 35]FIG. 10 is a diagram showing a case where the learning count tool in the upper row is selected. [Figure 36] FIG. 10 is a diagram illustrating a function for deleting overlapping detection between tools. [Figure 37] FIG. 10 is a diagram showing a case where a workpiece image that is properly arranged is input. [Figure 38] FIG. 10 is a diagram showing a case where a part of the workpiece is missing. [Figure 39] FIG. 10 is a diagram illustrating a case where a missing part detection setting is performed. [Figure 40] FIG. 10 is a diagram showing a number assignment setting window. [Figure 41] FIG. 10 is a diagram illustrating a method for detecting a missing part. DETAILED DESCRIPTION OF THE INVENTION

[0023] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that the following description of the preferred embodiments is merely exemplary in nature and is not intended to limit the present invention, its applications, or its uses.

[0024] FIG. 1 is a diagram illustrating an image inspection device S according to an embodiment of the present invention during operation. The image inspection device S captures an image of a workpiece W transported by a transport means A according to an imaging setting to acquire inference image data, detects the workpiece W in the image of the acquired inference image data, and outputs the detection result to an external device. An example of the external device is a programmable logic controller (PLC) 5, but a device other than the PLC 5 may also be the external device. The PLC 5 controls the transport means A based on the received detection result, for example, to separate the storage destination of the workpiece W. In the following description, a case will be described in which the external device is the PLC 5. Note that the workpiece W may be a workpiece that is not transported by the transport means A. In the following description, the workpiece is also referred to as the target object.

[0025] The image inspection device S includes an imaging unit 1 for imaging the workpiece W, a control unit 2 to which inference image data captured by the imaging unit 1 is input, a PC (personal computer) 3 for configuring the image inspection device S, and a display device 4 for displaying a setting screen, a selection screen, workpiece images, detection results, etc. The control unit 2 is capable of executing a trained model for detecting the workpiece W in the image of the input inference image data. The control unit 2 outputs to the PLC 5 according to the detection results obtained by the trained model.

[0026] Here, the image inspection device S may be used, for example, to inspect the workpiece W from various angles at various points in a manufacturing device or production line. For this reason, multiple image inspection devices S may be installed in a single manufacturing device or production line, potentially resulting in insufficient installation space and power supply. Therefore, the image inspection device S must be compact to accommodate the installation space and power-efficient to accommodate the power supply. To meet these requirements, the image inspection device S according to this embodiment does not have a GPU. In other words, the control unit 2 can execute a trained model that has been trained to the extent that basic workpiece W detection is possible, but the desired detection accuracy can be achieved without the user having to perform advanced additional training, for which the use of a GPU is recommended. Furthermore, not requiring the user to perform advanced training means that the user does not need to prepare a GPU for training, thereby reducing the time required for training. Note that a single image inspection device S may be installed and operated in a manufacturing device or production line. The image inspection device S may also be referred to as an image sensor.

[0027] (Configuration of imaging unit) The imaging unit 1 is separate from the control unit 2 and is installed so as to be able to capture an image of the workpiece W from a desired direction. The workpieces W are sequentially transported by the transport means A into the imaging field of view of the imaging unit 1.

[0028] As shown in FIG. 2, the imaging unit 1 includes an illumination module 10 for illuminating the workpiece W, and a camera module 11 for capturing an image of the workpiece W illuminated by the illumination module 10.

[0029] The lighting module 10 has an LED (light emitting diode) 10a that irradiates light toward the workpiece W, and an LED driver 10b that controls the light intensity and light emission timing of the LED 10a. The LED driver 10b is connected to a head communication section 20 (described later) of the control unit 2, and is controlled by a control section 21 (described later) of the control unit 2.

[0030] The camera module 11 has an AF motor 11a and an imaging board 11b. The AF motor 11a is a member for automatically focusing on the workpiece W by driving a focusing lens of an optical system (not shown). The autofocus method is not particularly limited, and examples include a contrast method.

[0031] The imaging board 11b is equipped with a CMOS sensor 11c, an FPGA 11d, and a DSP 11e. The CMOS sensor 11c is an image sensor that receives light that is irradiated from the LED 10a onto the workpiece W and reflected by the workpiece W. This CMOS sensor 11c is connected to the head communication section 20 of the control unit 2, and is controlled by the control section 21 of the control unit 2 to perform exposure processing at a predetermined timing for a predetermined time.

[0032] The FPGA 11d is a processing device whose internal processing contents can be changed. The DSP 11e is a signal processing device. A light-receiving amount signal of the light-receiving element of the CMOS sensor 11c is output to the FPGA 11d for processing, and is also output to the DSP 11e for processing. The processing by the FPGA 11d and the DSP 11e is not particularly limited, but examples thereof include various types of filter processing. The image data processed by the FPGA 11d and the DSP 11e is transmitted from the imaging unit 1 to the control unit 2.

[0033] The imaging unit 1 and the control unit 2 are connected via a communication cable 6. Therefore, the control unit 2 can be installed in a location away from the location where the imaging unit 1 is installed.

[0034] (PC configuration) PC3 is configured as a general-purpose personal computer or the like. In this example, the personal computer can be used as PC3 by installing a predetermined program on it. PC3 includes operating devices such as a keyboard 3a and a mouse (not shown). A user of the image inspection device S can perform setting operations and selection operations on the image inspection device S by operating the operating devices on PC3. Specific setting operations and selection operations will be described later.

[0035] The PC 3 and the communication board 22 of the control unit 2 are connected to be able to communicate with each other, and information based on setting operations by the user is sent from the PC 3 to the control unit 2. In addition, the PC 3 is able to receive image data of the workpiece W, inspection results, etc. output from the control unit 2. The PC 3 and the control unit 2 are connected via a communication cable 7. Therefore, the PC 3 can be installed in a location away from where the control unit 2 is installed.

[0036] (Configuration of display device 4) The display device 4 is configured, for example, with a liquid crystal display or an organic EL display. In this example, the display device 4 includes a touch panel 4a. The touch panel 4a is a component capable of detecting operations by the user's finger. The type of the touch panel 4a is not particularly limited, and examples include a capacitive type and an infrared type. The display device 4 and the communication board 22 of the control unit 2 are connected to be able to communicate with each other. Operation information of the touch panel 4a by the user is transmitted from the display device 4 to the control unit 2. In addition, the display device 4 is capable of receiving image data of the workpiece W output from the control unit 2. The display device 4 and the control unit 2 are connected via a communication cable 7. Therefore, the display device 4 can be installed in a location away from the installation location of the control unit 2.

[0037] The PC 3 and the display device 4 may be configured as an integrated unit. For example, the display device 4 may be configured as a display device that the PC 3 has. In this case, the main body of the PC 3 and the display device 4 may be integrated or may be separate. In this example, the communication board 22 and the PLC 5 are connected via a communication cable 7.

[0038] (Configuration of control unit 2) 2, the control unit 2 includes a head communication section 20, a control section 21, a communication board 22, a power supply 23, a connector board 24, an I / O board 25, and a storage device (storage section) 26. The head communication section 20 is connected to the control section 21 and is a section that executes mutual communication between the control section 21 and the imaging unit 1. A control signal for the imaging unit 1 output from the control section 21 is transmitted to the imaging unit 1 via the head communication section 20. The control signal for the imaging unit 1 includes a signal that controls the light emission timing and light emission amount of the LED 10a, and a signal that controls the AF motor 11a and the imaging board 11b. Furthermore, image data acquired by the imaging unit 1 is output from the imaging unit 1 and then transmitted to the control section 21 via the head communication section 20.

[0039] The control unit 21 has a DSP 21a and FPGA 21b that perform various signal processing, an accelerator 21c that speeds up the processing, and a memory 21d made up of RAM, ROM, etc. The specific configuration of the control unit 21 will be described later.

[0040] The communication board 22 is connected to the control unit 21, and is a member that executes mutual communication between the control unit 21 and the PC 3, the display device 4, and the PLC 5.

[0041] The connector board 24 includes a power supply interface 24a. A power cable (not shown) for supplying power from an external source is connected to the power supply interface 24a. The connector board 24 is connected to a power supply 23, and power supplied from the outside to the power supply interface 24a is adjusted to a predetermined voltage by the power supply 23 and then supplied to the control unit 21. The power supplied to the control unit 21 is supplied to the imaging unit 1 via the head communication unit 20.

[0042] The I / O board 25 is connected to the control unit 21. The inspection results output from the control unit 21 are input to the PLC 5 via the I / O board 25.

[0043] (Details of DSP21a) 3A is a functional block diagram of the DSP 21a included in the control unit 2. The DSP 21a includes an imaging setting unit, a rule-based related functional block, and a function block related to a detection task using a trained model.

[0044] The imaging setting unit sets imaging setting parameters and reflects the set imaging setting parameters when capturing an image of the workpiece W. The imaging setting parameters include, for example, multiple parameters such as the timing of illumination by the illumination module 10, brightness (light emission amount), exposure time by the camera module 11, and focus (focal position) of the camera module 11. When setting the image inspection device S, the imaging setting unit displays a GUI (graphical user interface) related to setting the imaging setting parameters on the display device 4. Although not shown, the GUI related to setting the imaging setting parameters has areas where the illumination brightness, exposure time, focus, etc. can be individually set. When the user inputs each parameter on the GUI using the touch panel 4a, keyboard 3a, etc., the imaging setting unit accepts the user's setting operation and outputs each parameter to, for example, the storage device 26 for storage therein. Each parameter can be read from the storage device 26 as needed.

[0045] When acquiring image data of the workpiece W, the imaging setting unit outputs the imaging setting data including imaging setting parameters to the camera module 11. When the camera module 11 receives the imaging setting data, it sets the imaging setting parameters included in the received imaging setting data within the camera module 11 so that they are reflected when capturing an image. When the camera module 11 receives an imaging trigger signal, it illuminates the workpiece W in accordance with the imaging setting parameters and captures an image of the workpiece W to acquire image data of the workpiece W.

[0046] The rule-based related functional block has a parameter setting unit and a tool selection unit. The tool selection unit allows the user to select a tool to be used for inspection. When setting up the image inspection device S, the tool selection unit displays a GUI related to tool selection on the display device 4. Although not shown, the GUI related to tool selection has an area where the user can select a desired tool from multiple tools. When the user selects an arbitrary tool on the GUI using the touch panel 4a, keyboard 3a, etc., the tool selection unit accepts the user's selection operation and outputs the selected tool to, for example, the storage device 26 for storage in the storage device 26. The selected tool can be read from the storage device 26 as needed.

[0047] Information about the tool selected by the tool selection unit is output to the parameter setting unit as selected tool information. When setting the image inspection device S, the parameter setting unit displays a GUI (graphical user interface) related to setting tool parameters on the display device 4. Tool parameters are parameters that can be set for each tool. When the user sets tool parameters on the GUI using the touch panel 4a, keyboard 3a, etc., the parameter setting unit accepts the setting operation by the user and outputs the tool parameters to, for example, the storage device 26, where they are stored. Tool parameters can be read from the storage device 26 as needed.

[0048] The detection task-related functional block includes a learning data setting unit 310, a feature extraction unit, and an inspection operation unit. The learning data setting unit 310 performs settings related to the inspection of captured images and reflects the settings related to the inspection. When configuring the image inspection device S, the learning data setting unit 310 displays a GUI on the display device 4 for configuring settings related to the inspection of captured images. When a user inputs parameters into the GUI using the touch panel 4a, keyboard 3a, or the like, the learning data setting unit 310 accepts the user's setting operation and outputs each setting to, for example, the storage device 26 for storage in the storage device 26.

[0049] The settings made by the learning data setting unit 310 can be broadly divided into setting the inspection range of the captured image, a setting called learning, and setting the inspection specifications based on the judgment results. The setting of the inspection range can be performed by the master image registration unit and the inspection window setting unit. The setting called learning can be performed by the learning data setting unit 310 and the classifier update unit 315. The setting of the inspection specifications based on the judgment results can be performed by the inspection condition setting unit. Once these settings are made, the learning data setting unit 310 reflects the settings when the inspection is performed (when the image inspection device S is operating).

[0050] The master image registration unit is a part that registers a captured image as a master image. Specifically, the master image registration unit incorporates the captured image into a GUI and displays it on the display device 4, thereby presenting it to the user. When the user registers the captured image displayed on the display device 4 as a master image, the user performs a registration operation using the touch panel 4a, keyboard 3a, etc. The master image registration unit registers the captured image for which the user has performed the registration operation as a master image, and outputs it to, for example, the storage device 26, where it is stored.

[0051] The inspection window setting unit receives instructions, for example, regarding the position, range, shape, etc. of the inspection window relative to the captured image. Specifically, the inspection window setting unit displays a GUI for setting the inspection window on the display device 4. When the user inputs the position, range, shape, etc. of the inspection window on the GUI using the touch panel 4a, the keyboard 3a, etc., the inspection window setting unit receives the setting operation by the user and outputs the information related to the inspection window to, for example, the storage device 26, and stores it in the storage device 26. The inspection window setting unit reflects the setting of the inspection range relative to the imaging range based on the positional relationship of the inspection window relative to the captured image.

[0052] The learning data setting unit 310 is a unit that sets learning data D1 for updating a classifier, which is a pre-trained model. Specifically, the learning data setting unit 310 includes a learning image selection unit 310a, a learning data generation unit 310b, and a label information setting unit 310c, and the label information setting unit 310c includes a setting screen display unit 311 and a candidate area estimation unit 312. As will be described in detail later, the setting screen display unit 311 is a unit that causes the display device 4 to display a setting screen including a setting image display area that displays a setting image, and the candidate area estimation unit 312 is a unit that estimates a candidate area in the setting image data. In this way, the control unit 21 controls the calculation processing by the pre-trained model and also functions as the learning data setting unit 310 that sets learning data D1 for on-site learning of the pre-trained model.

[0053] Here, one of the classes resulting from the AI ​​object detection is the workpiece W, i.e., the “object (foreground),” and the other is the portion other than the workpiece W, i.e., the “background.” In other words, all image regions included in a captured image are classified as either the “foreground” or the “background.” Therefore, when a captured image is displayed on the display device 4 and a user specifies the position of the workpiece W in the captured image, the specified position in the captured image is classified as the “object (foreground)” class, and the unspecified position in the captured image is classified as the “background” class. In this way, the learning data setting unit 310 acquires two types of label information by accepting a user's specification of the position of the workpiece W in the captured image, and stores the acquired label information in the storage device 26, etc. When this label information is combined with the image data displayed when the label information is set, an image region and a label corresponding to the image region are identified. Therefore, the learning data setting unit 310 outputs information related to the combination of the image data of the displayed captured image and the label information assigned to the image data of the captured image to the learning data generation unit 310b.

[0054] At this time, all positions in the captured image that are not designated as "object (foreground)" class are processed as "background" class. For this reason, it is necessary to designate all positions that correspond to the "foreground" class, and the displayed GUI and related functions are configured to assist in designating all positions in the "foreground" class.

[0055] The training data generation unit 310b is a unit that acquires various pieces of information from the training image selection unit 310a and the label information setting unit 310c. As will be described in detail later, the training data generation unit 310b is a unit that generates training data D1 based on the setting image data of the setting image I3 that is selected as the setting image (training image) I3 and displayed on the setting screen, and the candidate areas estimated by the candidate area estimation unit 312. If the candidate areas estimated by the candidate area estimation unit 312 include, for example, a first candidate area and a second candidate area, the training data generation unit 310b generates training data D1 based on the setting image data, the first candidate area, and the second candidate area.

[0056] The learning data generation unit 310b outputs the generated learning data D1 to the classifier update unit 315. The classifier update unit 315 is a learning execution unit that executes on-site learning based on the learning data D1 received from the learning data generation unit 310b. In this way, the control unit 21 also functions as a learning execution unit that executes additional learning based on the learning data.

[0057] The classifier update unit 315 includes a classifier update unit 315a and a class addition unit 315b. The classifier update unit 315 updates the classifier used in the test by performing a predetermined update method based on the received learning data. In other words, the classifier update unit 315 performs learning using the learning data.

[0058] In the "learning tool" mode, the classifier update unit 315a updates the SVM (support vector machine) used as a classifier. Specifically, the classifier update unit 315a inputs image data (captured images) into a feature extractor controlled by the feature extraction unit to obtain feature vectors. This allows the training data group to be treated as a group of "combinations of feature vectors x class labels corresponding to each feature vector." Based on this group of combinations, a boundary surface is generated so that each class identified by the class label can be distinguished in a feature space onto which the feature vectors are mapped. When additional training data is available, the boundary surface is readjusted so that the additional feature vectors in the feature space are classified into the class associated with the additional feature vector.

[0059] The classifier update unit 315a also updates the SVM as a classifier in the AI ​​object detection mode. The difference from the "learning tool" mode is that a region image is input to obtain a feature vector, but the classifier is otherwise updated in the same way as in the "learning tool" mode.

[0060] The inspection condition setting unit is a part that sets what conditions the judgment result must satisfy to output a predetermined inspection result. The inspection condition setting unit displays a GUI on the display device 4 that enables selection of a judgment mode and setting of judgment conditions. Although not shown, this GUI has an area where judgment conditions can be set. When a user sets judgment conditions on the GUI using the touch panel 4a, keyboard 3a, etc., the inspection condition setting unit accepts the user's setting operation of the judgment conditions. For example, an example of a judgment condition is "how many objects must be detected for the inspection result to be 'good'" when the judgment mode is AI object detection mode.

[0061] The feature extraction unit is configured, for example, by a convolutional neural network. The feature extraction unit extracts features that indicate the characteristics of input image data from the image data, and can also extract features for any position in the input image data, for example. The feature output unit outputs a class to which the image data belongs as information that indicates the characteristics of the image data based on the features extracted by the feature extraction unit, and detects objects in the image data.

[0062] The feature extraction unit has an image feature extraction unit and a feature map output unit. The image feature extraction unit inputs image data (captured image) to the feature extractor to obtain a multidimensional feature vector. The feature map output unit inputs image data to the feature extractor to obtain a feature map in which a feature vector is calculated for each convolution pixel corresponding to a certain range of pixel area in the input image data.

[0063] The inspection operation unit has a judgment unit and an inspection result output unit, and generates a judgment result by the judgment unit and an inspection result based on the judgment result, and outputs the generated inspection result from the I / O board 25 to the PLC 5 or the like.

[0064] The determination unit is a part that outputs a determination result based on an input feature amount (a feature amount vector or a feature map), and has an image classification unit and an image region classification unit.

[0065] In the "learning tool" mode, the image classification unit classifies the feature vectors input from the image feature extraction unit using a classifier (SVM) that classifies feature vectors into "good" and "bad" classes. The judgment result output from the image classification unit indicates which class the image data corresponding to the feature vector belongs to.

[0066] In AI object detection mode, the image region classification unit uses a classifier (SVM) that classifies the feature map input from the feature map output unit into "foreground" and "background" classes to classify the feature vectors of the input feature map. The image region classification unit classifies the convolved pixels of the feature map into "foreground" or "background," and this classification result makes it possible to determine where the "foreground," i.e., the workpiece W, exists in the image data corresponding to the feature map.

[0067] The inspection result output unit is a part that generates and outputs inspection results based on the judgment results output from the judgment unit. In the "learning tool" mode, the inspection result output unit obtains the classification result of "good" or "bad" for the image data as the judgment result. The inspection result output unit outputs the obtained judgment result as it is as the inspection result for the image.

[0068] The inspection result output unit has an inspection result generation unit. In the AI ​​object detection mode, the inspection result generation unit obtains a judgment result as to where the work W is located in the image data corresponding to the input feature map based on the classification result by the image area classification unit. For example, if the inspection condition is "there is a specific number of any target object (work W) (one for search, any number for count) or more," the inspection result generation unit can output the inspection result based on the number of work W in the image data indicated in the judgment result, and display the area corresponding to the detected work W on the target image data.

[0069] It is possible to register multiple types of objects for object detection. For example, by registering a first object and a second object that has a different shape, color, or size from the first object, the inspection result generating unit obtains as a determination result where and at what angle the first object exists in the image data, and also obtains as a determination result where and at what angle the second object exists in the image data.

[0070] The inspection result output unit may combine multiple inspection results obtained in AI object detection mode to generate and output further inspection results. For example, if the "learning tool" judges the image as "good" and there are two objects in the image (the inspection mode is count and the judgment tool used is AI object detection mode), the inspection result may be output as "good."

[0071] The DSP 21a also has an additional training image designation unit, which allows the user to designate training images when additional training is required on the trained model.

[0072] 3B is a conceptual diagram of the learning tool execution unit executing the AI ​​object detection tool T2. The learning tool execution unit executes the feature extraction unit and the determination unit as a pre-trained model.

[0073] Image data of the captured image I1 is input to a feature extraction unit executed by a learning tool execution unit. The feature extraction unit outputs a feature F5. The feature F5 is a feature map that reflects the spatial information of the captured image I1.

[0074] The feature F5 extracted by the feature extraction unit is input to the determination unit. The determination unit identifies and outputs the position of the object in the captured image I1 based on the feature F5. More specifically, the determination unit compares the feature F2 indicating the object with the feature corresponding to each convolved pixel that constitutes the feature F5, determines that an area with a high degree of similarity is an object area where the object is located, and outputs information indicating the position of the object area in the captured image I1.

[0075] (When setting up an image inspection device) 4 is a flowchart showing the procedure for setting up the image inspection device S. In step SA1 of imaging setting after the start, imaging setting parameters are set by the imaging setting section. In step SA2, the imaging setting parameters set in step SA1 are applied to cause the imaging unit 1 to perform imaging. Image data acquired by the imaging unit 1 is stored in the storage device 26 and is treated as a master image or a learning image.

[0076] In step SA3, the inspection tool is set according to the judgment mode. In step SA4, the input of signals from outside, the output of signals to outside, and communication settings are performed.

[0077] Details of step SA3 will be described with reference to the flowchart shown in Fig. 5. In step SB1, the image acquired in step SA2 shown in Fig. 4 is registered as a master image by a master image registration unit. In step SB2, an inspection window setting unit sets an inspection range on the master image registered in step SB1.

[0078] In step SB3, a learning image is added. In the case of a "learning tool," it is possible to simultaneously perform the addition of the learning image and the determination setting in step SB4.

[0079] In the AI ​​object detection mode and AI-OCR mode, displaying a training image on the GUI corresponds to adding the training image (step SB3), and accepting a designation for the displayed image corresponds to judgment setting (step SB4). The judgment setting in step SB4 generates training data after accepting the designation, and then repeats the process of additionally training the classifier using the training data, and then ends.

[0080] In step SB5, the conditions for generating inspection results based on the judgment results are set. For example, if counting is selected as the inspection tool, the number of areas that are judged as targets in AI object detection mode must be set to output a "pass" inspection result.

[0081] (Estimation of candidate learning areas) 6 is a flowchart showing the procedure when the learning data setting unit 310 estimates a candidate area. This flowchart can be started after step SB2 in the flowchart shown in FIG. 5. In step SC1, which corresponds to step SB3, a setting image (learning image) I3 to be used when estimating a candidate area is selected. Specifically, the learning data setting unit 310 causes the setting screen display unit 311 to display a setting screen 600 such as that shown in FIG. 7 on the display device 4.

[0082] The setting screen 600 displays a setting image display area 601 in which setting image data is displayed, and an image acquisition window 602. The image acquisition window 602 is provided with a master image selection button 602a and a live image selection button 602b. When the learning data setting unit 310 detects that the master image selection button 602a has been operated, it sets the image data registered as the master image I2 by the master image registration unit in step SB2 shown in FIG. 5 as setting image data, and displays a setting image I3 based on the setting image data in the setting image display area 601. In this way, the image used when setting the inspection range by the inspection window setting unit can be set as the setting image I3.

[0083] On the other hand, when the learning data setting unit 310 detects that the live image selection button 602b has been operated, it calls up image data currently captured by the imaging unit 1 from the imaging unit 1, sets the called image data as setting image data, and displays a setting image I3 based on this setting image data in the setting image display area 601. In this way, the live view image can also be set as the setting image I3.

[0084] The image acquisition window 602 is provided with an image history button 602c, a learning history button 603d, and a file selection button 602e. When the learning data setting unit 310 detects that the image history button 602c has been operated, it displays images that have been acquired by the imaging unit 1 up to that point in a list format on the setting screen 600 and accepts image selection by the user. The learning data setting unit 310 displays the image selected by the user in the setting image display area 601 as setting image I3.

[0085] When the learning data setting unit 310 detects that the learning history button 603d has been operated, it displays images that have been used as learning images in a list format on the setting screen 600 and accepts the user's selection of a learning image. The learning data setting unit 310 displays the learning image selected by the user in the setting image display area 601 as setting image I3.

[0086] When the learning data setting unit 310 detects that the file selection button 602e has been operated, it displays the images stored in the storage device 26 in a list format on the setting screen 600 and accepts the user's selection of an image. The learning data setting unit 310 displays the image selected by the user in the setting image display area 601 as the setting image I3.

[0087] After setting the setting image I3, the learning data setting unit 310 displays a setting screen 600 as shown in FIG. 8 on the display device 4. The setting screen 600 shown in FIG. 8 displays a list display window 610 for displaying a list of tool setting items. The list display window 610 is provided with a search range edit button 610a, a learning button 610b, and a judgment condition setting button 610c. When the search range edit button 610a and the learning button 610b are operated, the learning data setting unit 310 accepts, based on a user operation, settings regarding the range in the setting image in which to count objects and to perform object detection for counting objects.

[0088] When the judgment condition setting button 610c is operated, the learning data setting unit 310 accepts the setting of the conditions for determining the inspection result as a tool from the object detection result based on the user's operation. For example, if a predetermined number is set as the condition for the number of objects to be counted, the predetermined number becomes the condition for determining the inspection result.

[0089] After step SC1, in step SC2, the label information setting unit 310c specifies a first setting area in the setting image data set in step SC1. In step SC2, a setting screen 600 shown in FIG. 9 is displayed on the display device 4. The setting screen 600 shown in FIG. 9 displays a procedure display window 620 showing the procedure for specifying a setting area. In addition, in the setting image display area 601 of the setting screen 600 shown in FIG. 9, a designation box 621, which the user uses to designate a setting area as the first setting area, is superimposed on the setting image I3. The position, size, shape, angle, etc. of the designation box 621 can be changed by the user using a mouse or the like; this change operation is called an editing operation. When the training data setting unit 310 accepts an editing operation, the position, size, shape, angle, etc. of the designation box 621 on the setting image display area 601 are changed and displayed so as to reflect the editing operation. In this way, the training data setting unit 310 can accept the designation of a first setting area through a user operation on the setting image I3 displayed in the setting image display area 601.

[0090] The first setting area designated in step SC2 may be automatically detected by the learning data setting unit 310 based on, for example, feature values ​​corresponding to a known class. That is, the setting image I3 displayed in the setting image display area 601 included in the setting screen may be input to the pre-trained model to acquire feature values ​​for each pixel. Then, the candidate area estimation unit 312 may estimate a candidate area as the first setting area in the setting image data based on the feature values ​​for each pixel. At this time, the setting screen display unit 311 superimposes a box of a different form from the designation box 621 on the setting image I3 so as to surround the estimated candidate area. That is, by displaying a box superimposed on the setting image I3 as information indicating the position of the candidate area, the setting screen display unit 311 can present to the user which area is the candidate area estimated by the pre-trained model. Furthermore, the information indicating the position of the first setting area is not limited to the designation box 621. For example, the information indicating the position of the first setting area may also include displaying the first setting area lighter or darker than other areas. Furthermore, the method of designating the first setting area is not limited to using the designation box 621. For example, the user may specify a part of the area that he or she wishes to specify as the set area, and the label information setting unit 310c may estimate the size and angle of the set area based on the specified part.

[0091] Thereafter, the process proceeds to step SC3 of the flowchart shown in Fig. 6. In step SC3, a set area determination flow is started in which the learning data setting unit 310 and the user determine a set area based on the first set area specified in step SC2, and the process proceeds to step SC4. In step SC4, an automatic estimation instruction is input by the user. Specifically, the learning data setting unit 310 causes the display device 4 to display a setting screen 600 as shown in Fig. 10. A set area determination window 630 is displayed on the setting screen 600 shown in Fig. 10.

[0092] The setting area determination window 630 includes an automatic estimation button 630a, an add button 630b, a delete button 630c, an undo button 630d, a delete all button 630e, and a start learning button 630f. The automatic estimation button 630a is a button for instructing automatic estimation of a candidate area. As shown in the setting image display area 601 in FIG. 10 , when the learning data setting unit 310 receives a user operation of the automatic estimation button 630a while displaying information (designation box 621) indicating the position of the first setting area superimposed on the setting image I3, the candidate area estimation unit 312 estimates a candidate area such that an area other than the first setting area is set as the candidate area. More specifically, the candidate area estimation unit 312 executes the pre-trained model to identify an area from the setting image I3 that has features similar to those of the first setting area specified in step SC2, and estimates the area as the candidate area. This is the processing of step SC5 shown in FIG. 6. The user's operation of the automatic estimation button 630a is an instruction to execute estimation of the learning candidate area.

[0093] In step SC6, the setting screen display unit 311 displays a candidate box 622 enclosing the area automatically estimated as the candidate area, superimposed on the setting image I3 in the setting image display area 601. The line type of the candidate box 622 is different from the line type (solid line) of the specification box 621, specifically, a dashed line. The estimation result by the candidate area estimation unit 312 may differ from the user's intention. In this case, the user identifies the estimated candidate area and performs an editing operation. For this reason, the candidate box 622 is displayed in a manner that allows the candidate area estimated in step SC5 to be distinguished from the setting area specified in step SC2. As long as the specification box 621 and the candidate box 622 can be distinguished, for example, instead of changing the line type of the specification box 621 and the candidate box 622, they may be displayed in different colors.

[0094] After the second learning candidate area is automatically estimated, when a user operation of the automatic estimation button 630a is accepted, the candidate area estimation unit 312 estimates a learning candidate area such that an area other than the first learning candidate area and the second learning candidate area is set as the third learning candidate area. For example, after estimating the second learning candidate area, the candidate area estimation unit 312 estimates, based on the features of each pixel, a position corresponding to a class feature that is similar to the reference class feature within a certain range as the third learning candidate area. In this case, the candidate area estimation unit 312 can estimate the third learning candidate area using the class feature of the second learning candidate area as the reference class feature.

[0095] In step SC7 shown in FIG. 6, it is determined whether the user has issued an instruction to delete the estimated area. The user looks at the display in step SC6 and determines whether all of the candidate areas estimated by the candidate area estimation unit 312 are as intended by the user. If the position, size, and angle of the candidate area differ from the user's intention, the user performs an editing operation on the candidate box 622, and if there is an unnecessary estimated area, the candidate box 622 indicating that estimated area is deleted, thereby issuing an instruction to delete the estimated area. The learning data setting unit 310 determines whether this deletion instruction has been received. If there is a deletion instruction, the process proceeds to step SC8; if there is no deletion instruction, the process proceeds to step SC9.

[0096] In step SC8, the learning data setting unit 310 executes a process to delete the candidate area. The learning data setting unit 310 deletes the candidate box 622 and holds the candidate area for which the deletion instruction was received as a deletion set position. The deletion set position is information that is read in subsequent automatic estimation based on a re-estimation instruction, which will be described later, and the candidate area estimation unit 312 estimates the candidate area so that a position identical to the deletion set position is not set as a candidate area. In other words, the learning data setting unit 310 executes a process to remove a position identical to the deleted candidate area from the candidate areas that will be the result of subsequent automatic estimation.

[0097] In step SC9, it is determined whether a re-estimation instruction has been issued. The user looks at the display in step SC6 and determines whether there is an area in the setting image I3 that should be estimated as a candidate area and for which the candidate box 622 is not displayed. That is, the user determines whether there is an area in the setting image I3 that should be added as a setting area. When it is determined that there is an area that should be added, the user can instruct automatic estimation again, and the learning data setting unit 310 determines whether this instruction has been accepted. If there is a re-estimation instruction, the process proceeds to step SC5 to perform automatic estimation again. If there is no re-estimation instruction, the process proceeds to step SC10 to determine the candidate area at this point as the setting area, and the setting area determination flow ends.

[0098] The processing of step SC5, which is executed when it is determined in step SC9 that a re-estimation instruction has been issued, will be described below. For ease of explanation, the candidate area estimated by the first processing of step SC5 will be referred to as the first candidate area, the candidate box 622 indicating the first candidate area will be referred to as first candidate box 622a, the candidate area estimated by the second processing of step SC5, which is executed in response to a re-estimation instruction, will be referred to as the second candidate area, and the second candidate box 622 indicating the second candidate area will be referred to as 622b.

[0099] First, when the determination in step SC9 is made, the setting screen display unit 311 is in the state of the display in step SC6. That is, the setting screen display unit 311 accepts a re-estimation instruction while displaying a first candidate box 622a indicating the position of the first candidate area estimated by the candidate area estimation unit 312, superimposed on the setting image I3. When accepting the re-estimation instruction, the setting screen display unit 311 may change the appearance of the first candidate box 622a. This is because the user can perform an editing operation on the first candidate box 622a before issuing an instruction for re-estimation, and therefore accepting the re-estimation instruction can be considered as confirmation of the position of the first candidate area indicated in the first candidate box 622a by the user. If the appearance of the changed first candidate box 622a is the same as or similar to the appearance of the specified box 621, changing the appearance of the first box 622a makes it easier for the user to distinguish between an area designated or confirmed by the user and a candidate area estimated by the candidate area estimation unit 312 but not confirmed by the user. Furthermore, by making the aspect of the changed first box 622a different from the aspect of the specified box 621, it becomes easier for the user to distinguish whether or not the candidate area is one estimated by the candidate area estimation unit 312.

[0100] Furthermore, before accepting a re-estimation instruction, it is preferable to prompt the user to perform an editing operation on the first candidate box 622a. The first candidate area estimated by the candidate area estimation unit 312 may be incorrect, and the user can correct the position and size of the candidate area by operating a mouse or the like to move the first candidate box 622a or change the shape, size, or angle of the first box 622a. If an instruction to estimate a second candidate area is subsequently issued, this means that the first candidate area is correct, and by reflecting this correction and estimating the second candidate area, the accuracy of the estimation result of the second candidate area can be improved.

[0101] When step SC5 is executed as a result of the determination in step SC9, candidate area estimation unit 312 estimates a second candidate area so that an area other than the area between the first set area and the first candidate area is set as the second candidate area. At this time, if the processing of step SC5 by candidate area estimation unit 312 is the same as the processing of step SC5 the first time, i.e., the automatic estimation processing that estimates the first candidate area, areas other than the first candidate area will not be estimated as candidate areas. For this reason, the processing of step SC5 the second time is different from that of step SC5 the first time.

[0102] FIG. 11 is a conceptual diagram illustrating the difference between the first processing of step SC5, ie, the estimation of the first candidate region, and the second processing of step SC5, ie, the estimation of the second candidate region.

[0103] The estimation of the first candidate area will now be described. The candidate area estimation unit 312 uses the feature amount of the first set area, i.e., the area containing object W1 specified by the specification box 621, as a reference feature amount, and estimates an area having a feature amount similar to the reference feature amount as the first candidate area. If the feature amount is determined by two axes, the shape and color of the object, in estimating the first candidate area, an area containing object W2 with a shape similar to the shape of the object is identified as an area from which feature amounts similar to those of the first set area can be obtained. In this way, the candidate area estimation unit 312 estimates the area containing object W2 as the first candidate area, and displays the first candidate box 622a.

[0104] The estimation of the second candidate area will now be described. In this embodiment, the candidate area estimation unit 312 combines two methods to estimate an area different from the first set area and the first candidate area as the second candidate area. First, the candidate area estimation unit 312 uses the feature values ​​of the first set area, i.e., the area containing the object W1 specified by the designation box 621, as the reference feature values, and expands the similarity range compared to when estimating the first candidate area to identify an area in the set image I3 that has feature values ​​similar to the reference feature values. Figure 11 shows a state in which, as a result of expanding the similarity range, an area containing an object W4 with a similar color to the object W1 has been identified as an area that has feature values ​​similar to those of the first set area. Next, the candidate area estimation unit 312 uses the feature values ​​of the first candidate box indicated by the first candidate box 622a as the reference feature values ​​to identify an area in the set image I3 that has feature values ​​similar to the reference feature values. In the estimation of the second candidate area in this embodiment, the feature values ​​are determined to be similar if the colors of the objects are similar. Therefore, an area including an object W3 with a similar color to the object W2 in the first candidate area is identified as an area with feature values ​​similar to those of the first candidate area. In this manner, an area different from the first set area and the first candidate area is estimated as the second candidate area. In this embodiment, the range of similarity is expanded, and the feature values ​​of the first candidate area estimated in step SC5 are used as reference feature values ​​to estimate the second candidate area. However, the second candidate area may be estimated using either one method or the other method. Furthermore, in this embodiment, when the feature values ​​of the first candidate area are used as reference feature values ​​to estimate the second candidate area, the similarity range is expanded from that used to estimate the first candidate area. However, the similarity range may be the same as that used to estimate the first candidate area. For example, an area including an object with a similar shape to the object W2 located in the first candidate area may be estimated as the second candidate area.

[0105] After the second processing of step SC5, the estimation result by candidate area estimation unit 312 is displayed in step SC5, just as after the first processing of step SC5. The user can look at the display in step SC6 and again determine whether there is an area that should be added as a setting area to setting image I3, and can give an estimation instruction as many times as necessary until they determine that there is no area that should be added. In this way, the user can obtain an estimation result by candidate area estimation unit 312 that has increased the number of candidate areas, simply by giving an estimation instruction.

[0106] When the process proceeds to step SC10 and the set area determination flow is completed, the training data generation unit 310b generates training data D1 based on the image data of the set image I3, the set area set as label information, and the candidate area corresponding to the candidate box 622 displayed when the set area determination flow was completed. The candidate area corresponding to the candidate box 622 displayed when the set area determination flow was completed is considered to have been confirmed by the user. That is, at the time of step SC10, the area identified by the designation box 621 and the candidate box 622 displayed superimposed on the set image I3 is treated as the set area specified by the user in the label information setting unit 310c, and the training data generation unit 310b generates training data D1 based on the set area and the image data of the set image I3. More specifically, a feature F2 indicating the target object is identified from the set image I3 based on the set area, and the determination unit is updated accordingly.

[0107] In the flowchart of Fig. 6, steps related to user instructions include step SC7 for determining whether a deletion instruction is given and step SC9 for determining whether a re-estimation instruction is given, but the user can give various instructions after step SC6. This will be explained using Fig. 10.

[0108] As described above, the automatic estimation button 630a is a button for instructing automatic estimation of a candidate area. The user can instruct re-estimation by operating the automatic estimation button 630a.

[0109] The Add button 630b is a button operated by the user when manually adding a setting region. When the user adds a second setting region from a state in which a first setting region is specified and a designation box 621 is displayed, a first candidate region is estimated, and a first candidate box 622a is displayed, as shown in FIG. 10 , the learning data setting unit 310, upon receiving a user operation of the Add button 630b, displays a designation box 621 separate from the designation box 621 corresponding to the first setting region at an arbitrary position in the setting image display area 601. The user can move the displayed designation box 621 by operating a mouse or the like, or change the shape, size, or angle of the designation box 621, and arrange it so that it surrounds the second setting region. This allows the second setting region to be manually added.

[0110] The delete button 630c is a button that the user operates when deleting a selected candidate box 622. For example, when the learning data setting unit 310 detects that the user has operated the delete button 630c while a candidate box 622 is selected, the learning data setting unit 310 determines in step SC7 that a deletion instruction for a candidate area has been issued. The learning data setting unit 310 then deletes the candidate box 622 and executes a process of removing the area surrounded by the deleted candidate box 622 from the candidate area as an estimation result of subsequent automatic estimation. When multiple candidate areas are estimated in step SC5 and multiple candidate boxes 622 are displayed in step SC6, any number of the candidate boxes 622 can be deleted.

[0111] The Undo button 630d is a button that is operated to undo an operation performed in the setting region determination window 630. When the learning data setting unit 310 accepts a user operation of the Undo button 630d, the learning data setting unit 310 cancels the immediately previous operation, thereby returning to the state before the immediately previous operation. For example, if the Undo button 630d is operated immediately after the designation box 621 is added, the designation box 621 is erased. Also, if the Undo button 630d is operated immediately after the candidate box 622 is deleted, the candidate box 622 is displayed. Also, if the Undo button 630d is operated immediately after an editing operation of the candidate box 622 is performed, the state returns to the state immediately before the editing operation.

[0112] The all delete button 630e is a button for simultaneously deleting all of the candidate boxes 622 shown in Fig. 10. When the learning data setting unit 310 receives a user operation of the all delete button 630e, it deletes all of the candidate boxes 622 and executes a process of removing each candidate area surrounded by each candidate box 622 from the candidate areas.

[0113] The process for estimating the candidate area is not particularly limited, but may be the following process. For example, the candidate area estimation unit 312 acquires image data of the setting image I3 and information indicating the position of the first setting area that is already superimposed and displayed on the setting image display area 601. The information indicating the position of the first setting area may be manually specified by the user or may be automatically estimated.

[0114] Next, candidate area estimation unit 312 estimates candidate areas. At this time, candidate areas can be estimated using a predetermined method, and if multiple candidate areas are estimated, candidate boxes 622 indicating the multiple candidate areas are displayed superimposed on the same setting image I3.

[0115] The candidate area estimation unit 312 also calculates feature amounts corresponding to "information indicating the position of the first set candidate area that is already superimposed" and feature amounts of each area obtained from the image data of the setting image I3. The candidate area estimation unit 312 then calculates the similarity between the feature amounts of each area obtained from the image data of the setting image I3, using one of the feature amounts corresponding to "information indicating the position of the first set candidate area that is already superimposed" as a reference. The candidate area estimation unit 312 then determines a candidate area to be presented to the user based on the calculated similarity. At this time, if there is a history of a deletion instruction from the user, the candidate area is determined so that the location of the deletion instruction is not included in the candidate area to be presented to the user.

[0116] The candidate area estimation unit 312 can also store predetermined reference features. These reference features are features that are used when a general object is present; in other words, they are determined not by the user but by the business that provides the image inspection device S, and are set at the time of shipment. The candidate area estimation unit 312 identifies similar coordinates point by point based on the reference features, and sets the area containing these coordinates as a learning candidate area.

[0117] In this process, it is assumed that the predetermined reference feature amounts are in units of points, but reference feature amounts that take into account the size and tilt angle of the image area of ​​a general object may be held, and image areas that correspond to feature amounts with a high degree of similarity to the set image data may be set as learning candidate areas. In this case, it is possible to estimate learning candidate areas that take into account the size and angle of the image area while suppressing an increase in the load of the comparison process between the reference feature amounts and the set image data.

[0118] Alternatively, feature quantities related to size and tilt angle may be stored in advance, and the identified coordinate group may be set as first-stage candidates, and then the candidate learning areas may be further narrowed down based on the similarity between the feature quantities calculated for each coordinate and the size and tilt angle feature quantities. In this case, the feature quantities calculated for each coordinate indicate the size and tilt angle of the image area when the coordinates are included in the image area of ​​a general object.

[0119] The first set area is also narrowed down based on the "information indicating the position of the first set area that is already superimposed." The candidate area estimation unit 312 extracts features from the "information indicating the position of the first set area that is already superimposed," identifies similar coordinates point by point, and designates the area containing those coordinates as the candidate area. Comparing point by point reduces the load of the comparison process, and because the feature of the set image data is compared with the instructed data ("information indicating the position of the first set area that is already superimposed"), there is a reduced risk that an area similar to the target object to be detected will not be estimated as a candidate area, or that an area containing an object other than the target object to be detected will be estimated as a candidate area.

[0120] At this time, it is also possible to compare the coordinates based on size feature values ​​and angle feature values. That is, the candidate areas can be narrowed down based on the size and angle of the "information indicating the position of the first setting area that is already superimposed and displayed." At this time, the values ​​indicating the size and angle of the "information indicating the position of the first setting area that is already superimposed and displayed" themselves may be used to narrow down the candidate areas, or feature values ​​indicating these may be extracted, and the similarity with feature values ​​calculated for each coordinate in the coordinate group in a similar manner may be used to narrow down the candidate areas.

[0121] As shown in FIG. 10, the setting area determination window 630 is provided with an operation explanation section 631, in which an operation method is displayed as a moving image.

[0122] When the classifier update unit 315 detects that the learning start button 630f has been operated, it updates the pre-trained model based on the generated learning data D1 and the feature F2, which is the feature corresponding to the set area. An example of an image used during learning is shown in FIG. 12. In FIG. 12, a test run window 640 is displayed, and the setting image I3 displayed in the setting image display area 601 allows the user to set whether or not to update the pre-trained model. The test run window 640 is provided with an additional learning button 640a, which allows the user to determine whether or not to perform additional learning using this setting image. In this example, a designation box 621 and six candidate boxes 622 are displayed, and a total of seven set areas are set.

[0123] During training, the classifier update unit 315 trains the trained model to classify the set area into an object class and the area outside the set area into a background class. Furthermore, the classifier update unit 315 can learn the boundary separating the object class from the background class. In this case, the trained model can detect the object in the inference image data image based on the feature amount and the boundary for each pixel.

[0124] (When operating image inspection equipment) Fig. 13 is a flowchart showing the procedure during operation of the image inspection device S. The image inspection device S can be switched between a setting mode in which settings are made as shown in Figs. 4 to 6, and an operation mode shown in Fig. 13, but this mode switching does not have to be clear, and continuous operation is also possible, for example, by temporarily switching to the setting mode to change settings while operating in the operation mode, and then returning to the operation mode.

[0125] In step SD1 after the start, the imaging unit 1 captures an image of the workpiece W and acquires image data of the workpiece W. In step SD2, inspection based on the set inspection tool is started for the image of the workpiece W acquired in step SD1. First, in step SD3, the range to be inspected is cut out from the image of the workpiece W acquired in step SD1 based on the inspection range set in the inspection window setting unit.

[0126] In step SD4, the feature extraction unit extracts features from the inspection range extracted in step SD3. In step SD5, the features extracted in step SD4 are judged by a classifier. As mentioned above, in steps SD4 and SD5, the features vary depending on the inspection tool or the classifier used.

[0127] In step SD6, the inspection results using the inspection tools are created and compared with the inspection conditions. In step SD7, it is determined whether there are any unprocessed configured inspection tools. If there are any unprocessed configured inspection tools, the process proceeds to step SD3. If there are no unprocessed configured inspection tools, the process proceeds to step SD8. In step SD8, the inspection based on the configured inspection tools is terminated.

[0128] In step SD9, the inspection result output unit outputs the inspection results. If there is one inspection tool configured, the inspection result for that configured inspection tool is output as is, but if multiple configured inspection tools are combined, further inspection results are generated based on the inspection results of each inspection tool and output.

[0129] (Learning count function) Even when multiple candidate regions exist in the image of the input image data, the image inspection device S can update the pre-trained model so that the training data generation unit 310b classifies the candidate regions into object classes based on the training data. In other words, this is particularly effective when there are multiple objects in the imaging field of view, and is a suitable configuration for a training counting tool.

[0130] When a tool setting operation is performed by the user, the control unit 21 generates a tool setting screen 700 as shown in FIG. 14 and displays it on the display device 4. The tool setting screen 700 includes an image display area 701 in which a workpiece image captured by the imaging unit 1 is displayed, and a setting area 702 in which various setting operations can be performed. The setting area 702 includes an add tool button 702a for adding a tool to be used for judgment, an edit button 702b for editing a tool, a copy button 702c for copying a tool, and a delete button 702d for deleting a tool. Each of the buttons 702a, 702b, 702c, and 702d is configured as an image displayed on the display device 4, and therefore can be touch-operated using the touch panel 4a. However, this is not a limitation, and the buttons may be configured as physically operable buttons. The same applies to buttons hereinafter.

[0131] When the control unit 21 detects that the tool addition button 702a has been operated, it generates a tool selection window 710 shown in FIG. 15 and displays it on the display device 4. The tool selection window 710 is provided with a tool display area 711 in which basic tools can be selected, and an explanation display area 712 in which an explanation of the selected tool is displayed. The tool display area 711 displays buttons corresponding to multiple tools, such as a learning tool, learning search, learning count, learning OCR (character recognition), color area, and position correction, and is capable of accepting selection operations by the user. An explanation of the tool selected in the tool display area 711 is displayed in the explanation display area 712. The explanation of the tool includes an explanation of the tool's function, an explanation of how to use the tool, etc. In this example, a case is shown in which the learning count tool has been selected.

[0132] When the control unit 21 detects that the learning count tool has been selected, it displays a learning setting button 720 for detailed tool settings and a threshold setting button 721 for setting a threshold value in the setting area 702, as shown in FIG. 16. When the control unit 21 detects that the learning setting button 720 has been operated, it generates a box setting area 730 for setting a first box 701a surrounding a reference master workpiece, as shown in FIG. 17, and displays the box setting area 730 on the display device 4. The box setting area 730 includes a shape selection area 731 for selecting whether the shape of the first box 701a is rectangular or circular, and a range setting area 732 for selecting whether the search range is the entire range or a specified range. The first box 701a can be created by, for example, a touch operation or the like on the image displayed in the image display area 701. For example, the first box 701a can be created by dragging from the upper left to the lower right of the area to be surrounded. The size and aspect ratio of the first box 701a can be freely changed. When the first box 701a is formed, the control unit 21 identifies the image area surrounded by the first box 701a and accepts the designation of the image area.

[0133] Once the image area is specified using the first box 701a, as shown in FIG. 18, the control unit 21 displays the specified image area as a master and generates a count object setting area 740 for setting a count object, which is displayed on the display device 4. The count object setting area 740 includes an automatic detection button 740a for automatically detecting a count object, an add button 740b for adding a count object, and a delete button 740c for deleting a count object. The count object setting area 740 also includes a selection area 741 for selecting whether to automatically learn the rotated state of the workpiece, a count number display area 742 for displaying the count number, and a start learning button 743. Automatically learning the rotated state of the workpiece is called automatic rotation learning. To perform automatic rotation learning, select "enable" in the selection area 741. To disable automatic rotation learning, select "disable" in the selection area 741. Automatic rotation learning will be described later. In FIG. 18, only the master is being counted by the count tool, so the count number display area 742 displays "1." This makes it easy to see how many annotations a user has made.

[0134] When the control unit 21 detects that the automatic detection button 740a has been operated, it automatically detects whether or not a workpiece identical to or similar to the master workpiece is included in the image displayed in the image display area 701. As shown in FIG. 19, when a workpiece identical to or similar to the master workpiece is detected, the control unit 21 superimposes and displays a second box 701b surrounding the workpiece. The example shown in FIG. 19 shows a case where nine workpieces other than the master workpiece have been detected, and therefore, nine second boxes 701b are displayed. Furthermore, since a total of 10 workpieces including the master workpiece are counted, "10" is displayed in the count number display area 742. Note that the first box 701a and the second box 701b indicate whether or not a workpiece is a detected workpiece, and may be displayed in a form other than a box. For example, the image area including the workpiece may be colored in a different color from the other areas, or may be displayed using letters, symbols, or the like.

[0135] 20 is a diagram showing an example of an image used in automatic rotation learning. As shown in this figure, the control unit 21 automatically generates multiple rotated images, and the generated images can be used as learning images. This makes it possible to automatically detect a workpiece even if an image of the workpiece whose posture has changed is input during operation of the image inspection device S.

[0136] Before executing learning, the control unit 21 generates a warning display window 750 as shown in Fig. 21 and displays it on the display device 4. The warning display window 750 is a screen for final confirmation immediately before learning, and by displaying this screen, it is possible to prevent a decrease in learning accuracy when learning is performed taking into account unset items.

[0137] The warning display window 750 displays messages and illustrations to prompt the user to check, for example, whether there are any unspecified objects in the tool or whether there are any workpieces that have not been enclosed. This is because workpieces that are not enclosed will be registered as defective workpieces. It is also possible to issue a warning if an object that is clearly similar to a master workpiece has been forgotten to be enclosed. During automatic detection, the control unit 21 calculates the similarity with objects that have already been enclosed, and for objects that are not enclosed, determines whether or not they should be enclosed based on the calculated similarity. The above warning can be issued when an object with a high similarity remains unenclosed by a box. The warning display window 750 also displays a count number display area 751.

[0138] After checking the warning display window 750 in Fig. 21, if there are no problems, the user operates the learning start button 743 shown in Fig. 18. When the control unit 21 detects that the learning start button 743 has been operated, it executes additional learning of the model.

[0139] The control unit 21 can generate a setting window 760 (shown in FIG. 22) for setting the upper and lower limits of the count number and display it on the display device 4. The upper and lower limits of the count number are automatically set according to the number of master images. In the setting window 760, the range of the count number can be set, and the upper limit (HI) and lower limit (LO) of the count number can be changed individually.

[0140] When the control unit 21 receives a test run instruction from the user, it executes a process in which the model after additional learning detects an object in an image. Figure 23 shows a test screen that the control unit 21 generates during the test run and displays on the display device 4, and the image input during the test run is displayed in the image display area 701. A third box 701c surrounding the workpiece detected by the model is superimposed on the image.

[0141] The control unit 21 generates a test operation / additional learning window 770 and displays it on the display device 4. The test operation / additional learning window 770 is provided with an additional learning button 771 for additionally learning the image displayed in the image display area 701, and a learning count display area 772. The learning count display area 772 displays the number of workpieces detected by the model, i.e., the count number. In this example, since 10 workpieces were detected, "10" is displayed.

[0142] The control unit 21 generates an extended function setting window 780 (shown in FIG. 24) and displays it on the display device 4. The setting window 780 can accept requests to change the learning image to another image, such as when the counting target is not included in the master image, to change the detection threshold value for the counting target, and to change the position correction setting.

[0143] FIG. 25 shows an operation screen 800 that is displayed when the image inspection device S is in operation; the control unit 21 generates the operation screen 800 and displays it on the display device 4. The operation screen 800 has an image display area 801 in which the image input during operation is displayed, and a result display area 802. The result display area 802 displays the judgment results of each tool; in the example shown in FIG. 25, the judgment result by the learning count tool is displayed along with the count number. Furthermore, in the image input during operation, the workpiece is tilted, but because the model is trained using a learning image such as that shown in FIG. 20, tilted workpieces can also be detected stably.

[0144] FIG. 26 shows an example of the display of the judgment results using the sum tool. The sum tool counts each location where the tool is set and makes a final judgment using both the number of each location and the total. FIG. 26 shows a case where a learning count tool is set in each of the top, middle, and bottom rows, and the learning count tool set in the middle row should have detected four workpieces, but only three workpieces were detected. Since the result display area 802 displays pass / fail for each tool, the user can easily confirm which tool location is defective. In short, multiple tools can be set within the imaging field of view, and a sum tool that totals the counts of those tools can also be set. Therefore, the results for each tool setting position allow the user to confirm which areas are free of target objects. In addition to setting tools in the top, middle, and bottom rows, setting tools in the left, center, and right rows is also possible.

[0145] Figure 27 shows a case where a learning count tool is set in the middle row. As shown in this figure, the user creates a fourth box 701d to surround the four works in the middle row as the search range. The fourth box 701d can be created in the same way as the first box 701a.

[0146] The workpiece surrounded by the first box 701a in the fourth box 701d becomes the master workpiece, and as shown in FIG. 28, the control unit 21 displays the specified image area as the master. Then, as shown in FIG. 29, the count target is automatically detected in the fourth box 701d. For example, as shown in FIG. 30, if the search range is later changed and the annotated object falls outside the search range, the control unit 21 displays a warning with diagonal lines. The warning display is not limited to diagonal lines, but may also be displayed in a color such as red, or may be displayed with letters or symbols. In other words, since the user can be notified when annotation is performed outside the tool, it is possible to prevent objects that are inappropriate for learning data from being set when the objects to be detected change depending on the tool.

[0147] FIG. 31 shows a specific example of a display in which learning count tools are set in the top, middle, and bottom rows. "01" is the learning count tool set in the top row, "02" is the learning count tool set in the middle row, and "03" is the learning count tool set in the bottom row. To add a learning count tool, simply operate the count tool add button 702e. When the control unit 21 detects that the count tool add button 702e has been operated, it adds a new learning count tool.

[0148] To set a summing tool that sums the determination results of these three learning count tools, simply operate the Add Sum Tool button 702f. When the control unit 21 detects that the Add Sum Tool button 702f has been operated, it displays a check box 702h corresponding to the learning count tool and accepts a selection operation by the user, as shown in FIG. 32. The learning count tool checked in the check box 702h is subject to summing processing by the summing tool. In this example, three learning count tools are subject to summing processing by the summing tool. When using the summing tool, it is also possible to set the upper and lower limits of the count number using a setting window 760, as shown in FIG. 22. After setting the summing tool, the summing tool is displayed together with the learning count tool, as shown in FIG. 33, allowing the user to confirm the setting status of the summing tool and the learning count tool.

[0149] 34 is a diagram showing an example of an operation screen 800 that is displayed when the total tool is in operation. As shown in this figure, the judgment results of the three learning count tools are displayed individually, and the judgment result of the total tool is also displayed. For example, when the user selects the upper learning count tool, as shown in FIG. 35, the control unit 21 causes the display device 4 to display a fourth box 701d that indicates the search range of the upper learning count tool, so that the user can easily understand where and how the learning count tool is set on the image.

[0150] When multiple learning count tools are set, the control unit 21 has a function to eliminate duplicate detections between tools. For example, as shown in the image display area 901 in FIG. 36, when a model is trained using a large-sized workpiece on the left and a small-sized workpiece on the right, the intermediate-sized workpiece in the center is detected by both the tool that detects large works and the tool that detects small works. However, in this embodiment, as shown in the result display area 902, the tool that detects large works (01 learning count) is used to detect the intermediate-sized workpiece. Therefore, in this example, the tool that detects small works (02 learning count) detects only one workpiece. This function is particularly effective, for example, when sorting workpieces with different postures or sizes, or when learning workpieces with different postures or sizes and determining the total. Note that the tool that detects small works may be used to detect intermediate-sized workpieces. In this case, the tool that detects large works detects only one workpiece.

[0151] For example, if one tool cannot correctly detect all objects, two types of tools may be set in the same area. In this case, if an object is detected by both tools, it will be counted twice. However, in this example, duplicates can be deleted based on the detection location, so all objects can be counted by multiple tools without duplication in the numbers.

[0152] (Disconnection detection function) The image inspection device S has a function to detect missing workpieces, for example, when inspecting multiple lined up workpieces and some of the workpieces are missing. Figure 37 shows the case where an image capturing an arrangement of 10 workpieces vertically and 10 workpieces horizontally is input. In this case, the count is 100, which is no problem.

[0153] On the other hand, Fig. 38 shows a case where an image capturing six workpieces missing out of 100 workpieces is input. In this case, the count becomes 94, and the control unit 21 detects that workpieces are missing, determines that they are defective, and displays this in the result display area 802. If some workpieces are missing, the numbers of the missing locations are displayed in the image display area 801. That is, since there are 10 workpieces lined up vertically and horizontally, the control unit 21 assigns numbers 1 to 100 in order from the top left to the bottom right, for example, and displays the numbers of the locations determined to be missing in the image display area 801. This makes it easier for the user to determine which workpieces are missing.

[0154] To enable the missing element detection function, enablement is performed in a missing element detection setting area 723 shown in Fig. 39. The missing element detection setting area 723 is classified as an extended function. When the control unit 21 detects that the user has operated the extended function tab, it causes the missing element detection setting area 723 to be displayed on the display device 4.

[0155] When setting the missing work detection function, the assignment of numbers to each of multiple workpieces is also set. Figure 40 shows the number assignment setting window 724 displayed. The number assignment setting window 724 is provided with a rule setting area 724a for setting assignment rules and a number display area 724b that shows the arrangement of the numbers to be assigned. The number assignment rule may be set in detail, such as vertically and horizontally, left and right, or up and down, or may be set simply by specifying a preferred direction, such as vertically or horizontally.

[0156] A specific method for detecting missing workpieces will be described with reference to FIG. 41. An image of five circular workpieces lined up is assumed as the master. The five workpieces are assigned numbers 1 to 5 according to the allocation rules described above. In the input image input during operation, the fourth workpiece is missing. The control unit 21 determines the correspondence between the master and the input image, and determines that the locations where there is no correspondence are locations where a workpiece is missing, and presents this to the user.

[0157] The above-described embodiments are merely examples in all respects and should not be construed as limiting. Furthermore, all modifications and variations within the scope of the claims are within the scope of the present invention. [Industrial Applicability]

[0158] As described above, the image inspection device according to the present disclosure can be used to inspect various types of workpieces. [Explanation of symbols]

[0159] 4 Display device 21 Control section 310b Learning data generation unit 310 Learning data setting unit 600 Settings screen 601 Setting image display area S Image inspection device

Claims

1. An image inspection device in which a pre-trained model is executed to detect an object in an image of input image data, a control unit that executes the pre-trained model and functions as a learning data setting unit that sets learning data for on-site learning of the pre-trained model, and a learning execution unit that updates the pre-trained model based on the learning data; The pre-trained model is a feature extraction unit that extracts, from the input image data, a feature that indicates a feature of the image data; a classification unit that detects the object in the image of the image data by outputting a class to which an image region of the image data belongs based on the feature extracted by the feature extraction unit, The learning data setting unit a setting screen display unit that displays a setting screen including a setting image display area on a display device; a candidate area estimation unit that executes a pre-trained model to estimate a candidate area of ​​the set image; a training data generation unit that generates the training data based on the setting image and the estimated candidate area; when a re-estimation instruction to execute estimation of the candidate area is received in a state in which the setting screen display unit displays the position of a first candidate area, which is the candidate area estimated by the candidate area estimation unit, superimposed on the setting image, the candidate area estimation unit estimates a second candidate area such that an area other than the first candidate area is set as the second candidate area; the learning data generation unit generates the learning data based on the setting image, the first candidate area, and the second candidate area; The learning execution unit updates the pre-trained model based on the learning data so as to classify the candidate region into the object class.

2. 2. The image inspection device according to claim 1, The setting screen display unit displays the second candidate area in a manner different from that of the first candidate area.

3. 3. The image inspection device according to claim 2, When the learning data setting unit receives the re-estimation instruction, the setting screen display unit changes the state of the first candidate area.

4. 2. The image inspection device according to claim 1, the learning data setting unit receives a user-specified area by a user's specification of the setting image data displayed in the setting image display area; The setting screen display unit displays the user-specified area and the candidate area in different formats.

5. 2. The image inspection device according to claim 1, the learning data setting unit accepts a deletion instruction for the candidate area displayed by the setting screen display unit, and holds the candidate area as a deletion setting position; The candidate area estimation unit estimates the candidate area so as not to set the same position as the deletion set position as the candidate area.

6. 2. The image inspection device according to claim 1, The candidate area estimation unit A region having a feature quantity similar to the reference feature quantity is estimated as a first candidate region; An image inspection device that uses a feature quantity corresponding to the first candidate area displayed when the re-estimation instruction is received as the reference feature quantity and estimates an area having a similar feature quantity as a second candidate area.

7. 2. The image inspection device according to claim 1, The candidate area estimation unit A region having a feature quantity similar to the reference feature quantity within a certain range is estimated as a first candidate region; An image inspection device that estimates, as a second candidate region, a region having a similar feature amount in a range expanded from when estimating the first candidate region for the reference feature amount.

8. 2. The image inspection device according to claim 1, The candidate area estimation unit A region having a feature quantity similar to the reference feature quantity within a certain range is estimated as a first candidate region; an image inspection device that uses a feature quantity corresponding to the first candidate area displayed when the re-estimation instruction is received as the reference feature quantity, and estimates, as a second candidate area, an area having a similar feature quantity in a range expanded relative to the reference feature quantity compared to when the first candidate area was estimated.

9. 2. The image inspection device according to claim 1, The learning execution unit updates the pre-trained model so as to classify the candidate region into the object class and to classify regions other than the candidate region into the background class.

10. 10. The image inspection device according to claim 9, the learning execution unit learns a boundary separating the object class and the background class; The pre-trained model detects an object in an image of the inference image data based on the feature and the boundary, an image inspection device.

Images