Image inspection device, and image inspection program

Abstract

To easily and appropriately provide information that becomes an object guide for a detection object expected to be stably detected in image inspection using a machine learning model.SOLUTION: An image inspection device having a machine learning model for generating a feature map on the basis of an inputted inspection object image to detect a detection object included in the inspection object image on the basis of feature map information includes: a learning unit for storing a machine learning model; and a control unit for calculating an object size of the detection object expected to be stably detected in an inspection object area on the basis of the size of the inspection object area being the whole or a portion of the inspection object image and the size of a feature map generated by the machine learning model on the basis of the inspection object area, and displays a guide object corresponding to the object size together with the inspection object area in a display unit.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a technique for detecting a detection target object included in an inspection target area based on feature information by a machine learning model having a machine learning model for extracting feature information.

Background Art

[0002] Conventionally, as an image inspection technique, a rule-based image inspection technique is known. In the rule-based image inspection technique, for example, using data for each pixel, a change in brightness from surrounding pixels is detected as a defect (for example, a scratch, dirt, foreign matter, rust). Therefore, while fine defects can be detected using an image with high pixel resolution, if all processing for the image is performed in units of one pixel, it takes an enormous amount of processing time, and noise may also affect the results.

[0003] Therefore, in defect detection, processing is performed using the average value of a small unit (segment) of several pixels.

[0004] If the segment size used for processing is set to be approximately the same size as the size of the defect to be detected, the defect can be stably detected. For example, if the segment size is too large, the level of detecting the defect to be detected may decrease, and there is a risk of overlooking it. On the other hand, if the segment size is too small, there is a risk of misdetecting fine noise components.

[0005] Therefore, the segment size needs to be appropriately set according to the size of the defect to be detected. Therefore, in an image inspection tool for performing defect detection, for example, as shown in Patent Document 1, the segment size is received from the user. In such an image inspection tool, the segment size was displayed on the inspection image as a reference size of the defect that can be detected as reference information for the user.

[0006] In recent years, as an image inspection technology, an image inspection technology using AI (Artificial Intelligence) (AI image inspection) has come to be used.

[0007] In AI image inspection, regarding what size of defects can be detected, it depends not only on the inspection image but also on the design of the machine learning model used as AI (for example, the number of convolutional layers, the size of the convolutional filter, the size of the stride, the pooling method), etc., and it is not necessary to directly accept the segment size from the user as in the rule-based method.

Prior Art Documents

Patent Documents

[0008]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0009] For example, in the rule-based image inspection technology, the user needs to specify the segment size, and the standard size of the detection target that can be detected according to this segment size is displayed. On the other hand, in AI image inspection, the user does not need to specify the segment size, and information serving as a standard regarding what size of detection target will be detected has not been provided to the user.

[0010] The present invention has been made in view of the above circumstances, and an object thereof is to provide a technology capable of easily and appropriately providing information serving as a standard for the object size of a detection target for which stable detection is expected in image inspection using a machine learning model.

Means for Solving the Problems

[0011] To achieve the above object, an image inspection apparatus according to one aspect has a machine learning model that generates a feature map based on an input inspection target image, and is an image inspection apparatus that detects a detection target object included in the inspection target image based on the feature map, including a storage unit that stores the machine learning model, the size of an inspection target region that is all or part of the inspection target image, and based on the size of the feature map generated by the machine learning model based on the inspection target region, calculates an object size of a detection target object for which stable detection is expected in the inspection target region, and a control unit that causes a reference object corresponding to the object size to be displayed on a display unit together with the inspection target region.

Advantages of the Invention

[0012] According to the present invention, information serving as a standard for the object size of a detection target object for which stable detection is expected in image inspection using a machine learning model can be easily and appropriately provided.

Brief Description of the Drawings

[0013]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

[0014] Embodiments will be described with reference to the drawings. Note that the embodiments described below do not limit the invention according to the claims, and not all of the elements and combinations thereof described in the embodiments are essential for the solution means of the invention.

[0015] FIG. 1 is an overall configuration diagram of an image inspection apparatus according to an embodiment.

[0016] The image inspection apparatus 1 is a device for performing an inspection process for inspecting whether a detection object exists in an inspection object based on an image obtained by imaging an inspection object such as various parts and products, and is used at a production site such as a factory. The inspection object in the image inspection apparatus 1 may be the entire inspection object, a part of the inspection object, or a plurality of parts of the inspection object. Also, the image may include a plurality of inspection objects.

[0017] The image inspection apparatus 1 includes a control unit 2, an imaging unit 3, a display device 4, and a personal computer 5. The personal computer 5 is not essential and may be omitted. Also, the personal computer 5 may be used instead of the display device 4. Further, the control unit 2 may be realized by the personal computer 5.

[0018] In the example of FIG. 1, the image inspection apparatus 1 separates the control unit 2, the imaging unit 3, the display device 4, and the personal computer 5, but a plurality of arbitrary configurations may be combined and integrated. For example, the control unit 2 and the imaging unit 3 may be integrated to form a so-called smart camera, and the control unit 2 and the display device 4 may be integrated. Further, the control unit 2 may be divided into a plurality of units, and a part of the divided units may be incorporated into other configurations such as the imaging unit 3 or the display device 4. Further, the imaging unit 3 may be divided into a plurality of units, and a part of the divided units may be incorporated into other configurations.

[0019] FIG. 2 is a hardware configuration diagram of an image inspection apparatus according to an embodiment.

[0020] (Configuration of Imaging Unit 3) The imaging unit 3 includes a camera module 14 and an illumination module 15. The camera module 14 includes an AF (Auto Focus) motor 141 that drives the imaging optical system and an imaging substrate 142. The AF motor 141 automatically performs focus adjustment by driving the lens of the imaging optical system. The AF motor 141 has its focus adjustment controlled by a technique such as well-known contrast autofocus.

[0021] The imaging substrate 142 includes a CMOS (Complementary Metal-Oxide-Semiconductor) sensor 143 as a light-receiving element that receives light incident from the imaging optical system, an FPGA (Field Programmable Gate Array) 144, and a DSP (Digital Signal Processor) 145. The CMOS sensor 143 is an imaging sensor configured to be able to acquire a color image. The CMOS sensor 143 is configured to be able to output a live image, that is, the currently captured image, at any time with a short frame rate. The CMOS sensor 143 inputs an image (image signal) to the FPGA 144 and the DSP 145. Instead of the CMOS sensor 143, a light-receiving element such as a CCD (Charged-Coupled Device) sensor may be used. The FPGA 144 and the DSP 145 control the camera module 14 and perform predetermined image processing on the image from the CMOS sensor 143. The CMOS sensor 143 starts imaging in accordance with an imaging control signal from a control unit 13A (to be described later) of the control unit 2, and adjusts the exposure time to an arbitrary time to perform imaging. That is, the imaging unit 3 images within the field of view of the CMOS sensor 143 in accordance with the imaging control signal output from the control unit 13A. Therefore, if there is an inspection object within the field of view, the CMOS sensor 143 images the inspection object, and if there is an object other than the inspection object within the field of view, the CMOS sensor 143 also images the object other than the inspection object. For example, at the time of setting the image inspection apparatus 1, the imaging unit 3 can image a plurality of images for associating (assigning) the attributes of good products or defective products by the user. Also, during the operation of the image inspection apparatus 1, the imaging unit 3 can image an inspection object for determining a good product or a defective product.

[0022] The illumination module 15 includes an LED (light emitting diode) 151 as a light emitter that illuminates an imaging area including an object to be inspected, and an LED driver 152 that controls the LED 151. The light emission timing, light emission time, and light emission amount of the LED 151 can be arbitrarily set by the LED driver 152. In the example of FIG. 2, the LED 151 is provided integrally with the imaging unit 3, but it may be provided as an external illumination unit separate from the imaging unit 3. Although not shown, the illumination module 15 includes a reflector that reflects the light emitted from the LED 151, a lens through which the light emitted from the LED 151 passes, and the like. In the illumination module 15, the irradiation range of the LED 151 is set so that the light emitted from the LED 151 irradiates the object to be inspected and the peripheral area of the object to be inspected. Instead of the LED 151, other light emitters may be used. The LED driver 152 switches the on / off of the LED 151 and adjusts the lighting time in response to an illumination control signal from the control unit 13A of the control unit 2, and also adjusts the light amount and the like of the LED 151.

[0023] (Configuration of Control Unit 2) The control unit 2 includes a main board 13, a connector board 16, a communication board 17, a power supply board 18, and a storage device 19. The main board 13 is mounted with an FPGA 131 and a DSP 132 that constitute the control unit 13A, and a memory 133 as an example of a storage unit. Note that the FPGA 131, the DSP 132, and the memory 133 may be configured as an integrated unit.

[0024] The control unit 13A comprehensively controls the operations of each board and module connected to the main board 13. For example, the control unit 13A outputs an illumination control signal for controlling the lighting / extinguishing of the LED 151 to the LED driver 152 of the illumination module 15. Further, the control unit 13A outputs an imaging control signal for controlling the CMOS sensor 143 to the imaging board 142 of the camera module 14. With this imaging control signal, the control unit 13A can cause the imaging unit 3 to capture various images. When the imaging by the imaging unit 3 is completed, the control unit 13A acquires the image (image signal) output from the imaging unit 3. In the present embodiment, the image signal is input to the FPGA 131, processed by the FPGA 131 and the DSP 132, and stored in the memory 133. Also, the control unit 13A outputs an AF control signal for controlling the AF motor driver 181 (described later) of the power supply board 18.

[0025] The connector board 16 has a power supply interface 161 and receives power supply from the outside via a power supply connector (not shown) provided on the power supply interface 161.

[0026] The power supply board 18 distributes the power received by the connector board 16 to each board and module. Specifically, the power supply board 18 distributes power to the illumination module 15, the camera module 14, the main board 13, and the communication board 17. The power supply board 18 includes an AF motor driver 181. The AF motor driver 181 supplies driving power to the AF motor 141 of the camera module 14 to realize the autofocus function. The AF motor driver 181 adjusts the power supplied to the AF motor 141 according to the AF control signal from the control unit 13A.

[0027] The communication board 17 outputs the image data, user interface, etc. about the inspection object output from the control unit 13A to the display device 4, the personal computer 5, an external control device (not shown), etc.

[0028] In addition, the communication board 17 receives various operations of the user input from the touch panel 41 of the display device 4, the keyboard 51 of the personal computer 5, and the like. The communication form between the communication board 17 and other devices may be wired or wireless, and any communication form can be realized by a well-known communication module.

[0029] The storage device 19 is a storage device such as a hard disk drive. The storage device 19 stores a program file 100 including an image inspection program for enabling various processes described later to be executed by hardware, a setting file, etc. (software), a learning image, a verification image, and the like. For example, the program file 100 and the setting file may be stored in a storage medium 90 such as an optical disk, and the program file 100 and the setting file stored in the storage medium 90 may be installed in the control unit 2.

[0030] The display device 4 has a touch panel 41. The touch panel 41 of the display device 4 displays an image transmitted from the control unit 2, a screen of a user interface, and the like. Further, the touch panel 41 is a well-known touch-type operation panel equipped with, for example, a pressure sensor, and detects a touch operation by the user and outputs it to the communication board 17.

[0031] The personal computer 5 has a keyboard 51, a display panel made of, for example, a liquid crystal panel, a mouse, and the like. The personal computer 5 receives various operations of the user by operation devices such as the keyboard 51 and the mouse and transmits them to the control unit 2. Further, the personal computer 5 displays an image transmitted from the control unit 2, a screen of a user interface, and the like.

[0032] Next, the functional configuration of the image inspection apparatus 1 will be described.

[0033] FIG. 3 is a functional configuration diagram of an image inspection apparatus according to an embodiment.

[0034] The image inspection apparatus 1 (mainly, the control unit 2) includes an image reception unit 21, a resizing processing unit 22, a learning unit 23, an inspection verification unit 25, a display processing unit 26, and an instruction reception unit 27. Each of these functional units (21 to 27) is configured, for example, by the control unit 13A executing a program file 100. Note that at least one of the functional units may be configured only by hardware.

[0035] The image reception unit 21 receives a learning image for learning the machine learning models 24 (24A to 24C) of the learning unit 23 and a plurality of verification images in which the correct answers for evaluating the accuracy of the machine learning models 24 are associated. In this embodiment, in each machine learning model 24, the same learning image and verification image may be used. As a method for receiving images such as learning images and verification images, the name of an image group that is grouped and stored in advance in a storage device 19 or the like, or the designation of the image name of each image may be received. Further, the image reception unit 21 may receive a designation of a learning image for an image. Note that a learning image may be included in the verification images. Further, the image reception unit 21 receives an image (image to be inspected) for inspection during operation using the learned machine learning model 24 by the inspection verification unit 25. Note that the learning image and the verification image are also examples of the image to be inspected. Further, when the inspection target area reception unit included in the instruction reception unit 27 receives a designation of an inspection area (ROI: Region of Interest) 315 in the inspection target image 311, the image reception unit 21 also receives an inspection area image 210 corresponding to the inspection area 315.

[0036] The resizing processing unit 22 is an example of an adjustment unit, and performs adjustment processing for adjusting an image of an inspection target area (inspection area image) in the image received by the image reception unit 21 to a model input image of a predetermined size for inputting to the machine learning model 24. In this embodiment, the model input images for inputting to the machine learning models 24A to 24C have the same size (for example, 512 (pixels) × 512 (pixels)).

[0037] The learning unit 23 has a plurality of machine learning models 24 (24A to 24C). The learning unit 23 executes machine learning on the machine learning model 24 based on the learning image received by the image reception unit 21.

[0038] The machine learning model 24 has a plurality of layers. The machine learning model 24 is, for example, an AutoEncoder and includes, for example, a convolutional layer, a MAX (maximum value) pooling layer, an AVG (average) pooling layer, a downsampling layer, and the like. The machine learning model 24 creates a multi-dimensional feature map (feature information) indicating the features in the model input image with the model input image as an input.

[0039] In the present embodiment, the machine learning model 24 includes a machine learning model 24A suitable for detecting a detection target of a large size (abnormal size large), a machine learning model 24B suitable for detecting a detection target of a medium size (abnormal size medium), and a machine learning model 24C suitable for detecting a detection target of a small size (abnormal size small). The size of the smallest feature map in the machine learning model 24A is, for example, 32×32, the size of the smallest feature map in the machine learning model 24B is, for example, 64×64, and the size of the smallest feature map in the machine learning model 24C is, for example, 128×128. Note that the dimension of the feature map in the machine learning model 24 increases as the size of the feature map decreases. In the machine learning model 24, since the inspection area image is aggregated into the smallest feature map, for a detection target smaller than the size corresponding to a 1×1 element (conveniently referred to as a pixel) in the smallest feature map in the inspection area image, the detection stability tends to be low.

[0040] The inspection and verification unit 25 verifies the machine learning model 24 by inspecting whether the object to be detected is detected for the verification image using the machine learning model 24 learned by the learning unit 23. Further, the inspection and verification unit 25 performs a process of selecting a machine learning model that satisfies a predetermined condition based on the verification results by a plurality of machine learning models 24. In this case, the inspection and verification unit 25 is an example of a selection unit. Also, the inspection and verification unit 52 inspects whether the object to be detected is detected for the image of the object to be inspected taken in operation using the machine learning model 24 learned by the learning unit 23.

[0041] The display processing unit 26 displays learning images and verification images, and displays a screen of a user interface (machine learning model screen 301 (see FIG. 4)) used for learning such as accepting designation of a machine learning model to be used, designation of an inspection target area in the image, etc. The display processing unit 26 calculates an object size (stable detection size: detection target size) of the object to be detected for which stable detection is expected when inspecting the designated inspection target area using the selected machine learning model, and performs a process (reference object display process) of displaying a reference object corresponding to the calculated detection target size on the machine learning model screen 301. The machine learning model 24 detects an object having a size equal to or larger than the stable detection size. In the present embodiment, as an example of image inspection, the object to be detected is a defect, but as another example of image inspection involving character recognition processing, the object to be detected may be a character.

[0042] The instruction reception unit 27 receives various instructions from the user via the displayed screen or the like, and notifies the learning unit 23, the inspection and verification unit 25, and the display processing unit 26. For example, the instruction reception unit 27 receives designation of the machine learning model 24 to be selected and designation of the inspection target area via the machine learning model screen 301, and notifies the display processing unit 26. The instruction reception unit 27 is an example of an inspection target area reception unit, a model reception unit, and a selection unit.

[0043] Next, the machine learning model screen 301 displayed by the display processing unit 26 will be described.

[0044] Figure 4 is a configuration diagram of a screen for a machine learning model according to an embodiment.

[0045] The screen 301 for the machine learning model includes an image view 310, a learning setting area 320, a verification image display area 330, and a comprehensive determination display area 340.

[0046] The image view 310 is an area where an image (inspection target image) 311 to be operated or referred to by the user is displayed. In the example of FIG. 4, in the inspection target image 311, an image of a product 311a to be inspected is displayed, and a defect 311b to be detected exists on the product 311a.

[0047] In the image view 310, an inspection target area 315 is displayed on the inspection target image 311. The inspection target area 315 can indicate its position and size according to an operation instruction. The size of the inspection target area 315 can be changed by an operation such as drag and drop. In the example of FIG. 4, the inspection target area 315 is rectangular, but it may be circular, polygonal, an arbitrary shape, etc.

[0048] Also, in the image view 310, a reference object 316 based on the selected machine learning model and the size and shape of the inspection target area 315 is displayed. When the selected machine learning model or the size of the inspection target area 315 is changed, the reference object 316 is changed to a corresponding size and / or shape. Also, when the position of the inspection target area 315 changes, the position of the reference object 316 also moves following the changed position of the inspection target area 315. Thereby, regardless of the position where the inspection target area 315 is set within the inspection target image 311, it becomes easier to compare an object (for example, the defect 311b) included in the inspection target area 315 with the reference object 316. In the example of FIG. 4, the reference object 316 is circular, but it may be polygonal such as rectangular, an arbitrary shape, or a scale bar indicating the size.

[0049] The learning setting area 320 includes a used learning model setting area 321, a learning image display area 324, and a learning button 325. In the used learning model setting area 321, a used learning model selection area 322 and an automatic selection box 323 are displayed.

[0050] The used learning model selection area 322 is an area for receiving a selection of a machine learning model to be used from among a plurality of machine learning models by the user. In the example of FIG. 4, the size of the detection target object for which the machine learning model is suitable (such as large abnormal size, medium abnormal size, small abnormal size, etc.) is displayed, and a suitable machine learning model can be selected according to the size of the abnormal size.

[0051] The automatic selection box 323 is a box for receiving a designation when automatically selecting the machine learning model to be used. When the automatic selection box 323 is designated, a learning model determination process described later for automatically determining (selecting) the machine learning model is executed.

[0052] In the learning image display area 324, one or more learning images 326 designated by the user are displayed.

[0053] The learning button 325 is a button for receiving an instruction to learn with the set content. When the learning button 325 is pressed, the instruction receiving unit 27 notifies the learning unit 23 of the designated inspection target area 315 and the selected machine learning model when the machine learning model is selected and the inspection target area 315 is designated, and causes the learning unit 23 to start the learning process. On the other hand, when the automatic selection box 323 is designated, a process of learning the machine learning model and selecting an appropriate machine learning model is started.

[0054] In the verification image display area 330, a verification image 331 designated by the user is displayed.

[0055] In the comprehensive determination display area 340, it is displayed whether the image displayed on the image view 310 is a non-defective product (OK) that does not contain the detection target or a defective product (NG) that contains the detection target.

[0056] Next, the processing operation by the image inspection apparatus 1 will be described.

[0057] FIG. 5 is a flowchart of the learning process according to an embodiment.

[0058] The learning process is, for example, a process executed before starting the operation of the image inspection apparatus 1 in an inspection for a new inspection target.

[0059] First, the display processing unit 26 displays the machine learning model screen 301 in the initial state, and the image reception unit 21 receives one or more learning images for learning the machine learning model 24 of the learning unit 23 and a plurality of verification images associated with the correct answers for evaluating the accuracy of the machine learning model 24 (S11). Note that the learning image may be an image associated with a non-defective product or an image associated with a defective product. The image received by the image reception unit 21 is displayed on the machine learning model screen 301. In the image view 310 of the machine learning model screen 301, either the learning image or the verification image is displayed by the display processing unit 26, and the displayed image can be switched according to the user's instruction.

[0060] Here, the following processing shows the processing when a specific machine learning model 24 is selected by the user in the used learning model selection area 322 of the machine learning model screen 301.

[0061] The instruction reception unit 27 receives the selected machine learning model 24 and notifies the display processing unit 26 (S12).

[0062] Next, the instruction receiving unit 27 receives the region of interest (ROI) in the inspection target image 311 of the image view 310 on the machine learning model screen 301, and notifies the display processing unit 26 (S13).

[0063] Based on the received machine learning model and the received inspection target region, the display processing unit 26 performs reference object display processing, and displays the reference object 316 on the machine learning model screen 301 on the inspection target image 311 including the image of the inspection target region 315 (inspection region image 210) (S14). Details of the reference object display processing by the display processing unit 26 will be described later. With this reference object 316, the user can easily determine whether the machine learning model to be used and the designated inspection target region are suitable for the detection target object.

[0064] Next, the instruction receiving unit 27 determines whether there is an instruction to change the machine learning model 24 or the inspection target region 315 from the user (S15). If there is an instruction to change (S15: Yes), the process proceeds to step S12.

[0065] On the other hand, if there is no instruction to change (S15: No), the instruction receiving unit 27 determines whether there is a learning instruction, that is, whether the learning button 325 has been pressed (S16). As a result, if there is no learning instruction (S16: No), the instruction receiving unit 27 proceeds to step S15.

[0066] On the other hand, if there is a learning instruction (S16: Yes), the instruction receiving unit 27 notifies the learning unit 23 of the designated inspection target region 315 and the selected machine learning model 24. The learning unit 23 that has received the notification causes the resizing processing unit 22 to execute an image adjustment process of adjusting the image of the inspection target region 315 to a model input image for each learning image (S17). Details of the image adjustment process by the resizing processing unit 22 will be described later.

[0067] Next, the learning unit 23 performs learning processing on the machine learning model 24 received from the instruction receiving unit 27, using the model input image adjusted by the resizing processing unit 22 as input (S18). Specifically, the learning unit 23 determines the parameters of the machine learning model 24 through the learning process.

[0068] Next, the inspection and verification unit 25 causes the resizing processing unit 22 to execute an image adjustment process for adjusting the image of the inspection target area 315 to the model input image for each of the verification images, inputs each model input image into the machine learning model 24 learned by the learning unit 23, and verifies the detection status of the object to be detected by the machine learning model 24 based on the output of the machine learning model 24 (S19). Note that the verification results for each verification image are displayed on the machine learning model screen 301.

[0069] Next, the instruction receiving unit 27 determines whether there is an instruction to end the setting (S20). If there is no instruction to end the setting (S20: No), the process proceeds to step S11. On the other hand, if there is an instruction to end the setting (S20: Yes), the instruction receiving unit 27 notifies the inspection and verification unit 25 that there is an instruction to end the setting. The inspection and verification unit 25 registers the learned machine learning model 24 as a machine learning model (operation model) to be used during operation (S21), and ends the learning process. As a result, the image inspection apparatus 1 can perform appropriate inspections using the newly registered operation model.

[0070] Next, the learning model determination process executed when the automatic selection selection box 323 in the used learning model selection area 322 of the machine learning model screen 301 is specified will be described.

[0071] FIG. 6 is a flowchart of the learning model determination process according to an embodiment. In FIG. 6, for the processing steps similar to the learning process shown in FIG. 5, the same reference numerals are given, and duplicate explanations may be omitted.

[0072] In the learning model determination process, the instruction reception unit 27 determines whether there is an instruction to change the inspection target area 315 (S31). If there is an instruction to change (S31: Yes), the process proceeds to step S32.

[0073] On the other hand, if there is no instruction to change (S31: No), the instruction reception unit 27 determines whether there is a learning instruction, that is, whether the learning button 325 has been pressed (S32). As a result, if there is no learning instruction (S32: No), the instruction reception unit 27 proceeds the process to step S31.

[0074] On the other hand, if there is a learning instruction (S32: Yes), the instruction reception unit 27 notifies the specified inspection target area 315 to the learning unit 23 and the display processing unit 26. The learning unit 23 that has received the notification causes the resizing processing unit 22 to execute an image adjustment process of adjusting the image of the inspection target area 315 to the model input image for each of the learning images (S17).

[0075] Next, the learning unit 23 performs a learning process of the machine learning model 24 with the model input image adjusted by the resizing processing unit 22 as an input for each of the plurality of machine learning models 24 (S33).

[0076] Next, the inspection verification unit 25 causes the resizing processing unit 22 to execute an image adjustment process of adjusting the image of the inspection target area 315 to the model input image for each of the verification images, inputs each of the model input images to the respective machine learning models 24 learned by the learning unit 23, and verifies the detection status of the detection target object by each of the machine learning models 24 based on the output of the machine learning model 24 (S34).

[0077] Next, based on the verification results of the detection status of each machine learning model 24, the inspection and verification unit 25 selects one machine learning model 24 to be used in operation and notifies the selected machine learning model 24 to the display processing unit 26 (S35). Here, the inspection and verification unit 25 may, for example, select the machine learning model 24 with the highest detection accuracy among the plurality of machine learning models 24, or may select according to a predetermined condition (for example, the one with a low processing load, etc.) from among the machine learning models 24 with a detection accuracy equal to or higher than a predetermined level.

[0078] Next, based on the received inspection target area and the machine learning model 24 notified from the inspection and verification unit 25, the display processing unit 26 performs reference object display processing to display the reference object 316 on the machine learning model screen 301 (S36). With this reference object 316, the user can easily determine whether the selected machine learning model and the designated inspection target area are suitable for the detection target object.

[0079] Next, the instruction reception unit 27 determines whether an instruction to end the setting has been given (S37). If there is no instruction to end the setting (S37: No), the process proceeds to step S11. On the other hand, if there is an instruction to end the setting (S37: Yes), the instruction reception unit 27 notifies the inspection and verification unit 25 that an instruction to end the setting has been given. The inspection and verification unit 25 registers the learned machine learning model 24 as the machine learning model (operation model) to be used during operation (S38), and ends the learning model determination process. As a result, the image inspection apparatus 1 can perform appropriate inspection using the newly registered operation model.

[0080] According to this learning model determination process, an appropriate machine learning model can be automatically selected.

[0081] Next, in operation, an image inspection process for inspecting an inspection target image using the learned machine learning model 24 (operation model) set for operation will be described.

[0082] FIG. 7 is a flowchart of the image inspection process according to an embodiment.

[0083] First, the image reception unit 21 receives an inspection target image to be inspected during operation (S41).

[0084] The instruction reception unit 27 receives the inspection target region (ROI) in the inspection target image via the image view 310 of the machine learning model screen 301, and notifies the display processing unit 26 (S42). Note that when the inspection target regions of the learning image and the inspection target image during operation are the same region, since the inspection target region in the inspection target image is determined during learning, step S42 is not executed.

[0085] Next, the display processing unit 26 performs reference object display processing based on the operation model and the received or predetermined inspection target region, and displays the reference object 316 on the image view 310 (S43). With this reference object 316, the user can easily determine whether the operation model is suitable for the detection target object.

[0086] Next, the instruction reception unit 27 determines whether there is an inspection instruction (S44). As a result, if there is no inspection instruction (S44: No), the instruction reception unit 27 ends the process.

[0087] On the other hand, if there is an inspection instruction (S44: Yes), the instruction reception unit 27 notifies the inspection target region 315 to the inspection verification unit 25, and the inspection verification unit 25 causes the resizing processing unit 22 to execute an image adjustment process of adjusting the image of the inspection target region 315 of the inspection target image to a model input image (S45). Next, the inspection verification unit 25 inputs the model input image into the operation model, inspects the detection target object based on the output of the operation model, and notifies the inspection result to the display processing unit 26 (S46).

[0088] Next, the display processing unit 26 displays the inspection result of the inspection target image (see FIG. 12) (S47).

[0089] Next, the instruction reception unit 27 determines whether there is an instruction to end the inspection (S48). If there is no instruction to end the inspection (S48: No), the process proceeds to step S41, and processing for the next inspection target image is performed. On the other hand, if there is an instruction to end the inspection (S48: Yes), the instruction reception unit 27 ends the image inspection process. As a result, the image inspection apparatus 1 can operate an appropriate inspection using the registered operation model.

[0090] Next, a process of extracting feature information from the inspection target image using a machine learning model will be described.

[0091] FIG. 8 is a diagram for explaining an extraction process (the same in the inspection process and the learning process) of extracting feature information from an inspection target image according to an embodiment using a machine learning model.

[0092] Note that before the extraction process, an inspection target area 315 is specified for the inspection target image 311.

[0093] First, the resizing processing unit 22 adjusts the image (inspection area image 210) of the inspection target area 315 of the inspection target image 311 to a model input image 220 of a predetermined size (for example, 512×512). For example, the inspection area image 210 is reduced and converted into the model input image 220. Here, let the reduction rate (adjustment rate) for the horizontal and vertical sizes be A.

[0094] Next, when the model input image 220 is input to the machine learning model 24, the machine learning model 24 extracts features from the model input image 220 and creates a feature map 230. The size of the feature map 230 is different depending on the machine learning model 24 used. In this embodiment, the size of the smallest feature map in the machine learning model 24A corresponding to the large abnormal size is, for example, 32×32, the size of the smallest feature map in the machine learning model 24B corresponding to the medium abnormal size is, for example, 64×64, and the size of the smallest feature map in the machine learning model 24C corresponding to the small abnormal size is, for example, 128×128.

[0095] Here, assuming that the reduction rate in each direction (vertical and horizontal) from the model input image 220 to the feature map 230 is B, in the case of the machine learning model 24A, the reduction rate B is 1 / 16, in the case of the machine learning model 24B, the reduction rate B is 1 / 8, and in the case of the machine learning model 24C, the reduction rate B is 1 / 4.

[0096] In this case, the reduction rate in each direction (overall reduction rate) from the inspection area image 210 to the feature map 230 can be expressed as the reduction rate A × the reduction rate B.

[0097] Each element of the feature map 230 (in this embodiment, conveniently referred to as a pixel) strongly indicates the features of the corresponding range of the inspection target area 315. Therefore, the object to be detected can be stably detected with respect to the size (detection target size: stable detection size) of the range (inspection area image 210) of the inspection target area 315 corresponding to each pixel of the feature map 230. Note that as the threshold value of the size in the feature map 230 corresponding to the stable detection size of the inspection area image 210, for example, it is 1×1, but it may be larger than 1×1. In some cases, it may be smaller than 1×1, but the condition is that at least a little feature remains in the feature map 230. In this example, the threshold value of the size in the feature map 230 is described as 1×1. When the threshold value of the size in the feature map 230 is 1×1, the size of the corresponding range in the inspection target area 315 is 1 / overall reduction rate in each of the vertical and horizontal directions.

[0098] For example, therefore, assuming that the reduction rate A is 1 / 2, the stable detection size when using the machine learning model 24A is 32×32, the stable detection size when using the machine learning model 24B is 16×16, and the stable detection size when using the machine learning model 24C is 8×8.

[0099] In this embodiment, the display processing unit 26 calculates a stable detection size based on the correspondence between the sizes of the inspection target area 315 and the feature map 230 of the machine learning model 24, and displays an object corresponding to this stable detection size as a reference object 316. In this example, when the machine learning model 24 is determined, the reduction rate B in each direction from the model input image 220 to the feature map 230 is determined. Therefore, by multiplying the reduction rate A in each direction from the inspection area image 210 to the feature map 230 by this reduction rate, the overall reduction rate is calculated, and the stable detection size is calculated based on this overall reduction rate.

[0100] As described above, since the stable detection size is determined by the correspondence between the sizes of the inspection area image 210 and the feature map 230 in the machine learning model 24, when the size of the inspection area image 210 changes, and when the machine learning model 24 used changes, the stable detection size changes. As a result, the size of the reference object 316 changes.

[0101] Next, the change in the size of the reference object 316 with respect to the difference in the machine learning model 24 and the size of the inspection target area 315 will be described.

[0102] FIG. 9 is a diagram for explaining the display of the reference object according to an embodiment.

[0103] Here, FIG. 9(A) shows the reference object 316 when the machine learning model 24A with a large abnormal size is used and the inspection target area 315 is set. FIG. 9(B) shows the reference object 316 when the inspection target area 315 remains the same as in FIG. 9(A) and the machine learning model 24B with a small abnormal size is used. FIG. 9(C) shows the reference object 316 when the machine learning model 24A with a large abnormal size is used and the inspection target area 315 is set to be small. FIG. 9(D) shows the reference object 316 when the inspection target area 315 is set to the same size as in FIG. 9(C) and the machine learning model 24B with a small abnormal size is used.

[0104] For example, when setting to use the machine learning model 24B with a small abnormal size from the state shown in FIG. 9(A), as shown in FIG. 9(B), the size of the reference object 316 becomes smaller.

[0105] Also, for example, when setting the inspection target area 315 to be smaller from the state shown in FIG. 9(A), as shown in FIG. 9(C), the size of the reference object 316 becomes smaller.

[0106] Also, for example, when setting to use the machine learning model 24B with a small abnormal size and setting the inspection target area 315 to be smaller from the state shown in FIG. 9(A), as shown in FIG. 9(D), the size of the reference object 316 is smaller than the states shown in FIGS. 9(B) and 9(C).

[0107] As shown in FIG. 9, by changing the machine learning model 24 to be used and the size of the inspection target area 315, the stable detection size changes, and the size of the corresponding reference object 316 is changed. The smaller the stable detection size is than the size of the actual detection target object (for example, a defect or a character), the easier it is for the machine learning model 24 to detect an object smaller than the detection target object (for example, a noise component). Also, the larger the stable detection size is than the size of the actual detection target object, the more difficult it is for the machine learning model 24 to detect the detection target object. Therefore, in order to improve the accuracy of image inspection, it is necessary for the user to accurately recognize the stable detection size. According to the present embodiment, the user can easily perform a setting suitable for detecting the detection target object by changing the machine learning model 24 to be used and the size of the inspection target area 315 while referring to the reference object 316.

[0108] In FIGS. 8 and 9 described above, the inspection target area 315 was a square shape that was simple to process within a rectangular shape for ease of explanation and was an area larger than the model input image. However, the rectangular shape of the inspection target area 315 is not limited to such a shape. In the present embodiment, the resizing processing unit 22 performs different image adjustment processes according to the size and shape of the inspection target area 315.

[0109] Next, the image adjustment process by the resizing processing unit 22 will be described.

[0110] FIG. 10 is a diagram for explaining an image adjustment process according to an embodiment.

[0111] When the inspection area image is smaller than the size of the model input image, the resizing processing unit 22 pads the difference from the size of the model input image with predetermined pixels (for example, black pixels with a luminance value of 0) without changing the size of the inspection area image as shown in FIG. 10(A), thereby adjusting it to the model input image.

[0112] On the other hand, when at least one of the vertical or horizontal sizes of the inspection area image is larger than the size of the model input image, the resizing processing unit 22 determines whether it is possible to make the size in the short side direction equal to or greater than the lower limit value while reducing the aspect ratio of the inspection area image without changing it and making the long side direction the same as the corresponding direction of the model input image. Here, the lower limit value may be, for example, a size such that the size corresponding to the short side direction of the feature map generated by inputting the model input image into the machine learning model 24 to be used is equal to or greater than a predetermined threshold value (for example, 1 pixel).

[0113] As a result, when the short side direction can be made equal to or greater than the lower limit value, as shown in FIG. 10(B-1), without changing the aspect ratio of the inspection area image, it is reduced so that the long side direction becomes the same as the corresponding direction of the model input image, and padding is performed with a predetermined pixel (for example, a black pixel) for the difference from the predetermined size of the model input image, thereby adjusting it to the model input image. Note that, after padding the portion that is insufficient for the predetermined size of the model input image with a predetermined pixel, it may be reduced to the predetermined size.

[0114] On the other hand, when the short side direction cannot be made equal to or greater than the lower limit value, as shown in FIG. 10(B-2), the aspect ratio of the inspection area image is changed, and it is reduced so that the long side direction becomes equal to or less than the corresponding direction of the model input image and the short side direction becomes equal to or greater than the lower limit value, and padding is performed with a predetermined pixel (for example, a black pixel) for the difference from the size of the model input image, thereby adjusting it to the model input image. Note that, based on the changed aspect ratio of the inspection area image, after padding the portion that is insufficient for the predetermined size with a predetermined pixel, the inspection area image may be adjusted to the model input image by reducing it to a size equal to or less than the predetermined size so that the size corresponding to the short side direction of the feature map does not become less than a predetermined threshold value (for example, 1 pixel).

[0115] Next, a specific example of the reference object will be described.

[0116] FIG. 11 is a diagram for explaining a specific example of a reference object according to an embodiment. FIG. 11 shows a reference object (stable detection size) when the model input image is 512×512.

[0117] FIG. 11(A) shows the size of the reference object when the inspection area image is 1024×1024 and the machine learning model 24C with a small abnormal size is specified.

[0118] In this example, the reduction ratio in both the vertical and horizontal directions from the inspection area image to the model input image is 1 / 2. Also, the reduction ratio from the model input image to the feature map in the machine learning model 24C is 1 / 4. In this case, the stable detection size of the inspection area image corresponding to a 1×1 pixel of the feature map is 8×8. Therefore, the reference object is an object corresponding to a size of 8×8. Note that the reference object may be a rectangle of 8×8 or a circle with a diameter of 8.

[0119] Figure 11(B) shows the size of the reference object when the inspection area image is 1024×1024 and the machine learning model 24A with a large abnormal size is specified.

[0120] In this example, the reduction ratio in both the vertical and horizontal directions from the inspection area image to the model input image is 1 / 2. Also, the reduction ratio from the model input image to the feature map in the machine learning model 24A is 1 / 16. In this case, the stable detection size of the inspection area image corresponding to a 1×1 pixel of the feature map is 32×32. Therefore, the reference object is an object corresponding to a size of 32×32. Note that the reference object may be a rectangle of 32×32 or a circle with a diameter of 32.

[0121] Figure 11(C) shows the size of the reference object when the inspection area image is 30×1024 and the machine learning model 24A with a large abnormal size is specified.

[0122] In this example, for the inspection area image, even if at least one of the vertical or horizontal sizes of the inspection area image is larger than the size of the model input image and it is reduced so that the longitudinal direction becomes the same as the corresponding direction of the model input image without changing the aspect ratio of the inspection area image, the short side cannot be made equal to or greater than the lower limit value (in this example, the size corresponding to the short side of the feature map is 1 pixel or more). In this case, the inspection area image is adjusted by the resizing processing unit 22 as shown in FIG. 10(B-2). Specifically, the inspection target image is adjusted to the model input image with the vertical reduction rate being 1 / 2 and the horizontal reduction rate being 1 (i.e., non-reduction).

[0123] In this case, since the stable detection size of the inspection area image corresponding to 1×1 pixel of the feature map is 16 times in the horizontal direction and 32 times in the vertical direction, it becomes 16×32. Therefore, the reference object becomes an object corresponding to the size of 16×32. Note that the reference object may be a rectangle of 16×32 or an ellipse with the major axis being 32 and the minor axis being 16.

[0124] Next, an example of displaying the inspection result in the image inspection process will be described.

[0125] FIG. 12 is a diagram for explaining an example of displaying the inspection result according to an embodiment.

[0126] The detection result by the inspection verification unit 25 is displayed by the display processing unit 26, for example, overlaid on the inspection target image 311 displayed on the image viewer 310. Note that on the inspection target image 311, an inspection target area 315 and a reference object 316 are displayed. On this inspection target image 311, the detection result is displayed, for example, as a heat map in which areas where there is a high possibility of the presence of the detection target object are emphasized more. In the example of FIG. 12, a high possibility area 317a indicating that there is the highest possibility of the presence of the detection target object is displayed in the range where the defect 311b exists, and a medium possibility area 317b indicating that there is a medium possibility of the presence of the detection target object is displayed around it.

[0127] Next, the process when a non-rectangular region is specified as the inspection target region will be described.

[0128] FIG. 13 is a diagram for explaining the process when a non-rectangular region according to an embodiment is specified.

[0129] In the above example, an example where the region specified as the inspection target region is rectangular has been shown. However, in the present embodiment, the specification of the inspection target region may be any shape such as a polygon, a star shape, a circular shape, etc., that is, non-rectangular. Thus, when a non-rectangular shape is specified as the inspection target region, a corrected inspection region image obtained by correcting the specified region into a rectangle may be subjected to the same processing as the inspection region image described above.

[0130] Here, for example, when a star-shaped region 402 is specified for the image 401, the resizing processing unit 22 sets an image of a rectangle circumscribing the specified region 402 as the corrected inspection region image 403, and pads the difference between the corrected inspection region image 403 and the region 402 with a predetermined pixel (for example, a black pixel).

[0131] Note that the present invention is not limited to the above-described embodiments and modified examples, and can be appropriately modified and implemented without departing from the spirit of the present invention.

[0132] For example, in the above embodiment, the reference object may be a rectangle of a stable detection size, a circle having a diameter corresponding to the stable detection size, or a scale bar indicating at least one of the horizontal or vertical direction of the stable detection size. In short, any object that allows the user to grasp the stable detection size may be used.

[0133] For example, in the above embodiment, the machine learning model is an AutoEncorder, but the present invention is not limited to this, and any machine learning model that compresses or reduces the input image by a convolutional layer, a pooling layer, or a downsampling layer may be used, and a machine learning model that does not include a decoder may also be used.

[0134] Also, the machine learning model in the above embodiment may be a model that combines a plurality of machine learning models.

[0135] Also, in the present embodiment, for example, an image inspection apparatus, an image inspection method, and an image inspection program according to the following Supplementary Note 1 to Supplementary Note 14 are included.

[0136] (Supplementary Note 1) An image inspection apparatus having a machine learning model that generates a feature map based on an input inspection target image, and detecting a detection target object included in the inspection target image based on the feature map, a storage unit that stores the machine learning model, Based on the size of an inspection target area that is all or part of the inspection target image and the size of a feature map generated by the machine learning model based on the inspection target area, calculate an object size of a detection target object for which stable detection is expected in the inspection target area, and cause a reference object corresponding to the object size to be displayed on a display unit together with the inspection target area. A control unit, An image inspection apparatus comprising:

[0137] (Supplementary Note 2) The storage unit stores a plurality of machine learning models having different sizes of the feature maps, The image inspection apparatus further includes a selection unit that selects one machine learning model to be used from among the plurality of machine learning models, The control unit calculates the object size based on the size of the inspection target area and the size of a feature map generated based on the inspection target area by the selected machine learning model The image inspection apparatus according to Supplementary Note 1.

[0138] (Supplementary Note 3) The image inspection apparatus further includes a learning unit that learns the plurality of machine learning models based on an image of the inspection target area, The selection unit selects a machine learning model to be used from among the plurality of machine learning models based on the learning results of the plurality of machine learning models. The image inspection apparatus according to Supplementary Note 2.

[0139] (Supplementary Note 4) An inspection target area reception unit that receives designation and size change of the inspection target area in the inspection target image, A model reception unit that receives designation of a machine learning model to be used, and further includes: The selection unit selects, as the machine learning model to be used, the machine learning model received by the model reception unit, Based on the size of the inspection target area whose change has been received and the size of the feature map generated based on the inspection target area by the selected machine learning model, the control unit calculates the object size, and causes the display unit to display a new reference object corresponding to the object size in association with the inspection target area whose size has been changed. The image inspection apparatus according to Supplementary Note 2.

[0140] (Supplementary Note 5) Further includes an inspection target area reception unit that receives designation and size change of the inspection target area in the inspection target image, Based on the size of the inspection target area whose change has been received and the size of the feature map generated based on the inspection target area by the machine learning model, the control unit calculates the object size, and causes the display unit to display a new reference object corresponding to the object size in association with the inspection target area whose size has been changed. The image inspection apparatus according to Supplementary Note 1.

[0141] (Supplementary Note 6) The control unit Adjusts the size of the inspection area image corresponding to the inspection target area to generate a model input image that is an image of a predetermined size, Is configured to input the model input image into the machine learning model to generate the feature map. The image inspection apparatus according to Supplementary Note 1.

[0142] (Supplementary Note 7) The control unit calculates the object size based on the adjustment ratio from the inspection area image to the model input image and the reduction ratio from the model input image to the feature map. The image inspection apparatus according to Supplementary Note 6.

[0143] (Supplementary Note 8) When at least one of the vertical or horizontal sizes of the inspection area image is larger than the corresponding size of the model input image in the corresponding direction, the control unit reduces the inspection area image to be equal to or smaller than the predetermined size while maintaining the aspect ratio of the inspection area image, and pads the portion that is less than the predetermined size with predetermined pixels to adjust it to the model input image. The image inspection apparatus according to Supplementary Note 6.

[0144] (Supplementary Note 9) The control unit When either the vertical or horizontal size of the inspection area image is larger than the corresponding size of the model input image in the corresponding direction, and when reducing the inspection area image to be equal to or smaller than the predetermined size while maintaining the aspect ratio of the inspection area image, if the size in the shorter direction (either vertical or horizontal) satisfies a predetermined condition, the control unit changes the aspect ratio by varying the reduction ratios for the vertical and horizontal directions of the inspection area image, reduces the inspection area image to be equal to or smaller than the predetermined size so that the size in the shorter direction does not satisfy the predetermined condition, and pads the portion that is less than the predetermined size with predetermined pixels to adjust it to the model input image. The image inspection apparatus according to Supplementary Note 8.

[0145] (Supplementary Note 10) The predetermined condition is that when reducing the model input image to the feature map, the size corresponding to the shorter direction of the feature map becomes less than a predetermined threshold. The image inspection apparatus according to Supplementary Note 9.

[0146] (Appendix 11) The control unit calculates the object size based on the aspect ratio of the inspection area image after change according to the reduction ratios in the vertical and horizontal directions of the inspection area image and the reduction ratio from the model input image to the feature map. The image inspection apparatus according to Appendix 9.

[0147] (Appendix 12) When the size of the inspection area image is smaller than the size of the model input image, the control unit adjusts the inspection area image to the model input image by padding the portion lacking the predetermined size with predetermined pixels. The image inspection apparatus according to Appendix 6.

[0148] (Appendix 13) The inspection target area reception unit can receive a designation of a non-rectangular area as the designation of the inspection target area in the inspection target image. When the designation of the inspection target area is a non-rectangular area, the control unit calculates the object size based on a rectangular area circumscribing the non-rectangular area. The image inspection apparatus according to Appendix 1.

[0149] (Appendix 14) An image inspection program that causes a computer to have a machine learning model that generates a feature map based on an input inspection target image and detects a detection target object included in the inspection target image based on the feature map, causing the computer to function as a control unit that calculates an object size of a detection target object for which stable detection is expected in the inspection target area based on the size of the inspection target area that is all or part of the inspection target image and the size of the feature map generated by the machine learning model based on the inspection target area, and causes a reference object corresponding to the object size to be displayed on the display unit together with the inspection target area.

Explanation of Signs

[0150] 1... Image inspection device, 2... Control unit, 3... Imaging unit, 4... Display device, 5... Personal computer, 13A... Control section, 21... Image reception section, 22... Resizing process, 23... Learning section, 24A, 24B, 24C... Machine learning models, 25... Inspection verification section, 26... Display processing section, 27... Instruction reception section

Claims

1. An image inspection apparatus having a machine learning model that generates a feature map based on an input inspection target image, and detecting a detection target object included in the inspection target image based on the feature map, a storage unit that stores the machine learning model; a control unit that calculates an object size of a detection target object for which stable detection is expected in the inspection target area based on the size of the inspection target area that is all or part of the inspection target image and the size of the feature map generated by the machine learning model based on the inspection target area, and causes a reference object corresponding to the object size to be displayed on a display unit together with the inspection target area; An image inspection apparatus comprising:

2. The storage unit stores a plurality of machine learning models having different sizes of the feature maps, The image inspection apparatus further includes a selection unit that selects one machine learning model to be used from among the plurality of machine learning models, The control unit calculates the object size based on the size of the inspection target area and the size of the feature map generated based on the inspection target area by the selected machine learning model The image inspection apparatus according to claim 1.

3. The apparatus further includes a learning unit that learns the plurality of machine learning models, The selection unit selects a machine learning model to be used from among the plurality of machine learning models based on learning results of the plurality of machine learning models The image inspection apparatus according to claim 2.

4. an inspection target area reception unit that receives designation and size change of the inspection target area in the inspection target image; a model reception unit that receives designation of a machine learning model to be used, and further includes: The selection unit selects, as a machine learning model to be used, the machine learning model received by the model reception unit, The control unit calculates the object size based on the size of the inspection target area whose change has been received and the size of the feature map generated based on the inspection target area by the selected machine learning model, and causes a new reference object corresponding to the object size to be displayed on the display unit in correspondence with the inspection target area whose size has been changed. The image inspection apparatus according to claim 2.

5. The apparatus further includes an inspection target area reception unit that receives designation and size change of the inspection target area in the inspection target image, The control unit calculates the object size based on the size of the inspection target area for which the change has been received and the size of the feature map generated based on the inspection target area by the machine learning model, and causes the display unit to display a new reference object corresponding to the object size in association with the inspection target area for which the size change has been received. The image inspection apparatus according to claim 1.

6. The control unit adjusts the size of the inspection area image corresponding to the inspection target area to generate a model input image that is an image of a predetermined size, and is configured to input the model input image into the machine learning model to generate the feature map. The image inspection apparatus according to claim 1.

7. The control unit calculates the object size based on the adjustment rate from the inspection area image to the model input image and the reduction rate from the model input image to the feature map. The image inspection apparatus according to claim 6.

8. When at least one of the vertical or horizontal sizes of the inspection area image is larger than the corresponding size in the model input image in the corresponding direction, the control unit reduces the inspection area image to be equal to or less than the predetermined size while maintaining the aspect ratio of the inspection area image, and pads the portion lacking the predetermined size with predetermined pixels to adjust the inspection area image to the model input image. The image inspection apparatus according to claim 6.

9. The control unit When one of the vertical or horizontal sizes of the inspection area image is larger than the corresponding size in the model input image in the corresponding direction, and when reducing the inspection area image to be equal to or less than the predetermined size while maintaining the aspect ratio of the inspection area image, if the size in the shorter direction in the vertical or horizontal direction satisfies a predetermined condition, the reduction rates for the vertical and horizontal directions of the inspection area image are made different to change the aspect ratio, and the inspection area image is reduced to be equal to or less than the predetermined size so that the size in the shorter direction does not satisfy the predetermined condition, and the portion lacking the predetermined size is padded with predetermined pixels to adjust the inspection area image to the model input image. The image inspection apparatus according to claim 8.

10. The predetermined condition is that when the model input image is reduced to the feature map, the size corresponding to the shorter direction of the feature map becomes less than a predetermined threshold. The image inspection apparatus according to claim 9.

11. The control unit calculates the object size based on the aspect ratio of the inspection region image after change according to the reduction ratios in the vertical and horizontal directions of the inspection region image, and the reduction ratio from the model input image to the feature map. The image inspection apparatus according to claim 9.

12. When the size of the inspection region image is smaller than the size of the model input image, the control unit pads the portion of the inspection region image that is insufficient for the predetermined size with predetermined pixels to adjust it to the model input image. The image inspection apparatus according to claim 6.

13. The inspection target region reception unit can receive the designation of a non-rectangular region as the designation of the inspection target region in the inspection target image. When the designation of the inspection target region is a non-rectangular region, the control unit calculates the object size based on a rectangular region circumscribing the non-rectangular region. The image inspection apparatus according to claim 5.

14. An image inspection program for causing a computer to execute, which has a machine learning model that generates a feature map based on an input inspection target image, and detects a detection target object included in the inspection target image based on the feature map, the computer to function as a control unit that calculates the object size of a detection target object for which stable detection is expected in the inspection target region based on the size of the inspection target region, which is all or part of the inspection target image, and the size of the feature map generated by the machine learning model based on the inspection target region, and causes the display unit to display a reference object corresponding to the object size together with the inspection target region.

Images