Image inspection device, image inspection method, and image inspection program

Abstract

To automate inspection work more simply and with higher accuracy.SOLUTION: An image inspection device according to one aspect of an embodiment comprises a control unit. The control unit acquires captured images of picked items. Furthermore, the control unit performs simplified training on a base model, which is a large-scale pre-trained AI model, using at least one of the captured images as a sample image. Further, the control unit executes an article inspection process for detecting and counting the article from the photographed image using the base model on which the simple instruction has been performed. In the inspection process, the control unit counts the articles after adjusting at least the size of the articles to the detection result by the base model.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The disclosed embodiments relate to an image inspection device, an image inspection method, and an image inspection program. [Background technology]

[0002] Conventionally, in the field of AI (Artificial Intelligence), a technique for detecting an object in an image by image recognition using a deep learning model such as a CNN (Convolutional Neural Network) has been known (see, for example, Patent Document 1).

[0003] This technology could be used, for example, in a warehouse management system (WMS) to automate inspection tasks such as identifying and counting items picked onto picking carts. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2018-022484 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the above-mentioned conventional techniques have room for further improvement in order to automate inspection work more easily and with higher accuracy.

[0006] For example, improving object detection accuracy requires learning from all kinds of pattern images, which requires the collection and training of a huge number of training images. Specifically, for example, an object detection method that detects objects in an image using a rectangular detection frame requires annotation work to teach the correct rectangle to each training image. With conventional technology, much of this work had to be done manually, which was extremely time-consuming.

[0007] One possible solution to this problem is to use object detection methods that use image processing such as template matching and background subtraction. This method has the advantage of being able to detect objects with a simple teaching process, such as registering a single template. However, it also has the disadvantage that detection can be difficult if the color, shape, background, etc. of the object differs from the template, making it difficult to ensure detection accuracy.

[0008] In particular, in inspection work, items of various sizes are often mixed together on a picking cart. For this reason, when a single image shows both large and small items, it is desirable to prevent false detections, such as detecting part of a large item as a small item.

[0009] One aspect of the embodiment has been made in consideration of the above, and aims to provide an image inspection device, an image inspection method, and an image inspection program that can automate inspection work more easily and with higher accuracy. [Means for solving the problem]

[0010] An image inspection device according to one aspect of the embodiment includes a control unit. The control unit acquires captured images of picked items. The control unit also provides simplified instruction to a base model, which is a large-scale trained AI model, using at least one of the captured images as a sample image. The control unit then uses the simplified instruction to execute an inspection process that detects and counts the items from the captured images using the base model. During the inspection process, the control unit counts the items after adjusting at least the size of the items based on the detection results from the base model. [Effects of the Invention]

[0011] According to one aspect of the embodiment, it is possible to automate inspection work more easily and with higher accuracy. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram illustrating an outline of an image inspection method according to an embodiment. [Figure 2] FIG. 2 is a block diagram of an image inspection device according to the embodiment. [Figure 3] FIG. 3 is a diagram showing an example of the simple teaching screen. [Figure 4] FIG. 4 is a diagram showing an example of a processing result screen of the inspection processing. [Figure 5] FIG. 5 is an explanatory diagram of size limitation by segmentation. [Figure 6] FIG. 6 is an explanatory diagram of additional registration. [Figure 7] FIG. 7 is an explanatory diagram of a case where the size of the inspection object can be specified. [Figure 8] FIG. 8 is a block diagram of the inspection terminal according to the embodiment. [Figure 9] FIG. 9 is a diagram showing a processing sequence executed by the image inspection system according to the embodiment. [Figure 10] FIG. 10 is a hardware configuration diagram showing an example of a computer that realizes the functions of the image inspection device. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of an image inspection device, an image inspection method, and an image inspection program disclosed in the present application will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to the embodiments described below.

[0014] First, an overview of an image inspection method according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram illustrating an overview of an image inspection method according to an embodiment.

[0015] In the following, an example will be described in which various items picked onto the picking cart 50 based on a picking instruction from the WMS 10 are detected and counted based on photographed images of these items.

[0016] As shown in FIG. 1, the image inspection system 1 according to the embodiment includes a WMS 10, a picking cart 50, an image inspection device 100, a camera 150, a light 170, and an inspection terminal 200.

[0017] The image inspection device 100, the camera 150, the lighting 170, and the inspection terminal 200 can be mounted on a picking cart 50.

[0018] The WMS 10 is a warehouse management system equipped with functions such as inventory management and warehousing management of cargo, materials, and products in a warehouse at a logistics center, etc. The WMS 10 is set up so as to be able to communicate with the image inspection device 100 via a network N1 such as an intranet, the Internet, or a mobile phone network.

[0019] The WMS 10 transmits a picking instruction via the network N1 to the image inspection device 100. The WMS 10 also receives a picking result, which is the result of the inspection work based on the picking instruction, from the image inspection device 100 via the network N1.

[0020] The picking cart 50 is a material handling device specialized for picking items based on picking instructions from the WMS 10. The picking cart 50 is transported, for example, by a worker performing picking and inspection work. Alternatively, the picking cart 50 may be self-propelled, for example, by electric power. The picking cart 50 loads items picked by the worker.

[0021] The picked items are placed on a tray T1. The lighting 170 is provided so as to be able to illuminate the tray T1. The camera 150 is provided so as to be able to photograph the tray T1. The camera 150 also transmits the photographed image of the tray T1 to the image inspection device 100.

[0022] The image inspection device 100 performs simple teaching, which is a simple teaching operation for the deep learning model for object detection, based on the image captured by the camera 150. Through this simple teaching, the deep learning model for object detection is linked to the characteristics of the item to be picked.

[0023] Specifically, the image inspection device 100 has a foundation model 103c as a deep learning model for object detection. The foundation model 103c is a large-scale trained AI model that has been trained in advance using a huge dataset. In this embodiment, the foundation model 103c is based on, for example, ChatGPT (Chat Generative Pre-trained Transformer), DALL-E, CLIP, etc. The foundation model 103c is personalized for the inspection task using the simple teaching described above.

[0024] The simplified teaching is realized by one-shot learning, in which one sample image is registered in the base model 103c, or by few-shot learning, in which at most several sample images are registered. The specific content of the simplified teaching will be described later using FIG. 3 etc.

[0025] Then, the image inspection device 100 uses the base model 103c that has been given this simplified instruction to perform an inspection process that detects and counts items from the image captured by the camera 150. The image inspection device 100 also transmits the picking results, which are registered as the results of the inspection process, to the WMS 10 via the network N1.

[0026] The inspection terminal 200 is a terminal device used by an operator. The inspection terminal 200 is realized by a mobile computer such as a tablet PC (Personal Computer), a smartphone, or a wearable device.

[0027] The inspection terminal 200 is provided so as to be able to communicate with the image inspection device 100, for example, by short-range wireless communication, etc. As shown in Fig. 1, the inspection terminal 200 exchanges information with the image inspection device 100 regarding image registration (simple instruction), inspection processing, picking result registration, etc., and provides an HMI (Human Machine Interface) related to this information to the worker.

[0028] 1 shows an example of a processing result screen of the inspection process as a screen displayed by the inspection terminal 200. The specific contents of these screens will be described later using FIG. 4 etc.

[0029] Furthermore, during inspection processing, the image inspection device 100 counts the items after adjusting the size of the items based on the detection results from the base model 103c. For example, the image inspection device 100 performs pixel-level analysis using segmentation to prevent erroneous detection in the counting results when the size of the detection frame differs depending on the orientation of the item relative to the camera 150, even for items of the same type. A specific example of this will be described later using Figure 5, etc.

[0030] As described above, in the image inspection method according to the embodiment, the image inspection device 100 acquires photographed images of picked items. The image inspection device 100 also performs simplified teaching on the base model 103c, which is a large-scale trained AI model, using at least one of the photographed images as a sample image. The image inspection device 100 then performs an inspection process using the simplified teaching on the base model 103c to detect and count items from the photographed images. During the inspection process, the image inspection device 100 adjusts at least the size of the items based on the detection results from the base model 103c before counting the items. Therefore, the image inspection method according to the embodiment enables easier and more accurate automation of inspection work.

[0031] The configuration of the image inspection system 1 to which the image inspection method according to the above-described embodiment is applied will be described in more detail below.

[0032] Fig. 2 is a block diagram of an image inspection device 100 according to an embodiment. Note that in Fig. 2 and Fig. 8 shown later, components necessary for explaining the features of this embodiment are shown as functional blocks, and descriptions of general components are omitted.

[0033] 2 and 8 are functional concepts and do not necessarily have to be physically configured as shown. For example, the specific form of distribution and integration of each functional block is not limited to that shown, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

[0034] In the description using FIGS. 2 and 8, the description of the components that have already been described may be simplified or omitted.

[0035] As shown in FIG. 2, the image inspection device 100 includes a first communication unit 101, a second communication unit 102, a storage unit 103, and a control unit 104.

[0036] The first communication unit 101 and the second communication unit 102 are realized by, for example, a network adapter etc. The first communication unit 101 is connected to the network N1 by wire or wirelessly, and transmits and receives information to and from the WMS10.

[0037] The second communication unit 102 is connected to the camera 150 and the inspection terminal 200 by wire or wirelessly, and transmits and receives information to and from the camera 150 and the inspection terminal 200 .

[0038] The storage unit 103 is realized by a storage device such as a random access memory (RAM), a read only memory (ROM), a flash memory, a hard disk drive, an optical disk drive, etc. In the example shown in Fig. 2, the storage unit 103 stores picking instruction information 103a, picking result information 103b, and a base model 103c.

[0039] The picking instruction information 103a stores picking instructions from the WMS 10 acquired by the first acquisition unit 104a, which will be described later. The picking result information 103b stores picking results, which are the results of the inspection process executed by the inspection processing unit 104f, which will be described later. The foundation model 103c has already been explained, so its explanation will be omitted here. Note that the foundation model 103c operates as a foundation AI (which can also be read as "large-scale trained AI") when loaded into the control unit 104.

[0040] The control unit 104 corresponds to a so-called processor or controller. The control unit 104 is realized by, for example, a central processing unit (CPU), a micro processing unit (MPU), or a graphical processing unit (GPU). The functions of the control unit 104 are realized by executing an image inspection program according to an embodiment (not shown) stored in the storage unit 103 using RAM as a working area. The control unit 104 can also be realized by, for example, an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0041] The control unit 104 has a first acquisition unit 104a, a transmission unit 104b, a second acquisition unit 104c, a simple teaching unit 104d, a detection unit 104e, and an inspection processing unit 104f, and realizes or executes the information processing functions and actions described below.

[0042] The first acquisition unit 104a acquires a picking instruction from the WMS 10 via the first communication unit 101. The first acquisition unit 104a also stores the acquired picking instruction in the picking instruction information 103a.

[0043] The transmission unit 104b acquires the picking result, which is the result of the inspection process by the inspection processing unit 104f, from the picking result information 103b, and transmits it to the WMS 10 via the first communication unit 101.

[0044] The second acquisition unit 104c acquires the captured image from the camera 150 via the second communication unit 102. The second acquisition unit 104c also acquires various pieces of information transmitted from the inspection terminal 200 via the second communication unit 102.

[0045] The simple teaching unit 104d performs simple teaching on the base model 103c using at least one of the photographed images acquired by the second acquisition unit 104c as a sample image. When performing this simple teaching, the simple teaching unit 104d inputs the sample image into the base model 103c before the simple teaching and acquires the detection result by the base AI.

[0046] Furthermore, the simple teaching unit 104d generates a simple teaching screen to be presented to the inspection terminal 200 based on the acquired detection results, and transmits it to the inspection terminal 200 via the second communication unit 102. Furthermore, the simple teaching unit 104d acquires various input values ​​to be input to the simple teaching screen in the inspection terminal 200 via the second communication unit 102. Furthermore, the simple teaching unit 104d reflects the various acquired input values ​​in the base model 103c.

[0047] Here, Fig. 3 is a diagram showing an example of a simple instruction screen. During simple instruction, the inspection terminal 200 displays a simple instruction screen as shown in Fig. 3. As shown in Fig. 3, the simple instruction screen presents inspection candidate areas that have been automatically detected by the base AI from the sample image. Each inspection candidate area includes a detection frame for each item shown in the sample image and a tag consisting of an inspection candidate ID and a score (detection accuracy).

[0048] For each of the presented candidate areas for examination, the operator enters the candidate area ID and type via the simple instruction screen as shown in Figure 3, and registers the area. This generates a registered data list in which each candidate area ID is associated with a type that serves as a label.

[0049] Then, when the operator operates the registration completion button, one-shot learning is performed in which the contents of the registered data list are reflected in the base model 103c. This makes it possible to easily link the characteristics of the object to be inspected to the base model 103c without collecting a large number of learning images or performing time-consuming annotation work.

[0050] Returning to the explanation of Fig. 2, the detection unit 104e inputs the image captured by the camera 150 acquired by the second acquisition unit 104c to the post-simple teaching base model 103c, and obtains the detection result of the article appearing in the captured image. The detection unit 104e also passes the obtained detection result to the inspection processing unit 104f.

[0051] The inspection processing unit 104f recognizes each item to be inspected based on the picking instructions included in the picking instruction information 103a and the detection results by the detection unit 104e, and then performs inspection processing to count the number of each item. The inspection processing unit 104f also generates a processing result screen based on the processing results of the inspection processing and transmits it to the inspection terminal 200 via the second communication unit 102. The inspection processing unit 104f also acquires various input values ​​to be input to the processing result screen in the inspection terminal 200 via the second communication unit 102. The inspection processing unit 104f also reflects the picking results based on the acquired various input values ​​in the picking result information 103b.

[0052] Here, Fig. 4 is a diagram showing an example of a processing result screen for the inspection process. During the inspection process, the inspection terminal 200 displays a processing result screen as shown in Fig. 4. As shown in Fig. 4, the processing result screen for the inspection process presents the results of counting after the infrastructure AI detects the inspection target from the captured image.

[0053] Figure 4 shows an example in which the picking instruction from WMS 10 is to pick 20 screws, but the picked items contain one nut and one extra screw. In this case, the processing result screen displays an error message such as "Picking instruction does not match. Please check the number of items picked," as shown in Figure 4. If the worker redoes the picking in response to this error (in this case, removing one nut and one screw) and presses the registration button, the picking result will be reflected in picking result information 103b and sent to WMS 10.

[0054] Returning to the explanation of Figure 2, the inspection processing unit 104f counts the items after adjusting the size of the items based on the detection results from the base model 103c during inspection processing. For example, the inspection processing unit 104f performs pixel-level analysis by segmentation to prevent false detections in the count results when the size of the detection frame differs depending on the orientation of the item relative to the camera 150, even for items of the same type.

[0055] A more detailed explanation will be given below. Fig. 5 is an explanatory diagram of size limitation by segmentation. As shown in Fig. 5, the detection results using deep learning models for object detection, including the base model 103c, will have variations in the size of the detection frame depending on the orientation of the object, even for the same type of object.

[0056] 5 shows an example in which, of detection frames BB1 and BB2 for the same screw, detection frame BB2 is larger than detection frame BB1 ​​(see the vertical inequality sign in the figure). To prevent false positives in the counting results in such cases, the inspection processing unit 104f extracts the area in which the item exists by segmentation, and treats items as belonging to the same type if their sizes are approximately equal at the pixel level (see the vertical equality sign in the figure). This makes it possible to prevent false positives in the counting results due to variations in the size of the detection frames.

[0057] Furthermore, there may be cases where an article of a certain type is mistakenly detected as an article of a different type due to the orientation of the article relative to the camera 150. In such cases, the image inspection device 100 may use the captured image where the mistaken detection occurred as a new sample image, and may perform additional registration to the base model 103c by simple teaching based on this sample image.

[0058] This will be explained in more detail. Fig. 6 is an explanatory diagram of additional registration. As shown in Fig. 6, for example, one of the screws is oriented vertically with its head facing downwards when viewed from the camera 150, so that the "screw" is mistakenly detected as a "nut."

[0059] In this case, the control unit 104 of the image inspection device 100 operates to be able to transition to a simple teaching screen (see FIG. 3) in which the captured image in which this erroneous detection occurred is used as a sample image. Then, based on the input values ​​entered into this simple teaching screen in the inspection terminal 200, the control unit 104 additionally registers the screw that was erroneously detected as a "nut" as the type "screw."

[0060] This makes it possible to easily prevent false detections due to the orientation of an article relative to the camera 150, for example, without preparing a large number of learning images.

[0061] Furthermore, during inspection work, items of various sizes are often mixed together on the picking cart 50. For this reason, during inspection work, when a large item and a small item are captured in a single captured image, it is desirable to avoid false detections such as detecting part of a large item as a small item.

[0062] Here, for operational reasons, if it is possible to inspect large and small items separately, the size of the items to be inspected may be specified so that only items that fit the specified size are counted.

[0063] A more specific explanation will be given below. Fig. 7 is an explanatory diagram of a case where the size of the inspection object can be specified. In this case, as shown in Fig. 7, the control unit 104 of the image inspection device 100 generates a screen on which the size of the item to be inspected can be specified, and presents it to the inspection terminal 200.

[0064] Then, the worker specifies the size of the inspection object during inspection, as shown in Figure 7. Note that the "size" referred to here corresponds to, for example, the area of ​​each detection frame in the entire captured image. Figure 7 shows an example in which this size can be specified relatively as "large," "medium," or "small."

[0065] If the inspection target is a "large" or "medium" sized item, such as a bundle of cables, the worker can specify "large" or "medium" to count the items of that size. Also, if the inspection target is a "small" sized item, such as a screw or nut, the worker can specify "small" to count the items of that size.

[0066] Note that "large," "medium," and "small" may be selectable using an OR condition, such as "large" or "medium," or may be selectable only as a single option. Furthermore, in terms of operation, it is not limited to cases where items of different sizes can be inspected separately by size, and when these are mixed, different sizes may be specified in stages for inspection, and the picking results may be integrated at the end.

[0067] In this way, by making it possible to specify the size of the items to be counted, it is possible to prevent erroneous detection due to differences in size.

[0068] Next, a description will be given of an example of the configuration of the inspection terminal 200. Fig. 8 is a block diagram of the inspection terminal 200 according to the embodiment.

[0069] As shown in FIG. 8, the inspection terminal 200 includes a communication unit 201, an HMI unit 202, a storage unit 203, and a control unit 204.

[0070] The communication unit 201 is realized by, for example, a network adapter, etc. The communication unit 201 is connected to the image inspection device 100 by wire or wirelessly, and transmits and receives information to and from the image inspection device 100.

[0071] The HMI unit 202 is a component that provides interface components related to input and output to the worker using the inspection terminal 200. The HMI unit 202 includes an input interface that accepts input operations from the worker. The input interface is realized by, for example, a touch panel. The input interface may also be realized by a keyboard, a mouse, a pen tablet, a microphone, etc. The input interface may also be realized by software components.

[0072] The HMI unit 202 also includes an output interface that presents visual and audio information to the operator. The output interface is realized by, for example, a display, a speaker, etc. The HMI unit 202 may also be configured such that the input and output interfaces are integrated into one unit, for example, by a touch panel display.

[0073] The storage unit 203 is realized by a storage device such as a RAM, a ROM, a flash memory, a hard disk drive, an optical disk drive, etc. In the example shown in Fig. 8, the storage unit 203 stores HMI information 203a. The HMI information 203a is information related to input / output control of the HMI unit 202 in the inspection terminal 200.

[0074] The control unit 204 corresponds to a so-called processor or controller. The control unit 204 is realized by, for example, a CPU, an MPU, a GPU, or the like. The functions of the control unit 204 are realized by executing an inspection terminal program (not shown) stored in the storage unit 203 using RAM as a work area. The control unit 204 can also be realized by, for example, an integrated circuit such as an ASIC or an FPGA.

[0075] The control unit 204 has an acquisition unit 204a and an HMI control unit 204b, and realizes or executes the functions and actions of information processing described below.

[0076] The acquisition unit 204a acquires various screens related to the inspection work transmitted from the image inspection device 100 via the communication unit 201. The HMI control unit 204b presents the screens acquired by the acquisition unit 204a to the HMI unit 202 based on the HMI information 203a. The HMI control unit 204b also acquires various input values ​​input by the worker via the HMI unit 202 and transmits them to the image inspection device 100 via the communication unit 201.

[0077] Next, a processing procedure executed by the image inspection system 1 according to the embodiment will be described with reference to Fig. 9. Fig. 9 is a diagram showing a processing sequence executed by the image inspection system according to the embodiment.

[0078] 9, first, the image inspection device 100 acquires a sample image for simple teaching from the camera 150 (step S101). Note that the image inspection device 100 may acquire the sample image for the first time based on a picking instruction from the WMS 10 shown in step S108.

[0079] The image inspection device 100 then inputs the acquired sample image to the base model 103c (step S102) and obtains a detection result from the base model 103c (step S103).The image inspection device 100 then generates a simple instruction screen based on the acquired detection result (step S104) and transmits it to the inspection terminal 200 (step S105).

[0080] The inspection terminal 200 accepts input from the worker on the simple teaching screen and transmits the various input values ​​as teaching information to the image inspection device 100 (step S106). The image inspection device 100 reflects the teaching information received from the inspection terminal 200 in the base model 103c (step S107) and ends the simple teaching.

[0081] Then, when the image inspection device 100 receives a picking instruction from the WMS 10 (step S108), it starts the inspection process (step S109). In the inspection process, the image inspection device 100 acquires a photographed image of the picked item from the camera 150 (step S110), inputs this to the base model 103c after simple teaching, and obtains a detection result by the base model 103c (steps S111, S112).

[0082] Then, the image inspection device 100 counts the inspection objects based on the detection results obtained by the substrate model 103c (step S113). At this time, the image inspection device 100 appropriately adjusts at least the size of the objects based on the detection results obtained by the substrate model 103c before counting the objects.

[0083] Then, the image inspection device 100 generates a processing result screen of the inspection process (step S114) and transmits it to the inspection terminal 200 (step S115). The inspection terminal 200 registers the results based on the worker's input on the processing result screen (step S116). Then, the image inspection device 100 transmits the picking results based on this result registration to the WMS 10 (step S117).

[0084] The image inspection device 100 and the inspection terminal 200 according to the above-described embodiments are realized by a computer 1000 configured as shown in Fig. 10, for example. The image inspection device 100 will be described as an example. Fig. 10 is a hardware configuration diagram showing an example of a computer that realizes the functions of the image inspection device 100. The computer 1000 includes a CPU 1100, a RAM 1200, a ROM 1300, a HDD (Hard Disk Drive) 1400, a communication interface (I / F) 1500, an input / output interface (I / F) 1600, and a media interface (I / F) 1700.

[0085] The CPU 1100 operates and controls each unit based on programs stored in the ROM 1300 or the HDD 1400. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 starts up, programs that depend on the hardware of the computer 1000, and the like.

[0086] HDD 1400 stores programs executed by CPU 1100 and data used by the programs. Communication interface 1500 receives data from other devices via a communication network (e.g., network N1) and sends the data to CPU 1100, and transmits data generated by CPU 1100 to other devices via the communication network.

[0087] The CPU 1100 controls input / output devices, i.e., output devices such as a display and a printer, and input devices such as a keyboard and a mouse, via the input / output interface 1600. The CPU 1100 acquires data from the input devices via the input / output interface 1600. The CPU 1100 also outputs generated data to the output devices via the input / output interface 1600.

[0088] Media interface 1700 reads a program or data stored in recording medium 1800 and provides it to CPU 1100 via RAM 1200. CPU 1100 loads the program from recording medium 1800 onto RAM 1200 via media interface 1700 and executes the loaded program. Recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase Change Rewritable Disc), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0089] For example, when the computer 1000 functions as the image inspection device 100 according to the embodiment, the CPU 1100 of the computer 1000 executes programs loaded onto the RAM 1200 to realize the functions of the control unit 104. In addition, the HDD 1400 stores data in the storage unit 103. The CPU 1100 of the computer 1000 reads and executes these programs from the recording medium 1800, but as another example, the CPU 1100 may acquire these programs from another device via a communication network.

[0090] As described above, the image inspection device 100 according to the embodiment includes a control unit 104. The control unit 104 acquires captured images of picked items. The control unit 104 also provides simple instruction to the base model 103c, which is a large-scale trained AI model, using at least one of the captured images as a sample image. The control unit 104 then uses the simply taught base model 103c to execute an inspection process that detects and counts the items from the captured images. During the inspection process, the control unit 104 adjusts at least the size of the items based on the detection results from the base model 103c, and then counts the items.

[0091] Therefore, the image inspection device 100 according to the embodiment can automate inspection work more easily and with higher accuracy.

[0092] In the above-described embodiment, the picking cart 50 is used as an example of material handling equipment for picking, but the present invention is not limited to this, and a picking robot or the like may also be used.

[0093] Further advantages and modifications will readily occur to those skilled in the art. Therefore, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents. [Explanation of symbols]

[0094] 1. Image inspection system 50 Picking Carts 100 Image inspection device 101 First Communications Department 102 Second Communications Department 103 Storage section 103c base model 104 Control Unit 104a 1st acquisition part 104b Transmitter 104c 2nd acquisition part 104d Simple teaching section 104e Detection unit 104f Inspection Department 150 cameras 200 Inspection terminal

Claims

1. Acquire a photograph of the picked item, A basic model, which is a large-scale trained AI model, is simply taught using at least one of the captured images as a sample image; performing an inspection process for detecting and counting the items from the captured image using the base model that has been simply taught; a control unit that, during the inspection process, counts the items after adjusting at least the size of the items based on the detection result obtained by the base model; An image inspection device comprising:

2. The control unit performing the simple teaching by one-shot learning or few-shot learning using the sample image; 2. The image inspection device according to claim 1.

3. The control unit extracting an area where the object exists by segmentation from the detection result obtained by the base model, and treating the object as belonging to the same type when the size is approximately the same at the pixel level; 2. The image inspection device according to claim 1.

4. The control unit When an erroneous detection of the object occurs in the detection result by the base model, the captured image in which the erroneous detection occurred is added as a new sample image, thereby enabling the simplified teaching.

2. The image inspection device according to claim 1.

5. The control unit When the size of the object in the detection result by the base model is specified, the object corresponding to the specified size is counted.

2. The image inspection device according to claim 1.

6. It is installed on material handling equipment used for picking, 6. The image inspection device according to claim 1, wherein the image inspection device is a device for inspecting an object.

7. A computer-implemented image inspection method, comprising: Acquiring a photographed image of the picked item; A basic model that is a large-scale trained AI model is simply taught using at least one of the captured images as a sample image; performing an inspection process for detecting and counting the items from the captured image using the base model that has been simply taught; During the inspection process, counting the items after adjusting at least the size of the items based on the detection result of the base model; An image inspection method comprising:

8. Acquiring a photographed image of the picked item; A basic model that is a large-scale trained AI model is simply taught using at least one of the captured images as a sample image; performing an inspection process for detecting and counting the items from the captured image using the base model that has been simply taught; During the inspection process, counting the items after adjusting at least the size of the items based on the detection result of the base model; An image inspection program characterized by causing a computer to execute the above.

Images