Image inspection device and image inspection method

Abstract

To provide an image inspection device and image inspection method that can more accurately inspect various types of inspected items (parts to be inspected). [Solution] The image inspection device comprises a variety database in which variety information linked to images and features of defect-free parts is registered, and an LLM variety inspection unit that inputs an image of the product to be inspected, which is the part to be inspected, and a prompt created by referring to the variety database to identify the variety of the product to be inspected, into an LLM (Large Language Models), and identifies the variety of the product to be inspected based on the output of the LLM.

Description

[Technical Field]

[0001] The present disclosure relates to an image inspection apparatus and an image inspection method. [Background Art]

[0002] Patent Document 1 discloses a system that uses a large-scale language model trained on a large amount of up-to-date medical knowledge to support medical staff in creating order documents and record documents in electronic medical records, thereby supporting efficient medical activities. This system includes an electronic medical record item extracting means that extracts information of items necessary for processing documents of an arbitrarily specified document category from an electronic medical record, and a transcribing means that transcribes the extracted electronic medical record content into a prompt for a large-scale language model and provides the prompt to the large-scale language model.

[0003] For example, in manufacturing sites and the like, utilization of large-scale language models such as those described above is being studied to automate or improve the efficiency of inspection of a wide variety of products and components. [Prior Art Literature] [Patent Literature]

[0004] [Patent Document 1] Japanese Patent No. 7441391 [Summary of the Invention] [Problem to be Solved by the Invention]

[0005] The present disclosure has been conceived in view of the aforementioned conventional circumstances, and an object thereof is to provide an image inspection apparatus and an image inspection method capable of more accurately inspecting inspection products (components to be inspected) of various product types. [Means for Solving the Problem]

[0006] This disclosure provides an image inspection device comprising: a variety database in which variety information linked to images and characteristics of defect-free parts is registered; and an LLM (Large Language Models) variety inspection unit that inputs an image of an inspected part, which is the part to be inspected, and a prompt created by referring to the variety database to identify the variety of the inspected part, to an LLM, and identifies the variety of the inspected part based on the output of the LLM.

[0007] This disclosure provides an image inspection device comprising: a defect database in which defect information is registered, which is linked to images of defective parts and the characteristics of the defects; and an LLM (Large Language Models) defect inspection unit that inputs an image of an inspected product, which is a part to be inspected, and prompts created by referring to the defect database to detect defects in the inspected product, to an LLM, and inspects the inspected product based on the output of the LLM.

[0008] This disclosure provides an image inspection method in which an image of an item to be inspected, which is a part to be inspected, and a prompt created by referring to a variety database in which variety information linked to images and features of defect-free parts is registered in order to identify the variety of the item to be inspected are input to an LLM (Large Language Model), and the variety of the item to be inspected is identified based on the output of the LLM.

[0009] This disclosure provides an image inspection device comprising: a database of fallen objects on a road in which images of fallen objects on the road are registered; an AI model for detecting fallen objects on a road based on an image that includes the road; and an LLM (Large Language Model) fallen object inspection unit that, if the AI ​​model for detecting fallen objects cannot determine that there are fallen objects on the road, inputs the image and a prompt created by referring to the database of fallen objects to detect the fallen objects to an LLM, and determines the presence or absence of the fallen objects based on the output of the LLM. [Effects of the Invention]

[0010] According to this disclosure, it is possible to inspect various types of inspected items (parts to be inspected) more accurately. [Brief explanation of the drawing]

[0011] [Figure 1] Block diagram showing an example configuration of an image inspection system including an image inspection device according to Embodiment 1. [Figure 2] Block diagram showing an example of the hardware configuration of the image inspection apparatus according to Embodiment 1. [Figure 3] Block diagram showing an example of the functional configuration of the image inspection device according to Embodiment 1. [Figure 4] This diagram illustrates the variety identification process performed by the LLM variety inspection unit in the image inspection apparatus according to Embodiment 1. [Figure 5] This diagram illustrates the inspection process performed by the LLM defect inspection unit in the image inspection apparatus according to Embodiment 1. [Figure 6] This figure shows an example of displaying inspection results on the display unit in the image inspection apparatus according to Embodiment 1. [Figure 7] A flowchart illustrating the processing flow of the image inspection method in the image inspection apparatus according to Embodiment 1 when the type of inspected item flowing down the lane is not set. [Figure 8] A flowchart illustrating the processing flow of the image inspection method in the image inspection apparatus according to Embodiment 1 when the type of inspected item flowing down the lane is not set. [Figure 9] A flowchart illustrating the processing flow of the image inspection method when the types of inspected items flowing along the lane are set in the image inspection apparatus according to Embodiment 1. [Figure 10] A flowchart illustrating the processing flow of the image inspection method when the types of inspected items flowing along the lane are set in the image inspection apparatus according to Embodiment 1. [Figure 11] Block diagram showing an example of the functional configuration of an image inspection device according to a modified embodiment of Embodiment 1. [Figure 12] A diagram illustrating the outline of an image inspection device according to a modified example of Embodiment 1. [Figure 13] Flowchart illustrating the processing flow of a falling object inspection method in an image inspection apparatus according to a modification of the first embodiment MODES FOR CARRYING OUT THE INVENTION

[0012] Background Leading to the Present Disclosure Conventionally, an AI visual inspection system has been proposed that captures images of inspection target components (inspection articles) flowing on a lane using a camera or the like, and inspects the inspection articles with respect to the captured images of each inspection article using AI (Artificial Intelligence) for image analysis. This AI visual inspection system requires AI learning using data such as product type information and defect information for each product type of inspection article. Therefore, when inspecting many product types, man-hours are required for training the AI for image analysis accordingly, which makes it unfavorable for multi-product-type inspection. Further, when an unknown product type is included among the inspection articles, no training data for that product type exists in advance for the AI for image analysis, so that the aforementioned AI visual inspection system has difficulty in inspecting unknown product types. Further, when inspecting products using an LLM (Large Language Model) instead of AI for image analysis, it is not preferable because inspection takes a long time.

[0013] Therefore, in the following embodiments, an image inspection apparatus and an image inspection method capable of more accurately inspecting inspection articles (components to be inspected) of various product types will be described.

[0014] Hereinafter, embodiments that specifically disclose the image inspection apparatus and the image inspection method according to the present disclosure will be described in detail with appropriate reference to the drawings. However, unnecessary detailed description may be omitted. For example, detailed description of already well-known matters and repeated description of substantially the same configuration may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding for those skilled in the art. The accompanying drawings and the following description are provided for those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0015] (Embodiment 1) <System Configuration> FIG. 1 is a block diagram showing a configuration example of an image inspection system IIS including an image inspection apparatus 10 according to Embodiment 1. The system configuration shown in FIG. 1 is an example; one apparatus may be divided into a plurality of components, or a plurality of apparatuses may be integrated into one component. Further, a plurality of the apparatuses shown in FIG. 1 may be provided. Furthermore, the processing entities described below are examples, and a part of the functions of a certain apparatus may be implemented as functions of another apparatus.

[0016] The image inspection system IIS is configured to include a lane 1, a camera 2, a lane control device 3, and the image inspection apparatus 10. Each component constituting the image inspection system IIS is configured to be able to communicate with each other via a network, for example.

[0017] The lane 1 is a conveying device such as a belt conveyor or a roller conveyor that conveys a component to be inspected (also referred to as an "inspection product"). Inspection products of a pre-associated product type (also referred to as a "set product type") or inspection products in which a plurality of different product types are mixed flow through the lane 1.

[0018] The camera 2 is an imaging device for imaging an inspection product flowing on the lane 1. The image inspection apparatus 10 analyzes an image of the inspection product captured by the camera 2 and performs inspection of the inspection product (see description below).

[0019] The lane control device 3 controls the lane 1 based on an inspection result obtained by the image inspection apparatus 10. As an example, the lane control device 3 controls the lane 1 by switching the conveyance destination so as to remove an inspection product determined to be a defective product by the image inspection apparatus 10 from the line. Similarly, the lane control device 3 controls the lane 1 by switching the conveyance destination so as to convey an inspection product determined to be a non-defective product by the image inspection apparatus 10 to a work site for the next process.

[0020] In this specification, we use wires with connectors as an example of "inspection items," but the inspection items are not limited to these as long as they can be moved along lane 1.

[0021] <Example of hardware configuration for an image inspection system> Figure 2 is a block diagram showing an example of the hardware configuration of the image inspection apparatus 10 according to Embodiment 1. The hardware configuration of the image inspection apparatus 10 described herein is an example, and some parts may be omitted or other parts added as needed.

[0022] The image inspection device 10 comprises a control unit 11, a storage unit 12, a communication unit 13, an input unit 14, and a display unit 15. Each component is configured to communicate via an internal interface 16.

[0023] The control unit 11 may be configured using, for example, a CPU (Central Processing Unit), a GPU (Graphical Processing Unit), an MPU (Micro Processing Unit), a DSP (Digital Signal Processor), or an FPGA (Field-Programmable Gate Array). The control unit 11 implements various functions described later by, for example, referring to various databases stored in the storage unit 12 or reading programs.

[0024] The memory unit 12 is a memory unit for storing various data and programs, and may consist of volatile / non-volatile memory units such as RAM (Random Access Memory), ROM (Read Only Memory), and HDD (Hard Disk Drive). The memory unit 12 stores a variety database 51 (Figure 4, see below), a defect database 52 (Figure 5, see below), and the like.

[0025] The communication unit 13 is an interface for communicating with external devices via a network. The communication standards supported by the communication unit 13 are not particularly limited and can be wired or wireless. Furthermore, it may support multiple communication standards. Therefore, the network may be configured by combining networks using multiple communication standards.

[0026] The input unit 14 accepts, for example, operations and instructions from an inspector. The input unit 14 may consist of a mouse, keyboard, touch panel display, or the like.

[0027] The display unit 15 displays various user interfaces to the user. The display unit 15 may consist of a liquid crystal display, a touch panel display, a lamp, etc.

[0028] <Example of Functional Configuration of an Image Inspection System> Figure 3 is a block diagram showing an example of the functional configuration of the image inspection apparatus 10 according to Embodiment 1. Each part shown in Figure 3 may be realized, for example, by the control unit 11 reading a program stored in the storage unit 12, or by providing dedicated hardware.

[0029] Note that the configuration shown in Figure 3 is just one example; the parts shown in Figure 3 may be combined into a single configuration, or they may be further divided into more detailed configurations. Also, the links and arrows in Figure 3 show an example of data transmission and reception, but this connection configuration is not the only one that may be used; other connections and linkages may also be used.

[0030] The image inspection device 10 is comprised of an image acquisition unit 21, an inspection location extraction AI model 22, a product classification AI model 23, a product matching determination unit 24, a defect inspection AI model 25, an LLM product inspection unit 26, an LLM defect inspection unit 27, a retraining data generation unit 28, a defective product sorting unit 29, a good product sorting unit 30, an inspection result display unit 31, a final confirmation reception unit 32, an inspection location extraction AI model creation unit 33, a product classification AI model creation unit 34, a defect inspection AI model creation unit 35, a product identification few-shot prompt generation unit 36, a product database management unit 37, a product identification prompt input unit 38, a defect detection few-shot prompt generation unit 39, a defect database management unit 40, and a defect detection prompt input unit 41.

[0031] The image acquisition unit 21 acquires images of the inspected items moving along lane 1, which are captured by camera 2, from camera 2 when inspecting the inspected items.

[0032] The inspection area extraction AI model 22 is a pre-trained AI model for extracting inspection areas from images of inspected products, and extracts inspection areas from images of inspected products acquired by the image acquisition unit 21. Alternatively, instead of the inspection area extraction AI model 22, methods such as coordinate extraction or image processing (pattern matching) may be used for extracting inspection areas. In this case, the inspection area extraction AI model creation unit 33 is omitted.

[0033] The variety classification AI model 23 is a trained AI model for classifying (identifying) the variety of an inspected product based on the image IPI of the inspected product. It classifies (identifies) the variety of an inspected product based on the image IPI of the inspected product from which the inspection locations have been extracted by the inspection location extraction AI model 22. The variety classification AI model 23 may also have the functionality of the inspection location extraction AI model 22. In this case, the inspection location extraction AI model 22 and the inspection location extraction AI model creation unit 33 are omitted.

[0034] The variety matching determination unit 24 determines whether the variety of the inspected product flowing through lane 1 matches the set variety for lane 1, provided that the variety of the inspected product is set and the variety classification AI model 23 is able to classify the variety of the inspected product. If the variety of the inspected product flowing through lane 1 is not set, the determination process by the variety matching determination unit 24 is unnecessary.

[0035] Defect inspection AI model 25 is a trained AI model for detecting defects in inspected products based on image IPI of the inspected product, and detects defects in the inspected product based on image IPI of the inspected product if any one of the following conditions (1) to (3) is met. (1) When the variety of the product to be inspected flowing through lane 1 is set, and the variety matching determination unit 24 determines that the variety of the product to be inspected matches the set variety. (2) When the variety of the inspected items flowing through lane 1 is not set (for example, when multiple different varieties of inspected items flow through lane 1), and the variety classification AI model 23 is able to classify the variety of the inspected items. (3) When the LLM variety inspection unit 26 determines, based on the output of the LLM, that the variety of the inspected product is an existing variety registered in the variety database 51.

[0036] The LLM variety inspection unit 26 inputs an image IPI of the product to be inspected and a prompt to identify the variety of the product (also called the "variety identification prompt Pt1") to the LLM 100, and identifies the variety of the product based on the output of the LLM 100 (see Figure 4, described later). The LLM variety inspection unit 26 identifies the variety of the product to be inspected as described above if either (4) or (5) below applies. (4) When the variety classification AI model 23 cannot classify the variety of the inspected product (5) When the variety of the product to be inspected flowing through lane 1 is set, and the variety matching determination unit 24 determines that the variety of the product to be inspected does not match the set variety.

[0037] If the LLM Variety Inspection Unit 26 determines, based on the output of the LLM 100, that the variety of the inspected product is a new variety not registered in the variety database 51, the LLM Defect Inspection Unit 27 inputs a prompt (also called "Defect Detection Prompt Pt2") to the LLM 100 for detecting defects in the inspected product determined to be a new variety (including an image IPI of the inspected product), and inspects the inspected product based on the output of the LLM 100 (see Figure 5, described later).

[0038] The retraining data generation unit 28 generates training data for use in retraining the variety classification AI model 23, which includes image IPI of the inspected product determined to be a new variety and variety information linked to its features, when the LLM variety inspection unit 26 determines, based on the output of the LLM 100, that the variety of the inspected product is a new variety not registered in the variety database 51. The retraining data generation unit 28 also generates training data for use in retraining the defect inspection AI model 25, which includes image IPI of the inspected product determined to be a new variety and defect information linked to the features of the defects, when the LLM variety inspection unit 26 determines, based on the output of the LLM 100, that the variety of the inspected product is a new variety not registered in the variety database 51.

[0039] The defective product sorting unit 29 generates a control signal to control lane 1 to sort defective products when the defect inspection AI model 25 or LLM defect inspection unit 27 detects a defect in an inspected product, i.e., determines that the inspected product is defective, and outputs it to the lane control device 3.

[0040] The good product sorting unit 30 generates a control signal to control lane 1 to sort good products when the defect inspection AI model 25 or LLM defect inspection unit 27 determines that the inspected product is good, i.e., the inspected product is good, and outputs it to the lane control device 3.

[0041] The inspection result display unit 31 generates display information regarding the inspection results of the inspected product and outputs it to the display unit 15.

[0042] The final confirmation reception unit 32 generates display information to accept the final confirmation by the inspector and outputs it to the display unit 15.

[0043] The inspection area extraction AI model creation unit 33 receives training data from a training data generation unit (not shown) or a retraining data generation unit 28, and creates an inspection area extraction AI model 22 by training (AI learning) using the training data with a neural network or deep learning.

[0044] The variety classification AI model creation unit 34 receives training data from a training data generation unit (not shown) or a retraining data generation unit 28, and creates a variety classification AI model 23 by training (AI learning) using the training data with a neural network or deep learning.

[0045] The defect inspection AI model creation unit 35 receives training data from a training data generation unit (not shown) or a retraining data generation unit 28, and creates a defect inspection AI model 25 by training (AI learning) using the training data with a neural network or deep learning.

[0046] The variety identification few-shot prompt generation unit 36 ​​generates variety identification few-shot prompts based on the classification results of the variety classification AI model 23 or the determination results of the variety matching determination unit 24, etc.

[0047] The Variety DB Management Unit 37 manages the Variety Database 51. For example, if the LLM Variety Inspection Unit 26 determines, based on the output of the LLM 100, that the variety of the inspected product is a new variety not registered in the Variety Database 51, the Variety DB Management Unit 37 registers the variety information, linked to the image IPI and characteristics of the inspected product determined to be a new variety, into the Variety Database 51.

[0048] The variety identification prompt input unit 38 generates a variety identification prompt Pt1 based on the variety identification few-shot prompt obtained from the variety identification few-shot prompt generation unit 36 ​​and the variety information obtained from the variety DB management unit 37, and outputs it to the LLM variety inspection unit 26.

[0049] The Few-shot prompt generation unit for defect detection 39 generates a Few-shot prompt for defect detection based on the inspection result of the LLM product type inspection unit 26 and the like.

[0050] The defect DB management unit 40 manages the defect database 52. For example, when the LLM defect inspection unit 27 detects a defect in an inspection product based on the output of the LLM 100, the defect DB management unit 40 registers, in the defect database 52, defect information in which the image IPI and features of the inspection product in which the defect was detected are associated with each other.

[0051] The defect detection prompt input unit 41 generates a defect detection prompt Pt2 based on the Few-shot prompt for defect detection acquired from the Few-shot prompt generation unit for defect detection 39 and the defect information acquired from the defect DB management unit 40, and outputs the defect detection prompt Pt2 to the LLM defect inspection unit 27.

[0052] <Product Type Identification Processing by LLM Product Type Inspection Unit> FIG. 4 is a diagram illustrating product type identification processing performed by the LLM product type inspection unit 26 in the image inspection apparatus 10 according to the first embodiment.

[0053] [Product Type Database] In the product type database 51, product type information in which images of non-defective components (images of normal components) and features thereof are associated with each other is registered. In this example, as shown in FIG. 4, the "image of a non-defective component" is an image of a component from which an inspection location has been extracted, and the "features of a non-defective component" are component features such as the number of pins and wiring color. The "product type information" is information in which "images and features of a non-defective component" are associated with each product type of the component. That is, product type information is associated with each of a plurality of different product types of components to be inspected, and this product type information is registered in the product type database 51.

[0054] [Prompt for Product Type Identification] The prompt Pt1 for product type identification is a prompt for identifying the product type of an inspection item, and is generated based on, for example, instructions. For example, as the prompt Pt1 for product type identification, the instruction content is described as "This lane is product type 1. Please inspect the incoming images based on the image of the non-defective product. If there is any discrepancy, the result will be NG. Pay attention to the front and back surfaces. Also consider the color sequence. Example output: "Result: OK / NG, Number of pins: 3, Colors: red, yellow, blue, Orientation: front"". Note that the above example assumes a case where the product type of the inspection item flowing on lane 1 is set. For example, if the product type of the inspection item flowing on the lane is not set, the instruction content is described as "Please inspect the incoming images. Pay attention to the front and back surfaces. Also consider the color sequence. Example output: "Result: Product type 3, Number of pins: 3, Colors: yellow, black, red, Orientation: front"".

[0055] [Product Type Identification Processing] For the product type identification processing performed by the LLM product type inspection unit 26, first, the image IPI of the inspection item, the product type information in the product type database 51, and the prompt Pt1 for product type identification are input to the LLM 100. Then, output information OI1 is generated based on the input information and output to the LLM product type inspection unit 26. Thereby, the LLM product type inspection unit 26 identifies the product type of the inspection item based on the output (output information OI1) of the LLM 100.

[0056] <Inspection Processing by LLM Defect Inspection Unit> FIG. 5 is a diagram illustrating the inspection processing by the LLM defect inspection unit 27 in the image inspection apparatus 10 according to the first embodiment.

[0057] [Defect Database] In the defect database 52, defect information in which images of defective parts and defect features are associated is registered. In this example, as shown in FIG. 5, the "image of a defective part" is an image of a defective product in which the defect is captured, and "the defect features of the defective part" are features such as the type of defect and what kind of defect it is. That is, regardless of product type, defect information that can generally occur in parts of all product types is registered in the defect database 52.

[0058] [Prompt for defect detection] The defect detection prompt Pt2 is a prompt for detecting defects in inspected products that have been determined to be a new type by the LLM variety inspection unit 26. For example, the defect detection prompt Pt2 may contain instructions such as, "Inspect the incoming images based on the image of the defective product. If even one defect is found, it will be considered NG. Pay attention to the characteristics of the defect as well. Example output: 'Result: OK / NG, Defect characteristics: Crack'."

[0059] [Inspection process] The inspection process by the LLM defect inspection unit 27 begins by inputting the image IPI of the product to be inspected, the defective product information from the defect database 52, and the defect detection prompt Pt2 to the LLM 100. Based on the input information, output information OI2 is generated and output to the LLM defect inspection unit 27. The LLM defect inspection unit 27 then identifies the type of product to be inspected based on the output (output information OI2) from the LLM 100.

[0060] <Example of how test results are displayed> Figure 6 shows an example of the display of inspection results on the display unit 15 in the image inspection device 10 according to Embodiment 1. The inspection results of the inspected product performed by the image inspection device 10 are displayed on the display unit 15, for example, as shown in the figure. Various items are displayed, such as "image of the inspected product," "inspection result (OK / NG in this example)," "comments on the inspection result (NG points in this example)," "confirmation items," "confirmation of registration to the database," and "generation of retraining data."

[0061] <Example of the first operating procedure for an image inspection device> The first example of the operation procedure of the image inspection device 10 according to Embodiment 1 will be described below with reference to Figures 7 and 8. Figure 7 is a flowchart illustrating the processing flow of the image inspection method in the image inspection device 10 according to Embodiment 1 when the type of inspected product flowing through lane 1 is not set. Figure 8 is a flowchart illustrating the processing flow of the image inspection method in the image inspection device 10 according to Embodiment 1 when the type of inspected product flowing through lane 1 is not set.

[0062] The image inspection device 10 acquires an image of the inspected product from the camera 2 using the image acquisition unit 21 (Sp101).

[0063] The image inspection device 10 extracts inspection areas from the image of the inspected product acquired by the image acquisition unit 21 using the inspection area extraction AI model 22 (Sp102).

[0064] The image inspection device 10 classifies the variety of the inspected product based on the image IPI of the inspected product, from which the inspection area has been extracted by the inspection area extraction AI model 22, using the variety classification AI model 23 (Sp103).

[0065] The image inspection device 10 determines whether the variety of the inspected item can be classified using the variety classification AI model 23 (Sp104). Here, the determination of whether the variety of the inspected item can be classified is made based on whether the reliability of the classification result by the variety classification AI model 23 is above a predetermined threshold.

[0066] If the type of product to be inspected can be classified, i.e., the confidence level is above the threshold (YES in Sp104), the image inspection device 10 inspects the product using the defect inspection AI model 25 (Sp105). Here, the defect inspection AI model 25 detects defects in the product based on the image IPI of the product. Then, the image inspection device 10 proceeds to step Sp106. On the other hand, if the type of product to be inspected cannot be classified, i.e., the confidence level is below the threshold (NO in Sp104), the image inspection device 10 proceeds to step Sp111.

[0067] The image inspection device 10 determines whether or not a defect has been detected in the inspected product using the defect inspection AI model 25 (Sp106).

[0068] If a defect is detected in the inspected product (YES in Sp106), the image inspection device 10 separates the defective product using the defective product sorting unit 29 (Sp107). Then, the image inspection device 10 proceeds to step Sp109.

[0069] On the other hand, if no defects are detected in the inspected product (NO in Sp106), the image inspection device 10 separates the good products using the good product sorting unit 30 (Sp108). Then, the image inspection device 10 proceeds to step Sp109.

[0070] The image inspection device 10 displays the inspection results on the display unit 15 (Sp109). Here, display information regarding the inspection results of the inspected product, generated by the inspection result display unit 31, is displayed on the display unit 15 (see Figure 6).

[0071] The image inspection device 10 accepts the final confirmation by the inspector (Sp110). Here, the display information indicating acceptance of the final confirmation by the inspector, generated by the final confirmation acceptance unit 32, is displayed on the display unit 15. Then, this processing flow ends.

[0072] If the variety of the inspected item cannot be classified, i.e., the confidence level is below the threshold (NO in Sp4), the image inspection device 10 performs a variety identification process for the inspected item using the LLM variety inspection unit 26 (Sp111).

[0073] The image inspection device 10 uses the LLM variety inspection unit 26 to determine whether the variety of the inspected product is a new variety or not based on the output of the LLM 100 (Sp112). Here, the LLM variety inspection unit 26 determines whether the variety of the inspected product is a new variety not registered in the variety database 51, or an existing variety registered in the variety database 51.

[0074] If the variety of the inspected product is a new variety (YES in Sp112), the image inspection device 10 registers the inspected product, which has been determined to be a new variety, in the variety database 51 via the variety DB management unit 37 (Sp113). Then, the image inspection device 10 proceeds to step Sp114. On the other hand, if the variety of the inspected product is an existing variety (NO in Sp112), the image inspection device 10 proceeds to step Sp105.

[0075] The image inspection device 10 generates training data using the retraining data generation unit 28, which includes image IPIs of inspected items determined to be new varieties and variety information linked to their features, for use in retraining the variety classification AI model 23 (Sp114).

[0076] The image inspection device 10 performs inspection processing of the inspected product using the LLM defect inspection unit 27 (Sp115).

[0077] The image inspection device 10 determines whether or not a defect has been detected in the inspected product by the LLM defect inspection unit 27 (Sp116).

[0078] If a defect is detected in the inspected product (YES in Sp116), the image inspection device 10 registers the inspected product, which has been determined to be a new type and defective, in the defect database 52 by the defect DB management unit 40 (Sp117). Then, the image inspection device 10 proceeds to step Sp118. On the other hand, if no defect is detected in the inspected product (NO in Sp116), the image inspection device 10 proceeds to step Sp108.

[0079] The image inspection device 10 generates training data using the retraining data generation unit 28, which includes images of newly identified defective inspected items and defect information linked to the characteristics of the defects, for use in retraining the defect inspection AI model 25 (Sp118). The image inspection device 10 then proceeds to step Sp107.

[0080] The first example of the operation procedure of the image inspection device 10 according to Embodiment 1 has been described above. As described above, the variety of inspected items can be quickly classified by the variety classification AI model 23, and the variety of inspected items that could not be classified by the variety classification AI model 23 can be identified by the LLM variety inspection unit 26, so that inspected items can be inspected quickly and accurately.

[0081] (Example of the second operating procedure for the image inspection device) The following describes a second example of the operation procedure of the image inspection device 10 according to Embodiment 1, using Figures 9 and 10. Figure 9 is a flowchart illustrating the processing flow of the image inspection method when the type of inspected product flowing through lane 1 is set in the image inspection device 10 according to Embodiment 1. Figure 10 is a flowchart illustrating the processing flow of the image inspection method when the type of inspected product flowing through lane 1 is set in the image inspection device 10 according to Embodiment 1.

[0082] The image inspection device 10 acquires an image of the inspected product from the camera 2 using the image acquisition unit 21 (Sp121).

[0083] The image inspection device 10 extracts inspection areas from the image of the inspected product acquired by the image acquisition unit 21 using the inspection area extraction AI model 22 (Sp122).

[0084] The image inspection device 10 classifies the variety of the inspected product based on the image IPI of the inspected product, from which the inspection area has been extracted by the inspection area extraction AI model 22, using the variety classification AI model 23 (Sp123).

[0085] The image inspection device 10 determines whether the variety of the inspected item can be classified using the variety classification AI model 23 (Sp124). Here, the determination of whether the variety of the inspected item can be classified is made based on whether the reliability of the classification result by the variety classification AI model 23 is above a predetermined threshold.

[0086] If the variety of the inspected item can be classified, i.e., the confidence level is above the threshold (YES in Sp124), the image inspection device 10 uses the variety matching determination unit 24 to determine whether the classification result matches the set variety (Sp125). Here, it is determined whether the variety of the inspected item identified by the variety classification AI model 23, i.e., the classification result, matches the set variety of the inspected item associated with lane 1. On the other hand, if the variety of the inspected item cannot be classified, i.e., the confidence level is below the threshold (NO in Sp124), the image inspection device 10 proceeds to step Sp132.

[0087] If the classification result matches the set product type (YES in Sp125), the image inspection device 10 inspects the product using the defect inspection AI model 25 (Sp126). Then, the image inspection device 10 proceeds to step Sp127. On the other hand, if the classification result does not match the set product type (NO in Sp125), the image inspection device 10 proceeds to step Sp132.

[0088] The image inspection device 10 determines whether or not a defect has been detected in the inspected product using the defect inspection AI model 25 (Sp127).

[0089] If a defect is detected in the inspected product (YES in Sp127), the image inspection device 10 separates the defective product using the defective product sorting unit 29 (Sp128). Then, the image inspection device 10 proceeds to step Sp130.

[0090] On the other hand, if no defects are detected in the inspected product (NO in Sp127), the image inspection device 10 separates the good products using the good product sorting unit 30 (Sp129). Then, the image inspection device 10 proceeds to step Sp130.

[0091] The image inspection device 10 displays the inspection results on the display unit 15 (Sp130). Here, display information regarding the inspection results of the inspected product, generated by the inspection result display unit 31, is displayed on the display unit 15 (see Figure 6).

[0092] The image inspection device 10 accepts the final confirmation by the inspector (Sp131). Here, the display information indicating acceptance of the final confirmation by the inspector, generated by the final confirmation acceptance unit 32, is displayed on the display unit 15. Then, this processing flow ends.

[0093] If the variety of the inspected item cannot be classified, i.e., the confidence level is below the threshold (NO in Sp124), or if the classification result does not match the set variety (NO in Sp125), the image inspection device 10 performs variety identification processing of the inspected item using the LLM variety inspection unit 26 (Sp132).

[0094] The image inspection device 10 uses the LLM variety inspection unit 26 to determine whether the variety of the inspected product is a new variety or not based on the output of the LLM 100 (Sp133). Here, the LLM variety inspection unit 26 determines whether the variety of the inspected product is a new variety not registered in the variety database 51, or an existing variety registered in the variety database 51.

[0095] If the variety of the inspected product is a new variety (YES in Sp133), the image inspection device 10 registers the inspected product, which has been determined to be a new variety, in the variety database 51 via the variety DB management unit 37 (Sp134). Then, the image inspection device 10 proceeds to step Sp135. On the other hand, if the variety of the inspected product is an existing variety (NO in Sp133), the image inspection device 10 proceeds to step Sp136.

[0096] The image inspection device 10 generates training data using the retraining data generation unit 28, which includes image data of inspected items determined to be new varieties and variety information linked to their characteristics, for use in retraining the variety classification AI model 23 (Sp135). Then, the image inspection device 10 proceeds to step Sp128.

[0097] If the variety of the product being inspected is an existing variety (NO in Sp133), the image inspection device 10 determines whether the identified result matches the set variety using the LLM variety inspection unit 26 (Sp136). Here, the LLM variety inspection unit 26 determines whether the variety of the product being inspected, i.e., the identified result, matches the set variety of the product being inspected that is associated with lane 1.

[0098] If the identification result matches the set variety (YES in Sp136), the image inspection device 10 proceeds to step Sp126. On the other hand, if the identification result does not match the set variety (NO in Sp136), the image inspection device 10 proceeds to step Sp128.

[0099] The second example of the operation procedure of the image inspection device 10 according to Embodiment 1 has been described above. As described above, the variety of inspected items can be quickly classified by the variety classification AI model 23, and the variety of inspected items that could not be classified by the variety classification AI model 23 can be identified by the LLM variety inspection unit 26, so that inspected items can be inspected quickly and accurately.

[0100] As described above, the image inspection device 10 and inspection method according to Embodiment 1 can more accurately inspect various types of inspected items (parts to be inspected).

[0101] (Other variations) In the above-described embodiment 1, an example was given in which the image inspection device 10 is used to inspect items flowing along lane 1. However, the image inspection device 10 can also be used to inspect fallen objects on roads such as expressways.

[0102] Conventionally, systems have been proposed that use cameras or other means to capture images of fallen objects on roads and then use AI for image analysis to detect the objects from the captured images. However, similar to AI visual inspection systems, this system also requires significant effort to train the AI ​​for image analysis, leaving room for improvement. Furthermore, using LLM instead of AI for image analysis to detect fallen objects is time-consuming and therefore unsuitable. By using an image inspection device according to a modification of Embodiment 1, fallen objects on roads can be detected quickly and more accurately.

[0103] First, an example of the functional configuration of the image inspection device 10 according to a modified example of Embodiment 1 will be described with reference to Figure 11. Figure 11 is a block diagram showing an example of the functional configuration of the image inspection device 10 according to a modified example of Embodiment 1. The hardware configuration example of the image inspection device 10 is, for example, the same as that of Embodiment 1 described above. Each part shown in Figure 11 may be realized, for example, by the control unit 11 reading a program stored in the storage unit 12, or by providing dedicated hardware.

[0104] Note that the configuration shown in Figure 11 is just one example; the parts shown in Figure 11 may be combined into a single configuration, or they may be further divided into more detailed configurations. Also, the links and arrows in Figure 11 show an example of data transmission and reception, but this connection configuration is not the only one that may be used; other connections and linkages may also be used.

[0105] The image inspection device 10 is comprised of an image acquisition unit 71, a falling object inspection AI model 72, an LLM inspection image generation unit 73, an LLM falling object inspection unit 74, an inspection result display unit 75, a final confirmation reception unit 76, a retraining data generation unit 77, a falling object inspection AI model creation unit 78, a falling object inspection few-shot prompt generation unit 79, a falling object DB management unit 80, and a falling object inspection prompt input unit 81.

[0106] The image acquisition unit 21 acquires images of the road captured by a camera (not shown) mounted on a vehicle or the like when inspecting the product being inspected.

[0107] The falling object inspection AI model 72 is a pre-trained AI model for detecting falling objects from images of roads, and detects falling objects from images of roads acquired by the image acquisition unit 71.

[0108] The LLM inspection image generation unit 73 processes the image of the road to generate an LLM inspection image, making it easier for the LLM fallen object inspection unit 74 to determine the presence or absence of fallen objects (see below). The LLM inspection image generation unit 73 may be omitted.

[0109] The LLM falling object inspection unit 74 inputs an image generated by the LLM inspection image generation unit 73 (or an image acquired by the image acquisition unit 21) and a prompt for detecting a falling object (also called the "falling object inspection prompt") to the LLM 100, and determines the presence or absence of a falling object based on the output of the LLM 100. If the LLM falling object inspection unit 74 cannot determine the presence or absence of a falling object by the falling object inspection AI model 72 (more specifically, if the falling object inspection AI model 72 cannot definitively determine whether it is a falling object), it determines the presence or absence of a falling object as described above.

[0110] The inspection result display unit 75 generates display information regarding the inspection results of fallen objects (such as the presence or absence of fallen objects) and outputs it to the display unit 15.

[0111] The final confirmation reception unit 76 generates display information to accept the final confirmation by the inspector and outputs it to the display unit 15.

[0112] The retraining data generation unit 77 generates training data containing fallen object information, including images of the fallen object, for use in retraining the fallen object inspection AI model 72, when the LLM fallen object inspection unit 74 determines, based on the output of the LLM 100, that there is a fallen object on the road.

[0113] The falling object inspection AI model creation unit 78 receives training data from a training data generation unit (not shown) or a retraining data generation unit 77, and creates a falling object inspection AI model 72 by training (AI learning) using the training data with a neural network or deep learning.

[0114] The Few-shot prompt generation unit 79 for falling object inspection generates Few-shot prompts for falling object inspection based on the inspection results of the AI ​​model 72 for falling object inspection, etc.

[0115] The Fallen Object Database Management Unit 80 manages the fallen object database. For example, if the LLM Fallen Object Inspection Unit 74 determines, based on the output of the LLM 100, that there is a fallen object on the road, the Fallen Object Database Management Unit 80 registers the fallen object information, including an image of the fallen object, into the Fallen Object Database. The Fallen Object Database referred to here is stored, for example, in the Storage Unit 12. Storing the Fallen Object Database in the Storage Unit 12 allows for the detection of specific fallen objects, but general fallen objects can be detected even if the Fallen Object Database is not stored in the Storage Unit 12. Therefore, the Fallen Object Database Management Unit 80 does not need to be stored in the Storage Unit 12. In this case, the Fallen Object Database Management Unit 80 is omitted.

[0116] The falling object inspection prompt input unit 81 generates a falling object inspection prompt based on the falling object inspection few-shot prompt obtained from the falling object inspection few-shot prompt generation unit 79 and the falling object information obtained from the falling object DB management unit 80, and outputs it to the LLM inspection image generation unit 73. If the falling object database is not stored in the storage unit 12, i.e., the falling object DB management unit 80 is omitted, the falling object inspection prompt input unit 81 generates a falling object inspection prompt based on the falling object inspection few-shot prompt.

[0117] Next, with reference to Figure 12, an overview of the image inspection apparatus according to a modified example of Embodiment 1 will be described. Figure 12 is a diagram illustrating an overview of the image inspection apparatus 10 according to a modified example of Embodiment 1.

[0118] As shown in Figure 12(a), when the falling object detection AI model 72 detects an object (hereinafter referred to as "detected object") from an image of a road, it determines whether or not the detected object is a fallen object. If the falling object detection AI model 72 cannot determine that the detected object is a fallen object, it outputs the corresponding image to the LLM inspection image generation unit 73. The falling object detection AI model 72 may also draw a frame around the detected object on the image of the road (see dotted frame).

[0119] As shown in Figure 12(b), the LLM inspection image generation unit 73 crops an image of a certain size centered on the detected object (see thick frame). In addition, as shown in Figure 12(b), the LLM inspection image generation unit 73 draws a frame (see dashed-dot frame) that is larger than the frame (dotted line frame) drawn by the falling object inspection AI model 72. In this way, by the LLM inspection image generation unit 73 cropping an image of a certain size centered on the detected object (see thick frame), background information can be made known to the LLM 100, while unnecessary information is not made known to the LLM, thus improving the accuracy of falling object inspection. Furthermore, as mentioned above, by the LLM inspection image generation unit 73 drawing a frame (see dashed-dot frame) that is larger than the frame (dotted line frame) drawn by the falling object inspection AI model 72, the gaze point of the LLM 100 is limited, thus improving the accuracy of falling object inspection.

[0120] The LLM falling object inspection unit 74 inputs the image generated by the LLM inspection image generation unit 73 (see Figure 12(c)) and the falling object inspection prompt to the LLM 100, and determines whether the detected object is a falling object based on the output of the LLM 100. This makes it possible to determine whether or not there is a falling object. The falling object inspection prompt contains instructions such as, for example, "Is there a falling object in the dashed-dotted frame? Output example: "YES / NO"".

[0121] Next, with reference to Figure 13, an example of the operation procedure of the image inspection device 10 according to a modified example of Embodiment 1 will be described. Figure 13 is a flowchart illustrating the processing flow of the falling object inspection method in the image inspection device 10 according to a modified example of Embodiment 1.

[0122] The image inspection device 10 acquires images of the road from a camera (not shown) using the image acquisition unit 21 (Sp201).

[0123] The image inspection device 10 inspects the falling object using the falling object inspection AI model 72 (Sp202).

[0124] The image inspection device 10 determines whether or not there is a fallen object using the fallen object detection AI model 72 (Sp203). Here, the determination of whether there is a fallen object or whether it might be a fallen object (gray) is made based on whether the reliability of the inspection result by the fallen object detection AI model 72 is above a predetermined threshold. By setting a second threshold with a lower value than the aforementioned threshold, the image inspection device 10 can determine that there is no fallen object (no fallen object, or a different object that is not a fallen object) if the reliability of the inspection result by the fallen object detection AI model 72 is below the second threshold.

[0125] If the device determines that a fallen object is present, i.e., the confidence level is above the threshold (YES in Sp203), the image inspection device 10 displays the inspection result on the display unit 15 (Sp204). Here, the display information regarding the inspection result of the inspected item, generated by the inspection result display unit 75, is displayed on the display unit 15.

[0126] The image inspection device 10 accepts the final confirmation by the inspector (Sp205). Here, the display information indicating acceptance of the final confirmation by the inspector, generated by the final confirmation acceptance unit 76, is displayed on the display unit 15. Then, this processing flow ends.

[0127] If the confidence level of the fallen object is below the first threshold but above the second threshold, i.e., it cannot be definitively determined that it is a fallen object (NO in Sp203), the image inspection device 10 uses the LLM inspection image generation unit 73 to extract the inspection area from the image (Sp206). That is, the image inspection device 10 uses the LLM inspection image generation unit 73 to generate an LLM inspection image. Note that this processing may be omitted.

[0128] The image inspection device 10 performs inspection processing of fallen objects using the LLM fallen object inspection unit 74 (Sp207).

[0129] The image inspection device 10 determines the presence or absence of a fallen object based on the output of the LLM 100 using the LLM fallen object inspection unit 74 (Sp208). If it is determined that a fallen object is present (YES in Sp208), the image inspection device 10 proceeds to step Sp209. On the other hand, if it is determined that there is no fallen object (NO in Sp208), the image inspection device 10 proceeds to step Sp204.

[0130] The image inspection device 10 registers new types of fallen objects that could not be identified by the fallen object inspection AI model 72 into the fallen object database via the fallen object DB management unit 80 (Sp209).

[0131] The image inspection device 10 generates training data containing information about fallen objects, including images of new types of fallen objects, using the retraining data generation unit 77 for use in retraining the fallen object inspection AI model 72 (Sp210). Then, this processing flow ends.

[0132] As described above, the image inspection device 10 according to the modified embodiment 1 can quickly determine the presence or absence of a fallen object using the fallen object inspection AI model 72, and if the fallen object inspection AI model 72 cannot determine the presence or absence of a fallen object, it can be identified by the LLM fallen object inspection unit 74, thus enabling quick and accurate determination of the presence or absence of a fallen object.

[0133] Furthermore, this disclosure also applies to programs and storage media that supply programs that realize the functions of the apparatus of the above-described embodiment to the apparatus via a network or various storage media, and which are read and executed by the computer within the apparatus.

[0134] Although various embodiments have been described above with reference to the drawings, it goes without saying that this disclosure is not limited to these examples. It will be clear to those skilled in the art that various modifications, alterations, substitutions, additions, deletions, and equivalents can occur within the scope of the claims, and these will naturally fall within the technical scope of this disclosure. Furthermore, the components of the various embodiments described above can be combined arbitrarily without departing from the spirit of the invention.

[0135] (Note) The following technologies are disclosed based on the above description of embodiments.

[0136] <Technology 1> A variety database 51 in which variety information is registered, linked to images and characteristics of parts without defects, An LLM (Large Language Models) 100 receives an image IPI of the part to be inspected, which is the part to be inspected, and a prompt (prompt for identifying the type of the part to be inspected) created by referring to the type database 51 to identify the type of the part to be inspected (LLM type inspection unit 26), and identifies the type of the part to be inspected based on the output of the LLM 100. Image inspection device 10, which includes the following:

[0137] This configuration improves the accuracy of identifying inspected items by using LLM to identify the variety of the inspected item based on variety information registered in the variety database. This allows for the application of inspection standards appropriate to the inspected item's variety, thus improving inspection accuracy. Consequently, it becomes possible to inspect various varieties of inspected items more accurately.

[0138] <Technology 2> The system further includes a variety classification AI model 23 that identifies the variety of the inspected product based on an image of the inspected product. The aforementioned LLM variety inspection unit 26 is If the aforementioned variety classification AI model 23 cannot classify the variety of the inspected product, the AI ​​model identifies the variety of the inspected product that could not be classified. Image inspection apparatus 10 as described in Technical 1.

[0139] This configuration allows for rapid classification of the varieties of inspected items using a variety classification AI model, and enables the LLM variety inspection unit to identify any varieties of inspected items that could not be classified by the variety classification AI model, thus enabling rapid and accurate identification of the varieties of inspected items.

[0140] <Technology 3> The LLM variety inspection unit 26, based on the output of the LLM 100, determines that the variety of the inspected product is a new variety not registered in the variety database 51, and further comprises a variety database management unit 37 that registers variety information, linked to the image and characteristics of the inspected product determined to be a new variety, in the variety database 51. Image inspection apparatus 10 as described in Technical 2.

[0141] This configuration ensures that the variety database is continuously updated and that it can flexibly respond to new varieties of samples being tested.

[0142] <Technology 4> The LLM variety inspection unit 26, based on the output of the LLM 100, determines that the variety of the inspected product is a new variety not registered in the variety database 51, and further comprises a retraining data generation unit 28 that generates training data for use in retraining the variety classification AI model 23, which includes an image of the inspected product determined to be a new variety and variety information linked to its characteristics. Image inspection apparatus 10 as described in Technology 2 or Technology 3.

[0143] This configuration expands the range of varieties that can be classified by the variety classification AI model, improving the future inspection accuracy of the variety classification AI model.

[0144] <Technology 5> If the aforementioned variety classification AI model 23 is able to classify the variety of the inspected product, the variety matching determination unit 24 further determines whether the classified variety of the inspected product matches the set variety associated with the lane 1 through which the inspected product flows. The aforementioned LLM variety inspection unit 26 is If the variety matching determination unit 24 determines that the variety of the inspected product does not match the set variety, it identifies the variety of the inspected product. An image inspection device 10 as described in any one of the technologies 2 to 4.

[0145] This configuration prevents misinspections caused by discrepancies between the product type being inspected and the specified product type, thereby improving inspection accuracy.

[0146] <Technology 6> A defect database 52 in which defect information is registered, which is linked to an image of the defective part and the characteristics of the defect, If the LLM variety inspection unit 26 determines, based on the output of the LLM 100, that the variety of the inspected product is a new variety not registered in the variety database 51, then the LLM defect inspection unit 27 inputs a prompt (defect detection prompt Pt2) created by referring to the defect database 52 to detect defects in the inspected product determined to be a new variety to the LLM 100, and inspects the inspected product based on the output of the LLM 100. It also has, An image inspection device 10 as described in any one of the technologies 1 to 5.

[0147] This configuration allows for the proper detection of defects even in new types of inspected items, improving both inspection accuracy and versatility.

[0148] <Technology 7> The LLM defect inspection unit 27 further includes a defect database management unit 40 that, when it detects a defect in the inspected product based on the output of the LLM 100, registers defect information, linked to the image and characteristics of the inspected product in the defect database 52. Image inspection apparatus 10 as described in Technical 6.

[0149] This configuration ensures that the defect database is continuously updated, improving future inspection procedures.

[0150] <Technology 8> The system further includes a defect inspection AI model 25 that detects defects in the inspected product based on an image of the inspected product. The aforementioned defect inspection AI model 25 is If the LLM variety inspection unit 26 determines, based on the output of the LLM 100, that the variety of the product to be inspected is an existing variety registered in the variety database 51, it inspects the product that has been determined to be an existing variety. An image inspection device 10 as described in any one of the technologies 1 to 7.

[0151] This configuration allows for efficient and accurate defect detection of existing varieties of inspected products.

[0152] <Technology 9> A defect database 52 registers defect information, which is linked to images of defective parts and the characteristics of the defects. An LLM (Large Language Models) 100 receives an image IPI of the part to be inspected, which is the part to be inspected, and a prompt (defect detection prompt Pt2) created by referring to the defect database 52 to detect defects in the part to be inspected, and inspects the part based on the output of the LLM 100. An LLM defect inspection unit 27 Image inspection device 10, which includes the following:

[0153] This configuration ensures that the defect database is continuously updated, and by using LLM to detect defects in inspected items based on the defect information registered in the database, the accuracy of defect detection in inspected items is improved, thus improving inspection accuracy. Therefore, it becomes possible to inspect various types of items more accurately.

[0154] <Technology 10> The Large Language Models (LLM) 100 receives an IPI image of the part to be inspected, and a prompt (prompt Pt1 for part identification) created by referring to a part database 51 in which part identification information is registered, which is linked to images and features of parts without defects, in order to identify the part of the inspected part. Based on the output of the LLM100, the type of the inspected product is identified. Image inspection methods.

[0155] This configuration improves the accuracy of identifying inspected items by using LLM to identify the variety of the inspected item based on variety information registered in the variety database. This allows for the application of inspection standards appropriate to the inspected item's variety, thus improving inspection accuracy. Consequently, it becomes possible to inspect various varieties of inspected items more accurately.

[0156] <Technology 11> A database of fallen objects where images of objects on roads are registered, A falling object inspection AI model 72 that determines fallen objects on the road based on an image that includes the road, If the aforementioned falling object inspection AI model 72 cannot determine that there is a fallen object on the road, the LLM (Large Language Models) 100 is input with the image and a prompt created by referring to the falling object database to detect the fallen object, and the LLM falling object inspection unit 74 determines the presence or absence of the fallen object based on the output of the LLM 100. An image inspection device equipped with the following features.

[0157] This configuration improves the accuracy of falling object inspection by using LLM to determine the presence or absence of falling objects. [Industrial applicability]

[0158] This disclosure is useful as an image inspection device and image inspection method that can more accurately inspect various types of inspected items (parts to be inspected). [Explanation of symbols]

[0159] 1 lane 2 cameras 3-lane control system 10. Image inspection device 11 Control Unit 12 Storage section 13 Communications Department 14 Input section 15 Display 16 Internal Interface 21 Image acquisition unit 22. AI Model for Extracting Inspection Locations 23. Variety Classification AI Model 24 Type matching determination section 25. AI Model for Defect Inspection 26 LLM Variety Inspection Department 27 LLM Defect Inspection Department 28 Retraining Data Generation Unit 37 Product DB Management Department 40 Defective Database Management Department 51 Variety Database 52 Defective Databases IIS Image Inspection System Images of IPI inspected items Pt1 Prompt for variety identification Pt2 Defect Detection Prompt 71 Image acquisition unit 72. AI Model for Inspecting Falling Objects 73 LLM Inspection Image Generation Unit 74 LLM Falling Object Inspection Department 75. Inspection Result Display Unit 76 Final Confirmation Reception Department 77 Retraining Data Generation Unit 78. AI Model Creation Department for Falling Object Inspection 79. Few-shot prompt generation unit for falling object inspection. 80 Fallen Object DB Management Department 81 Prompt input section for falling object inspection

Claims

1. A variety database in which variety information is registered, linked to images and characteristics of defect-free parts, An LLM (Large Language Models) unit inputs an image of the part to be inspected and a prompt created by referring to the variety database to identify the variety of the inspected part, and identifies the variety of the inspected part based on the output of the LLM. An image inspection device equipped with the following features.

2. The system further includes a variety classification AI model that identifies the variety of the inspected product based on an image of the inspected product. The aforementioned LLM variety inspection unit is If the aforementioned variety classification AI model cannot classify the variety of the inspected product, the AI ​​model identifies the variety of the inspected product that could not be classified. The image inspection apparatus according to claim 1.

3. The LLM variety inspection unit, when it determines from the output of the LLM that the variety of the inspected product is a new variety not registered in the variety database, further comprises a variety database management unit that registers variety information, linked to the image and characteristics of the inspected product determined to be a new variety, in the variety database. The image inspection apparatus according to claim 2.

4. If the LLM variety inspection unit determines, based on the output of the LLM, that the variety of the inspected product is a new variety not registered in the variety database, the LLM further comprises a retraining data generation unit that generates training data for use in retraining the variety classification AI model, which includes an image of the inspected product determined to be a new variety and variety information linked to its characteristics. The image inspection apparatus according to claim 2.

5. If the aforementioned variety classification AI model is able to classify the variety of the inspected product, the system further includes a variety matching determination unit that determines whether the classified variety of the inspected product matches the set variety associated with the lane through which the inspected product flows. The aforementioned LLM variety inspection unit is If the variety matching determination unit determines that the variety of the inspected product does not match the set variety, it identifies the variety of the inspected product. The image inspection apparatus according to claim 2.

6. A defect database in which defect information is registered, which is linked to an image of the defective part and the characteristics of the defect, If the LLM variety inspection unit determines, based on the output of the LLM, that the variety of the inspected product is a new variety not registered in the variety database, the LLM inputs a prompt created by referring to the defect database to detect defects in the inspected product determined to be a new variety, and then inspects the inspected product based on the output of the LLM; this is the LLM defect inspection unit. It also has, The image inspection apparatus according to claim 1.

7. The LLM defect inspection unit further comprises a defect DB management unit that, when it detects a defect in the inspected product based on the output of the LLM, registers defect information, linked to the image and characteristics of the inspected product in which the defect was detected, into the defect database. The image inspection apparatus according to claim 6.

8. The system further includes a defect inspection AI model that detects defects in the inspected product based on an image of the inspected product. The aforementioned defect inspection AI model is If the LLM variety inspection unit determines, based on the output of the LLM, that the variety of the product to be inspected is an existing variety registered in the variety database, then the product to be inspected, which has been determined to be an existing variety, is inspected. The image inspection apparatus according to claim 1.

9. A defect database in which defect information is registered, which is linked to images of defective parts and the characteristics of the defects, An LLM (Large Language Model) defect inspection unit inputs an image of the part to be inspected, which is the part to be inspected, and prompts created by referring to the defect database to detect defects in the part to be inspected, and inspects the part based on the output of the LLM. An image inspection device equipped with the following features.

10. In LLM (Large Language Models), an image of the part to be inspected and a prompt created by referring to a variety database in which variety information linked to images and characteristics of defect-free parts is registered in order to identify the variety of the inspected part are input. Based on the output of the LLM, the variety of the inspected product is identified. Image inspection methods.

11. A database of fallen objects where images of objects on roads are registered, A falling object inspection AI model that determines fallen objects on the road based on an image that includes the road, If the aforementioned falling object inspection AI model cannot determine that there is a fallen object on the road, the LLM (Large Language Model) inputs the image and a prompt created by referring to the falling object database to detect the fallen object, and determines the presence or absence of the fallen object based on the output of the LLM, the LLM falling object inspection unit, An image inspection device equipped with the following features.

Images