Method for generating teacher data, foreign object inspection device, foreign object inspection method, and foreign object detection program

The training data generation method divides visible light images into RGB regions and uses learning models to accurately detect foreign objects in food materials by identifying color differences, enhancing detection accuracy.

JP2024160730AActive Publication Date: 2024-11-15KANAYA FOODS CO LTD

Patent Information

Application Number
JP2023075954
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-02
Publication Date
2024-11-15
Estimated Expiration
2043-05-02

AI Technical Summary

Technical Problem

Existing systems struggle to accurately detect foreign objects in food materials due to the inability to designate and learn which foreign objects should be imaged and excluded, particularly those with different colors from the inspection object.

Method used

A training data generation method that images foreign matter samples with visible light, divides the visible light images into regions using RGB color space, and designates foreign objects based on color differences exceeding a threshold, combined with learning models for accurate detection.

Benefits of technology

Enables appropriate detection of foreign objects by distinguishing them from the inspection object based on color differences, improving detection accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024160730000001_ABST
    Figure 2024160730000001_ABST
Patent Text Reader

Abstract

To detect foreign objects appropriately.SOLUTION: In a method for generating teacher data, a plurality of foreign matter samples are captured in visible light in advance and the visible light images of the captured foreign matter samples are learned. The method for generating teacher data includes a storage step of capturing images of foreign matter samples to be learned, and storing the captured visible light images of the foreign matter samples in a memory unit, and a foreign matter designation step of designating an image of the foreign matter sample as a foreign matter from among the stored visible light images of the foreign matter samples. The foreign matter designation step divides the visible light image into three or more areas using RGB in the RGB color space as a color standard and designates, as a foreign matter, the foreign matter sample in the divided areas other than the area of an inspected object 8 and in the area where the color difference from the inspected object 8 in the RGB color space exceeds a color difference threshold.SELECTED DRAWING: Figure 5
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a teacher data generating method for appropriately detecting foreign matter, a foreign matter inspection device, a foreign matter inspection method, and a foreign matter detection program. [Background technology]

[0002] 2. Description of the Related Art In the field of food processing, for food safety reasons, it has been necessary to detect and remove foreign matter that has become mixed into food materials.

[0003] For example, Patent Document 1 proposes a technology in which a food material is imaged with a visible light camera and a near-infrared camera, frequency component images are created which are imaged images of the visible light wavelength components and near-infrared wavelength components contained in the visible light image and the near-infrared image, respectively, and image features are extracted from the multiple frequency component images thus generated, taking into consideration their mutual relationships, and the quality of the object being inspected and the presence of objects other than the object being inspected are analyzed based on the image features. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2020-51981 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the technology described in Patent Document 1, food materials are imaged with a visible light camera and the quality of the object being inspected and the presence of objects different from the object being inspected are analyzed based on image features. However, since the technology does not image the foreign objects themselves or specify and learn which foreign objects should be excluded, it is difficult to properly detect foreign objects.

[0006] Here, various foreign objects can be assumed, such as hair, resin, vinyl, rubber, paper, etc., and such foreign objects have various colors. In particular, it is necessary to appropriately remove foreign objects having a color different from that of the inspection object, but the technology of Patent Document 1 does not learn about foreign objects having a color different from that of the inspection object, so it is difficult to improve the accuracy of detecting foreign objects.

[0007] The present invention has been made in consideration of the above problems, and has an object to provide a teacher data generation method, a foreign matter inspection device, a foreign matter inspection method, and a foreign matter detection program for appropriately detecting foreign matters. [Means for solving the problem]

[0008] In order to achieve the above object, a first feature of the teaching data generation method according to the present invention is to A teacher data generation method for capturing images of a plurality of foreign matter samples using visible light in advance and learning the visible light images of the captured foreign matter samples, a storage step of capturing an image of a foreign substance sample to be studied and storing the captured visible light image of the foreign substance sample in a storage unit; a foreign matter designation step of designating an image of a foreign matter sample as a foreign matter from among the stored visible light images of the foreign matter sample, The foreign object designation step includes: The visible light image is divided into three or more regions using RGB in an RGB color space as a color reference, and a foreign substance sample that is in a region other than the region of the object to be inspected and has a color difference with the object to be inspected in the RGB color space that exceeds a color difference threshold is designated as a foreign substance. It is characterized by: Effect of the Invention

[0009] According to the teaching data generating method, foreign matter inspection device, foreign matter inspection method, and foreign matter detection program of the present invention, foreign matters can be appropriately detected. [Brief description of the drawings]

[0010] [Figure 1]1 is a configuration diagram showing an example of a foreign matter inspection device according to a first embodiment of the present invention. [Diagram 2] 5 is a flowchart showing the processing contents of a setting process in a processing device of the foreign matter inspection device according to the first embodiment of the present invention. [Diagram 3] 5 is a flowchart showing the processing contents of a visible light image learning process in the processing device of the foreign matter inspection device according to the first embodiment of the present invention. [Figure 4] FIG. 11 is an explanatory diagram illustrating a color region. [Diagram 5] 5 is a schematic plan view of the RGB color space shown in FIG. 4 for the purpose of explanation. [Figure 6] FIG. 2 is an explanatory diagram illustrating a color region. [Figure 7] 5 is a flowchart showing the processing contents of a foreign matter determination process for a visible light image in the processing device of the foreign matter inspection device according to the first embodiment of the present invention. [Figure 8] 10 is a flowchart showing the processing contents of a visible light image learning process in a processing device of a foreign matter inspection device according to Example 2 of the present invention. [Figure 9] 10 is a flowchart showing the processing contents of a foreign matter determination process for a visible light image and a near-infrared image in a processing device of a foreign matter inspection device according to Example 2 of the present invention. [Figure 10] 11 is a flowchart showing the processing contents of a foreign matter determination process for a visible light image and a near-infrared image in a processing device of a foreign matter inspection device according to Example 3 of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The same or equivalent parts and components are designated by the same or equivalent reference numerals throughout the drawings. However, it should be noted that the drawings are schematic and may differ from the actual product. In addition, the drawings may include parts with different dimensional relationships and ratios.

[0012] The embodiments described below are merely examples of devices for embodying the technical idea of ​​the present invention, and the technical idea of ​​the present invention does not limit the arrangement of each component to that described below. Various modifications can be made to the technical idea of ​​the present invention within the scope of the claims.

[0013] <Example 1> FIG. 1 is a configuration diagram showing an example of a foreign matter inspection device according to a first embodiment of the present invention. The foreign body inspection apparatus 1 includes a transport device 2 that transports an object 8 to be inspected, a near-infrared light irradiation device 3, a near-infrared camera 4, a visible light illumination device 5, a visible light camera 6, and a processing device 10 that includes a computer. The processing device 10 includes a display 13, an input device, a storage device, and a CPU (not shown). The processing device 10 includes, as its functional configuration, a learning means 11 that learns about foreign bodies, and a foreign body determination means 12 that determines foreign bodies.

[0014] The transport device 2 transports the object 8 to be inspected in a specific direction. In FIG. 1, the object 8 to be inspected is transported in the direction indicated by the arrow in FIG. A belt conveyor, a pallet conveyor, etc. can be used as the conveying device 2. In Fig. 1, a belt conveyor is used as the conveying device 2, in which an endless belt 2b is stretched between two parallel rollers 2a, and an inspection object 8 is placed on the upper surface of the endless belt 2b and conveyed. Fig. 1 shows an example in which dried vegetables, which are food materials, are conveyed as the inspection object on the upper surface (conveying surface) of this endless belt 2b.

[0015] The near-infrared light irradiation device 3 is disposed above the endless belt 2b and emits near-infrared light toward the upper surface of the endless belt 2b. The near-infrared light irradiation device 3 emits near-infrared light so that the area irradiated with the near-infrared light becomes a linear area (hereinafter referred to as the irradiation area 3a) extending in a direction perpendicular to the conveying direction of the inspection object 8 in a plan view (hereinafter referred to as the width direction). A halogen lighting device or an LED can be applied as the near-infrared light irradiation device 3. The irradiation area 3a extends from one end to the other end in the width direction of the endless belt 2b. As a result, all of the inspection objects 8 conveyed by the conveying device 2 pass through the irradiation area 3a, and as a result, all of the inspection objects 8 being conveyed are irradiated with near-infrared light. The near-infrared light irradiation device 3 irradiates near-infrared light at a timing designated by a control signal from the processing device 10.

[0016] The near-infrared camera 4 is disposed at a position where it can photograph the linear irradiation area 3a on the upper surface of the endless belt 2b. The near-infrared camera 4 photographs the irradiation area 3a, and collects the near-infrared light reflected by the object 8 passing through the irradiation area 3a, and also collects the near-infrared light reflected by the foreign matter, etc., when the object 8 is mixed with the object 8, to detect three different wavelength components λ0 to λ2 (hereinafter also referred to as wavelength components λ0 to λ2), and outputs a captured image (hereinafter also referred to as a near-infrared image) consisting of these three wavelength components to the processing device 10. The three wavelengths λ0 to λ2 are set to wavelengths that can detect foreign matter, components of the object 8, etc., depending on the characteristics of the object to be detected, such as foreign matter mixed in the object 8 being conveyed and components of the object 8. The wavelength components detected by the near-infrared camera 4 are set to wavelengths of, for example, about 1200 nm or more and 1600 nm or less. For example, if you want to detect proteins and fats, you can set the central wavelength (λ0) to 1180 to 1240 nm, and if you want to detect components related to moisture, you can set the central wavelength (λ1) to 1400 to 1460 nm. On the other hand, a wavelength that is not easily absorbed by either of them can be set as the reference wavelength, with the central wavelength (λ2) set to 1500 to 1620 nm. The near-infrared camera 4 captures images at a timing specified by a control signal from the processing device 10. As the near-infrared camera 4, for example, a line sensor camera with about 512 pixels can also be used.

[0017] The visible light illumination device 5 emits visible light so that the area illuminated by the visible light becomes a linear area (hereinafter referred to as the illumination area 5a) extending in a direction perpendicular to the conveying direction of the inspection object 8 in a plan view. Specifically, the visible light illumination device 5 illuminates light of three colors, R (Red), G (Green), and B (Blue). For example, a white LED illumination device can be used as the visible light illumination device 5. The illumination area 5a extends from one end to the other end in the width direction of the endless belt 2b. As a result, all of the inspection objects 8 conveyed by the conveying device 2 pass through the illumination area 5a, and as a result, visible light is irradiated onto all of the inspection objects 8 being conveyed.

[0018] The visible light camera 6 is disposed at a position where it can capture an image of the linear irradiation area 5a on the upper surface of the endless belt 2b, and is located upstream of the near-infrared camera 4 in the conveying direction and does not interfere with the near-infrared camera 4. The visible light camera 6 collects visible light reflected by the object 8 passing through the irradiation area 5a, and also collects visible light reflected by foreign matter, etc., if the object 8 is mixed with the foreign matter, and outputs a captured image (hereinafter also referred to as a visible light image) consisting of three wavelength components of R, G, and B to the processing device 10. The visible light camera 6 also captures images at a timing specified by a control signal from the processing device 10. As the visible light camera 6, for example, a color line sensor camera with about 256 pixels or 4096 pixels can be used.

[0019] The processing device 10 includes a learning means 11 and a foreign matter determining means 12 . The learning means 11 generates training data in which a near-infrared image consisting of wavelength components λ0 to λ2 as near-infrared wavelength components from the near-infrared camera 4, a visible light image consisting of wavelength components R, G, and B as visible light wavelength components from the visible light camera 6, and the results of designating a foreign object are associated with each other, and generates a learning model by learning based on this training data. The learning means 11 does not necessarily need to use a near-infrared image, but may generate training data in which a visible light image consisting of wavelength components R, G, and B as visible light wavelength components from the visible light camera 6 is associated with the result of designating a foreign object, and generate a learning model by learning based on this training data.

[0020] The foreign matter determination means 12 inputs a near-infrared image consisting of wavelength components λ0 to λ2 as near-infrared wavelength components from the near-infrared camera 4 and a visible light image consisting of wavelength components R, G, and B as visible light wavelength components from the visible light camera 6, and performs analysis processing using a learning model learned by the learning means 11 based on these images to determine the presence or absence of a foreign matter, etc. The foreign matter determination means 12 does not necessarily need to use a near-infrared image. It may also be configured to input a visible light image consisting of wavelength components R, G, and B as visible light wavelength components from the visible light camera 6, and execute an analysis process using a learning model learned by the learning means 11 based on the visible light image to determine the presence or absence of a foreign matter, etc.

[0021] Furthermore, the processing device 10 determines the photographing timings of the near-infrared camera 4 and the visible light camera 6 based on the conveying speed of the conveying device 2, the position of the irradiation area 3a, the position of the irradiation area 5a, etc., so as to photograph the object 8 to be inspected in the irradiation areas 3a, 5a at the timing when the object 8 to be inspected passes through the irradiation areas 3a and 5a, and outputs a control signal designating the photographing timing to the near-infrared camera 4 and the visible light camera 6. Also, it outputs a control signal to the near-infrared light irradiation device 3 and the visible light illumination device 5 so as to irradiate the irradiation areas 3a, 5a with light at the photographing timings of the near-infrared camera 4 and the visible light camera 6.

[0022] 1, for example, since the inspection objects 8 are continuously transported, images are captured at regular intervals by the near-infrared camera 4 and the visible light camera 6 so that all the inspection objects 8 being transported are captured by each of them. At this time, images may be captured so that the images captured at the previous and next shooting timings by the near-infrared camera 4 and the visible light camera 6 overlap. By setting the shooting timing in this way so that the previous and next captured images partially overlap, it is preferable to avoid the existence of an inspection object 8 that is not photographed between the previous and next shooting timings.

[0023] (Explanation of the setting process) 2 is a flowchart showing the processing contents of the setting processing in the processing device 10 of the foreign matter inspection device according to the embodiment 1 of the present invention. Various settings are input in advance to the processing device 10 in order to appropriately inspect the object 8 to be inspected.

[0024] As shown in FIG. 2, in step S11, a visible light image of the object to be inspected 8 captured by the visible light camera 6 is input, and the user inputs attribute information of the object to be inspected 8, such as the name, color, shape, wavelength, etc. of the object to be inspected 8, from an input device.

[0025] In step S13, the user inputs a color difference threshold value from the input device. The color difference threshold value is a value used in a process of specifying a foreign object, which will be described later. For example, it is possible to input "70" as a color difference threshold value in the L*a*b* color space. When a color difference threshold value in the L*a*b* color space is input, it is converted to the RGB color space and set. The color difference threshold value may be input as a value in the RGB color space. The color difference threshold value may be set according to the color of the object 8 to be inspected. In this case, the color difference threshold value may be set so that it is higher as the range of color variation in the RGB color space of the object 8 to be inspected is wider. In this way, as the range of color variation in the RGB color space of the object 8 to be inspected is wider, a foreign object sample having a color farther away from the object 8 to be inspected can be specified as a foreign object, and overdetection can be prevented.

[0026] In step S15, the user inputs the number of divisions N of the regions from the input device. Alternatively, a setting file of the color regions previously generated based on the user input may be read. As described later, by default, the region is divided into three regions with each of RGB in the RGB color space as a color reference. Here, the number of divisions N is input to further divide the region. For example, when the number of divisions N is "1", the color is divided into three regions by default, and the number of color regions M is "3". When the number of divisions N is "2", the number of color regions M is "6", that is, the color is divided into six regions obtained by multiplying the number of divisions N by the three regions by default.

[0027] In step S17, the processing device 10 associates the input visible light image, the attribute information of the inspection object 8, the color difference threshold value, and the color region, and stores them in the storage device as inspection object information.

[0028] In step S15, the number of divided regions N is input by the user, or a setting file for color regions generated in advance based on user input is read, but this is not limiting. For example, without setting the number of divided regions N, the user may input color regions from an input device so as to divide the RGB color space into three or more regions based on RGB as the color reference, and the input color regions may be set.

[0029] (Learning processing of visible light images) FIG. 3 is a flowchart showing the processing contents of the visible light image learning processing in the processing device 10 of the foreign matter inspection device according to the first embodiment of the present invention. 3, in step S101, the learning means 11 specifies the object to be inspected 8 based on a user input to the input device. For example, when "cabbage" is specified as the object to be inspected 8, in this learning process, in order to reliably detect foreign objects such as red thread or blue rubber in the cabbage, a foreign object sample having a color as different as possible from the color of the cabbage (far away in the RGB color space) is transported by the transport device 2. At this time, the foreign object sample may be red thread or blue rubber, or may be a color pattern such as red or blue.

[0030] In step S103, the learning means 11 initializes a counter i and assigns "1" to it.

[0031] In step S105, the visible light camera 6 captures an image of the foreign matter sample transported by the transport device 2.

[0032] In step S107, the learning means 11 calculates the color difference in the RGB color space between the object 8 to be inspected specified in step S101 and the foreign substance sample included in the visible light image captured in step S103.

[0033] In step S109, the learning means 11 causes the display 13 to display the visible light image captured in step S103.

[0034] In step S123, the learning means 11 judges whether or not the color of the captured foreign substance sample is included in the color region of the object 8 or in a color region that has already been specified, among the regions divided based on the color criterion. Here, the color criterion refers to three colors R (255,0,0), G (0,255,0), and B (0,0,255) in the RGB color space that are preset as color criterions. Note that, here, the RGB color space is a space expressed by gradations of R (0 to 255), G (0 to 255), and B (0 to 255).

[0035] If it is determined that the color of the captured foreign matter sample is included in the color region of the inspection object 8 or in a color region that has already been designated (step S123; YES), it is necessary to designate a foreign matter in the undesignated color region.

[0036] Therefore, in step S125, the learning means 11 recommends a color region that has not been pointed out. For example, the learning means 11 displays a message on the display 13 such as "The captured foreign body sample has already been designated. Please load another foreign body sample." Note that, although the unpointed color region is recommended and not designated as a foreign body here, multiple foreign bodies may be designated in the same color region other than the color region of the inspection object 8, so that the unpointed color region may be recommended and then designated as a foreign body.

[0037] In step S135, the learning means 11 increments the counter i, and moves the process to step S105. On the other hand, if it is determined that the color of the imaged foreign matter sample is outside the color range of the object 8 to be inspected and is not included in the color range already designated (step S123; NO), in step S127, the learning means 11 designates the color of the imaged foreign matter sample as a foreign matter.

[0038] In step S129, the learning means 11 judges whether the counter i is equal to or greater than the number of color regions M. Here, the number of color regions M is, as described above, a value obtained by multiplying the default number of divisions "3" by the input number of divisions N, or the number of regions into which the RGB color space is divided by the user with RGB as the color reference.

[0039] In step S131, the learning means 11 deletes foreign matter whose color difference is within the color difference threshold range, thereby making it possible to exclude foreign matter whose color is close to that of the object under inspection 8 from foreign matter samples. Note that, here, the learning means 11 deletes foreign matter designated as foreign matter in step S127 whose color difference is within the color difference threshold range, but this is not limited to this. For example, it is also possible to designate as a foreign matter a foreign matter sample that is in an undesignated color area and whose color difference in the RGB color space with the object to be inspected 8 exceeds the color difference threshold. Also, here, the foreign matter samples are imaged one by one and the foreign matter designation process is executed, but this is not limited to this. It is also possible to image a plurality of foreign matter samples simultaneously and execute the foreign matter designation process for each image.

[0040] 4 is an explanatory diagram illustrating color regions. Here, the number of divisions N is "1", that is, the color region is divided into three.

[0041] As shown in FIG. 4, it can be expressed as a three-dimensional RGB space with P(0,0,0) as the origin, R(255,0,0), G(0,255,0), and B(0,0,255).

[0042] When the number of divisions N is "1", the number of color regions M is "3", and the colors are divided into the default three regions with R (255,0,0), G (0,255,0), and B (0,0,255) as the color references.

[0043] In the example shown in FIG. 4, the color region is divided into three color regions: color region A101 surrounded by P0 (0,0,0), P1 (0,255,255), P4 (255,255,255), P3 (255,0,255), and B (0,0,255); color region A102 surrounded by P0 (0,0,0), P3 (255,0,255), P4 (255,255,255), P2 (255,255,0), and R (255,0,0); and color region A103 surrounded by P0 (0,0,0), P1 (0,255,255), P4 (255,255,255), P2 (255,255,0), and G (0,255,0).

[0044] FIG. 5 is a schematic plan view of the RGB color space shown in FIG. 4 for the purpose of explanation. As shown in Fig. 5, the center of the circle is P0 (0,0,0), and R (255,0,0), G (0,255,0), and B (0,0,255) are shown at intervals of 120 degrees on the circumference. Note that, although the RGB three-dimensional space of R (255,0,0), G (0,255,0), and B (0,0,255) is here illustrated as a two-dimensional schematic representation with P (0,0,0) as the origin, a hue wheel in which hues are arranged in an orderly circular ring may also be used.

[0045] The color area is divided based on color criteria, so the color area surrounded by V101, P0, and V102 with B(0,0,255) as the color reference will be referred to as A101, the color area surrounded by V102, P0, and V103 with R(255,0,0) as the color reference will be referred to as A102, and the color area surrounded by V103, P0, and V101 with G(0,255,0) as the color reference will be referred to as A103.

[0046] In FIG. 5, if the color of the object 8 to be inspected is Q100, among the color regions A101 to A103, the color region that includes the object Q100 to be inspected is the color region A103, and therefore the learning means 11 excludes the color region A103 based on the color reference G(0,255,0).

[0047] Here, the color reference with a color difference within the color difference threshold range is within the range determined by the color difference threshold Th1 based on Q100. The color difference threshold Th1 is the color difference threshold input in step S13 of the flowchart shown in Fig. 2. Here, for example, the color difference threshold Th1 corresponding to the color difference threshold "70" in the L*a*b* color space is regarded as an angle, and the area sandwiched between the straight lines X101 and X102 passing through the center P0(0,0,0), which is the range indicated by the color difference threshold Th1 and including Q100 at its circumferential center, is defined as the range determined by the color difference threshold Th1 based on Q100.

[0048] Therefore, the learning means 11 deletes the color region A103 based on the color reference G(0,255,0), and deletes the foreign matter that is in a color region in which the color difference from the inspection object 8 in the RGB color space is within the color difference threshold Th1. As a result, the learning means 11 divides the visible light image into color regions A101-A103 using RGB in the RGB color space as the color reference, and can designate as a foreign matter a foreign matter sample that is in the color region A101-A102, which is outside the region of the inspection object 8 among the divided color regions A101-A103, and whose color difference from the inspection object 8 in the RGB color space exceeds the color difference threshold Th1.

[0049] For example, the color difference threshold Th1 corresponding to the color difference threshold value "70" in the L*a*b* color space may be regarded as the distance from Q100, and a region C101 having Q100 as its center and the color difference threshold Th1 as its radius may be defined as a range determined by the color difference threshold Th1 based on Q100. In this case, foreign matter samples that are outside the region C101 among the color regions A101-A102 are designated as foreign matter.

[0050] 3, in step S133, the learning means 11 extracts the color of the image of the specified foreign object and learns it as teacher data. For the learning, deep learning using a neural network or the like is used, and a learning model is generated by this learning.

[0051] In the example shown in FIG. 5, the learning means 11 excludes foreign matter Q101 present in color region A103 from the learning object, but includes foreign matter Q102 as the learning object since it is present in color region A102.

[0052] 6 is an explanatory diagram that illustrates a color region in a schematic manner, in which the division number N is "2", that is, the region is divided into six color regions. As shown in Figure 6, the center of the circle is P0 (0,0,0), and R (255,0,0), P2 (255,255,0), G (0,255,0), P1 (0,255,255), B (0,0,255), and P3 (255,0,255) are shown at intervals of 60 degrees on the circumference.

[0053] The color area is divided based on the color reference, so the color area surrounded by V201, P0, and V202 with B(0,0,255) as the color reference is A201, the color area surrounded by V202, P0, and V203 with P3(255,0,255) as the color reference is A202, and the color area surrounded by V203, P0, and V204 with R(255,0,0) as the color reference is A203. Let us assume that the color area surrounded by V204, P0, and V205 with P2(255,255,0) as the color reference is A204, the color area surrounded by V205, P0, and V206 with G(0,255,0) as the color reference is A205, and the color area surrounded by V206, P0, and V201 with P1(0,255,255) as the color reference is A206.

[0054] In Figure 6, as in Figure 5, if the color of the object 8 to be inspected is Q100, among the color areas A201 to A206, the color area that includes the object Q100 to be inspected is color area A205, and therefore the learning means 11 excludes color area A205 based on the color reference G (0, 255, 0).

[0055] Here, the color reference with a color difference within the color difference threshold range is within the range determined by the color difference threshold Th1 based on Q100. The color difference threshold Th1 is the color difference threshold input in step S13 of the flowchart shown in Fig. 2. Here, the color difference threshold Th1, which corresponds to the color difference threshold value "70" in the L*a*b* color space, is regarded as an angle, and the area sandwiched between the straight lines X101 and X102 passing through the center P0(0,0,0), which is the range indicated by the color difference threshold Th1 and includes Q100 at its circumferential center, is defined as the range determined by the color difference threshold Th1 based on Q100.

[0056] Therefore, the learning means 11 deletes the color region A205 based on the color reference G(0,255,0), and deletes foreign matter that is in a color region in which the color difference from the object under inspection 8 in the RGB color space is within the color difference threshold Th1. As a result, the learning means 11 divides the visible light image into color regions A201-A106 using RGB in the RGB color space as the color reference, and can designate as foreign matter any foreign matter sample that is in the color regions A201-A204, A206 that are outside the region of the object under inspection 8 among the divided color regions A201-A206 and whose color difference from the object under inspection 8 in the RGB color space exceeds the color difference threshold Th1.

[0057] (Foreign object detection processing of visible light images) FIG. 7 is a flowchart showing the processing contents of the foreign matter determination processing of a visible light image in the processing device 10 of the foreign matter inspection device according to the first embodiment of the present invention. As shown in FIG. 7, in step S201, the user designates one of the objects to be inspected 8 set in advance through an input device.

[0058] In step S203, the visible light camera 6 captures an image of the inspection object transported by the transport device 2.

[0059] In step S207, the foreign matter determination means 12 uses a learning model previously learned by the learning means 11 to determine whether or not the inspection object captured in the acquired visible light image is a foreign matter. If it is determined that the object is a foreign object (step S207; YES), in step S209, the foreign object determining means 12 notifies the display 13 that it is a foreign object. On the other hand, if it is determined that the object is not a foreign object (step S207; NO), in step S211, the foreign object determining means 12 notifies the display 13 that the object is not a foreign object.

[0060] <Example 2> In the first embodiment of the present invention, a learning process is performed on a visible light image, and a foreign object determination process is executed on the visible light image. In the second embodiment, a learning process is performed on a visible light image and a near-infrared image, and a foreign object determination process is executed on the visible light image and the near-infrared image.

[0061] (Learning processing of visible light images and near infrared images) Fig. 8 is a flowchart showing the processing contents of a visible light image learning process in the processing device 10 of the foreign matter inspection device according to the second embodiment of the present invention. Note that the configuration of the foreign matter inspection device according to the second embodiment of the present invention is the same as the configuration of the foreign matter inspection device according to the first embodiment of the present invention, so a description thereof will be omitted. Also, in Fig. 8, the processing contents of steps S101 to S103 are the same as the processing contents of steps S101 to S103 shown in Fig. 3, so a description thereof will be omitted. As shown in FIG. 8, in step S104, the visible light camera 6 captures an image of the foreign matter sample transported by the transport device 2, and the near-infrared camera 4 captures an image of the foreign matter sample transported by the transport device 2.

[0062] In step S107, the learning means 11 calculates the color difference in the RGB color space between the object 8 to be inspected specified in step S101 and the foreign substance sample included in the visible light image captured in step S103.

[0063] In step S109, the learning means 11 causes the display 13 to display the visible light image captured in step S103.

[0064] In step S111, the learning means 11 judges whether or not the color difference with the object to be inspected 8 in the RGB color space exceeds a color difference threshold value.

[0065] If it is determined that the color difference with the inspection object 8 in the RGB color space exceeds the color difference threshold (step S111; YES), in step S123, the learning means 11 determines whether or not the color of the captured foreign matter sample is included in the color area of ​​the inspection object 8 or an already specified color area among the areas divided based on the color criterion. The processing of steps S125 to S133 is the same as the processing of steps S125 to S133 shown in FIG. 3, so a description thereof will be omitted.

[0066] On the other hand, if it is determined that the color difference with the object to be inspected 8 in the RGB color space is equal to or less than the color difference threshold value (step S111; NO), in step S141, the learning means 11 does not designate it as a foreign object in the visible light image, but stores in the storage means the position recognized as a foreign object in the visible light image.

[0067] In step S143, the learning means 11 displays the captured near-infrared image and visible light image side by side on the display 13. At this time, the captured near-infrared image and visible light image are displayed in parallel at the same time so that their positions correspond to each other. That is, the near-infrared image and the visible light image are displayed so that their image positions (positions in the conveying direction of the belt conveyor) match on the left and right sides of the screen of the display 13, and the captured foreign body sample and the image of the dried vegetable are displayed so that they correspond to each other and can be seen by the operator. While looking at the left and right near-infrared image and the visible light image, the operator marks the image of the foreign body sample on the near-infrared image using a pointer such as a mouse, which is an input device, to specify that it is a foreign body sample.

[0068] In step S145, the learning means 11 judges whether or not the image of the foreign substance sample on the near-infrared image has been marked by a user operation. If it is judged that the image of the foreign substance sample on the near-infrared image has been marked (step S145; YES), in step S133, the learning means 11 designates the marked foreign substance sample as a foreign substance.

[0069] In step S149, the learning means 11 extracts features such as color and shape of the image of the specified foreign object and learns them as teacher data. For the learning, deep learning using a neural network or the like is used, and a learning model is generated by this learning.

[0070] (Foreign object detection processing of visible light images and near-infrared images) FIG. 9 is a flowchart showing the processing contents of the foreign matter determination processing of the visible light image and the near-infrared image in the processing device 10 of the foreign matter inspection device according to the second embodiment of the present invention. As shown in FIG. 9, in step S301, the user designates one of the objects to be inspected 8 set in advance through an input device.

[0071] In step S303, the visible light camera 6 captures an image of the inspection object transported by the transport device 2, and in step S305, the near-infrared camera 4 captures an image of the inspection object transported by the transport device 2.

[0072] In step S307, the foreign matter determination means 12 uses a learning model previously learned by the learning means 11 based on the visible light image to determine whether or not the inspection object captured in the acquired visible light image is a foreign matter.

[0073] If it is determined that a foreign object is present in the visible light image (step S307; YES), in step S317, the foreign object determination means 12 notifies the display 13 that a foreign object is present.

[0074] On the other hand, if it is determined that the object is not a foreign object in the visible light image (step S307; NO), in step S309, the foreign object determination means 12 uses a learning model that has been previously trained by the learning means 11 based on the near-infrared image to determine whether or not the object to be inspected shown in the acquired near-infrared image is a foreign object.

[0075] If it is determined that a foreign object is present in the near-infrared image (step S309; ​​YES), in step S313, the foreign object determination means 12 notifies the display 13 that a foreign object is present.

[0076] On the other hand, if it is determined that the object is not a foreign object in the near-infrared image, i.e., that the object is not a foreign object in either the visible light image or the near-infrared image (step S309; ​​NO), in step S311, the foreign object determination means 12 notifies the display 13 that the object is not a foreign object.

[0077] <Example 3> In the second embodiment of the present invention, the foreign object determination process is performed on the visible light image and the near-infrared image using a learning model trained by machine learning using training data of the visible light image and the near-infrared image. However, the present invention is not limited to this, and foreign object determination by a user's direct vision may also be combined. 10 is a flowchart showing the processing contents of the foreign matter determination processing of the visible light image and the near-infrared image in the processing device 10 of the foreign matter inspection device according to the third embodiment of the present invention. Note that the foreign matter count described later is set to "0" as the initial value. As shown in FIG. 10, in step S401, the user designates one of the objects to be inspected 8 set in advance through an input device.

[0078] In step S403, the visible light camera 6 captures an image of the inspection object transported by the transport device 2, and in step S405, the near-infrared camera 4 captures an image of the inspection object transported by the transport device 2.

[0079] In step S407, the foreign matter determination means 12 uses a learning model previously machine-learned by the learning means 11 based on the visible light image to determine whether or not the inspection object captured in the acquired visible light image is a foreign matter.

[0080] When it is determined that the object is a foreign object in the determination by machine learning based on the visible light image (step S407; YES), the foreign object determining means 12 increments the foreign object count C in step S409.

[0081] In step S411, the foreign object determination means 12 determines whether or not the inspection object included in the visible light image is a foreign object by direct vision based on the visible light image captured by the visible light camera 6. Specifically, when the user performs an operation to designate the inspection object as a foreign object while looking at the visible light image displayed on the display 13, the foreign object determination means 12 may determine whether or not the inspection object is a foreign object. Furthermore, the foreign object determination means 12 may determine that the inspection object is a foreign object when the color and size of the inspection object included in the visible light image captured by the visible light camera 6 are outside the range of colors and sizes included in the pre-stored attribute data of the inspection object 8.

[0082] When it is determined that there is a foreign object in the direct visual determination based on the visible light image (step S411; YES), the foreign object determining means 12 increments the foreign object count C in step S413.

[0083] In step S415, the foreign matter determination means 12 uses a learning model previously machine-learned by the learning means 11 based on the near-infrared image to determine whether or not the inspection object captured in the acquired near-infrared image is a foreign matter.

[0084] When it is determined that the object is a foreign object through machine learning based on the near-infrared image (step S415; YES), the foreign object determining means 12 increments the foreign object count C in step S417.

[0085] In step S419, the foreign substance determination means 12 determines whether or not the inspection object included in the near-infrared image captured by the near-infrared camera 4 is a foreign substance by direct vision based on the near-infrared image captured by the near-infrared camera 4. Specifically, when the reflectance of near-infrared rays with different wavelengths (here, three wavelengths of wavelength components λ0 to λ2 as near-infrared wavelength components) for the inspection object included in the near-infrared image captured by the near-infrared camera 4 is all outside a predetermined range, the foreign substance determination means 12 may determine that the material of the inspection object is different from the material of the inspection object 8 and therefore the inspection object is a foreign substance. Here, the predetermined range is included in the attribute data of the inspection object 8 stored in advance. Furthermore, when the user performs an operation to designate the inspection object as a foreign substance while looking at the near-infrared image displayed on the display 13, the foreign substance determination means 12 may determine whether or not the inspection object is a foreign substance.

[0086] When it is determined that there is a foreign object in the direct visual determination based on the near-infrared image (step S419; YES), the foreign object determining means 12 increments the foreign object count C in step S421.

[0087] In step S423, the foreign matter determination means 12 determines whether or not the foreign matter count C has exceeded a preset threshold value Th. Here, the threshold value Th is preset by the user within the range of 1 to 4 based on the detection accuracy.

[0088] If it is determined that the foreign object count C exceeds the threshold value Th (step S423; YES), in step S425, the foreign object determination means 12 notifies the display 13 that a foreign object has been detected.

[0089] On the other hand, if it is determined that the foreign object count C is equal to or less than the threshold value Th (step S423; NO), in step S427, the foreign object determination means 12 notifies the display 13 that there is no foreign object.

[0090] Here, when it is determined that the foreign object count C exceeds the threshold value Th, the foreign object determination means 12 notifies the display 13 that there is a foreign object, but this is not limited to the above. For example, the threshold value Th may be set to 1, and when it is determined that there is a foreign object in a machine learning determination based on a visible light image (step S407; YES), or when it is determined that there is a foreign object in a direct visual determination based on a visible light image (step S411; YES), the foreign object determination means 12 may notify the display 13 that there is a foreign object.

[0091] (Additional Note) This application discloses the following inventions. (Appendix 1)

[0092] A teacher data generation method for capturing images of a plurality of foreign matter samples using visible light in advance and learning the visible light images of the captured foreign matter samples, a storage step of capturing an image of a foreign substance sample to be studied and storing the captured visible light image of the foreign substance sample in a storage unit; a foreign matter designation step of designating an image of a foreign matter sample as a foreign matter from among the stored visible light images of the foreign matter sample, The foreign object designation step includes: The visible light image is divided into three or more regions using RGB in an RGB color space as a color reference, and a foreign substance sample that is in a region other than the region of the object to be inspected and has a color difference with the object to be inspected in the RGB color space that exceeds a color difference threshold is designated as a foreign substance. A teacher data generating method comprising:

[0093] (Appendix 2) The foreign object designation step includes: The visible light image is divided into 3N (N≧1) regions using RGB in the RGB color space as a color reference, and a foreign substance sample that is in a region other than the region of the object to be inspected and has a color difference with the object to be inspected in the RGB color space that exceeds a color difference threshold is designated as a foreign substance. The teacher data generation method according to claim 1,

[0094] (Appendix 3) The color difference threshold is set to be higher as the range of color variation in the color space of the object to be inspected becomes wider. The teacher data generation method according to claim 1,

[0095] (Appendix 4) A foreign object detection device that optically detects foreign objects mixed in an object to be inspected that is placed on a transport path and transported, comprising: a visible light camera that captures an image of the object to be inspected using visible light irradiated by a light irradiating means that irradiates the object with visible light; and a foreign matter determining means that determines whether a foreign matter is present in the object to be inspected based on the visible light image of the object to be inspected captured by the visible light camera, The foreign matter determination means is a means for determining whether or not the object to be inspected is a foreign matter by referring to teacher data of a visible light image of a foreign matter previously learned by a learning means, The teacher data is, in the learning process, The visible light image is divided into three or more regions using RGB in an RGB color space as a color reference, and a foreign substance sample that is in a region other than the region of the object to be inspected and has a color difference with the object to be inspected in the RGB color space that exceeds a color difference threshold is designated as a foreign substance and the image is learned. A foreign body inspection device comprising:

[0096] (Appendix 5) The foreign object determination means includes: a first foreign matter determination means for determining whether or not the object to be inspected is a foreign matter by using a learning model that is trained based on the teacher data; a second foreign matter determination means for determining that an object to be determined, contained in the visible light image captured by the visible light camera, is a foreign matter mixed in the object to be inspected, when the color and size of the object are outside of a predetermined range; a notification means for notifying the presence of a foreign object when either the first foreign object determination means or the second foreign object determination means determines that a foreign object has been detected; 5. The foreign body inspection device according to claim 4, further comprising:

[0097] (Appendix 6) A teaching data generation method for capturing images of a plurality of foreign matter samples using visible light and near-infrared light in advance, and learning the visible light images and near-infrared images of the captured foreign matter samples, a storage step of capturing an image of a foreign substance sample to be studied and storing a visible light image and a near-infrared image of the captured foreign substance sample in a storage unit; a foreign matter designation step of designating an image of a foreign matter sample as a foreign matter from among the stored visible light images and near infrared images of the foreign matter sample, The foreign object designation step includes: Dividing the visible light image into three or more regions using RGB in an RGB color space as a color reference, and designating as a foreign substance a foreign substance sample that is in a region other than the region of the object to be inspected in the RGB color space and has a color difference from the object to be inspected that exceeds a color difference threshold value among the divided regions; For a foreign substance sample in an area among the divided areas where the color difference in the RGB color space from the object to be inspected is equal to or less than a color difference threshold value, the foreign substance is not designated as a foreign substance in the visible light image, but if the shape of the foreign substance can be confirmed in an image of a position on the near-infrared image corresponding to the position recognized as a foreign substance on the visible light image, the image on the near-infrared image is designated as a foreign substance. A teacher data generating method comprising:

[0098] (Appendix 7) A foreign object detection device that optically detects foreign objects mixed in an object to be inspected that is placed on a transport path and transported, comprising: a visible light camera that captures an image of the inspection object illuminated with visible light by a first light illumination means that illuminates the inspection object with visible light; a near-infrared camera that captures an image of the inspection object irradiated with near-infrared light by a second light irradiating means that irradiates the inspection object with near-infrared light; a foreign matter determination means for determining whether a foreign matter is present in the object to be inspected based on a visible light image of the object to be inspected captured by the visible light camera and a near-infrared image of the object to be inspected captured by the near-infrared camera, The foreign matter determination means is a means for determining whether the object to be inspected is a foreign matter by referring to teacher data of a visible light image and a near-infrared light image of the foreign matter previously learned by a learning means, The teacher data is, in the learning process, Dividing the visible light image into three or more regions using RGB in an RGB color space as a color reference, and designating as a foreign substance a foreign substance sample that is in a region other than the region of the object to be inspected in the RGB color space and has a color difference from the object to be inspected that exceeds a color difference threshold value among the divided regions; For a foreign object sample in an area of ​​the divided areas where the color difference in the RGB color space with respect to the object to be inspected is equal to or less than the color difference threshold, the foreign object is not designated as a foreign object in the visible light image, but if the shape of the foreign object can be confirmed in an image at a position on the near-infrared image corresponding to the position recognized as a foreign object on the visible light image, the image on the near-infrared image is designated as a foreign object and learning is performed. A foreign body inspection device comprising:

[0099] (Appendix 8) The foreign object determination means includes: a first foreign matter determination means for determining whether or not a foreign matter has been mixed into the object to be inspected by using a learning model that has been trained on the basis of teacher data generated on the basis of a visible light image; a second foreign matter determination means for determining that an object to be determined, contained in the visible light image captured by the visible light camera, is a foreign matter mixed in the object to be inspected, when the color and size of the object are outside of a predetermined range; a third foreign matter determination means for determining whether or not a foreign matter has been mixed into the object to be inspected by using a learning model that has been trained based on the teacher data generated based on near-infrared light; a fourth foreign matter determination means for determining that a foreign matter has been mixed into the object to be inspected when a reflectance of near-infrared light having a different wavelength for the object to be determined, which is included in the near-infrared image captured by the near-infrared camera, is outside a predetermined range; a notification means for notifying the presence of a foreign matter based on the results of the determinations by the first foreign matter determination means, the second foreign matter determination means, the third foreign matter determination means, and the fourth foreign matter determination means; 8. The foreign matter inspection device according to claim 7, further comprising:

[0100] (Appendix 9) A foreign object detection method for optically detecting foreign objects present in an object to be inspected that is placed on a transport path and transported, comprising the steps of: a foreign matter determination step of determining whether a foreign matter is present in the object to be inspected based on a visible light image of the object to be inspected, the object being irradiated with visible light by a light irradiation means for irradiating the object to be inspected with visible light and captured by a visible light camera; The foreign matter determination step is a step of determining whether the object to be inspected is a foreign matter by referring to teacher data of a visible light image of the foreign matter previously learned in the learning step, The teacher data is, in the learning process, The visible light image is divided into three or more regions using RGB in an RGB color space as a color reference, and a foreign substance sample that is in a region other than the region of the object to be inspected and has a color difference with the object to be inspected in the RGB color space that exceeds a color difference threshold is designated as a foreign substance and the image is learned. 13. A foreign body inspection method comprising:

[0101] (Appendix 10) A foreign matter detection program to be executed by a foreign matter inspection device that optically detects foreign matters mixed in an object to be inspected that is placed on a transport path and transported, comprising: a foreign matter determination step of determining whether a foreign matter is present in the object to be inspected based on a visible light image of the object to be inspected, the object being irradiated with visible light by a light irradiation means that irradiates the object to be inspected with visible light and captured by a visible light camera; The foreign matter determination step is a step of determining whether the object to be inspected is a foreign matter by referring to teacher data of a visible light image of the foreign matter previously learned in the learning step, The teacher data is, in the learning process, The visible light image is divided into three or more regions using RGB in an RGB color space as a color reference, and a foreign substance sample that is in a region other than the region of the object to be inspected and has a color difference with the object to be inspected in the RGB color space that exceeds a color difference threshold is designated as a foreign substance and the image is learned. A foreign object detection program comprising: [Explanation of symbols]

[0102] 1 Foreign body inspection equipment 2. Conveyor device 2a Laura 2b endless belt 3 Near-infrared light irradiation device 3a Irradiation area 4. Near-infrared camera 5 Visible light illumination device 5a Irradiation area 6. Visible Light Camera 8. Inspection item 10 Processing equipment 11 Learning tools 12 Foreign object determination means 13. Display

Claims

1. A teacher data generation method for capturing images of a plurality of foreign matter samples using visible light in advance and learning the visible light images of the captured foreign matter samples, a storage step of capturing an image of a foreign substance sample to be studied and storing the captured visible light image of the foreign substance sample in a storage unit; a foreign matter designation step of designating an image of a foreign matter sample as a foreign matter from among the stored visible light images of the foreign matter sample, The foreign object designation step includes: The visible light image is divided into three or more regions using RGB in an RGB color space as a color reference, and a foreign substance sample that is in a region other than the region of the object to be inspected and has a color difference from the object to be inspected in the RGB color space that exceeds a color difference threshold is designated as a foreign substance. A teacher data generating method comprising:

2. The foreign object designation step includes: The visible light image is divided into 3N (N≧1) regions using RGB in the RGB color space as a color reference, and a foreign substance sample that is in a region other than the region of the object to be inspected and has a color difference from the object to be inspected in the RGB color space that exceeds a color difference threshold is designated as a foreign substance.

2. The teaching data generating method according to claim 1,

3. The color difference threshold is set to be higher as the range of color variation in the color space of the object to be inspected becomes wider.

2. The teaching data generating method according to claim 1,

4. A foreign object detection device that optically detects foreign objects mixed in an object to be inspected that is placed on a transport path and transported, comprising: a visible light camera that captures an image of the object to be inspected using visible light irradiated by a light irradiating means that irradiates the object with visible light; and a foreign matter determining means that determines whether a foreign matter is present in the object to be inspected based on the visible light image of the object to be inspected captured by the visible light camera, The foreign matter determination means is a means for determining whether or not a foreign matter is present in the object to be inspected by referring to teacher data of a visible light image of a foreign matter previously learned by a learning means, The teacher data is, in the learning process, The visible light image is divided into three or more regions using RGB in an RGB color space as a color reference, and a foreign substance sample that is in a region other than the region of the object to be inspected and has a color difference from the object to be inspected in the RGB color space that exceeds a color difference threshold is designated as a foreign substance and learning is performed. A foreign body inspection device comprising:

5. The foreign object determination means includes: a first foreign matter determination means for determining whether or not the object to be inspected is a foreign matter by using a learning model that is trained based on the teacher data; a second foreign matter determination means for determining that an object to be determined, contained in the visible light image captured by the visible light camera, is a foreign matter mixed in the object to be inspected, when the color and size of the object to be determined are outside of a predetermined range; a notification means for notifying the presence of a foreign object when either the first foreign object determination means or the second foreign object determination means determines that a foreign object has been detected; 5. The foreign matter inspection apparatus according to claim 4, further comprising:

6. A teaching data generation method for capturing images of a plurality of foreign matter samples using visible light and near-infrared light in advance, and learning the visible light images and near-infrared images of the captured foreign matter samples, a storage step of capturing an image of a foreign substance sample to be studied and storing a visible light image and a near-infrared image of the captured foreign substance sample in a storage unit; a foreign matter designation step of designating an image of a foreign matter sample as a foreign matter from among the stored visible light images and near infrared images of the foreign matter sample, The foreign object designation step includes: Dividing the visible light image into three or more regions using RGB in an RGB color space as a color reference, and designating as a foreign substance a foreign substance sample that is in a region other than the region of the object to be inspected in the RGB color space and has a color difference from the object to be inspected that exceeds a color difference threshold value among the divided regions; For a foreign substance sample in an area among the divided areas where the color difference in the RGB color space from the object to be inspected is equal to or less than a color difference threshold value, the foreign substance is not designated as a foreign substance in the visible light image, but if the shape of the foreign substance can be confirmed in an image of a position on the near-infrared image corresponding to the position recognized as a foreign substance on the visible light image, the image on the near-infrared image is designated as a foreign substance. A teacher data generating method comprising:

7. A foreign object detection device that optically detects foreign objects mixed in an object to be inspected that is placed on a transport path and transported, comprising: a visible light camera that captures an image of the inspection object illuminated by a first light illuminating means that illuminates the inspection object with visible light; and a near-infrared camera that captures an image of the inspection object irradiated with near-infrared light by a second light irradiating means that irradiates the inspection object with near-infrared light; a foreign matter determination means for determining whether a foreign matter is present in the object to be inspected based on a visible light image of the object to be inspected captured by the visible light camera and a near-infrared image of the object to be inspected captured by the near-infrared camera, The foreign matter determination means is a means for determining whether the object to be inspected is a foreign matter by referring to teacher data of a visible light image and a near-infrared light image of the foreign matter previously learned by a learning means, The teacher data is, in the learning process, Dividing the visible light image into three or more regions using RGB in an RGB color space as a color reference, and designating as a foreign substance a foreign substance sample that is in a region other than the region of the object to be inspected in the RGB color space and has a color difference from the object to be inspected that exceeds a color difference threshold value among the divided regions; For a foreign substance sample in an area among the divided areas where the color difference in the RGB color space from the object to be inspected is equal to or less than the color difference threshold, the foreign substance is not designated as a foreign substance in the visible light image, but if the shape of the foreign substance can be confirmed in an image at a position on the near-infrared image corresponding to the position recognized as a foreign substance on the visible light image, the image on the near-infrared image is designated as a foreign substance and learning is performed. A foreign body inspection device comprising:

8. The foreign object determination means includes: a first foreign matter determination means for determining whether or not a foreign matter has been mixed into the object to be inspected by using a learning model that has been trained on the basis of teacher data generated on the basis of a visible light image; a second foreign matter determination means for determining that an object to be determined, contained in the visible light image captured by the visible light camera, is a foreign matter mixed in the object to be inspected, when the color and size of the object to be determined are outside of a predetermined range; a third foreign matter determination means for determining whether or not a foreign matter has been mixed into the object to be inspected by using a learning model that has been trained on the basis of the teacher data generated on the basis of near-infrared light; a fourth foreign matter determination means for determining that a foreign matter has been mixed into the object to be inspected when a reflectance of near-infrared light having a different wavelength for the object to be determined, which is included in the near-infrared image captured by the near-infrared camera, is outside a predetermined range; a notification means for notifying the presence of a foreign matter based on the results of the determinations made by the first foreign matter determination means, the second foreign matter determination means, the third foreign matter determination means, and the fourth foreign matter determination means; 8. The foreign matter inspection apparatus according to claim 7, further comprising:

9. A foreign object detection method for optically detecting foreign objects present in an object to be inspected that is placed on a transport path and transported, comprising the steps of: a foreign matter determination step of determining whether a foreign matter is present in the object to be inspected based on a visible light image of the object to be inspected, the object being irradiated with visible light by a light irradiation means that irradiates the object to be inspected with visible light and captured with a visible light camera; The foreign matter determination step is a step of determining whether the object to be inspected is a foreign matter by referring to teacher data of a visible light image of the foreign matter previously learned in the learning step, The teacher data is, in the learning process, The visible light image is divided into three or more regions using RGB in an RGB color space as a color reference, and a foreign substance sample that is in a region other than the region of the object to be inspected and has a color difference from the object to be inspected in the RGB color space that exceeds a color difference threshold is designated as a foreign substance and learning is performed.

13. A foreign body inspection method comprising:

10. A foreign matter detection program to be executed by a foreign matter inspection device that optically detects foreign matters mixed in an object to be inspected that is placed on a transport path and transported, comprising: a foreign matter determination step of determining whether a foreign matter is present in the inspection object based on a visible light image of the inspection object obtained by capturing an image of the inspection object with visible light using a visible light camera after the inspection object has been irradiated with visible light by a light irradiating means for irradiating the inspection object with visible light, The foreign matter determination step is a step of determining whether the object to be inspected is a foreign matter by referring to teacher data of a visible light image of the foreign matter previously learned in the learning step, The teacher data is, in the learning process, The visible light image is divided into three or more regions using RGB in an RGB color space as a color reference, and a foreign substance sample that is in a region other than the region of the object to be inspected and has a color difference from the object to be inspected in the RGB color space that exceeds a color difference threshold is designated as a foreign substance and learning is performed. A foreign object detection program comprising:

Citation Information

Patent Citations

  • Plastic sorting system having front image processing and reflecting type spectrum unit

    CN106824824A

  • Tobacco leaf sorting method based on convolutional neural network

    CN109420622A

  • Visual inspection device and visual inspection method

    JP2015114226A

  • Inspection device

    JP2020051981A

  • Image inspection device and inspection model construction system

    JP2020051982A

Cited By

  • Vapor phase growth apparatus

    CN121700365A