System and method for determining the presence or absence of foreign matter in a liquid sealed in a container

Abstract

In foreign object inspection, this system provides a method for more reliably determining or identifying whether floating objects are foreign objects or air bubbles. [Solution] A system according to one embodiment of the present invention is a system for determining the presence or absence of foreign matter in a liquid sealed in a container, comprising: a floating object detection illumination device; a bubble foreign matter identification illumination device that emits light for identifying bubbles among the floating matter; an imaging device capable of imaging the liquid in the container, positioned to receive light emitted from the bubble foreign matter identification illumination device and totally reflected by bubbles; and a control device that identifies whether the floating matter in the liquid is foreign matter or bubbles based on the light intensity in a predetermined band including at least a part of the wavelength band of the light emitted by the bubble foreign matter identification illumination device in a region of the image acquired by the imaging device using the light emitted from the floating object detection illumination device and the bubble foreign matter identification illumination device.

Description

[Technical Field]

[0001] The present invention relates to a system and method for determining the presence or absence of foreign matter in a liquid sealed in a container. [Background technology]

[0002] Products containing liquids, such as injectable drugs, sealed in transparent containers are sold as good products only after passing various product inspections. One of these inspections involves checking for foreign matter inside the container; if foreign matter is present, the product is deemed defective.

[0003] Foreign object inspection is generally performed using the following procedure. (i) Illuminate the container with a light source. (ii) Take an image of the container with a camera. (iii) A data processing device detects floating objects in the image acquired by the camera and extracts the area containing floating objects if any are found. (iv) If floating matter is found, identify whether it is air bubbles or foreign matter, and if it is foreign matter, determine that "foreign matter present".

[0004] During the manufacturing process, bubbles often form in the liquid inside the container due to rotation of the container or agitation of the liquid. Bubbles are not solids and therefore not foreign matter, but both foreign matter and bubbles are "suspended matter" in the liquid and are often detected as suspended matter in step (iii) above. Step (iv) above is used to distinguish between foreign matter and bubbles, but if bubbles are mistakenly identified as foreign matter and judged as defective products, it will lead to a decrease in product yield, so it is important to distinguish between foreign matter and bubbles with high accuracy.

[0005] Foreign matter is introduced during the product manufacturing process, but the causes are diverse, and it is difficult to accurately predict the material, shape, color, transparency, weight, size, etc., of the foreign matter. On the other hand, the characteristics of bubbles are the same except for their size. The contents of a bubble are gas, which is colorless and transparent, has almost zero weight compared to a liquid, and has a smooth spherical shape.

[0006] For example, Patent Documents 1 to 3 disclose techniques for distinguishing between foreign matter and bubbles using the difference in their movement in a liquid as a clue. Patent Document 2 discloses a technique for determining that floating objects that trace a trajectory in the anti-gravity direction in a liquid are bubbles, and floating objects that do not trace such a trajectory are foreign matter. Patent Document 3 discloses a technique for improving the accuracy of foreign matter / bubble identification by changing the orientation of the container (the direction of gravity as seen from the container) and capturing the movement of floating objects caused by the change.

[0007] For example, Patent Documents 4 and 5 disclose techniques that utilize the shape characteristics of bubbles. Patent Document 4 discloses a method for distinguishing foreign objects from bubbles using feature quantities such as the differential value distribution of an image, the number of edge pixels, and the perimeter length, while Patent Document 5 discloses a technique for distinguishing foreign objects from bubbles by measuring whether they have a hollow shape characteristic of bubbles or a tilt characteristic of foreign objects.

[0008] For example, Patent Documents 6 and 7 disclose techniques for distinguishing between foreign objects and bubbles by utilizing the characteristics that bubbles with smooth surfaces are prone to specular and total internal reflection, and that bubbles are transparent. For example, Patent Document 6 discloses a technique for distinguishing between bubbles that appear high brightness and foreign objects that appear low brightness by adjusting the relative light intensity of the light sources of the transmitted and reflective light sources, using the difference in brightness. Patent Document 7 also discloses a technique for determining that floating objects that strongly exhibit total and scattered reflection are bubbles by shining a high-brightness light from diagonally above and in front of a camera. [Prior art documents] [Patent Documents]

[0009] [Patent Document 1] Japanese Patent Publication No. 2019-174346 [Patent Document 2] WO2021 / 214994 [Patent Document 3] Japanese Patent Publication No. 2023-125640 [Patent Document 4] Japanese Patent Publication No. 2004-354100 [Patent Document 5] Japanese Patent Publication No. 2013-44688 [Patent Document 6] Japanese Patent Publication No. 2001-116703 [Patent Document 7] Japanese Patent Publication No. 2015-10965 [Overview of the Initiative] [Problems that the invention aims to solve]

[0010] When foreign matter and bubbles are between 100 μm and 300 μm in size, gravity and buoyancy are weaker compared to larger objects, and the force dragging them by the liquid due to its viscosity becomes relatively stronger. Therefore, the differences in the movement of foreign matter and bubbles become less pronounced as their size decreases, which is why the technologies disclosed in Patents 1-3 have the challenge of being unable to distinguish between small foreign matter and bubbles smaller than approximately 300 μm.

[0011] When imaging small foreign objects or bubbles measuring approximately 100 μm to 300 μm, it is desirable that each pixel in the image be a fraction of that size (10 μm to several tens of μm). To achieve such resolution, it is necessary to select a camera with a small pixel pitch and high optical magnification, which results in a small depth of field for such a camera. When the thickness of the container is several centimeters or more from the camera's perspective, it is difficult to focus on the entire area, and some degree of image blurring is unavoidable. The technologies disclosed in Patent Documents 4 and 5 are based on the premise that the shape characteristics of bubbles are clearly captured, and there was a problem in that it was difficult to distinguish between small foreign objects and bubbles measuring approximately 300 μm or less in size.

[0012] The method described in Patent Document 6 causes specular reflection on the container by irradiating it with light from the front, making it difficult to see what is inside the container. Generally, in such cases, the position of the lighting is shifted to reduce the likelihood of specular reflection on the container, but in the case of Patent Document 6, there is a condition that the lighting be placed in a position where specular reflection or total internal reflection occurs due to air bubbles, so there is little freedom in the lighting placement and it is difficult to prevent specular reflection on the container. The method described in Patent Document 7 is difficult to distinguish from air bubbles if specular reflection occurs when the foreign object is transparent or made of metal. The fact that it assumes the foreign object is opaque is also a limitation in its use. Thus, although the technologies disclosed in Patent Documents 6 and 7 could solve the problems shown in Patent Documents 1 to 5, they had the problem of being difficult to prevent the effects of specular reflection.

[0013] In one aspect, the present invention has been made to solve such problems and aims to provide a system that can more reliably determine or identify whether floating objects are foreign objects or air bubbles in foreign object inspection. [Means for solving the problem]

[0014] [1] The system of one embodiment of the present invention is A system for determining the presence or absence of foreign matter in a liquid sealed inside a container, A floating object detection lighting device that emits light to detect floating objects, A bubble foreign object identification illumination device that emits light to identify bubbles among suspended particles, An imaging device capable of imaging the liquid inside a container using light emitted from either or both of the floating object detection illumination device and the bubble foreign object identification illumination device, wherein the imaging device is positioned to receive light emitted from the bubble foreign object identification illumination device that has been totally reflected by bubbles, A control device that discriminates whether a floating object in a liquid is a foreign object or a bubble based on the light intensity in a predetermined band including at least a part of the wavelength band of the light emitted by the bubble foreign object discrimination illumination device in a region corresponding to the floating object in the image acquired by the imaging device using the light emitted from the floating object detection illumination device and the bubble foreign object discrimination illumination device. A system comprising the above.

[0015] 〔2〕In one embodiment of the present invention, The spectra of the light emitted by the floating object detection illumination device and the bubble foreign object discrimination illumination device are different. The imaging device is a multi-band camera capable of acquiring the light intensity in a plurality of wavelength bands including the predetermined band. The control device discriminates whether a floating object in a liquid is a foreign object or a bubble based on the light intensity in the predetermined band and the light intensity in the plurality of wavelength bands in a region corresponding to the floating object in the image acquired by the imaging device. The system according to 〔1〕.

[0016] 〔3〕In one embodiment of the present invention, The control device discriminates whether a floating object in a liquid is a foreign object or a bubble based on the change in the light intensity in the predetermined band in a region corresponding to the floating object in the first image acquired by the imaging device using the light emitted from the bubble foreign object discrimination illumination device and the floating object detection illumination device and the second image acquired by the imaging device using only the light emitted from the floating object detection illumination device. The system according to 〔1〕 or 〔2〕.

[0017] 〔4〕In one embodiment of the present invention, The control device is a system according to any one of [1] to [3], which calculates the movement speed of a floating object based on a first image acquired by the imaging device using light emitted from the bubble foreign object identification illumination device and the floating object detection illumination device, and a second image acquired by the imaging device at a predetermined time difference from the first image using light emitted only from the floating object detection illumination device, and further identifies whether the floating object in the liquid is a foreign object or a bubble based on the calculated movement speed.

[0018] [5] In one embodiment of the present invention, The floating object detection illumination device includes a first floating object detection illumination device arranged to penetrate the liquid and reach the imaging device, and a second floating object detection illumination device arranged so that light reflected by the floating object reaches the imaging device. The control device is a system according to any one of [1] to [4], which identifies whether floating objects in a liquid are foreign objects or bubbles based on one or both of the following: a change in light intensity in a predetermined band in the region corresponding to floating objects in a first image acquired by the imaging device using light emitted from the bubble foreign object identification lighting device and the first floating object detection lighting device, and a second image acquired by the imaging device using light emitted only from the first floating object detection lighting device; and a change in light intensity in a predetermined band in the region corresponding to floating objects in a third image acquired by the imaging device using light emitted from the bubble foreign object identification lighting device and the second floating object detection lighting device, and a fourth image acquired by the imaging device using light emitted only from the second floating object detection lighting device.

[0019] [6] In one embodiment of the present invention, The control device is The system is as described in [5], wherein the system identifies whether the floating object in the liquid is a foreign object or a bubble based on the change in light intensity in a predetermined band in the region corresponding to the floating object in a first image acquired by the imaging device using light emitted from the bubble foreign object identification illumination device and the first floating object detection illumination device, and a second image acquired by the imaging device using light emitted only from the first floating object detection illumination device, and if the floating object is identified as a bubble, it further identifies whether the floating object in the liquid is a foreign object or a bubble based on the change in light intensity in a predetermined band in the region corresponding to the floating object in a third image acquired by the imaging device using light emitted from the bubble foreign object identification illumination device and the second floating object detection illumination device, and a fourth image acquired by the imaging device using light emitted only from the second floating object detection illumination device.

[0020] [7] In one embodiment of the present invention, The floating object detection illumination device includes a first floating object detection illumination device arranged so that emitted light passes through the liquid and reaches the imaging device, and a second floating object detection illumination device arranged so that light reflected from the floating object reaches the imaging device. The system according to any one of [1] to [6], wherein the wavelength bands of the light emitted by the first floating object detection lighting device, the second floating object detection lighting device, and the bubble foreign object identification lighting device are different from each other, and the imaging device is a multiband camera that includes bands of a frequency band that includes at least a portion of the respective light bands.

[0021] [8] In one embodiment of the present invention, The system is as described in any one of [1] to [7], wherein the light emitted by the floating object detection illumination device is white light including blue, green, and red light, the light emitted by the bubble foreign object identification illumination device is blue, green, or red light, and the imaging device is a multiband camera including a band in a frequency range that includes at least a portion of the frequency range of the light emitted by the bubble foreign object identification illumination device.

[0022] [9] The system of one embodiment of the present invention is A system for determining the presence or absence of foreign matter in a liquid sealed inside a container, A floating object detection lighting device that emits light to detect floating objects, A bubble foreign object identification illumination device that emits light to identify bubbles among suspended particles, An imaging device capable of imaging the liquid inside a container using light emitted from either or both of the floating object detection illumination device and the bubble foreign object identification illumination device, wherein the imaging device is positioned to receive light emitted from the bubble foreign object identification illumination device that has been totally reflected by bubbles, A control device that identifies whether a floating object in a liquid is a foreign object or a bubble based on the light intensity of the region corresponding to the floating object in the image acquired by the imaging device using light emitted only from the floating object detection illumination device and the image acquired by the imaging device using light emitted only from the bubble foreign object identification illumination device. It is a system that includes [this feature].

[0023]

[10] In one embodiment of the present invention, The floating object detection illumination device includes a first floating object detection illumination device arranged to penetrate the liquid and reach the imaging device, and a second floating object detection illumination device arranged so that light reflected from the floating object reaches the imaging device. The control device is the system described in [9], which calculates the movement speed of a floating object based on one or both of two images acquired by the imaging device at a predetermined time difference using light emitted from the first floating object detection illumination device, and two images acquired by the imaging device at a predetermined time difference using light emitted from the second floating object detection illumination device, and further identifies whether the floating object in the liquid is a foreign object or a bubble based on the calculated movement speed.

[0024]

[11] The method of one embodiment of the present invention is A method for determining the presence or absence of foreign matter in a liquid sealed inside a container, This includes using light for detecting suspended matter and light for identifying bubbles and foreign objects in a liquid to identify whether the suspended matter in the liquid is a foreign object or a bubble, Identifying whether floating objects in a liquid are foreign objects or bubbles involves an imaging device positioned to receive bubble foreign object identification light totally reflected by bubbles in the liquid within the container, and identifying them based on the light intensity in a predetermined band including at least a portion of the wavelength band of the bubble foreign object identification light in the region corresponding to the floating objects in the image acquired using the floating object detection light and the bubble foreign object identification light. It is a method.

[0025]

[12] In one embodiment of the present invention, The spectra of the light used for detecting floating objects and the light used for identifying foreign objects in bubbles are different. The imaging device is a multiband camera capable of acquiring light intensity in multiple wavelength bands including the predetermined band, Identifying whether or not suspended matter in a liquid is a bubble includes identifying it based on the light intensity in a predetermined band and the light intensity in a plurality of wavelength bands in a region corresponding to the suspended matter in an image acquired using the imaging device. The method described in

[11] .

[0026]

[13] In one embodiment of the present invention, Identifying whether suspended particles in a liquid are bubbles or not is possible. The first image is acquired using light for detecting floating objects and light for identifying foreign objects in bubbles, To acquire a second image using only the light for detecting floating objects, Identifying whether the suspended matter in the liquid is foreign matter or air bubbles based on the change in light intensity in the predetermined band of the region corresponding to the suspended matter in the first and second images, The method described in

[11] or

[12] , including the following:

[0027]

[14] In one embodiment of the present invention, Identifying whether suspended particles in a liquid are bubbles or not is possible. The first image is acquired using light for detecting floating objects and light for identifying foreign objects in bubbles, A second image is acquired using only the light for detecting floating objects, either a predetermined time before or after the acquisition of the first image. The movement speed of the floating object is calculated based on the first and second images, Based on the calculated movement speed, it is possible to further identify whether the suspended matter in the liquid is foreign matter or air bubbles, The method is one of any one of

[11] to

[13] , including the method described above.

[0028]

[15] In one embodiment of the present invention, Identifying whether suspended particles in a liquid are bubbles or not is possible. The floating object detection light includes a first floating object detection light emitted to penetrate the liquid and reach the imaging device, and a second floating object detection light emitted to reflect light from the floating object and reach the imaging device. The wavelength bands of the first floating object detection light, the second floating object detection light, and the bubble foreign object identification light are different from each other, and the imaging device is a multiband camera that includes bands in a frequency range that includes at least a portion of the respective light bands. The method is one of any one of

[11] to

[14] , including the method described above.

[0029]

[16] The method of one embodiment of the present invention is A method for determining the presence or absence of foreign matter in a liquid sealed inside a container, Identifying whether suspended particles in a liquid are bubbles or not is possible. The first image is acquired using only light for detecting floating objects, The second image is acquired using only the light for identifying air bubbles and foreign objects, Based on the light intensity of the regions corresponding to the suspended objects in the first and second images, it is possible to identify whether the suspended objects in the liquid are foreign objects or bubbles, This method includes [something]. [Effects of the Invention]

[0030] In one respect, according to the present invention, it is possible to more reliably determine or identify whether floating objects are foreign objects or air bubbles in foreign object inspection. [Brief explanation of the drawing]

[0031] [Figure 1] This is a schematic diagram of the inspection system according to the first embodiment of the present invention, and is a schematic plan view of the inspection system as seen from above. [Figure 2] This is a diagram illustrating total internal reflection. [Figure 3] This figure shows an example of the spectral distribution (spectrum) of light emitted by the floating object detection illumination device and the bubble foreign object identification illumination device according to the first embodiment of the present invention, and an example of the spectral sensitivity of the imaging device. [Figure 4] This is a block diagram showing the hardware configuration of a control device according to the first embodiment of the present invention. [Figure 5] This is a functional block diagram of the control device according to the first embodiment of the present invention. [Figure 6] This figure shows a lighting pattern used in the first embodiment of the present invention. [Figure 7] This is an example of an image (conceptual diagram) of the liquid inside a container acquired by the imaging device of the first embodiment of the present invention. [Figure 8] This figure shows an example of pixel values ​​in each wavelength band (B band, G band, R band) of the pixel region corresponding to a bubble and the pixel region corresponding to a red foreign object in an image acquired by an imaging device when the bubble foreign object identification illumination device is turned on and turned off. [Figure 9] This figure shows an example of boundary setting for detecting air bubbles according to the color change index (CCI) in the first embodiment of the present invention. [Figure 10] This figure shows an example flowchart illustrating the process by which the inspection system in the first embodiment of the present invention determines the presence or absence of foreign matter in the liquid inside a container. [Figure 11] This figure shows an example of a flowchart illustrating the processing of the data processing unit in step S103 of Figure 10. [Figure 12] This table shows the types and sizes of foreign matter contained in the liquid sealed in the container during the example experiment, as well as the detection results for these foreign matter and the results of distinguishing them from air bubbles. [Figure 13] This figure shows the two-dimensional feature space and decision boundary, which consist of two features, color change index and movement speed, obtained from the experiment in the example. [Figure 14] This figure shows the lighting pattern used in the second embodiment. [Figure 15] This figure shows an example of the spectral distribution (spectrum) of light emitted by the floating object detection illumination device and the bubble foreign object identification illumination device according to the second embodiment of the present invention, and an example of the spectral sensitivity of the imaging device. [Figure 16] This figure shows the lighting pattern used in the fourth embodiment. [Modes for carrying out the invention]

[0032] Embodiments of the present invention will be described below with reference to the drawings. The systems (apparatus) and methods of the embodiments of the present invention are for identifying (determining) whether suspended matter is a foreign substance or a bubble when suspended matter is detected, present, or known to be present in a liquid sealed in a container during foreign matter inspection. One embodiment of the present invention is an inspection system for determining the presence or absence of foreign matter in a liquid sealed in a container, and may consist of one or more devices as long as they perform the operations described in the embodiments of the present invention. The methods of the embodiments of the present invention can have the same effects as the inspection system. In the embodiments of the present invention, the container containing the liquid to be inspected is a transparent container, and a transparent container means a container in which at least a portion of the light can be transmitted or refracted and pass through the container when light is shone on the container. In the embodiments of the present invention, suspended matter is either a bubble or a foreign substance. In one example, the liquid is a liquid medicine such as an injectable drug, and the container is a sealed container for medicine. Foreign matter inspection may also include re-inspection.

[0033] The systems and methods of the embodiments of the present invention are for identifying whether floating objects are foreign objects or bubbles, but since the information processing involves determination, identification may sometimes mean determination, and determination may sometimes mean identification. In this specification, arranging one device to have a predetermined positional relationship with another device can mean arranging one device and another device to have a predetermined positional relationship, and it can also mean arranging the other device to have a predetermined positional relationship with the one device. For the sake of explanation, the vertical and horizontal scales of components or parts may be shown differently from the actual scales.

[0034] [First Embodiment] Figure 1 is a schematic diagram of the inspection system 1 according to the first embodiment of the present invention, and is a schematic plan view of the inspection system 1 as seen from above. The inspection system 1 comprises a holding unit 4, a control device 10, an imaging device 30, a floating object detection illumination device 40, and a bubble foreign object identification illumination device 50. In Figure 1, the holding unit 4 and the container 6 are intentionally depicted as large for ease of explanation, but the size of the objects (devices, etc.) shown in Figure 1 does not represent the actual size.

[0035] The holding unit 4 is a holding unit or holding mechanism for holding the container 6, and can be anything that can hold or fix the container 6. For example, it can be a fixing stand on which the container 6 can be fixed and placed. The container 6 that is held contains liquid 8. For the sake of convenience, in the explanation of the operation of the inspection system 1, it will be assumed that the liquid 8 sealed in the container 6 contains air bubbles 61 and foreign matter 62.

[0036] The imaging device 30 is a device capable of imaging (photographing) the liquid 8 inside the container 6, and is positioned to image the entire container 6. The imaging device 30 is a general RGB camera and can acquire the light intensity (light intensity, light intensity information) of the respective wavelength bands (B band, G band, R band) of blue, green, and red. The imaging device 30 can image the liquid 8 inside the container 6 using light emitted from either or both of the floating object detection illumination device 40 and the bubble foreign object identification illumination device 50. In the embodiment of the present invention, the side of the container 6 on which the imaging device 30 is positioned is called the front side, and the side facing the imaging device 30 across the container 6 is called the back side.

[0037] The floating object detection lighting device 40 is a lighting device that emits light to detect floating objects in the liquid 8, and is positioned to irradiate the liquid 8 in the container 6 with light. The floating object detection lighting device 40 includes a rear-side lighting device 41 located on the rear side and a front-side lighting device 42 located on the front side.

[0038] The imaging device 30 and the rear-side illumination device 41 are positioned so that light emitted from the rear-side illumination device 41 can pass through the liquid 8 and reach the imaging device 30. In one example, the rear-side illumination device 41 is large enough to include the entire container 6 within the imaging range (shooting range) of the imaging device 30, and appears as a uniform black when the light is off.

[0039] The front-side illumination device 42 includes a first front-side illumination device 43 and a second front-side illumination device 44. The imaging device 30 and the first front-side illumination device 43 are positioned so that light emitted from the first front-side illumination device 43 and reflected by floating objects can reach the imaging device 30. Similarly, the imaging device 30 and the second front-side illumination device 44 are positioned so that light emitted from the second front-side illumination device 44 and reflected by floating objects can reach the imaging device 30. In one example, as shown in Figure 1, the first front-side illumination device 43 and the second front-side illumination device 44 are positioned symmetrically with respect to a line connecting the imaging device 30 and the container 6.

[0040] The bubble foreign object identification illumination device 50 is an illumination device that emits light for identifying bubbles among suspended matter, and is positioned to irradiate the liquid 8 in the container 6 with light. In the embodiment of the present invention, suspended matter is either bubbles or foreign matter, and if suspended matter is identified as not being bubbles, it is identified as foreign matter. Therefore, the light for identifying bubbles among suspended matter is the light for identifying whether the suspended matter is foreign matter or bubbles. The bubble foreign object identification illumination device 50 includes a first identification illumination device 51 and a second identification illumination device 52. The imaging device 30 and the first identification illumination device 51 are positioned so that the imaging device 30 can receive light emitted from the first identification illumination device 51 that has been totally reflected by bubbles. Similarly, the imaging device 30 and the second identification illumination device 52 are positioned so that the imaging device 30 can receive light emitted from the second identification illumination device 52 that has been totally reflected by bubbles. The light path 72 shown in Figure 1 is an example of the path of light emitted from the first identification illumination device 51 that undergoes total internal reflection by the bubble 61 and reaches the imaging device 30.

[0041] In embodiments of the present invention, lighting devices such as the floating object detection lighting device 40 and the bubble foreign object identification lighting device 50 are light sources that emit light, and their on / off states can be controlled by a computer such as the control device 10, and are, for example, LEDs that light up when power is supplied.

[0042] Figure 2 illustrates total internal reflection. When light travels from a medium with a higher refractive index to a medium with a lower refractive index, it normally splits into refracted and reflected light. However, when light is incident on the interface at an angle above a certain incidence angle (called the critical angle), the refracted light disappears, and only reflected light remains. This phenomenon is called total internal reflection. The critical angle θ when light is incident from water with a refractive index n1 of approximately 1.333 to air with a refractive index n2 of approximately 1.000. c is, θ c =sin -1 (n2 / n1) is used to find θ c It is approximately 48.6 degrees.

[0043] The inside of the bubble 61 is a gas with a refractive index lower than that of the surrounding liquid, and a clear, smooth interface exists between the liquid and the gas. As a result, most of the light that passes through the liquid and enters the bubble is reflected. In other words, total internal reflection occurs. In addition, because the shape is close to a sphere and composed of surfaces facing various directions, there is a certain width in the direction of propagation of the light after total internal reflection. No matter where the bubble 61 is located in the liquid 8, some of the light that has been totally reflected reaches the imaging device 30. In the case of total internal reflection, unlike diffuse reflection, a large portion of the energy of the incident light is reflected in one direction without diffusion and reaches the imaging device 30. Therefore, the intensity of the light received by the imaging device 30 in that area becomes particularly high, or in other words, the pixel value in that area becomes particularly high. On the other hand, foreign objects are irregular in shape and their material varies. Even if some of the light emitted by the bubble foreign object identification illumination device 50 is reflected by the foreign object, in many cases the proportion of reflection is not as large as in the case of bubbles. Therefore, in the image acquired by the imaging device 30, areas where there is a lot of light from the bubble foreign object identification illumination device 50 are likely to contain bubbles. The inventors have focused on the physical properties of bubbles and total internal reflection described above, and these physical properties are utilized in the embodiments of the present invention.

[0044] In one example, the first identification illumination device 51 is positioned as shown in Figure 1 such that the angle 71 between the imaging device 30, the center of the container 6, and the first identification illumination device 51 is approximately 100 to 120 degrees. This positioning allows the imaging device 30 to capture the light totally reflected by the bubbles. For example, if the refractive index of the liquid in the container is close to that of water and the refractive index of the gas is close to that of air, the critical angle is approximately 50 degrees as described above, and theoretically, total internal reflection will occur if the angle 71 is greater than twice that, approximately 100 degrees. On the other hand, if the angle 71 is too large, the totally reflected light will not reach the imaging device 30. In one example, the positions of the imaging device 30 and the bubble foreign object identification illumination device 50 are determined by adjusting them while observing the images actually acquired by the imaging device 30, using the above-mentioned value of 100 to 120 degrees as a reference.

[0045] Figure 3(a) shows an example of the spectral distribution of light emitted by the floating object detection illumination device 40 and the bubble foreign object identification illumination device 50 according to the first embodiment of the present invention. The detection light 40L emitted by the floating object detection illumination device 40 is white light containing components in the wavelength band of the visible light region, i.e., white light containing blue, green, and red light. The identification light 50L emitted by the bubble foreign object identification illumination device 50 is red light. For example, the blue light is light having a predetermined wavelength band centered around 460 nm, the green light is light having a predetermined wavelength band centered around 540 nm, and the red light is light having a predetermined wavelength band centered around 620 nm. Hereinafter, the light emitted by the floating object detection illumination device 40 may be simply referred to as the detection light 40L, and the light emitted by the bubble foreign object identification illumination device 50 may be simply referred to as the identification light 50L. In Figure 3(a), for ease of understanding, the detection light 40L is shown as having a constant intensity across the entire visible light wavelength range, but this is just one example of the case where the detection light 40L is white light. As shown in Figure 3(a), the spectral distributions of the detection light 40L and the identification light 50L are different, and these lights have different wavelength bands.

[0046] Figure 3(b) shows an example of the spectral sensitivity of the imaging device 30 according to the first embodiment of the present invention. The imaging device 30 is a general RGB camera and is sensitive to the respective wavelength bands of blue (B), green (G), and red (R) light. Since the imaging device 30 is sensitive to the wavelength band of red light corresponding to the red light of the identification light 50L (R-band sensitivity), it can detect the red light of the identification light 50L. In order to obtain the effects of this embodiment, the imaging device 30 and the bubble foreign object identification illumination device 50 can be configured such that the wavelength band of red light to which the imaging device 30 is sensitive and the wavelength band of red light of the identification light 50L overlap at least partially so that the identification light 50L is detected by the imaging device 30. The image captured by the imaging device 30, unless there is an error during shooting, includes the pixel values ​​of each wavelength band (B-band, G-band, R-band), i.e., the intensity of light received by the image sensor, for all pixels.

[0047] The control device 10 is communicatively connected to the imaging device 30, the floating object detection illumination device 40, and the bubble foreign object identification illumination device 50, and controls these devices.

[0048] Figure 4 is a block diagram showing the hardware configuration of a control device 10 according to a first embodiment of the present invention. The control device 10 comprises a processor 11, a display device 12, an input device 13, a storage device 14, and a communication device 15. These components are connected by a bus 16. Interfaces are interposed between the bus 16 and each component as needed. For example, the control device 10 may consist of one or more computers, or it may include dedicated modules or devices for controlling the device.

[0049] The processor 11 controls the operation of the control device 10. For example, the processor 11 is a CPU. The processor 11 performs various processes by reading and executing programs and data stored in the storage device 14. The processor 11 may be composed of multiple processors.

[0050] The display device 12 is a display that shows application screens and the like to the user of the inspection system 1. The input device 13 is a user interface that receives input from the user to the inspection system 1, such as a keyboard or mouse.

[0051] The storage device 14 includes main memory and auxiliary storage. The main memory is, for example, volatile memory capable of high-speed reading and writing of information, and is used as a storage area and work area when the processor 11 processes information. The auxiliary storage stores various programs and data used by the processor 11 when executing each program. The auxiliary storage is non-volatile storage or non-volatile memory, and may be removable.

[0052] The communication device 15 is a module, device, or apparatus capable of exchanging data with the imaging device 30, the floating object detection illumination device 40, and the bubble foreign object identification illumination device 50 via a network. The communication device 15 can be a wireless communication device or module, or a wired communication device or module. If the user input / output of the inspection system 1 is limited to input / output via the communication device 15, the inspection system 1 does not need to be equipped with a display device 12 and an input device 13.

[0053] Figure 5 is a functional block diagram of a control device 10 (inspection system 1) according to the first embodiment of the present invention. The control device 10 comprises a device control unit 21 and a data processing unit 22. For example, if these functions are realized by a program being executed by a processor 11, one part (function) may be comprised of part or all of another part. However, in this case, these functions may also be realized by hardware by configuring electronic circuits, etc., to realize part or all of the device control unit 21 and the data processing unit 22.

[0054] The device control unit 21 controls the floating object detection lighting device 40 and the bubble foreign object identification lighting device 50, turning each light on or off. The device control unit 21 turns on or off the lights of the bubble foreign object identification lighting device 50, the rear lighting device 41, and the front lighting device 42, respectively, to cause the imaging device 30 to image the liquid 8 in the container 6. Note that turning on the bubble foreign object identification lighting device 50 means that the two lighting devices 51 and 52, the first identification lighting device 51 and the second identification lighting device 52, are turned on, and turning off the bubble foreign object identification lighting device 50 means that these two lighting devices are turned off. Similarly, turning on the front lighting device 42 means that the two lighting devices 43 and 44, the first front lighting device 43 and the second front lighting device 44, are turned on, and turning off the front lighting device 42 means that these two lighting devices 43 and 44 are turned off. In the embodiments of the present invention, for the sake of explanation, the on / off state of each light is represented using a lighting pattern that indicates the on / off state of three lights: the bubble foreign object identification lighting device 50, the rear lighting device 41, and the front lighting device 42, with "1" representing the on state of each light and "0" representing the off state. For example, lighting pattern "110" indicates that the bubble foreign object identification lighting device 50 and the rear lighting device 41 are on, and the front lighting device 42 is off.

[0055] Figure 6 shows an illumination pattern used in the first embodiment of the present invention. The imaging device 30 captures images for each of the illumination patterns shown in Figure 6 and acquires an image. In this specification, the expression "image captured by the imaging device 30" or "acquired image" can be understood to mean that the image captured by the imaging device 30 is stored in the memory or storage device 14 of the imaging device 30 to acquire an image.

[0056] Figure 7 is an example of an image (conceptual diagram) of the liquid 8 in a container 6 acquired by an imaging device 30 of the first embodiment of the present invention. This image 160 is an image captured by the imaging device 30 under illumination pattern "110". Image 160 shows bubbles and foreign matter. That is, it includes a pixel area 161 corresponding to bubbles and a pixel area 162 corresponding to foreign matter. Before distinguishing between foreign matter and bubbles, these are detected as pixel areas corresponding to floating objects. For the sake of explanation, Figure 7 shows an image in which bubbles and foreign matter can be distinguished. The pixel area 161 corresponding to bubbles is generally shown with a reddish tint due to the effect of total internal reflection of bubbles. The pixel area 162 corresponding to foreign matter is not displayed with a reddish tint and is shown darker compared to the background (around the foreign matter). For example, if the foreign matter is opaque, the pixel area 162 corresponding to the foreign matter is shown darker than the background, and if the foreign matter is somewhat transparent, the pixel area 162 corresponding to the foreign matter is shown slightly darker than the background. For example, if the pixel area corresponding to a floating object is several pixels or larger in both the horizontal and vertical directions, it can be confirmed that there is a difference between the pixel area 161 corresponding to a bubble and the pixel area 162 corresponding to a foreign object, as shown in Figure 7. Similarly, in the image captured by the imaging device 30 under illumination pattern "101", there is a difference between the two pixel areas. Note that the pixel area corresponding to a floating object varies depending on the dimensions of the inspection system 1, the size of the floating object, the spatial resolution of the image, etc., but it is often around several pixels to about 10 pixels in both the horizontal and vertical directions.

[0057] The data processing unit 22 determines whether a pixel region corresponding to a floating object in an image acquired by the imaging device 30 using detection light 40L and identification light 50L is a foreign object or a bubble, based on the light intensity in a predetermined band that includes at least a portion of the wavelength band of the identification light 50L in the pixel region corresponding to the floating object in the image acquired by the imaging device 30 using detection light 40L and identification light 50L. In the first embodiment, more specifically, the data processing unit 22 identifies / determines whether a floating object is a foreign object or a bubble based on the change in light intensity in a predetermined band that includes at least a portion of the wavelength band of the identification light 50L in the pixel region corresponding to a floating object in an image acquired by the imaging device 30 using detection light 40L and identification light 50L. It is understood that determining whether a pixel region corresponding to a floating object is a foreign object or a bubble can mean identifying / determining whether the floating object in the image is a foreign object or a bubble.

[0058] The image captured by the imaging device 30 shows the light intensity of each pixel, which is the sum of the light intensities of the blue, green, and red wavelength bands. The data processing unit 22 calculates a color index CI in the image acquired by the imaging device 30, which is an index that indicates the extent to which the pixel area corresponding to the floating object contains red light intensity corresponding to the wavelength band (R band) of the light emitted by the bubble foreign object identification illumination device 50. The color index CI is calculated by the following equation 1. CI = R / (B + G + R) (Equation 1) Here, R, G, and B are the average pixel values ​​of the R band, G band, and B band, respectively, in the pixel region corresponding to the floating object.

[0059] Furthermore, the data processing unit 22 calculates the color change index CCI, which is an indicator showing the degree of change in the color index in the image when the bubble foreign object identification illumination device 50 is turned on and turned off. The color change index CCI is calculated by the following equation 2. CCI=CI ON -CI OFF (Formula 2) Here, CION This is the CI calculated from the image when the bubble foreign object identification lighting device 50 is lit, and CI OFF This is the CI calculated from the image taken when the bubble foreign object identification lighting device 50 was turned off.

[0060] Figure 8 shows an example of pixel values ​​in each wavelength band (B band, G band, R band) of the pixel region corresponding to a bubble and the pixel region corresponding to a red foreign object in an image acquired by the imaging device 30 when the bubble foreign object identification illumination device 50 is lit and when it is turned off, according to the first embodiment of the present invention. A red foreign object is a foreign object that is red in appearance. The pixel value indicates the intensity of the light acquired by the imaging device 30. In Figure 8, the pixel values ​​of each band in the pixel region corresponding to the bubble and the pixel values ​​of each band in the pixel region corresponding to the foreign object represent the average value of the pixel values ​​of each band of the pixels included in each region. When the illumination pattern is "010", the bubble foreign object identification illumination device 50 is turned off, and in the pixel region corresponding to the bubble (b), all bands have similar pixel values ​​(light intensity) corresponding to the white light from the back-side illumination device 41, whereas in the pixel region corresponding to the red foreign object (a), the pixel value of the R band is slightly larger, corresponding to the color of the foreign object. When the lighting pattern is "110", the bubble foreign object identification lighting device 50 is lit, and in the pixel area (d) corresponding to the bubble, the pixel value of the R band increases significantly, while in the pixel area (c) corresponding to the red foreign object, only the pixel value of the R band increases slightly, similar to when the bubble foreign object identification lighting device 50 is turned off.

[0061] When the bubble foreign object identification lighting device 50 is off, only the floating object detection lighting device 40 is lit, resulting in a small color index for bubbles and a large color index for red foreign objects due to the white light. When the bubble foreign object identification lighting device 50 is lit, the color index of bubbles increases significantly due to total internal reflection, whereas foreign objects generally do not undergo total internal reflection, resulting in a small increase in the color index of foreign objects. Since the color change index is relatively large for bubbles and relatively small for foreign objects, it is possible to distinguish between bubbles and foreign objects. However, objects with an extremely large color change index may be unexpected foreign objects, so it is necessary to identify them as foreign objects.

[0062] The data processing unit 22 uses the calculated color change index (CCI) value to determine whether the pixel region corresponding to a floating object corresponds to a foreign object or a bubble. Figure 9 shows an example of boundary setting for bubble / foreign object determination according to the color change index (CCI) in the first embodiment of the present invention. CCI is a one-dimensional feature vector, and it can be used to distinguish floating objects into either the bubble class or the foreign object class. The class determination boundary can be determined experimentally. In one example, the CCI for which a bubble is determined is set within the range of a lower limit and an upper limit. This is because a larger CCI is more likely to be a bubble, but if it is larger than a certain size, it is more likely to be a foreign object rather than a bubble.

[0063] In the first embodiment, the data processing unit 22 determines whether the floating object is a foreign object or a bubble based on one or both of the following: a CCI showing the change in light intensity in the R band of the region corresponding to the floating object in the image of the liquid 8 in the container 6 captured by the imaging device 30 with illumination pattern "110" and the image of the liquid 8 in the container 6 captured by the imaging device 30 with illumination pattern "010"; and a CCI showing the change in light intensity in the R band of the region corresponding to the floating object in the image of the liquid 8 in the container 6 captured by the imaging device 30 with illumination pattern "101" and the image of the liquid 8 in the container 6 captured by the imaging device 30 with illumination pattern "001".

[0064] Figure 10 is a diagram illustrating an example of a flowchart that explains the process by which the inspection system 1 in the first embodiment of the present invention determines whether or not there are foreign substances in the liquid 8 in the container 6.

[0065] In step S101, the container 6 to be inspected is fixed in place by the holding unit 4. In step S102, the device control unit 21 instructs the imaging device 30 to image the liquid 8 inside the container 6 using four illumination patterns: "110", "010", "101", and "001", and acquire images. In step S102, the device control unit 21 instructs the imaging device 30 to take images with illumination patterns "110" and "010" with a predetermined time difference, and stores the two acquired images as images of image group A. In step S103, the data processing unit 22 determines the presence or absence of foreign matter in the liquid 8 inside the container 6 based on the images acquired in step S102.

[0066] Figure 11 is a diagram showing an example of a flowchart illustrating the processing of the data processing unit 22 in step S103 of Figure 10.

[0067] In step S201, the data processing unit 22 uses the images captured by the imaging device 30 in step S102 with illumination patterns "010" and "001" to determine whether or not the liquid 8 in the container 6 contains suspended matter. For example, the data processing unit 22 can determine the presence or absence of suspended matter using a known method. In one example, the data processing unit 22 can make the determination by comparing the image captured by the imaging device 30 in step S102 with illumination pattern "010" with an image of liquid 8 that does not contain suspended matter, which was previously captured by the imaging device 30 with illumination pattern "010", and further by comparing the image captured by the imaging device 30 in step S102 with illumination pattern "001" with an image of liquid 8 that does not contain suspended matter, which was previously captured by the imaging device 30 with illumination pattern "001". The image used by the data processing unit 22 does not have to be an image captured by the imaging device 30 with illumination pattern "010", but may be one of the images acquired in step S102.

[0068] In step S201, if it is determined that there is no floating matter, the data processing unit 22 determines "no foreign matter" in step S211, and this flowchart ends.

[0069] In step S201, if it is determined that there is floating matter, the data processing unit 22 selects one image group in step S202. In the first step S202, the data processing unit 22 selects image group A.

[0070] In step S203, the data processing unit 22 selects one of the floating matters in the image captured by the imaging device 30 with the illumination pattern "010" in step S102. In other words, the data processing unit 22 selects a pixel region corresponding to one of the floating matters in the pixel regions corresponding to the floating matters in the image captured by the imaging device 30 with the illumination pattern "010" in step S102. The data processing unit 22 may determine the pixel region corresponding to the floating matter in the image captured by the imaging device 30 with the illumination pattern "010" in step S102 in step 203, or in step S201.

[0071] In step S204, the data processing unit 22 determines whether the floating matter selected in step S203 is a bubble or a foreign object. More specifically, the data processing unit 22 calculates the color change index (CCI) and the moving speed of the pixel region corresponding to the floating matter, and determines whether the pixel region corresponding to the floating matter selected in step S203 is a pixel region corresponding to a bubble or a pixel region corresponding to a foreign object.

[0072] In step S204, the data processing unit 22 uses Equation 2 to set the CI calculated in the image obtained with the illumination pattern "110" as CI ON and sets the CI calculated in the image obtained with the illumination pattern "010" as CI OFFThe Color Change Index (CCI) is calculated. The data processing unit 22 also calculates the distance traveled by the floating object in the image captured with illumination pattern "010" and the floating object in the image captured with illumination pattern "110" in step S102, using, for example, the number of pixels at the center position. Since the two images are acquired with a predetermined time difference, the data processing unit 22 determines the movement speed of the floating object from the distance and time. The predetermined time difference is determined from a predetermined imaging time, for example, 0.01 seconds to several seconds. The range of CCI and movement speed that a bubble can take is predetermined, and the data processing unit 22 determines that the floating object selected in step S203 is a bubble if the calculated CCI and movement speed are within the predetermined range, and determines it to be a foreign object otherwise.

[0073] If a foreign object is detected in step S204, the data processing unit 22 determines "Foreign object present" in step S212 and terminates this flowchart.

[0074] If it is determined in step S204 that the floating object is a bubble, the data processing unit 22 determines in step S205 whether there are any other floating objects for which the determination process in step S204 has not been performed. In other words, if it is determined in step S204 that the pixel area corresponding to the floating object selected in step S203 corresponds to a bubble, the data processing unit 22 determines in step S205 whether there are any other pixel areas corresponding to floating objects for which the determination process in step S204 has not been performed.

[0075] If it is determined in step S205 that other floating objects are present, this flowchart proceeds to step S203, and in the second and subsequent steps of step S203, the data processing unit 22 selects one of the other floating objects for which the determination process in step S204 has not been performed. In other words, in the second and subsequent steps of step S203, the data processing unit 22 selects a pixel area corresponding to one of the other floating objects for which the determination process in step S204 has not been performed in the image captured by the imaging device 30 with the illumination pattern "010" in step S102. Hereafter, this flowchart executes steps S203 to S205 until there are no more floating objects (pixel areas corresponding to floating objects) for which the determination process in step S204 has not been performed, or until it is determined in step S212 that foreign objects are present.

[0076] In step S206, the data processing unit 22 determines whether there are other image groups. If it determines that there are none, it determines "no foreign objects" in step S207, and this flowchart ends. In step S206, the data processing unit 22 determines whether there are other image groups. If it determines that there are, this flowchart proceeds to step S202. In the second step S202, the data processing unit 22 selects image group B. Alternatively, the data processing unit 22 may be configured to select image group B in the first step S202 and image group A in the second step S202.

[0077] As described above, the flowchart shown in Figure 11 can be said to include acquiring a first image of the liquid 8 in the container 6 with the imaging device 30 using the illumination pattern "110", acquiring a second image of the liquid 8 in the container 6 with the imaging device 30 using the illumination pattern "010", and determining whether the floating objects are foreign matter or bubbles based on the change in light intensity in the R band of the region corresponding to the floating objects in the first and second images. Alternatively, the flowchart shown in Figure 11 can be said to include acquiring a first image of the liquid 8 in the container 6 with the imaging device 30 using the illumination pattern "110", acquiring a second image of the liquid 8 in the container 6 with the imaging device 30 using the illumination pattern "010" a predetermined time before or after acquiring the first image, calculating the movement speed of the floating objects based on the first and second images, and determining whether the floating objects are foreign matter or bubbles based on the change in light intensity in the R band of the region corresponding to the floating objects in the first and second images and the calculated movement speed.

[0078] Next, the operation and effects of the inspection system 1 according to the first embodiment of the present invention will be described. The inspection system 1 comprises a floating object detection illumination device 40, a bubble foreign object identification illumination device 50, an imaging device 30, and a control device 10. The detection light 40L emitted by the floating object detection illumination device 40 is white light, and the identification light 50L emitted by the bubble foreign object identification illumination device 50 is red light, and the spectral distributions of these lights are different. The imaging device 30 is an RGB camera that is sensitive to the wavelength bands of blue (B), green (G), and red (R) light, respectively. The imaging device 30 has sensitivity in the R band corresponding to the red light of the identification light 50L and can detect the identification light 50L, and since it has sensitivity in the B, G, and R bands, it can also detect the detection light 40L. The imaging device 30 can image the liquid 8 in the container 6 using either or both of the detection light 40L and the identification light 50L, and is positioned to receive the identification light 50L that has been totally reflected by bubbles. The imaging device 30 takes images with illumination patterns "110" and "010" with a predetermined time difference and stores the two acquired images as images of image group A. It then takes images with illumination patterns "101" and "001" with a predetermined time difference and stores the two acquired images as images of image group B. The control device 10 uses the images taken by the imaging device 30 with illumination patterns "010" and "001" to determine whether or not the liquid 8 in the container 6 contains suspended matter. If it is determined that no suspended matter is present, the control device 10 determines "no foreign matter" and terminates the process. If it is determined that suspended matter is present, the control device 10 calculates the CCI and movement speed from the two images of image group A for the region corresponding to one suspended matter in the image and determines whether the target suspended matter is a foreign object or a bubble. If it is determined to be a foreign object, the control device 10 determines "foreign object present" and terminates the process. If it is determined that there is no foreign object, the control device 10 calculates the CCI and movement speed for each region corresponding to one floating object in the image, and determines whether the target floating object is a foreign object or a bubble. If it is determined that none of them are foreign objects, the same determination process is performed on the two images of image group B as on the images of image group A. If either determination is found to be "foreign object present", the control device 10 determines that "foreign object present" and terminates the process. If it is determined that none of them are foreign objects, the control device 10 determines that "no foreign object present" and terminates the process.

[0079] As described above, in this embodiment, the imaging device 30 is configured to be positioned to receive the identification light 50L totally reflected by the bubble, while the detection light 40L and the identification light 50L have different spectral distributions, and the imaging device 30 is configured to have sensitivity in multiple wavelength bands corresponding to the identification light 50L. This makes it possible to detect the identification light 50L totally reflected from the bubble with a stronger light intensity, and to separate the two lights by the difference in pixel values ​​(light intensity) between bands, thereby enabling the separation of the detection light 40L and the identification light 50L with a certain level of accuracy. This configuration is unprecedented and makes it possible to distinguish between foreign objects and bubbles, even for small suspended particles of about 100 μm to 300 μm (for example, unlike the image shown in Figure 7, where it is difficult to distinguish between bubbles and foreign objects based solely on the shape of the suspended particles in the image), as well as for larger suspended particles. For example, since the inspection system 1 of this embodiment does not use a method that relies on edges or shape as clues, it is possible to detect small foreign objects of about 100 μm to 300 μm in a container of about several centimeters in size without mistaking them for bubbles, even if the captured image cannot be accurately focused across the entire area.

[0080] In this embodiment, by using a color change index, it becomes possible to determine whether the light intensity of the identification light 50L in the pixel region corresponding to the floating object in the image is relatively strong or not, and it becomes possible to determine whether the floating object is either a bubble or a foreign object. Calculating a color change index for the pixel region corresponding to a floating object and using it to determine whether the floating object is either a bubble or a foreign object, and selecting or determining a floating object detection illumination device 40 and a bubble / foreign object identification illumination device 50 that consider spectral distribution and sensitivity, and an imaging device 30 that considers sensitivity, so that the color change index can be used for determination, are things that have not been done in the past. In this embodiment, it is possible to detect the identification light 50L with a larger pixel value and determination can also be made using the color index, but as shown in Figure 8, in special cases such as when the foreign object itself is red, the color change index can distinguish bubbles and foreign objects more clearly.

[0081] Furthermore, in this embodiment, by using two indicators—a color change index and conventionally used movement speed—for example, by obtaining a two-dimensional feature vector and making the determination in a two-dimensional feature space, it becomes possible to distinguish with even greater accuracy whether the floating object is a bubble or a foreign object. In this embodiment, it is also possible to make the determination using only the color change index, and even in that case, it is possible to distinguish with greater accuracy whether the floating object is a bubble or a foreign object. When making the determination using only the color change index, it is not necessary to calculate the movement speed.

[0082] In this embodiment, the floating object detection lighting device 40 includes a rear lighting device 41 and a front lighting device 42, thereby improving the floating object detection performance compared to the case where only one side (rear or front) is illuminated. Illumination from the rear of the container 6 has the effect of making dark-colored foreign objects stand out against a white background, while illumination from the front has the effect of making white-colored foreign objects stand out, which is particularly effective when the background is black. Illumination from the front makes it easier to detect even transparent foreign objects because reflection occurs from the foreign objects. Depending on the location of the front lighting, specular reflection may occur on the container 6, making it difficult to see what is inside the container 6. However, in this embodiment, there are no particular restrictions on the position of the front lighting device 42, so it can be placed in a location where specular reflection on the container 6 is minimized.

[0083] In this embodiment, by configuring the imaging device 30 to capture images using four illumination patterns, "110," "010," "101," and "001," that is, by individually illuminating the rear illumination device 41 and the front illumination device 42 rather than simultaneously, it is possible to prevent black foreign objects from blending into the background when they are lit simultaneously, thereby improving the accuracy of detecting floating objects. Furthermore, in this embodiment, by using white light covering a wide wavelength range for the detection light 40L, it is possible to make it easier to detect foreign objects of various colors.

[0084] As described above, in this embodiment, unlike the technologies disclosed in Patent Documents 6 and 7, there are no particular restrictions on the position of the front-side illumination device 42, so it can be placed in a location where specular reflection on the container 6 is reduced. In this respect as well, compared to the conventional technology, it is possible to more reliably determine or identify whether floating objects are foreign objects or air bubbles in foreign object inspection.

[0085] Furthermore, in this embodiment, by dividing the acquired images into two groups, image group A and B, and configuring the judgment process to be performed sequentially, if a foreign object is determined to be present in image group A, the processing of image group B can be skipped, thereby shortening the processing time and thus the inspection time.

[0086] As described above, in this embodiment, it is possible to more reliably determine or identify whether floating objects are foreign objects or air bubbles during foreign object inspection. This eliminates or reduces the possibility of misidentifying air bubbles as foreign objects, thereby improving product yield.

[0087] The effects described above are the same in other embodiments and other examples unless otherwise specified.

[0088] In the first embodiment of the present invention, the range of color change index and movement speed (determination boundary) that the bubble can take, which the data processing unit 22 references in step S204, may be determined to be unique, or it may be determined for each lighting pattern. When determined for each lighting pattern, in step S204, the data processing unit 22 can determine that the floating object shown in the image selected in step S203 is a bubble if the calculated CCI and movement speed are within the range of the color change index and movement speed predetermined for image group A, and determine it to be a foreign object if it is not. In the case of processing image group B, the data processing unit 22 can determine that the floating object shown in the image selected in step S203 is a bubble if the calculated CCI and movement speed are within the range of the color change index and movement speed predetermined for image group B, and determine it to be a foreign object if it is not.

[0089] [Examples] The following describes the details of an experiment conducted to confirm the operation of the inspection system 1 or inspection method of the first embodiment.

[0090] In the experiment of the embodiment, a lighting device emitting white light was used as the lighting device 40 for detecting floating objects, a lighting device emitting red light was used as the lighting device 50 for identifying bubble foreign objects, and a 3-band (RGB) color camera was used as the imaging device 30. The image size of the color camera was 2592 pixels horizontally and 1944 pixels vertically, and its spatial resolution was approximately 16 μm. In the experiment of the embodiment, multiple containers 6 containing liquid 8 containing only foreign objects were prepared, as well as multiple containers 6 containing liquid 8 containing only bubbles. Figure 12 is a table showing the types and sizes of foreign objects contained in the liquid 8 sealed in the containers 6 in the experiment of the embodiment, the detection results for these foreign objects, and the identification results for bubbles. A total of eight types of foreign objects were tested, including metal and rubber. The size of each was approximately 100 μm to 1000 μm. The "Rear Illumination" column indicates whether floating objects could be detected when only the rear illumination device 41 was lit, and the "Front Illumination" column indicates whether floating objects could be detected when only the front illumination device 42 was lit. The "Overall" column indicates whether floating objects could be detected by any of the above illumination methods. For example, it is shown that metal was detected when only the rear illumination device 41 was lit, but not when only the front illumination device 42 was lit. Figure 12 shows that the inspection system 1 of the first embodiment was able to detect floating objects for all of the foreign objects used in the experiment of the embodiment.

[0091] Figure 13 shows a two-dimensional feature space and decision boundary, consisting of two features, color change index and movement speed, obtained from the experiment of the embodiment. In Figure 13, the set of color change index and movement speed calculated by the data processing unit 22 from each floating object region (pixel region corresponding to floating object) in the image obtained by imaging the prepared container 6 (liquid 8 inside the container 6) is reflected in each data section of the two-dimensional feature space. A positive movement speed means that the floating object is sinking, and a negative movement speed means that it is floating. Figure 13(a) shows the color change index and movement speed calculated from the floating object regions in the images obtained with the rear illumination device 41 turned on and the front illumination device 42 turned off for each of the prepared containers 6, that is, with illumination patterns "110" and "010", reflected in each data section of the two-dimensional feature space. Figure 13(b) shows the color change index and movement speed calculated from images obtained for each of the prepared containers 6 with the rear illumination device 41 turned off and the front illumination device 42 turned on, that is, from the floating object regions in the images obtained with illumination patterns "101" and "001", reflected in each data section in the two-dimensional feature space.

[0092] For example, in Figure 13(a), the data category for a color change index of "0.100-0.125" and a movement speed of "-2.0~-1.5 (mm / s)" is labeled "Bubble 1". This indicates that, among the containers 6 prepared for the experiment in the example, one container 6 contained liquid 8 containing only bubbles, and the color change index and movement speed calculated from one floating object region (bubble region) in the captured image fell within this data category (the values ​​were within this data category). Similarly, "Bubble 4" indicates that, among the containers 6 prepared for the experiment in the example, one container 6 contained liquid 8 containing only bubbles, and the color change index and movement speed calculated from four floating object regions (bubble regions) in the captured image fell within this data category. And "Foreign Matter 3" indicates that, among the containers 6 prepared for the experiment in the example, one container 6 contained liquid 8 containing foreign matter, and the color change index and movement speed calculated from three floating object regions (foreign matter regions) in the captured image fell within this data category. Furthermore, whether the floating objects were foreign objects was confirmed visually during imaging. The foreign objects in Figure 13 include all eight types shown in Figure 12, but the types of foreign objects are not described separately. Therefore, for example, "Foreign object 3" does not necessarily mean that there are three foreign objects of the same type.

[0093] Although the lighting patterns differ in Figures 13(a) and 13(b), in both cases, bubbles are clustered within a certain range in the two-dimensional feature space, while foreign objects are widely distributed, confirming that the two are easily distinguishable in this feature space. In fact, as shown in the figure, it can be confirmed that bubbles and foreign objects can be distinguished by a simple rectangular decision boundary. From Figure 13, it can be confirmed that the color change index for determining a bubble is preferably set within a range of lower and upper limits, and is even more preferable when combined with the movement speed. Furthermore, Figures 13(a) and (b) show that the optimal range of color change index and movement speed that a predetermined bubble can take (optimal decision boundary) may differ depending on the lighting pattern. Therefore, the optimal decision boundary may be determined by considering the lighting pattern.

[0094] [Second Embodiment] Next, we will describe the inspection system 1 of the second embodiment of the present invention. The differences from the first embodiment will be described below.

[0095] In the second embodiment, the data processing unit 22 acquires only two images in step S102. Figure 14 shows the illumination patterns used in the second embodiment. In step S102, the device control unit 21 causes the imaging device 30 to image the liquid 8 in the container 6 and acquire images using two illumination patterns, "111" and "011". In the second embodiment, the rear illumination device 41 and the front illumination device 42 are turned on simultaneously, reducing the number of images taken compared to the first embodiment. In the second embodiment, there are no image groups, so the flowchart of the data processing unit 22 does not include steps S202 and S206.

[0096] Figure 15(a) shows an example of the spectral distribution of light emitted by the floating object detection illumination device 40 and the bubble foreign object identification illumination device 50 according to the second embodiment of the present invention. Figure 15(b) shows an example of the spectral sensitivity of the imaging device 30 according to the second embodiment of the present invention. The detection light 40L (rear), detection light 40L (front), and identification light 50L emitted by the rear illumination device 41, the front illumination device 42, and the bubble foreign object identification illumination device 50, respectively, are light having different wavelength bands, for example, as shown in Figure 15(a), they are blue light, green light, and red light. The imaging device 30 is sensitive to the wavelength bands corresponding to the respective wavelength bands of the detection light 40L (rear), detection light 40L (front), and identification light 50L, and is sensitive to the wavelength bands corresponding to blue light, green light, and red light, for example, as shown in Figure 12(b).

[0097] By configuring the rear-side illumination device 41, the front-side illumination device 42, and the bubble foreign object identification illumination device 50 to emit light with different spectral distributions, it becomes possible to distinguish these lights on the image. Although the inspection system 1 of the first embodiment, which does not light up simultaneously, has higher performance in detecting floating objects, the second embodiment can have the same effects as the first embodiment. Furthermore, in the second embodiment, the number of imaging cycles can be reduced compared to the inspection system 1 of the first embodiment, shortening the processing time and thus the inspection time.

[0098] In the second embodiment, the imaging device 30 can be any multiband camera (multispectral camera, hyperspectral camera) having N (N≧3) bands (wavelength bands). By using a camera with three or more bands, it becomes possible to distinguish and detect the light emitted by the rear illumination device 41, the front illumination device 42, and the bubble foreign object identification illumination device 50, and the same effects as in the first embodiment can be achieved.

[0099] [Third Embodiment] Next, we will describe the inspection system 1 of the third embodiment of the present invention. The differences from the first embodiment will be described below.

[0100] In the third embodiment, the data processing unit 22 uses a color index instead of a color change index, does not calculate the color change index, and does not calculate the movement speed. In the third embodiment, the identification of bubbles and foreign objects using the color index involves analyzing the light intensity of the pixel area corresponding to the floating object in the image, and identifying it as a bubble if a certain percentage or more of that light intensity is emitted by the bubble and foreign object identification illumination device 50, and identifying it as a foreign object otherwise.

[0101] The data processing unit 22 acquires only two images in step S102. In step S102, the device control unit 21 instructs the imaging device 30 to image the liquid 8 in the container 6 and acquire images under two illumination patterns, "110" and "101". In the third embodiment, the number of images taken is reduced compared to the first embodiment by using a color index instead of a color change index. In the third embodiment, there are no image groups, so the flowchart of the data processing unit 22 does not include steps S202 and S206.

[0102] In step S204, the data processing unit 22 calculates the color index (CI) of the pixel area corresponding to the floating object and determines whether the pixel area corresponding to the floating object selected in step S203 corresponds to a bubble or a foreign object. The range of CIs that a bubble can take is predetermined, and the data processing unit 22 determines that the floating object is a bubble if the calculated CI is within the predetermined range of CIs, and determines that it is a foreign object otherwise.

[0103] In special cases, such as when the foreign object itself is red, the color change index can more clearly distinguish between bubbles and foreign objects. However, in other cases, the third embodiment can have the same effects as the first embodiment. Furthermore, in the third embodiment, compared to the inspection system 1 of the first embodiment, the number of imaging cycles can be reduced, shortening the processing time and thus the inspection time.

[0104] In the third embodiment, the data processing unit 22 may acquire only one image in step S102. In this case, the device control unit 21 may instruct the imaging device 30 to image the liquid 8 in the container 6 and acquire an image in one of the illumination patterns "111" in step S102. In this case, although the performance in detecting suspended matter and identifying air bubbles and foreign objects is reduced, the number of imaging cycles can be significantly reduced compared to the inspection system 1 of the first embodiment, thus shortening the processing time and the inspection time.

[0105] [Fourth Embodiment] Next, we will describe the inspection system 1 of the fourth embodiment of the present invention. The differences from the first embodiment will be described below.

[0106] In the fourth embodiment, the data processing unit 22 can distinguish whether the floating objects in the image are foreign objects or bubbles based on the light intensity of the region corresponding to the floating objects in the image acquired by the imaging device 30 using only the detection light 40L and the image acquired by the imaging device 30 using only the identification light 50L. In the fourth embodiment, the light emitted by the floating object detection illumination device 40 and the bubble foreign object identification illumination device 50 can be any spectral distribution light, as long as it is within the wavelength band to which the imaging device 30 is sensitive.

[0107] Figure 16 shows the illumination patterns used in the fourth embodiment. The imaging device 30 captures images for each of the illumination patterns shown in Figure 16. In step S102, the device control unit 21 instructs the imaging device 30 to capture images of the liquid 8 in the container 6 for three illumination patterns: "010", "100", and "001". In step S102, the device control unit 21 instructs the imaging device 30 to take two images for illumination pattern "010" with a predetermined time difference, take one image for illumination pattern "100", and take two images for illumination pattern "001" with a predetermined time difference. In the fourth embodiment, there are no image groups, so the flowchart of the data processing unit 22 does not include steps S202 and S206.

[0108] In the fourth embodiment, the data processing unit 22 does not calculate a color index or a color change index, but instead calculates a brightness index LI, which is an index indicating the relative intensity of the identification light 50L of the pixel region corresponding to the floating object in the image acquired by the imaging device 30. The brightness index LI is calculated using the following equation 3. LI=I(100) / (I(010)+I(001)) (Formula 3) Here, I(100), I(010), and I(001) are the average pixel values ​​in the pixel regions corresponding to floating objects in the images acquired by the imaging device 30 for three illumination patterns: "010", "100", and "001", respectively. The pixel regions corresponding to bubbles generally have a large brightness index LI.

[0109] In the fourth embodiment, the data processing unit 22 uses a brightness index instead of a color change index. For example, in step S204, the data processing unit 22 calculates the brightness index (LI) and movement speed of the pixel area corresponding to the floating object, and determines whether the pixel area corresponding to the floating object selected in step S203 corresponds to a bubble or a foreign object. For example, in step S204, the data processing unit 22 determines that the floating object is a bubble if the calculated LI and movement speed are within a predetermined range of LI and movement speed, and determines that it is a foreign object otherwise.

[0110] In the fourth embodiment, the flowchart shown in Figure 11 can be said to include acquiring a first image of the liquid 8 in the container 6 by the imaging device 30 with illumination pattern "010", acquiring a second image of the liquid 8 in the container 6 by the imaging device 30 with illumination pattern "100", and determining whether the floating objects are foreign objects or bubbles based on the change in light intensity of the regions corresponding to the floating objects in the first and second images.

[0111] Next, the operation and effects of the inspection system 1 according to the fourth embodiment of the present invention will be described.

[0112] As described above, in this embodiment, the imaging device 30 is configured to be positioned to receive the identification light 50L totally reflected by the bubbles, and the detection light 40L and the identification light 50L are illuminated one at a time rather than simultaneously to acquire an image of the liquid 8. This makes it possible to detect the identification light 50L totally reflected from the bubbles with a stronger light intensity while temporally separating the two lights, and to separate the detection light 40L and the identification light 50L with a certain level of accuracy. This configuration is unprecedented and makes it possible to distinguish between foreign objects and bubbles, whether they are small floating objects of about 100 μm to 300 μm or larger. For example, since the inspection system 1 of this embodiment does not use a method that relies on edges or shapes as clues, even if the captured image cannot be focused accurately across the entire area, it is possible to detect small foreign objects of about 100 μm to 300 μm in a container of about several centimeters in size without mistaking them for bubbles.

[0113] Furthermore, in this embodiment, although the inspection time increases due to the increased number of imaging cycles compared to the first embodiment, unlike the first embodiment, there are no constraints regarding the spectral distribution. Therefore, a single-band camera (monochrome camera) can be used as the imaging device 30, which reduces costs and expands the range of camera options. In addition, since an image is obtained with only the identification light 50L illuminated, regions with strong identification light 50L in the image can be extracted without being affected by the detection light 40L. This allows for more reliable determination or identification of whether floating objects are foreign objects or air bubbles during foreign object inspection. It should be noted that in parts with a similar configuration, the fourth embodiment can also have the same effects as the first to fourth embodiments.

[0114] The effects described above are the same in other embodiments and other examples unless otherwise specified.

[0115] In the fourth embodiment, the data processing unit 22 can determine in step S204 whether the pixel area corresponding to the floating object selected in step S203 corresponds to a bubble or a foreign object, solely by brightness index determination, without calculating the movement speed. Although the identification performance of bubbles and foreign objects is higher when movement speed is used, the effects of the fourth embodiment can still be achieved in this case, and the number of imaging cycles can be reduced, shortening the processing time and thus the inspection time.

[0116] [Various embodiments] In the following description, we will explain that, in embodiments of the present invention, for example, the first to fourth embodiments can be combined as appropriate to form various embodiments without departing from the spirit of the present invention.

[0117] Embodiments of the present invention may also include a method or program for implementing the functions of the inspection system 1 described above or the information processing shown in the flowchart, or a computer-readable storage medium storing said program. Furthermore, embodiments of the present invention may also include a server capable of supplying a computer with a program for implementing the functions of the embodiments of the present invention described above or the information processing shown in the flowchart.

[0118] In embodiments of the present invention, the data processing unit 22 does not need to perform step S201. In this case, the inspection system 1 does not perform suspended matter detection and can identify whether the suspended matter contained in the liquid 8 in the container 6 is foreign matter or air bubbles. For example, the inspection system 1 can be implemented as a re-inspection system (re-inspection device) and can identify whether the suspended matter contained in the liquid 8 in the container 6 is foreign matter or air bubbles, based on the fact that an inspection has already been performed and there is a high probability that suspended matter is present.

[0119] In embodiments of the present invention, the data processing unit 22 may, in step S201, determine whether or not the liquid 8 in the container 6 contains suspended matter using either the image captured by the imaging device 30 in step S102 with illumination patterns "010" and "001". In embodiments of the present invention, the data processing unit 22 may be configured to execute step S201 after step S202.

[0120] In embodiments of the present invention, the front-side illumination device 42 does not necessarily include the second front-side illumination device 44. For example, if the front-side illumination device 42 does not include the second front-side illumination device 44, the illumination of the front-side illumination device 42 in the illumination pattern means the illumination of the first front-side illumination device 43. In embodiments of the present invention, the bubble foreign object identification illumination device 50 does not necessarily include the second identification illumination device 52. For example, if the bubble foreign object identification illumination device 50 does not include the second identification illumination device 52, the illumination of the bubble foreign object identification illumination device 50 in the illumination pattern means the illumination of the first identification illumination device 51.

[0121] In embodiments of the present invention, the rear illumination device 41, the front illumination device 42, and the bubble and foreign matter identification illumination device 50 may include any number of illumination devices to cover the entire container 6.

[0122] In embodiments of the present invention, the inspection system 1 of the embodiment of the present invention can be realized using an existing foreign object inspection device. For example, in the first to third embodiments, an existing illumination device and imaging device can be used, and an optical filter such as a color filter that allows only light in a predetermined wavelength band to pass through can be attached to these devices as needed to acquire an image, calculate the color index or color change index of the pixel area corresponding to the floating object in the image, and be configured to identify whether the floating object is a foreign object or a bubble. For example, in the fourth embodiment, an existing illumination device and imaging device can be used to acquire an image, calculate the brightness index of the pixel area corresponding to the floating object in the image, and be configured to identify whether the floating object is a foreign object or a bubble.

[0123] In an embodiment of the present invention, the data processing unit 22 can be configured to save an image to the storage device 14 when it identifies a floating object in the image as a foreign object. This configuration makes it possible to obtain actual image data in which a foreign object has been detected. Generally, cases in which foreign objects are detected in actual product inspections are rare, so such image data is valuable.

[0124] In embodiments of the present invention, the data processing unit 22 may be configured to train a known learning model (e.g., a CNN) with images in which floating objects in the image are bubbles and images in which floating objects in the image are foreign objects, for each lighting pattern, in order to generate a trained model. In this case, the flowchart does not need to include steps S202, S203, S205, and S206, and in step S204, the data processing unit 22 may be configured to determine whether floating objects in an image captured with a single lighting pattern are foreign objects or bubbles, using a trained model corresponding to that lighting pattern, without calculating the color index, color change index, or movement speed.

[0125] In the first and second embodiments, the data processing unit 22 can determine in step S204 whether the pixel region corresponding to the floating object selected in step S203 corresponds to a bubble or a foreign object, solely by determining the color change index without calculating the movement speed. In this case, for example, if a boundary as shown in Figure 9 is set in advance, the data processing unit 22 can determine that the pixel region corresponds to a bubble if the color change index is within the range of the boundary between the lower and upper limits, and determine that it corresponds to a foreign object otherwise.

[0126] In the first and second embodiments, if the foreign matter that the liquid 8 in the container 6 may contain is limited to black or white, the floating object detection illumination device 40 can consist of only one of the rear illumination device 41 and the front illumination device 42. This reduces the number of imaging cycles, shortens processing time, and shortens inspection time.

[0127] In the first to third embodiments, the identification light 50L emitted by the bubble foreign object identification illumination device 50 can be blue or green light instead of red light. For example, in this case, the color index CI shown in Equation 1 will have R replaced by B in the molecule for blue light, and R replaced by G in the molecule for green light.

[0128] In the first to third embodiments, the color index CI can be any formula other than Equation 1, as long as it can indicate to what extent the pixel area corresponding to the floating object contains light intensity corresponding to the wavelength band of light emitted by the bubble foreign object identification illumination device 50. For example, if the imaging device 30 is a two-band camera with sensitivity to G and R only, the color index CI can be defined as the denominator of Equation 1 with B removed, and the data processing unit 22 can calculate the CI. For example, in the third embodiment, in step S204, the data processing unit 22 can identify whether the floating object in the liquid 8 is a foreign object or a bubble based on the pixel value (light intensity) in a predetermined band including the wavelength band of the identification light 50L in the area corresponding to the floating object in the image acquired by the imaging device 30, and the pixel values ​​in multiple bands of the imaging device 30.

[0129] In the first to third embodiments, the color change index CCI can be any formula other than Equation 2, as long as it can show the degree of change in the color index in the image when the bubble foreign object identification illumination device 50 is turned on and when it is turned off.

[0130] In the first and third embodiments, the imaging device 30 can be any multiband camera (multispectral camera, hyperspectral camera). For example, if the imaging device 30 is a camera with N (N≧2) bands (wavelength bands), the floating object detection illumination device 40 and the bubble foreign object identification illumination device 50 can be configured such that the wavelength band of the detection light 40L overlaps with part or all of one or more bands (N1), and the wavelength band of the identification light 50L overlaps with part or all of one or more bands (N2) different from N1, so that the inspection system 1 can obtain the effects of this embodiment. For example, if the imaging device 30 is a camera with five wavelength bands for blue light, green light, yellow light, orange light, and red light, the floating object detection illumination device 40 and the bubble foreign object identification illumination device 50 can be configured such that the wavelength band of the identification light 50L overlaps with part or all of one of the yellow light bands so that the identification light 50L is sufficiently detected only in that band, and the wavelength band of the detection light 40L overlaps with part or all of at least one band other than yellow light so that the detection light 40L is sufficiently detected in that band.

[0131] In the first to third embodiments, the imaging device 30 can be any multiband camera having sensitivity in bands substantially matching the wavelength band of the detection light 40L and sensitivity in bands substantially matching the wavelength band of the identification light 50L.

[0132] In the first to third embodiments, the imaging device 30 can be any of several cameras equipped with color filters having spectral transmittances that allow the imaging device 30 to have sensitivity in the same wavelength band as a multiband camera. This configuration makes it possible to finely adjust the sensitivity of the camera using the color filters.

[0133] In the first to third embodiments, the floating object detection illumination device 40 and the bubble foreign object identification illumination device 50 can be configured to emit detection light 40L and identification light 50L, which include wavelengths in the near-infrared region centered on visible light. In this case, the imaging device 30 can be a camera with sensitivity corresponding to those wavelengths.

[0134] In the first and third embodiments, the floating object detection illumination device 40 and the bubble foreign object identification illumination device 50 can be configured such that the detection light 40L is white light, and the identification light 50L is light in a specific wavelength band such as blue light, green light, or red light. With this configuration, it is possible to detect a wide range of floating objects while separating and identifying the detection light 40L and the identification light 50L.

[0135] In the first and third embodiments, the floating object detection lighting device 40 and the bubble foreign object identification lighting device 50 can be configured such that the wavelength bands of the detection light 40L and the identification light 50L do not overlap, for example, when the type of floating object can be identified to some extent. With this configuration, it becomes possible to distinguish between the detection light 40L and the identification light 50L more clearly.

[0136] In the processes or operations described above, the processes or operations can be freely modified, as long as no inconsistencies arise in the processes or operations, such as using data that should not yet be available at a given step. Furthermore, the embodiments described above are illustrative examples for explaining the present invention, and the present invention is not limited to these embodiments. The present invention can be implemented in various forms without departing from its essence. [Explanation of Symbols]

[0137] 1: Inspection system, 4: Holding unit, 6: Container, 8: Liquid, 10: Control device, 11: Processor, 12: Display device, 13: Input device, 14: Storage device, 15: Communication device, 16: Bus, 21: Device control unit, 22: Data processing unit, 30: Imaging device, 40: Illumination device for detecting floating objects, 41: Rear illumination device, 42: Front illumination device, 43: First front illumination device, 44: Second front illumination device, 50: Illumination device for identifying bubbles and foreign objects, 51: First identification illumination device, 52: Second identification illumination device, 61: Bubble, 62: Foreign object, 71: Angle, Path: 72, 160: Image, 161: Pixel area, 162: Pixel area

Claims

1. A system for determining the presence or absence of foreign matter in a liquid sealed inside a container, A floating object detection lighting device that emits light to detect floating objects, A bubble foreign object identification illumination device that emits light to identify bubbles among suspended particles, An imaging device capable of imaging the liquid inside a container using light emitted from either or both of the floating object detection illumination device and the bubble foreign object identification illumination device, wherein the imaging device is positioned to receive light emitted from the bubble foreign object identification illumination device that has been totally reflected by bubbles, A control device that identifies whether a floating object in a liquid is a foreign object or a bubble based on the light intensity in a predetermined band including at least a portion of the wavelength band of the light emitted by the bubble foreign object identification lighting device in an image acquired by the imaging device using light emitted from the floating object detection lighting device and the bubble foreign object identification lighting device, A system that includes these features.

2. The light spectra emitted by the aforementioned floating object detection lighting device and the aforementioned bubble foreign object identification lighting device are different. The imaging device is a multiband camera capable of acquiring light intensity in multiple wavelength bands including the predetermined band, The control device identifies whether the floating object in the liquid is a foreign object or a bubble based on the light intensity in a predetermined band and the light intensity in a plurality of wavelength bands in the region corresponding to the floating object in the image acquired by the imaging device. The system according to claim 1.

3. The system according to claim 2, wherein the control device identifies whether the floating object in the liquid is a foreign object or a bubble based on the change in light intensity in a predetermined band in the region corresponding to the floating object in the first image acquired by the imaging device using light emitted from the bubble foreign object identification illumination device and the floating object detection illumination device, and the second image acquired by the imaging device using light emitted only from the floating object detection illumination device.

4. The control device calculates the movement speed of the floating object based on a first image acquired by the imaging device using light emitted from the bubble foreign object identification illumination device and the floating object detection illumination device, and a second image acquired by the imaging device at a predetermined time difference from the first image using light emitted only from the floating object detection illumination device, and further identifies whether the floating object in the liquid is a foreign object or a bubble based on the calculated movement speed, according to claim 3.

5. The floating object detection illumination device includes a first floating object detection illumination device arranged to penetrate the liquid and reach the imaging device, and a second floating object detection illumination device arranged so that light reflected by the floating object reaches the imaging device. The control device identifies whether a floating object in a liquid is a foreign object or a bubble based on either or both of the following: a change in light intensity in a predetermined band in the region corresponding to a floating object in a first image acquired by the imaging device using light emitted from the bubble foreign object identification illumination device and the first floating object detection illumination device, and a second image acquired by the imaging device using light emitted only from the first floating object detection illumination device; and a change in light intensity in a predetermined band in the region corresponding to a floating object in a third image acquired by the imaging device using light emitted from the bubble foreign object identification illumination device and the second floating object detection illumination device, and a fourth image acquired by the imaging device using light emitted only from the second floating object detection illumination device. The system according to claim 3 or 4.

6. The control device is Based on the change in light intensity in the predetermined band of the region corresponding to the floating object in the first image acquired by the imaging device using light emitted from the bubble foreign object identification illumination device and the first floating object detection illumination device, and the change in light intensity in the predetermined band of the region corresponding to the floating object in the second image acquired by the imaging device using light emitted only from the first floating object detection illumination device, the system identifies whether the floating object in the liquid is a foreign object or a bubble. If the floating object is identified as a bubble, the system further identifies whether the floating object in the liquid is a foreign object or a bubble based on the change in light intensity in the predetermined band of the region corresponding to the floating object in the third image acquired by the imaging device using light emitted from the bubble foreign object identification illumination device and the second floating object detection illumination device, and the change in light intensity in the predetermined band of the region corresponding to the floating object in the fourth image acquired by the imaging device using light emitted only from the second floating object detection illumination device. The system according to claim 5.

7. The floating object detection illumination device includes a first floating object detection illumination device arranged so that emitted light passes through the liquid and reaches the imaging device, and a second floating object detection illumination device arranged so that light reflected from the floating object reaches the imaging device. The wavelength bands of the light emitted by the first floating object detection illumination device, the second floating object detection illumination device, and the bubble foreign object identification illumination device are different from each other, and the imaging device is a multiband camera that includes bands in a frequency band that includes at least a portion of the respective light frequency bands. The system according to any one of claims 1 to 4.

8. The system according to any one of claims 1 to 4, wherein the light emitted by the floating object detection illumination device is white light including blue, green, and red light, the light emitted by the bubble foreign object identification illumination device is blue, green, or red light, and the imaging device is a multiband camera including a band in a frequency range that includes at least a portion of the frequency range of the light emitted by the bubble foreign object identification illumination device.

9. A system for determining the presence or absence of foreign matter in a liquid sealed inside a container, A floating object detection lighting device that emits light to detect floating objects, A bubble foreign object identification illumination device that emits light to identify bubbles among suspended particles, An imaging device capable of imaging the liquid inside a container using light emitted from either or both of the floating object detection illumination device and the bubble foreign object identification illumination device, wherein the imaging device is positioned to receive light emitted from the bubble foreign object identification illumination device that has been totally reflected by bubbles, A control device that identifies whether a floating object in a liquid is a foreign object or a bubble based on the light intensity of the region corresponding to the floating object in the image acquired by the imaging device using light emitted only from the floating object detection illumination device and the image acquired by the imaging device using light emitted only from the bubble foreign object identification illumination device. A system that includes these features.

10. The floating object detection illumination device includes a first floating object detection illumination device arranged to penetrate the liquid and reach the imaging device, and a second floating object detection illumination device arranged so that light reflected from the floating object reaches the imaging device. The control device calculates the movement speed of a floating object based on one or both of two images acquired by the imaging device at a predetermined time difference using light emitted from the first floating object detection illumination device, and two images acquired by the imaging device at a predetermined time difference using light emitted from the second floating object detection illumination device, and further identifies whether the floating object in the liquid is a foreign object or a bubble based on the calculated movement speed, according to claim 9.

11. A method for determining the presence or absence of foreign matter in a liquid sealed inside a container, This includes using light for detecting suspended matter and light for identifying bubbles and foreign objects in a liquid to identify whether the suspended matter in the liquid is a foreign object or a bubble, Identifying whether floating objects in a liquid are foreign objects or bubbles involves an imaging device positioned to receive bubble foreign object identification light totally reflected by bubbles in the liquid within the container, and identifying them based on the light intensity in a predetermined band including at least a portion of the wavelength band of the bubble foreign object identification light in the region corresponding to the floating objects in the image acquired using the floating object detection light and the bubble foreign object identification light. method.

12. The spectra of the light used for detecting floating objects and the light used for identifying foreign objects in bubbles are different. The imaging device is a multiband camera capable of acquiring light intensity in multiple wavelength bands including the predetermined band, Identifying whether or not suspended matter in a liquid is a bubble includes identifying it based on the light intensity in a predetermined band and the light intensity in a plurality of wavelength bands in a region corresponding to the suspended matter in an image acquired using the imaging device. The method according to claim 11.

13. Identifying whether suspended particles in a liquid are bubbles or not is possible. The first image is acquired using light for detecting floating objects and light for identifying foreign objects in bubbles, To acquire a second image using only the light for detecting floating objects, Identifying whether the suspended matter in the liquid is foreign matter or air bubbles based on the change in light intensity in the predetermined band of the region corresponding to the suspended matter in the first and second images, The method according to claim 12, including the method described in claim 12.

14. Identifying whether suspended particles in a liquid are bubbles or not is possible. The first image is acquired using light for detecting floating objects and light for identifying foreign objects in bubbles, A second image is acquired using only the light for detecting floating objects, either a predetermined time before or after the acquisition of the first image. The movement speed of the floating object is calculated based on the first and second images, Based on the calculated movement speed, it is possible to further identify whether the suspended matter in the liquid is foreign matter or air bubbles, The method according to claim 13, including the method described in claim 13.

15. Identifying whether suspended particles in a liquid are bubbles or not is possible. The light for detecting floating objects includes a first light for detecting floating objects that is emitted so as to penetrate the liquid and reach the imaging device, and a second light for detecting floating objects that is emitted so as to reflect light from the floating objects and reach the imaging device. The wavelength bands of the first floating object detection light, the second floating object detection light, and the bubble foreign object identification light are different from each other, and the imaging device is a multiband camera that includes a band in a frequency band that includes at least a portion of the respective light frequency bands. The method according to claim 13, including the method described in claim 13.

16. A method for determining the presence or absence of foreign matter in a liquid sealed inside a container, Identifying whether suspended particles in a liquid are bubbles or not is possible. The first image is acquired using only light for detecting floating objects, The second image is acquired using only the light for identifying air bubbles and foreign objects. Based on the light intensity of the regions corresponding to the suspended objects in the first and second images, it is possible to identify whether the suspended objects in the liquid are foreign objects or bubbles, Methods that include...

Images