Inspection device

Abstract

This inspection device includes: an acquisition means for acquiring a plurality of images obtained by continuously photographing a container filled with liquid; and a processing means for processing the plurality of acquired images. The processing means sets, on the basis of the position of the liquid surface of the liquid in the container, at least one inspection region for each of a region at the same level as or above the liquid surface of the plurality of images and a region below the liquid surface, sets the type of a binarization method for each of the plurality of set inspection regions, and detects an image of a foreign object candidate from each of the plurality of inspection regions on the basis of the set type of binarization method.

Description

Inspection Equipment

[0001] The present disclosure relates to an inspection apparatus, an inspection method, and a recording medium.

[0002] Patent Document 1 describes an example of a technology for inspecting a container filled with liquid for foreign matter. The technology described in Patent Document 1 (hereinafter referred to as the first related technology) includes a means for photographing a container filled with liquid and a means for processing the photographed image of the container. The image processing means sets multiple inspection areas in the image of the container based on the location of the container and the time of day the container was photographed, and sets a filter strength for each inspection area to remove bubbles that cause noise. The image processing means then uses a filter to remove noise from each inspection area, and then uses a predetermined binarization method to create a binary image required for foreign matter detection.

[0003] JP 2022-122072 A JP 2016-66111 A

[0004] As shown in the first related art, when multiple inspection areas are set in an image of a container based on the location of the container and the time of day the container was photographed, the liquid level is not taken into consideration, so there is a possibility that the area below the liquid level and the area above the liquid level in the container will be included in the same inspection area. However, because liquid is present in the area below the liquid level in the container and not in the area above the liquid level, the images of foreign objects appear differently in the two areas. Therefore, it is difficult to detect images of foreign objects present in the area below the liquid level in the container and the area above the liquid level using the same processing method.

[0005] An object of the present disclosure is to provide an inspection device that solves the above-mentioned problems.

[0006] The inspection device according to the present disclosure comprises an acquisition means for acquiring a plurality of images obtained by successively photographing a container filled with liquid, and a processing means for processing the acquired plurality of images, wherein the processing means is configured to set at least one inspection area in each of an area above the liquid level and an area below the liquid level in the plurality of images based on the position of the liquid level in the container, set a type of binarization method for each of the set plurality of inspection areas, and detect images of potential foreign objects from each of the set plurality of inspection areas based on the set type of binarization method.

[0007] Furthermore, the inspection method according to the present disclosure is configured so that a computer acquires a plurality of images obtained by successively photographing a container filled with a liquid, processes the acquired plurality of images, and in processing the plurality of images, sets at least one inspection area in each of an area above the liquid level and an area below the liquid level in the plurality of images based on the position of the liquid level in the container, sets a type of binarization method for each of the set plurality of inspection areas, and detects images of potential foreign objects from each of the set plurality of inspection areas based on the set type of binarization method.

[0008] Furthermore, a computer-readable recording medium according to the present disclosure is configured to record a program that causes a computer to perform the following processes: acquiring a plurality of images obtained by successively photographing a container filled with a liquid; and processing the acquired plurality of images; and in processing the plurality of images, setting at least one inspection area in each of an area above the liquid level and an area below the liquid level in the plurality of images based on the position of the liquid level in the container, setting a type of binarization method for each of the set plurality of inspection areas, and detecting images of foreign matter candidates from each of the plurality of inspection areas based on the set type of binarization method.

[0009] By having the configuration described above, the present disclosure can easily extract images of foreign object candidates present in the area below the liquid surface of the liquid filled in the container and images of foreign object candidates present in the area above the liquid surface.

[0010] 1 is a block diagram illustrating an example of an inspection device according to the present disclosure. FIG. 2 is a block diagram illustrating an example of an information processing device constituting the inspection device according to the present disclosure. FIG. 3 is a diagram illustrating an example of image information acquired by the inspection device according to the present disclosure. FIG. 4 is a diagram illustrating an example of tracking information generated by the inspection device according to the present disclosure. FIG. 5 is a diagram illustrating an example of inspection result information generated by the inspection device according to the present disclosure. FIG. 6 is a flowchart illustrating an example of an inspection method performed by the inspection device according to the present disclosure. FIG. 7 is a diagram illustrating an example of a rotation angle timeline of the inspection device according to the present disclosure. FIG. 8 is a schematic diagram illustrating a state in which a vial in an upright position is photographed by a camera device in the inspection device according to the present disclosure. FIG. 9 is a schematic diagram illustrating a state in which a vial in a sideways position is photographed by a camera device in the inspection device according to the present disclosure. FIG. 10 is a schematic diagram illustrating a state in which the positions of foreign matter, air bubbles, and scratches on the bottle body in a vial that is rotating sideways in the inspection device according to the present disclosure, viewed from a direction parallel to the rotation axis. FIG. 11 is a schematic diagram illustrating a state in which the positions of foreign matter, air bubbles, and scratches on the bottle body in a vial that is rotating sideways in the inspection device according to the present disclosure, viewed from a direction perpendicular to the rotation axis. FIG. 12 is a flowchart illustrating an example of processing performed by a detection unit of the inspection device according to the present disclosure. 1 is a diagram illustrating an example of an inspection area set on an image of a container in an inspection device according to the present disclosure;

[0011] Next, embodiments of the present disclosure will be described in detail with reference to the drawings. Each embodiment of the present disclosure can be utilized in the medical healthcare field. Specifically, it can be used to inspect pharmaceutical liquids for foreign substances. In the following description, when multiple identical or similar elements exist, a common reference numeral may be used to describe each element without distinguishing between them. A subnumber may be added to the common reference numeral to describe each element distinctly. [First Embodiment] Referring to FIG. 1 , an inspection device 100 according to a first embodiment of the present disclosure is an apparatus for inspecting filled vials 110. The inspection device 100 includes, as main components, a gripping and rotating device 200, an illumination device 300, a camera device 400, an information processing device 500, and a display device 600.

[0012] A filled vial (hereinafter simply referred to as a vial) 110 is, for example, a bottle that has been filled with a drug solution for storage under sterile conditions, has its opening sealed with a rubber stopper, and is then covered with an aluminum cap. In the final process before product shipment, a plastic cap is fitted to cover the aluminum cap. The rubber stopper and the aluminum and plastic caps form the lid of the vial 110. In an upright position, the vial 110 is composed of, from top to bottom, a seam, a bottle head (container head), a truncated conical bottle shoulder (container shoulder), a cylindrical bottle body (container body), and a bottle bottom that closes the bottle body.

[0013] The amount of liquid filled into the vial 110 in this example is approximately half the vial's capacity. That is, the liquid level R of the vial 110 in this example is approximately at the center of the vial body. However, the vials to which the present disclosure is applicable are not limited to the above. Furthermore, the amount of liquid filled into the vial 110 does not need to be approximately half the vial's capacity; it may be more than half or less than half. The vial 110 may have various defects. For example, foreign matter may be present in the vial 110. Examples of foreign matter include glass fragments, metal fragments, rubber fragments, hair, fiber fragments, soot, etc. The vial 110 may also have cracks, scratches, dirt, poor seaming, insufficient medicine, etc. The inspection device 100 may be a device that inspects the vial 110 for various defects that may occur.

[0014] The gripping and rotating device 200 is a device that can rotate a vial 110 while gripping it. The gripping and rotating device 200 has two mutually perpendicular rotation axes (rotation axis A and rotation axis B), and is capable of rotating the gripped vial 110 about rotation axis A and also about rotation axis B. The gripping and rotating device 200 includes a flat plate-like member 201, an upper arm 202 connected to the upper end of the flat plate-like member 201, and a lower arm 203 connected to the lower end of the flat plate-like member 201. A lower gripping part 204 is connected to the end of the lower arm 203 opposite to the end connected to the flat plate-like member 201.

[0015] The lower gripping portion 204 functions as a base for placing the vial 110. A rotatable plate 205 is attached to the upper surface of the lower gripping portion 204.

[0016] A chuck mechanism 206 having chuck fingers 211 for chucking the vial 110 is provided at the end of the upper arm 202 opposite to the end to which the flat plate-like member 201 is connected. For example, the chuck mechanism 206 may be configured as, but is not limited to, a parallel open / close air chuck. The chuck fingers 211 are rotatable about a rotation axis A and are movable up and down along the rotation axis A. The chuck mechanism 206 closes, opens, rotates, and moves up and down the chuck fingers 211 in accordance with commands sent from the information processing device 500. When the vial 110 is placed on the plate 205 in an upright position and the chuck fingers 211 are lowered to chuck the top of the vial 110, the vial 110 is gripped by the gripping / rotating device 200 so that its central axis (the axis passing through the center of the top and bottom; also referred to as the upright central axis) coincides with the rotation axis A. When the chuck fingers 211 are rotated in this state, the vial 110 rotates around the rotation axis A. A rotation angle detector 209 such as an encoder provided on the upper arm 202 is configured to detect the rotation angle of the chuck fingers 211, and therefore the rotation angle of the vial 110 chucked by the chuck fingers 211 about the rotation axis A, and output it to the information processing device 500.

[0017] The lighting device 300 is a surface light source that illuminates the bottle body of the vial 110 chucked by the chuck mechanism 206 of the gripping and rotating device 200 from a direction perpendicular to the rotation axis A, and is attached to the flat plate-like member 201. The lighting device 300 is installed on the opposite side of the vial 110 from the camera device 400.

[0018] The plate-like member 201 is supported by a rotating shaft 208 that is rotated by a motor 207. The motor 207 is fixed by a support member (not shown). When the rotating shaft 208 is rotated by the motor 207, the plate-like member 201 rotates. In response to this, all elements directly or indirectly connected or attached to the plate-like member 201, i.e., the upper arm 202, the lower arm 203, the lower gripping portion 204, the chuck fingers 211, the chuck mechanism 206, the plate 205, and the illumination device 300, rotate. Therefore, the vial 110 placed on the plate 205 and chucked by the chuck mechanism 206 also rotates about the rotation axis B. The dimensions and mounting positions of each part of the gripping / rotating device 200 are specified so that the vial 110 rotates about an axis perpendicular to the rotation axis A and passing through the center of the vial 110. A rotation angle detector 210 such as an encoder provided on the flat plate-like member 201 is configured to detect the rotation angle of the rotating shaft 208, and therefore the rotation angle of the vial 110 chucked by the chuck finger 211 around the rotation axis B, and output it to the information processing device 500.

[0019] The camera device 400 is a high-speed camera having a wide-angle lens that continuously captures images of the vial body of the vial 110 at a predetermined frame rate (100 fps or higher) from a predetermined position on the opposite side of the vial 110 from the side where the illumination device 300 is installed. The camera device 400 may have a telecentric lens instead of a wide-angle lens. The optical axis of the camera device 400 is parallel to the rotation axis B. The field of view (angle of view) of the camera device 400 is adjusted so that at least the entire vial body of the vial 110 is included in the capture range. The focus value of the camera device 400 is also adjusted so that, for example, scratches on the outer wall of the bottle body close to the camera device 400, foreign matter stuck to the inner wall, and foreign matter in the liquid can be clearly captured. The camera device 400 may be configured to include, for example, a color camera or a black-and-white camera equipped with a CCD (Charge-Coupled Device) image sensor or a CMOS (Complementary MOS) image sensor having a pixel capacity of several million pixels. The camera device 400 is connected to the information processing device 500 via a wired or wireless connection. The camera device 400 is configured to transmit time-series images obtained by capturing images to the information processing device 500 together with information indicating the capture time, etc.

[0020] The display device 600 is a display device such as an LCD (Liquid Crystal Display). The display device 600 is connected to the information processing device 500 by wire or wirelessly. The display device 600 is configured to display the results of an inspection of the vial 110 performed by the information processing device 500.

[0021] The information processing device 500 is a device that performs image processing on time-series images captured by the camera device 400 and inspects defects in the vial 110. The information processing device 500 is connected to the gripping / rotating device 200, the camera device 400, and the display device 600 by wire or wirelessly.

[0022] Referring to FIG. 2, an example of the information processing device 500 includes a communication I / F unit 510 , an operation input unit 520 , a storage unit 530 , and an arithmetic processing unit 540 .

[0023] The communication I / F unit 510 is composed of a data communication circuit and is configured to perform data communication with the gripping and rotating device 200, the lighting device 300, the camera device 400, the display device 600, and other external devices (not shown) via wired or wireless connections. The operation input unit 520 is composed of operation input devices such as a keyboard and a mouse, and is configured to detect operations by the operator and output the detected operations to the calculation processing unit 540.

[0024] The storage unit 530 is composed of one or more storage devices of one or more types, such as a hard disk or memory, and is configured to store processing information and programs 531 required for various processes in the arithmetic processing unit 540. The programs 531 are programs that are read into the arithmetic processing unit 540 and executed to realize various processing units, and are read in advance from an external device or recording medium (not shown) via a data input / output function such as the communication I / F unit 510 and stored in the storage unit 530. The main processing information stored in the storage unit 530 includes image information 532, tracking information 533, and inspection result information 534.

[0025] The image information 532 includes time-series images obtained by successively photographing the vial 110 with the camera device 400 .

[0026] 3 , an example of image information 532 is composed of entries each including a container ID 5321, a camera ID 5322, a shooting time 5323, a rotation angle 5324, a rotation angle 5325, and a frame image 5326. The container ID 5321 field contains an ID that uniquely identifies the vial 110 held by the gripping / rotating device 200. The container ID 5321 may be a serial number assigned to the vial 110, a barcode affixed to the vial 110, or Fingerprint of the Object information collected from the cap of the vial 110. The camera ID 5322 field contains an ID that uniquely identifies the camera device 400 that captured the frame image. The shooting time 5323 field contains a shooting time accurate enough (for example, in milliseconds) to distinguish the frame image from other adjacent frame images. The rotation angle 5324 field contains a rotation angle of the vial 110 around the rotation axis A when the frame image was captured. The rotation angle 5325 field contains the rotation angle of the vial 110 around the rotation axis B when the frame image was captured. The frame image 5326 field contains the acquired frame image. The frame image is a multi-value image. Hereinafter, the frame image is assumed to be, but is not limited to, a gray image in which each pixel has 0 to 255 gradations (brightness 0 to 255). The entries in the image information 532 are arranged in order of the camera ID 5322. Multiple entries with the same camera ID 5322 are arranged in order of the capture time 5323. In the example of FIG. 3, a pair of a container ID and a camera ID is associated with each frame image 5326, but a pair of a container ID and a camera ID may also be associated with each group of multiple frame images 5326. The camera ID may also be omitted.

[0027] The tracking information 533 includes time-series data representing the movement trajectory of the detected and tracked image of a potential foreign object present in the vial 110 .

[0028] 4, an example of the tracking information 533 is composed of entries for a container ID 5331 and a pair of a tracking ID 5332 and a pointer 5333. An ID that uniquely identifies the vial 110 is set in the container ID 5331 entry, and an entry consisting of a pair of a tracking ID 5332 and a pointer 5333 is provided for each foreign substance candidate to be tracked. An ID for distinguishing the foreign substance candidate to be tracked from other foreign substance candidates in the same vial 110 is set in the tracking ID 5332 item. For example, the tracking ID 5332 can be composed of a combination of an ID that identifies the inspection area, an ID that identifies a sub-inspection area, and an ID that identifies an image of the foreign substance candidate within the sub-inspection area, but is not limited to this. A pointer to movement trajectory information 5334 of the foreign substance candidate to be tracked is set in the pointer 5333 item.

[0029] The movement trajectory information 5334 is composed of entries each consisting of a set of a time 53341, position information 53342, size 53343, brightness distribution 53344, and shape 53345. The items of the time 53341, position information 53342, size 53343, brightness distribution 53344, and shape 53345 contain the image capture time, coordinate values ​​indicating the position of the foreign object candidate to be tracked at that image capture time (e.g., the center of gravity of the image of the foreign object candidate), the size of the foreign object candidate, the brightness distribution of the foreign object candidate, and the shape of the foreign object candidate. The image capture time 5323 of the frame image is used as the image capture time set for the time 53341. The coordinate values ​​may be coordinate values ​​in a predetermined coordinate system. Furthermore, the predetermined coordinate system may be a camera coordinate system centered on the camera, or a world coordinate system centered on a certain position in real space. The entries of the movement trajectory information 5334 are arranged in order of the time 53341. The time 53341 of the first entry is the tracing start time. The time 53341 of the last entry is the tracing end time. The time 53341 of the entries other than the first and last entries is the tracing intermediate time.

[0030] The inspection result information 534 includes information according to the inspection result of the vial 110. Referring to Fig. 5, an example of the inspection result information 534 is composed of the following entries: a container ID 5341, a pair of a tracking ID 5342 and a determination result 5343, and an inspection result 5347. An ID that uniquely identifies the vial 110 to be inspected is set in the entry for container ID 5341. An entry consisting of a pair of a tracking ID 5342 and a determination result 5343 is provided for each tracked foreign substance candidate. The tracking ID 5332 of the tracking information 533 is set in the item for tracking ID 5342. The determination result 5343 is set to indicate whether the foreign substance candidate to be tracked identified by the tracking ID 5342 is a foreign substance or something other than a foreign substance (such as an air bubble). In the entry for inspection result 5347, an inspection result of OK (inspection passed) is set if not a single foreign object is detected, and an inspection result of NG (inspection failed) is set if at least one foreign object is detected.

[0031] The arithmetic processing unit 540 has a processor such as a CPU (Central Processing Unit) and its peripheral circuits, and is configured to read and execute a program 531 from the storage unit 530, thereby realizing various processing units through cooperation between the above hardware and the program 531. The main processing units realized by the arithmetic processing unit 540 include a gripping / rotation control unit 541, an acquisition unit 542, a detection unit 543, and a display control unit 544.

[0032] The gripping / rotation control unit 541 is configured to control the gripping / rotating device 200. The gripping / rotation control unit 541 controls operations such as lowering, closing, rotating, opening, and raising of the chuck fingers 211 by transmitting and receiving signals to and from the chuck mechanism 206 of the gripping / rotating device 200 via the communication I / F unit 510. The gripping / rotation control unit 541 also controls the rotation of the vial 110 gripped by the gripping / rotating device 200 about the rotation axis B by transmitting and receiving signals to and from the motor 207 via the communication I / F unit 510. The gripping / rotation control unit 541 also monitors the rotation angles of the vial 110 gripped by the gripping / rotating control unit 541 about the rotation axes A and B by transmitting and receiving signals to and from the rotation angle detectors 209 and 210 via the communication I / F unit 510.

[0033] The acquisition unit 542 is configured to control the lighting device 300 and the camera device 400. The acquisition unit 542 controls the lighting device 300, such as turning on and off, by transmitting and receiving signals to and from the lighting device 300 via the communication I / F unit 510. The acquisition unit 542 also controls the photographing of the vial 110 held by the gripping and rotating device 200 and acquires time-series images obtained by photographing the vial 110 by transmitting and receiving signals to and from the camera device 400 via the communication I / F unit 510. The acquisition unit 542 also creates image information 532 based on the images acquired from the camera device 400 and information on the rotation angles of the vial 110 about the rotation axis A and the rotation axis B monitored by the rotation angle detectors 209 and 210, and stores the image information 532 in the storage unit 530.

[0034] The detection unit 543 is configured to perform a foreign substance inspection based on the image information 532 acquired by the acquisition unit 542. For example, the detection unit 543 is configured to extract images of foreign substance candidates from the image of the container and generate tracking information 533 including time-series data representing the movement trajectories of the foreign substance candidates. The detection unit 543 is also configured to determine whether each foreign substance candidate is a foreign substance based on the tracking information 533. The detection unit 543 is further configured to create inspection result information 534 based on the results of the above determination and store the information in the storage unit 530.

[0035] The display control unit 544 is configured to output part or all of the test result information 534 created by the detection unit 543 to the display device 600 and / or transmit it to an external device (not shown) via the communication I / F unit 510.

[0036] Next, the operation of the inspection device 100 according to this embodiment will be described with reference to Fig. 6. The inspection device 100 performs the process shown in Fig. 6 for each vial 110 to be inspected. Dust and other particles that may adhere to the outside of the vial 110 to be inspected are blown away with air immediately before inspection.

[0037] When the process of FIG. 6 starts, the gripping / rotating device 200 is in an initial state. In the initial state, the chuck fingers 211 of the gripping / rotating device 200 are open, raised, and stopped from rotating. At this time, the rotation angle about the rotation axis A detected by the rotation angle detector 209 is set to 0°. The motor 207 of the gripping / rotating device 200 also stops the rotation of the flat plate-like member 201 about the rotation axis B at an angle where the rotation axis A coincides with the vertical. At this time, the rotation angle about the rotation axis B detected by the rotation angle detector 210 is set to 0°. In this initial state, the gripping / rotating device 200 of the inspection apparatus 100 loads the vial 110 to be inspected (step S1). At this time, the gripping / rotating control unit 541 uses, for example, a robot arm (not shown) or a human hand to place the vial 110 in an upright position at a predetermined position on the plate 205 of the gripping / rotating device 200. Next, the gripping / rotation control unit 541 controls the chuck mechanism 206 to lower the chuck fingers 211 and chuck the head of the vial 110. As a result, the vial 110 to be inspected is gripped in an upright position by the gripping / rotation device 200. At this time, the central axis of the vial 110 is aligned substantially vertically with the rotation axis A.

[0038] Next, the inspection device 100 rotates and photographs the vial 110 (step S2). In step S2, the gripping / rotation control unit 541 controls the chuck mechanism 206 and the motor 207 to rotate the vial 110 about rotation axis A and rotation axis B according to a preset rotation angle timeline. When starting to rotate the vial 110 about rotation axis A, the gripping / rotation control unit 541 issues a rotation start command specifying the rotation direction and rotation speed to the chuck mechanism 206, and when terminating the rotation, issues a rotation end command to the chuck mechanism 206. Furthermore, when rotating the vial 110 about rotation axis B, the gripping / rotation control unit 541 issues a rotation start command specifying the rotation direction and rotation speed to the motor 207, and when terminating the rotation, issues a rotation end command to the motor 207. During rotation, the gripping and rotation control unit 541 also monitors the angles of rotation around the rotation axis A and the rotation axis B detected by the rotation angle detector 209 and the rotation angle detector 210 .

[0039] Meanwhile, in step S2, when the camera device 400 starts photographing the vial 110, the acquisition unit 542 issues a photographing start command to the camera device 400 and a turn-on command to the lighting device 300. When the photographing ends, the acquisition unit 542 issues a turn-off command to the lighting device 300 and a photographing end command to the camera device 400. However, the lighting device 300 may be kept on all the time. The acquisition unit 542 may also issue a command to control the zoom and / or focus to the camera device 400 so as to change the zoom amount and / or focus value of the camera device 400 during photographing. Note that, when rotation and photographing are started simultaneously in step S2, the gripping / rotation control unit 541 and the acquisition unit 542 are configured to operate synchronously. For example, when the vial 110 is rotated around the rotation axis A and photographing is started simultaneously with the camera device 400, the gripping / rotation control unit 541 issues a rotation start command to the chuck mechanism 206, and the acquisition unit 542 issues a photographing start command to the camera device 400 in synchronization with the rotation start command.

[0040] Also, in step S2, while the camera device 400 is capturing images, the acquisition unit 542 receives time-series images sent from the camera device 400 along with information indicating the capture time. The acquisition unit 542 also receives information on the monitored rotation angles around the rotation axis A and the rotation axis B from the rotation angle detectors 209, 210 via the grip / rotation control unit 541. The acquisition unit 542 then associates the time-series images received from the camera device 400 with the capture time and the rotation angles around the rotation axis A and the rotation axis B, and stores the associated information as image information 532 in the storage unit 530. In the above example, the acquisition unit 542 used information on the monitored rotation angles around the rotation axis A and the rotation axis B. However, since the rotation angle timeline is set in advance and known, processing may be performed assuming that rotation always starts and ends at fixed times. In other words, if the grasping / rotating device 200 is configured to perform programmed operations in milliseconds in response to an operation start instruction from the information processing device 500, the acquisition unit 542 will automatically determine the programmed rotation angles of rotation axis A and rotation axis B from the time information of the operation start instruction from the information processing device 500, associate them with time-series images, and store them in the memory unit 530 as image information 532.

[0041] Next, the detection unit 543 of the inspection device 100 inspects the vial 110 for defects based on the acquired image information 532, creates inspection result information 534 based on the inspection results, and stores it in the storage unit 530 (step S3). Next, the display control unit 544 of the inspection device 100 displays the inspection result information 534 on the display device 600 and / or transmits it to an external device (not shown) via the communication I / F unit 510 (step S4). Next, the inspection device 100 removes the inspected vial 110 (step S5). At this time, the gripping / rotation control unit 541 controls the chuck mechanism 206 to release and then raise the chuck fingers 211, and moves the vial 110 on the plate 205 of the gripping / rotation device 200 to a storage location corresponding to the inspection results using a robot arm or manual labor (not shown). In the above example, defect detection by image analysis (step S3) was performed after the vial 110 was rotated and photographed (step S2). However, the rotation and photographing of the vial 110 (step S2) and the defect detection by image analysis (step S3) may be performed in parallel.

[0042] Next, a specific example of the rotation angle timeline will be described.

[0043] FIG. 7 is a diagram showing an example of a rotation angle timeline. In this example, the vial 110 is rotated as follows: Interval 1 (time t0-t1): The vial 110 is stationary in an upright position. Interval 2 (time t1-t2): The vial 110 rotates 360° around rotation axis A in an upright position. Interval 3 (time t2-t3): The vial 110 rotates 90° around rotation axis B. At time t3, the vial 110 assumes a sideways position. Interval 4 (time t3-t4): The vial 110 rotates 720° around rotation axis A in a sideways position. The rotation direction is such that the top of the vial 110 approaches the back and the bottom approaches the front as viewed from the camera device 400. At time t4, the vial 110 assumes a sideways position with the same rotation angle as at time t3. Section 5 (time t4-t5): The vial 110 rotates around rotation axis A at a relatively slow rotation speed (for example, one rotation per second) while rotating -90° around rotation axis B. At time t5, the vial 110 assumes an upright position with the same rotation angle as at time t1. Section 6 (time t5-t6): The vial 110 rotates another 720° around rotation axis A in the same direction and speed as in section 5 while in an upright position. Section 7 (time t6-t7): The vial 110 remains stationary in an upright position.

[0044] When the vial 110 is rotated along the rotation angle timeline shown in FIG. 7 , the acquisition unit 542, for example, at time t0, sends a turn-on command to the lighting device 300 and a start image capture command to the camera device 400, and at time t7, sends a turn-off command to the lighting device 300 and a stop image capture command to the camera device 400. As a result, the camera device 400 captures the vial 110 over all sections of the rotation angle timeline in FIG. 6 . However, image capture by the camera device 400 may be limited to some sections. For example, image capture by the camera device 400 may be limited to sections 2 and 6 in which the vial 110 is rotating around the upright central axis in an upright position, and section 4 in which the vial 110 is rotating around the upright central axis in a sideways position.

[0045] 8 is a schematic diagram showing how the camera device 400 captures images of the upright vial 110 in section 2 and section 6. In section 2, the vial 110 rotates 360° around the rotation axis A in an upright position, and in section 6, the vial 110 rotates 720° around the rotation axis A in an upright position. The camera device 400 captures images of the rotating vial 110 from a direction perpendicular to the rotation axis A and parallel to the surface of the liquid in the vial 110.

[0046] 9 is a schematic diagram showing how the camera device 400 captures an image of the vial 110 in a sideways position in section 4. In section 4, the vial 110 rotates a total of 720° about the rotation axis A in a sideways position. The camera device 400 captures an image of the bottle body of the rotating vial 110 from a direction perpendicular to the rotation axis A and parallel to the liquid surface in the vial 110. As described above, the rotation direction of the vial 110 at this time is such that the upper side of the vial 110 approaches the back and the lower side approaches the front when viewed from the camera device 400.

[0047] 10 and 11 are schematic diagrams showing how the position of foreign matter, etc. changes in a vial 110 that is rotated around a rotation axis A in a sideways position, as viewed from directions parallel and perpendicular to the rotation axis A.

[0048] The present inventors discovered the following phenomena in the process of searching for a method for inspecting vials for foreign matter.

[0049] First, as shown in Figures 10 and 11, it was confirmed that heavy and large foreign matter 111, such as metal fragments or glass fragments, repeatedly rises due to friction with the container on the curved inner wall of the bottle body and then falls due to gravity in the area 131 below the liquid surface as the vial 110 rotates in a sideways position. The area directly below and slightly behind the bottle body of the vial 110 (the side farther from the camera device 400) becomes invisible due to refraction. Therefore, the area directly below and in front of the bottle body (the side closer to the camera device 400) is the area where the movement of the foreign matter 111 can be reliably observed. Therefore, in order to be able to observe the movement of the foreign matter 111 in such an area, it is desirable that the rotation direction of the vial 110 be such that the upper side of the vial 110 faces the back and the lower side faces the front, as viewed from the camera device 400.

[0050] Furthermore, in region 131, as shown in Figures 10 and 11, air bubbles 112 present in the liquid were observed to consistently rise during the rotation of vial 110, and to repeatedly descend and rise (near the liquid surface) while riding on the liquid flow near the inner wall due to the rotation, unlike the repeated rising and settling of foreign matter 111. Furthermore, unlike heavy and large foreign matter 111 such as metal fragments or glass pieces, small foreign matter that sink slowly did not remain at the bottom and instead behaved more like air bubbles. However, small submerged foreign matter and air bubbles can be distinguished from each other based on detailed differences such as appearance and initial position. Furthermore, scratches on the bottle body, like scratches 115, move in sync with the rotation of vial 110 while maintaining a constant shape.

[0051] Furthermore, some of the foreign matter mixed in the vial 110 floated on the liquid surface. Furthermore, among the foreign matter floating on the liquid surface, there were foreign matter such as foreign matter 113 and 114 that adhered to the inner surface of the vial 110 and moved as the vial 110 rotated on its side and around its upright central axis. On the other hand, some foreign matter remained on the liquid surface, depending on its affinity with the liquid and its initial position, even when the vial 110 was turned on its side and rotated around its upright central axis. Foreign matter that remained on the liquid surface was captured as a darkened image according to its reflection coefficient. However, due to various factors, darkened areas that were difficult to distinguish from the image of the foreign matter appeared on the liquid surface, making it difficult to reliably detect foreign matter that remained on the liquid surface.

[0052] It was also confirmed that even when the vial 110 was placed in an inverted position and rotated around its upright central axis, foreign matter remaining on the liquid surface remained primarily within region 133 shown in FIG. 11 . Region 133 is located near the bottom of the vial 110, and because the liquid surface is curved due to the surface tension of the liquid, it is particularly difficult to observe the image of foreign matter in this region. Therefore, the inventors of the present disclosure confirmed that when the vial 110 was rotated around its upright central axis at a relatively slow speed of approximately one rotation per second while changing the orientation of the vial 110 from an inverted position to an upright position, foreign matter that had been in region 133 appeared in an area above the liquid surface. Furthermore, it was confirmed that foreign matter that appeared in an area above the liquid surface near the left and right sides of the vial 110 as viewed from the camera device 400 when the vial 110 was placed in the upright position moved to an area above the liquid surface on the front side (the side closer to the camera device 400) of the vial 110, which rotates around the upright central axis, even after the vial 110 was placed in the upright position. It was confirmed that this phenomenon does not occur rarely and is highly reproducible. Note that, when a foreign particle appears in the area above the liquid surface immediately after the vial 110 is placed in the upright position, it moves in the direction of rotation of the vial 110 while gradually falling toward the water surface due to the influence of gravity and the rotation of the vial 110, which rotates around the upright central axis in the upright position, and finally enters the black band on the water surface, making it difficult to observe again.

[0053] Next, an example of a method in which the detection unit 543 detects foreign matter based on image information 532 captured by the camera device 400 while the vial 110 is rotating along the rotation angle timeline shown in Figure 7 will be described.

[0054] 12, the detection unit 543 first acquires image information for a predetermined section (e.g., image information for sections 2, 4, and 6) from the image information 532 shown in FIG. 3 (step S11). Next, the detection unit 543 sets multiple inspection areas in the image of the container in the frame image included in the acquired image information based on the position of the liquid surface in the container (step S12). Next, the detection unit 543 sets at least one sub-inspection area for each of the set inspection areas based on the brightness of the image of the inspection area (step S13). Next, the detection unit 543 sets, for each of the set sub-inspection areas, the type of binarization method for extracting an image of a foreign substance candidate from the image of the sub-inspection area (step S14).

[0055] Next, the detection unit 543 focuses on one of the multiple inspection areas set above (step S15). Next, the detection unit 543 focuses on one sub-inspection area of ​​the currently selected inspection area (step S16). Next, the detection unit 543 applies the type of binarization method set for the currently selected sub-inspection area to the image of the currently selected sub-inspection area in each frame image included in the image information for the specified section acquired in step S11 to detect an image of a foreign object candidate (step S17). Next, the detection unit 543 tracks the image of the detected foreign object candidate across multiple frame images to create a movement trajectory of the foreign object candidate (step S18). Next, the detection unit 543 determines whether the foreign object candidate is a foreign object or noise such as an air bubble based on the movement trajectory of the foreign object candidate created above (step S19).

[0056] After completing the above series of processes for the currently selected sub-inspection area, the detection unit 543 shifts its attention to the next sub-inspection area of ​​the currently selected inspection area (step S20), returns to step S17, and executes the same series of processes described above for the newly selected sub-inspection area. After completing the above series of processes for all sub-inspection areas set in the currently selected inspection area (YES in step S21), the detection unit 543 shifts its attention to the next inspection area (step S22), returns to step S16, and executes the same series of processes described above for the newly selected inspection area. After completing the above series of processes for all set inspection areas (YES in step S23), the detection unit 543 creates inspection result information 534 based on the foreign substance determination results for the foreign substance candidates in the sub-inspection areas of all inspection areas (step S24). The detection unit 543 then terminates the process shown in FIG. 12 .

[0057] Next, the main processing steps in FIG. 12 will be described in detail.

[0058] First, an example of setting the inspection areas will be described. Fig. 13 is a schematic diagram showing an example of the inspection areas set in the image of a container. In this example, the detection unit 543 sets four inspection areas 124, 123, 121, and 127 in order from the top of the bottle to the bottom of the bottle in the image of the upright container, and sets two inspection areas 125 and 122 in order from the top to the bottom of the container in the image of the container lying on its side.

[0059] Due to various factors, a black area (hereinafter referred to as a black band) 126 as shown in Fig. 13 appears at the position of the liquid surface in the image of the container. The detection unit 543 detects the position of the liquid surface by, for example, detecting the black band 126 from the image. Then, the detection unit 543 sets the above-mentioned multiple inspection areas based on the detected position of the liquid surface and the orientation of the container.

[0060] 13, inspection area 122 is set in the portion of the container below black band 126 detected from the image of the container in a sideways position, and inspection area 125 is set in the portion of the container above black band 126. That is, inspection area 122 is set to cover the in-liquid area below the liquid surface, and inspection area 125 is set to cover the non-in-liquid area above the liquid surface. Hereinafter, inspection area 122 will also be referred to as the 90-degree in-liquid area, and inspection area 125 will also be referred to as the 90-degree wall area.

[0061] The 90-degree liquid region 122 may capture images of foreign objects (microparticles, hair, fiber fragments, etc.) that are heavier than the liquid and floating in the liquid, foreign objects (metal, glass, etc.) that are heavier than the liquid and sink to the bottom and do not move, and air bubbles in the liquid. Meanwhile, the 90-degree wall region 125 may capture images of foreign objects (hair, fiber fragments, soot, etc.) that have low affinity with the liquid and do not sink in the liquid, such as foreign objects that adhere to the inner wall surface of the container while the container is shaking, or foreign objects with a light specific gravity that appear above the liquid surface. In the example shown in FIG. 13 , no inspection area is set for the container seam or bottle top in the image of the container in a sideways position, but inspection areas may also be set for these areas. Also, in the example shown in FIG. 13 , two inspection areas are set for the image of the container in a sideways position, but the number of inspection areas set is not limited to two and may be one, three, or more. However, although it is possible to set one inspection area across the area below the black band 126 and the area above the black band 126, this is not preferable.

[0062] 13 , inspection area 124 is set in the portion of the container above black band 126, including black band 126 detected from the image of the upright container, and the portion of the container immediately below black band 126 but excluding black band 126 is divided into three parts, and inspection area 123, inspection area 121, and inspection area 127 are set in that order from top to bottom. That is, inspection area 124 is set to cover the liquid surface area and non-submerged area, inspection area 123 is set to cover the submerged area near the liquid surface, inspection area 121 is set to cover the submerged area sufficiently below the liquid surface, and inspection area 127 is set to cover the area near the bottom of the bottle. Hereinafter, inspection area 121 will also be referred to as the 0-degree submerged area, inspection area 123 is set to cover the 0-degree liquid surface area, inspection area 124 is set to cover the 0-degree wall area, and inspection area 127 is set to cover the 0-degree bottom area.

[0063] The 0-degree wall region 124 may capture images of foreign matter (e.g., hair, fiber fragments, soot, etc.) that has a low affinity with the liquid and does not sink in the liquid, such as foreign matter adhering to the inner wall surface of the container while the container is shaking, or foreign matter with a low specific gravity that appears above the liquid surface. The 0-degree liquid surface region 123 may capture images of foreign matter (e.g., hair, fiber fragments, soot, etc.) that has a low affinity with the liquid and does not sink in the liquid. The 0-degree liquid region 121 may capture images of foreign matter (e.g., microparticles, hair, fiber fragments, etc.) that is heavier than the liquid and floats in the liquid, as well as images of air bubbles in the liquid. The 0-degree bottom region 127 may capture images of foreign matter (e.g., metal, glass, etc.) that is heavier than the liquid and sinks downward and does not move, as well as images of air bubbles in the liquid. In the example shown in FIG. 13 , no inspection region is set for the container's seam or bottle head in the image of the upright container, but inspection regions may also be set for these areas. 13, four inspection areas are set in the image of the upright container, but the number of inspection areas set is not limited to four and may be one, two, three, five or more. In the example shown in FIG. 13, there are no overlapping areas between the inspection areas, but two adjacent inspection areas may partially overlap. One inspection area may be set across both the area below the black band 126 and the area above the black band 126, but this is not preferred.

[0064] Next, the types of binarization methods used in this embodiment will be described. In this embodiment, the following eight types of binarization methods are used. However, the binarization methods used are not limited to the following, and may be fewer than eight types or nine types or more. (1) Simple binarization method (2) Adaptive binarization method (3) Temporal difference binarization method (4) Same angle difference binarization method (5) Adaptive AND same angle difference binarization method (6) Adaptive OR temporal difference binarization method (7) Temporal difference AND same angle difference binarization method (8) Adaptive AND temporal difference binarization method

[0065] The simple binarization method binarizes each pixel of the image to be binarized using a preset fixed threshold value. The simple binarization method can be used to distinguish between light and dark areas of an image (bright and dark areas, described below). Light areas of an image include, for example, uniform backgrounds and translucent objects. Dark areas of an image include, for example, patterns that appear on the bottom or side of a container image or opaque objects.

[0066] The adaptive thresholding method binarizes each pixel of an image to be binarized using a local threshold for each pixel. The adaptive thresholding method can extract potential foreign object images from a single image. The local threshold, for example, uses the average or median value of multiple pixels within a rectangle (window) of a predetermined size that includes the pixel to be binarized. Possible window sizes include 3x3, 15x15, and 41x41. Furthermore, the luminance of the pixel to be binarized can be binarized using either the luminance of the pixel itself or a representative value (e.g., the average or median) calculated from the luminance of the pixel and its neighboring pixels. When used in conjunction with a median filter, the adaptive thresholding method can also extract areas with little difference in luminance from the background. However, a large amount of noise may occur near areas where the background luminance changes suddenly.

[0067] The temporal difference binarization method creates a difference image between the current frame image and an old frame image taken N frames before the current frame image, and binarizes each pixel of the difference image using a preset fixed threshold value. The temporal difference binarization method is used to extract images of potential foreign objects from the current frame image. N is determined depending on the frame rate and the rotation speed of the container. The temporal difference binarization method can extract areas of moving objects with little difference in brightness from the background, but cannot extract images of stationary or quasi-stationary objects. Note that instead of the old frame image, a frame image taken N frames after the current frame image may be used.

[0068] The same-angle difference binarization method creates a difference image between the current frame image and an older frame image taken of a container at the same rotation angle before the current frame image, and then binarizes each pixel of the difference image using a preset fixed threshold value. The same-angle difference binarization method is used to extract images of potential foreign objects from the current frame image. The same-angle difference binarization method does not extract scratches on the container as images of potential foreign objects, and has performance comparable to that of the time-dependent difference binarization method near the center of the bottle body. However, the same-angle difference binarization method is prone to noise due to phase shifts in the bottle position and rotation near the bottom and sides of the bottle. Note that instead of the older frame image, a frame image taken at the same rotation angle after the current frame image may be used.

[0069] The adaptive AND same-angle difference binarization method extracts images of foreign substance candidates from the image to be binarized by taking the logical product (AND) of the images of foreign substance candidates extracted by the adaptive binarization method and the images of foreign substance candidates extracted by the same-angle difference binarization method.

[0070] The adaptive OR temporal difference binarization method extracts images of foreign substance candidates from the image to be binarized by taking the logical sum (OR) of the images of foreign substance candidates extracted by the adaptive binarization method and the images of foreign substance candidates extracted by the temporal difference binarization method.

[0071] The temporal difference AND same angle difference binarization method extracts images of foreign substance candidates from the image to be binarized by taking the logical product (AND) of the images of foreign substance candidates extracted by the temporal difference binarization method and the images of foreign substance candidates extracted by the same angle difference binarization method.

[0072] The adaptive AND temporal difference binarization method extracts images of foreign substance candidates from the image to be binarized by taking the logical product (AND) of the images of foreign substance candidates extracted by the adaptive binarization method and the images of foreign substance candidates extracted by the temporal difference binarization method.

[0073] Next, an example of setting sub-inspection areas in the inspection area and an example of setting the type of binarization method for extracting images of foreign substance candidates from each sub-inspection area will be described. Note that the set inspection areas are five types in total, excluding the 0-degree bottom surface area 127 shown in FIG. 13.

[0074] (1) 0-degree submerged region 121 and 90-degree submerged region 122 (1-1) Example of sub-inspection region setting: The detection unit 543 sets two sub-inspection regions in each of the 0-degree submerged region 121 and the 90-degree submerged region 122: a dark region composed of pixels with a brightness value below a predetermined threshold and a bright region composed of pixels with a brightness value equal to or greater than the predetermined threshold. The predetermined threshold may be, for example, a brightness value of "150," but is not limited thereto. (1-2) Characteristics of the dark regions: The dark regions in the 0-degree submerged region 121 and the 90-degree submerged region 122 are characterized by the appearance of patterns on the bottom surface of the container or dark foreign matter. Furthermore, foreign matter initially positioned at the bottom is characterized by not leaving the vicinity of the bottom surface of the container when the container is rotated around its upright central axis in a sideways position. Furthermore, relatively heavy (large) foreign matter is characterized by a quasi-stationary state due to the balance between the upward movement due to the rotation and the sedimentation due to gravity. (1-3) Example of Setting the Binarization Method for Dark Regions The detection unit 543 sets the adaptive AND same-angle difference binarization method for the dark regions of the 0-degree submerged region 121 and the 90-degree submerged region 122. This is to reliably extract images of dark foreign object candidates stagnating near the bottom while reducing pattern noise. The adaptive binarization method used in the adaptive AND same-angle difference binarization method may use the average or median brightness value of multiple pixels within a 41 x 41 rectangle centered on the pixel to be binarized as a local threshold for the pixel to be binarized, and may further extract only pixels whose brightness is 20 or more points lower than the local threshold. The same-angle difference binarization method used in the adaptive AND same-angle difference binarization method may extract pixels whose brightness is 20 or more points lower than that of the previous frame image as pixels of the foreign object candidate image. (1-4) Characteristics of Bright Regions The bright regions of the 0-degree submerged region 121 and the 90-degree submerged region 122 may capture images of translucent or tiny foreign objects whose brightness is extremely small compared to the background brightness. Images of both fast-moving foreign objects and quasi-static foreign objects may also be captured. (1-5) Example of Setting the Binarization Method for Bright Regions The detection unit 543 sets an adaptive OR time-dependent difference binarization method for the bright regions of the 0-degree submerged region 121 and the 90-degree submerged region 122. This is to ensure that images of foreign objects whose brightness is extremely small compared to the background brightness are extracted.The adaptive binarization method used in the adaptive AND temporal difference binarization method may be to calculate the difference between the median value within a 3 × 3 window centered on the pixel to be binarized and the median or average value within a window of a predetermined size. The predetermined size may be a window size (e.g., 15 × 15) that ensures approximately three times the area of ​​the minute foreign object. By setting such a binarization method, it is possible to extract, as an image of a foreign object, an area where pixels that are slightly darker than the background are densely concentrated.

[0075] (2) 0-Degree Wall Surface Region 124 (2-1) Example of Setting Sub-Inspection Regions The detection unit 543 sets two sub-inspection regions in the 0-degree wall surface region 124: a dark region composed of pixels with a brightness value below a predetermined threshold and a bright region composed of pixels with a brightness value equal to or greater than the predetermined threshold. The predetermined threshold may be, for example, a brightness value of "80," but is not limited thereto. (2-2) Characteristics of the Dark Region The dark region in the 0-degree wall surface region 124 corresponds to a relatively black area on the periphery of the liquid surface. This dark region may capture an image of a foreign object moving in synchronization with the rotation of the container. That is, foreign objects present in the dark region of the 0-degree wall surface region 124 are characterized by not being stationary. Furthermore, foreign objects present in the dark region of the 0-degree wall surface region 124 are characterized by appearing brighter than the background. Furthermore, the 0-degree wall surface region 124 is characterized by the liquid surface state changing before and after the container is shaken. (2-3) Example of Setting the Binarization Method for Dark Regions: The detection unit 543 sets a time-dependent difference binarization method for the dark regions of the 0-degree wall surface region 124 to extract bright regions within the dark regions as foreign object candidate images. For example, the threshold value may be set to "10." If the brightness value of a foreign object present in the dark region is less than 80, the same as the sub-inspection region setting threshold, the binarization method set above will not extract the region of the foreign object itself. However, since there will always be a bright region (brightness 80 or higher) around the foreign object, this can be extracted as the foreign object candidate image. (2-4) Characteristics of Bright Regions: Foreign objects present in the bright regions of the 0-degree wall surface region 124 are attached to the inner wall surface of the container with liquid, and therefore thin fibers and other small objects are clearly visible and the foreign object size is large. Furthermore, the bright regions of the 0-degree wall surface region 124 are characterized by low noise. (2-5) Example of Setting the Binarization Method for Bright Regions The detection unit 543 sets the time-dependent difference binarization method or the same-angle difference binarization method for extracting images of foreign matter candidates using, for example, a threshold value of "20" for the bright regions of the 0-degree wall surface region 124. Because foreign matters present in the bright regions have the characteristics described above, sufficient performance can be ensured by using either the time-dependent difference binarization method or the same-angle difference binarization method alone.

[0076] (3) 90-Degree Wall Surface Region 125 (3-1) Example of Setting Sub-Inspection Regions The detection unit 543 sets two sub-inspection regions in the 90-degree wall surface region 125: a dark region composed of pixels with brightness values ​​below a predetermined threshold, and a bright region composed of pixels with brightness values ​​equal to or greater than a threshold smaller than the predetermined threshold. The predetermined threshold may be, for example, a brightness value of "150," and the threshold smaller than the predetermined threshold may be, for example, a brightness value of "80," but these are not limited thereto. (3-2) Characteristics of Dark Regions The dark region in the 90-degree wall surface region 125 corresponds to a bottom surface pattern or the like. The image of a foreign object present in this dark region is characterized by being regionally connected to the pattern on the bottom surface of the container. (3-3) Example of Setting Binarization Method for Dark Regions The detection unit 543 sets a time-dependent difference AND same-angle difference binarization method for the dark region in the 90-degree wall surface region 125. This allows the bottom pattern to be separated from the area where the foreign object image and the bottom pattern are connected, allowing only the image of the foreign object candidate to be extracted. (3-4) Characteristics of the Bright Region Foreign objects present in the bright region of the 90-degree wall region 125 move in sync with the rotation of the container and do not remain stationary. Furthermore, foreign objects present in this bright region are characterized by their attachment to the inner wall surface of the container accompanied by liquid, making thin fibers and other objects clearly visible and their size large. Furthermore, the bright region has less noise near the liquid surface and the container side than the 0-degree wall. (3-5) Example of Setting the Binarization Method for the Bright Region The detection unit 543 sets an adaptive AND temporal difference binarization method for the bright region of the 90-degree wall region 125. This allows the foreign object region to be reliably extracted without being affected by noise.

[0077] (4) 0-Degree Liquid Surface Region 123 (4-1) Example of Setting a Sub-Inspection Region The detection unit 543 sets two sub-inspection regions in the 0-degree liquid surface region 123: a dark region composed of pixels with a brightness value below a predetermined threshold, and a bright region composed of pixels with a brightness value equal to or greater than the predetermined threshold. The predetermined threshold may be, for example, a brightness value of "80," but is not limited to this. (4-2) Characteristics of the Dark Region The dark region in the 0-degree liquid surface region 123 corresponds to the black area on the periphery of the liquid surface. The image of a foreign object present in this dark region is characterized by being adjacent to or connected to the black band on the liquid surface. Furthermore, the image of a foreign object present in the dark region of the 0-degree liquid surface region 123 may be observed against the black band on the liquid surface as a background. (4-3) Example of Setting a Binarization Method for the Dark Region The detection unit 543 sets a time-dependent difference AND same-angle difference binarization method for the dark region of the 0-degree liquid surface region 123. This allows the black band on the liquid surface to be separated from the area where the image of the foreign object and the black band on the liquid surface are connected, allowing only the image of the foreign object candidate to be extracted. (4-4) Characteristics of the Bright Region Foreign objects present in the bright region of the 0-degree liquid surface region 123 move in sync with the rotation of the container and do not remain stationary. Furthermore, foreign objects present in the bright region of the 0-degree liquid surface region 123 adhere to the inner wall surface of the container along with the liquid, making thin fibers and other small objects clearly visible, and the foreign object size is also large. Furthermore, there is more noise near the liquid surface and the container side in the bright region of the 0-degree liquid surface region 123 than in the 0-degree wall surface. (4-5) Example of Setting the Binarization Method for the Bright Region The detection unit 543 sets an adaptive AND temporal difference binarization method for the bright region of the 0-degree liquid surface region 123. This allows the foreign object region to be reliably extracted without being affected by noise.

[0078] Next, an example of the operation for creating a movement trajectory of a foreign object candidate will be described.

[0079] The detection unit 543 detects images of foreign substance candidates for each section of the rotation angle timeline and for each sub-inspection region of the inspection region, and creates a movement trajectory. For example, in section 4, in which the vial 110 is in a sideways position and rotates 720° around the upright central axis, the detection unit 543 independently detects images of foreign substance candidates for a total of four sub-regions, namely, a dark region in the 90° submerged region, a bright region in the 90° submerged region, a dark region in the 90° wall region, and a bright region in the 90° wall region, and creates a movement trajectory. Also, in section 6, in which the vial 110 is in an upright position and rotates 720° around the upright central axis, the detection unit 543 independently detects images of foreign substance candidates for a total of six sub-regions, namely, a dark region in the 0° submerged region 121, a bright region in the 0° submerged region 121, a dark region in the 0° liquid surface region, a bright region in the 0° liquid surface region, a dark region in the 0° wall region, and a bright region in the 0° wall region, and creates a movement trajectory.

[0080] An example of a method for detecting an image of a foreign substance candidate from a sub-inspection area of ​​the inspection area in each section and creating a movement trajectory of the foreign substance candidate will be described below.

[0081] <Dark Regions in the 90° Submerged Region 122 in Section 4> The detection unit 543 acquires image information including the camera ID 5322 of the camera device 400 and the shooting time 5323 included in Section 4 from the image information 532 shown in FIG. 3 . Next, the detection unit 543 sets a dark region in the 90° submerged region 122 for each frame image included in the acquired image information, and also sets an adaptive AND same-angle difference binarization method. Next, the detection unit 543 extracts images of foreign substance candidates from the dark regions set in each frame image using the set binarization method. Because the same-angle difference binarization method requires pairs of images at the same angle, the detection unit 543 uses pairs of images at the same angle in Section 4 to extract images of foreign substance candidates by the same-angle difference binarization method. Furthermore, the detection unit 543 extracts images of foreign substance candidates from the frame images in Section 4 using the adaptive binarization method. The detection unit 543 then takes the logical product of the foreign substance candidate images detected by both methods, and extracts the foreign substance candidate images from the frame images of section 4 using the adaptive AND same-angle difference binarization method.

[0082] Next, the detection unit 543 tracks the image of the detected foreign matter candidate in the time-series frame images included in the acquired image information of Section 4, and, based on the tracking results, generates tracking information 533 related to the dark region of the 90° submerged region 122 in Section 4. The detection unit 543 may generate the tracking information 533, for example, by the following method.

[0083] First, the detection unit 543 initializes the tracking information 533 related to the dark region of the 90° submerged region 122 in section 4. In this initialization, the container ID of the vial 110 is set in the entry for container ID 5331 in Fig. 4. Next, the detection unit 543 tracks the images of the foreign substance candidate in the time-series frame images using the method described below, and creates, for each image of the foreign substance candidate, an entry for a pair of tracking ID 5332 and pointer 5333 in Fig. 4, and movement trajectory information 5334, depending on the tracking results.

[0084] First, the detection unit 543 focuses on the frame image with the oldest capture time among the frame images in the latter half of section 4. Next, the detection unit 543 assigns a unique tracking ID to each image of a foreign object candidate detected in the frame image of interest. Next, for each detected foreign object candidate image, the detection unit 543 sets the tracking ID assigned to the image of the foreign object candidate detected in the frame image of interest in the tracking ID 5332 field in FIG. 4 , sets the capture time of the frame image of interest in the time 53341 field of the first entry of movement trajectory information 5334 indicated by the corresponding pointer 5333, and sets the coordinate values, size, brightness distribution, and shape of the image of the foreign object candidate in the frame image of interest in the position information 53342, size 53343, brightness distribution 53344, and shape 53345 fields.

[0085] Next, the detection unit 543 shifts its attention to the frame image that is one frame after the current frame image in section 4. Next, the detection unit 543 focuses on one of the foreign object candidate images detected in the current frame image. The detection unit 543 then compares the position of the current foreign object candidate image with the position of the foreign object candidate image detected in the frame image one frame before (hereinafter referred to as the preceding frame image), and if the foreign object candidate image is located within a predetermined distance threshold from the current foreign object candidate image, it determines that the current foreign object candidate image and the foreign object candidate image that was located within that distance threshold are images of the same foreign object candidate. In this case, the detection unit 543 assigns the tracking ID assigned to the image of the foreign object candidate determined to be identical to the current foreign object candidate image. The detection unit 543 then secures a new entry in the movement trajectory information 5334 pointed to by the pointer 5333 of the entry in the tracking information 533 to which the assigned tracking ID 5332 is set, and sets the shooting time of the frame image under consideration and the coordinate values, size, brightness distribution, and shape of the image of the foreign object candidate under consideration to the time 53341, position information 53342, size 53343, brightness distribution 53344, and shape 53345 of the secured entry.

[0086] On the other hand, if no image of a foreign substance candidate is present in the previous frame image within the distance threshold from the image of the foreign substance candidate under consideration, the detection unit 543 determines that the image of the foreign substance candidate under consideration is an image of a new foreign substance candidate and assigns a new tracking ID to it. Next, the detection unit 543 sets the tracking ID assigned to the image of the foreign substance candidate under consideration in the tracking ID 5332 field of the newly secured entry in Figure 4, sets the shooting time of the frame image under consideration in the time 53341 field of the first entry of the movement trajectory information 5334 indicated by the corresponding pointer 5333, and assigns the coordinate values, size, luminance distribution, and shape of the image of the foreign substance candidate under consideration to the position information 53342, size 53343, luminance distribution 53344, and shape 53345 fields.

[0087] When the detection unit 543 has finished processing the image of the foreign substance candidate under consideration, it shifts its attention to the image of the next foreign substance candidate detected in the frame image under consideration, and repeats the same processing as described above. Then, when the detection unit 543 has finished focusing on all foreign substance candidate images detected in the frame image under consideration, it shifts its attention to the frame image one frame later in section 4, and repeats the same processing as described above. Then, when the detection unit 543 has finished focusing on the last frame image in the image information 532 of section 4, it ends the tracking processing.

[0088] In the above description, the detection unit 543 performed tracking based on the distance between the images of the foreign substance candidate in two adjacent frame images. However, the detection unit 543 may also perform tracking based on the distance between the images of the foreign substance candidate in two adjacent frame images separated by n frames (n is a positive integer of 1 or greater). Furthermore, the detection unit 543 may also perform tracking by comprehensively determining the tracking result obtained by performing tracking based on the distance between the images of the foreign substance candidate in two adjacent frame images separated by m frames (m is a positive integer of 0 or greater) and the tracking result obtained by performing tracking based on the distance between the images of the foreign substance candidate in two adjacent frame images separated by m+j frames (j is a positive integer of 1 or greater).

[0089] Furthermore, the detection unit 543 may combine multiple fragmented trajectories that are estimated to belong to the same foreign object to generate a new trajectory. Any method, such as a method based on frame overlap and proximity between trajectories, may be used to combine multiple fragmented trajectories to generate a new trajectory. Furthermore, a technique similar to that used by the movement line combination information creation unit described in Patent Document 2 may be used.

[0090] <Bright Regions of the 90° Submerged Region 122 in Section 4> The detection unit 543 acquires image information including the camera ID 5322 of the camera device 400 and the image capture time 5323 included in Section 4 from the image information 532 shown in FIG. 3 . Next, the detection unit 543 sets bright regions of the 90° submerged region 122 for each frame image included in the acquired image information, and also sets an adaptive OR temporal difference binarization method. Next, the detection unit 543 extracts images of foreign substance candidates from each frame image in Section 4 using the adaptive OR temporal difference binarization method. Next, the detection unit 543 tracks the images of the detected foreign substance candidates in the time-series frame images included in the image information for Section 4, and generates tracking information 533 related to the bright regions of the 90° submerged region 122 in Section 4 based on the tracking results. The method for generating the tracking information 533 for the bright region of the 90° submerged region 122 in section 4 is basically the same as the method for generating the tracking information 533 for the dark region of the 90° submerged region 122 in section 4.

[0091] <Dark Region of 90-Degree Wall Surface Region 125 in Section 4> The detection unit 543 acquires image information including the camera ID 5322 of the camera device 400 and the shooting time 5323 included in Section 4 from the image information 532 shown in FIG. Next, the detection unit 543 sets a dark region of the 90-degree wall surface region 125 for each frame image included in the acquired image information, and also sets a temporal difference AND same-angle difference binarization method. Next, the detection unit 543 uses a pair of images at the same angle in Section 4 to extract images of foreign object candidates using the same-angle difference binarization method. The detection unit 543 also extracts images of foreign object candidates from the frame images in Section 4 using the temporal difference binarization method. The detection unit 543 then takes the logical product of the foreign object candidate images detected by both methods, thereby extracting images of foreign object candidates from the frame images in Section 4 using the temporal difference AND same-angle difference binarization method. Next, the detection unit 543 tracks the image of the detected foreign object candidate in the time-series frame images included in the image information of section 4 using the same method as described above, and generates tracking information 533 related to the dark area of ​​the 90° wall surface area 125 of section 4 based on the tracking results.

[0092] <Bright Region of 90-Degree Wall Surface Region 125 in Section 4> The detection unit 543 acquires image information including the camera ID 5322 of the camera device 400 and the shooting time 5323 included in Section 4 from the image information 532 shown in FIG. Next, the detection unit 543 sets a bright region of the 90-degree wall surface region 125 for each frame image included in the acquired image information, and also sets an adaptive AND temporal difference binarization method. Next, the detection unit 543 extracts images of foreign object candidates from the frame images in Section 4 using the adaptive AND temporal difference binarization method. Next, the detection unit 543 tracks the detected images of foreign object candidates in the time-series frame images included in the image information of Section 4 using the same method as above, and generates tracking information 533 related to the bright region of the 90-degree wall surface region 125 in Section 4 based on the tracking results.

[0093] <Dark Region in 0° Submerged Region 121 in Section 6> The detection unit 543 acquires image information including the camera ID 5322 of the camera device 400 and the shooting time 5323 included in Section 6 from the image information 532 shown in FIG. 3 . Next, the detection unit 543 sets a dark region in the 0° submerged region 121 for each frame image included in the acquired image information, and also sets an adaptive AND same-angle difference binarization method. Next, the detection unit 543 extracts images of foreign substance candidates from the frame images in Section 6 using the adaptive AND same-angle difference binarization method. When detecting images of foreign substance candidates from the frame images in Section 6 using the same-angle difference binarization method, the detection unit 543 uses image pairs with the same angle in Section 6. Alternatively, when detecting images of foreign substance candidates from the frame images in Section 6 using the same-angle difference binarization method, the detection unit 543 may use image pairs with the same angle in Sections 4 and 6. Next, the detection unit 543 tracks the image of the detected foreign object candidate in the time series frame images included in the image information of section 6 using the same method as described above, and generates tracking information 533 related to the dark area of ​​the 0° liquid area 121 of section 6 based on the tracking results.

[0094] <Bright Regions in the 0° Submerged Region 121 in Section 6> The detection unit 543 acquires image information including the camera ID 5322 of the camera device 400 and the image capture time 5323 included in Section 6 from the image information 532 shown in FIG. 3 . Next, the detection unit 543 sets bright regions in the 0° submerged region 121 for each frame image included in the acquired image information, and also sets an adaptive OR temporal difference binarization method. Next, the detection unit 543 extracts images of foreign substance candidates from the frame images in Section 6 using the adaptive OR temporal difference binarization method. Next, the detection unit 543 tracks the detected images of foreign substance candidates in the time-series frame images included in the image information for Section 6 using the same method as above, and generates tracking information 533 related to the bright regions in the 0° submerged region 121 in Section 6 based on the tracking results.

[0095] <Dark Region of the 0° Liquid Surface Region 123 in Section 6> The detection unit 543 acquires image information including the camera ID 5322 of the camera device 400 and the shooting time 5323 included in Section 6 from the image information 532 shown in FIG. 3 . Next, the detection unit 543 sets a dark region of the 0° liquid surface region 123 for each frame image included in the acquired image information, and also sets a temporal difference AND same-angle difference binarization method. Next, the detection unit 543 extracts images of foreign substance candidates from the frame images in Section 6 using the temporal difference AND same-angle difference binarization method. When detecting images of foreign substance candidates from the frame images in Section 6 using the same-angle difference binarization method, the detection unit 543 uses image pairs with the same angle in Section 6. Alternatively, when detecting images of foreign substance candidates from the frame images in Section 6 using the same-angle difference binarization method, the detection unit 543 may use image pairs with the same angle in Section 6 and Section 4. Next, the detection unit 543 tracks the image of the detected foreign object candidate in the time series frame images included in the image information of section 6 using the same method as described above, and generates tracking information 533 related to the dark area of ​​the 0° liquid surface area 123 of section 6 based on the tracking results.

[0096] <Bright Region of the 0° Liquid Surface Region 123 in Section 6> The detection unit 543 acquires image information including the camera ID 5322 of the camera device 400 and the shooting time 5323 included in Section 6 from the image information 532 shown in FIG. 3 . Next, the detection unit 543 sets a bright region of the 0° liquid surface region 123 for each frame image included in the acquired image information, and also sets an adaptive AND temporal difference binarization method. Next, the detection unit 543 extracts images of foreign object candidates from each frame image in Section 6 using the adaptive AND temporal difference binarization method. Next, the detection unit 543 tracks the images of the detected foreign object candidates in the time-series frame images included in the image information of Section 6 using the same method as above, and generates tracking information 533 related to the bright region of the 0° liquid surface region 123 in Section 6 based on the tracking results.

[0097] <Dark Region in 0° Wall Surface Region 124 in Section 6> The detection unit 543 acquires image information including the camera ID 5322 of the camera device 400 and the shooting time 5323 included in Section 6 from the image information 532 shown in FIG. 3 . Next, the detection unit 543 sets a dark region in the 0° wall surface region 124 for each frame image included in the acquired image information, and also sets a temporal difference binarization method. Next, the detection unit 543 extracts images of foreign object candidates from the frame images of Section 6 using the temporal difference binarization method. Next, the detection unit 543 tracks the images of the detected foreign object candidates in the time-series frame images included in the image information of Section 6 using the same method as above, and generates tracking information 533 related to the dark region in the 0° wall surface region 124 in Section 6 based on the tracking results.

[0098] <Bright Region of 0° Wall Surface Region 124 in Section 6> The detection unit 543 acquires image information including the camera ID 5322 of the camera device 400 and the shooting time 5323 included in Section 6 from the image information 532 shown in FIG. 3 . Next, the detection unit 543 sets a bright region of the 0° wall surface region 124 for each frame image included in the acquired image information, and also sets a time-dependent difference binarization method or a same-angle difference binarization method. Next, the detection unit 543 extracts images of foreign object candidates from the frame images in Section 6 using the time-dependent difference binarization method or the same-angle difference binarization method. When detecting images of foreign object candidates from the frame images in Section 6 using the same-angle difference binarization method, the detection unit 543 may use an image pair with the same angle in Section 6, or may use an image pair with the same angle in Section 4 and Section 6. Next, the detection unit 543 tracks the image of the detected foreign object candidate in the time-series frame images included in the image information of section 6 using the same method as above, and generates tracking information 533 related to the bright region of the 0° wall surface region 124 of section 6 based on the tracking results.

[0099] Next, an example of a method for determining whether a foreign object exists based on the movement trajectory of a foreign object candidate will be described.

[0100] For each image of a foreign object candidate identified by a tracking ID 5332 included in tracking information 533, detection unit 543 determines whether the image of the foreign object candidate is a foreign object or not, based on at least one of the movement trajectory identified by its movement trajectory information 5334 and the image features of the image (position information, size, brightness distribution, shape).

[0101] For example, among the images of foreign object candidates detected in the dark and light regions of the sub-inspection regions of each inspection area, the detection unit 543 determines, for example, foreign object candidates that move upward and then downward as the vial 110 rotates, foreign object candidates that move exclusively downward, and foreign object candidates that are quasi-stationary as foreign objects. Furthermore, among the images of foreign object candidates detected in the dark and light regions of the sub-inspection regions of each inspection area, the detection unit 543 determines, for example, foreign object candidates that move exclusively upward as air bubbles rather than foreign objects. Furthermore, among the images of foreign object candidates detected in the dark and light regions of the sub-inspection regions of each inspection area, the detection unit 543 may determine, for example, images having a brightness distribution or shape typical of air bubbles as air bubbles, and may determine the rest as foreign objects. Furthermore, among the images of foreign object candidates detected in the dark and light regions of the sub-inspection regions of each inspection area, the detection unit 543 may determine, for example, images having a brightness distribution or shape typical of water droplets as water droplets rather than foreign objects. Alternatively, the detection unit 543 may determine whether an object is a foreign object or a bubble by using a time-series information discrimination model, such as a machine learning model such as a neural network, based on more detailed differences in appearance and movement.

[0102] As described above, this embodiment includes an acquisition unit 542 that acquires multiple images obtained by continuously photographing a vial 110 filled with liquid, and a detection unit 543 that is a processing unit that processes the multiple acquired images. Furthermore, the detection unit 543 sets at least one inspection area in each of the areas above and below the liquid level of the multiple images based on the position of the liquid level in the vial 110. Furthermore, the detection unit 543 sets a type of binarization method for each of the multiple inspection areas, and detects images of foreign substance candidates from each of the multiple inspection areas based on the set type of binarization method. Therefore, this embodiment can easily extract images of foreign substance candidates present in areas below the liquid level of the liquid filled in the vial 110 and images of foreign substance candidates present in areas above the liquid level.

[0103] This embodiment can be modified in various ways, such as the following.

[0104] Although vials were used as the test object, bottles and containers other than vials can also be used as the test object as long as they are transparent or translucent containers filled with liquid such as drinking water.

[0105] In the above description, the inspection device 100 rotates the vial 110 360 degrees in an upright position in section 2 of the rotation angle timeline of FIG. 7 . However, because there is section 6 in which the vial 110 is rotated 720 degrees in an upright position, section 2 may be omitted. Furthermore, the inspection device 100 rotates the vial 110 720 degrees in the same direction in section 6, but it is sufficient to rotate the vial 110 at least 180 degrees. Furthermore, the inspection device 100 may rotate the vial 110 360 degrees or more in section 6 by reversing the direction twice. Furthermore, the inspection device 100 may insert sections in which the rotation stops before and after the rotation or between reversals in the rotation angle timeline illustrated in FIG. 7 . Furthermore, the inspection device 100 may rotate the vial 110 at different speeds in multiple sections. Furthermore, in order to move foreign matter that is difficult to move from the bottom surface or inner wall, the inspection device 100 may insert a section of forward rotation or reverse rotation at a high rotation speed while rotating at least 720 degrees in the same rotation direction around the rotation axis A. In addition, various rotation angle timelines may be used in the present disclosure.

[0106] In the above description, the inspection device 100 sets two sub-inspection areas in the set inspection area, but the number of sub-inspection areas set in one inspection area may be one, or three or more. Also, one inspection area itself may be used as one sub-inspection area.

[0107] The display control unit 544 may display inspection result information 534, which is an example of the foreign substance inspection result, on a terminal used by the inspector. In addition, the foreign substance inspection result may be recorded and presented by any other method.

[0108] The display control unit 544 may issue an alert to an inspector when a foreign object or a specific foreign object is detected. Furthermore, the cause of the foreign object contamination and the location of the foreign object contamination can be identified based on the nature of the foreign object. For example, in the case of a fiber fragment or a hair fragment, a person may be the cause of the foreign object contamination, and production line A may be the location of the foreign object contamination. Furthermore, in the case of a metal fragment, production line B or C may be the location of the foreign object contamination. Therefore, the display control unit 544 may issue an alert to a specific inspector in charge of the cause of the foreign object contamination and the location of the foreign object contamination. In addition, any alert may be issued based on the foreign object inspection results.

[0109] Countermeasures to prevent recurrence can be identified based on the type of foreign object. Therefore, the display control unit 544 may output countermeasures according to the type of foreign object. For example, the display control unit 544 may determine the countermeasure based on a model generated by machine learning the correspondence between foreign object inspection results and countermeasures and the subject's estimation results. Alternatively, the display control unit 544 may search for and acquire a countermeasure corresponding to the type of foreign object identified in the foreign object inspection results from a correspondence table in which the type of foreign object and the countermeasures are previously associated and recorded. Countermeasures to prevent recurrence may include, but are not limited to, the following: (1) Remove the foreign object. For example, in the case of a fiber fragment, cleaning of production line A is performed, and in the case of a metal fragment, loosening of bolts on production line B is performed. (2) Share information. For example, in the case of a fiber fragment, only the line manager is notified. On the other hand, in the case of a metal fragment, the line manager and the factory manager are contacted and an incident report is created.

[0110] As described above, issuing an alert or presenting a solution based on the foreign substance inspection results can optimize the inspector's actions or support the inspector's decision-making based on the foreign substance inspection results.

[0111] Second Embodiment Next, a second embodiment of the present disclosure will be described with reference to Fig. 14. Fig. 14 is a block diagram showing the configuration of an inspection device according to this embodiment. This embodiment will provide an overview of the inspection device according to the present disclosure.

[0112] As shown in FIG. 14, the inspection device 1 in this embodiment includes an acquisition unit 2 and a processing unit 3.

[0113] The acquisition unit 2 is configured to acquire a plurality of images obtained by successively photographing a container filled with a liquid. The acquisition unit 2 can be configured similarly to, for example, the acquisition unit 542 in FIG. 2, but is not limited thereto.

[0114] The processing unit 3 is configured to process the multiple images acquired by the acquisition unit 2. Specifically, the processing unit 3 is configured to set at least one inspection area in each of the areas above the liquid level and below the liquid level of the multiple images based on the position of the liquid level in the container. The processing unit 3 is also configured to set a type of binarization method for each of the multiple set inspection areas. The processing unit 3 is also configured to detect images of foreign substance candidates from each of the multiple inspection areas based on the set type of binarization method. The processing unit 3 can be configured similarly to the detection unit 543 in FIG. 2, for example, but is not limited to this.

[0115] The inspection device 1 configured as described above operates as follows. First, the acquisition unit 2 acquires multiple images obtained by successively photographing a container filled with liquid. Next, the processing unit 3 sets at least one inspection area in each of the areas above and below the liquid surface of the multiple images based on the position of the liquid surface in the container. Next, the processing unit 3 sets a type of binarization method for each of the multiple inspection areas set above. Next, the processing unit 3 detects images of foreign substance candidates from each of the multiple inspection areas based on the set type of binarization method.

[0116] With the inspection device 1 configured and operating as described above, it is possible to easily extract images of foreign substance candidates present in the region below the liquid surface of the container and images of foreign substance candidates present in the region above the liquid surface. This is because the images of foreign substance candidates present in the region below the liquid surface of the container and images of foreign substance candidates present in the region above the liquid surface are detected based on different types of binarization methods set for each region.

[0117] Although the present disclosure has been described above with reference to the above-described embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. For example, instead of the above-described CPU (Central Processing Unit), the information processing device can use a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an MPU (Micro Processing Unit), an FPU (Floating Number Processing Unit), a PPU (Physics Processing Unit), a TPU (Tensor Processing Unit), a quantum processor, a microcontroller, or a combination thereof.

[0118] The present disclosure can be used in the field of inspecting containers such as vials filled with liquid.

[0119] Some or all of the above embodiments can be described as, but are not limited to, the following supplementary notes. [Supplementary Note 1] An inspection apparatus comprising: an acquisition means for acquiring a plurality of images obtained by successively photographing a container filled with a liquid; and a processing means for processing the acquired plurality of images, wherein the processing means is configured to set at least one inspection area in each of an area above the liquid level and an area below the liquid level in the plurality of images based on the position of the liquid level in the container, set a type of binarization method for each of the set plurality of inspection areas, and detect an image of a foreign substance candidate from each of the plurality of inspection areas based on the set type of binarization method. [Supplementary Note 2] The inspection apparatus according to Supplementary Note 1, wherein the processing means is further configured to set a plurality of sub-inspection areas in at least one of the plurality of inspection areas based on the brightness of the image of the inspection area, set a type of binarization method for each of the set plurality of sub-inspection areas, and detect an image of a foreign substance candidate from each of the plurality of sub-inspection areas based on the set type of binarization method. [Supplementary Note 3] The inspection device according to Supplementary Note 3, wherein the processing means tracks the image of the foreign substance candidate among the plurality of images and identifies whether the image of the foreign substance candidate is a foreign substance based on at least one of image features and a movement trajectory of the image of the foreign substance candidate moving within the region above the liquid surface. [Supplementary Note 4] The inspection device according to Supplementary Note 3, wherein the processing means combines a plurality of fragmented movement trajectories presumed to be those of the same foreign substance to generate a new movement trajectory. [Supplementary Note 5] The inspection device according to Supplementary Note 3, wherein the processing means further comprises a display control unit that outputs foreign substance inspection results. [Supplementary Note 6] The inspection device according to Supplementary Note 5, wherein the processing means further outputs a countermeasure to assist an inspector in making a decision based on the foreign substance inspection results. [Supplementary Note 7] The inspection device according to Supplementary Note 1, wherein the processing means selects the binarization method of the type to be set from a plurality of types of binarization methods including at least a binarization method that takes a logical product (AND) of an image of a foreign substance candidate extracted by a time-dependent difference binarization method and an image of a foreign substance candidate extracted by a same-angle difference binarization method.[Supplementary Note 8] The inspection device of Supplementary Note 1, wherein the processing means selects the binarization method of the type to be set from a plurality of types of binarization methods including at least a binarization method that takes a logical sum (OR) of images of foreign substance candidates extracted by an adaptive binarization method and images of foreign substance candidates extracted by a temporal difference binarization method. [Supplementary Note 9] The inspection device of Supplementary Note 1, wherein the processing means selects the binarization method of the type to be set from a plurality of types of binarization methods including at least a binarization method that takes a logical product (AND) of images of foreign substance candidates extracted by the adaptive binarization method and images of foreign substance candidates extracted by a same-angle difference binarization method. [Supplementary Note 10] The inspection device of Supplementary Note 1, wherein the processing means selects the binarization method of the type to be set from a plurality of types of binarization methods including at least a binarization method that takes a logical product (AND) of images of foreign substance candidates extracted by the adaptive binarization method and images of foreign substance candidates extracted by a temporal difference binarization method. [Supplementary Note 11] The inspection device described in Supplementary Note 1, wherein the plurality of images include a plurality of first images obtained by successively photographing the container rotating about its upright central axis in a sideways position from a viewpoint substantially perpendicular to the upright central axis and substantially parallel to the liquid surface in the container, and the processing means sets at least one first inspection area in an area above the liquid surface in the plurality of first images. [Supplementary Note 12] The inspection device described in Supplementary Note 11, wherein the processing means sets, in the first inspection area, at least one of a first sub-inspection area consisting of pixels having a brightness below a predetermined threshold and a second sub-inspection area consisting of pixels having a brightness equal to or greater than a threshold that is smaller than the predetermined threshold. [Supplementary Note 13] The inspection device described in Supplementary Note 12, wherein the processing means sets, in the first sub-inspection area, a binarization method that takes a logical product (AND) of an image of a foreign substance candidate extracted by a time-dependent difference binarization method and an image of a foreign substance candidate extracted by a same-angle difference binarization method. [Supplementary Note 14] The inspection device according to Supplementary Note 12, wherein the processing means sets a binarization method for extracting images of foreign substance candidates from the image to be binarized by taking a logical product (AND) of images of foreign substance candidates extracted by an adaptive binarization method and images of foreign substance candidates extracted by a time-dependent difference binarization method in the second sub-inspection area.[Supplementary Note 15] The inspection device described in Supplementary Note 11, wherein the processing means tracks the image of the foreign substance candidate in the plurality of first images and identifies whether the image of the foreign substance candidate is a foreign substance or not based on at least one of image features and a movement trajectory of the image of the foreign substance candidate moving within the first inspection area. [Supplementary Note 16] The inspection device described in Supplementary Note 1, wherein the plurality of images include a plurality of first images obtained by successively photographing the container in an upright position and rotating around a central axis of upright position from a viewpoint that is approximately perpendicular to the central axis of upright position and approximately horizontal to the surface of the liquid in the container, and the processing means sets at least one first inspection area in an area above the surface of the liquid in the first image. [Supplementary Note 17] The inspection device described in Supplementary Note 16, wherein the processing means sets at least one of a first sub-inspection area consisting of pixels having a brightness below a predetermined threshold and a second sub-inspection area consisting of pixels having a brightness above the predetermined threshold, within the first inspection area. [Supplementary Note 18] The inspection device according to Supplementary Note 17, wherein the processing means sets a temporal difference binarization method for the first sub-inspection area. [Supplementary Note 19] The inspection device according to Supplementary Note 17, wherein the processing means sets a temporal difference binarization method or an angular difference binarization method for the second sub-inspection area. [Supplementary Note 20] The inspection device according to Supplementary Note 16, wherein the processing means tracks an image of the foreign substance candidate in the plurality of first images and identifies whether the image of the foreign substance candidate is a foreign substance or not based on at least one of image features and a movement trajectory of the image of the foreign substance candidate moving within the first inspection area. [Supplementary Note 21] An inspection method, comprising: a computer acquiring a plurality of images obtained by successively photographing a container filled with a liquid; processing the acquired plurality of images; and in processing the plurality of images, setting at least one inspection area in each of an area above the liquid level and an area below the liquid level in the plurality of images based on the position of the liquid level in the container; setting a type of binarization method for each of the set plurality of inspection areas; and detecting an image of a foreign object candidate from each of the set plurality of inspection areas based on the set type of binarization method.[Supplementary Note 22] A computer-readable recording medium having recorded thereon a program that causes a computer to perform a process of acquiring a plurality of images obtained by successively photographing a container filled with a liquid, and a process of processing the acquired plurality of images, wherein in the processing of the plurality of images, based on the position of the liquid level of the liquid in the container, at least one inspection area is set in each of an area above the liquid level and an area below the liquid level in the plurality of images, a type of binarization method is set for each of the set plurality of inspection areas, and an image of a foreign substance candidate is detected from each of the plurality of inspection areas based on the set type of binarization method.

[0120] REFERENCE SIGNS LIST 1 Inspection device 2 Acquisition unit 3 Processing unit 100 Inspection device 110 Vial 200 Gripping and rotation device 201 Flat plate-like member 202 Upper arm 203 Lower arm 204 Lower gripping unit 205 Plate 206 Chuck mechanism 207 Motor 208 Rotating shaft 209, 210 Rotation angle detector 211 Chuck finger 300 Lighting device 400 Camera device 500 Information processing device 510 Communication I / F unit 520 Operation input unit 530 Memory unit 531 Program 532 Image information 533 Tracking information 534 Inspection result information 540 Arithmetic processing unit 541 Gripping and rotation control unit 542 Acquisition unit 543 Detection unit 544 Display control unit 600 Display device

Claims

1. An inspection apparatus comprising: an acquisition means for acquiring a plurality of images obtained by continuously photographing a container filled with a liquid; and a processing means for processing the acquired plurality of images, wherein the processing means sets at least one inspection region in each of the regions above and below the liquid level in the plurality of images based on the position of the liquid level of the liquid in the container, sets the type of binarization method for each of the set plurality of inspection regions, and is configured to detect an image of a foreign object candidate from each of the set plurality of inspection regions based on the set type of binarization method.

2. The inspection apparatus according to claim 1, wherein the processing means further sets a plurality of sub-inspection regions in at least one inspection region of the plurality of inspection regions based on the luminance of the image of the inspection region, sets the type of binarization method for each of the set plurality of sub-inspection regions, and is configured to detect an image of a foreign object candidate from each of the set plurality of sub-inspection regions based on the set type of binarization method.

3. The inspection apparatus according to claim 1, wherein the processing means tracks the image of the foreign object candidate among the plurality of images, and identifies whether the image of the foreign object candidate is a foreign object based on at least one of the image features and the movement trajectory of the image of the foreign object candidate moving within the region above the liquid level.

4. The inspection apparatus according to claim 3, wherein the processing means combines fragmented plurality of movement trajectories estimated to be of the same foreign object to generate a new movement trajectory.

5. The inspection apparatus according to claim 3, further comprising a display control unit that outputs an inspection result of a foreign object.

6. The inspection apparatus according to claim 5, wherein the processing means further outputs a countermeasure method for assisting the decision-making of an inspector based on the inspection result of a foreign object.

7. The inspection apparatus according to claim 1, wherein the processing means selects the type of binarization method to be set from among a plurality of types of binarization methods including at least a binarization method that takes a logical product (AND) of an image of a foreign object candidate extracted by a temporal difference binarization method and an image of a foreign object candidate extracted by a same-angle difference binarization method.

8. The processing means selects the type of binarization method to be set from among a plurality of types of binarization methods including at least a binarization method that takes a logical sum (OR) of an image of a foreign object candidate extracted by an adaptive binarization method and an image of a foreign object candidate extracted by a temporal difference binarization method. The inspection apparatus according to claim 1.

9. The processing means selects the type of binarization method to be set from among a plurality of types of binarization methods including at least a binarization method that takes a logical product (AND) of an image of a foreign object candidate extracted by an adaptive binarization method and an image of a foreign object candidate extracted by a same-angle difference binarization method. The inspection apparatus according to claim 1.

10. The processing means selects the type of binarization method to be set from among a plurality of types of binarization methods including at least a binarization method that takes a logical product (AND) of an image of a foreign object candidate extracted by an adaptive binarization method and an image of a foreign object candidate extracted by a temporal difference binarization method. The inspection apparatus according to claim 1.

11. The plurality of images include a plurality of first images obtained by continuously photographing the container rotating about a vertical central axis in a horizontal-lying posture from a viewpoint in a direction substantially perpendicular to the vertical central axis and substantially horizontal to the liquid level in the container. The processing means sets at least one first inspection area in an area above the liquid level of the liquid in the plurality of first images. The inspection apparatus according to claim 1.

12. The processing means sets at least one of a first sub-inspection area composed of pixels having a luminance less than a predetermined threshold value and a second sub-inspection area composed of pixels having a luminance greater than or equal to a threshold value smaller than the predetermined threshold value in the first inspection area. The inspection apparatus according to claim 11.

13. The processing means sets a binarization method that takes a logical product (AND) of an image of a foreign object candidate extracted by a temporal difference binarization method and an image of a foreign object candidate extracted by a same-angle difference binarization method in the first sub-inspection area. The inspection apparatus according to claim 12.

14. The processing means sets a binarization method for extracting an image of a foreign object candidate from an image to be binarized by taking a logical product (AND) of an image of a foreign object candidate extracted by an adaptive binarization method and an image of a foreign object candidate extracted by a temporal difference binarization method in the second sub-inspection area. The inspection apparatus according to claim 12.

15. The inspection apparatus according to claim 11, wherein the processing means tracks an image of the foreign object candidate among the plurality of first images, and based on at least one of an image feature and a movement trajectory of the image of the foreign object candidate moving within the first inspection area, determines whether the image of the foreign object candidate is a foreign object.

16. The plurality of images include a plurality of first images obtained by continuously photographing the container rotating about an upright central axis in an upright posture from a viewpoint in a direction substantially perpendicular to the upright central axis and substantially horizontal to the liquid level in the container. The processing means sets at least one first inspection area in an area above the liquid level of the liquid in the first image. The inspection apparatus according to claim 1.

17. The inspection apparatus according to claim 16, wherein the processing means sets at least one of a first sub-inspection area composed of pixels having a luminance less than a predetermined threshold value and a second sub-inspection area composed of pixels having a luminance greater than or equal to the predetermined threshold value in the first inspection area.

18. The inspection apparatus according to claim 17, wherein the processing means sets a temporal difference binarization method in the first sub-inspection area.

19. The inspection apparatus according to claim 17, wherein the processing means sets a temporal difference binarization method or an angular difference binarization method in the second sub-inspection area.

20. The inspection apparatus according to claim 16, wherein the processing means tracks an image of the foreign object candidate among the plurality of first images, and based on at least one of an image feature and a movement trajectory of the image of the foreign object candidate moving within the first inspection area, determines whether the image of the foreign object candidate is a foreign object.

21. A computer acquires a plurality of images obtained by continuously photographing a container filled with liquid, processes the acquired plurality of images, and in the processing of the plurality of images, based on the position of the liquid level of the liquid in the container, sets at least one inspection area in each of the area above the liquid level and the area below the liquid level in the plurality of images, sets the type of binarization method for each of the set plurality of inspection areas, and detects an image of a foreign object candidate from each of the set plurality of inspection areas based on the set type of binarization method. Inspection method.

22. A computer-readable recording medium recording a program that causes a computer to perform a process of acquiring a plurality of images obtained by continuously photographing a container filled with a liquid, and a process of processing the acquired plurality of images. In the process of processing the plurality of images, at least one inspection region is set in each of the region above the liquid surface and the region below the liquid surface in the plurality of images based on the position of the liquid surface of the liquid in the container, the type of binarization method is set for each of the set plurality of inspection regions, and an image of a foreign matter candidate is detected from each of the set plurality of inspection regions based on the set type of binarization method.

Images