Foreign object detection device and foreign object detection system

Abstract

To provide a foreign matter detection and sorting device that detects and sorts articles containing foreign matter. [Solution] The system includes an inverted truncated cone shaped chute (51) that allows items discharged from a container to fall towards the center, a passage (70) located at the bottom of the chute and allowing items discharged from the chute to fall, a camera (80) that photographs the freely falling items, a foreign object detection device (90) that inputs the image taken by the camera and determines whether the image contains foreign objects other than the item, and a sorting device (100) located below the passage and removes items that contain foreign objects, a lower chute is located in the passage, and the bottom end of the chute and the top end of the lower chute are arranged spaced apart in the vertical direction, and the items are configured to fall freely between the chute and the lower chute, and multiple camera devices are located on the side between the bottom end of the chute and the top end of the lower chute to photograph the freely falling items from different directions.

Description

Technical Field

[0001] The present invention relates to a foreign object detection and sorting device.

Background Art

[0002] Regarding devices for detecting and removing foreign objects contained in articles, the following inventions have been filed. Patent Document 1 (Japanese Patent Laid-Open No. 63-148130) discloses a combination weighing and packaging device that solves the above problems by being able to distinguish a packaged article containing metal from a packaged article not containing metal when metal is mixed in the packaged article sent out from a packaging device.

[0003] The combination weighing and packaging device described in Patent Document 1 combines the measured values of articles respectively accommodated in a plurality of containers in various ways, selects a combination whose total value conforms to the conditions from these combinations, and discharges the articles constituting the selected combination from the above containers accommodating these articles by free fall. A combination scale, a packaging device arranged below this combination scale for packaging each article freely falling from the above combination scale into one packaged article, a passage provided between the above combination scale and the above packaging device for collecting the articles freely falling from the above combination scale and sending them to the above packaging device, and a metal detector provided in this passage for generating a metal detection signal when metal passes through this passage. The packaging device is characterized in that it is controlled based on the above metal detection signal so that the form of the packaged article containing metal sent out from the above packaging device is different from the form of the packaged article not containing metal.

[0004] Patent Document 2 (Japanese Patent Laid-Open No. 2002-312762) discloses a grain sorting device using a neural network for solving problems such as the sorting rate of rice changing depending on the amount of rice handled in a color sorter for rice sorting used in a rice mill or the like.

[0005] In the grain sorting device described in Patent Document 2, templates are prepared for each type of rice, and a neural network is used as the system configuration method, employing a neurotemplate matching sorting technique that enables high-performance sorting even for ambiguous images. Furthermore, grains falling in an unbroken state are captured by a camera, a set of image frames is constructed, and multiple rice images are extracted from these constructed images, one grain at a time, and these are used as the target of the sorting system.

[0006] Patent Document 3 (Japanese Patent Publication No. 2021-42955) discloses a food inspection device that can improve the robustness of food inspections to confirm the absence of foreign matter.

[0007] The food inspection apparatus described in Patent Document 3 comprises a light irradiation means for irradiating light onto the object to be inspected, an imaging means for capturing an image of the object to be inspected, and an identification processing device. The identification processing device is equipped with a food reconstruction neural network that has been pre-trained to identify only good food products. The food reconstruction neural network is used to reconstruct a hypothetical food image from the image obtained by the imaging means, and the difference between the image obtained by the imaging means and the hypothetical food image is calculated. Objects A whose difference exceeds a pre-set threshold are identified as foreign objects.

[0008] Patent Document 4 (Reprint No. 2018 / 142988) discloses a weighing and packaging system that, when a foreign object is mixed in with the Nth item (where N is an integer of 2 or more) discharged from the weighing unit, and the foreign object falls faster than the item, can eliminate the possibility of the foreign object entering the item that is discharged from the weighing unit as the (N-1)th item.

[0009] The weighing and packaging system described in Patent Document 4 includes a weighing unit that weighs and sequentially discharges articles, a discharge chute located below the weighing unit through which articles discharged from the weighing unit pass, a foreign object detection unit that detects foreign objects contained in articles passing through the discharge chute, a storage unit that stores quality information regarding the quality of articles based on the detection results of the foreign object detection unit, and a control unit that controls the storage unit. If the foreign object detection unit detects a foreign object in the Nth (N is an integer of 2 or more) article discharged from the weighing unit, the control unit forcibly stores the quality information of the N-1th article discharged from the weighing unit as "not" in the storage unit. [Prior art documents] [Patent Documents]

[0010] [Patent Document 1] Japanese Patent Application Laid-Open No. 63-148130 [Patent Document 2] Japanese Patent Publication No. 2002-312762 [Patent Document 3] Japanese Patent Publication No. 2021-42955 [Patent Document 4] Re-tabled publication 2018 / 142988 [Overview of the Initiative] [Problems that the invention aims to solve]

[0011] As a tool for automating and saving labor in production sites, combination weighing machines that package items of varying sizes and shapes in fixed quantities are well known. However, when packaging items such as food using combination weighing machines, there is a risk that the packaged items may contain foreign objects such as pebbles, insects, and / or spoiled food. Therefore, it is necessary to inspect the items for foreign objects before putting them through the combination weighing machine, or to inspect the packaged items for foreign objects after packaging. In this case, foreign object inspection and combined weighing are required separately, and even if a combined weighing machine is used, it does not lead to sufficient automation and labor saving in the production site.

[0012] A weighing and packaging device is known that packages multiple items into packages of a predetermined weight and removes packages containing foreign matter. For example, Patent Document 1 discloses a weighing and packaging device in which a packaging device is provided below a combination scale that combines and discharges items to a certain weight, and a metal detector is provided in the passage between the combination scale and the packaging device that generates a metal detection signal when metal passes through, thereby removing packaged products containing metal.

[0013] However, while the weighing and packaging device described in Patent Document 1 can remove packaged goods that contain metal, it cannot remove packaged goods that contain foreign matter such as pebbles, insects, and / or spoiled parts of food.

[0014] In the grain sorting device described in Patent Document 2, rice grains falling from a chute are photographed by a line sensor, and individual grain images are extracted from the image data of a large amount of rice. A neural network is used to sort good rice from damaged rice, and damaged rice is removed by applying pneumatic pressure to that part based on its location information.

[0015] In this case, since good rice and damaged rice are sorted using a neural network from image data, it is possible to sort foreign objects other than metal as foreign objects. However, when photographing items (rice grains) that have fallen from the chute, there is a problem that accurate sorting is not possible if the items overlap in the captured image. The grain sorting device described in Patent Document 2 also describes that the center position of the image of an item (grain) that has been cut out and separated one grain at a time is determined, and when multiple items make up an overlapping image, the edge portion is contracted toward the center to separate and cut off from adjacent items, thereby accurately obtaining image data of the items around the center position. The edge portion of the image of the item is contracted toward the center to forcibly separate it from adjacent items (paragraph

[0055] of the specification). However, for example, if the image of the overlapping part of the item contains a rotten part, that item cannot be removed as a foreign object.

[0016] In the food inspection device described in Patent Document 3, food is placed on a conveying means such as a belt conveyor and then imaged by an imaging means. Therefore, if the items are placed on the conveying means without overlapping, the items will not overlap in the captured image. However, if workers are positioned to avoid overlapping, labor saving in the production site cannot be achieved. Furthermore, when this food inspection device is combined with a weighing machine, there is a problem that the device becomes large and complex.

[0017] The weighing and packaging system described in Patent Document 4 is characterized by determining that an item previously discharged from the weighing unit contains a foreign object (metal) if the foreign object falls faster than the item, as the foreign object may enter the item that was previously discharged from the weighing unit. However, the foreign object detection unit of the weighing and packaging system described in Patent Document 4 is a metal detection device and cannot remove packaged goods that contain foreign objects such as pebbles, insects, and / or spoiled parts of food.

[0018] The main objective of the present invention is to provide an automatic weighing, foreign object detection, and packaging device that can be easily miniaturized, has a simple structure, and can be used to package multiple articles that may contain metal and non-metallic foreign objects into packages of a predetermined weight, and to remove packages containing foreign objects. [Means for solving the problem]

[0019] (1) The weighing and foreign object detection packaging device follows a single plane and packages multiple articles into a package of a predetermined weight, while also removing packages containing foreign objects. The device includes a combination scale that combines the weighing values ​​of articles contained in multiple containers in various ways, selects a combination whose total weight is predetermined, and opens the outlets of the containers constituting the selected combination to discharge the articles; a packaging device positioned below the combination scale that packages the articles discharged from the combination scale into a single package; an inverted truncated cone-shaped chute that causes the articles discharged from each container to fall toward the center; a passage provided at the bottom of the chute that sends the articles discharged from the chute to the packaging device; a photographing device provided at the top of the chute and / or on the side between the chute and the passage that photographs the falling articles; a foreign object detection device that inputs the image captured by the photographing device and determines whether or not the image contains foreign objects other than articles; and a sorting device provided after the packaging device that removes packages containing foreign objects. The combination scale sequentially opens the outlets of the containers constituting the selected combination at predetermined time intervals.

[0020] To realize an automated weighing, foreign object detection, and packaging device with a simple structure and miniaturization, it is desirable to place the packaging device below the combination scale and the foreign object inspection device between the combination scale and the packaging device. However, while a metal detector can be used as a foreign object detection device when the foreign object is metal, a metal detector cannot be used as a foreign object detection device when the foreign object is a pebble, insect, and / or food containing spoiled parts. In this case, one possible approach is to photograph the items as they pass (fall) from the combination scale to the packaging device using a camera, and then use the captured images to determine whether or not foreign objects are present. However, if multiple items overlap in the captured image, it is difficult to accurately determine whether or not they are foreign objects.

[0021] On the one hand, a combination scale usually includes about 10 to 20 containers (weighing hoppers), and selects a combination of containers whose total weight of the articles becomes a predetermined weight, and opens the outlets of those containers. In this case, usually the outlets of the selected containers are opened simultaneously, but in a weighing foreign object detection packaging device according to a certain aspect, the outlets of the containers constituting the selected combination are sequentially opened at a predetermined time interval, thereby restricting the number of articles passing through the shooter and the passage at one time, and preventing the articles from overlapping in the image taken by the imaging device.

[0022] Also, when detecting foreign objects in the articles, how to photograph the articles is also important. As a first method, it is conceivable to arrange an imaging device in a direction facing the inclined surface of the shooter and photograph the state in which the articles discharged from each container roll or slide down toward the hole at the lower center of the shooter. In this case, since the articles on the shooter move in a scattered state, the overlap of the articles in the taken image can be minimized. Also, since the distance between the hole at the lower part of the shooter and the passage can be shortened, the height of the weighing foreign object detection packaging device can be kept low. However, since a plurality of containers are arranged and the position of the shooter where the articles move also varies depending on each container, a plurality of imaging devices (for example, 4 units) are required.

[0023] As a second method, it is conceivable to arrange an imaging device on the side surface between the shooter and the passage and photograph the articles freely falling from the hole at the lower part of the shooter toward the passage. In this case, even if a plurality of articles fall, the overlap of the articles in the taken image can be minimized. In particular, if a plurality of imaging devices are arranged on the side surface and the articles are photographed from different directions, the possibility of obtaining an image without overlap of the articles is increased. Also, in the case of free fall, by taking a plurality of images at a certain time interval and calculating the falling speed, for example, when a metal nail or the like is mixed as a foreign object, it is possible to easily determine the articles and the foreign object.

[0024] A third method, similar to the first method, involves positioning the imaging device at the top of the chute, but with a wide-angle lens positioned vertically downwards at the center of the top of the chute, allowing a single imaging device to capture the items discharged from each container. However, in this case, since the image of the item is contained within a narrow area of ​​the entire captured image, it becomes necessary to increase the resolution of the imaging device and to extract the portion of the image in which the item is captured.

[0025] Furthermore, to improve the accuracy of distinguishing between items and foreign objects, it is desirable to place imaging devices on both the top of the chute and the side between the chute and the passage. In this case, the first or third method described above is used in combination with the second method. By determining the presence or absence of foreign objects in the image taken from the top of the chute and the image taken from the side between the chute and the passage, and if a foreign object is determined to be present in either image, the final determination result is set to "foreign object present," thereby further improving the accuracy of the determination.

[0026] (2) The weighing and foreign object detection packaging device according to the second invention is a weighing and foreign object detection packaging device that follows one aspect, wherein the imaging device may photograph the article from multiple different directions.

[0027] In this case, even if two objects overlap in an image taken from one direction, the overlapping objects will be separated in an image taken from another direction, allowing for more reliable detection of the foreign object. This method is particularly effective when photographing objects from the side. In this case, it is desirable that the angle between the shooting directions of adjacent imaging devices be approximately the same. Specifically, for example, if there are two imaging devices, it is desirable that the shooting directions be opposite to each other, and if there are three devices, it is desirable that the angle between the shooting directions be 120 degrees.

[0028] (3) The weighing foreign object detection packaging device according to the third invention, in one aspect or in the weighing foreign object detection packaging device according to the second invention, may determine whether or not a foreign object other than the article is contained in each of the multiple images that have been captured, and if a foreign object is contained in at least one of the multiple images, it may be determined that a foreign object is contained.

[0029] For example, when photographing an object from the side, if the object is photographed from multiple different directions, and a foreign object is not detected in the image taken from one direction, but is detected in the image taken from another direction, it is likely that the foreign object was hidden behind the object in the image where it was not detected. Furthermore, if imaging devices are placed both on the top of the chute and on the side between the chute and the passage, and images of the item are taken with each imaging device, and a foreign object is not detected in one image but is detected in the other, it is likely that the foreign object was hidden behind the item in the image where it was not detected. In these cases, the accuracy of foreign object detection can be improved by determining that a foreign object is present if at least one of the multiple images contains one.

[0030] (4) The fourth invention relates to a weighing and foreign object detection packaging device, which in one aspect relates to the third invention, wherein the foreign object detection device is composed of a multilayer neural network and may be pre-trained using images of articles and / or images of foreign objects.

[0031] If the shape or color of a foreign object is consistent, it is possible to determine whether it is an object or a foreign object by setting a threshold. However, in many cases, foreign objects found in items such as food are diverse, including pebbles, insects, or food containing spoiled parts. A foreign object detection device composed of a multilayer neural network can reliably determine whether a diverse range of foreign objects are foreign objects by training it in advance using images of both objects and foreign objects. It is also possible to train the multilayer neural network using only objects or only foreign objects as training data to determine whether something is an object or a foreign object. The multilayer neural network used in foreign object inspection devices should be capable of detecting the class (type) and bounding box (region) of each item and foreign object from an image containing multiple items and foreign objects. For example, Mask R-CNN or the YOLO system (you only look once) can be used. Furthermore, multilayer neural networks such as the YOLO system are preferable because they can identify the class (type) and bounding box (region) of each item and foreign object, even when two items and foreign objects overlap in an image.

[0032] (5) The fifth invention relates to a weighing and foreign object detection packaging device, which in one aspect relates to the fourth invention, wherein the article may be configured to freely fall through a passage from the chute to the packaging device.

[0033] In this case, the time it takes for the goods to travel from the chute through the aisle and fall into the packaging device is shorter, allowing for faster packaging of the items. [Brief explanation of the drawing]

[0034] [Figure 1] This is a schematic diagram showing the overall configuration of the weighing, foreign object detection, and packaging device according to the first embodiment. [Figure 2] This is a schematic diagram showing the configuration of a combination scale. [Figure 3]Figures 3(a), 3(b), and 3(c) are schematic diagrams showing three different variations of the imaging device. [Figure 4] This is a schematic block diagram showing the circuit configuration of a weighing, foreign object detection, and packaging device. [Figure 5] This is a schematic diagram showing the result of adding class (type) and bounding box (region) information to an image captured by a camera using a foreign object detection device. [Figure 6] This is a schematic diagram showing the result of adding a class and bounding box using a foreign object detection device in an image where an object and a foreign object partially overlap. [Figure 7] Figures 7(a) and 7(b) are schematic diagrams showing the results of adding class and bounding box information using a foreign object detection device to two images taken at regular time intervals. [Figure 8] This flowchart shows the entire process from weighing an item to shipping it using a weighing and foreign object detection packaging device. [Modes for carrying out the invention]

[0035] Embodiments of the present invention will be described below with reference to the drawings. In the following description, the same parts will be denoted by the same reference numerals. Furthermore, in the case of the same reference numerals, their names and functions are also the same. Therefore, detailed descriptions of them will not be repeated.

[0036] [First Embodiment] Figure 1 is a schematic diagram showing the overall configuration of the weighing, foreign object detection, and packaging device 200 according to the first embodiment. The weighing, foreign object detection, and packaging device 200 comprises a combination scale 50, a chute 51, a packaging device 60, a passage 70 provided between the chute 51 and the packaging device 60, a camera 80, a foreign object detection device 90, and a sorting device 100.

[0037] (Combination scale 50) Figure 2 is a schematic diagram showing the configuration of the combination scale 50. In Figure 2, the combination scale 50 is equipped with multiple (10 in Figure 2) containers 40, and, upon instruction from the control system, measures the weight of one or more items 10 (and foreign objects 20) contained inside the containers 40, and opens the outlet (not shown) of the containers 40 to drop the items 10 (and foreign objects 20). The operation of the combination scale 50 is as follows: First, the weight of the items 10 (and foreign objects 20) contained in each container 40 is measured. Next, a combination of items 10 (and foreign objects 20) from each container 40 is selected such that the sum of the measured values ​​equals a predetermined target weight. If no combination exactly equals the predetermined weight, the combination that exceeds the predetermined weight but is closest to it is selected. After a combination of articles 10 (and foreign matter 20) is selected, the outlet of each selected container 40 is opened and the articles 10 (and foreign matter 20) fall into the passage 70. In this embodiment, however, the outlets of the containers 40 constituting the selected combination are opened sequentially at predetermined time intervals, and the articles 10 (and foreign matter 20) discharged from the containers 40 roll down or slide down the inverted truncated cone-shaped chute 51 towards the hole at the bottom center. This predetermined time interval is set so that the articles 10 (and foreign objects 20) falling from the container 40 whose outlet is open do not overlap in the passage with the articles 10 (and foreign objects 20) falling from the container 40 whose outlet is next opened. This predetermined time interval varies depending on the type of articles 10 (and foreign objects 20), but can be selected from 0.02 seconds to 3.0 seconds, for example, about 1.0 second.

[0038] (Imaging device 80) In Figure 1, the imaging device 80 is positioned opposite the inclined surface of the chute 51 and films the state in which the article 10 rolls down or slides down toward the hole at the bottom center of the chute 51. In this invention, since there are multiple containers 40, multiple imaging devices 80 are also required. However, it is not the case that an imaging device 80 is required for each container 40; for example, four devices can be used to film the article 10 discharged from each container 40. In this case, since the items 10 on the chute 51 move independently of each other, the overlap of the items 10 in the captured image 81 can be minimized. Also, (since there is no need to place the imaging device 80a on the side between the chute 51 and the passage 70) the vertical distance between the hole at the bottom of the chute 51 and the passage 70 can be shortened, so the height of the weighing foreign object detection packaging device 200 can be kept low. The imaging device 80 photographs the item 10 at the moment when the item 10 reaches the inclined surface of the chute 51 facing the imaging device 80, after a certain period of time has elapsed since the container 40 discharged the item 10. Alternatively, the imaging device 80 may capture a movie or a series of images 81, and the foreign object detection device 90 may select an image 81 containing the item 10 (and foreign object 20) from among these images 81 and input the selected image 81 into a multilayer neural network 91 for foreign object detection (see Figure 4).

[0039] (Modified image of the camera device 80) Figures 3(a), (b), and (c) show modified examples of the imaging device 80. Figure 3(a) shows that in addition to the imaging device 80 positioned opposite the inclined surface of the chute 51, there is also an imaging device 80a positioned on the side between the chute 51 and the passage 70. The imaging device 80a photographs the article 10 as it free-falls from the hole at the bottom of the chute 51 toward the passage 70. In this case, even if multiple articles 10 fall, the overlap of the articles 10 in the captured image 81 can be minimized. In particular, by positioning multiple imaging devices 80a on the side and photographing the article 10 from different directions, the likelihood of obtaining an image 81 without overlapping articles 10 increases. Furthermore, in free fall, by taking multiple images 81 at regular time intervals and calculating the falling velocity, it is possible to easily distinguish between the item 10 and the foreign object 20, for example, if a metal nail or similar object is mixed in. Furthermore, by determining the presence or absence of foreign matter 20 in both the image 81 captured by the imaging device 80 and the image 81 captured by the imaging device 80a, and determining that foreign matter 20 is present if foreign matter 20 is detected in either the image 81 captured by the imaging device 80 or the image 81 captured by the imaging device 80a, the accuracy of determining the presence of foreign matter 20 can be further improved. However, in order to photograph the item 10 as it free-falls from the hole at the bottom of the chute 51 toward the passage 70 using the imaging device 80a, it is necessary to maintain a certain vertical distance between the hole at the bottom of the chute 51 and the passage 70, which requires a slightly higher weighing foreign object detection and packaging device 200.

[0040] Figure 3(b) shows only an imaging device 80a positioned on the side between the chute 51 and the passage 70. The imaging device 80a photographs the article 10 as it free-falls from the hole at the bottom of the chute 51 toward the passage 70. In this case, even if multiple articles 10 fall, the overlap of the articles 10 in the captured image 81 can be minimized. In particular, by positioning multiple imaging devices 80a on the side and photographing the article 10 from different directions, the likelihood of obtaining an image 81 without overlapping articles 10 increases. Furthermore, in free fall, by taking multiple images 81 at regular time intervals and calculating the falling velocity, it is possible to easily distinguish between the item 10 and the foreign object 20, for example, if a metal nail or similar object is mixed in. However, in order to photograph the item 10 as it free-falls from the hole at the bottom of the chute 51 toward the passage 70 using the imaging device 80a, it is necessary to maintain a certain vertical distance between the hole at the bottom of the chute 51 and the passage 70, which requires a slightly higher weighing foreign object detection and packaging device 200.

[0041] In Figure 3(c), an imaging device 80b equipped with a wide-angle lens is positioned vertically downward at the top center of the chute 51. By equipping the imaging device 80b with a wide-angle lens, all the articles 10 discharged from multiple containers 40 can be photographed, enabling simplification and cost reduction of the device. However, in this case, since the image 81 of the article 10 is contained within a narrow range of the entire captured image, it becomes necessary to increase the resolution of the imaging device 80b and to extract the portion of the entire image in which the article 10 is photographed.

[0042] (Foreign object detection device 90) Figure 4 is a schematic block diagram showing the overall circuit configuration of the weighing, foreign object detection, and packaging device 200, with the foreign object detection device 90 at the center. In Figure 4, the foreign object detection device 90 includes a multilayer neural network 91 that takes an image 81 as input and determines whether it is an item 10 or a foreign object 20, as well as a control unit 95 that controls the combination scale 50, the imaging device 80, the packaging device 60, and the sorting device 100, and a recording unit 96 that records the coefficients of the multilayer neural network 91, the parameters of each device, etc. In this embodiment, the foreign object detection device 90 includes a control unit 95 and a recording unit 96, but the control unit 95 and the recording unit 96 may be provided independently of the foreign object detection device 90.

[0043] (Multilayer neural network 91) The multilayer neural network 91 used in the foreign object inspection device should be capable of detecting the class 92 (type) and bounding box 93 (region) of each item 10 and foreign object 20 from an image 81 containing multiple items 10 and foreign objects 20. For example, Mask R-CNN or the YOLO system (you only look once) can be used. The multilayer neural network 91 prepares many items 10 and foreign objects 20 as training data in advance, learns coefficients using the training data, and records them in the recording unit 96. When detecting a foreign object, it uses the learned coefficients to detect the class 92 (type) and bounding box 93 (region) of the item 10 and foreign object 20. Alternatively, it is possible to train the coefficients of the multilayer neural network 91 using only item 10 or only foreign object 20 as training data, and if a class other than item 10 is detected, or if the class of foreign object 20 is detected, it can be determined to be a foreign object 20.

[0044] Figure 5 is an image 81 taken by the imaging device 80, which includes multiple items 10 and foreign objects 20, with the class 92 and bounding box 93 detected by the multilayer neural network 91 added. Figure 4 is an example of the output of the YOLO system. Figure 4 includes item 10, which is a bell pepper G1, and foreign objects 20, which include a caterpillar NG1 and a partially rotten bell pepper NG2. In the case of image 81 in Figure 4, multiple items 10 (and foreign objects 20) are photographed without overlapping. Note that the figure is for illustrative purposes only and illustrates the operation of the multilayer neural network 91. In an actual weighing, foreign object detection, and packaging device 200, the multilayer neural network 91 may not output an image 81 like the one in Figure 5, but rather only whether or not foreign objects 20 such as NG1 and NG2 are included in the input image 81.

[0045] In this embodiment, the outlets of the containers 40 constituting the selected combination are opened sequentially at predetermined time intervals to prevent the article 10 and the foreign object 20 from overlapping in the captured image 81. Furthermore, it is preferable to determine whether the foreign object 20 is included in each of the multiple images 81 captured by the imaging devices 80, 80a, and 80b positioned at different locations, or by capturing images from multiple different directions, and if the foreign object 20 is included in at least one of the multiple images 81, then it is determined that the foreign object 20 is included. This further reduces the possibility of misjudgment due to the overlap of the article 10 and the foreign object 20 in the image 81. Furthermore, if the images 81 captured by the imaging device 80 are a movie or a series of still images, it may be determined that foreign matter 20 is present if at least one of those images 81 contains foreign matter 20.

[0046] Figure 6 is also an image 81 taken by the imaging device 80, which includes multiple items 10 and foreign objects 20, with the class 92 and bounding box 93 detected by the multilayer neural network 91 added. In image 81 of Figure 6, the bell pepper G1 (item 10) and the caterpillar NG1 (foreign object 20) partially overlap, but the class 92 and bounding box 93 are correctly detected. A multilayer neural network 91 such as the YOLO system is preferable because it can detect the respective class 92 and bounding box 93 even when objects partially overlap.

[0047] Figures 7(a) and 7(b) are schematic diagrams showing the results of adding a class 92 and a bounding box 93 by the foreign object detection device 90 to two images 81 taken at regular time intervals. By comparing Figures 7(a) and 7(b), it can be seen that in Figure 7(b), which is an image 81 taken after a certain time has elapsed, NG3 (iron nail) is at a lower position, and NG3 (iron nail) has fallen faster than G1 (bell pepper). Based on these comparison results, NG3 (iron nail) can be determined to be a foreign object 20 because its falling speed is faster than that of G1 (bell pepper).

[0048] (Packaging equipment 60) Referring again to Figure 1, the packaging device 60 consists of a heater 61, a knife 62, and a drum 63, and packages articles 10 (and foreign matter 20) that fall sequentially from a plurality of containers 40 constituting a selected combination into a tape-shaped synthetic resin film 64. The packaging process is as follows. First, the heater 61 heat-seals the lower part of the synthetic resin film 64 that has been fed out from the drum 63, and also heat-seals both ends of the tape to form a bag shape with the synthetic resin film 64. Next, the items 10 (and foreign matter 20) are dropped sequentially from the container 40 and placed inside the bag-shaped synthetic resin film 64. Furthermore, the bag-shaped synthetic resin film 64 containing the article 10 (and foreign matter 20) is moved under the heater 61, the top of the bag-shaped synthetic resin film 64 is heat-sealed by the heater 61, and then cut with a knife 62. Through these steps, the packaged product 30 containing the article 10 (and foreign matter 20) is completed. While the packaging device 60 in this embodiment is as described above, various other packaging devices 60 can be used, such as packaging devices 60 that enclose the product in a box.

[0049] (sorting device 100) Referring further to Figure 1, the structure consists of a chute board 101, a shipping belt conveyor 102, and a waste box 103. After the items 10 (and foreign matter 20) have been placed in the bag-shaped synthetic resin film 64, and before the bag-shaped synthetic resin film 64 is moved under the heater 61, information is sent from the foreign matter detection device 90 to the sorting device 100 regarding whether the bag-shaped synthetic resin film 64 (packaged goods 30) contains foreign matter 20. The chute plate 101 of the sorting device 100 is then fixed to the position of the solid line in Figure 1 if the bag-shaped synthetic resin film 64 does not contain foreign matter 20, and to the position of the dotted line in Figure 1 if it does. When the chute plate 101 is in the position of the solid line, the packaged goods 30 are placed on the shipping belt conveyor 102 and shipped, and when it is in the position of the dotted line, it falls into the waste box 103 and is discarded. While the sorting device 100 in this embodiment is as described above, other configurations of the sorting device 100 can be used, such as one in which all packaged items 30 are placed on a belt conveyor 102, an actuator is placed on one side of the belt conveyor 102 and a piston rod is used to push defective items to the other side, or a configuration in which defective items are blown away with an air gun.

[0050] (flowchart) Figure 8 is a flowchart illustrating the overall steps of the weighing, foreign object detection, packaging, and sorting process of the weighing, foreign object detection, and packaging device 200 of this embodiment. The details of each step will be explained below. (S1) Place the articles 10 (and foreign matter 20) into each of the multiple containers 40, and weigh the articles 10 (and foreign matter 20) placed in each container 40. (S2) Based on the weight of each of the articles 10 (and foreign objects 20) contained in the multiple containers 40, a combination of articles 10 (and foreign objects 20) that results in a predetermined weight is selected. If no combination exactly matches the predetermined weight, a combination that exceeds the predetermined weight but is closest to it may be selected. (S3) The outlets of the containers 40 corresponding to the selected combination are opened sequentially at predetermined time intervals.

[0051] (S4) After a certain amount of time has elapsed since each exit was opened, images 81 of the articles 10 passing through the chute 51 and / or between the chute 51 and the passage 70 are captured by the imaging devices 80, 80a, and 80b. For example, if there are 5 containers 40 in the combination, images will be taken 5 times. The imaging devices 80, 80a, and 80b may capture a movie or multiple consecutive photographs.

[0052] (S5) Images 81 captured by the imaging device 80 are input to the multilayer neural network 91 of the foreign object detection device 90 to detect whether each image 81 contains a foreign object 20. In this case, the number of images 81 input to the multilayer neural network 91 is at least the number of combinations of selected containers 40, and the number of images 81 will be even larger if images 81 are captured by different imaging devices 80, 80a, and 80b, if images 81 are captured from multiple different directions, or if images 81 captured in a movie are input. However, because the latest multilayer neural networks 91 such as the YOLO system have a high speed of object detection, they can handle a large number of input images 81. For example, the YOLO system is said to be able to process 45-155 images 81 per second.

[0053] (S6) After all the articles 10 (and foreign matter 20) in the selected combination of containers 40 have fallen out and been placed in the packaging container (in this embodiment, a bag-shaped synthetic resin film 64), the packaging container is sealed to complete the packaged product 30. (S7-S9) The multilayer neural network 91 is checked to see if any of the multiple images 81 input to it contain the foreign object 20. If there are no images 81 containing the foreign object 20, the packaged product 30 is shipped. If there are images 81 containing the foreign object 20, the packaged product 30 is discarded. In the case of the weighing foreign object detection and packaging device 200 shown in Figure 1, the position of the chute plate 101 of the sorting device 100 must be set before the bag-shaped synthetic resin film 64 moves to the underside of the heater 61. Therefore, the decision in step S7 is made before step S6 is completed.

[0054] As described above, in the weighing and foreign object detection packaging device 200 of the first embodiment, imaging devices 80, 80a, and 80b are placed between the combination scale 50 and the packaging device 60 of the weighing and packaging device which is composed of a combination scale 50, a packaging device 60, and a sorting device 100. Multiple images 81 captured by imaging devices 80, 80a, and 80b are input to the multilayer neural network 91 of the foreign object detection device 90 to determine the presence or absence of foreign objects 20. This realizes a weighing and foreign object detection packaging device 200 that is simple in structure and can be made compact, allowing multiple articles 10 to be packaged into packages 30 of a predetermined weight, and packages 30 containing foreign objects 20 to be removed.

[0055] In the present invention, article 10 corresponds to "article", foreign object 20 corresponds to "foreign object", packaged product 30 corresponds to "packaged product", container 40 corresponds to "container", combination scale 50 corresponds to "combination scale", chute 51 corresponds to "chute", packaging device 60 corresponds to "packaging device", passage 70 corresponds to "passage", imaging devices 80, 80a, 80b correspond to "imaging devices", image 81 corresponds to "image", foreign object detection device 90 corresponds to "foreign object detection device", sorting device 100 corresponds to "sorting device", multilayer neural network 91 corresponds to "multilayer neural network", and weighing foreign object detection packaging device 200 corresponds to "weighing foreign object detection packaging device".

[0056] While the above describes a preferred embodiment of the present invention, the invention is not limited thereto. It will be understood that various other embodiments can be made without departing from the spirit and scope of the invention. Furthermore, although the operation and effects of the configuration of the present invention are described in this embodiment, these operations and effects are examples and do not limit the invention. [Explanation of symbols]

[0057] 10 Goods 20 Foreign object 30 Packaging 40 containers 50 Combination Scales 51 Shooter 60 Packaging equipment 70 aisles 80, 80a, 80b imaging equipment 81 images 90 Foreign object detection device 91-Layer Neural Network 100 sorting device 200 Weighing, foreign object detection, and packaging device

Claims

1. A truncated cone-shaped chute that causes the items dispensed from the container to fall towards the center, A passage provided at the lower part of the chute for dropping the article discharged from the chute, A photographic device for photographing the object in free fall, The device includes a determination device which takes an image captured by the aforementioned photographic device as input and determines whether or not the image contains foreign objects other than the aforementioned article, The passage is provided with a lower chute, and the lower end of the chute and the upper end of the lower chute are spaced apart in the vertical direction, and the article is configured to free-fall between the chute and the lower chute. The aforementioned imaging device is a foreign object detection device provided in multiple locations on the side surface between the lower end of the chute and the upper end of the lower chute, and photographs the free-falling article from different directions a predetermined time after the article is discharged.

2. The determination device determines whether each of the multiple images captured contains a foreign object other than the article, and determines that a foreign object is present if at least one of the multiple images contains the foreign object.

3. The foreign object detection device according to claim 2, wherein the determination device is composed of a multilayer neural network and is pre-trained using the image of the article and / or the image of the foreign object.

4. The aforementioned shooting device captures a movie or a plurality of the aforementioned images, The foreign object detection device according to claim 3, wherein the determination device calculates the falling speed of the article and / or the foreign object in the image using the movie or a plurality of images, and detects whether or not the foreign object is present.

5. A truncated cone-shaped chute that causes the discharged articles to fall toward the center, A passage provided at the lower part of the chute for dropping the article discharged from the chute, A photographic device for photographing the object in free fall, The device includes a determination device which takes an image captured by the aforementioned photographic device as input and determines whether or not the image contains foreign objects other than the aforementioned article, The passage is provided with a lower chute, and the lower end of the chute and the upper end of the lower chute are spaced apart in the vertical direction, and the article is configured to free-fall between the chute and the lower chute. The aforementioned imaging device is provided in multiple locations on the side surface between the lower end of the chute and the upper end of the lower chute, and is a foreign object detection system that photographs the free-falling object from different directions a predetermined time after the object is discharged.

Images