Inspection device, inspection method and program
The inspection device precisely detects and corrects errors in product placement within cases by comparing captured images to master data, enhancing inspection accuracy and reducing inspector workload.
Patent Information
- Application Number
- JP2023076898
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-08
- Publication Date
- 2024-11-20
AI Technical Summary
Existing inspection systems struggle to accurately and efficiently determine the orientation, type, and position of products within cases, especially when large shipments are involved, leading to increased workload for inspectors.
An inspection device that uses a camera to detect cases and articles within them, recognizes their positions and types, and compares them to master data, notifying an error if they do not match, utilizing a processor to control the inspection process and provide precise feedback.
The device enables high-precision inspection by accurately detecting and correcting errors in product placement, reducing inspector workload and improving production efficiency by minimizing redundant warnings.
Smart Images

Figure 2024161829000001_ABST
Abstract
Description
[Technical Field]
[0001] The present disclosure relates to an inspection device, an inspection method, and a program. [Background technology]
[0002] Patent Document 1 discloses an inspection system for supporting product shipping inspections. The inspection system includes a weighing device for weighing the actual weight of a product, a photographing device for photographing an actual product image, and a control device for inputting data on the actual weight measured by the weighing device and data on the actual product image photographed by the photographing device and saving the data. The inspection system also includes a host storage device for storing data for outputting information on the shipping instruction weight of a product and information on a sample shipping instruction image of the product based on shipping instruction information indicating the shipping instruction content of the product. The inspection system also includes a display device for inputting data transmitted from the control device and data transmitted from the host storage device, and displaying the actual weight and shipping instruction weight side by side, as well as the actual product image and the sample shipping instruction image side by side, for comparison and collation when inspecting the product. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-210646 Summary of the Invention [Problem to be solved by the invention]
[0004] When packing products into cases, it is necessary to check whether they are packaged correctly. For example, if the product is a gift, the orientation, number, and type of product must be packed more strictly. Therefore, when there are a large number of shipments, the workload of the inspectors increases.
[0005] In Patent Document 1, a shipping instruction sample image of a product to be shipped is compared with an actual image of the product to be shipped captured in real time by a camera, either visually or by a computer, and a person (for example, an inspector) or a computer determines whether the actual image matches the shipping instruction sample image. However, when comparing the entire shipping instruction sample image with the entire actual image, there is a problem in that it is difficult to detect the state of each product included in each image (orientation, type, number, position, etc. of each product).
[0006] The present disclosure has been devised in view of the above-described conventional situation, and aims to inspect an object with high accuracy. [Means for solving the problem]
[0007] The present disclosure provides an inspection device including a processor that detects a case of an object to be inspected from an image captured by a camera of the object to be inspected, detects multiple items from within the case, recognizes the positions and types of the items, and determines whether the type and position of each item match the master data using master data that indicates information related to the state in which the items of a specified type are placed in a specified position, and if it is determined that the type and position of at least one of the items do not match the master data, notifies a notification device of an error.
[0008] The present disclosure also provides an inspection method that detects a case of an object to be inspected from an image captured by a camera of the object to be inspected, detects multiple items from within the case, recognizes the positions and types of the items, and uses master data indicating information relating to a state in which the items of a specified type are arranged in a specified position to determine whether the type and position of each of the items match the master data, and if it is determined that the type and position of at least one of the items do not match the master data, notifies a notification device of an error.
[0009] The present disclosure also provides a program that causes a computer to execute the following processes: detecting a case of the object to be inspected from an image captured by a camera of the object to be inspected; detecting multiple items from within the case; recognizing the positions and types of the items; using master data indicating information relating to the state in which the items of a specified type are placed in a specified position, determining whether the type and position of each item match the master data; and, if it is determined that the type and position of at least one of the items do not match the master data, notifying a notification device of an error.
[0010] These comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium. [Effects of the Invention]
[0011] According to the present disclosure, an object can be inspected with high accuracy. [Brief explanation of the drawings]
[0012] [Figure 1] A diagram showing an example of a scene in which an inspection object is inspected. [Figure 2] 1 is a system configuration diagram of an inspection system according to an embodiment of the present invention. [Figure 3] FIG. 10 is a diagram showing an example of imaging data at each time point. [Figure 4] FIG. 10 is a diagram showing an example of an inspection object detected from a captured image. [Figure 5] FIG. 1 is a diagram illustrating an example of a method for calculating coordinates of an item. [Figure 6] A diagram showing an example of how to calculate the size of the case [Figure 7] 1 is a flowchart of a first process for inspecting an object to be inspected, which is performed by an inspection device; [Figure 8] 10 is a flowchart of a second process for inspecting an object to be inspected, which is executed by the inspection device. [Figure 9] 10 is a flowchart of a third process for inspecting an inspection object performed by an inspection device. [Figure 10] FIG. 10 is a diagram showing a first example of a screen displayed on a notification device or a display device. [Figure 11] FIG. 2 is a diagram showing a second example of a screen displayed on a notification device or a display device. [Figure 12] FIG. 1 is a diagram showing an example of a method for measuring the number of items using an inspection system. DETAILED DESCRIPTION OF THE INVENTION
[0013] Hereinafter, with reference to the drawings as appropriate, detailed descriptions will be given of embodiments specifically disclosing an inspection apparatus, an inspection method, and a program according to the present disclosure. However, more detailed descriptions than necessary may be omitted. For example, detailed descriptions of well-known matters and redundant descriptions of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. Note that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter of the claims.
[0014] An example of a scene in which an inspection object is inspected will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of a scene in which an inspection object is inspected.
[0015] An inspection object OB flows on the lane LA. The lane LA is a conveying machine, such as a belt conveyor, a roller conveyor, or a belt chain conveyor. A camera 12 captures an image of the inspection object OB flowing on the lane LA.
[0016] The object to be inspected OB is, for example, a case B1 in which multiple items CA are packed. The case B1 is, for example, a box, a bag, or a basket. The items CA shown in FIG. 1 are cans and sweets. Note that the items CA are not limited to cans and sweets, and may be various items such as food, cosmetics, miscellaneous goods, tableware, clothing, or electrical appliances. Furthermore, although the multiple items CA in the case B1 shown in FIG. 1 are of different shapes, they are not limited to different shapes and may be of the same shape and size.
[0017] The inspection device 30 (see FIG. 2) determines whether the inspection object OB is properly packed in the case B1 using an image of the inspection object OB captured by the camera 12. The detailed determination method of the inspection device 30 will be described with reference to FIG.
[0018] When the inspection device 30 determines that the items CA are not packed correctly in the case B1 (for example, there are not enough items CA, the items CA are placed in the wrong position, or the items CA are facing the wrong direction), the inspection device 30 notifies the notification device. In the example shown in Fig. 1, when the inspection device 30 determines that the items CA are not packed correctly in the case B1, the lamp PL lights up to notify the inspector hm.
[0019] Next, a system configuration diagram of the inspection system 1 according to this embodiment will be described with reference to FIG. 2. FIG. 2 is a system configuration diagram of the inspection system 1 according to this embodiment.
[0020] The inspection system 1 is installed in, for example, a factory or the like, and is used for the purpose of inspecting whether or not articles are correctly placed in cases.
[0021] The inspection system 1 includes lane monitoring devices 10 and 20, an inspection device 30, and a terminal device 40.
[0022] The lane monitoring device 10 is installed in one of the multiple lanes LA and captures an image of the inspection object OB using a camera 12. The number of lane monitoring devices is not limited to two, and may be one or more than two. The lane monitoring device 10 includes a light 11, a camera 12, a communication I / F 13, and a notification device 14.
[0023] The lighting 11 illuminates the inspection object OB traveling on the lane LA with an appropriate illuminance. The appropriate illuminance is an illuminance bright enough for the inspection device 30 to analyze the inspection object OB shown in the captured image. The lighting 11 is, for example, a halogen lamp, a light-emitting diode, or an incandescent lamp. Note that the lighting 11 may be omitted from the lane monitoring device 10.
[0024] The camera 12 captures an image of the inspection object OB traveling on the lane LA. The camera 12 includes at least a lens (not shown) and an image sensor (not shown) as optical elements. The lens receives light reflected by an object within the angle of view of the area captured by the camera 12 and forms an optical image of the object on the light receiving surface (i.e., the imaging surface) of the image sensor. The image sensor is, for example, a solid-state imaging element such as a Charged Coupled Device or a Complementary Metal Oxide Semiconductor. The image sensor converts the optical image formed on the imaging surface via the lens into an electrical signal every predetermined time (e.g., 1 / 30 (second)). For example, if the predetermined time is 1 / 30 (second), the frame rate of the camera 12 is 30 fps. The camera 12 may also generate image data (video data) by performing predetermined signal processing on the electrical signal every predetermined time. The camera 12 outputs the image data (video data) to the communication I / F 13. When the communication I / F 13 is included in the camera 12, the camera 12 outputs image data (video data) directly to the inspection device 30. Hereinafter, the image data and video data will be referred to as imaging data.
[0025] The communication I / F 13 is an interface circuit that performs wireless or wired communication between a lane monitoring device (e.g., lane monitoring devices 10 and 20) and the inspection device 30. Here, I / F refers to an interface. When wired communication is performed between the lane monitoring device 10 and the inspection device 30, the communication I / F 13 is, for example, a Power over Ethernet (hereinafter referred to as "PoE") hub. Communication between the lane monitoring device 10 and the inspection device 30 may be via a network. The communication method used by the communication I / F 13 is, for example, a Wide Area Network (hereinafter referred to as "WAN"), a Local Area Network (hereinafter referred to as "LAN"), Long Term Evolution (hereinafter referred to as "LTE"), mobile communication such as 5G, power line communication, short-range wireless communication (e.g., Bluetooth (registered trademark) communication), or communication for mobile phones. The communication I / F 13 may be included in the camera 12.
[0026] The notification device 14 is a device that notifies the inspection target OB of the results of the inspection performed by the inspection apparatus 30. The notification device 14 may be, for example, a lamp such as a patrol lamp, a speaker, a display, a touch panel display, or a laptop. If the notification device 14 is a lamp, the notification device 14 lights up or flashes based on an instruction from the inspection apparatus 30. The notification device 14 may light up or flash in different colors depending on the instruction from the inspection apparatus 30. If the notification device 14 is a speaker, the notification device 14 may sound an alarm based on an instruction from the inspection apparatus 30 or read text based on the instruction. Examples of the text include "Insufficient number of items," "The item is in the wrong position," or "The item is facing the wrong direction." Note that the text is merely an example and is not limited to these. If the notification device 14 is a display, a touch panel display, or a laptop, the notification device 14 displays the instruction from the inspection apparatus 30 (see FIGS. 10 and 11). The notification device 14 may be configured with two or more of a lamp such as a patrol lamp, a speaker, a display, a touch panel display, or a laptop computer.
[0027] The lane monitoring device 20 is installed in one of the multiple lanes LA and captures an image of an inspection object OB using a camera 12. The lane monitoring device 20 includes a light 11, a camera 12, a communication I / F 13, and a notification device 14.
[0028] The inspection device 30 analyzes the image data captured by the lane monitoring device 20 and inspects the object OB (i.e., whether the item CA is properly packed in the case B1). The inspection device 30 is, for example, a personal computer. The inspection device 30 includes a communication I / F 31, a processor 32, a memory 33, a database 34, a display device 35, and an input device 36.
[0029] The communication I / F 31 is an interface circuit that performs wireless or wired communication between the inspection device 30 and the communication I / F 13. Here, I / F represents an interface. When wired communication is performed between the inspection device 30 and the communication I / F 13, the communication I / F 31 is, for example, a PoE hub. The communication between the inspection device 30 and the communication I / F 13 may be via a network. The communication method used by the communication I / F 31 is, for example, mobile communication such as WAN, LAN, LTE, or 5G, power line communication, short-range wireless communication (for example, Bluetooth (registered trademark) communication), or communication for mobile phones.
[0030] The processor 32 may be a semiconductor chip equipped with at least one of electronic devices such as a central processing unit, a digital signal processor, a graphic processing unit, or a field programmable gate array. The processor 32 functions as a controller that manages the overall operation of the inspection device 30, and performs control processing for managing the operations of each component of the inspection device 30, data input / output processing between each component of the inspection device 30, data calculation processing, and data storage processing. The processor 32 operates according to a program stored in the memory 33. The processor 32 may also read and process data from an external server device or the like when the inspection device 30 is started. The processor 32 uses the memory 33 during operation and temporarily stores data generated or acquired by the processor 32 in the memory 33. The processor 32 inspects the inspection target OB using the image data transmitted from the lane monitoring devices 10 and 20 (see FIG. 7).
[0031] The memory 33 includes at least a read only memory (hereinafter referred to as "ROM") that stores programs that define the various processes performed by the processor 32 and data used during execution of the programs, and a random access memory (hereinafter referred to as "RAM") that serves as a work memory used when executing the various processes performed by the processor 32. The ROM stores programs that define the various processes performed by the processor 32 and data used during execution of the programs. The RAM temporarily stores data or information (e.g., information such as the position, type, or orientation of the item CA) generated or acquired by the processor 32.
[0032] The database 34 stores master data. The master data is various information about the state of the inspection target OB that is predetermined by a user (e.g., a person responsible for the production of the inspection target OB). The predetermined state of the inspection target OB refers to a state in which the number, type, position, arrangement, or orientation of the items CA inside the case B1 are in a state determined by the user (hereinafter referred to as the correct state). The various information is, for example, a captured image of the correct state, numerical data, matrix data, or table data. The database 34 may store the captured image data at each time sent from the lane monitoring devices 10 and 20 or the results of the inspection of the inspection target OB by the processor 32. The database 34 may also be provided outside the inspection device 30.
[0033] The display device 35 is a device that displays the current and past inspection results performed by the inspection apparatus 30. The display device 35 is, for example, a display. The display device 35 and the input device 36 may be integrated into one unit, such as a touch panel display. The display device 35 may also be omitted from the inspection apparatus 30.
[0034] The input device 36 is a device through which a user inputs instructions to the inspection apparatus 30. The input device 36 is, for example, a keyboard or a touch panel display. The input device 36 may be omitted from the inspection apparatus 30.
[0035] The terminal device 40 is communicably connected to the inspection device 30. The terminal device 40 is, for example, a terminal carried by an inspector working or a manager of the installation location of the inspection device 30. The terminal device 40 is, for example, a smartphone or a personal handy-phone system (hereinafter referred to as "PHS"). The terminal device 40 displays data transmitted from the inspection device 30 (for example, information about the inspection target OB in which an error has occurred). The terminal device 40 may be omitted from the inspection system 1.
[0036] Next, an example of imaging data at each time will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of imaging data at each time.
[0037] 3 shows one frame of imaging data at each time (times t1, t2, and t3). Time flows from time t1 to time t3. That is, time t2 is later than time t1. Also, time t3 is later than time t2. Hereinafter, a frame of imaging data at each time will be referred to as a captured image.
[0038] Captured image P1 is an image captured at time t1 in the imaging data. Multiple inspection objects are flowing on lane LA in the direction of the arrow, and captured image P1 shows the lane LA captured by camera 12 at time t1. Captured image P1 includes the entire inspection object OB1 and a part of inspection object OB2.
[0039] Captured image P2 is an image captured at time t2 in the imaging data. At time t2, inspection objects OB1 and OB2 have progressed further along lane LA in the direction of the arrow than at time t1. Also, at time t2, a portion of inspection object OB3 enters the imaging range of camera 12 and is displayed in captured image P2. Captured image P2 includes the entire inspection objects OB1 and OB2 and a portion of inspection object OB3.
[0040] Captured image P3 is an image captured at time t3 in the imaging data. At time t3, inspection objects OB1, OB2, and OB3 have progressed further along lane LA in the direction of the arrow than at time t2. Captured image P3 includes the entire inspection objects OB1 and OB2 and a portion of inspection object OB3.
[0041] The inspection device 30 analyzes the captured images at each time (for example, times t1, t2, and t3) and inspects the inspection objects OB (for example, inspection objects OB1, OB2, and OB3) traveling on the lane LA.
[0042] Next, an example of the inspection object OB detected from the captured image will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of the inspection object OB detected from the captured image.
[0043] Captured image P4 includes the entire inspection objects OB1 and OB2 and a portion of inspection object OB3. As an example, the inspection objects (e.g., inspection objects OB1, OB2, OB3) contain one bag (e.g., item C1) containing candy A, three small boxes (e.g., items C2, C3, C4) containing candy B, and two cans (e.g., items C5, C6) containing tea leaves. Candy B comes in two designs: two with cat marks and one with a bear mark. Note that the items contained in the inspection objects are merely examples and are not limited to the above examples.
[0044] The processor 32 detects the case B1 from the captured image P4 using known techniques such as image recognition. The processor 32 sets a frame FB1 for the detected case B1, with the case B1 inside. The processor 32 detects the case B2 from the captured image P4. The processor 32 sets a frame FB2 for the detected case B2, with the case B2 inside. The frames FB1 and FB2 are, for example, rectangular. Note that the shape may be modified to a polygon or the like to fit the object.
[0045] Alternatively, the processor 32 may detect the outer frame of the case B1 from the captured image and set a frame FB1 corresponding to the detected outer frame of the case B1. The processor 32 may set the frame FB1 to, for example, the same size as the outer frame of the case B1 or a size several percent larger than the size of the outer frame of the case B1.
[0046] A label corresponding to the detected object (hereinafter referred to as the detected object) may be displayed in a portion of the frames FB1 and FB2. Text indicating that the detected object is a case (e.g., "BOX") may be displayed in the frames FB1 and FB2. A numerical value indicating the likelihood of the detected object may also be displayed in a portion of the frames FB1 and FB2. The likelihood is the match rate when the detected object (here, the case) is compared with master data. For example, if the cases B1 and B2 are partially damaged, the match rate decreases. The likelihood is expressed as a percentage, and the frames FB1 and FB2 indicate 100% (i.e., there is no abnormality in the cases B1 and B2). Note that the likelihood is not limited to a percentage, and may be a scaled evaluation, etc. Processor 32 calculates the likelihood using a known technique such as image recognition.
[0047] The processor 32 detects the items placed in the cases B1 and B2 from the captured image using known techniques such as image recognition.
[0048] The processor 32 detects the item C1 placed in the case B1 from the captured image. The processor 32 sets a frame FC1 that includes the item C1 for the detected item C1.
[0049] The processor 32 detects the item C2 placed in the case B1 from the captured image. The processor 32 sets a frame FC2 that includes the item C2 for the detected item C2.
[0050] The processor 32 detects the item C3 placed in the case B1 from the captured image. The processor 32 sets a frame FC3 that includes the item C3 for the detected item C3.
[0051] The processor 32 detects the item C4 placed in the case B1 from the captured image. The processor 32 sets a frame FC4 that includes the item C4 for the detected item C4.
[0052] The processor 32 detects the item C5 placed in the case B1 from the captured image. The processor 32 sets a frame FC5 that includes the item C5 for the detected item C5.
[0053] The processor 32 detects the item C6 placed in the case B1 from the captured image. The processor 32 sets a frame FC6 that includes the item C6 for the detected item C6.
[0054] The processor 32 detects the item C7 placed in the case B2 from the captured image. The processor 32 sets a frame FC7 that includes the item C7 for the detected item C7.
[0055] The processor 32 detects the item C8 placed in the case B2 from the captured image. The processor 32 sets a frame FC8 that includes the item C8 for the detected item C8.
[0056] The processor 32 detects the item C9 placed in the case B2 from the captured image. The processor 32 sets a frame FC9 that includes the item C9 for the detected item C9.
[0057] The processor 32 detects the item C10 placed in the case B2 from the captured image. The processor 32 sets a frame FC10 that includes the item C10 for the detected item C10.
[0058] The frames FC1, FC2, FC3, FC4, FC5, and FC6 are, for example, rectangular. Note that the shape of the frames FC1, FC2, FC3, FC4, FC5, and FC6 is not limited to a rectangular shape, and the shape may be modified to a polygon or the like to fit the object.
[0059] Labeling according to the detection object may be displayed in some of the frames FC1, FC2, FC3, FC4, FC5, FC6, FC7, FC8, FC9, and FC10. Frame FC1 displays text (e.g., "A") indicating that the detection object is candy A. Frames FC2, FC4, FC7, and FC8 display text (e.g., "B") indicating that the detection object is candy B with a cat mark. Frames FC3 and FC9 display text (e.g., "C") indicating that the inspection object is candy C with a bear mark. Frames FC5, FC6, and FC10 display text (e.g., "D") indicating that the detection object is a can of tea leaves.
[0060] Furthermore, a numerical value indicating the likelihood of the detected object (here, an object) may be displayed in a portion of the boxes FC1, FC2, FC3, FC4, FC5, FC6, FC7, FC8, FC9, and FC10. The likelihood decreases, for example, when the orientation of the object (e.g., objects C1, C2, C3, C4, C5, C6, C7, C8, C9, and C10) is different (e.g., the object is upside down, facing diagonally, or backwards). For example, if object C7 is packed upside down in case B2, the likelihood displayed in box FC7 is "50." If object C10 is packed into case B2 with its logo facing diagonally, the likelihood displayed in box FC10 is "70." In this way, the inspection device 30 can estimate the orientation of the object based on the numerical likelihood value.
[0061] Additionally, items C8 and C9 are items that are placed at positions different from the predetermined positions inside case B2.
[0062] Inspection object OB2 contains items (items C7, C8, C9, C10) that are arranged in a state different from the predetermined state, and is therefore detected as having an abnormality by inspection device 30. Detailed processing related to the detection of an abnormality by inspection device 30 will be described with reference to FIG.
[0063] Only a portion of the case B3 is captured in the captured image P4 as the inspection object OB3. Therefore, it is not detected by the processor 32, or even if it is detected, the numerical value indicating the likelihood of the detected object (here, the case) is low, so it is not selected as the inspection object. Whether or not the inspection object is detected by the processor 32 will be described with reference to FIG. 6.
[0064] Next, an example of a method for calculating the coordinates of an item will be described with reference to Fig. 5. Fig. 5 is a diagram for explaining an example of a method for calculating the coordinates of an item.
[0065] FIG. 5 shows an item C1 detected by processor 32 and having a frame FC1 set thereon.
[0066] Processor 32 detects the coordinates of the four corners of frame FC1 (coordinates k1, k2, k3, k4) using known techniques such as image recognition. The coordinates are, for example, XY coordinates related to the XY axes set in the captured image. Processor 32 calculates the coordinate of the midpoint of frame FC1 (coordinate k5) using coordinates k1, k2, k3, k4. Processor 32 uses the calculated coordinates of the midpoints of each item to arrange information about each detected item (for example, the type or likelihood of the item, etc.) in order.
[0067] Next, an example of a method for calculating the size of the case will be described with reference to Fig. 6. Fig. 6 is a diagram showing an example of a method for calculating the size of the case.
[0068] FIG. 6 shows case B1, which is detected by processor 32 and in which frame FB1 is set.
[0069] The processor 32 detects the coordinates of the four corners of the frame FB1 (coordinates k6, k7, k8, and k9) using a known technique such as image recognition. The processor 32 calculates the perimeter of the frame FB1 using the coordinates k6, k7, k8, and k9. The processor 32 detects the case B1 if the perimeter of the frame FB1 is equal to or greater than a predetermined threshold (for example, the length when the entire case B1 is imaged), and does not detect the case B1 if the perimeter is less than the predetermined threshold. Note that the processor 32 is not limited to the perimeter of the frame FB1, and may also calculate the length from the coordinate k6 to the coordinate k7 (or the length from the coordinate k8 to the coordinate k9) or the area of the frame FB1.
[0070] Next, a flowchart of a first process for inspecting an inspection object OB executed by the inspection device 30 will be described with reference to Fig. 7. Fig. 7 is a flowchart of a first process for inspecting an inspection object OB executed by the inspection device 30. Each process in the flowchart of Fig. 7 is executed by the processor 32.
[0071] The processor 32 acquires imaging data of the inspection object OB from the camera 12 (step St100).
[0072] The processor 32 detects the items and the case from the captured images at each time of the acquired imaging data (step St101). The method by which the processor 32 detects the items and the case from the captured images is a method using a known technique such as image recognition. The processor 32 recognizes the type of each item. In the processing of step St101, the processor 32 calculates the coordinates of the case (see FIG. 6) and the coordinates of each item (see FIG. 5).
[0073] Processor 32 calculates the size of the case (for example, the outer periphery of the case) (see FIG. 6). Processor 32 determines whether the size of the case is equal to or greater than a predetermined threshold (step St102). When using the outer periphery of the case to make the determination related to the processing of step St102, processor 32 determines whether the outer periphery of the case is equal to or greater than a predetermined length.
[0074] When processor 32 determines that the size of the case is less than the predetermined threshold value (step St102, NO), the processing of processor 32 ends.
[0075] If the processor 32 determines that the size of the case is equal to or larger than a predetermined threshold (i.e., the entire case is captured in the captured image) (step St102, YES), it calculates the midpoint of the item present in the coordinates of the case (see FIG. 5 for a method for calculating the midpoint of the item). The processor 32 then arranges the information relating to the type of each item recognized in the processing of step St101 in the order of the coordinates of the calculated midpoints of the items (step St103). Hereinafter, the information relating to the type of each item arranged in the order of the coordinates of the midpoints of the items will be referred to as the created data. Note that if multiple cases are detected in the captured image, the processor 32 rearranges the information relating to the type of each item for each case in the order of the coordinates of the midpoints.
[0076] The processor 32 acquires the master data from the database 34. The processor 32 compares the data of the items detected in the processing of step St101 with the master data to calculate the orientation of each item (i.e., the likelihood) (step St104). The processor 32 calculates the likelihood, for example, depending on how much it differs from the master data (i.e., the orientation of the item in the correct state). The processor 32 adds the likelihood to the created data. Note that the processor 32 may calculate the likelihood before the processing of step St103, and generate the created data by arranging information related to the type of item and the likelihood in coordinate order.
[0077] The processor 32 compares the created data with the master data to determine whether the number of items is the same (step St105).
[0078] When the processor 32 compares the created data with the master data and determines that the number of items is different (step St105, NO), it records the captured image in the database 34 (step St106). For example, the processor 32 may record the captured image in the database 34 in association with the captured image the time when the captured image was captured, information about the lane LA in which the image was captured (for example, information indicating which lane LA in the case where there are multiple lanes LA), the number of detected items, or the number of items that is either excessive or insufficient.
[0079] The processor 32 outputs a signal related to a warning to the notification device 14 (step St107). The signal related to a warning is a signal notifying that there is an abnormality (i.e., an error) in the inspection target by comparing it with the master data. The signal related to a warning in the processing of step St107 is a signal notifying that the number of items is different from the master data by comparing it with the master data. For example, if the notification device 14 is a lamp, the processor 32 outputs an instruction to turn on or blink the lamp. If the notification device 14 is a lamp that can light up in multiple colors, the processor 32 may turn on the lamp in different colors depending on the content of the warning. For example, the processor 32 turns on the lamp in red as a warning related to the number. Note that the color of the lamp is just an example and is not limited to red. If the notification device 14 is a display, the processor 32 may display a message such as "Number is insufficient," "Number is one too many," or "Number is different" on the notification device 14 as a warning. If the notification device 14 is a speaker, the processor 32 may cause the notification device 14 to output a warning message such as "Quantity is insufficient," "Quantity is one too many," or "Quantity is incorrect." The processor 32 may also cause the notification device 14 to output a beep (i.e., a warning sound). The processor 32 may output a warning to the notification device 14 by turning on a lamp, displaying a message on a display, outputting a message through a speaker, turning on a lamp, or a combination of these. The processing of the processor 32 proceeds to processing of step St114 after processing of step St107. Note that if the inspector confirms the warning in the processing of step St107 and inputs a signal to the input device 36 indicating that the abnormal item has been corrected, the processing of the processor 32 may proceed to processing of step St114 after processing of step St107. "Abnormal" refers to the state of the inspection target, case, or item being different from the normal state.
[0080] When the processor 32 compares the creation data with the master data and determines that the number of items is the same (step St105, YES), it compares the creation data with the master data and determines whether the arrangement of the items is the same (step St108). Note that the processor 32 may simultaneously perform the processes of steps St105 and St108. That is, when the processor 32 compares the creation data with the master data and determines whether the arrangement of the items is the same, it counts the number of each item. When the processor 32 compares the creation data with the master data and determines that at least one of the number of items and the arrangement of the items is not the same, it records the captured image in the database 34 and outputs a warning. When outputting a warning, the processor 32 may output different warnings depending on whether the number of items is different and whether the arrangement of the items is different. Furthermore, when both the number of items and the arrangement of the items are different, the processor 32 may output a warning indicating that both the number of items and the arrangement of the items are different.
[0081] When the processor 32 compares the created data with the master data and determines that the arrangement of the items is different (step St108, NO), the processor 32 records the captured image in the database 34 (step St109). For example, when the arrangement of items C8 and C9 in the inspection object OB2 is different in the captured image P4 shown in FIG. 4, the processor 32 determines that the arrangement of the items is different.
[0082] The processor 32 outputs a signal related to a warning to the notification device 14 (step St110). The signal related to the warning in the processing of step St110 is a signal notifying that the arrangement of the items is different from the master data. For example, if the notification device 14 is a lamp that can light up in multiple colors, the processor 32 may, for example, light up the lamp in yellow as a warning related to the arrangement. Note that the color of the lamp is just an example and is not limited to yellow. If the notification device 14 is a display, the processor 32 may cause the notification device 14 to display a message such as "The arrangement is different" as a warning. If the notification device 14 is a speaker, the processor 32 may cause the notification device 14 to output a message such as "The arrangement is different" as a warning. The processor 32 may also cause the notification device 14 to output a beep (i.e., a warning sound). The processor 32 may output a warning to the notification device 14 by performing a combination of the following: lighting up a lamp, displaying on a display, outputting to a speaker, lighting up a lamp, etc. The processing of the processor 32 proceeds to processing of step St114 after processing of step St110. Note that if the inspector confirms the warning of the processing of step St110 and inputs a signal to the input device 36 indicating that the abnormal item has been changed to a normal state, the processing of the processor 32 may proceed to processing of step St114 after processing of step St110.
[0083] Processor 32 determines whether the orientation of the article is correct based on the information related to the orientation (i.e., the likelihood) calculated in the process of step St104 (step St111). The correct orientation is the orientation predetermined by the user (i.e., the orientation of the article in the correct state). Processor 32 determines whether the orientation of the article is correct, for example, according to the likelihood value.
[0084] When the processor 32 determines that the orientation of the article is not correct (step St111, NO), it records the captured image in the database 34 (step St112). For example, when the orientations of the articles are different, such as the articles C7 and C10 shown in FIG. 4, the processor 32 determines that the orientation of the article is not correct.
[0085] The processor 32 outputs a signal related to the warning to the notification device 14 (step St113). The signal related to the warning in the processing of step St113 is a signal notifying that the orientation of the item is incorrect. For example, if the notification device 14 is a lamp that can light up in multiple colors, the processor 32 may, for example, light up the lamp in blue as a warning related to the orientation. Note that the color of the lamp is just an example and is not limited to blue. If the notification device 14 is a display, the processor 32 may cause the notification device 14 to display a message such as "The orientation of the item is incorrect" as a warning. If the notification device 14 is a speaker, the processor 32 may cause the notification device 14 to output a message such as "The orientation of the item is incorrect" as a warning. The processor 32 may also cause the notification device 14 to output a beep (i.e., a warning sound). The processor 32 may output a warning to the notification device 14 by performing a combination of the following: turning on a lamp, displaying on a display, outputting to a speaker, turning on a lamp, etc. The processing of the processor 32 proceeds to the processing of step St114 after the processing of step St113. Note that if the inspector acknowledges the warning of the processing of step St113 and inputs a signal to the input device 36 indicating that the abnormal item has been changed to a normal state, the processing of the processor 32 may proceed to the processing of step St114 after the processing of step St113.
[0086] After the processing of steps St107, St110, and St113, the processor 32 may return to the processing of step St105 again. In other words, when an abnormality is detected, the processor 32 may determine again whether the inspector or the like has restored the article to the correct state.
[0087] When the processor 32 determines that the orientation of the object is correct (step St111, YES), it displays the results of the inspection of the inspection object on the display device 35 (step St114). The processor 32 may, for example, display the inspection result as text (e.g., "Normal" or "Abnormality detected") or may display the captured image used for the detection. When an abnormality is detected in the processing of steps St105, St108, and St111, the processor 32 may superimpose a mark or the like indicating which object has the abnormality on the captured image and display it on the display device 35. Note that a frame (e.g., frame FC1 in FIG. 4) set when an object is detected is set at a corresponding position on the captured image and then superimposed on the captured image. Here, "setting" refers to, for example, numerically setting the center coordinates and the size of the frame in the coordinates on the captured image. "Superimposing" refers to displaying the shape of the frame at a corresponding position on the captured image according to the setting when displaying on the display device 35. The processor 32 may display the details of the detected abnormality on the display device 35. The processor 32 may combine the captured image, the inspection result, the time or information on the lane LA, and the like, and display them on the display device 35. The processor 32 may also cause the notification device 14 to display the results.
[0088] Next, a flowchart of a second process for inspecting the inspection object OB, which is executed by the inspection device 30, will be described with reference to Fig. 8. Fig. 8 is a flowchart of a second process for inspecting the inspection object OB, which is executed by the inspection device 30. Each process in the flowchart of Fig. 8 is executed by the processor 32. The same processes as those in Fig. 7 are denoted by the same reference numerals, and their description will be omitted.
[0089] Processor 32 first detects the case from the captured image (step St200). Processor 32 calculates the coordinates of the case (see FIG. 6) in the process of step St200.
[0090] Processor 32 calculates the size of the case (for example, the outer periphery of the case) (see FIG. 6). Processor 32 determines whether the size of the case is equal to or greater than a predetermined threshold (step St201). When using the outer periphery of the case to make the determination related to the processing of step St201, processor 32 determines whether the outer periphery of the case is equal to or greater than a predetermined length.
[0091] If the size of the case is less than the predetermined threshold (step St201, NO), processor 32 ends the process.
[0092] If the size of the case is equal to or larger than a predetermined threshold (YES in step St201), processor 32 detects the items included in the coordinates of the case (step St202). Processor 32 recognizes the type of item in the processing of step St202. Processor 32 calculates the coordinates of each item (see FIG. 5) in the processing of step St202. The processing thereafter is the same as the processing from step St103 onwards in the flowchart of FIG. 7, and therefore description thereof will be omitted.
[0093] In this way, the processor 32 may first detect cases and then detect items from among the detected cases. This prevents the inspection device 30 from mistakenly determining that an item in an inspection object moving along the lane LA is of a different type, orientation, or number because the case is not partially visible in the captured image. In other words, the inspection device 30 can inspect the inspection object with high accuracy. Furthermore, if the inspection device 30 determines that only a portion of the inspection object is abnormal, and if other portions of the inspection object (i.e., portions not included in the captured image) are also abnormal, the inspection device 30 may repeatedly output warnings to the notification device 14, potentially increasing the inspector's workload. Stopping the lane LA every time the inspection device 30 determines that an inspection object is abnormal may result in a decrease in production efficiency. By detecting items from among cases whose size is equal to or exceeds a threshold and proceeding with the inspection, the inspection device 30 can prevent multiple warnings from being output for the same inspection object, thereby reducing the burden on the operator. Furthermore, the inspection device 30 can prevent the lane LA from being stopped multiple times for the same inspection object, thereby improving production efficiency.
[0094] Next, a flowchart of a third process for inspecting an inspection object OB, which is executed by the inspection device 30, will be described with reference to Fig. 9. Fig. 9 is a flowchart of a third process for inspecting an inspection object OB, which is executed by the inspection device 30. Each process in the flowchart of Fig. 9 is executed by the processor 32. The same processes as those in Fig. 7 are denoted by the same reference numerals, and description thereof will be omitted.
[0095] Processor 32 compares the production data with the master data to determine whether the items are in the correct state (step St300). That is, processor 32 compares the production data with the master data to determine whether the number, arrangement, and orientation of the items are the same.
[0096] When processor 32 compares the created data with the master data and determines that the item is in a normal condition (YES in step St300), the process of processor 32 proceeds to step St114.
[0097] When the processor 32 compares the created data with the master data and determines that the article is not in the correct state (step St300, NO), the processor 32 records the captured image in the database 34 (step St301).
[0098] The processor 32 outputs a warning to the notification device 14 according to the determination result of the processing in step St300 (step St302). For example, if the notification device 14 is a lamp with multiple colors, the processor 32 lights or flashes the lamp in different colors when the number of items is different, when the arrangement is different, or when the orientation is different. If the processor 32 determines that there are multiple types of abnormalities (i.e., when the number of items is different, the arrangement is different, or the orientation is different), the processor 32 may light or flash the lamp in multiple colors sequentially or simultaneously. If the notification device 14 is a display, the processor 32 displays text according to the content of the abnormality. If the notification device 14 is a speaker, the processor 32 outputs text according to the above content.
[0099] As a result, when multiple items in a single inspection target are found to have abnormalities, the inspection device 30 can determine all abnormalities in the inspection target and issue a warning in a single process, allowing the inspection device 30 to efficiently notify inspectors of warnings and improve production efficiency.
[0100] The inspection device 30 may be configured to not issue warnings for the same error content for a certain period of time, depending on the frame rate of the camera 12 and the size of the case. For example, if the camera 12 inspects each frame at a frame rate of 30 fps, warnings for the same error may be issued frequently as the case passes through the camera 12, which can be bothersome to the inspector. By not issuing warnings for the same error content for a certain period of time, the inspection device 30 can prevent frequent warnings and unnecessary interruptions to the inspector's work. The inspection device 30 may also track the coordinates of detected cases and not issue warnings for errors related to cases that have already been detected. The inspection device 30 may also set the frame rate and the size of the detection area to be greater than the size of the case but less than the size of two cases, thereby timing each image capture for one box at a time.
[0101] Next, a first example of a screen displayed on the notification device 14 or the display device 35 will be described with reference to Fig. 10. Fig. 10 is a diagram showing a first example of a screen displayed on the notification device 14 or the display device 35.
[0102] Screen SC1 is an example of a screen displayed on the notification device 14 or the display device 35. Note that the screen displayed on the notification device 14 or the display device 35 is not limited to screen SC1. Screen SC1 includes an image IM1 and a list Li1.
[0103] Image IM1 is a captured image used in the inspection process of the inspection object by inspection device 30. The number "1" displayed on image IM1 corresponds to the number "1" in list Li1. In other words, the inspection result using image IM1 corresponds to the information related to number 1 in list Li1.
[0104] List Li1 is a list of captured images in which an abnormality was found during the inspection process of inspection device 30.
[0105] The "date and time" is the date and time when the captured image was captured. In the "date and time" field, the date and time are displayed, for example, "2022 / 12 / 15 16:03".
[0106] "Image" represents the file name of the captured image.
[0107] "Error content" displays the content of the abnormality detected in the captured image. For example, the error content corresponding to number 1 is "Candy B is upside down" and "The can is facing the wrong direction." The error content corresponding to number 2 is "An unnecessary item is inside." Here, an unnecessary item refers to an item other than the item that should be inside the case according to the master data. For example, if the master data shows that there is only one candy B with a bear mark, but there are two candy B with a bear mark, the one candy B with a bear mark becomes an unnecessary item. The error content corresponding to number 3 is "Candy A is missing." "Error content" may also display the content of multiple abnormalities detected in the captured image, as in number 1.
[0108] When there are multiple lanes LA at the location where the inspection system 1 is installed, the "lane" indicates in which lane LA the captured image was captured.
[0109] When the user selects each line of the list Li1, an image IM1 corresponding to each number is displayed on the screen SC1.
[0110] The screen SC1 may display content limited to the target lane (for example, lane 1). In this case, screens relating to other lanes (for example, lane 2) are also created.
[0111] In this way, the inspection device 30 can assist a user (e.g., an inspector, a manager, or an employee of a store selling the inspection object) in checking the details of an abnormality that occurred in the inspection object in the past. For example, when a user receives a complaint from a customer that there was a problem with the inspection object OB, the inspection device 30 can display the details of the abnormality that occurred in the past on the notification device 14 or the display device 35. In other words, the inspection device 30 assists the user in referring to past results, checking which lane in which factory the abnormality occurred, and making efforts to prevent recurrence. Note that, not only when an abnormality occurs, all information related to the captured images used in the inspection may be stored in the database 34 and displayed in the list Li1. In this case, the list Li1 may be narrowed down by distinguishing between cases where an abnormality occurred and cases where an abnormality did not occur.
[0112] Next, a second example of a screen displayed on the notification device 14 or the display device 35 will be described with reference to Fig. 11. Fig. 11 is a diagram showing a second example of a screen displayed on the notification device 14 or the display device 35.
[0113] Screen SC2 is an example of a screen displayed on the notification device 14 or the display device 35. The screen displayed on the notification device 14 or the display device 35 is not limited to screen SC2. Screen SC2 includes an image IM2 and a list Li2.
[0114] Image IM2 is a captured image used in the inspection process of the inspection object by the inspection device 30. Image IM2 displays marks (for example, rectangular marks R1, R2, R3, R4) superimposed on the items in which abnormalities have been detected. Numbers may be added to each mark and superimposed on image IM2. Note that the marks added to each mark are not limited to numbers, and may also be alphabets or symbols.
[0115] List Li2 is a list of abnormalities found in the inspection process of inspection device 30 for image IM2.
[0116] The "correction target number" is a number corresponding to the content of the abnormality that was discovered. For example, correction target number 1 corresponds to mark R1. Correction target number 2 corresponds to mark R2. Correction target number 3 corresponds to mark R3. Correction target number 4 corresponds to mark R4.
[0117] "Error content" displays the content of the abnormality detected in the captured image. For example, the error content corresponding to correction target number 1 is "Candy B is facing the wrong way." The error content corresponding to correction target number 2 is "An unnecessary item is inside." The error content corresponding to correction target number 3 is "The orientation of the can is wrong." The error content corresponding to correction target number 4 is "The position of candy B is wrong."
[0118] "Instructions" displays the instructions for the work that should be done in response to the detected abnormality. For example, the error content for correction target number 1 is "Please correct the orientation of candy B." The error content corresponding to correction target number 2 is "Please replace it with the correct item." The error content corresponding to correction target number 3 is "Please correct the orientation of the can." The error content corresponding to correction target number 4 is "Please correct the position of candy B."
[0119] This allows the inspection device 30 to notify the user of the detected abnormality and assist in correcting the abnormality in the inspection target.
[0120] Next, an example of a method for counting the number of items using the inspection system 1 will be described with reference to Fig. 12. Fig. 12 is a diagram showing an example of a method for counting the number of items using the inspection system 1. As mentioned above, counting the number of items may be processed together with determining whether the arrangement of the items is correct. This is possible by detecting the type of each item when checking the arrangement of the items, and counting the number of items at the same time.
[0121] The processor 32 counts the number of articles that have passed through the area GA, thereby enabling the inspection device 30 to count the number of articles packed in the case.
[0122] By setting the area GA to be at least the size of one case but less than the size of two cases, cases that are entirely contained within the area GA can be grouped together. This allows the processor 32 to detect each entire case. Furthermore, by adjusting the imaging frame rate and / or the size of the area GA, it is possible to prevent the same case from being counted multiple times without tracking the case. More specifically, after an entire case is imaged, images that do not entirely contain either case are captured between the time when the entire case is imaged and the time when the entire next case is imaged. This allows the processor 32 to recognize that a case has changed without tracking the cases and verifying whether they are the same case. In other words, since the processor 32 does not need to track the cases and verify whether they are the same case, it can reduce the overall processing time from imaging the cases to detecting abnormalities in the items. This also allows the processor 32 to prevent multiple counts of items in the same case, thereby accurately measuring the number of items that have passed through the area GA.
[0123] If there are multiple types of items in a case, the processor 32 may count the number of each type of item. For example, the processor 32 counts the number of items each time an item passes through the area GA. This allows the inspection device 30 to count the number of items packed in the case by type. This allows the inspection device 30 to assist an inspector or a manager of the location where the inspection device 30 is installed in determining the number of items of each type packed in the case.
[0124] Summary of the Disclosure The above description of the embodiments discloses the following techniques.
[0125] <Technology 1> The inspection device of this embodiment (e.g., inspection device 30) detects the case of the inspection object from an image captured by a camera of the inspection object, detects multiple items from within the case, recognizes the position and type of the items, and uses master data indicating information regarding the state in which a specified type of item is placed in a specified position to determine whether the type and position of each item match the master data.If it is determined that the type and position of at least one item do not match the master data, the inspection device is equipped with a processor (e.g., processor 32) that notifies a notification device of an error.
[0126] As a result, the inspection device according to this embodiment can detect each of the multiple items contained in the case and determine whether the type and position of each item matches the master data (i.e., whether it is in a normal state). This allows the inspection device to inspect the inspection object with high accuracy, and can assist the inspector in inspecting the inspection object.
[0127] <Technology 2> In the inspection device described in Technology 1, the processor calculates the coordinates of each of the multiple items in the captured image, rearranges the information on the recognized item types in coordinate order, and if it determines that the order does not match the order of the item type information in the master data, notifies the notification device of an error.
[0128] As a result, the inspection device according to this embodiment can determine whether the arrangement of the items matches the master data. If the arrangement of the items differs, the inspection device can report an error and assist the inspector in inspecting the items. For example, if a case contains a large number of items, the inspector must visually check the arrangement of all the items, which increases the workload. If such a case contains a large number of items, the inspection device can reduce the inspector's inspection workload by reporting an error if the arrangement of the items differs.
[0129] <Technology 3> In the inspection device according to the first or second technique, when it is determined that the type of the item does not match the master data, the processor notifies the notification device of the type of the item to be replaced.
[0130] As a result, the inspection device according to this embodiment can notify the inspector of the type of article that should be replaced when the types of articles packed in the case are different. This allows the inspector to easily understand the type of article that should be replaced. In other words, the inspection device can support the inspector's inspection work.
[0131] <Technology 4> In the inspection device according to any one of Techniques 1 to 3, when it is determined that the type of the item does not match the master data, the processor notifies the notification device of the location of the item in the case.
[0132] As a result, the inspection device according to this embodiment can assist the inspector in identifying the location of an item that is in a different type from the normal state.
[0133] <Technology 5> In the inspection device described in any one of Technology 1 to Technology 4, the processor calculates a difference between the detected item and master data as a numerical value, estimates the orientation of the item based on the numerical value, and if the orientation differs from the predetermined orientation indicated by the master data, notifies the notification device of an error.
[0134] As a result, the inspection device according to this embodiment can detect differences in the orientation of the object and notify the inspector. The inspection device can inspect the state of the inspection target with high accuracy by estimating the orientation of the object from the calculated numerical value.
[0135] <Technology 6> In the inspection device according to any one of the first to fifth techniques, the processor calculates the number of items, and if the number does not match the number of items in the master data, notifies the notification device of an error.
[0136] As a result, the inspection device according to this embodiment can notify the inspector when the number of articles in the case differs, thereby assisting the inspector in his or her work.
[0137] <Technology 7> In the inspection device according to any one of Techniques 1 to 6, the notification device is a lamp, and the processor causes the lamp to light up in different colors when it is determined that the number does not match the number of items in the master data, and when it is determined that the type and position of the item do not match the master data.
[0138] As a result, the inspection device according to this embodiment can visually notify the inspector of the content of the error by lighting the lamp in a different color depending on the content of each error, thereby helping the inspector to easily understand the content of the error.
[0139] <Technology 8> In the inspection device according to any one of the first to seventh techniques, the processor causes the notification device to display the captured image and the details of the error.
[0140] As a result, the inspection device of this embodiment can help a user (e.g., an inspector or a manager at the location where the inspection device is installed) visually check the contents of the detected error and easily resolve the error contents.
[0141] <Technology 9> In the inspection device according to any one of Techniques 1 to 8, the processor stores the captured image and the content of the error in association with each other.
[0142] This allows the inspection device according to this embodiment to assist the user in referring to the details of errors that have occurred in the past.
[0143] <Technology 10> In the inspection device according to any one of Techniques 1 to 9, the content of the error is at least one of the time when the captured image was captured, the type of the article, and the position of the article.
[0144] As a result, the inspection device according to this embodiment can assist the user in referencing the time when the error occurred and the type or location of the item where the error occurred.
[0145] <Technology 11> In the inspection device described in any one of Technology 1 to Technology 10, the object to be inspected is an object flowing on a conveyor, and the processor detects an item from within the case when the outer periphery of the case to be inspected flowing on the conveyor is equal to or greater than a predetermined threshold.
[0146] As a result, the inspection device according to this embodiment can inspect the inspection objects transported on the conveyor with high accuracy, and can assist the inspector in inspecting the inspection objects.
[0147] Although the embodiments have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications, alterations, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also fall within the technical scope of the present disclosure. Furthermore, the components in the above-described embodiments may be combined in any manner without departing from the spirit of the invention. [Industrial Applicability]
[0148] The technology disclosed herein is useful as an inspection device, an inspection method, and a program for inspecting an object with high accuracy. [Explanation of symbols]
[0149] 1. Inspection system 10,20 Lane monitoring device 11. Lighting 12 Camera 13,31 Communication I / F 14 Notification Devices 30 Inspection equipment 32 processors 33 Memory 34 Databases 35 Display Devices 36 Input Devices 40 Terminal Equipment LA Lane P1, P2, P3, P4 captured images CA,C1,C2,C3,C4,C5,C6,C7,C8,C9,C10 Goods B1, B2, B3 Case OB, OB1, OB2, OB3 Inspection target FB1, FB2, FC1, FC2, FC3, FC4, FC5, FC6, FC9, FC10 frame t1,t2,t3 time hm inspector PL lamp k1,k2,k3,k4,k5,k6,k7,k8,k9 coordinates SC1,SC2 screen IM1, IM2 images Li1,Li2 list R1, R2, R3, R4 marks GA Area
Claims
1. Detecting a case of the inspection object from a captured image of the inspection object captured by a camera; Detecting a plurality of articles from within the case; Recognizing the location and type of the item; using master data indicating information related to a state in which the items of a predetermined type are arranged at a predetermined position, determining whether or not the type and the position of each of the items match the master data; a processor for notifying a notification device of an error when determining that the type and the location of at least one of the items do not match the master data; Inspection equipment.
2. The processor, Calculating coordinates in the captured image of each of the plurality of articles; sorting the type information of the recognized item in order of coordinates; When it is determined that the order of the information of the types of the items in the master data does not match, the notification device is notified of the error.
2. The inspection device according to claim 1.
3. When the processor determines that the type of the item does not match the master data, the processor notifies the notification device of the type of the item to be replaced.
2. The inspection device according to claim 1.
4. and when the processor determines that the type of the item does not match the master data, notifying the notification device of the location of the item within the case.
2. The inspection device according to claim 1.
5. The processor, Calculating a numerical value of the difference between the detected item and the master data; Estimating an orientation of the article based on the value; If the orientation is different from the predetermined orientation indicated by the master data, notify the notification device of the error.
2. The inspection device according to claim 1.
6. The processor, Counting the number of items; if the number does not match the number of the items in the master data, notifying the notification device of the error; 2. The inspection device according to claim 1.
7. the notification device is a lamp; The processor causes the lamp to light up in a different color when it is determined that the number does not match the number of the items in the master data, and when it is determined that the type and the location of the items do not match the master data.
7. The inspection device according to claim 6.
8. The processor causes the notification device to display the captured image and the content of the error.
2. The inspection device according to claim 1.
9. The processor stores the captured image and the content of the error in association with each other.
2. The inspection device according to claim 1.
10. The content of the error is at least one of the time when the captured image was captured, the type of the item, or the position of the item.
10. The inspection apparatus according to claim 9.
11. The inspection object is an object flowing on a conveyor, the processor detects the item from within the case when a perimeter of the case to be inspected traveling on the conveyor is equal to or greater than a predetermined threshold value.
2. The inspection device according to claim 1.
12. Detecting a case of the inspection object from a captured image of the inspection object captured by a camera; Detecting a plurality of articles from within the case; Recognizing the location and type of the item; using master data indicating information related to a state in which the items of a predetermined type are arranged at a predetermined position, determining whether or not the type and the position of each of the items match the master data; if it is determined that the type and the location of at least one of the items do not match the master data, notifying a notification device of an error; Testing method.
13. A process of detecting a case of the inspection object from an image captured by a camera of the inspection object; detecting a plurality of items from within the case; A process for identifying the location and type of the item; a process of determining whether or not the type and the position of each of the items match the master data, using master data indicating information related to a state in which the items of a predetermined type are arranged at a predetermined position; and if it is determined that the type and the location of at least one of the items do not match the master data, notifying a notification device of an error. program.
Citation Information
Patent Citations
Commodity check system, commodity check device, and commodity check method
JP2014210646A
