Packaging inspection device, packaging inspection method, packaging inspection system, and program

The packaging inspection device and system accurately determine correct packaging by using case information, specification creation, and image analysis to ensure proper orientation and quantity of items within a single box, addressing the limitations of existing technologies in complex packing scenarios.

JP7589467B2Active Publication Date: 2024-11-26TOPPAN HOLDINGS INC
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
JP2020148319
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2020-09-03
Publication Date
2024-11-26
Estimated Expiration
2040-09-03

AI Technical Summary

Technical Problem

Existing packaging inspection technologies are inadequate for complex situations where multiple products are packed in a single box, as they cannot accurately determine the correct packaging state from above after storage.

Method used

A packaging inspection device and system that includes a case information acquisition unit, a packaging specification creation unit, an identification information acquisition unit, and imaging devices to capture and analyze images of packed items, determining correct packaging based on predefined specifications and identification results.

Benefits of technology

Enables accurate determination of correct packaging even when products are stacked in a single box, preventing errors and ensuring proper orientation and quantity of items within the packaging.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007589467000001
    Figure 0007589467000001
  • Figure 0007589467000002
    Figure 0007589467000002
  • Figure 0007589467000003
    Figure 0007589467000003
Patent Text Reader

Abstract

To provide a package inspection apparatus, a package inspection method, a package inspection system and a program which can determine whether or not the correct packaging is performed even when commodity products are packaged in one packaging box in an overlapped manner.SOLUTION: A package inspection apparatus comprises: a case information acquisition unit which acquires case information indicating an article group being an object for each case; a packaging specification creation unit which creates packaging specification information in which the packaging box and an article packaged in the packaging box are associated with each other for each case on the basis of the case information; an identification information acquisition unit which acquires identification information uniquely identifying the packaging box associated with the case; an article identification unit which identifies the article on the basis of an output result obtained by inputting an image obtained by imaging the article before packaging to an article identification model; and a packaging object inspection unit which determines whether or not the article before packaging is the object packaged in the packaging box on the basis of the packaging specification information and the identification result by the article identification unit.SELECTED DRAWING: Figure 3
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a packaging inspection device, a packaging inspection method, a packaging inspection system, and a program. [Background technology]

[0002] There is a technology for inspecting whether a product is properly packed at the site where the product is packed, etc. For example, a technology for determining whether the packaged state is correct or not by using an image of the packaged state has been disclosed (for example, see Patent Documents 1 and 2). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2014-210646 A [Patent Document 2] JP 2020-45132 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technologies of Patent Documents 1 and 2 assume a simple situation where the packaging state is fixed to one pattern, such as when single products are packaged one after another using a conveying mechanism like a line. In other words, they do not assume a situation where inspection cannot be performed by simply checking the packaging state. For example, in a case where many different types of products are packed in a single packaging box, each box being stacked on top of the other, it is difficult to inspect whether the products have been packaged correctly by simply checking the packaging state from above after the products have been placed in the box.

[0005] The present invention has been made in consideration of the above circumstances, and aims to provide a packaging inspection device, a packaging inspection method, a packaging inspection system, and a program that can determine whether products have been correctly packaged, even in cases where products are packed overlapping each other in a single packaging box. [Means for solving the problem]

[0006] The packaging inspection device of the present invention includes a case information acquisition unit that acquires case information indicating a group of target articles for each case, and a packaging box and articles to be packed in the packaging box for each case based on the case information. a packing state indicating whether the article is to be packed in the upper or lower layer when the article is stacked and packed in the packing box; a packaging specification creating unit that creates packaging specification information associated with the item; an identification information acquiring unit that acquires identification information that uniquely identifies the packaging box associated with the item; No. 1 picture To the statue a packaging target inspection unit that determines whether or not an unpackaged item is a target to be packed in a packaging box based on the packaging specification information and an identification result by the item identification unit; a packaging state inspection unit that judges whether the packaging state of the lower tier in the packing process is correct or not based on a second image capturing a state in which the items to be packed in the lower tier are packed during the packing process and the packing specification information, and allows the packing work for the next upper tier to proceed if the packaging state is correct; Equipped with.

[0007] The packaging inspection system of the present invention includes the packaging inspection device described above, a code reader that reads information from a code affixed to a packaging box and outputs the information to the packaging inspection device as identification information, and an imaging device that images an item before packaging and outputs the captured image to the packaging inspection device.

[0008] The packaging inspection method of the present invention is a packaging inspection method by a computer of a packaging inspection device, in which a case information acquisition unit acquires case information indicating a group of objects to be packed for each case, and a packaging specification creation unit creates a packaging box and an object to be packed in the packaging box for each case based on the case information. a packing state indicating whether the article is to be packed in the upper or lower layer when the article is stacked and packed in the packing box; The identification information acquisition unit acquires identification information that uniquely identifies the packaging box associated with the case, and the item identification unit acquires an image of the item before packaging. No. 1 picture To the statue A packaging object inspection unit determines whether the unpackaged object is a target to be packed in a packaging box based on the packaging specification information and the identification result by the item identification unit. The packaging state inspection unit judges whether the packaging state of the lower tier in the packing process is correct or not based on a second image in which the packaged state of the item to be packed in the lower tier is captured during the packing process and the packaging specification information, and if the packaged state is correct, the packaging work for the next upper tier is allowed to proceed. do.

[0009] The program of the present invention causes a computer of a packaging inspection device to acquire case information indicating a group of objects to be packed for each case, and to select a packing box and objects to be packed in the packing box for each case based on the case information. a packing state indicating whether the article is to be packed in the upper or lower layer when the article is stacked and packed in the packing box; and generating packaging specification information that corresponds to the item, acquiring identification information that uniquely identifies the packaging box associated with the item, and acquiring an image of the item before packaging. No. 1 picture To the statue and determining whether or not the unpackaged item is to be packed in a packaging box based on the packaging specification information and the identification result of the item. and determining whether the packaging state of the lower tier in the packing process is correct or not based on a second image in which the article to be packed in the lower tier is packed in a packed state and the packing specification information, and allowing the packing work of the next upper tier to proceed if the packaging state is correct. This is a program. Effect of the Invention

[0010] According to the present invention, even when products are packed in a single packaging box in an overlapping state, it is possible to determine whether the products are packed correctly. [Brief description of the drawings]

[0011] [Figure 1] 1 is a diagram showing an example of a packaging inspection system 1 to which a packaging inspection device 10 according to an embodiment of the present invention is applied. [Diagram 2] 1 is a diagram showing an example of a packaging inspection system 1 to which a packaging inspection device 10 according to an embodiment of the present invention is applied. [Diagram 3] 1 is a block diagram showing an example of the configuration of a packaging inspection device 10 according to an embodiment of the present invention. [Figure 4] FIG. 2 is a diagram showing an example of a configuration of packaging specification information 121 according to the embodiment of the present invention. [Diagram 5] 4 is a flowchart showing the flow of processing performed by a packaging inspection device 10 according to an embodiment of the present invention. [Figure 6] 4 is a flowchart showing the flow of processing performed by a packaging inspection device 10 according to an embodiment of the present invention. [Figure 7] 4 is a flowchart showing the flow of processing performed by a packaging inspection device 10 according to an embodiment of the present invention. [Figure 8]FIG. 13 is a diagram for explaining a learning method of an item identification model 122 according to the first modified example of an embodiment of the present invention. [Figure 9] FIG. 11 is a diagram for explaining a learning method of the item identification model 122 according to the second modified example of the embodiment of the present invention. [Figure 10] FIG. 13 is a diagram showing an example of a configuration of packing specification information 121 according to a third modified example of the embodiment of the present invention. [Figure 11] FIG. 11 is a diagram showing an example of a packaging state corresponding to the packaging specification information 121 of FIG. [Figure 12] FIG. 11 is a diagram showing an example of a packaging state corresponding to the packaging specification information 121 of FIG. [Figure 13] FIG. 13 is a diagram showing an example of a configuration of packing specification information 121 according to a fourth modified example of the embodiment of the present invention. [Figure 14] FIG. 14 is a diagram showing an example of a packaging state corresponding to the packaging specification information 121 of FIG. [Figure 15] FIG. 14 is a diagram showing an example of a packaging state corresponding to the packaging specification information 121 of FIG. [Figure 16] FIG. 13 is a diagram showing an example of a configuration of packing specification information 121 according to a fifth modified example of the embodiment of the present invention. [Figure 17] FIG. 17 is a diagram showing an example of a packaging state corresponding to the packaging specification information 121 of FIG. 16. [Figure 18] FIG. 13 is a diagram illustrating an imaging technique according to the sixth modified example of the embodiment of the present invention. [Figure 19] FIG. 13 is a diagram illustrating an imaging technique according to the sixth modified example of the embodiment of the present invention. [Figure 20] FIG. 13 is a diagram illustrating an imaging technique according to the sixth modified example of the embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0013] The packaging inspection system 1 is a system that inspects whether an item BP is correctly packaged in a packaging box BX. The packaging box in this embodiment is a container used to store an item. The item includes any product, novelty goods, promotional materials, etc. that can be delivered in a state where it is packaged in a packaging box. The packaging box is, for example, a cardboard box, various envelopes, an outer box, etc. The packaging box may include a package that encases the item, a packaging bag, etc. The shape of the packaging box may be any shape as long as it can store at least the item, and may be, for example, a box-shaped shape such as a cube or a rectangular parallelepiped, or a cylindrical shape.

[0014] 1 and 2 are diagrams showing an example of a packaging inspection system 1 to which a packaging inspection device 10 according to an embodiment of the present invention is applied. In this embodiment, two inspections, a "picking inspection" and a "packaging state inspection", are performed.

[0015] "Picking inspection" is an inspection that is performed before packing. In the picking inspection, it is determined whether the item that is about to be packed into the packing box BX is correct as the item to be packed. By performing the picking inspection, it is possible to confirm before packing that the item to be packed has been correctly selected. Therefore, even in a complex situation such as when packing a wide variety of items BP into one packing box BX, it is possible to prevent overpacking, where an item BP that is not the item to be packed is mistakenly packed, or a packing mistake, where only one item BP that is the item to be packed is not packed.

[0016] 1 shows an example in which the packaging inspection device 10 is applied to "picking inspection." In this case, the packaging inspection system 1 includes an imaging device CM, a code reader RD, and the packaging inspection device 10. The imaging device CM captures an image of an item BP and outputs the captured image to the packaging inspection device 10. The code reader RD reads a code CD attached to a packaging box BX and outputs the read information (packaging box ID) to the packaging inspection device 10.

[0017] The packaging inspection device 10 is a computer device such as a PC (Personal Computer). The packaging inspection device 10 stores, for example, packaging specification information 121 and an item identification model 122. The packaging specification information 121 is information that specifies the specifications of packaging, and is information in which a packaging box BX is associated with an item BP to be packed in the packaging box BX for each case. The item identification model 122 is a trained model that identifies an item based on an image.

[0018] The packaging inspection device 10 inputs an image acquired from the imaging device CM to the item identification model 122. The item identification model 122 identifies the item captured in the image based on the input image, and outputs the identification result. The packaging inspection device 10 refers to the packaging specification information 121 using the identification result obtained from the item identification model 122 and the packaging box ID acquired from the code reader RD. By referring to the packaging specification information 121, the packaging inspection device 10 determines whether the item BP is an item (packaging target) to be packed in the packaging box BX, and outputs the determination result. In the example of this figure, it is shown that the item BP is determined to be a packaging target, and "picking OK" is output. In the packaging inspection system 1, in response to the output of "picking OK", the process proceeds to a packing operation of storing the item BP in the packaging box BX.

[0019] "Packing status inspection" is an inspection carried out after the packing work. In the packing status inspection, it is determined whether the status (packing status) of the items BP contained in the packing box BX is correct or not. By carrying out the packing status inspection, it is possible to confirm whether the items BP are contained in the packing box BX in the correct orientation and quantity. This makes it possible to reduce the occurrence of defects such as, for example, not being able to contain all the items BP to be packed in the packing box BX or the contained items BP being damaged due to improper storage in the packing box BX.

[0020] 2 shows an example in which the packaging inspection device 10 is applied to "packaging state inspection." In this case, the packaging inspection system 1 includes an imaging device CM, a code reader RD, and the packaging inspection device 10. The imaging device CM captures an image of a packaging state KJ in which an item BP is contained in a packaging box BX, and outputs the captured image to the packaging inspection device 10. The code reader RD reads a code CD attached to the packaging box BX, and outputs the read information (packaging box ID) to the packaging inspection device 10.

[0021] The packaging inspection device 10 stores, for example, packaging specification information 121 and a packaging condition identification model 123. The packaging specification information 121 is information that is associated with a packaging condition when packaging an item BP in a packaging box BX. The packaging condition identification model 123 is a trained model that identifies whether or not the packaging condition is correct based on an image.

[0022] The packaging inspection device 10 inputs the image acquired from the imaging device CM to the packaging condition identification model 123. The packaging condition identification model 123 identifies whether the packaging condition captured in the image is correct or not based on the input image, and outputs the identification result. The packaging inspection device 10 refers to the packaging specification information 121 using the identification result obtained from the packaging condition identification model 123 and the packaging box ID acquired from the code reader RD. The packaging inspection device 10 judges whether the packaging condition KJ is correct or not by referring to the packaging specification information 121, and outputs the judgment result. In the example of this figure, it is shown that the packaging condition is judged to be correct, and "packaging condition OK" is output. In the packaging inspection system 1, in response to the output of "packaging condition OK", the operation proceeds to delivery of the packaging box BX containing the item BP.

[0023] 3 is a block diagram showing an example of the configuration of a packaging inspection device 10 according to an embodiment of the present invention. The packaging inspection device 10 includes, for example, a communication unit 11, a storage unit 12, and a control unit 13. The communication unit 11 communicates with an imaging device CM and a code reader RD.

[0024] The storage unit 12 is configured with a storage medium, for example, a hard disk drive (HDD), a flash memory, an electrically erasable programmable read only memory (EEPROM), a random access read / write memory (RAM), a read only memory (ROM), or any combination of these storage media. The storage unit 12 stores programs for executing various processes of the packaging inspection device 10, and temporary data used when performing various processes.

[0025] The storage unit 12 stores, for example, case information 120, packaging specification information 121, an item identification model 122, and a packaging condition identification model 123. The case information 120 is information related to a case. The case here is created, for example, in response to the contents of an order from a customer, and is information indicating a group of ordered items. The case information 120 may also include information such as the delivery address and delivery date.

[0026] As described above, the packing specification information 121 is information that specifies the packing specifications, and is information that associates a packing box BX with an item BP to be packed in the packing box BX for each case. The packing specification information 121 is also information that associates a packing state when packing the item BP in the packing box BX. The packing specification information 121 is created by the packing specification creation unit 131 based on the case information 120. A method by which the packing specification creation unit 131 creates the packing specification information 121 will be described in detail later.

[0027] The item identification model 122 is a trained model that identifies an item based on an image. The item identification model 122 is created by machine learning a learning data set in a learning model such as an RNN (Recurrent Neural Network). The learning data set here is information in which a learning image (input data) is associated with a label (correct answer data) indicating whether or not it is a specific item. By machine learning the learning data set, the learning model adjusts variables (parameters) in the learning model so that it can derive whether or not it is a specific item (correct answer data) based on the image (input data). In this way, the learning model with the parameters adjusted so that it can identify whether or not it is a specific item becomes a trained model, that is, the item identification model 122.

[0028] The packaging state identification model 123 is a trained model that identifies whether a packaging state is good or bad based on an image. The packaging state identification model 123 is created by machine learning a learning data set in a learning model such as an RNN (Recurrent Neural Network). The learning data set here is information in which a label (correct answer data) indicating whether the packaging state is correct or not is associated with a learning image (input data). By machine learning the learning data set, the learning model adjusts variables (parameters) in the learning model so that it can derive whether the packaging state is correct or not (correct answer data) based on the image (input data). In this way, the learning model with the parameters adjusted so that it can identify whether the packaging state is correct or not becomes a trained model, that is, the packaging state identification model 123. In this embodiment, for example, a trained model that identifies whether the packaging state is correct or not for each specific packaging state is created as the packaging state identification model 123.

[0029] The control unit 13 realizes its functions by executing programs stored in the memory unit 12 using processing units such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit) as hardware provided in the packaging inspection device 10.

[0030] The control unit 13 includes, for example, a case information acquisition unit 130, a packaging specification creation unit 131, a packaging box ID acquisition unit 132, an item identification unit 133, a packaging object inspection unit 134, and a packaging state inspection unit 135.

[0031] The case information acquisition unit 130 acquires the case information 120. The case information 120 may be information previously stored in the storage unit 12, or may be information notified from an external customer terminal (not shown). The case information acquisition unit 130 outputs the acquired case information 120 to the packaging specification creation unit 131.

[0032] The packaging specification creation unit 131 creates the packaging specification information 121 based on the case information 120. For example, the packaging specification creation unit 131 selects one or more packaging boxes BX in which to pack the items BP based on the group of items shown in the case information 120. The packaging specification creation unit 131 determines the items BP to be accommodated in the packaging box BX for each selected packaging box BX. In this case, the packaging specification creation unit 131 may determine the order in which the items BP are accommodated in the packaging box BX. For example, a relatively heavy item BP is accommodated in the bottom (lower level) of the packaging box BX, and a relatively light item BP is placed on top of it (upper level). This allows the light item BP to be accommodated without being damaged. The packaging specification creation unit 131 creates information that associates the items BP to be accommodated in the packaging box BX for each case. The packaging specification creation unit 131 stores the created information in the storage unit 12 as the packaging specification information 121.

[0033] The packing box ID acquisition unit 132 acquires a packing box ID. The packing box ID is information that uniquely identifies the packing box BX for each item, and is information embedded in the code CD attached to the packing box BX. The packing box ID acquisition unit 132 acquires the packing box ID by receiving information read by the code reader RD via the communication unit 11. The packing box ID acquisition unit 132 outputs the acquired packing box ID to the packing object inspection unit 134 and the packing state inspection unit 135.

[0034] The item identification unit 133 acquires the identification result of identifying the item BP in the "picking inspection". The item identification unit 133 acquires an image of the item BP captured by the imaging device CM, and inputs the acquired image to the item identification model 122. The item identification unit 133 acquires the output from the item identification model 122. The item identification unit 133 regards the output result from the item identification model 122 as the identification result of the item BP. The item identification unit 133 outputs the identification result of identifying the item BP to the packaging target inspection unit 134.

[0035] In "picking inspection", the packing target inspection unit 134 determines whether or not an item BP is a packing target. The packing target inspection unit 134 refers to the packing specification information 121 based on the packing box ID acquired from the packing box ID acquisition unit 132. By referring to the packing specification information 121, the packing target inspection unit 134 identifies the case associated with the packing box BX and specifies the item BP that should be contained in the packing box BX. Then, the packing target inspection unit 134 determines whether or not the item identified from the identification result of the item BP acquired from the item identification unit 133 is the item BP that should be contained in the packing box BX.

[0036] If the item identified from the image is an item BP that should be contained in the packaging box BX, the packaging object inspection unit 134 causes the packaging inspection device 10 to output "picking OK." On the other hand, if the item is not an item BP that should be contained in the packaging box BX, the packaging object inspection unit 134 causes the packaging inspection device 10 to output "picking NG."

[0037] The packaging state inspection unit 135 judges whether the packaging state is correct in the "packaging state inspection". The packaging object inspection unit 134 refers to the packaging specification information 121 based on the packaging box ID acquired from the packaging box ID acquisition unit 132. The packaging object inspection unit 134 identifies the packaging state associated with the packaging box BX by referring to the packaging specification information 121. Then, the packaging state inspection unit 135 acquires an image of the packaging state of the packaging box BX captured by the imaging device CM, and inputs the acquired image to the packaging state identification model 123. The packaging state inspection unit 135 identifies whether the packaging state is correct based on the output from the packaging state identification model 123.

[0038] When the packaging state identified from the image is a correct packaging state, the packaging state inspection unit 135 causes the packaging inspection device 10 to output "packaging state OK." On the other hand, when the packaging state is not a correct packaging state, the packaging state inspection unit 135 causes the packaging inspection device 10 to output "packaging state NG."

[0039] FIG. 4 is a diagram showing an example of the configuration of packing specification information 121 according to an embodiment of the present invention. The packing specification information 121 includes items such as a case ID, a packing box ID, an item ID, the number of items, and a packing status. The case ID is information that uniquely identifies a case. The packing box ID is information that uniquely identifies a packing box BX associated with a case. The item ID is information that uniquely identifies an item BP to be contained in the packing box BX. The number of items is the number of items identified by the item ID to be contained in the packing box BX. The packing status is information that indicates the state (packing status) of the items BP contained in the packing box BX.

[0040] 5 to 7 are flowcharts showing the flow of processing performed by the packaging inspection device 10 according to an embodiment of the present invention. Fig. 5 shows an overview of the processing performed by the packaging inspection device 10. Fig. 6 shows the flow of processing for "picking inspection" performed by the packaging inspection device 10. Fig. 7 shows the flow of processing for "packaging state inspection" performed by the packaging inspection device 10.

[0041] 5, the packaging inspection device 10 acquires case information 120 (step S10). Based on the acquired case information 120, the packaging inspection device 10 creates packaging specification information 121 (step S11). Then, in the picking process, the packaging inspection device 10 performs a picking inspection (step S12). If picking is OK, the packaging work is performed, and the packaging inspection device 10 performs a packaging state inspection after the packing work (step S13).

[0042] As shown in FIG. 6, in the picking inspection, an item BP before packaging and a packaging box BX are prepared. First, the packaging inspection device 10 acquires a packaging box ID (step S120). Next, the packaging inspection device 10 acquires an image (item image) of the item BP (step S121). The packaging inspection device 10 inputs the acquired image to the item identification model 122 (step S122). The packaging inspection device 10 acquires an identification result of the item BP from the item identification model 122 (step S123). The packaging inspection device 10 judges whether the item BP is an item to be packaged (step S124). The packaging inspection device 10 refers to the packaging specification information 121 based on the packaging box ID and identifies an item to be packaged in the packaging box BX. When the identified items include the item BP identified from the image, the packaging inspection device 10 judges that the item BP is an item to be packaged. On the other hand, if the identified items do not include the item BP recognized from the image, the packaging inspection device 10 determines that the item BP is not an item to be packaged.

[0043] If the packaging inspection device 10 determines that the item BP is an item to be packed, it outputs "Picking OK" (step S125). On the other hand, if the packaging inspection device 10 determines that the item BP is not an item to be packed, it outputs "Picking NG" (step S126). If the packaging inspection device 10 has output "Picking OK" for all of the groups of items to be packed in the packaging box BX, it ends the processing (step S127). On the other hand, if the packaging inspection device 10 has not output "Picking OK" for all of the groups of items to be packed in the packaging box BX, it returns to step S120 and repeats the determination of whether or not the items are to be packed.

[0044] As shown in FIG. 7, when a packaging operation is performed, an image of the packaging state is captured in a packaging state inspection. The packaging inspection device 10 acquires an image of the packaging state (packaging state image) (step S131). The packaging inspection device 10 inputs the acquired image to the packaging state identification model 123 (step S132). The packaging inspection device 10 acquires an identification result of identifying the packaging state from the packaging state identification model 123 (step S133). The packaging inspection device 10 judges whether the packaging state is correct or not (step S134). The packaging inspection device 10 refers to the packaging specification information 121 based on the packaging box ID and identifies the correct packaging state of the packaging box BX. If the identified packaging state matches the packaging state identified from the image, the packaging inspection device 10 judges that the packaging state is correct. On the other hand, if the identified packaging state does not match the packaging state identified from the image, the packaging inspection device 10 judges that the packaging state is not correct.

[0045] If the packaging inspection device 10 determines that the packaging condition is correct, it outputs "packaging condition OK" (step S135). On the other hand, if the packaging inspection device 10 determines that the packaging condition is incorrect, it outputs "packaging condition NG" (step S136). Then, the packaging inspection device 10 ends the process.

[0046] In the above-mentioned flow charts of Fig. 6 and Fig. 7, the case where the image is acquired after the packing box ID is acquired has been described as an example, but the present invention is not limited to this. The packing inspection device 10 may acquire the packing box ID after acquiring the image. Specifically, in Fig. 6, the process may be such that step S120 is performed after steps S121 to 123. In Fig. 7, the process may be such that step S130 is performed after steps S131 to 133.

[0047] As described above, the packaging inspection device 10 of the embodiment includes the case information acquisition unit 130, the packaging specification creation unit 131, the packaging box ID acquisition unit 132, the item identification unit 133, and the packaging target inspection unit 134. The case information acquisition unit 130 acquires the case information 120. The case information 120 is information indicating a group of items to be targeted for each case. The packaging specification creation unit 131 creates the packaging specification information 121 based on the case information 120. The packaging specification information 121 is information in which a packaging box BX and an item BP to be packed in the packaging box BX are associated with each case. The packaging box ID acquisition unit 132 acquires the packaging box ID. The packaging box ID is information that uniquely identifies the packaging box BX associated with the case. The packaging box ID acquisition unit 132 is an example of an "identification information acquisition unit." The packaging box ID is an example of "identification information." The item identification unit 133 identifies the item BP based on an output result obtained by inputting an image of the item BP before packaging into the item identification model 122. The packaging target inspection unit 134 judges whether the item BP before packaging is a target to be packaged in the packaging box BX.

[0048] As a result, the packaging inspection device 10 of the embodiment can determine whether or not the item BP to be packed in the packaging box BX is correct before packaging. As a result, even if products are packed in a single packaging box in an overlapping manner, it is possible to determine whether or not the products are packed correctly by performing a picking inspection before storing the products.

[0049] The packaging inspection system 1 of the embodiment includes a packaging inspection device 10, a code reader RD, and an imaging device CM. The code reader RD outputs information read from a code CD attached to a packaging box BX to the packaging inspection device 10 as a packaging box ID. The imaging device CM captures an image of the item BP before packaging, and outputs the captured image to the packaging inspection device 10. As a result, the packaging inspection device 10 of the embodiment provides the same effects as those described above.

[0050] (First Modification of the Embodiment) Here, a first modified example of the embodiment will be described. In this modified example, the packaging inspection device 10 generates an item identification model 122 according to the situation at the site where the picking work is performed.

[0051] FIG. 8 is a diagram for explaining a learning method of the item identification model 122 according to the first modified example of the embodiment of the present invention. In FIG. 8, the learning model M1 is trained so as to be able to identify the item A. As shown in FIG. 8, the packaging inspection device 10, for example, trains the learning model M1 with a learning data set in which an image of the item A is associated with a label indicating that the image is an image of the item A (a). The packaging inspection device 10 also trains the learning data set in which an image of the item A captured from a different viewpoint from the image in (a) is associated with a label indicating that the image is an image of the item A (b). The packaging inspection device 10 also trains the learning data set in which an image of an item BP# different from the item A is associated with a label indicating that the image is not an image of the item A (c). The item BP# is an item that can be picked instead of the item A at the site where the picking work is performed. The item that can be picked instead of the item A may be arbitrarily selected according to the layout of the site where the picking work is performed and the content of the work. That is, in this modification, the learning model is made to learn that an item that may be picked by mistake at the site is a different item from item A. This enables the item identification model 122 to accurately identify that an incorrect item is not a target for packaging when the incorrect item is picked.

[0052] As described above, in the packaging inspection device 10 according to the first modification of the embodiment, the item identification model 122 is a trained model that has learned the correspondence between an image and a target item based on a training data set in which a label indicating whether or not the learning image is a target item is associated with the learning image. The learning images of the item identification model 122 include a group of images in which the target item is captured from different positions, and images of non-target items that may be packed in a packaging box but are different from the target item. As a result, in the first modification of the embodiment, when an incorrect item is picked on-site, it is possible to accurately identify that the item is not a target item for packing.

[0053] (Modification 2 of the embodiment) Here, a second modification of the embodiment will be described. In this modification, the packaging inspection device 10 generates an item identification model 122 for identifying a plurality of items.

[0054] Fig. 9 is a diagram for explaining a learning method of the item identification model 122 according to the second modified example of the embodiment of the present invention. In Fig. 9, the learning model M2 is trained so as to be able to identify whether or not there are four items A. As shown in Fig. 9, the packaging inspection device 10, for example, trains the learning model M2 with a learning data set in which images of four items A placed on a tray are associated with labels indicating that the images are images of four items A (a). It is preferable to use trays similar to those used at the site where the picking work is performed.

[0055] The packaging inspection device 10 also trains a learning data set in which an image of four items A placed on a tray in a manner different from the image in (a) is associated with a label indicating that the image is an image of four items A (b). The packaging inspection device 10 also trains a learning data set in which an image of an item A and an item BP# different from the item A placed on a tray is associated with a label indicating that the image is not an image of four items A (c). That is, in this modified example, learning is performed according to the situation at the site, in this example, a situation in which four items A are placed on a tray and picking work is performed. As a result, the item identification model 122 does not need to individually image the items BP placed on the tray at the site, but only needs to image the state of the items placed on the tray, making the imaging work easier. The item identification model 122 is also capable of simultaneously identifying items and identifying the number of items.

[0056] As described above, in the packaging inspection device 10 according to the second modification of the embodiment, the learning images of the item identification model 122 are a trained model that has learned the correspondence between images and target items based on a learning dataset in which a label indicating whether or not an item is a target item is associated with learning images including images of multiple target items and images of a mixture of target items and non-target items. As a result, in the second modification of the embodiment, learning can be performed according to the on-site situation, imaging work can be facilitated, and item identification and number identification can be performed simultaneously, enabling efficient inspection.

[0057] (Modification 3 of the embodiment) Here, a third modification of the embodiment will be described. In this modification, the packaging inspection device 10 generates a packaging condition identification model 123 according to the situation at the site where the packaging work is performed. Figs. 10 to 12 are diagrams for explaining the third modification of the embodiment. Fig. 10 is a diagram showing an example of the configuration of packaging specification information 121 in the third modification of the embodiment of the present invention. Figs. 11 and 12 are diagrams showing an example of a packaging condition corresponding to the packaging specification information 121 of Fig. 10. As shown in Fig. 10, consider a case where six items with item ID (BP101), three items with item ID (BP202), and three items with item ID (BP203) are packed in a packaging box BX.

[0058] In this case, as shown in Fig. 11, six items with item ID (BP101) are lined up in a row along the left side of the packaging box BX. To the right of them, two rows of six items each with the same shape as the item with item ID (BP101) are lined up. To the right of them, three items with item ID (BP202) are lined up, and to the right of those, three items with item ID (BP203) are lined up.

[0059] In addition, in the case of the packaging specification information 121 as shown in Fig. 10, it is possible that the positions of the three items with the item ID (BP202) in Fig. 11 are reversed to the positions of the three items with the item ID (BP203). In this way, when various types of items BP are packed in one packaging box BX, multiple packaging states are possible.

[0060] In this modification, the packaging inspection device 10 makes the learning model learn both a correct packaging state and an incorrect packaging state among a plurality of packaging states. When the packaging state shown in the example of FIG. 11 is the correct packaging state, the packaging inspection device 10 makes the learning model learn a learning data set in which the image in FIG. 11 is associated with a label indicating a correct packaging state. Furthermore, the packaging inspection device 10 makes the learning model learn a learning data set in which the image in FIG. 12 is associated with a label indicating an incorrect packaging state. That is, in this modification, in a situation where the packaging target is properly picked up, the learning model learns both a correct packaging state and an incorrect packaging state. This enables the item identification model 122 to accurately identify that the packaging state is incorrect when the item is packed in an incorrect packaging state.

[0061] As described above, the packaging inspection device 10 according to the third modified example of the embodiment further includes a packaging state inspection unit 135. The packaging state inspection unit 135 judges whether or not the packaging state is correct based on the output result obtained by inputting an image of a packaging state in which the item BP is packaged in the packaging box BX to the packaging state identification model 123, and the packaging specification information 121. The packaging state identification model 123 is a trained model that has learned the correspondence between an image and a correct packaging state based on a learning data set in which a label indicating whether or not the packaging state is correct is associated with the learning image. The learning images of the packaging state identification model 123 include an image of a correct packaging state and an image of a packaging state in which the breakdown of the item group packaged in the packaging box is correct but the packaging state is incorrect. This enables the packaging inspection device 10 to accurately identify that the packaging state is incorrect when the item is packaged in an incorrect packaging state.

[0062] (Fourth Modification of the Embodiment) Here, a fourth modified example of the embodiment will be described. In this modified example, the packaging inspection device 10 generates a packaging state identification model 123 that has been trained on the process of the packaging work. FIGS. 13 to 15 are diagrams for explaining the fourth modified example of the embodiment. FIG. 13 is a diagram showing an example of the configuration of the packaging specification information 121 in the fourth modified example of the embodiment of the present invention. FIGS. 14 and 15 are diagrams showing an example of a packaging state corresponding to the packaging specification information 121 of FIG. 13. As shown in FIG. 13, a case is considered in which an item with an item ID (BP301) is packed in addition to the group of items shown in FIG. 10 in a packaging box BX. Then, the group of items is accommodated in the lower tier of the packaging box BX as shown in FIG. 14, and the item with the item ID (BP301) as shown in FIG. 15 is placed on the upper tier of the packaging box BX.

[0063] In this case, the packaging state of the group of items below the item with item ID (BP301) cannot be confirmed from the image captured after the item with item ID (BP301) is placed on the platform. As a countermeasure to this, in this modified example, images showing the intermediate packaging process are trained. This allows the packaging state identification model 123 to identify whether or not the packaging state, which cannot be confirmed at the final packaging stage, is correct.

[0064] In this modified example, during the packing process (for example, when the packing of the lower layer is completed), the packing state is captured by the imaging device CM, and the captured image is output to the packing inspection device 10. The packing state identification model 123 identifies whether the packing state of that packing process is correct or not, and if the packing state is correct, the work proceeds to the next packing process. This enables the packing state identification model 123 to identify whether the packing state of the packing process is correct or not.

[0065] As described above, in the packaging inspection device 10 according to the fourth modified example of the embodiment, the learning images for the packaging condition identification model 123 include images of the packaging condition during the packaging process. This enables the packaging inspection device 10 to identify whether the packaging condition during the packing process is correct, and enables accurate inspection of the packaging work.

[0066] (Fifth Modification of the Embodiment) Here, a fifth modified example of the embodiment will be described. In this modified example, the packaging inspection device 10 generates a packaging condition identification model 123 that has been trained to learn a state in which items are packed overlapping each other. Figs. 16 and 17 are diagrams for explaining the fifth modified example of the embodiment. Fig. 16 is a diagram showing an example of the configuration of packaging specification information 121 in the fifth modified example of the embodiment of the present invention. Fig. 17 is a diagram showing an example of a packaging condition corresponding to the packaging specification information 121 of Fig. 13.

[0067] As shown in FIG. 15, consider a case where items identified by item ID (BP401), item ID (BP402), and item ID (BP403) are packed one by one in a packing box BX. These three groups of items are stacked and stored in the packing box BX, and are packed in a state where only a part of the item with item ID (BP401) is exposed in the final packing stage. In such a case, when the packaging inspection device 10 can identify the orientation of the item with ID (BP401) using a part of the item with item ID (BP401) that is visible in the final packing stage and can identify whether the item is in a correct packing state or not, the packaging inspection device 10 uses a packaging state identification model 123 that has learned only images of the final packing stage. On the other hand, when the packaging inspection device 10 cannot identify whether the item is in a correct packing state or not only from the visible part in the final packing stage, the packaging inspection device 10 uses a packaging state identification model 123 that has learned both images of the packing process and the final stage. This allows the packaging inspection device 10 to inspect the packaging state in the packing process as necessary, and to inspect only the packaging state in the final stage if not necessary. Therefore, it is possible to inspect the packaging condition efficiently and accurately according to the circumstances.

[0068] (Sixth Modification of the Embodiment) Here, a sixth modified example of the embodiment will be described. In this modified example, a device that assists in capturing images for learning is used. As shown in FIG. 18, in this modified example, for example, an electric rail DR is used to capture images of an object BP from various viewpoints. An imaging device CM is fixed to the electric rail DR. The electric rail DR moves around the subject (object BP) in the direction of the arrow. The imaging device CM captures images continuously in conjunction with the movement of the electric rail DR. This makes it possible to easily obtain images of the object BP captured from various viewpoints.

[0069] As shown in Fig. 19, a subject (article BP) may be placed on a rotating platform KD. The rotating platform KD is rotated, and the imaging device CM captures images continuously from a fixed position in conjunction with the rotation. This makes it easy to obtain images of the article BP captured from various viewpoints.

[0070] As shown in Fig. 20, a simple darkroom may be used for imaging. This allows the imaging environment to be constant, making it easier to extract features from learning images. By using such a darkroom for the environment in which images are captured during picking and packing work, it is expected that the accuracy of identification will improve.

[0071] In the above-mentioned embodiment, the case where an item BP is packed in a packing box BX has been described as an example. However, the present invention is not limited to this. This embodiment can be applied to the inspection of a group of items and one composite item created by combining the group of items. For example, the present invention can be applied to a system that inspects whether a pamphlet and a sticker are combined and whether the sticker is attached to a predetermined location on the pamphlet. In this case, in the picking inspection, it is inspected whether the predetermined pamphlet and the predetermined sticker have been picked correctly. Then, in the packaging state inspection, it is inspected whether the sticker is attached to a predetermined location on the pamphlet.

[0072] This embodiment can also be applied to the inspection of inserting paper (such as an errata) into a booklet. In this case, the picking inspection checks whether a specific booklet and a specific inserted paper have been correctly picked. Then, the packaging state inspection checks whether the paper has been correctly inserted into a specific location in the booklet.

[0073] This embodiment can also be applied to the inspection of inserting a sample (such as a sample of a cosmetic product) into a mount. In this case, the picking inspection checks whether a specific mount and a specific sample have been correctly picked. Then, the packaging state inspection checks whether the sample has been correctly inserted into a specific location on the mount.

[0074] This embodiment can also be applied to the inspection of attaching accessories (such as covers, bands, and pockets) to booklets. In this case, the picking inspection checks whether a specific booklet and specific accessories have been correctly picked. Then, the packaging state inspection checks whether the accessories have been correctly attached to specific locations on the booklet.

[0075] This embodiment can also be applied to the inspection of attaching accessories (such as covers, bands, and pockets) to booklets. In this case, the picking inspection checks whether a specific booklet and specific accessories have been correctly picked. Then, the packaging state inspection checks whether the accessories have been correctly attached to specific locations on the booklet.

[0076] The packaging inspection system 1 and the packaging inspection device 10 in the above-mentioned embodiment may be realized in whole or in part by a computer. In that case, a program for realizing the function may be recorded in a computer-readable recording medium, and the program recorded in the recording medium may be read into a computer system and executed to realize the function. The term "computer system" as used herein includes hardware such as an OS and peripheral devices. The term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into a computer system. The term "computer-readable recording medium" may also include a medium that dynamically holds a program for a short period of time, such as a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line, and a medium that holds a program for a certain period of time, such as a volatile memory inside a computer system that is a server or client in that case. The above-mentioned program may be a program for realizing a part of the above-mentioned function, or may be a program that can realize the above-mentioned function in combination with a program already recorded in the computer system, or may be a program that is realized using a programmable logic device such as an FPGA.

[0077] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and designs that do not deviate from the gist of the present invention are also included. [Explanation of symbols]

[0078] 1. Packaging inspection system 10...Package inspection equipment 120…Project information 121...Packing specification information 122...Item Identification Model 123…Package status identification model 13...Control section 130…Project information acquisition department 131…Packing Specification Creation Department 132...packaging box ID acquisition unit (identification information acquisition unit) 133…Item Identification Division 134…Packaging inspection section 135…Packaging Condition Inspection Department CM: Imaging equipment RD…Code reader

Claims

1. a case information acquisition unit that acquires case information indicating a group of target items for each case; a packaging specification creation unit that creates, for each case based on the case information, packaging specification information in which a packaging box, an item to be packed in the packaging box, and a packaging state indicating whether the item should be packed in the upper layer or the lower layer when the items are stacked and packed in the packaging box are associated with each other; an identification information acquisition unit that acquires identification information that uniquely identifies a packing box associated with a case; an item identifying unit that identifies an item based on a first image obtained by capturing an image of the item before packaging; a packaging object inspection unit that determines whether an unpackaged item is an object to be packed in a packaging box based on the packaging specification information and an identification result by the item identification unit; a packaging state inspection unit that judges whether the packaging state of the lower tier in the packing process is correct or not based on a second image of the packaged state of the items to be packed in the lower tier during the packing process and the packaging specification information, and allows the packing work of the next upper tier to proceed if the packaging state is correct; A packaging inspection device comprising:

2. The item identification unit identifies an item based on an output result obtained by inputting the first image into an item identification model; the packaging state inspection unit determines whether or not a packaging state of a lower tier in a packaging process is correct based on an output result obtained by inputting the second image into a packaging state identification model; the packaging state identification model is a trained model that has learned a correspondence between an image and a correct packaging state based on a training dataset in which a learning image is associated with a label indicating whether the packaging state is correct or not, The learning images for the packaging state identification model include images of the packaging state of the lower tier in the packaging process. The packaging inspection device according to claim 1.

3. The item identification model is a trained model that has learned a correspondence relationship between an image and a target item based on a training dataset in which a learning image is associated with a label indicating whether or not the image is a target item, The learning images for the item identification model include a group of images in which the target item is captured from different positions, and an image of a non-target item that can be packed in a packaging box and is different from the target item. The packaging inspection device according to claim 2.

4. The learning images for the item identification model include images in which a plurality of target items are captured, and images in which target items and non-target items are mixed. The packaging inspection device according to claim 2 or 3.

5. The learning images for the packaging state identification model include images of a packaging state during a packaging process. The packaging inspection device according to claim 2.

6. The packaging inspection device according to any one of claims 1 to 5, a code reader that reads information from a code attached to a packaging box and outputs the information to the packaging inspection device as the identification information; an imaging device that captures an image of an item before packaging and outputs the captured image to the packaging inspection device; A packaging inspection system comprising:

7. A packaging inspection method by a computer of a packaging inspection device, comprising: The case information acquisition unit acquires case information indicating a group of items that are the subject of each case, A packaging specification creation unit creates packaging specification information for each project based on the project information, which corresponds a packaging box, an item to be packed in the packaging box, and a packaging state indicating whether the item is to be packed in the upper layer or the lower layer when the item is stacked and packed in the packaging box, for each project; an identification information acquisition unit acquires identification information that uniquely identifies a packing box associated with the case; The article identifying unit identifies the article based on a first image obtained by capturing an image of the article before packaging; a packaging object inspection unit determines whether or not the unpackaged object is an object to be packed in a packaging box based on the packaging specification information and the identification result by the object identification unit; a packing state inspection unit determines whether or not the packing state of the lower tier in the packing process is correct based on a second image capturing a state in which the items to be packed in the lower tier are packed during the packing process and the packing specification information, and allows the packing work of the next upper tier to proceed if the packing state is correct. Packaging inspection methods.

8. The computer of the packaging inspection device Obtain project information showing the target product group for each project, Based on the case information, packaging specification information is created for each case, which corresponds a packing box, an item to be packed in the packing box, and a packing state indicating whether the item is to be packed in the upper layer or the lower layer when the item is stacked and packed in the packing box, acquiring identification information that uniquely identifies a packing box associated with the item; Identifying the article based on a first image obtained by capturing the article before packaging; determining whether or not an unpackaged item is to be packed in a packaging box based on the packaging specification information and an identification result of the item; a determination is made as to whether or not the packaging state of the lower tier in the packing process is correct based on a second image in which the item to be packed in the lower tier is packed in a packed state during the packing process and the packing specification information, and if the packaging state is correct, the packing work for the next upper tier can be carried out. program.

Citation Information

Patent Citations

  • Id reading device, id reading method, id reading program, and inspecting device

    JP2008297086A

  • Information processor and program

    JP2014118259A

  • Commodity check system, commodity check device, and commodity check method

    JP2014210646A

  • Server apparatus, packing apparatus and program

    JP2019021065A

  • Box packing apparatus

    JP2020045132A