Package inspection device and package inspection method

Abstract

The invention provides a packaging body inspection device and the like, which can detect damage and the like of a packaging material with higher precision under the condition that a special packaging material with an ultraviolet absorption function is not used. A package (1) to be inspected is obtained by shrink-packaging or stretch-packaging an object to be packaged with a packaging material, and the outer surface of the object to be packaged is rougher than the outer surface of the packaging material. In a state in which the packaging body (1) is irradiated with ultraviolet light, the ultraviolet light reflected from the packaging body (1) is imaged, and whether or not the packaging material in the packaging body (1) is qualified is determined on the basis of specular reflection light image data relating to the packaging body (1) obtained by the imaging. In specular reflection light image data, a damaged position becomes relatively darker, while a normal position becomes relatively brighter, resulting in a relatively large difference in contrast between the normal position and the damaged position. Consequently, damage or the like of the packaging material can be detected with higher accuracy without using a special packaging material imparting an ultraviolet absorption function.

Description

Technical Field

[0001] The present invention relates to an inspection device and method for inspecting a packaged product obtained by shrink-packing or stretch-packing a packaged object with a specified packaging material. Background Art

[0002] It has long been known that a packaging body is obtained by shrink-packing or stretch-packing various packaged items such as food, cosmetics, medicines, and daily necessities using a predetermined packaging material. Such packaging bodies are widely used from the viewpoints of protecting the packaged items, improving the convenience of handling the packaged items, and improving the design. As packaging bodies, there are types in which the entire packaged item is covered with a packaging material, types in which only a part of the packaged item is partially covered with a packaging material, and the like.

[0003] However, if the packaging material in the package is damaged, it may cause various problems such as insufficient protection of the packaged object, reduced convenience or design, etc. Therefore, it is necessary to check whether the packaging material is damaged before shipping the product (package).

[0004] In this regard, as an inspection device for inspecting damage etc. in a label covering a packaged object, there is known a device that uses a label with an ultraviolet absorption function, irradiates the label with light containing ultraviolet light as inspection light, and inspects whether the label has damage etc. using an image (fluorescent image) based on fluorescence emitted from the label (for example, see Patent Document 1). With this inspection device, it is possible to inspect whether the label has damage etc. by utilizing the fact that the damaged portion of the label that has been damaged etc. no longer emits fluorescence.

[0005] Prior art literature

[0006] Patent Literature

[0007] Patent Document 1: Japanese Patent Application Publication No. 2004-12294 Summary of the invention

[0008] Problems to be solved by the present invention.

[0009] However, when the above inspection device is used to inspect the package, it is necessary to impart ultraviolet absorption function to the packaging material. However, imparting ultraviolet absorption function specifically for inspection may increase the manufacturing cost of the product (package).

[0010] In addition, in fluorescent images, the contrast difference between normal and damaged locations in packaging materials tends to be very small, and the boundary between normal and damaged locations tends to be "blurred." Therefore, it is difficult to obtain a clear inspection image, and the inspection accuracy may become insufficient.

[0011] The present invention is proposed in view of the above situation, and its purpose is to provide a packaging inspection device and a packaging inspection method, wherein damage of packaging materials can be detected with higher accuracy without using special packaging materials with ultraviolet absorption function.

[0012] Means of solving problems

[0013] The following will describe each appropriate technical solution for achieving the above-mentioned purpose in detail. In addition, as required, the specific effects will be attached after the corresponding technical solution.

[0014] Technical Solution 1: A packaging inspection device for inspecting a packaging body obtained by shrink-packing or stretch-packing a packaged object with a packaging material formed of a predetermined thermoplastic resin material, characterized in that:

[0015] The outer surface of the packaged object is rougher than the outer surface of the packaging material.

[0016] The packaging inspection device has:

[0017] an irradiation mechanism, the irradiation mechanism irradiating ultraviolet light to the packaging body;

[0018] a photographing mechanism for photographing the ultraviolet light reflected from the packaging body; and

[0019] A determination means determines whether the packaging material in the packaging body is acceptable based on the specularly reflected (regularly reflected) light image data of the packaging body obtained by the imaging means.

[0020] According to the above-mentioned technical solution 1, the outer surface of the packaged object is rougher than the outer surface of the packaging material. Therefore, in the mirror reflection light image data obtained by photographing the package body in a state where the package body is irradiated with ultraviolet rays, the damaged position (the exposed portion of the outer surface of the packaged object) with damage etc. becomes relatively dark, and on the other hand, the normal position becomes relatively bright. Moreover, by such a difference generated in the mirror reflection light image data, it is possible to determine whether the packaging material in the package body is qualified. Therefore, as a packaging material, it is possible to check whether the packaging material has damage etc. without using a special packaging material with an ultraviolet absorption function. Thereby, it is possible to seek to suppress the increase in the manufacturing cost of the product (package body).

[0021] In addition, since the specular reflection light image data is used, the contrast difference between the normal position and the damaged position in the packaging material can be made larger, and the "blurring" at the boundary between the normal position and the damaged position can be more reliably prevented. This makes it possible to obtain clear inspection images more reliably and detect damage to the packaging material with higher accuracy.

[0022] In addition, thermoplastic resin materials used to form packaging materials include, for example, polypropylene (PP), polyethylene (PE), polyolefin (PO), polyvinyl chloride (PVC), polyvinylidene chloride (PVDC), polystyrene (PS), polyethylene terephthalate (PET), polymethylpentene (PMP) or linear low-density polyethylene (LLDPE) and the like.

[0023] Technical Solution 2: The packaging inspection device according to Technical Solution 1 is characterized in that:

[0024] At least one of the irradiation mechanism and the imaging mechanism is configured to be movable relative to the packaging body.

[0025] The photographing mechanism is configured to obtain a plurality of brightness image data by photographing the package body multiple times when at least one of the irradiating mechanism and the photographing mechanism is relatively moved relative to the package body, wherein the brightness image data has information related to the brightness of multiple positions of the package body.

[0026] The packaging body inspection device has a mirror reflection light image data generating mechanism, in which the mirror reflection light image data generating mechanism generates the above-mentioned mirror reflection image data about the packaging body by taking the highest brightness among the multiple brightnesses at the same position in the above-mentioned packaging body respectively represented by the above-mentioned multiple brightness image data as the brightness of the position at each position of the packaging body.

[0027] According to the second technical solution, more accurate specular reflection light image data can be obtained relatively easily, thereby further improving the inspection accuracy of packaging materials.

[0028] Furthermore, by arranging the irradiation mechanism and the photographing mechanism in correspondence with the transport path of the package, and thereby configuring the irradiation mechanism and the photographing mechanism to be movable relative to the package, the inspection can be performed more effectively by photographing the package during transportation. Thus, the increase in manufacturing-related costs can be more effectively suppressed.

[0029] Technical Solution 3: The packaging inspection device according to Technical Solution 1 is characterized in that the above-mentioned irradiation mechanism is a shadowless lighting device having multiple ultraviolet light sources.

[0030] According to the above technical solution 3, the specular reflection light image data related to the package can be obtained with fewer shooting times (for example, once). As a result, the processing burden when obtaining the specular reflection light image data can be reduced. In addition, the inspection speed can be increased, thereby seeking to improve productivity.

[0031] Technical Solution 4: The packaging body inspection device according to Technical Solution 1 is characterized in that the above-mentioned judgment mechanism is constructed to, while binarizing the above-mentioned mirror reflection light image data to obtain binary image data, determine that the above-mentioned packaging material is unqualified when the area of ​​the connected area of ​​the dark part in the binary image data exceeds a preset threshold.

[0032] According to the above technical solution 4, the area of ​​the connected area of ​​the dark part is used to determine whether the packaging material is qualified. Therefore, it is possible to more reliably prevent the shadow caused by the fine "wrinkles" existing in the packaging material from being mistakenly determined as an unqualified part. Therefore, the inspection accuracy can be further improved.

[0033] Technical Solution 5: The packaging inspection device according to Technical Solution 1 is characterized in that the determination mechanism has:

[0034] The recognition means is generated by causing a neural network to learn only the mirror-reflected light image data related to the packaging body of a qualified product as learning data, and the neural network has: an encoding unit that extracts a feature amount from input image data, and a decoding unit that reconstructs image data based on the feature amount;

[0035] a reconstructed image data acquisition unit capable of acquiring the following image data as reconstructed image data, the image data being image data reconstructed by inputting the mirror-reflected light image data obtained by photographing the photographing unit as raw image data into the recognition unit; and

[0036] A comparison unit is provided for determining whether the packaging material is acceptable based on a comparison result between the original image data and the reconstructed image data.

[0037] Furthermore, the "mirror reflection light image data related to the packaging body of qualified products" used as the above-mentioned "learning data" may include mirror reflection light image data related to the packaging body of qualified products accumulated in previous inspections, mirror reflection light image data related to the packaging body of qualified products selected by the operator based on visual inspection, and hypothetical qualified product image data generated using these image data, etc.

[0038] Furthermore, the above-mentioned “neural network” includes, for example, a convolutional neural network having a plurality of convolutional layers. The above-mentioned “learning” includes, for example, deep learning. The above-mentioned “recognition mechanism (generation model)” includes, for example, an auto encoder and a convolutional auto encoder.

[0039] Furthermore, since the "recognition mechanism" is a mechanism generated by learning only the specular reflection light image data related to the package of a qualified product, when the original image data related to the package of a defective product is input to the recognition mechanism, the generated reconstructed image data is almost the same as the original image data with the noise portion (defective portion) removed. In other words, when there is a defective portion in the package, as the reconstructed image data, hypothetical specular reflection light image data related to the package is generated assuming that there is no defective portion.

[0040] According to the above-mentioned technical solution 5, the original image data and the reconstructed image data reconstructed by inputting the original image data into the recognition mechanism are compared, and the qualification of the packaging material is determined based on the comparison result. Therefore, the two image data to be compared are respectively related to the same packaging material. Therefore, since the shape and appearance of the packaging body are basically the same in the two image data to be compared, in order to prevent false detection caused by differences in shape and appearance, it is not necessary to set relatively loose judgment conditions, and more stringent judgment conditions can be set. In addition, in the two image data to be compared, the conditions when the image data are obtained (for example, the configuration position and configuration angle of the packaging body relative to the camera, the light and dark state and the viewing angle of the camera, etc.) can be made consistent. As a result, particularly excellent inspection accuracy can be obtained.

[0041] Technical Solution 6: A packaging inspection method for inspecting a packaging body obtained by shrink-packing or stretch-packing a packaged object with a packaging material formed of a specified thermoplastic resin material, characterized in that:

[0042] The outer surface of the packaged object is rougher than the outer surface of the packaging material.

[0043] The packaging inspection method has the following features:

[0044] an irradiation step of irradiating the packaging body with ultraviolet light;

[0045] a photographing step of photographing the ultraviolet light reflected from the packaging body; and

[0046] A determination step of determining whether the packaging material in the packaging body is acceptable based on the mirror-reflected light image data of the packaging body obtained by the imaging step.

[0047] According to the above-mentioned technical solution 6, the same effect as the above-mentioned technical solution 1 can be produced.

[0048] In addition, the technical contents related to the above technical solutions can be used in combination as appropriate. For example, the technical contents related to the above technical solutions 4 or 5 can be used for the technical contents related to the above technical solutions 2 or 3. The technical contents related to the above technical solutions 2 to 5 can be applied to the above technical solution 6 as appropriate. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A schematic diagram of a cross section of the packaging body.

[0050] Figure 2 It is a three-dimensional schematic diagram of the packaging inspection device.

[0051] Figure 3 This is a front view of the packaging inspection device.

[0052] Figure 4 It is a schematic diagram showing the difference in the reflection state of ultraviolet light generated depending on the presence or absence of a damaged position in the packaging material.

[0053] Figure 5 This is a block diagram showing the functional structure of the package inspection device.

[0054] Figure 6 A schematic diagram illustrating the structure of a neural network.

[0055] Figure 7 This is a flowchart showing the flow of the learning process of the neural network.

[0056] Figure 8 This is a flowchart showing the flow of the inspection process.

[0057] Fig. 9 FIG. 1 is a partially cutaway schematic front view of a package inspection device having a movable lighting device in another embodiment.

[0058] Fig.10 FIG. 1 is a partially cutaway front schematic diagram of a packaging inspection device having an illumination device formed as a shadowless illumination device in another embodiment.

[0059] Fig.11 is a rough diagram of binarized image data in another embodiment. DETAILED DESCRIPTION

[0060] An embodiment will be described below with reference to the drawings. First, a package body 1 to be inspected will be described.

[0061] like Figure 1 and Figure 2 As shown, the packaging body 1 is manufactured by shrink-wrapping or stretch-wrapping the packaged object 2 with the packaging material 3. Figure 1 In the figures, the packaging material 3 is shown to be much thicker than the actual thickness.

[0062] The packaged objects 2 may include, for example, food, cosmetics, medicines, daily necessities, electrical appliances, books, etc. Of course, the packaged objects 2 are not limited thereto. Furthermore, the packaged objects 2 may also be provided with boxes or cases made of paper, resin, etc. for storing the contents, and these boxes or cases may constitute the outer surface of the packaged objects 2. In this embodiment, the entire outer surface of the packaged objects 2 is covered with the packaging material 3. Furthermore, the packaging body 1 may also be a case in which a portion of the outer surface of the packaged objects 2 is partially covered with the packaging material 3.

[0063] The outer surface of the packaged object 2 is not a mirror surface but is rougher than the outer surface of the packaging material 3. The roughness of the outer surfaces of the packaged object 2 and the packaging material 3 can be measured using, for example, the arithmetic mean roughness Ra.

[0064] The packaging material 3 is a sheet (film) formed of a predetermined thermoplastic resin material, and at least the outer surface is a smooth surface. The thermoplastic resin material constituting the packaging material 3 may be PP, PE, PO, PVC, PVDC, PS, PET, PMP or LLDPE.

[0065] Next, a general structure of the package inspection device 10 for inspecting the above-mentioned package 1, particularly the packaging material 3, will be described.

[0066] like Figure 2 , Figure 3 As shown, the package inspection device 10 is equipped with a conveying device 11, an illuminating device 12, a line sensor camera 13 and a determination system 20. In this embodiment, the illuminating device 12 constitutes an "illuminating mechanism" and the line sensor camera 13 constitutes a "photographing mechanism".

[0067] The conveying device 11 is a device for conveying the package 1 to a predetermined post-process device (such as a boxing device, etc.). The conveying device 11 has a conveyor belt 11a for carrying the package 1, and the conveyor belt 11a is moved to continuously convey the package 1 in the horizontal direction at a constant speed. As the package 1 is conveyed by the conveying device 11, both the lighting device 12 and the line sensor camera 13 also move relative to the package 1.

[0068] The conveying device 11 is provided with a defective product discharge mechanism (not shown in the figure). When a predetermined defective product signal is input from the determination system 20, the defective product package 1 is discharged to the outside of the system (outside the conveying route of the package 1) by the defective product discharge mechanism. In this way, the defective product package 1 will not be conveyed to the above-mentioned post-process device.

[0069] The lighting device 12 and the line sensor camera 13 are respectively located above the conveying device 11 and are arranged corresponding to the conveying path of the package 1 .

[0070] The lighting device 12 irradiates ultraviolet light from above the conveying device 11 (conveyor belt 11a), that is, from above the predetermined area on the surface on which the package 1 is placed. Accordingly, the package 1 conveyed from the lighting device 12 to the above-mentioned area is irradiated with ultraviolet light from above. The irradiated ultraviolet light has a diffusion in the width direction of the conveying device 11 (the direction orthogonal to the conveying direction of the package 1). In this embodiment, the process of irradiating the package 1 with ultraviolet light by the lighting device 12 is equivalent to the "irradiation process".

[0071] The line sensor camera 13 is disposed at a position where the predetermined area in the conveying device 11 is sandwiched between the line sensor camera 13 and the lighting device 12 when viewed from above. The line sensor camera 13 includes a line sensor on which a plurality of detection elements capable of detecting ultraviolet light are arranged in a row, and is configured to capture (expose) ultraviolet light reflected from the package 1. In the present embodiment, the image data acquired by the line sensor camera 13 is brightness image data, and the brightness thereof increases or decreases according to the detection output of the line sensor.

[0072] Furthermore, when the package 1 is being transported (i.e., the lighting device 12 and the line sensor camera 13 are moving relative to the package 1), the line sensor camera 13 can take multiple shots of the ultraviolet light reflected from the package 1. Thus, multiple brightness image data can be obtained, and the multiple brightness image data have information related to the brightness of multiple positions of the package 1. In this embodiment, the process of taking pictures of the ultraviolet light reflected from the package 1 by the line sensor camera 13 corresponds to the "shooting process".

[0073] Furthermore, the ultraviolet light irradiated to the normal position in the packaging material 3 is reflected by the mirror surface of the packaging material 3, and the ultraviolet light after the mirror reflection is incident on the line sensor camera 13. On the other hand, the ultraviolet light irradiated to the damaged position (the position where the damage etc. occurs) in the packaging material 3 is not reflected by the mirror surface of the packaging material 3, and the ultraviolet light after the mirror reflection is not incident on the line sensor camera 13 (see respectively). Figure 4). Therefore, in the obtained brightness image data, the position (pixel) representing the normal position has high brightness, while the position representing the damaged position has low brightness.

[0074] Whenever a package 1 is transported in a specified amount, the brightness image data obtained by the line sensor camera 13 is converted into a digital signal (image signal) inside the line sensor camera 13, and then output to the determination system 20 (the mirror reflection light image data generation unit 24 described later) in the form of a digital signal.

[0075] Furthermore, in the present embodiment, the photographed area of ​​the package 1 by the line sensor camera 13 is set to partially overlap the photographed area of ​​the package 1 by the previous line sensor camera 13. Therefore, a pixel at a predetermined coordinate in a plurality of brightness image data obtained continuously overlaps at least a portion of a predetermined position in the package 1 (an area corresponding to one pixel in the area occupied by the package 1 in the mirror reflection light image data described later) and is slightly offset on the downstream side in the conveying direction of the package 1. Therefore, in the present embodiment, a plurality of brightness image data obtained continuously have pixels corresponding to (overlapping with) a pixel at a location in the mirror reflection light image data. Furthermore, as described later, the brightness of a pixel at a location in the mirror reflection light image data is set to the highest brightness among the brightnesses represented by a plurality of pixels corresponding to the pixel at a location in the plurality of brightness image data.

[0076] The determination system 20 is composed of a computer, input and output devices, a display device, etc. The computer includes: a CPU (central processing unit) that performs specified calculations, a ROM (read-only memory) that stores various programs and fixed value data, etc., a RAM (random access memory) that temporarily stores various data when performing various calculations, and peripheral circuits of these devices, etc.

[0077] like Figure 5 As shown, the determination system 20 is operated by the CPU according to various programs, thereby functioning as various functional units such as the main control unit 21, the lighting control unit 22, the camera control unit 23, the mirror reflection light image data generation unit 24, the learning unit 25, and the inspection unit 26 described later. In this embodiment, the mirror reflection light image data generation unit 24 constitutes a "mirror reflection light image data generation means".

[0078] However, the above-mentioned various functional units are realized through the cooperation of various hardware such as the above-mentioned CPU, ROM and RAM. There is no need to clearly distinguish whether the functions are realized by hardware or software. Some or all of these functions can be realized by hardware circuits such as integrated circuits.

[0079] In addition, the determination system 20 also includes: an input unit 20a composed of a keyboard, a mouse, a touch screen, etc.; a display unit 20b composed of a liquid crystal display screen, etc., each having a display screen capable of displaying various information; a storage unit 20c capable of storing various data, programs, calculation results, inspection results, etc.; a communication unit 20d capable of sending and receiving various data to and from the outside, etc.

[0080] Next, the above-mentioned various functional sections constituting the determination system 20 will be described in detail.

[0081] The main control unit 21 is a functional unit that controls the entire determination system 20 , and is configured to be able to send and receive various signals to other functional units such as the illumination control unit 22 and the camera control unit 23 .

[0082] The lighting control unit 22 is a functional unit that controls the lighting device 12 based on a command signal from the main control unit 21 .

[0083] The camera control unit 23 is a functional unit for controlling the line sensor camera 13, and controls the timing of photographing of the line sensor camera 13, etc., according to the command signal issued by the main control unit 21. The camera control unit 23 controls the timing of photographing of the line sensor camera 13 so that the photographed area of ​​the package 1 by the line sensor camera 13 partially overlaps with the photographed area of ​​the package 1 by the previous line sensor camera 13.

[0084] The specular reflection light image data generating unit 24 generates specular reflection light image data of the package 1 using a plurality of luminance image data regarding one package 1 acquired by the line sensor camera 13 .

[0085] More specifically, the specular reflection light image data generating unit 24 first obtains the highest brightness among the multiple brightnesses of the same position (one pixel in the specular reflection light image data finally obtained) represented by the multiple brightness image data. For example, if one pixel of each of the three brightness image data corresponds to the above-mentioned one pixel, the highest brightness among the three brightnesses represented by the three pixels is obtained. In addition, the same position includes the same position in the package 1.

[0086] On this basis, the mirror reflection light image data generating unit 24 performs a process of setting the obtained maximum brightness to the brightness of the same position (a pixel at one position in the mirror reflection light image data obtained in the end) for each pixel in the mirror reflection light image data obtained in the end. Therefore, as a result, the above-mentioned process is performed for each position of the package 1 (each pixel in the area occupied by the package 1 in the mirror reflection light image data obtained in the end). Moreover, the mirror reflection light image data generating unit 24 generates the mirror reflection light image data of the package 1 (including the information of the package 1) by setting the brightness of all pixels in the mirror reflection light image data as described above. In the present embodiment, the generated mirror reflection light image data is rectangular image data including the entire upper surface area of ​​the package 1.

[0087] Furthermore, in the present embodiment, all pixels of the brightness image data of the area other than the area occupied by the package 1 are used to generate the mirror reflection light image data of the package 1; however, it is also possible to extract only the area occupied by the package 1 from the brightness image data and generate the mirror reflection light image data of the package 1 using only the pixels of the area.

[0088] The learning unit 25 uses the learning data to perform a deep neural network 90 (hereinafter referred to as “neural network 90”, see Figure 6 ) learning, constructing the functional part of the AI ​​(artificial intelligence) model 100 as a "recognition mechanism".

[0089] In addition, the AI ​​model 100 in this embodiment is described below. It is a generative model constructed by deep learning the neural network 90 using only image data related to the packaging body 1 of a qualified product as learning data, and has a so-called autoencoder (self-encoder) structure.

[0090] Here, we will refer to Figure 6 The structure of neural network 90 will be described. Figure 6 : is a schematic diagram conceptually showing the structure of the neural network 90. ​​The neural network 90 has a structure of a convolutional auto-encoder (CAE), which has: an encoder unit 91, which is an "encoding unit" that extracts a feature quantity (latent variable) TA from input image data GA; and a decoder unit 92, which is a "decoding unit" that reconstructs image data GB based on the feature quantity TA.

[0091] Since the structure of the convolutional autoencoder is well known, a detailed description is omitted, but the encoder unit 91 has a plurality of convolution layers (Convolution Layer) 93, and in each convolution layer 93, a convolution operation using a plurality of filters (kernels) 94 is performed on the input data. The result is output as the input data of the next layer. Similarly, the decoder unit 92 has a plurality of deconvolution layers (Deconvolution Layer) 95, and in each deconvolution layer 95, a deconvolution operation using a plurality of filters (kernels) 96 is performed on the input data, and the result is output as the input data of the next layer. In addition, in the learning process described later, the weight (parameter) of each filter 94, 96 is updated.

[0092] The inspection unit 26 is a functional unit for determining whether the packaging material 3 in the packaging body 1 transported by the transport device 11 is acceptable. For example, in this embodiment, the inspection unit 26 is used to determine whether the packaging material 3 is damaged or not. In addition, if a printing unit is provided on the packaging material 3, the inspection unit 26 can also determine whether the printing unit is acceptable or not. In this embodiment, the inspection unit 26 constitutes a "determination unit".

[0093] The display unit 20b is configured to display various information stored in the storage unit 20c, for example, near the conveying device 11. Therefore, the display unit 20b can display various image data such as brightness image data, information for inspection, and acceptance determination results of the packaging material 3.

[0094] The storage unit 20c is composed of a HDD (hard disk drive), an SSD (solid state drive), etc., and has a predetermined storage area for storing, for example, the AI ​​model 100 (the neural network 90 and the learning information obtained by learning thereof). The storage unit 20c also has the function of storing various image data, information for inspection (such as various threshold values, etc.), and the acceptance determination result of the packaging material 3 by the inspection unit 26.

[0095] The communication unit 20d is equipped with a wireless communication interface conforming to a communication standard such as a wired LAN (local area network) or a wireless LAN, and is configured to send and receive various data to the outside. For example, a non-conforming signal based on the pass / fail determination result of the inspection unit 26 is output to the outside (such as the above-mentioned defective product discharge mechanism) through the communication unit 20d.

[0096] Next, we will refer to Figure 7 The flowchart in illustrates the learning process of the neural network 90 performed by the decision system 20.

[0097] When the learning process starts based on the execution of a prescribed learning program, the main control unit 21 first prepares the unlearned neural network 90 in step S101. For example, the neural network 90 pre-stored in a prescribed storage device or the like is read out. Alternatively, the neural network 90 is constructed based on the network configuration information (such as the number of layers of the neural network, the number of nodes in each layer, etc.) stored in the above storage device or the like.

[0098] Next, in step S102, reconstructed image data is obtained. That is, the learning data prepared in advance is provided as input data to the input layer of the neural network 90. ​​Then, the reconstructed image data output from the output layer of the neural network 90 is obtained. As learning data, there can be cited: mirror reflection light image data related to the packaging body 1 of qualified products accumulated in previous inspections, mirror reflection light image data related to the packaging body 1 of qualified products selected by the operator based on visual observation, and hypothetical qualified product image data generated using these image data, etc.

[0099] In the subsequent step S103, the learning data is compared with the reconstructed image data output by the neural network 90 in step S102, and it is determined whether the error is small enough (below a specified threshold).

[0100] Here, if the above error is small enough, it is determined in step S105 whether the end condition of the learning process is satisfied. For example, when the positive determination made in step S103 is performed continuously for a prescribed number of times without the processing of step S104 described later, or when the learning using all the prepared learning data is repeated for a prescribed number of times, it is determined that the end condition is satisfied. If the end condition is satisfied, the neural network 90 and its learning information (such as the updated parameters described later) are stored in the storage unit 20c as the AI ​​model 100, and the learning process ends.

[0101] On the one hand, if the end condition is not satisfied in step S105, the process returns to step S102 and the learning of the neural network 90 is performed again.

[0102] If the error is not small enough in step S103, a network update process (learning of the neural network 90) is performed in step S104, and then the process returns to step S102 to repeat the above series of processes.

[0103] In the network update process of step S104, the weights (parameters) of the filters 94 and 96 in the neural network 90 are updated to more appropriate weights in order to minimize the loss function representing the difference between the learning data and the reconstructed image data by using a known learning algorithm such as the backpropagation method. In addition, as the loss function, for example, BCE (binary cross entropy) can be used.

[0104] By repeating the processing of steps S102 to S104 multiple times, the neural network 90 can minimize the error between the learning data and the reconstructed image data, thereby outputting more accurate reconstructed image data.

[0105] Furthermore, when the specular reflection light image data related to the packaging body 1 of a qualified product is input, the AI ​​model 100 finally obtained will generate reconstructed image data that is substantially consistent with the input specular reflection light image data. Furthermore, when the specular reflection light image data related to the packaging body 1 of a defective product is input, the AI ​​model 100 will generate reconstructed image data that is substantially consistent with the input specular reflection light image data, with the noise portion (corresponding to the portion of the damaged position in the packaging material 3) removed. That is, when the packaging material 3 of the packaging body 1 is a defective product, as the reconstructed image data related to the packaging body 1, hypothetical image data related to the packaging body 1 assuming that there is no defective portion (damaged position) in the packaging material 3 is generated.

[0106] Next, we will refer to Figure 8 The flowchart in exemplifies the process (inspection process) related to the acceptance determination of the package 1 (particularly the packaging material 3) executed by the determination system 20. The inspection process is executed at a preset normal inspection process execution timing.

[0107] In the inspection process, first, in step S201, while the package 1 is being transported by the transport device 11, the ultraviolet light reflected from the package 1 is repeatedly photographed by the line sensor camera 13 while the package 1 is being irradiated with ultraviolet light from the lighting device 12. Thus, a plurality of brightness image data related to the package 1 are obtained. Furthermore, based on these brightness image data, the specular reflection light image data of the package 1 is obtained by the specular reflection light image data generation unit 24.

[0108] Next, in the determination process of step S202, the quality of the packaging material 3 is determined based on the acquired mirror reflection light image data. In the determination process, first, in step S301, the mirror reflection light image data (original image data) acquired in step S201 is input into the AI ​​model 100 by the inspection unit 26, thereby acquiring reconstructed image data. In this embodiment, the inspection unit 26 constitutes a "reconstructed image data acquisition mechanism".

[0109] In the subsequent step S302, the inspection unit 26 compares the specular reflection light image data as the original image data and the reconstructed image data, and determines the difference between the two image data. For example, the pixels at the same coordinates in the two image data are compared, and the pixels whose brightness difference is greater than a specified value are determined as the difference. In this embodiment, the inspection unit 26 that compares the original image data and the reconstructed image data constitutes a "comparison mechanism".

[0110] Next, in step S303, the area (number of pixels) of the differential connected region (that is, a block of pixels having a brightness difference of a predetermined value or more) is calculated.

[0111] Then, in step S304, the inspection unit 26 determines whether there is a damaged position (unqualified portion) in the packaging material 3. Specifically, the inspection unit 26 determines whether the area calculated in step S303 is greater than a predetermined threshold. If the calculated area is greater than the threshold, the packaging material 3 and thus the packaging body 1 are determined to be "unqualified" in step S305, and the inspection process is terminated. If it is determined to be "unqualified", an unqualified signal is output to the unqualified discharge mechanism.

[0112] On the other hand, if the calculated area is less than the above threshold, the packaging material 3 and thus the package 1 are determined to be "conforming products" in step S306, and the inspection process is terminated.

[0113] As described above, according to the present embodiment, the outer surface of the packaged object 2 is rougher than the outer surface of the packaging material 3. Therefore, in the mirror reflection light image data, the damaged position (the exposed portion of the outer surface of the packaged object 2) with damage or the like is relatively dark, while the normal position is relatively bright. Moreover, based on such a difference generated in the mirror reflection light image data, it can be determined whether the packaging material 3 in the packaging body 1 is qualified. Therefore, as the packaging material 3, it is not necessary to use a special packaging material that has an ultraviolet light absorption function, and it is possible to check whether the packaging material 3 has damage or the like. As a result, it is possible to seek to suppress the increase in costs associated with the manufacture of the product (packaging body 1).

[0114] Furthermore, since the mirror reflection light image data is used, the contrast difference between the normal position and the damaged position in the packaging material 3 can be relatively large, and the "blurring" generated at the boundary portion of the normal position and the damaged position can be more reliably prevented. As a result, a clear inspection image (mirror reflection light image data in this embodiment) can be obtained more reliably, and damage of the packaging material 3 can be inspected with higher accuracy.

[0115] Furthermore, by performing a process of setting the highest brightness among the multiple brightnesses of the same position (the one pixel in the finally obtained mirror-reflected light image data) respectively shown in the multiple brightness image data as the brightness of the above-mentioned same position (the one pixel), the mirror-reflected light image data is generated. Therefore, more accurate mirror-reflected light image data can be obtained relatively easily. As a result, the inspection accuracy of the packaging material 3 can be further improved.

[0116] In addition, the mirror reflection light image data (original image data) and the reconstructed image data reconstructed by inputting the original image data into the AI ​​model 100 are compared, and whether the packaging material 3 is qualified is determined based on the comparison result. Therefore, the two image data to be compared are respectively related to the same packaging body 1. Therefore, since the shape and appearance of the packaging body 1 are basically the same in the two image data to be compared, in order to prevent misinspection caused by differences in shape and appearance, it is not necessary to set relatively loose judgment conditions, but stricter judgment conditions can be set. In addition, in the two image data to be compared, the conditions for obtaining the image data (such as the configuration position and configuration angle of the packaging body 1 relative to the line sensor camera 13, the light and dark state, the viewing angle of the camera, etc.) can be consistent. Therefore, particularly excellent inspection accuracy can be obtained.

[0117] Furthermore, by determining whether the packaging material 3 is acceptable as described above, it is possible to more reliably prevent shadows caused by fine "wrinkles" present even in acceptable packaging bodies 3 from being erroneously determined as unacceptable portions. Therefore, the inspection accuracy can be further improved.

[0118] Furthermore, the present invention is not limited to the contents of the above-mentioned embodiments, and can be realized, for example, in the following manner. Of course, other application examples and modification examples not shown below are also possible.

[0119] (a) In the above embodiment, the package 1 is transported by the transport device 11, whereby both the lighting device 12 and the imaging mechanism (line sensor camera 13) are configured to be relatively movable relative to the package 1. In contrast, when the package 1 is in a stopped state, at least one of the lighting device 12 and the imaging mechanism (line sensor camera 13) can be configured to be relatively movable relative to the package 1 by a predetermined driving mechanism. Therefore, for example, Fig. 9 As shown, the lighting device 12 can be configured to move along a predetermined arc path around the package 1. In this configuration, each time the lighting device 12 moves a predetermined distance, the area camera 16 as a "photographing mechanism" photographs the ultraviolet light reflected from the package 1. The area camera 16 photographs a rectangular image.

[0120] In the case of the above-mentioned configuration, the specular reflection light image data is generated as follows. That is, the specular reflection light image data generating unit 24 obtains the highest brightness among a plurality of brightnesses, which are a plurality of brightnesses for the same position (a pixel at one location in the specular reflection light image data finally obtained) respectively represented by a plurality of brightness image data obtained by the area camera 16. For example, if there is only one corresponding pixel for each of the above-mentioned pixels at one location in a total of eight brightness image data obtained by eight shots, the specular reflection light image data generating unit 24 obtains the highest brightness among the eight brightnesses represented by the eight pixels.

[0121] On this basis, the specular reflection light image data generating unit 24 performs a process of setting the obtained maximum brightness to the brightness of the same position (a pixel at one position in the specular reflection light image data obtained in the final step) for each pixel in the specular reflection light image data obtained in the final step. As a result, the specular reflection light image data of the package 1 is generated.

[0122] Furthermore, the structure may be as follows: by providing a tubular ultraviolet light reflecting portion 17 that covers the moving path of the lighting device 12 from the outside, the ultraviolet light can be reflected by the inner surface of the ultraviolet light reflecting portion 17, thereby allowing the ultraviolet light to be more effectively irradiated onto the package body 1. Furthermore, a line sensor camera 13 may be provided instead of the area camera 16, and while the lighting device 12 and the package body 1 are moved, the reflected light from the package body 1 can be photographed by the line sensor camera 13.

[0123] (b) If Fig.10 As shown, the lighting device 15 as the "irradiation mechanism" can be configured as a shadowless lighting device, which has a plurality of ultraviolet light sources 15a arranged around the package 1, so that the area camera 16 as the "photographing mechanism" photographs the ultraviolet light reflected from the package 1. In addition, under this configuration, when photographing a package 1 multiple times, the position of the package 1 and the lighting device 12 is adjusted in such a manner that the relative position of the package 1 and the lighting device 12 is changed only a few times during each photographing.

[0124] When the lighting device 12 is configured as a shadowless lighting device, the specular reflection light image data related to the package 1 can be obtained with a relatively small number of shots (i.e., relatively small amounts of brightness image data). This can reduce the processing burden when obtaining the specular reflection light image data. In addition, the inspection speed can be increased, thereby seeking to improve productivity.

[0125] In addition, when the packaged object 2 is, for example, paper, vegetables, mushrooms, meat, fish, etc., and the ultraviolet reflectivity of the outer surface of the packaged object 2 is sufficiently low (the outer surface of the packaged object 2 usually absorbs ultraviolet light), the lighting device 15 can be configured as a shadowless lighting device, so that the mirror reflection light image data can be obtained with a very small number of shooting times (for example, once). In addition, when the mirror reflection light image data is obtained by one shooting, the mirror reflection light image data is the same as the brightness image data obtained by one shooting by the area camera 16. Therefore, it is not necessary to perform a process of obtaining the mirror reflection light image data from a plurality of brightness image data.

[0126] Furthermore, when the lighting device 15 is configured as a shadowless lighting device, a cylindrical ultraviolet light reflecting portion 17 may be provided, which covers all the multiple ultraviolet light sources 15a from the outside, and the ultraviolet light can be reflected by the inner surface of the ultraviolet light reflecting portion 17, so that the ultraviolet light can more effectively irradiate the package body 1. Furthermore, a line sensor camera 13 may be provided instead of the area camera 16, and the reflected light from the package body 1 may be photographed by the line sensor camera 13 while the package body 1 is moved.

[0127] (c) In the above embodiment, the inspection unit 26 is configured to determine whether the packaging material 3 is qualified by comparing the mirror reflection light image data (original image data) and the reconstructed image data using the AI ​​model 100. In contrast, the inspection unit 26 can determine whether the packaging material 3 is qualified based on a binarized image obtained by binarizing the mirror reflection light image data.

[0128] Specifically, the inspection unit 26 performs binarization processing on the mirror reflection light image data using a predetermined binarization threshold value to obtain binarized image data. For example, a binary image data is obtained in which pixels with brightness above the binarization threshold value are set to "1 (bright part)" and pixels with brightness below the binarization threshold value are set to "0 (dark part)". In the obtained binary image data, the damaged position X1 where damage or the like occurs and the shadow X2 caused by fine "wrinkles" become the dark part (refer to Fig.11 ). Furthermore, the inspection unit 26 calculates the area (number of pixels) of the connected dark area in the binary image data. If the calculated area exceeds a preset threshold, it is considered that the packaging material 3 is damaged, and the packaging material 3 is determined to be a "defective product". On the other hand, if the calculated area is below the threshold, the packaging material 3 is determined to be a "qualified product" by the inspection unit 26.

[0129] In this way, by using the area of ​​the connected region of the dark part to judge whether it is acceptable or not, it is possible to more reliably prevent the shadow X2 caused by the fine "wrinkles" that also exist in the packaging material 3 of the acceptable product from being misjudged as an unacceptable part. Therefore, the inspection accuracy can be further improved.

[0130] (d) In the above embodiment, the area of ​​the connection region is used to determine whether the packaging material 3 is qualified, but the length of the connection region may be used to determine whether the packaging material 3 is qualified. For example, when the length of the connection region exceeds a predetermined threshold, or when there are a predetermined number of connection regions whose length exceeds the predetermined threshold, it may be considered that there are cracks on the packaging material 3, and the packaging material 3 may be determined to be "unqualified".

[0131] Furthermore, as described in (c) above, a scheme using the length of the connected region may be applied to the pass / fail determination process using the binary image data.

[0132] (e) In the above embodiment, the conformity of the packaging material 3 is determined based on the state of the packaging material 3 itself. In contrast, the conformity of the packaging material 3 may be determined based on the state of the packaging material 3 relative to the packaged object 2. For example, when a part of the packaged object 2 is partially wrapped by the packaging material 3, the conformity of the packaging material 3 may be determined based on the relative position of the packaging material 3 relative to the packaged object 2.

[0133] (f) The structure of the AI ​​model 100 (neural network 90) as the "recognition mechanism" and its learning method are not limited to the above-mentioned embodiments. For example, it can be configured as follows: when executing the learning process of the neural network 90 and the acquisition process of the reconstructed image data, various data can be normalized as needed. In addition, the structure of the neural network 90 can be configured as follows, that is, it is not limited to Figure 6 The structure shown may include, for example, a pooling layer after the convolution layer 93. Of course, the number of layers of the neural network 90, the number of nodes in each layer, and the connection structure of each node may also have different configurations.

[0134] In addition, in the above embodiment, the AI ​​model 100 (neural network 90) is a generative model having a convolutional autoencoder (CAE) structure, but is not limited thereto. However, it can also be a generative model having a different type of autoencoder structure such as a variational autoencoder (VAE).

[0135] Furthermore, in the above-mentioned embodiment, the structure of the neural network 90 is learned by the error back propagation method, but the present invention is not limited to this, and learning may be performed by using other various learning algorithms.

[0136] In addition, the neural network 90 can also be composed of a dedicated circuit for AI processing such as a so-called AI chip. In this case, the AI ​​model 100 can be constructed in the following manner, that is, only learning information such as parameters is stored in the memory 20c, which is read by the dedicated circuit for AI processing and set as the neural network 90.

[0137] Meanwhile, in the above-mentioned embodiment, the determination system 20 has the learning unit 25 and is configured to perform learning of the neural network 90 within the determination system 20, but the present invention is not limited thereto. For example, the learning unit 25 may be omitted, and the learning of the neural network 90 may be performed outside the determination system 20, and the AI ​​model 100 (neural network 90 after learning) learned externally may be stored in the storage unit 20c.

[0138] Explanation of symbols

[0139] 1…Packaging body, 2…Packaged article, 3…Packaging material, 10…Packaging body inspection device, 12, 15…Illumination device (illumination mechanism), 13…Line sensor camera (shooting mechanism), 16…Area camera (shooting mechanism), 24…Mirror reflection light image data generating unit (mirror reflection light image data generating unit), 26…Inspection unit (determination mechanism, reconstructed image data acquisition mechanism, comparison mechanism), 91…Encoder unit (encoding unit), 92…Decoder unit (decoding unit), 100…AI model (recognition mechanism).

Claims

1. A packaging inspection device for inspecting a packaging body obtained by shrink-packing or stretch-packing a packaged object with a packaging material formed of a predetermined thermoplastic resin material, characterized in that: The outer surface of the packaged object is rougher than the outer surface of the packaging material. The packaging inspection device has: an irradiation mechanism, the irradiation mechanism irradiating ultraviolet light to the packaging body; a photographing mechanism for photographing the ultraviolet light reflected from the packaging body; and A determination means determines whether the packaging material in the packaging body is acceptable based on the mirror-reflected light image data of the packaging body obtained by the imaging means.

2. The packaging inspection device according to claim 1, characterized in that: At least one of the irradiation mechanism and the imaging mechanism is configured to be movable relative to the packaging body. The photographing mechanism is configured to obtain a plurality of brightness image data by photographing the package body multiple times when at least one of the irradiating mechanism and the photographing mechanism is relatively moved relative to the package body, wherein the brightness image data has information related to the brightness of multiple positions of the package body. The packaging body inspection device has a mirror reflection light image data generating mechanism, in which the mirror reflection light image data generating mechanism generates the above-mentioned mirror reflection image data about the packaging body by taking the highest brightness among the multiple brightnesses at the same position in the above-mentioned packaging body respectively represented by the above-mentioned multiple brightness image data as the brightness of the position at each position of the packaging body.

3. The packaging inspection device according to claim 1, characterized in that: The irradiation mechanism is a shadowless lighting device having a plurality of ultraviolet light sources.

4. The packaging inspection device according to claim 1, characterized in that: The determination mechanism is configured to perform binarization processing on the mirror reflection light image data to obtain binarized image data, and determine that the packaging material is unqualified when the area of ​​the connected region of the dark portion in the binarized image data exceeds a preset threshold.

5. The packaging inspection device according to claim 1, characterized in that: The above-mentioned determination body has: The recognition means is generated by causing a neural network to learn only the mirror-reflected light image data related to the packaging body of a qualified product as learning data, and the neural network has: an encoding unit that extracts a feature amount from input image data, and a decoding unit that reconstructs image data based on the feature amount; a reconstructed image data acquisition unit capable of acquiring the following image data as reconstructed image data, the image data being image data reconstructed by inputting the mirror-reflected light image data obtained by photographing the photographing unit as raw image data into the recognition unit; and A comparison unit is provided for determining whether the packaging material is acceptable based on a comparison result between the original image data and the reconstructed image data.

6. A packaging inspection method for inspecting a packaging body obtained by shrink-packing or stretch-packing a packaged object with a packaging material formed of a predetermined thermoplastic resin material, characterized in that: The outer surface of the packaged object is rougher than the outer surface of the packaging material. The packaging inspection method has the following features: an irradiation step of irradiating the packaging body with ultraviolet light; a photographing step of photographing the ultraviolet light reflected from the packaging body; and A determination step of determining whether the packaging material in the packaging body is acceptable based on the mirror-reflected light image data of the packaging body obtained by the imaging step.

Images