Anomaly Data Recording System

Abstract

This system provides an anomaly data recording system that makes it easier to find data indicating an anomaly within the target data. [Solution] The abnormal data recording system 50 includes a camera 52 that photographs the tablets being filled and / or transported to obtain video data, a ring buffer 61 that stores the video data, and an encoder 51 that can generate phases related to the transport of the tablets. When the trigger generation unit 62 generates a trigger, it extracts and saves target data from the video data in the ring buffer 61. The trigger generation unit 62 determines whether there is an abnormality related to the filling and / or transport of the tablets 5 based on still frame image data when a predetermined phase is generated by the encoder 51 in the video data and reconstructed image data obtained by inputting the image data into the tablet AI model 200, and generates a trigger if there is an abnormality.

Description

Technical Field

[0001] The present invention relates to contents that are filled in the pocket portion of a container film and are conveyed as the container film is conveyed, and to an abnormal data recording system for recording moving image data related to abnormalities in the filling and / or conveyance of the contents.

Background Art

[0002] Generally, in the field of pharmaceuticals and the like, blister sheets such as PTP (Press Through Pack) sheets are known. A blister sheet includes a container film having a pocket portion for accommodating contents (for example, tablets, etc.), and a cover film that is attached to the container film so as to seal the opening side of the pocket portion.

[0003] The above-described blister sheet can be manufactured by a blister packaging machine. The blister packaging machine includes means for forming a pocket portion in a conveyed belt-like container film, means for filling the pocket portion with contents, means for attaching a belt-like cover film to the container film so as to seal the opening side of the pocket portion (sealing means), means for obtaining a blister sheet by punching out a belt-like blister film composed of the container film and the cover film, and the like. The contents filled in the pocket portion are conveyed downstream (towards the sealing means side) as the container film is conveyed.

[0004] By the way, during the manufacture of the blister sheet, filling errors of the contents (for example, the contents do not fall at an appropriate timing during filling, etc.) or conveyance errors (for example, the contents jump out due to abnormal vibration of the container film, etc.) may occur, resulting in abnormal states such as a state where the contents are not accommodated in the pocket portion or a state where the contents remain on the blister packaging machine. When such an abnormal state occurs, it is necessary to take appropriate measures to prevent the same kind of state from occurring again.

[0005] Therefore, it is conceivable to use a status confirmation system equipped with a video recording means, an inspection means, and a target data storage means (see, for example, Patent Document 1). The video recording means captures the scene of filling the pocket with contents and transporting the contents downstream, thereby obtaining video data consisting of multiple still frame image data saved in chronological order. The inspection means is provided downstream of the video recording means along the transport path of the container film and performs an inspection to determine whether or not contents are contained in the pocket. The target data storage means uses the inspection means's determination of a defect, that is, the inspection means's determination that contents are not contained in the pocket, as a trigger to extract and save predetermined target data from the video data. The target data is a predetermined range within the video data, based on the still frame image data at the time the trigger occurred, and located at a position corresponding to the number of retrospective frames corresponding to the amount of contents transported from the video recording means to the inspection means. With this status confirmation system, by checking the still frame image data that shows the scene of the abnormality in the target data, it becomes possible to identify the cause of the state in which contents are not contained in the pocket. [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2024-72365 [Overview of the project] [Problems that the invention aims to solve]

[0007] Incidentally, even when a product is judged to be defective by an inspection method, the timing of the abnormal condition (the timing of the abnormality) is not necessarily constant and can vary. For example, the timing of the abnormality will differ between a case where a product is judged to be defective because the contents were not initially contained in the pocket due to a filling error, and a case where a product is judged to be defective because the contents were initially contained in the pocket but then spilled out due to a transport error after filling. Naturally, the timing of the contents spilling out of the pocket is also not constant.

[0008] On the other hand, the system described in Patent Document 1 is triggered by a defect detection by the inspection means, and as a result, the extracted target data is based on the assumption that the timing of the anomaly occurrence is constant. Therefore, there may be variations in the position of the static frame image data (data at the time of anomaly occurrence) that indicates the scene of the anomaly occurring within the target data. Consequently, the process of finding the data at the time of anomaly occurrence within the target data may be time-consuming and laborious.

[0009] In recent years, the manufacturing speed of blister sheets using blister packaging machines has become increasingly faster, resulting in faster conveying speeds for container film and contents, as well as faster filling speeds for contents. As a result, one second of video data may consist of numerous still frame images, which can make the process of finding data indicating an anomaly more cumbersome.

[0010] This invention has been made in view of the above circumstances, and its purpose is to provide an anomaly data recording system that makes it easier to find anomaly data from among the target data. [Means for solving the problem]

[0011] Below, we will describe, in separate sections, each means suitable for achieving the above objectives. Furthermore, we will add notes on the effects and benefits specific to each means as needed.

[0012] Means 1. An abnormality data recording system applied to a blister packaging machine for manufacturing a blister sheet, which is formed by attaching a cover film to a container film filled with contents in a pocket portion so as to close the opening of the pocket portion, and which records video data related to abnormalities in the filling and / or transport of contents that are filled in the pocket portion and transported along with the transport of the container film, A video recording means for capturing images of at least the contents filled in the pocket and / or the contents transported together with the container film, and obtaining video data consisting of multiple still frame image data stored in chronological order, A ring buffer for storing the video data obtained by the video recording means, A trigger generating means that generates a trigger when predetermined conditions are met, An encoder capable of generating the phase related to the transport of contents, The trigger generating means generates a trigger, and the target data storage means extracts and stores target data for a predetermined time period before and after the trigger generation from the video data stored in the ring buffer, when the trigger generating means generates a trigger, The trigger generation means is A neural network having an encoding unit that extracts features from input image data and a decoding unit that reconstructs image data from the features, is trained as training data to generate an object presence identification means, which consists of image data corresponding to the still frame image data when a predetermined phase is generated by the encoder, where the pocket contains contents and there are no abnormalities related to the filling and transport of the contents. A reconstructed image data acquisition means capable of acquiring reconstructed image data by inputting the still frame image data, which is the video data obtained by the video recording means when a predetermined phase is generated by the encoder, into the contents identification means, and reconstructing the image data. The system includes an abnormality determination means that determines whether or not there is an abnormality related to the filling and / or transport of the contents by comparing the reconstructed image data and the inspection image data, The system is configured to generate the trigger when the abnormality detection means determines that an abnormality exists. The trigger generation means includes a neural network having an encoding unit that extracts features from input image data and a decoding unit that reconstructs image data from the features, and a contentless identification means that generates contentless content identification by training the neural network with image data corresponding to the static frame image data when a predetermined phase is generated by the encoder, and only image data in a state where there is no content. The reconstructed image data acquisition means can acquire, as first reconstructed image data, the image data reconstructed by inputting the inspection image data into the content presence identification means, and as second reconstructed image data, the image data reconstructed by inputting the inspection image data into the content absence identification means. The abnormality determination means is configured to determine whether or not there is an abnormality related to the filling and / or transport of the contents, based on the area that coincides between the area showing the difference between the first reconstructed image data and the inspection image data obtained by comparing the first reconstructed image data and the area showing the difference between the second reconstructed image data and the inspection image data. An abnormal data recording system characterized by the following:

[0013] Furthermore, the "training data" may be image data obtained by actually photographing the contents during the manufacturing of the blister sheet, showing no abnormalities related to the filling and transport of the contents, or it may be virtual image data generated using such image data, showing no abnormalities related to the filling and transport of the contents.

[0014] Furthermore, the above-mentioned "neural network" includes, for example, a convolutional neural network having multiple convolutional layers. The above-mentioned "learning" includes, for example, deep learning. The above-mentioned "contents presence identification means (generative model)" includes, for example, an autoencoder or a convolutional autoencoder (the same applies to the "contents absence identification means" of means 2 described later).

[0015] In addition, since the "contents identification means" is generated by training only on image data in which contents are contained in the pocket and there are no abnormalities related to the filling and transport of the contents, the reconstructed image data generated when inspection image data with abnormal parts (for example, image data in which the contents are not contained in the pocket or the contents are outside the pocket) is input to the contents identification means will be almost identical to the inspection image data with the abnormal parts corrected. In other words, when there are abnormal parts in the inspection image data, the reconstructed image data will be a hypothetical image data that assumes there are no abnormal parts (for example, image data in which the contents are contained in the pocket and there are no contents outside the pocket).

[0016] According to the above-described means 1, the trigger generating means generates a trigger when the abnormality detection means determines that there is an abnormality related to the filling and / or transport of the contents. Therefore, the timing of trigger generation can be synchronized with the timing of an abnormal condition (such as when the contents are not contained in the pocket or when the contents remain on the blister packaging machine).

[0017] Furthermore, when the trigger generating means generates a trigger, the target data storage means extracts and stores target data for a predetermined time period before and after the trigger from the video data. As a result, variations in the position of still frame image data (data at the time of the anomaly) indicating the occurrence of the anomaly within the target data are less likely to occur. This makes it easier to find the data at the time of the anomaly within the target data. Consequently, it becomes easier to check for filling errors and transport errors of the contents, and to check for residual contents on the blister packaging machine, based on the data at the time of the anomaly and the video data before and after it. As a result, it becomes possible to efficiently identify the cause of the anomaly and recover the contents remaining on the blister packaging machine.

[0019] Since the "contentless identification means" is generated by learning only the image data in the state where there is no content (the state where the pocket part does not contain any content and there is no content protruding from the pocket part from flying out), when inspection image data in which the pocket part is filled with content is input into the contentless identification means, the second reconstructed image data generated will almost match the inspection image data with the content removed from the pocket part. That is, in the inspection image data, when all the pocket parts contain content, virtual image data assuming that there is no content in all the pocket parts is generated as the second reconstructed image data. Also, for example, when inspection image data in which there is content outside one pocket part while there is content in the other pocket parts is input into the contentless identification means, the second reconstructed image data generated is virtual image data assuming that all the content including the content protruding from the pocket part is removed and there is no content in all the pocket parts.

[0020] The above means 1 According to this, the abnormality determination means makes a determination based on the matching area between the area showing the difference between the first reconstructed image data and the inspection image data obtained by comparing them, and the area showing the difference between the second reconstructed image data and the inspection image data obtained by comparing them.

[0021] Here, in the inspection image data, for example, when there is no content in one pocket part while there is no image of the content protruding from the pocket part, a difference occurs in the part of the content that should be accommodated in the one pocket part between the inspection image data and the first reconstructed image data. Therefore, the area related to the content of the one pocket part is obtained as the area showing the difference. On the other hand, between the inspection image data and the second reconstructed image data, a difference occurs in the part of the content accommodated in the pocket parts other than the one pocket part. Therefore, the area related to the content of the pocket parts other than the one pocket part is obtained as the area showing the difference. Thus, since there is no matching area between these areas showing the differences, the abnormality determination means determines that there is no abnormality related to the conveyance of the content.

[0022] On the other hand, in the inspection image data, for example, when the pocket part of 1 does not contain the content and the content that has protruded from the pocket part is shown, there will be a difference between the inspection image data and the first reconstructed image data in the part of the content that should be accommodated in the pocket part of 1 and the part of the content that has protruded from the pocket part. Therefore, as the region indicating the difference, the regions of these contents are obtained. On the other hand, between the inspection image data and the second reconstructed image data, a difference occurs between the part of the content accommodated in the pocket part other than the pocket part of 1 and the part of the content that has protruded from the pocket part. Therefore, as the region indicating the difference, the regions of these contents are obtained. Therefore, between these regions indicating the difference, the region of the content that has protruded from the pocket part will match. Therefore, the abnormality determination means determines that there is an abnormality related to the filling and / or conveyance of the content.

[0023] Therefore, according to the above means 1, the trigger does not occur simply because the pocket part does not contain the content, and the trigger occurs only when there is content outside the pocket part, such as when the content has protruded from the pocket part. Therefore, based on the data at the time of abnormality occurrence and the video data before and after it, it is possible to more intensively confirm, in particular, filling mistakes and conveyance mistakes of the content that have caused a state where there is content outside the pocket part, and confirm the residue of the protruded content on the blister packaging machine.

[0024] Means 2 . This system is applied to a blister packaging machine for manufacturing blister sheets, which are formed by attaching a cover film to a container film filled with contents in a pocket portion, so as to close the opening of the pocket portion, and is an abnormality data recording system for recording video data related to abnormalities in the filling and / or transport of contents that are filled in the pocket portion and transported along with the transport of the container film, A video recording means for capturing images of at least the contents filled in the pocket and / or the contents transported together with the container film, and obtaining video data consisting of multiple still frame image data stored in chronological order, A ring buffer for storing the video data obtained by the video recording means, A trigger generating means that generates a trigger when predetermined conditions are met, An encoder capable of generating the phase related to the transport of contents, The trigger generating means generates a trigger, and the target data storage means extracts and stores target data for a predetermined time period before and after the trigger generation from the video data stored in the ring buffer, when the trigger generating means generates a trigger, The trigger generation means is A neural network having an encoding unit that extracts features from input image data and a decoding unit that reconstructs image data from the features, is trained as training data to generate an object presence identification means, which consists of image data corresponding to the still frame image data when a predetermined phase is generated by the encoder, where the pocket contains contents and there are no abnormalities related to the filling and transport of the contents. A reconstructed image data acquisition means capable of acquiring reconstructed image data by inputting the still frame image data, which is the video data obtained by the video recording means when a predetermined phase is generated by the encoder, into the contents identification means, and reconstructing the image data. The system includes an abnormality determination means that determines whether or not there is an abnormality related to the filling and / or transport of the contents by comparing the reconstructed image data and the inspection image data, The system is configured to generate the trigger when the abnormality detection means determines that an abnormality exists. Multiple means are provided to identify whether the contents are present. The multiple contents identification means correspond to each of the multiple phases generated by the encoder, The reconstructed image data acquisition means is characterized in that it is configured to acquire the reconstructed image data as reconstructed image data by inputting the inspection image data in the same phase as the corresponding phase to the content presence identification means. The difference A continuous data recording system.

[0025] The above means of transportation 2 According to this, multiple abnormality detection checks can be performed by the abnormality detection means during one phase change of the encoder. Therefore, abnormalities that occur in a very short time (for example, contents flying out of the pocket at high speed, or contents flying in and instantly replacing the contents in the pocket) can be detected with greater accuracy. As a result, target data can be obtained more reliably when abnormalities related to the filling and / or transport of contents occur.

[0026] Furthermore, the above means 1 The above means for the technical matters of 2 When combining the technical aspects, the contentless identification means is the same as the above means 2 It is configured similarly to the content presence identification means. That is, multiple content absence identification means are provided, and each of the multiple content absence identification means corresponds to one of the multiple phases generated by the encoder.

[0027] means 3 . This system is applied to a blister packaging machine for manufacturing blister sheets, which are formed by attaching a cover film to a container film filled with contents in a pocket portion, so as to close the opening of the pocket portion, and is an abnormality data recording system for recording video data related to abnormalities in the filling and / or transport of contents that are filled in the pocket portion and transported along with the transport of the container film, A video recording means for capturing images of at least the contents filled in the pocket and / or the contents transported together with the container film, and obtaining video data consisting of multiple still frame image data stored in chronological order, A ring buffer for storing the video data obtained by the video recording means, A trigger generating means that generates a trigger when predetermined conditions are met, An encoder capable of generating the phase related to the transport of contents, The trigger generating means generates a trigger, and the target data storage means extracts and stores target data for a predetermined time period before and after the trigger generation from the video data stored in the ring buffer, when the trigger generating means generates a trigger, The trigger generation means is A neural network having an encoding unit that extracts features from input image data and a decoding unit that reconstructs image data from the features, is trained as training data to generate an object presence identification means, which consists of image data corresponding to the still frame image data when a predetermined phase is generated by the encoder, where the pocket contains contents and there are no abnormalities related to the filling and transport of the contents. A reconstructed image data acquisition means capable of acquiring reconstructed image data by inputting the still frame image data, which is the video data obtained by the video recording means when a predetermined phase is generated by the encoder, into the contents identification means, and reconstructing the image data. The system includes an abnormality determination means that determines whether or not there is an abnormality related to the filling and / or transport of the contents by comparing the reconstructed image data and the inspection image data, The system is configured to generate the trigger when the abnormality detection means determines that an abnormality exists. The system is characterized by including a chapter marking means that adds a chapter mark to the location in the target data corresponding to the timing when the trigger generating means generates a trigger. The difference A continuous data recording system.

[0028] The above means of transportation 3 According to the report, using chapter markers makes it easier to find data where anomalies occurred within the target data. This makes it easier to check for errors in filling or transporting contents, as well as to check for residual contents on the blister packaging machine.

[0029] means 4 The means is characterized by comprising an abnormal region mark setting means for setting an abnormal region mark in the still frame image data constituting the target data, which indicates an abnormal region determined by the abnormality determination means to be abnormal. Any of 1 through 3 The abnormal data recording system described above.

[0030] The above means of transportation 4 According to this method, abnormal area marks can be added to the still frame image data of the target data to indicate areas that have been determined to be abnormal by the abnormality detection means (for example, areas corresponding to contents that should be present or contents that are outside the pocket area). Therefore, abnormal parts can be found more easily, and consequently, the cause of the abnormality and the recovery of spilled contents can be carried out more efficiently.

[0031] Furthermore, the technical aspects related to the above means may be combined as appropriate. Therefore, for example, the above means 1 Regarding the technical matters related thereto, the above means 2~4 At least one of the technical matters related to this may be combined. [Brief explanation of the drawing]

[0032] [Figure 1]This is a perspective view showing a PTP sheet. [Figure 2] This is a partially enlarged cross-sectional view of a PTP sheet. [Figure 3] This is a perspective view showing PTP film. [Figure 4] This is a schematic diagram showing the general configuration of a PTP packaging machine and the like. [Figure 5] This is a block diagram illustrating the schematic configuration of the abnormal data recording system. [Figure 6] This is an explanatory diagram for describing video data stored in a ring buffer. [Figure 7] This is a schematic diagram showing still frame image data with abnormal area markers. [Figure 8] This is a schematic diagram showing the training data used to train an AI model that includes tablets. [Figure 9] This is a schematic diagram illustrating the structure of a neural network. [Figure 10] This is a flowchart showing the learning process of a neural network. [Figure 11] This is a schematic diagram showing an example of image data for inspection. [Figure 12] This is a schematic diagram showing an example of reconstructed image data. [Figure 13] This is a schematic diagram showing an example of image data for inspection. [Figure 14] This block diagram shows the parts of the abnormal data recording system that are particularly related to judgment processing and trigger generation. [Figure 15] This is a flowchart showing the flow of the decision-making process. [Figure 16] This is a schematic diagram showing an example of differential image data. [Figure 17] In the second embodiment, this is a block diagram showing the part of the abnormal data recording system that is particularly related to the determination process and trigger generation. [Figure 18] This block diagram shows the third embodiment, specifically the part of the abnormal data recording system related to the determination process and trigger generation. [Figure 19]This is a schematic diagram showing the training data used to train an AI model without the need for tablets. [Figure 20] This flowchart shows the flow of the determination process in the third embodiment. [Figure 21] This is a schematic diagram showing an example of inspection image data in the third embodiment. [Figure 22] This is a schematic diagram showing an example of the first reconstructed image data in the third embodiment. [Figure 23] This is a schematic diagram showing an example of first differential image data in the third embodiment. [Figure 24] This is a schematic diagram showing an example of first differential image data in the third embodiment. [Figure 25] This is a schematic diagram showing an example of second reconstructed image data in the third embodiment. [Figure 26] This is a schematic diagram showing an example of second difference image data in the third embodiment. [Figure 27] This is a schematic diagram showing an example of second difference image data in the third embodiment. [Figure 28] This is a schematic diagram showing an example of additive image data in the third embodiment. [Figure 29] This is a schematic diagram showing an example of additive image data in the third embodiment. [Figure 30] This is a schematic diagram showing still frame image data with abnormal region marks in the third embodiment. [Figure 31] This block diagram shows a schematic configuration of an abnormal data recording system in another embodiment. [Modes for carrying out the invention]

[0033] The embodiments will be described below with reference to the drawings. [First Embodiment] First, let's explain the structure of the PTP sheet as a "blister sheet". As shown in Figures 1 and 2, the PTP sheet 1 has a container film 3 with multiple pockets 2 and a cover film 4 attached to the container film 3 so as to close the pockets 2.

[0034] The container film 3 is made of a transparent thermoplastic resin material such as PP (polypropylene) or PVC (polyvinyl chloride). On the other hand, the cover film 4 is made of an opaque material (such as aluminum foil) with a sealant made of polypropylene resin or the like applied to its surface. Of course, the materials of each film 3 and 4 are not limited to these, and other materials may be used.

[0035] The PTP sheet 1 has a rectangular shape when viewed from above. The PTP sheet 1 has two rows of pockets, each consisting of three pocket sections 2 arranged along its longitudinal direction, and two rows of pockets arranged along its short direction. Each pocket section 2 contains one tablet 5, which serves as the "contents".

[0036] Furthermore, the PTP sheet 1 is manufactured by punching out a strip-shaped PTP film 6 (see Figure 3) formed from a strip-shaped container film 3 and a strip-shaped cover film 4. In this embodiment, the PTP film 6 corresponds to a "blister film".

[0037] Next, the general configuration of the PTP packaging machine 10 for manufacturing the PTP sheet 1 described above will be explained. In this embodiment, the PTP packaging machine 10 corresponds to a "blister packaging machine".

[0038] As shown in Figure 4, at the upstream end of the PTP packaging machine 10, a strip of container film 3 is wound into a roll. The end of the roll of container film 3 is guided by a guide roll 13. Downstream of the guide roll 13, the container film 3 is mounted on an intermittent feed roll 14. The intermittent feed roll 14 intermittently conveys the container film 3.

[0039] Between the guide roll 13 and the intermittent feed roll 14, a preheating device 15 and a pocket forming device 16 are arranged along the transport path of the container film 3. When the container film 3 is preheated by the preheating device 15 and becomes relatively flexible, the pocket forming device 16 forms a plurality of pockets 2 at predetermined positions on the container film 3. The formation of the pockets 2 takes place during the intervals between the transport operations of the container film 3 by the intermittent feed roll 14.

[0040] The container film 3, fed from the intermittent feed roll 14, is mounted in the following order: tension roll 18, guide roll 19, and film receiving roll 20. The film receiving roll 20 is operated by a predetermined motor M2 to transport the container film 3 continuously and at a constant speed. The tension roll 18 prevents slack in the container film 3 due to differences in transport operation between the intermittent feed roll 14 and the film receiving roll 20, thereby keeping the container film 3 constantly taut.

[0041] Between the guide roll 19 and the film receiving roll 20, a filling device 22 is positioned along the transport path of the container film 3. The filling device 22 includes, for example, a cylindrical chute for accommodating tablets 5 in a single row, and a shutter (not shown) that can open and close the exit of the chute. The device fills the pocket section 2 with tablets 5 by opening the shutter at a predetermined timing. However, if a filling error occurs, such as a temporary jamming of tablets 5 in the chute or a malfunction of the shutter, there is a risk that the pocket section 2 will not contain tablets 5, or that tablets 5 will fall out of the pocket section 2.

[0042] Meanwhile, the raw material of the strip-shaped cover film 4 is wound into a roll at the upstream end. The end of the roll-shaped cover film 4 is guided toward the heating roll 25 by a guide roll 24.

[0043] The heating roll 25 is pressable against the film receiving roll 20, and the container film 3 and cover film 4 are fed between the two rolls 20 and 25. As the two films 3 and 4 pass between the two rolls 20 and 25 in a heated and pressed state, the cover film 4 is attached to the container film 3, and the pocket portions 2 are sealed with the cover film 4. This produces a strip-shaped PTP film 6 in which tablets 5 are contained in each pocket portion 2. However, if an error occurs in the transport of the tablets 5, such as abnormal vibration of the container film 3 or contact of the container film 3 with components of the PTP packaging machine 10, between the filling device 22 and the two rolls 20 and 25, there is a risk that the tablets 5 may not be contained in the pocket portions 2 or may fall out of the pocket portions 2.

[0044] The PTP film 6, fed from the film receiving roll 20, is mounted on the tension roll 27 and then the intermittent feed roll 28 in that order. The intermittent feed roll 28 intermittently transports the PTP film 6. The tension roll 27 prevents the PTP film 6 from slackening due to the difference in transport operation between the film receiving roll 20 and the intermittent feed roll 28, and keeps the PTP film 6 in a constantly taut state.

[0045] The PTP film 6 fed from the intermittent feed roll 28 is mounted on the tension roll 31 and then the intermittent feed roll 32 in that order. The intermittent feed roll 32 intermittently transports the PTP film 6. The tension roll 31 prevents the PTP film 6 from slackening between the intermittent feed rolls 28 and 32.

[0046] Between the intermittent feed roll 28 and the tension roll 31, a slitting device 33 and an engraving device 34 are arranged along the transport path of the PTP film 6. The slitting device 33 forms separation slits at predetermined positions on the PTP film 6. The engraving device 34 engraves markings at predetermined positions (e.g., tag areas) on the PTP film 6. Note that the separation slits and engravings are not shown in Figure 1, etc.

[0047] The PTP film 6, fed from the intermittent feed roll 32, is then mounted downstream on the tension roll 35 and the continuous feed roll 36 in that order. Between the intermittent feed roll 32 and the tension roll 35, a sheet punching device 37 is positioned along the transport path of the PTP film 6. The sheet punching device 37 has the function of punching out the outer edge of the PTP film 6 into units of PTP sheets, that is, separating the PTP sheet 1 from the PTP film 6.

[0048] The PTP sheets 1 obtained by the sheet punching machine 37 are transported by the conveyor 39 and temporarily stored in the finished product hopper 40. However, defective PTP sheets 1 are not sent to the finished product hopper 40 but are discharged separately by a defective sheet discharge mechanism (not shown).

[0049] A cutting device 42 is located downstream of the continuous feed roll 36. The scrap 43 remaining in strip form after punching by the sheet punching device 37 is guided to the tension roll 35 and the continuous feed roll 36, and then to the cutting device 42. The cutting device 42 cuts the scrap 43 to predetermined dimensions. The cut scrap 43 is stored in the scrap hopper 44 and then disposed of.

[0050] Next, the abnormal data recording system applied to the PTP packaging machine 10 described above will be explained. The abnormal data recording system is for recording video data related to abnormalities in the filling and / or transport of tablets 5 that are filled into the pocket section 2 and transported along with the transport of the container film 3. These abnormalities include abnormalities in the position of tablets 5 caused by filling errors or transport errors, such as abnormalities in the pocket section 2 not containing tablets 5, or abnormalities in the container film 3 (especially the sheet portion other than the pocket section 2) or tablets 5 remaining on the PTP packaging machine 10. As shown in Figure 5, the abnormal data recording system 50 includes an encoder 51, a camera 52, and a control device 60. In this embodiment, the camera 52 corresponds to the "video recording means".

[0051] The encoder 51 is used to acquire the phase related to the motor M2. In this embodiment, the encoder 51 is set so that each time the motor M2 operates continuously and the container film 3 is transported by a predetermined length (the length of one PTP sheet 1), the acquired phase advances by 360° and the rotation speed increases by 1. Therefore, the phase acquired by the encoder 51 corresponds to the amount of container film 3 transported by the motor M2.

[0052] Camera 52 is positioned to correspond to the filling position of the tablet 5 by the filling device 22 (see Figure 4). Camera 52 obtains video data by taking two-dimensional images of the tablet 5 being filled into the pocket section 2 and the tablet 5 being transported along with the transport of the container film 3. In other words, camera 52 obtains video data by taking two-dimensional images of the tablet 5 being filled into the pocket section 2 and the tablet 5 being transported together with the container film 3. The video data consists of multiple still frame image data stored in chronological order, and in this embodiment, one second of video data consists of multiple (for example, 60) still frame image data.

[0053] Furthermore, the camera 52 starts recording video data in conjunction with the activation of the encoder 51 and stops recording video data in conjunction with the deactivation of the encoder 51. As a result, the multiple still frame image data that make up the video data are associated with the rotation speed and phase of the encoder 51.

[0054] The video data obtained by camera 52 is input to control device 60 and stored in ring buffer 61, which will be described later (see Figure 6). In Figure 6, the data name consisting of time and frame number is shown as the still frame image data that constitutes the video data. In this embodiment, the frame number is a number from 0 to 59, for example, and indicates the acquisition order of multiple (for example, 60) still frame image data that constitute one second of video data. The time is obtained using the clock function of camera 52 or control device 60.

[0055] The control device 60 is responsible for controlling the operation of each device in the PTP packaging machine 10 and the abnormal data recording system 50. The control device 60 includes a CPU as a means of calculation, ROM for storing various programs, RAM for temporarily storing various data such as calculation data and input / output data, a storage medium for long-term storage of various data, an input device for inputting information (e.g., a keyboard), and a display device for displaying various information (e.g., a liquid crystal display).

[0056] As shown in Figure 6, the control device 60 includes a ring buffer 61, a trigger generation unit 62, a target data storage unit 63, and a learning unit 64. In this embodiment, the trigger generation unit 62 corresponds to the "trigger generation means," and the target data storage unit 63 corresponds to the "target data storage means."

[0057] The ring buffer 61 is composed of the storage medium of the control device 60, and treats a certain information storage area in this storage medium as a ring. The ring buffer 61 is configured so that after data has been stored in all information storage areas, it returns to the first information storage area and overwrites the data. The ring buffer 61 stores video data obtained by the camera 52, and in the ring buffer 61, new video data obtained by the camera 52 is sequentially stored, while old video data is sequentially erased.

[0058] The trigger generation unit 62 generates a trigger when predetermined conditions are met. The trigger generation unit 62 will be described later.

[0059] When the trigger generation unit 62 generates a trigger, the target data storage unit 63 extracts and saves target data for a predetermined time (for example, several seconds) from the video data stored in the ring buffer 61 around the time the trigger was generated (see Figure 6). The target data to be saved is video data consisting of multiple still frame image data, and is considered to be necessary for identifying the cause and timing of a malfunction. The length of the target data (the predetermined time mentioned above) can be changed as appropriate.

[0060] Furthermore, the target data storage unit 63 includes a chapter mark assignment unit 63a and an abnormal area mark setting unit 63b. In this embodiment, the chapter mark assignment unit 63a corresponds to the "chapter mark assignment means," and the abnormal area mark setting unit 63b corresponds to the "abnormal area mark setting means."

[0061] The chapter mark assignment unit 63a assigns chapter marks to the locations in the target data corresponding to the timing when the trigger generation unit 62 generates a trigger. The chapter marks indicate the divisions of the target data. In this embodiment, by using chapter marks, operators can view the target data starting from the still frame image data that constitutes the target data, particularly the one that triggered the trigger generation unit 62 to generate a trigger (i.e., the one used as inspection image data in the judgment process by the abnormality judgment unit 62c described later).

[0062] The abnormal area mark setting unit 63b provides an abnormal area mark Mk (see Figure 7) to the static frame image data Sf constituting the target data, indicating the area that the abnormality determination unit 62c (described later) has determined to be abnormal. In this embodiment, among the multiple static frame image data Sf constituting the target data, the one that triggered the trigger generation unit 62 (used as inspection image data in the determination process described later) has an abnormal area mark Mk (for example, a circular mark) provided in the part corresponding to the "abnormal part" identified by the abnormality determination unit 62c. The form of the abnormal area mark Mk may be changed as appropriate.

[0063] The learning unit 64 is a functional unit that uses training data to train a deep neural network 190 (hereinafter simply referred to as "neural network 190"; see Figure 9) and constructs an AI (Artificial Intelligence) model 200 that serves as a "means for identifying whether a tablet contains contents."

[0064] The tablet-containing AI model 200 is a generative model constructed by deep learning a neural network 190 using image data Gd1 (see Figure 8) that corresponds to static frame image data Sf when a predetermined phase (e.g., 0°) is generated by the encoder 51, where a tablet 5 is contained in the pocket section 2, and there are no abnormalities related to the filling and transport of the tablet 5. It has the structure of a so-called autoencoder. As described above, the encoder 51 advances its phase by 360° each time the container film 3 is transported by a predetermined length (the length of one PTP sheet 1). Therefore, the position of the pocket section 2 remains almost constant between static frame image data Sf when a predetermined phase (e.g., 0°) is generated by the encoder 51.

[0065] Here, the structure of the neural network 190 will be explained with reference to Figure 9. Figure 9 is a schematic diagram conceptually showing the structure of the neural network 190. As shown in Figure 9, the neural network 190 has the structure of a convolutional auto-encoder (CAE), comprising an encoder unit 191 as an "encoding unit" that extracts feature quantities (latent variables) TA from the input image data GA, and a decoder unit 192 as a "decoding unit" that reconstructs image data GB from the feature quantities TA.

[0066] The structure of the convolutional autoencoder is well known, so a detailed explanation will be omitted. The encoder unit 191 has multiple convolutional layers 193, and in each convolutional layer 193, the result of a convolution operation using multiple filters (kernels) 194 on the input data is output as input data for the next layer. Similarly, the decoder unit 192 has multiple deconvolutional layers 195, and in each deconvolutional layer 195, the result of a deconvolution operation using multiple filters (kernels) 196 on the input data is output as input data for the next layer. Then, in the learning process described below, the weights (parameters) of each filter 194,196 are updated.

[0067] Here, we will explain the learning process performed when generating the AI ​​model 200 with tablets. First, prior to the learning process, a large amount of learning data Gd1 is prepared. In this embodiment, the learning data Gd1 is still frame image data Sf from video data obtained when the camera 52 actually photographs the tablets 5, etc., and a predetermined phase (e.g., 0°) is generated by the encoder 51. This still frame image data Sf is free from any abnormalities related to the filling and transport of the tablets 5. In other words, the learning data Gd1 is such that tablets 5 are contained in each pocket 2, while no tablets 5 exist outside the pocket 2. Note that the learning data Gd1 may be virtually generated (virtual image data) rather than data obtained by the camera 52 actually photographing the tablets 5, etc.

[0068] As shown in Figure 10, in the learning process, first, in step S201, an untrained neural network 190 is prepared. For example, a neural network 190 that has been pre-stored in a predetermined storage device is read. Alternatively, the neural network 190 is constructed based on network configuration information (for example, the number of layers in the neural network and the number of nodes in each layer) stored in the storage device.

[0069] Next, in step S202, reconstructed image data is acquired. That is, the pre-prepared training data Gd1 is fed as input data to the input layer of the neural network 190. Then, the reconstructed image data output from the output layer of the neural network 190 is acquired.

[0070] In the following step S203, the training data Gd1 is compared with the reconstructed image data output by the neural network 190 in step S202, and it is determined whether the error is sufficiently small (whether it is below a predetermined threshold).

[0071] If the error is sufficiently small, step S205 determines whether the learning process termination conditions are met. For example, if a certain number of consecutive affirmative judgments are made in step S203 without going through the process of step S204 described later, or if learning using all of the prepared learning data is repeated a certain number of times, it is determined that the termination conditions are met. If the termination conditions are met, the neural network 190 and its learning information (updated parameters, etc., described later) are stored in the AI ​​memory unit 62b described later as the AI ​​model 200 with tablets, and the learning process is terminated.

[0072] On the other hand, if the termination condition is not met in step S205, the process returns to step S202 and the neural network 190 is trained again.

[0073] Furthermore, if the error is not sufficiently small in step S203, the network update process (training of the neural network 190) is performed in step S204, and then the process returns to step S202, repeating the above series of processes.

[0074] In the network update process of step S204, the weights (parameters) of each filter 194,196 in the neural network 190 are updated to more appropriate values, using a known learning algorithm such as backpropagation, so as to minimize the loss function that represents the difference between the training data Gd1 and the reconstructed image data. For example, BCE (Binary Cross-entropy) can be used as the loss function.

[0075] By repeatedly performing steps S202 to S204, the neural network 190 minimizes the error between the training data Gd1 and the reconstructed image data, resulting in the output of more accurate reconstructed image data.

[0076] The final AI model 200 with tablets will have inspection image data Kg, which is static frame image data Sf from the video data obtained by the camera 52 when a predetermined phase (e.g., 0°) is generated by the encoder 51. When data without abnormalities related to the filling and transport of tablets 5 (see Figure 11) is input, it will generate reconstructed image data Sk (see Figure 12) that is almost identical to the input inspection image data Kg.

[0077] On the other hand, when the AI ​​model 200 with tablets receives inspection image data Kg (see Figure 13) in which there are abnormalities related to the filling and / or transport of tablets 5, such as pockets 2 that are not filled with tablets 5 or tablets 5 that have popped out of pockets 2, the model generates reconstructed image data Sk (see Figure 12) that is corrected for noise (abnormal parts) and closely matches the input image data. In other words, if there are abnormalities related to the filling and / or transport of tablets 5 in the inspection image data Kg, a virtual image data is generated as the reconstructed image data Sk, assuming that there are no abnormalities.

[0078] Furthermore, it is not necessary to perform the above learning process each time the control device 60 is manufactured. The neural network 190 and its learning information (updated parameters, etc.) may be acquired in advance and stored in the AI ​​memory unit 62b of the trigger generation unit 62 as the AI ​​model 200 with a tablet.

[0079] Next, the trigger generation unit 62 will be described. As shown in Figure 5, the trigger generation unit 62 includes a reconstructed image data acquisition unit 62a, an AI storage unit 62b, and an anomaly determination unit 62c. In this embodiment, the reconstructed image data acquisition unit 62a corresponds to the "reconstructed image data acquisition means," and the anomaly determination unit 62c corresponds to the "anomaly determination means."

[0080] The reconstructed image data acquisition unit 62a causes the abnormality determination unit 62c to execute the determination process described later when a predetermined phase (for example, 0°) is generated by the encoder 51, and functions like a switch that is activated by the output from the encoder 51 (see Figure 14).

[0081] When the encoder 51 generates a predetermined phase (e.g., 0°), the reconstructed image data acquisition unit 62a inputs the inspection image data Kg obtained by the camera 52 at the time the encoder 51 generated that phase into the tablet AI model 200 and acquires the reconstructed image data as reconstructed image data Sk. More specifically, the reconstructed image data acquisition unit 62a provides the inspection image data Kg obtained by the camera 52 as input data to the input layer of the neural network 190. The reconstructed image data acquisition unit 62a then acquires the image data output from the output layer of the neural network 190 as reconstructed image data Sk. The acquired reconstructed image data Sk is input to the anomaly determination unit 62c.

[0082] Furthermore, when the encoder 51 generates a predetermined phase (for example, 0°), the reconstructed image data acquisition unit 62a inputs the inspection image data Kg obtained by the camera 52 directly to the abnormality determination unit 62c. Therefore, each time the encoder 51 generates a predetermined phase (0°), the inspection image data Kg and the reconstructed image data Sk obtained based on this inspection image data Kg are input to the abnormality determination unit 62c.

[0083] The AI ​​memory unit 62b is composed of the storage medium of the control device 60 and stores the AI ​​model 200 with tablets.

[0084] The abnormality detection unit 62c compares the input reconstructed image data Sk and the inspection image data Kg to determine whether there are any abnormalities related to the filling and / or transport of the tablet 5. When the abnormality detection unit 62c determines that there is an abnormality, it generates a trigger. This trigger is input to the target data storage unit 63.

[0085] Now, referring to Figures 14 and 15, we will explain the determination process performed by the abnormality determination unit 62c to determine whether or not there are any abnormalities related to the filling and / or transport of the tablets 5.

[0086] When the encoder 51 generates a predetermined phase (e.g., 0°), the determination process begins. First, in step S301, the reconstructed image data acquisition unit 62a inputs the inspection image data Kg obtained by the camera 52 when the predetermined phase (e.g., 0°) is generated into the tablet AI model 200. This acquires the reconstructed image data Sk. For example, if the inspection image data Kg does not contain a tablet 5 in the pocket 2 and the tablet 5 is sticking out of the pocket 2 (see Figure 13), the reconstructed image data Sk is obtained with the empty pocket 2 filled with a tablet 5 and the tablet 5 that has stuck out of the pocket 2 removed (see Figure 12).

[0087] Next, in step S302, the abnormality determination unit 62c compares the inspection image data Kg obtained by the camera 52 with the reconstructed image data Sk obtained based on the inspection image data Kg, and calculates the difference in brightness for each pixel of both data Kg and Sk. Pixels whose differences are outside a predetermined tolerance range are identified as difference pixels, and difference image data Sb (see Figure 16) representing these difference pixels is obtained. For example, if the inspection image data Kg is as shown in Figure 13 and the reconstructed image data Sk is as shown in Figure 12, the part corresponding to the tablet 5 that should be contained in the pocket 2 of 1 and the part corresponding to the tablet 5 that has popped out of the pocket 2 are identified as difference pixels, and the difference image data Sb representing these difference pixels is obtained as shown in Figure 16.

[0088] Next, in step S303, the area of ​​the connected component of the difference pixels in the difference image data Sb (defective area) is calculated.

[0089] Next, in step S304, it is determined whether the maximum area among the calculated areas is greater than or equal to a predetermined threshold. In other words, it is determined whether the maximum area is outside the acceptable range. If the maximum area is greater than or equal to the threshold, it is determined that there is an abnormality related to the filling and / or transport of the tablets 5. Then, the portion corresponding to the connected component of the difference pixels in the inspection image data Kg is identified as the "abnormal portion," and the determination process is terminated.

[0090] On the other hand, if the maximum area falls below the threshold, in step S306, it is determined that there are no abnormalities related to the filling and transport of the tablets 5, and the determination process is terminated. Note that the determination method described above is just one example, and the determination method may be changed as appropriate.

[0091] Then, if the abnormality detection unit 62c determines that there is an abnormality (step S305; Yes), the abnormality detection unit 62c (trigger generation unit 62) generates a trigger. Upon the generation of the trigger, the target data storage unit 63 extracts and saves target data for a predetermined time period before and after the generation of the trigger from the video data stored in the ring buffer 61. When saving the target data, chapter marks are added by the chapter mark assignment unit 63a and abnormal area marks Mk are set by the abnormal area mark setting unit 63b.

[0092] As described in detail above, according to this embodiment, the trigger generation unit 62 generates a trigger when the abnormality determination unit 62c determines that there is an abnormality related to the filling and / or transport of the tablets 5. Therefore, the timing of trigger generation can be synchronized with the timing of an abnormal condition (for example, when the tablets 5 are not contained in the pocket 2 or when the tablets 5 remain on the PTP packaging machine 10).

[0093] Furthermore, when the trigger generation unit 62 generates a trigger, the target data storage unit 63 extracts and stores target data for a predetermined time period before and after the trigger from the video data. As a result, variations in the position of still frame image data Sf (data at the time of abnormality occurrence), which indicates the scene of the abnormality, within the target data become less likely. This makes it easier to find the data at the time of abnormality occurrence within the target data. Consequently, it becomes easier to check for filling errors and transport errors of the tablets 5, and to check for any tablets 5 remaining on the PTP packaging machine 10 that have popped out, based on the data at the time of abnormality occurrence and the video data before and after it. As a result, it becomes possible to efficiently identify the cause of the abnormality and recover any tablets 5 remaining on the PTP packaging machine 10.

[0094] Furthermore, using chapter markers makes it even easier to find data where anomalies occurred within the target data. This makes it easier to check for errors in filling and transporting tablets 5, as well as to check for any residue of tablets 5 that have popped out on the PTP packaging machine 10.

[0095] In addition, an abnormal area mark Mk can be provided on the static frame image data Sf of the target data to indicate the area ("abnormal portion") determined to be abnormal by the abnormality determination unit 62c. Therefore, the abnormal portion can be found more easily, and consequently, the cause of the abnormality and the recovery of the ejected tablet 5 can be performed more efficiently. [Second Embodiment] Next, the second embodiment will be described, focusing on the differences from the first embodiment. In the first embodiment, one tablet-containing AI model 200 is provided, which is generated by training it with image data (training data Gd1) corresponding to the static frame image data Sf when a predetermined phase (e.g., 0°) is generated by the encoder 51. In other words, only one tablet-containing AI model 200 is provided for the predetermined phase (e.g., 0°).

[0096] In contrast, in this second embodiment, as shown in Figure 17, the AI ​​model 200 with a tablet includes an AI model (0°) 200 corresponding to a 0° phase, an AI model (10°) 200 corresponding to a 10° phase, ..., and an AI model (350°) 200 corresponding to a 350° phase. That is, multiple (e.g., 36) AI models 200 with tablets are provided, each corresponding to one of the multiple phases (e.g., 0°, 10°, ..., 350°) generated by the encoder 51.

[0097] Each AI model 200 with a tablet is generated by training it with image data corresponding to the static frame image data Sf generated by the encoder 51 when it generates the phase corresponding to its own. For example, the AI ​​model (0°) 200 with a tablet is generated by training it with image data corresponding to the static frame image data Sf generated by the encoder 51 when it generates a phase of 0°.

[0098] Furthermore, each tablet-containing AI model 200 is trained using only image data without abnormalities related to the filling and transport of the tablets 5, similar to the first embodiment described above. Also, similar to the first embodiment described above, the encoder 51 advances its phase by 360° each time the container film 3 is transported by a predetermined length (the length of one PTP sheet 1). Therefore, when one of the multiple phases is generated by the encoder 51, the position of the pocket portion 2 remains almost constant between static frame image data Sf. Accordingly, for example, when a phase of 10° is generated by the encoder 51, the position of the pocket portion 2 remains almost constant between static frame image data Sf.

[0099] Furthermore, in this second embodiment, an abnormality detection unit 62c is provided for each tablet-containing AI model 200. Therefore, for example, an abnormality detection unit 62c corresponding to the tablet-containing AI model (0°) 200 and an abnormality detection unit 62c corresponding to the tablet-containing AI model (10°) 200 are provided.

[0100] In this second embodiment, when the encoder 51 generates one of several phases (for example, 0°, 10°, ..., 350°), the reconstructed image data acquisition unit 62a acquires reconstructed image data Sk by inputting inspection image data Kg corresponding to that phase to the tablet-containing AI model 200 corresponding to the generated phase. For example, when the encoder 51 generates a phase of 10°, the reconstructed image data acquisition unit 62a acquires reconstructed image data Sk by inputting inspection image data Kg corresponding to the 10° phase to the tablet-containing AI model (10°) 200. The acquired reconstructed image data Sk is input to the abnormality determination unit 62c corresponding to the generated phase.

[0101] Furthermore, when the reconstructed image data acquisition unit 62a generates one of several phases (for example, 0°, 10°, ..., 350°) from the encoder 51, it inputs the inspection image data Kg corresponding to that phase to the anomaly determination unit 62c corresponding to that phase. Therefore, for example, each time the encoder 51 generates a phase of 10°, the inspection image data Kg corresponding to the 10° phase and the reconstructed image data Sk obtained based on this inspection image data Kg are input to the anomaly determination unit 62c corresponding to the 10° phase.

[0102] Then, the abnormality determination unit 62c, similar to the first embodiment, compares the input inspection image data Kg and reconstructed image data Sk to determine whether or not there is an abnormality related to the filling and / or transport of the tablet 5. Therefore, in this second embodiment, the presence or absence of an abnormality is determined each time one of the multiple phases is generated from the encoder 51.

[0103] Then, the abnormality detection unit 62c generates a trigger if it determines that an abnormality exists. Upon the generation of the trigger, the target data storage unit 63 extracts the target data from the ring buffer 61 and stores it.

[0104] In this second embodiment, the ability to input a trigger from each abnormality detection unit 62c to the target data storage unit 63 is switched according to the phase generated by the encoder 51. For example, when a phase of 10° is generated by the encoder 51, the input of a trigger from the abnormality detection unit 62c corresponding to the 10° phase to the target data storage unit 63 is permitted, while the input of a trigger from the abnormality detection unit 62c corresponding to other phases to the target data storage unit 63 is prohibited.

[0105] As described above, according to this second embodiment, the abnormality determination unit 62c can perform determination multiple times while the phase of the encoder 51 changes by one cycle. Therefore, abnormalities that occur in a very short time (for example, a tablet 5 flying out of the pocket 2 at high speed, or a flying tablet 5 instantly swapping places with a tablet 5 in the pocket 2) can be detected with greater accuracy. As a result, target data can be obtained more reliably when an abnormality occurs related to the filling and / or transport of the tablets 5. [Third Embodiment] Next, the third embodiment will be described, focusing on the differences from the first embodiment. In the first embodiment, a tablet 5 is contained in the pocket 2, and an AI model 200 with a tablet is provided, which is generated by training only image data without abnormalities related to the filling and transport of the tablet 5 as training data Gd1. In contrast, in this third embodiment, as shown in Figure 18, in addition to the AI ​​model 200 with a tablet, an AI model 201 without a tablet is provided. The AI ​​model 201 without a tablet corresponds to the "no contents identification means".

[0106] The tablet-free AI model 201 is a generative model constructed by deep learning a neural network 190 using only the image data Gd2 (see Figure 19) that corresponds to the static frame image data Sf when a predetermined phase (e.g., 0°) is generated by the encoder 51, and is the image data when there is no tablet 5. It has the structure of a so-called autoencoder.

[0107] Therefore, when the tablet-free AI model 201 receives an input of an inspection image data Kg containing a tablet 5, it generates a second reconstructed image data Sk2 that closely matches the inspection image data Kg from which the tablet 5 has been removed. For example, when all the pockets 2 in the inspection image data Kg contain a tablet 5, the second reconstructed image data Sk2 generates a hypothetical image data assuming that there are no tablets 5 in any of the pockets 2. Also, for example, when there are tablets 5 outside the pockets 2 in the inspection image data Kg, the second reconstructed image data Sk2 generates a hypothetical image data assuming that all tablets 5, including those outside the pockets 2, have been removed, and that there are no tablets 5 in any of the pockets 2.

[0108] Then, when the encoder 51 generates a predetermined phase (for example, 0°), the reconstructed image data acquisition unit 62a inputs the inspection image data Kg, which is the static frame image data Sf at the time the encoder 51 generated that phase, into the tablet-containing AI model 200 and acquires the reconstructed image data as the first reconstructed image data Sk1 (see Figure 22). The acquired first reconstructed image data Sk1 is input into the first difference calculation unit 62c1 of the abnormality determination unit 62c, which will be described later.

[0109] Furthermore, when the encoder 51 generates a predetermined phase (0°), the reconstructed image data acquisition unit 62a inputs the inspection image data Kg at the time the encoder 51 generated that phase into the tablet-less AI model 201 and acquires the reconstructed image data as the second reconstructed image data Sk2 (see Figure 25). The acquired second reconstructed image data Sk2 is input into the second difference calculation unit 62c2 of the abnormality determination unit 62c, which will be described later.

[0110] Furthermore, when the encoder 51 generates a predetermined phase (for example, 0°), the reconstructed image data acquisition unit 62a inputs the inspection image data Kg at the time the encoder 51 generated that phase to the first difference calculation unit 62c1 and the second difference calculation unit 62c2, respectively. Therefore, each time the encoder 51 generates a predetermined phase (0°), the inspection image data Kg and the first reconstructed image data Sk1 are input to the first difference calculation unit 62c1, and the inspection image data Kg and the second reconstructed image data Sk2 are input to the second difference calculation unit 62c2, respectively.

[0111] In addition, in this third embodiment, the abnormality determination unit 62c includes a first difference calculation unit 62c1, a second difference calculation unit 62c2, an addition unit 62c3, and a final determination unit 62c4.

[0112] The first difference calculation unit 62c1 compares the input inspection image data Kg with the first reconstructed image data Sk1 and calculates the difference in brightness for each pixel of both data. It then identifies pixels whose difference is outside a predetermined tolerance range as difference pixels and obtains first difference image data Sb1 (see Figure 23, etc.) that represents these difference pixels.

[0113] The second difference calculation unit 62c2 compares the input inspection image data Kg with the second reconstructed image data Sk2 and calculates the difference in brightness for each pixel of both data. It then identifies pixels whose difference is outside a predetermined tolerance range as difference pixels and obtains second difference image data Sb2 (see Figure 26, etc.) that shows these difference pixels.

[0114] The addition unit 62c3 obtains added image data Ag (see Figure 28, etc.) by performing a logical AND operation on each pixel of the two difference image data Sb1 and Sb2. The added image data Ag represents the area that shows the difference between the data Sk1 and Kg obtained by comparing the first reconstructed image data Sk1 and the inspection image data Kg (i.e., the difference pixels shown by the first difference image data Sb1) and the area that shows the difference between the data Sk2 and Kg obtained by comparing the second reconstructed image data Sk2 and the inspection image data Kg (i.e., the difference pixels shown by the second difference image data Sb2).

[0115] The final determination unit 62c4 determines whether or not there is an abnormality related to the filling and / or transport of the tablet 5 based on the summation image data Ag. More specifically, if a matching region exists in the summation image data Ag, the final determination unit 62c4 determines that there is an abnormality related to the filling and / or transport of the tablet 5 and generates a trigger. On the other hand, if no matching region exists in the summation image data Ag, the final determination unit 62c4 determines that there is no abnormality related to the filling and transport of the tablet 5. The final determination unit 62c4 may also make a determination based on the area of ​​the matching region, etc.

[0116] Furthermore, if the final determination unit 62c4 determines that there is an abnormality related to the filling and / or transport of the tablet 5, it identifies the portion corresponding to the matching area as the "abnormal portion".

[0117] Here, the determination process by the abnormality determination unit 62c in this third embodiment will be explained in more detail using the flowchart in Figure 20.

[0118] When a predetermined phase (e.g., 0°) is generated by the encoder 51, the determination process is started. First, in step S401, the inspection image data Kg is input to the AI ​​model 200 with a tablet, and reconstructed image data Sk1 is obtained. For example, if the pocket portion 2 of 1 in the inspection image data Kg is not filled with tablet 5 (see Figure 21), the first reconstructed image data Sk1 will be one in which the empty pocket portion 2 is filled with tablet 5 (see Figure 22). Also, for example, if the pocket portion 2 of 1 in the inspection image data Kg is not filled with tablet 5 and the tablet 5 is sticking out of the pocket portion 2 (see Figure 13), a similar first reconstructed image data Sk1 is obtained.

[0119] Next, in step S402, the first difference calculation unit 62c1 acquires the first difference image data Sb1. For example, if the inspection image data Kg is as shown in Figure 21, the portion corresponding to the tablet 5 that should be contained in the pocket portion 2 of 1 is identified as a difference pixel, and the first difference image data Sb1 showing this difference pixel is acquired (see Figure 23). On the other hand, for example, if the inspection image data Kg is as shown in Figure 13, the portion corresponding to the tablet 5 that should be contained in the pocket portion 2 and the portion corresponding to the tablet 5 that has popped out of the pocket portion 2 are identified as difference pixels, and the first difference image data Sb1 showing this difference pixel is acquired (see Figure 24).

[0120] Next, in step S403, the second reconstructed image data Sk2 is obtained by inputting the examination image data Kg into the tablet-less AI model 201. For example, if the examination image data Kg is as shown in Figure 21, the second reconstructed image data Sk2 will be as follows: all tablets 5, including those outside the pocket 2, have been erased, and all pocket 2 is empty (see Figure 25). Similarly, if the examination image data Kg is as shown in Figure 13 (see Figure 13), a similar second reconstructed image data Sk2 will be obtained.

[0121] Next, in step S404, the second difference calculation unit 62c2 acquires the second difference image data Sb2. For example, if the inspection image data Kg is as shown in Figure 21, the portion corresponding to the tablet 5 is identified as a difference pixel, and the second difference image data Sb2 showing this difference pixel is acquired (see Figure 26). On the other hand, for example, if the inspection image data Kg is as shown in Figure 13, the portion corresponding to all tablets 5, including the tablet 5 that has popped out of the pocket 2, is identified as a difference pixel, and the second difference image data Sb2 showing this difference pixel is acquired (see Figure 27).

[0122] Subsequently, in step S405, the addition unit 62c3 acquires the added image data Ad. For example, if the difference image data Sb1 and Sb2 are as shown in Figures 23 and 26, respectively, there are no regions that match these difference image data Sb1 and Sb2, so the added image data Ag (see Figure 28) indicating that there are no matching regions is acquired. On the other hand, for example, if the difference image data Sb1 and Sb2 are as shown in Figures 24 and 27, respectively, the regions related to the tablet 5 outside the pocket 2 match, so the added image data Ag (see Figure 29) indicating the region related to the tablet 5 is acquired.

[0123] Next, in step S406, the final determination unit 62c4 determines whether or not a matching region exists in the added image data Ag. If a matching region exists (step S406; Yes), in step S407, it is determined that there is an abnormality related to the filling and / or transport of the tablet 5. The final determination unit 62c4 also identifies the portion corresponding to the matching region as the "abnormal portion".

[0124] If an abnormality is detected, the abnormality detection unit 62c (final determination unit 62c4) generates a trigger, and as a result, the target data storage unit 63 extracts and stores target data for a predetermined time period before and after the trigger from the video data stored in the ring buffer 61. In addition, the abnormality area mark setting unit 63b places an abnormality area mark Mk (for example, a circular mark) on the part of the target data corresponding to the "abnormal part" identified by the final determination unit 62c4 in a predetermined still frame image data Sf (see Figure 30). In this third embodiment, the part of the pocket 2 that is not filled with tablets 5 is not identified as an "abnormal part," so no abnormality area mark Mk is placed on this part.

[0125] On the other hand, if there is no matching region in the added image data Ag (step S406; No), it is determined in step S408 that there is no abnormality related to the filling and transport of tablet 5.

[0126] As described above, according to this third embodiment, the trigger does not occur simply because the tablet 5 is not contained in the pocket 2. The trigger only occurs when the tablet 5 is outside the pocket 2, such as when the tablet 5 has popped out of the pocket 2. Therefore, based on the data at the time of the abnormality and the video data before and after, it is possible to more intensively check for errors in filling or transporting the tablet 5 that caused the situation where the tablet 5 was outside the pocket 2, as well as check for any residue of the popped-out tablet 5 on the PTP packaging machine 10.

[0127] Furthermore, the embodiment is not limited to the description above, and may be implemented as follows, for example. Of course, other applications and modifications not exemplified below are also possible.

[0128] (a) In the above embodiment, the camera 52 is a two-dimensional camera that takes two-dimensional images of the tablet 5, etc., but a three-dimensional camera that can obtain video data consisting of 3D image data [for example, a TOF (Time-of-Flight) camera] may be used as the camera 52.

[0129] Furthermore, as shown in Figure 31, both a first camera 52a consisting of a two-dimensional camera and a second camera 52b consisting of a three-dimensional camera may be provided, and the two-dimensional video data obtained by the first camera 52a may be stored separately in the ring buffer 61a, and the three-dimensional video data obtained by the second camera 52b may be stored separately in the ring buffer 61b. When the trigger generation unit 62 generates a trigger, the system may extract and save target data for a predetermined time before and after the trigger from the ring buffer 61a, 61b that stores the video data that caused the trigger. Alternatively, when the trigger generation unit 62 generates a trigger, the system may extract and save target data from both ring buffers 61a, 61b. Therefore, for example, when an abnormality is determined based on the video data obtained by the second camera 52b, not only the video data obtained by the second camera 52b but also the video data obtained by the first camera 52a may be saved as target data.

[0130] (b) In the above embodiment, the camera 52 is configured to photograph both the tablets 5 filled in the pocket 2 and the tablets 5 being transported together with the container film 3. In contrast, the camera 52 may be configured to photograph only one of the tablets 5 filled in the pocket 2 or the tablets 5 being transported together with the container film 3. However, it is not practical to photograph only the tablets 5 filled in the pocket 2, and if the camera attempts to photograph the tablets 5 filled in the pocket 2, the tablets 5 being transported together with the container film 3 will usually also be photographed.

[0131] Alternatively, multiple cameras 52 may be installed along the transport path of the container film 3. One camera 52 may capture the scene of filling the pocket portion 2 with tablets 5, while the other cameras may capture the scene of the tablets 5 being transported along with the transport of the container film 3.

[0132] (c) If the abnormality detection unit 62c detects that the "abnormal part" is outside the container film 3 (for example, that the tablet 5 is on the PTP packaging machine 10), the PTP packaging machine 10 may be stopped and the presence of the "abnormal part" may be notified to prompt the operator to remove the tablet 5.

[0133] Furthermore, if the "abnormal portion" is stationary outside the container film 3, in the determination process after determining that there is an abnormality in the filling and / or transport of the tablet 5 due to the presence of the "abnormal portion," the "abnormal portion" may be ignored, and the presence or absence of an abnormality in the filling and / or transport of the tablet 5 may be determined. In this case, the data related to the presence of this "abnormal portion" will be saved only once. Of course, the determination process may also be performed without ignoring this "abnormal portion" to determine the presence or absence of an abnormality. In this case, the data related to the presence of this "abnormal portion" will be saved each time the determination process is performed.

[0134] (d) In the above embodiment, the tablet 5 is given as the "contents," but the contents are not limited to tablets.

[0135] Furthermore, the types and shapes of tablets are not limited to the embodiments described above. For example, tablets include not only pharmaceutical tablets but also tablets used for food and drink. In addition, tablets include uncoated tablets, sugar-coated tablets, film-coated tablets, enteric-coated tablets, and gelatin-coated tablets, as well as various types of capsule tablets such as hard capsules and soft capsules.

[0136] Furthermore, regarding the shape of the tablet, it may not only be circular in plan view, but may also be polygonal in plan view, elliptical in plan view, or oval in plan view, for example.

[0137] (e) The configuration of the PTP sheet to be manufactured is not limited to the above embodiment. For example, the arrangement and number of pockets 2 in one PTP sheet unit are not limited in any way to the above embodiment.

[0138] Furthermore, in the above embodiment, the PTP film 6 has a configuration in which the number of pocket portions 2 corresponding to one sheet are arranged along its width direction, but it is not limited to this, and for example, it may have a configuration in which the number of pocket portions 2 corresponding to multiple sheets are arranged along its width direction.

[0139] Furthermore, in the above embodiment, the technical concept of the present invention is applied to a PTP packaging machine 10 that manufactures PTP sheets 1, but the technical concept of the present invention may also be applied to a blister packaging machine that manufactures blister sheets other than PTP sheets 1. [Explanation of Symbols]

[0140] 1...PTP sheet (blister sheet), 2...Pocket section, 3...Container film, 4...Cover film, 5...Tablet (contents), 10...PTP packaging machine (blister packaging machine), 50...Anomaly data recording system, 51...Encoder, 52...Camera (video recording means), 61...Ring buffer, 62...Trigger generation unit (trigger generation means), 62a...Reconstructed image data acquisition unit (reconstructed image data acquisition means), 62c...Anomaly determination unit (anomaly determination means), 63...Target data storage unit (target data storage means), 63a...Chapter mark assignment unit (chapter mark assignment means), 63b...Anomaly area mark setting unit (anomaly area mark setting means), 191...Encoder unit (encoding unit), 192...Decoder unit (decoding unit), 200...AI model with tablet (contents identification means), 201...AI model without tablet (contents identification means).

Claims

1. This system is applied to a blister packaging machine for manufacturing blister sheets, which are formed by attaching a cover film to a container film filled with contents in a pocket portion so as to close the opening of the pocket portion, and is an abnormality data recording system for recording video data related to abnormalities in the filling and / or transport of contents that are filled in the pocket portion and transported along with the transport of the container film, A video recording means for capturing images of at least the contents filled in the pocket and / or the contents transported together with the container film, and obtaining video data consisting of multiple still frame image data stored in chronological order, A ring buffer for storing the video data obtained by the video recording means, A trigger generating means that generates a trigger when predetermined conditions are met, An encoder capable of generating the phase related to the transport of contents, The trigger generating means generates a trigger, and the target data storage means extracts and stores target data for a predetermined time period before and after the trigger generation from the video data stored in the ring buffer, when the trigger generating means generates a trigger, The trigger generation means is A neural network having an encoding unit that extracts features from input image data and a decoding unit that reconstructs image data from the features, is trained as training data to generate an object presence identification means, which consists of image data corresponding to the still frame image data when a predetermined phase is generated by the encoder, where the pocket contains contents and there are no abnormalities related to the filling and transport of the contents. A reconstructed image data acquisition means capable of acquiring reconstructed image data by inputting the still frame image data, which is the video data obtained by the video recording means when a predetermined phase is generated by the encoder, into the contents identification means, and reconstructing the image data. The system includes an abnormality determination means that determines whether or not there is an abnormality related to the filling and / or transport of the contents by comparing the reconstructed image data and the inspection image data, The system is configured to generate the trigger when the abnormality detection means determines that an abnormality exists. The trigger generation means includes a neural network having an encoding unit that extracts features from input image data and a decoding unit that reconstructs image data from the features, and a contentless identification means that generates contentless content identification by training the neural network with image data corresponding to the static frame image data when a predetermined phase is generated by the encoder, and only image data in a state where there is no content. The reconstructed image data acquisition means can acquire, as first reconstructed image data, the image data reconstructed by inputting the inspection image data into the content presence identification means, and as second reconstructed image data, the image data reconstructed by inputting the inspection image data into the content absence identification means. An abnormality data recording system characterized in that the abnormality determination means is configured to determine whether or not there is an abnormality related to the filling and / or transport of contents, based on a region showing the difference between the first reconstructed image data and the inspection image data obtained by comparing the said data and a region showing the difference between the second reconstructed image data and the inspection image data that coincide.

2. This system is applied to a blister packaging machine for manufacturing blister sheets, which are formed by attaching a cover film to a container film filled with contents in a pocket portion so as to close the opening of the pocket portion, and is an abnormality data recording system for recording video data related to abnormalities in the filling and / or transport of contents that are filled in the pocket portion and transported along with the transport of the container film, A video recording means for capturing images of at least the contents filled in the pocket and / or the contents transported together with the container film, and obtaining video data consisting of multiple still frame image data stored in chronological order, A ring buffer for storing the video data obtained by the video recording means, A trigger generating means that generates a trigger when predetermined conditions are met, An encoder capable of generating the phase related to the transport of contents, The trigger generating means generates a trigger, and the target data storage means extracts and stores target data for a predetermined time period before and after the trigger generation from the video data stored in the ring buffer, when the trigger generating means generates a trigger, The trigger generation means is A neural network having an encoding unit that extracts features from input image data and a decoding unit that reconstructs image data from the features, is trained as training data to generate an object presence identification means, which consists of image data corresponding to the still frame image data when a predetermined phase is generated by the encoder, where the pocket contains contents and there are no abnormalities related to the filling and transport of the contents. A reconstructed image data acquisition means capable of acquiring reconstructed image data by inputting the still frame image data, which is the video data obtained by the video recording means when a predetermined phase is generated by the encoder, into the contents identification means, and reconstructing the image data. The system includes an abnormality determination means that determines whether or not there is an abnormality related to the filling and / or transport of the contents by comparing the reconstructed image data and the inspection image data, The system is configured to generate the trigger when the abnormality detection means determines that an abnormality exists. Multiple means are provided for identifying the presence of contents. The multiple contents identification means correspond to each of the multiple phases generated by the encoder, An abnormal data recording system characterized in that the reconstructed image data acquisition means is configured to input the inspection image data in the same phase as the corresponding phase to the content presence identification means and acquire the reconstructed image data as the reconstructed image data.

3. This system is applied to a blister packaging machine for manufacturing blister sheets, which are formed by attaching a cover film to a container film filled with contents in a pocket portion so as to close the opening of the pocket portion, and is an abnormality data recording system for recording video data related to abnormalities in the filling and / or transport of contents that are filled in the pocket portion and transported along with the transport of the container film, A video recording means for capturing images of at least the contents filled in the pocket and / or the contents transported together with the container film, and obtaining video data consisting of multiple still frame image data stored in chronological order, A ring buffer for storing the video data obtained by the video recording means, A trigger generating means that generates a trigger when predetermined conditions are met, An encoder capable of generating the phase related to the transport of contents, The trigger generating means generates a trigger, and the target data storage means extracts and stores target data for a predetermined time period before and after the trigger generation from the video data stored in the ring buffer, when the trigger generating means generates a trigger, The trigger generation means is A neural network having an encoding unit that extracts features from input image data and a decoding unit that reconstructs image data from the features, is trained as training data to generate an object presence identification means, which consists of image data corresponding to the still frame image data when a predetermined phase is generated by the encoder, where the pocket contains contents and there are no abnormalities related to the filling and transport of the contents. A reconstructed image data acquisition means capable of acquiring reconstructed image data by inputting the still frame image data, which is the video data obtained by the video recording means when a predetermined phase is generated by the encoder, into the contents identification means, and reconstructing the image data. The system includes an abnormality determination means that determines whether or not there is an abnormality related to the filling and / or transport of the contents by comparing the reconstructed image data and the inspection image data, The system is configured to generate the trigger when the abnormality detection means determines that an abnormality exists. An abnormal data recording system characterized by comprising a chapter marking means for assigning a chapter mark to the location in the target data corresponding to the timing when the trigger generating means assigned a trigger.

4. An abnormal data recording system according to any one of claims 1 to 3, characterized in that it includes an abnormal area mark setting means for setting an abnormal area mark in the still frame image data constituting the target data, which indicates an area that the abnormality determination means has determined to be abnormal.

Images