Abnormality time data recording system

Abstract

The present invention provides an abnormality time data recording system capable of more easily finding abnormality occurrence time data from object data. An abnormality time data recording system 50 includes: a camera 52 for photographing a tablet to be filled and / or a tablet to be conveyed so as to obtain video data; a ring buffer 61 for storing the video data; and an encoder 51 capable of generating a phase related to the conveyance of the tablet. When a trigger generation unit 62 generates a trigger, object data is extracted from the video data of the ring buffer 61 and is stored. The trigger generation unit 62 determines the presence or absence of an abnormality related to the filling and / or conveyance of a tablet 5 on the basis of still frame image data, from when a prescribed phase is generated by the encoder 51, in the video data and reconstructed image data obtained by inputting the image data into a tablet presence AI model 200, and generates the trigger when there is an abnormality.

Description

Abnormal-time Data Recording System

[0001] The present invention relates to an abnormal-time data recording system for recording video data related to an abnormality in filling and / or conveyance of contents, where the contents are filled into pocket portions of a container film and conveyed along with conveyance of the container film.

[0002] Generally, in the field of pharmaceuticals and the like, blister sheets such as PTP (press through pack) sheets are known. A blister sheet includes a container film having a pocket portion for accommodating contents (e.g., tablets or the like), and a cover film attached to the container film so as to seal the opening side of the pocket portion.

[0003] The blister sheet as described above can be manufactured by a blister packaging machine. The blister packaging machine includes means for forming pocket portions on a conveyed belt-shaped container film, means for filling the pocket portions with contents, means (sealing means) for attaching a belt-shaped cover film to the container film so as to seal the opening side of the pocket portions, means for obtaining a blister sheet by punching a belt-shaped blister film composed of the container film and the cover film, and the like. The contents filled in the pocket portions are conveyed to the downstream side (the sealing means side) along with the conveyance of the container film.

[0004] By the way, during the production of blister sheets, filling errors of contents (for example, the contents do not drop at an appropriate timing during filling) and conveyance errors (for example, the contents jump out due to abnormal vibration of the container film, etc.) may occur, resulting in abnormal conditions such as a state where no contents are accommodated in the pocket portion or a state where the contents remain on the blister packaging machine. When such an abnormal condition occurs, it is necessary to take appropriate measures to prevent the same type of condition from occurring again.

[0005] Therefore, it is conceivable to use a status confirmation system equipped with a video recording means, an inspection means, and a target data storage means (see, for example, Patent Document 1). The video recording means captures the scene of filling the pocket with contents and transporting the contents downstream, thereby obtaining video data consisting of multiple still frame image data saved in chronological order. The inspection means is provided downstream of the video recording means along the transport path of the container film and performs an inspection to determine whether or not contents are contained in the pocket. The target data storage means uses the inspection means's determination of a defect, that is, the inspection means's determination that contents are not contained in the pocket, as a trigger to extract and save predetermined target data from the video data. The target data is a predetermined range within the video data, based on the still frame image data at the time the trigger occurred, and located at a position corresponding to the number of retrospective frames corresponding to the amount of contents transported from the video recording means to the inspection means. With this status confirmation system, by checking the still frame image data that shows the scene of the abnormality in the target data, it becomes possible to identify the cause of the state in which contents are not contained in the pocket.

[0006] Japanese Patent Publication No. 2024-72365

[0007] Incidentally, even when a product is judged to be defective by inspection methods, the timing of the abnormal condition (the timing of the abnormality) is not necessarily constant and can vary. For example, the timing of the abnormality will differ between a case where a product is judged to be defective because the contents were not initially contained in the pocket due to a filling error, and a case where a product is judged to be defective because the contents were initially contained in the pocket but then spilled out due to a transport error after filling. Naturally, the timing of the contents spilling out of the pocket is also not constant.

[0008] On the other hand, the system described in Patent Document 1 is triggered by a defect detection by the inspection means, and as a result, the extracted target data is based on the assumption that the timing of the anomaly occurrence is constant. Therefore, there may be variations in the position of the static frame image data (data at the time of anomaly occurrence) that indicates the scene of the anomaly occurring within the target data. Consequently, the process of finding the data at the time of anomaly occurrence within the target data may be time-consuming and laborious.

[0009] In recent years, the manufacturing speed of blister sheets using blister packaging machines has become increasingly faster, resulting in faster conveying speeds for container film and contents, as well as faster filling speeds for contents. As a result, one second of video data may consist of numerous still frame images, which can make the process of finding data indicating an anomaly more cumbersome.

[0010] This invention has been made in view of the above circumstances, and its purpose is to provide an anomaly data recording system that makes it easier to find anomaly data from among the target data.

[0011] Below, we will describe, in separate sections, each means suitable for achieving the above objectives. Furthermore, we will add notes on the effects and benefits specific to each means as needed.

[0012] Means 1. An abnormal time data recording system for recording video data relating to abnormalities in the filling and / or transport of contents, applied to a blister packaging machine for manufacturing a blister sheet, which is formed by attaching a cover film to a container film filled with contents in a pocket portion so as to close the opening of the pocket portion, and relating to contents that are filled in the pocket portion and transported along with the transport of the container film, comprising: a video shooting means that photographs at least the contents filled in the pocket portion and / or the contents transported together with the container film and obtains video data consisting of a plurality of still frame image data stored in chronological order; a ring buffer that stores the video data obtained by the video shooting means; a trigger generating means that generates a trigger when predetermined conditions are met; an encoder that can generate a phase relating to the transport of contents; and a target data storage means that, when the trigger generating means generates a trigger, extracts and stores target data for a predetermined time before and after the occurrence of the trigger from the video data stored in the ring buffer, wherein the trigger generating means An abnormal data recording system comprising: an encoding unit for extracting feature quantities from input image data and a decoding unit for reconstructing image data from the feature quantities, wherein the neural network is trained as training data to generate image data corresponding to the still frame image data when a predetermined phase is generated by the encoder, wherein the pocket contains contents and there are no abnormalities related to the filling and transport of the contents; a reconstructed image data acquisition means capable of acquiring reconstructed image data as reconstructed image data by inputting inspection image data, which is the still frame image data from the video recording means obtained by the encoder when a predetermined phase is generated, into the contents identification means; and an abnormality determination means for determining whether there are any abnormalities related to the filling and / or transport of the contents by comparing the reconstructed image data and the inspection image data, wherein the system is configured to generate the trigger when the abnormality determination means determines that there is an abnormality.

[0013] Furthermore, the "training data" may be image data obtained by actually photographing the contents during the manufacturing of the blister sheet, showing no abnormalities related to the filling and transport of the contents, or it may be virtual image data generated using such image data, showing no abnormalities related to the filling and transport of the contents.

[0014] Furthermore, the above-mentioned "neural network" includes, for example, a convolutional neural network having multiple convolutional layers. The above-mentioned "learning" includes, for example, deep learning. The above-mentioned "contents presence identification means (generative model)" includes, for example, an autoencoder or a convolutional autoencoder (the same applies to the "contents absence identification means" of means 2 described later).

[0015] In addition, since the "contents identification means" is generated by training only on image data in which contents are contained in the pocket and there are no abnormalities related to the filling and transport of the contents, the reconstructed image data generated when inspection image data with abnormal parts (for example, image data in which the contents are not contained in the pocket or the contents are outside the pocket) is input to the contents identification means will be almost identical to the inspection image data with the abnormal parts corrected. In other words, when there are abnormal parts in the inspection image data, the reconstructed image data will be a hypothetical image data that assumes there are no abnormal parts (for example, image data in which the contents are contained in the pocket and there are no contents outside the pocket).

[0016] According to the above-described means 1, the trigger generating means generates a trigger when the abnormality detection means determines that there is an abnormality related to the filling and / or transport of the contents. Therefore, the timing of trigger generation can be synchronized with the timing of an abnormal condition (such as when the contents are not contained in the pocket or when the contents remain on the blister packaging machine).

[0017] Furthermore, when the trigger generating means generates a trigger, the target data storage means extracts and stores target data for a predetermined time period before and after the trigger from the video data. As a result, variations in the position of still frame image data (data at the time of the anomaly) indicating the occurrence of the anomaly within the target data are less likely to occur. This makes it easier to find the data at the time of the anomaly within the target data. Consequently, it becomes easier to check for errors in filling or transporting contents, and to check for residual contents on the blister packaging machine, based on the data at the time of the anomaly and the video data before and after it. As a result, it becomes possible to efficiently identify the cause of the anomaly and recover the contents remaining on the blister packaging machine.

[0018] Means 2. The trigger generating means comprises a neural network having an encoding unit for extracting feature quantities from input image data and a decoding unit for reconstructing image data from the feature quantities, and is trained to generate an empty content identification means by training only image data corresponding to the static frame image data when a predetermined phase is generated by the encoder, and which is in a state where there is no content; the reconstructed image data acquisition means is capable of acquiring, as first reconstructed image data, the image data reconstructed by inputting the inspection image data to the content presence identification means, and as second reconstructed image data, the image data reconstructed by inputting the inspection image data to the empty content identification means; and the abnormality determination means is configured to determine whether or not there is an abnormality related to the filling and / or transport of content based on a matching region between a region showing the difference between the first reconstructed image data and the inspection image data obtained by comparing the second reconstructed image data and the inspection image data. This is the abnormality data recording system according to Means 1.

[0019] Furthermore, since the "empty contents identification means" is generated by training only on image data of a state without contents (a state in which no contents are contained in the pockets and no contents have protruded from the pockets), the second reconstructed image data generated when inspection image data in which contents are contained in the pockets is input to the empty contents identification means will be almost identical to the inspection image data in which contents have been removed from the pockets. In other words, when contents are contained in all pockets in the inspection image data, the second reconstructed image data generated will be a hypothetical image data assuming that no contents are contained in any of the pockets. Also, for example, when inspection image data in which contents are outside of pocket 1 but contents are contained in other pockets is input to the empty contents identification means, the second reconstructed image data generated will be a hypothetical image data assuming that all contents, including contents that have protruded from the pockets, have been removed and no contents are contained in any of the pockets.

[0020] According to the above means 2, the abnormality determination means makes a determination based on the matching region between the region showing the difference between the first reconstructed image data and the inspection image data obtained by comparing the said data and the region showing the difference between the second reconstructed image data and the inspection image data.

[0021] Here, in the inspection image data, for example, if there is no content in pocket 1, but no content has spilled out of the pocket, a difference will occur between the inspection image data and the first reconstructed image data in the portion of the content that should be contained in pocket 1. As a result, the region representing the difference will be the region of the content related to pocket 1. On the other hand, a difference will occur between the inspection image data and the second reconstructed image data in the portion of the content contained in pockets other than pocket 1. As a result, the region representing the difference will be the region of the content related to pockets other than pocket 1. Therefore, there will be no matching region between these regions representing the difference, and the abnormality determination means will determine that there is no abnormality related to the transport of the content.

[0022] In contrast, if, for example, the inspection image data shows that pocket 1 is empty and the contents have spilled out of the pocket, a difference will occur between the inspection image data and the first reconstructed image data between the portion of the contents that should be contained in pocket 1 and the portion of the contents that have spilled out of the pocket. Therefore, the regions of these contents will be obtained as areas indicating the difference. On the other hand, a difference will occur between the inspection image data and the second reconstructed image data between the portion of the contents contained in pockets other than pocket 1 and the portion of the contents that have spilled out of the pocket. Therefore, the regions of these contents will be obtained as areas indicating the difference. Consequently, the regions of the contents that have spilled out of the pockets will coincide between these regions indicating the difference. Therefore, the abnormality determination means will determine that there is an abnormality related to the filling and / or transport of the contents.

[0023] Therefore, according to the above means 2, the trigger will not occur simply because the pocket is not filled with contents, but only when the contents are outside the pocket, such as when the contents have spilled out of the pocket. As a result, based on the data at the time of the abnormality and the video data before and after, it is possible to more intensively check for filling errors or transport errors of the contents that caused the contents to be outside the pocket, as well as check for any residue of spilled contents on the blister packaging machine.

[0024] Means 3. The abnormal data recording system according to Means 1, characterized in that a plurality of contents presence identification means are provided, each of the plurality of contents presence identification means corresponds to each of the plurality of phases generated by the encoder, and the reconstructed image data acquisition means is configured to input the inspection image data in the same phase as the phase to which the contents presence identification means corresponds to the input of the contents presence identification means and acquire the reconstructed image data as the reconstructed image data.

[0025] According to the above means 3, the abnormality detection means can perform multiple determinations while the encoder's phase changes for one cycle. Therefore, abnormalities that occur in a very short time (for example, contents flying out of the pocket at high speed, or contents flying in and instantly replacing the contents in the pocket) can be detected with greater accuracy. As a result, target data can be obtained more reliably when an abnormality occurs related to the filling and / or transport of contents.

[0026] Furthermore, when the technical aspects of means 2 are combined with the technical aspects of means 3, the empty contents identification means is configured in the same way as the contents identification means of means 3. That is, multiple empty contents identification means are provided, and each of the multiple empty contents identification means corresponds to one of the multiple phases generated by the encoder.

[0027] Means 4. The abnormal data recording system according to Means 1, further comprising a chapter marking means for assigning a chapter mark to a location in the target data corresponding to the timing at which the trigger generating means assigned a trigger.

[0028] According to method 4 described above, using chapter markers makes it easier to find data where anomalies occurred in the target data. This makes it easier to check for errors in filling or transporting contents, as well as for residual contents on the blister packaging machine.

[0029] Means 5. The abnormal data recording system according to Means 1, further comprising an abnormal area mark setting means for providing an abnormal area mark indicating an area determined to be abnormal by the abnormality determination means to the static frame image data constituting the target data.

[0030] According to the above means 5, an abnormal area mark can be added to the still frame image data of the target data to indicate an area determined to be abnormal by the abnormality determination means (for example, an area corresponding to contents that should be present or contents that are outside the pocket). Therefore, the abnormal part can be found more easily, and consequently, the cause of the abnormality and the recovery of the spilled contents can be carried out more efficiently.

[0031] Furthermore, the technical aspects related to the above means may be combined as appropriate. For example, at least one of the technical aspects related to means 3 to 5 may be combined with the technical aspects related to means 2.

[0032] This is a perspective view showing a PTP sheet. This is a partially enlarged cross-sectional view of a PTP sheet. This is a perspective view showing a PTP film. This is a schematic diagram showing the general configuration of a PTP packaging machine, etc. This is a block diagram showing the general configuration of an abnormal data recording system. This is an explanatory diagram for explaining video data stored in a ring buffer, etc. This is a schematic diagram showing still frame image data with abnormal area marks. This is a schematic diagram showing training data used for training an AI model with tablets. This is a schematic diagram for explaining the structure of a neural network. This is a flowchart showing the learning process flow of a neural network. This is a schematic diagram showing an example of image data for inspection. This is a schematic diagram showing an example of reconstructed image data. This is a schematic diagram showing an example of image data for inspection. This is a block diagram showing the part of the abnormal data recording system that is particularly related to judgment processing and trigger generation. This is a flowchart showing the flow of judgment processing. This is a schematic diagram showing an example of differential image data. This is a block diagram showing the part of the abnormal data recording system that is particularly related to judgment processing and trigger generation in the second embodiment. This is a block diagram showing the part of the abnormal data recording system that is particularly related to judgment processing and trigger generation in the third embodiment. This is a schematic diagram showing training data used for training an AI model without tablets. This is a flowchart showing the flow of judgment processing in the third embodiment. This is a schematic diagram showing an example of inspection image data in the third embodiment. This is a schematic diagram showing an example of first reconstructed image data in the third embodiment. This is a schematic diagram showing an example of first difference image data in the third embodiment. This is a schematic diagram showing an example of first difference image data in the third embodiment. This is a schematic diagram showing an example of second reconstructed image data in the third embodiment. This is a schematic diagram showing an example of second difference image data in the third embodiment. This is a schematic diagram showing an example of second difference image data in the third embodiment. This is a schematic diagram showing an example of additive image data in the third embodiment. This is a schematic diagram showing an example of additive image data in the third embodiment. This is a schematic diagram showing a static frame image data with an abnormality area mark in the third embodiment. This is a block diagram showing the schematic configuration of an abnormality data recording system in another embodiment.

[0033] The embodiments will be described below with reference to the drawings. [First Embodiment] First, the configuration of the PTP sheet as a "blister sheet" will be described. As shown in Figures 1 and 2, the PTP sheet 1 has a container film 3 with a plurality of pocket portions 2 and a cover film 4 attached to the container film 3 so as to close the pocket portions 2.

[0034] The container film 3 is made of a transparent thermoplastic resin material such as PP (polypropylene) or PVC (polyvinyl chloride). On the other hand, the cover film 4 is made of an opaque material (such as aluminum foil) with a sealant made of polypropylene resin or the like applied to its surface. Of course, the materials of each film 3 and 4 are not limited to these, and other materials may be used.

[0035] The PTP sheet 1 has a rectangular shape when viewed from above. The PTP sheet 1 has two rows of pockets, each consisting of three pocket sections 2 arranged along its longitudinal direction, and two rows of pockets arranged along its shorter direction. Each pocket section 2 contains one tablet 5 as the "contents".

[0036] Furthermore, the PTP sheet 1 is manufactured by punching out a strip-shaped PTP film 6 (see Figure 3) formed from a strip-shaped container film 3 and a strip-shaped cover film 4. In this embodiment, the PTP film 6 corresponds to a "blister film".

[0037] Next, the general configuration of the PTP packaging machine 10 for manufacturing the PTP sheet 1 described above will be explained. In this embodiment, the PTP packaging machine 10 corresponds to a "blister packaging machine".

[0038] As shown in Figure 4, at the upstream end of the PTP packaging machine 10, a strip of container film 3 is wound into a roll. The end of the roll of container film 3 is guided by a guide roll 13. Downstream of the guide roll 13, the container film 3 is mounted on an intermittent feed roll 14. The intermittent feed roll 14 intermittently conveys the container film 3.

[0039] Between the guide roll 13 and the intermittent feed roll 14, a preheating device 15 and a pocket forming device 16 are arranged along the transport path of the container film 3. When the container film 3 is preheated by the preheating device 15 and becomes relatively flexible, the pocket forming device 16 forms a plurality of pockets 2 at predetermined positions on the container film 3. The formation of the pockets 2 is performed during the intervals between the transport operations of the container film 3 by the intermittent feed roll 14.

[0040] The container film 3, fed from the intermittent feed roll 14, is mounted in the following order: tension roll 18, guide roll 19, and film receiving roll 20. The film receiving roll 20 is operated by a predetermined motor M2 to transport the container film 3 continuously and at a constant speed. The tension roll 18 prevents the container film 3 from slackening due to differences in transport operation between the intermittent feed roll 14 and the film receiving roll 20, thereby keeping the container film 3 constantly taut.

[0041] Between the guide roll 19 and the film receiving roll 20, a filling device 22 is arranged along the transport path of the container film 3. The filling device 22 includes, for example, a cylindrical chute for accommodating tablets 5 in a single row, and a shutter (not shown) that can open and close the exit of the chute. By opening the shutter at a predetermined timing, the tablets 5 are filled into the pocket section 2. However, if a filling error occurs, such as a temporary jamming of tablets 5 in the chute or a malfunction of the shutter, there is a risk that the pocket section 2 will not contain tablets 5 or that tablets 5 will fall out of the pocket section 2.

[0042] Meanwhile, the raw material of the strip-shaped cover film 4 is wound into a roll at the upstream end. The end of the roll-shaped cover film 4 is guided toward the heating roll 25 by a guide roll 24.

[0043] The heating roll 25 is configured to be capable of press-contacting the film receiving roll 20, and the container film 3 and the cover film 4 are fed between the two rolls 20 and 25. Then, when both the films 3 and 4 pass between the two rolls 20 and 25 in a heated and pressed state, the cover film 4 is attached to the container film 3, and the pocket portions 2 are closed by the cover film 4. Accordingly, a belt-shaped PTP film 6 in which tablets 5 are housed in the respective pocket portions 2 is manufactured. It should be noted that if a conveyance error of the tablet 5, such as abnormal vibration of the container film 3 or contact of the container film 3 with a component of the PTP packaging machine 10, occurs between the filling device 22 and the two rolls 20 and 25, there is a risk that a situation where no tablet 5 is housed in the pocket portion 2 or a situation where the tablet 5 jumps out of the pocket portion 2 may occur.

[0044] The PTP film 6 fed out from the film receiving roll 20 is wound around a tension roll 27 and an intermittent feed roll 28 in this order. The intermittent feed roll 28 intermittently conveys the PTP film 6. The tension roll 27 prevents slack of the PTP film 6 caused by a difference in conveyance operation between the film receiving roll 20 and the intermittent feed roll 28, and keeps the PTP film 6 in a constantly tensioned state.

[0045] The PTP film 6 fed out from the intermittent feed roll 28 is wound around a tension roll 31 and an intermittent feed roll 32 in this order. The intermittent feed roll 32 intermittently conveys the PTP film 6. The tension roll 31 prevents slack of the PTP film 6 between the intermittent feed rolls 28 and 32.

[0046] Between the intermittent feed roll 28 and the tension roll 31, a slit forming device 33 and a stamping device 34 are disposed along the conveyance path of the PTP film 6. The slit forming device 33 forms separation slits at predetermined positions of the PTP film 6. The stamping device 34 stamps a mark at a predetermined position (for example, a tag portion) of the PTP film 6. It should be noted that illustration of the separation slits and the stamped marks is omitted in FIG. 1 and other drawings.

[0047] The PTP film 6 fed from the intermittent feed roll 32 is wound around the tension roll 35 and the continuous feed roll 36 in this order on the downstream side thereof. Between the intermittent feed roll 32 and the tension roll 35, a sheet punching device 37 is disposed along the conveyance path of the PTP film 6. The sheet punching device 37 has a function of punching the outer edge of the PTP film 6 into units of one PTP sheet 1, that is, a function of separating the PTP sheet 1 from the PTP film 6.

[0048] The PTP sheet 1 obtained by the sheet punching device 37 is conveyed by a conveyor 39 and temporarily stored in a finished product hopper 40. However, defective PTP sheets 1 are not fed to the finished product hopper 40, and are separately discharged by a defective sheet discharge mechanism (not shown).

[0049] A cutting device 42 is disposed on the downstream side of the continuous feed roll 36. The scrap 43 remaining in a strip shape after punching by the sheet punching device 37 is guided to the tension roll 35 and the continuous feed roll 36, and then guided to the cutting device 42. The cutting device 42 cuts the scrap 43 into a predetermined size. The cut scrap 43 is stored in a scrap hopper 44 and then disposed of as waste.

[0050] Next, an abnormal-time data recording system applied to the above-described PTP packaging machine 10 will be described. The abnormal-time data recording system relates to the tablet 5 that is filled in the pocket portion 2 and conveyed along with the conveyance of the container film 3, and handles abnormalities related to filling and / or conveyance of the tablet 5 [such as an abnormality where no tablet 5 is accommodated in the pocket portion 2, an abnormality where the tablet 5 remains on the container film 3 (especially the sheet portion other than the pocket portion 2) or the PTP packaging machine 10, and other abnormalities related to the position of the tablet 5 caused by filling errors or conveyance errors], and is used for recording video data related to such abnormalities. As shown in FIG. 5, the abnormal-time data recording system 50 includes an encoder 51, a camera 52, and a control device 60. In the present embodiment, the camera 52 corresponds to a "video photographing means".

[0051] The encoder 51 is used to acquire the phase related to the motor M2. In this embodiment, the encoder 51 is set so that each time the motor M2 operates continuously and the container film 3 is transported by a predetermined length (the length of one PTP sheet 1), the acquired phase advances by 360° and the rotation speed increases by 1. Therefore, the phase acquired by the encoder 51 corresponds to the amount of container film 3 transported by the motor M2.

[0052] Camera 52 is positioned to correspond to the filling position of the tablet 5 by the filling device 22 (see Figure 4). Camera 52 obtains video data by taking two-dimensional images of the tablet 5 being filled into the pocket 2 and the tablet 5 being transported along with the transport of the container film 3. In other words, camera 52 obtains video data by taking two-dimensional images of the tablet 5 being filled into the pocket 2 and the tablet 5 being transported together with the container film 3. The video data consists of multiple still frame image data stored in chronological order, and in this embodiment, one second of video data consists of multiple (for example, 60) still frame image data.

[0053] Furthermore, the camera 52 starts recording video data in conjunction with the activation of the encoder 51 and stops recording video data in conjunction with the deactivation of the encoder 51. As a result, the multiple still frame image data that make up the video data are associated with the rotation speed and phase of the encoder 51.

[0054] The video data obtained by the camera 52 is input to the control device 60 and stored in the ring buffer 61, which will be described later (see Figure 6). In Figure 6, the data name consisting of the time and frame number is shown as the still frame image data that constitutes the video data. In this embodiment, the frame number is a number from 0 to 59, for example, and indicates the acquisition order of multiple (for example, 60) still frame image data that constitute one second of video data. The time is obtained using the clock function of the camera 52 or the control device 60.

[0055] The control device 60 is responsible for controlling the operation of each device in the PTP packaging machine 10 and the abnormal data recording system 50. The control device 60 includes a CPU as a means of calculation, ROM for storing various programs, RAM for temporarily storing various data such as calculation data and input / output data, a storage medium for long-term storage of various data, an input device for inputting information (e.g., a keyboard), and a display device for displaying various information (e.g., a liquid crystal display).

[0056] As shown in Figure 6, the control device 60 includes a ring buffer 61, a trigger generation unit 62, a target data storage unit 63, and a learning unit 64. In this embodiment, the trigger generation unit 62 corresponds to the "trigger generation means," and the target data storage unit 63 corresponds to the "target data storage means."

[0057] The ring buffer 61 is composed of the storage medium of the control device 60, and it treats a certain information storage area in this storage medium as a ring. The ring buffer 61 is configured so that after data has been stored in all information storage areas, it returns to the first information storage area and overwrites the data. The ring buffer 61 stores video data obtained by the camera 52, and in the ring buffer 61, new video data obtained by the camera 52 is sequentially stored, while old video data is sequentially erased.

[0058] The trigger generation unit 62 generates a trigger when predetermined conditions are met. The trigger generation unit 62 will be described later.

[0059] When the trigger generation unit 62 generates a trigger, the target data storage unit 63 extracts and saves target data for a predetermined time (for example, several seconds) from the video data stored in the ring buffer 61 around the time the trigger was generated (see Figure 6). The target data to be saved is video data consisting of multiple still frame image data, and is considered to be necessary for identifying the cause and timing of a malfunction. The length of the target data (the predetermined time mentioned above) can be changed as appropriate.

[0060] Furthermore, the target data storage unit 63 includes a chapter mark assignment unit 63a and an abnormal area mark setting unit 63b. In this embodiment, the chapter mark assignment unit 63a corresponds to the "chapter mark assignment means," and the abnormal area mark setting unit 63b corresponds to the "abnormal area mark setting means."

[0061] The chapter mark assignment unit 63a assigns chapter marks to locations in the target data that correspond to the timing when the trigger generation unit 62 generates a trigger. The chapter marks indicate the divisions of the target data. In this embodiment, by using chapter marks, operators can view the target data starting from the still frame image data that constitutes the target data, particularly those that triggered the trigger generation unit 62 to generate a trigger (i.e., those used as inspection image data in the judgment process by the abnormality judgment unit 62c described later).

[0062] The abnormal area mark setting unit 63b provides an abnormal area mark Mk (see Figure 7) to the static frame image data Sf constituting the target data, indicating the area that the abnormality determination unit 62c (described later) has determined to be abnormal. In this embodiment, among the multiple static frame image data Sf constituting the target data, the one that triggered the trigger generation unit 62 (used as inspection image data in the determination process described later) has an abnormal area mark Mk (for example, a circular mark) provided in the part corresponding to the "abnormal part" identified by the abnormality determination unit 62c. The form of the abnormal area mark Mk may be changed as appropriate.

[0063] The learning unit 64 is a functional unit that uses training data to train a deep neural network 190 (hereinafter simply referred to as "neural network 190"; see Figure 9) and constructs an AI (Artificial Intelligence) model 200 that serves as a "means for identifying whether a tablet contains contents."

[0064] The tablet-containing AI model 200 is a generative model constructed by deep learning a neural network 190 using image data Gd1 (see Figure 8) that corresponds to static frame image data Sf when a predetermined phase (e.g., 0°) is generated by the encoder 51, where a tablet 5 is contained in the pocket portion 2, and there are no abnormalities related to the filling and transport of the tablet 5. It has the structure of a so-called autoencoder. As described above, the encoder 51 advances its phase by 360° each time the container film 3 is transported by a predetermined length (the length of one PTP sheet 1). Therefore, the position of the pocket portion 2 remains almost constant between static frame image data Sf when a predetermined phase (e.g., 0°) is generated by the encoder 51.

[0065] Here, the structure of the neural network 190 will be explained with reference to Figure 9. Figure 9 is a schematic diagram conceptually showing the structure of the neural network 190. As shown in Figure 9, the neural network 190 has the structure of a convolutional auto-encoder (CAE), comprising an encoder unit 191 as an "encoding unit" that extracts feature quantities (latent variables) TA from the input image data GA, and a decoder unit 192 as a "decoding unit" that reconstructs image data GB from the feature quantities TA.

[0066] The structure of the convolutional autoencoder is well known, so a detailed explanation will be omitted. The encoder unit 191 has multiple convolutional layers 193, and in each convolutional layer 193, the result of a convolution operation using multiple filters (kernels) 194 on the input data is output as input data for the next layer. Similarly, the decoder unit 192 has multiple deconvolutional layers 195, and in each deconvolutional layer 195, the result of a deconvolution operation using multiple filters (kernels) 196 on the input data is output as input data for the next layer. Then, in the learning process described below, the weights (parameters) of each filter 194, 196 are updated.

[0067] Here, we will explain the learning process performed when generating the AI ​​model 200 with tablets. First, prior to the learning process, a large amount of learning data Gd1 is prepared. In this embodiment, the learning data Gd1 is still frame image data Sf from video data obtained when the camera 52 actually photographs the tablets 5, etc., and a predetermined phase (for example, 0°) is generated by the encoder 51. This still frame image data Sf is free from any abnormalities related to the filling and transport of the tablets 5. In other words, the learning data Gd1 is such that tablets 5 are contained in each pocket 2, while no tablets 5 exist outside the pocket 2. Note that the learning data Gd1 may be virtually generated (virtual image data) rather than data obtained by the camera 52 actually photographing the tablets 5, etc.

[0068] As shown in Figure 10, in the learning process, first, in step S201, an untrained neural network 190 is prepared. For example, a neural network 190 that has been pre-stored in a predetermined storage device is read. Alternatively, the neural network 190 is constructed based on network configuration information (for example, the number of layers in the neural network and the number of nodes in each layer) stored in the storage device.

[0069] Next, in step S202, reconstructed image data is acquired. That is, the pre-prepared training data Gd1 is fed as input data to the input layer of the neural network 190. Then, the reconstructed image data output from the output layer of the neural network 190 is acquired.

[0070] In the subsequent step S203, the training data Gd1 is compared with the reconstructed image data output by the neural network 190 in step S202, and it is determined whether the error is sufficiently small (whether it is below a predetermined threshold).

[0071] Here, if the error is sufficiently small, step S205 determines whether the learning process termination conditions are met. For example, if a certain number of consecutive affirmative judgments are made in step S203 without going through the process of step S204 described later, or if learning using all of the prepared learning data is repeated a predetermined number of times, it is determined that the termination conditions are met. If the termination conditions are met, the neural network 190 and its learning information (updated parameters, etc., described later) are stored in the AI ​​storage unit 62b described later as the AI ​​model 200 with tablets, and the learning process is terminated.

[0072] On the other hand, if the termination condition is not met in step S205, the process returns to step S202 and the neural network 190 is trained again.

[0073] Furthermore, if the error is not sufficiently small in step S203, the network update process (training of the neural network 190) is performed in step S204, and then the process returns to step S202, and the above series of processes is repeated.

[0074] In the network update process of step S204, the weights (parameters) of each filter 194, 196 in the neural network 190 are updated to more appropriate values, using a known learning algorithm such as backpropagation, so that the loss function representing the difference between the training data Gd1 and the reconstructed image data is minimized. For example, BCE (Binary Cross-entropy) can be used as the loss function.

[0075] By repeatedly performing steps S202 to S204, the neural network 190 minimizes the error between the training data Gd1 and the reconstructed image data, resulting in the output of more accurate reconstructed image data.

[0076] The final AI model 200 with tablets will have inspection image data Kg, which is static frame image data Sf from the video data obtained by the camera 52 when a predetermined phase (e.g., 0°) is generated by the encoder 51. When data without abnormalities related to the filling and transport of the tablets 5 (see Figure 11) is input, it will generate reconstructed image data Sk (see Figure 12) that is almost identical to the input inspection image data Kg.

[0077] On the other hand, when the AI ​​model 200 with tablets receives inspection image data Kg (see Figure 13) in which there are abnormalities related to the filling and / or transport of tablets 5, such as pockets 2 that are not filled with tablets 5 or tablets 5 that have popped out of pockets 2, it generates reconstructed image data Sk (see Figure 12) that is corrected for noise (abnormal parts) and closely matches the input image data. In other words, if there are abnormalities related to the filling and / or transport of tablets 5 in the inspection image data Kg, a virtual image data is generated as the reconstructed image data Sk, assuming that there are no abnormalities.

[0078] Furthermore, it is not necessary to perform the above learning process each time the control device 60 is manufactured. The neural network 190 and its learning information (updated parameters, etc.) may be acquired in advance and stored in the AI ​​memory unit 62b of the trigger generation unit 62 as the tablet-containing AI model 200.

[0079] Next, the trigger generation unit 62 will be described. As shown in Figure 5, the trigger generation unit 62 includes a reconstructed image data acquisition unit 62a, an AI storage unit 62b, and an anomaly determination unit 62c. In this embodiment, the reconstructed image data acquisition unit 62a corresponds to the "reconstructed image data acquisition means," and the anomaly determination unit 62c corresponds to the "anomaly determination means."

[0080] The reconstructed image data acquisition unit 62a causes the abnormality determination unit 62c to execute the determination process described later when a predetermined phase (for example, 0°) is generated by the encoder 51, and functions like a switch that operates based on the output from the encoder 51 (see Figure 14).

[0081] When the encoder 51 generates a predetermined phase (for example, 0°), the reconstructed image data acquisition unit 62a inputs the inspection image data Kg obtained by the camera 52 at the time the encoder 51 generated that phase into the tablet-containing AI model 200 and acquires the reconstructed image data as reconstructed image data Sk. More specifically, the reconstructed image data acquisition unit 62a provides the inspection image data Kg obtained by the camera 52 as input data to the input layer of the neural network 190. The reconstructed image data acquisition unit 62a then acquires the image data output from the output layer of the neural network 190 as reconstructed image data Sk. The acquired reconstructed image data Sk is input to the abnormality determination unit 62c.

[0082] Furthermore, the reconstructed image data acquisition unit 62a inputs the inspection image data Kg obtained by the camera 52 directly to the abnormality determination unit 62c when the encoder 51 generates a predetermined phase (for example, 0°). Therefore, each time the encoder 51 generates a predetermined phase (0°), the inspection image data Kg and the reconstructed image data Sk obtained based on this inspection image data Kg are input to the abnormality determination unit 62c.

[0083] The AI ​​memory unit 62b is composed of the storage medium of the control device 60 and stores the AI ​​model 200 with tablets.

[0084] The abnormality detection unit 62c compares the input reconstructed image data Sk and the inspection image data Kg to determine whether there are any abnormalities related to the filling and / or transport of the tablet 5. When the abnormality detection unit 62c determines that there is an abnormality, it generates a trigger. This trigger is input to the target data storage unit 63.

[0085] Now, referring to Figures 14 and 15, we will explain the determination process performed by the abnormality determination unit 62c to determine whether or not there is an abnormality related to the filling and / or transport of the tablets 5.

[0086] When the encoder 51 generates a predetermined phase (e.g., 0°), the determination process begins. First, in step S301, the reconstructed image data acquisition unit 62a inputs the inspection image data Kg obtained by the camera 52 when the predetermined phase (e.g., 0°) is generated into the tablet-containing AI model 200. This acquires the reconstructed image data Sk. For example, in the inspection image data Kg, if the pocket portion 2 is not filled with tablets 5 and tablets 5 are sticking out of the pocket portion 2 (see Figure 13), the reconstructed image data Sk is obtained with the empty pocket portion 2 filled with tablets 5 and the tablets 5 that have stuck out of the pocket portion 2 removed (see Figure 12).

[0087] Next, in step S302, the abnormality determination unit 62c compares the inspection image data Kg obtained by the camera 52 with the reconstructed image data Sk obtained based on the inspection image data Kg, and calculates the difference in brightness for each pixel of both data Kg and Sk. Pixels whose differences are outside a predetermined tolerance range are identified as difference pixels, and difference image data Sb (see Figure 16) representing these difference pixels is obtained. For example, if the inspection image data Kg is as shown in Figure 13 and the reconstructed image data Sk is as shown in Figure 12, the portion corresponding to the tablet 5 that should be contained in the pocket portion 2 of 1 and the portion corresponding to the tablet 5 that has popped out of the pocket portion 2 are identified as difference pixels, and the difference image data Sb representing these difference pixels is obtained as shown in Figure 16.

[0088] Next, in step S303, the area of ​​the connected component of the difference pixels in the difference image data Sb (defective area) is calculated.

[0089] Next, in step S304, it is determined whether the maximum area among the calculated areas is greater than or equal to a predetermined threshold. In other words, it is determined whether the maximum area is outside the acceptable range. If the maximum area is greater than or equal to the threshold, it is determined that there is an abnormality related to the filling and / or transport of the tablets 5. Then, the portion corresponding to the connected component of the difference pixels in the inspection image data Kg is identified as the "abnormal portion," and the determination process is terminated.

[0090] On the other hand, if the maximum area falls below the threshold, in step S306, it is determined that there are no abnormalities related to the filling and transport of the tablets 5, and the determination process is terminated. Note that the determination method described above is just one example, and the determination method may be changed as appropriate.

[0091] Then, if the abnormality detection unit 62c determines that there is an abnormality (step S305; Yes), the abnormality detection unit 62c (trigger generation unit 62) generates a trigger. Upon the generation of the trigger, the target data storage unit 63 extracts and saves target data for a predetermined time period before and after the generation of the trigger from the video data stored in the ring buffer 61. When saving the target data, chapter marks are added by the chapter mark assignment unit 63a and abnormal area marks Mk are set by the abnormal area mark setting unit 63b.

[0092] As described in detail above, according to this embodiment, the trigger generation unit 62 generates a trigger when the abnormality determination unit 62c determines that there is an abnormality related to the filling and / or transport of the tablets 5. Therefore, the timing of trigger generation can be synchronized with the timing of an abnormal condition (for example, when the tablets 5 are not contained in the pocket unit 2, or when the tablets 5 remain on the PTP packaging machine 10).

[0093] Furthermore, when the trigger generation unit 62 generates a trigger, the target data storage unit 63 extracts and stores target data for a predetermined time before and after the trigger from the video data. As a result, variations in the position of still frame image data Sf (data at the time of abnormality occurrence), which indicates the scene of the abnormality, within the target data become less likely. This makes it easier to find the data at the time of abnormality occurrence within the target data. Consequently, it becomes easier to check for errors in filling and transporting the tablets 5, and to check for any tablets 5 remaining on the PTP packaging machine 10 that have popped out, based on the data at the time of abnormality occurrence and the video data before and after it. As a result, it becomes possible to efficiently identify the cause of the abnormality and recover any tablets 5 remaining on the PTP packaging machine 10.

[0094] Furthermore, using chapter markers makes it easier to find data where anomalies occurred within the target data. This makes it even easier to check for errors in filling and transporting tablets 5, and to check for any residue of ejected tablets 5 remaining on the PTP packaging machine 10.

[0095] In addition, an abnormal region mark Mk indicating an abnormal region ("abnormal portion") determined by the abnormality determination unit 62c can be provided in the static frame image data Sf of the target data. Therefore, the abnormal portion can be found more easily, and consequently, the cause of the abnormality and the recovery of the ejected tablet 5 can be performed more efficiently. [Second Embodiment] Next, the second embodiment will be described, focusing on the differences from the first embodiment. In the first embodiment, one tablet-containing AI model 200 is provided, which is generated by training the encoder 51 with image data (learning data Gd1) corresponding to the static frame image data Sf when a predetermined phase (e.g., 0°) is generated. In other words, only one tablet-containing AI model 200 for a predetermined phase (e.g., 0°) is provided.

[0096] In contrast, in this second embodiment, as shown in Figure 17, the AI ​​models 200 with tablets include an AI model (0°) 200 corresponding to a 0° phase, an AI model (10°) 200 corresponding to a 10° phase, ..., and an AI model (350°) 200 corresponding to a 350° phase. That is, there are multiple (e.g., 36) AI models 200 with tablets, each corresponding to one of the multiple phases (e.g., 0°, 10°, ..., 350°) generated by the encoder 51.

[0097] Each AI model 200 with a tablet is generated by training it with image data corresponding to the static frame image data Sf generated by the encoder 51 when it generates the phase corresponding to its own. For example, the AI ​​model (0°) 200 with a tablet is generated by training it with image data corresponding to the static frame image data Sf generated by the encoder 51 when it generates a phase of 0°.

[0098] Furthermore, each tablet-containing AI model 200 is trained using only image data without abnormalities related to the filling and transport of the tablets 5, similar to the first embodiment described above. Also, similar to the first embodiment described above, the encoder 51 advances its phase by 360° each time the container film 3 is transported by a predetermined length (the length of one PTP sheet 1). Therefore, when one of the multiple phases is generated by the encoder 51, the position of the pocket portion 2 remains almost constant between static frame image data Sf. Accordingly, for example, when a phase of 10° is generated by the encoder 51, the position of the pocket portion 2 remains almost constant between static frame image data Sf.

[0099] Furthermore, in this second embodiment, an abnormality detection unit 62c is provided for each tablet-containing AI model 200. Therefore, for example, an abnormality detection unit 62c corresponding to the tablet-containing AI model (0°) 200 and an abnormality detection unit 62c corresponding to the tablet-containing AI model (10°) 200 are provided.

[0100] In this second embodiment, when the encoder 51 generates one of a plurality of phases (for example, 0°, 10°, ..., 350°), the reconstructed image data acquisition unit 62a acquires reconstructed image data Sk by inputting inspection image data Kg corresponding to the generated phase to the AI ​​model 200 with a tablet. For example, when the encoder 51 generates a phase of 10°, the reconstructed image data acquisition unit 62a acquires reconstructed image data Sk by inputting inspection image data Kg corresponding to the 10° phase to the AI ​​model (10°) 200 with a tablet. The acquired reconstructed image data Sk is input to the abnormality determination unit 62c corresponding to the generated phase.

[0101] Furthermore, when the reconstructed image data acquisition unit 62a generates one of several phases (for example, 0°, 10°, ..., 350°) from the encoder 51, it inputs the inspection image data Kg corresponding to that phase to the abnormality determination unit 62c corresponding to that phase.Therefore, for example, each time the encoder 51 generates a phase of 10°, the inspection image data Kg corresponding to the 10° phase and the reconstructed image data Sk obtained based on this inspection image data Kg are input to the abnormality determination unit 62c corresponding to the 10° phase.

[0102] Then, the abnormality determination unit 62c, similar to the first embodiment, compares the input inspection image data Kg and the reconstructed image data Sk to determine whether or not there is an abnormality related to the filling and / or transport of the tablet 5. Therefore, in this second embodiment, the presence or absence of an abnormality is determined each time one of the multiple phases is generated from the encoder 51.

[0103] Then, the abnormality detection unit 62c generates a trigger if it determines that an abnormality exists. Upon the generation of the trigger, the target data storage unit 63 extracts the target data from the ring buffer 61 and stores it.

[0104] In this second embodiment, the ability to input a trigger from each abnormality detection unit 62c to the target data storage unit 63 is switched according to the phase generated by the encoder 51. For example, when a phase of 10° is generated by the encoder 51, the input of a trigger from the abnormality detection unit 62c corresponding to the 10° phase to the target data storage unit 63 is permitted, while the input of a trigger from the abnormality detection unit 62c corresponding to other phases to the target data storage unit 63 is prohibited.

[0105] As described above, according to this second embodiment, the abnormality determination unit 62c can perform determination multiple times while the phase of the encoder 51 changes by one cycle. Therefore, abnormalities that occur in a very short time (for example, a tablet 5 flying out of the pocket 2 at high speed, or a flying tablet 5 instantly swapping places with a tablet 5 in the pocket 2) can be detected with greater accuracy. As a result, target data can be obtained more reliably when an abnormality occurs related to the filling and / or transport of the tablet 5. [Third Embodiment] Next, the third embodiment will be described, focusing on the differences from the first embodiment. In the first embodiment, a tablet 5 is contained in the pocket 2, and an AI model 200 with a tablet is provided, which is generated by training only image data without abnormalities related to the filling and transport of the tablet 5 as training data Gd1. In contrast, in this third embodiment, as shown in Figure 18, in addition to the AI ​​model 200 with a tablet, an AI model 201 without a tablet is provided. The AI ​​model 201 without a tablet corresponds to the "no contents identification means".

[0106] The tablet-free AI model 201 is a generative model constructed by deep learning a neural network 190 using only the image data Gd2 (see Figure 19) of a state without tablets 5, which corresponds to the static frame image data Sf when a predetermined phase (e.g., 0°) is generated by the encoder 51. It has the structure of a so-called autoencoder.

[0107] Therefore, when the tablet-free AI model 201 receives an input of an inspection image data Kg containing a tablet 5, it generates a second reconstructed image data Sk2 that is nearly identical to the inspection image data Kg from which the tablet 5 has been removed. For example, when all the pockets 2 in the inspection image data Kg contain tablets 5, the second reconstructed image data Sk2 generates a hypothetical image data assuming that there are no tablets 5 in any of the pockets 2. Also, for example, when there are tablets 5 outside the pockets 2 in the inspection image data Kg, the second reconstructed image data Sk2 generates a hypothetical image data assuming that all tablets 5, including those outside the pockets 2, have been removed, and that there are no tablets 5 in any of the pockets 2.

[0108] Then, when the encoder 51 generates a predetermined phase (for example, 0°), the reconstructed image data acquisition unit 62a inputs the static frame image data Sf at the time the encoder 51 generated that phase, which is the inspection image data Kg, into the tablet-containing AI model 200 and acquires the reconstructed image data as the first reconstructed image data Sk1 (see Figure 22). The acquired first reconstructed image data Sk1 is input into the first difference calculation unit 62c1 of the abnormality determination unit 62c, which will be described later.

[0109] Furthermore, when the encoder 51 generates a predetermined phase (0°), the reconstructed image data acquisition unit 62a inputs the inspection image data Kg at the time the encoder 51 generated that phase into the tablet-less AI model 201 and acquires the reconstructed image data as the second reconstructed image data Sk2 (see Figure 25). The acquired second reconstructed image data Sk2 is input into the second difference calculation unit 62c2 of the abnormality determination unit 62c, which will be described later.

[0110] Furthermore, when the encoder 51 generates a predetermined phase (for example, 0°), the reconstructed image data acquisition unit 62a inputs the inspection image data Kg at the time the encoder 51 generated that phase to the first difference calculation unit 62c1 and the second difference calculation unit 62c2, respectively. Therefore, each time the encoder 51 generates a predetermined phase (0°), the inspection image data Kg and the first reconstructed image data Sk1 are input to the first difference calculation unit 62c1, and the inspection image data Kg and the second reconstructed image data Sk2 are input to the second difference calculation unit 62c2, respectively.

[0111] In addition, in this third embodiment, the abnormality determination unit 62c includes a first difference calculation unit 62c1, a second difference calculation unit 62c2, an addition unit 62c3, and a final determination unit 62c4.

[0112] The first difference calculation unit 62c1 compares the input inspection image data Kg with the first reconstructed image data Sk1 and calculates the difference in brightness for each pixel of both data. It then identifies pixels whose difference is not within a predetermined tolerance range as difference pixels and obtains first difference image data Sb1 (see Figure 23, etc.) that shows these difference pixels.

[0113] The second difference calculation unit 62c2 compares the input inspection image data Kg with the second reconstructed image data Sk2 and calculates the difference in brightness for each pixel of both data. It then identifies pixels whose difference is not within a predetermined tolerance range as difference pixels and obtains second difference image data Sb2 (see Figure 26, etc.) that shows these difference pixels.

[0114] The addition unit 62c3 obtains added image data Ag (see Figure 28, etc.) by performing a logical AND operation on each pixel of the two difference image data Sb1 and Sb2. The added image data Ag represents the area that shows the difference between the data Sk1 and Kg obtained by comparing the first reconstructed image data Sk1 and the inspection image data Kg (i.e., the difference pixels shown by the first difference image data Sb1) and the area that shows the difference between the data Sk2 and Kg obtained by comparing the second reconstructed image data Sk2 and the inspection image data Kg (i.e., the difference pixels shown by the second difference image data Sb2).

[0115] The final determination unit 62c4 determines whether or not there is an abnormality related to the filling and / or transport of the tablet 5 based on the summed image data Ag. More specifically, if a matching region exists in the summed image data Ag, the final determination unit 62c4 determines that there is an abnormality related to the filling and / or transport of the tablet 5 and generates a trigger. On the other hand, if no matching region exists in the summed image data Ag, the final determination unit 62c4 determines that there is no abnormality related to the filling and transport of the tablet 5. The final determination unit 62c4 may also make a determination based on the area of ​​the matching region, etc.

[0116] Furthermore, if the final determination unit 62c4 determines that there is an abnormality related to the filling and / or transport of the tablet 5, it identifies the portion corresponding to the matching area as the "abnormal portion".

[0117] Here, the determination process by the abnormality determination unit 62c in this third embodiment will be explained in more detail using the flowchart in Figure 20.

[0118] When a predetermined phase (e.g., 0°) is generated by the encoder 51, the determination process is started. First, in step S401, the inspection image data Kg is input to the AI ​​model 200 with a tablet, and reconstructed image data Sk1 is obtained. For example, if the pocket portion 2 of 1 in the inspection image data Kg is not filled with a tablet 5 (see Figure 21), the first reconstructed image data Sk1 will be one in which the empty pocket portion 2 is filled with a tablet 5 (see Figure 22). Also, for example, if the pocket portion 2 of 1 in the inspection image data Kg is not filled with a tablet 5 and the tablet 5 is sticking out of the pocket portion 2 (see Figure 13), a similar first reconstructed image data Sk1 is obtained.

[0119] Next, in step S402, the first difference calculation unit 62c1 acquires the first difference image data Sb1. For example, if the inspection image data Kg is as shown in Figure 21, the portion corresponding to the tablet 5 that should be contained in the pocket portion 2 of 1 is identified as a difference pixel, and the first difference image data Sb1 showing this difference pixel is acquired (see Figure 23). On the other hand, for example, if the inspection image data Kg is as shown in Figure 13, the portion corresponding to the tablet 5 that should be contained in the pocket portion 2 and the portion corresponding to the tablet 5 that has popped out of the pocket portion 2 are identified as difference pixels, and the first difference image data Sb1 showing this difference pixel is acquired (see Figure 24).

[0120] Next, in step S403, the second reconstructed image data Sk2 is obtained by inputting the inspection image data Kg into the tablet-less AI model 201. For example, if the inspection image data Kg is as shown in Figure 21, the second reconstructed image data Sk2 will be as follows: all tablets 5, including those outside the pocket 2, have been erased, and all pocket 2 is empty (see Figure 25). Similarly, if the inspection image data Kg is as shown in Figure 13 (see Figure 13), a similar second reconstructed image data Sk2 will be obtained.

[0121] Next, in step S404, the second difference calculation unit 62c2 acquires the second difference image data Sb2. For example, if the inspection image data Kg is as shown in Figure 21, the portion corresponding to the tablet 5 is identified as a difference pixel, and the second difference image data Sb2 showing this difference pixel is acquired (see Figure 26). On the other hand, for example, if the inspection image data Kg is as shown in Figure 13, the portion corresponding to all tablets 5, including the tablet 5 that has popped out of the pocket 2, is identified as a difference pixel, and the second difference image data Sb2 showing this difference pixel is acquired (see Figure 27).

[0122] Subsequently, in step S405, the addition unit 62c3 acquires the added image data Ad. For example, if the difference image data Sb1 and Sb2 are as shown in Figures 23 and 26, respectively, there are no regions that match these difference image data Sb1 and Sb2, so the added image data Ag (see Figure 28) indicating that there are no matching regions is acquired. On the other hand, for example, if the difference image data Sb1 and Sb2 are as shown in Figures 24 and 27, respectively, the regions related to the tablet 5 outside the pocket 2 match, so the added image data Ag (see Figure 29) indicating the region related to the tablet 5 is acquired.

[0123] Next, in step S406, the final determination unit 62c4 determines whether or not a matching region exists in the added image data Ag. If a matching region exists (step S406; Yes), in step S407, it is determined that there is an abnormality related to the filling and / or transport of the tablet 5. The final determination unit 62c4 also identifies the portion corresponding to the matching region as the "abnormal portion".

[0124] If an abnormality is detected, the abnormality detection unit 62c (final determination unit 62c4) generates a trigger, and as a result, the target data storage unit 63 extracts and stores target data for a predetermined time period before and after the trigger from the video data stored in the ring buffer 61. In addition, the abnormality area mark setting unit 63b places an abnormality area mark Mk (for example, a circular mark) on the part of the target data corresponding to the "abnormal part" identified by the final determination unit 62c4 in a predetermined still frame image data Sf (see Figure 30). In this third embodiment, the part of the pocket 2 that is not filled with tablets 5 is not identified as an "abnormal part," so no abnormality area mark Mk is placed on this part.

[0125] On the other hand, if there is no matching region in the added image data Ag (step S406; No), it is determined in step S408 that there is no abnormality related to the filling and transport of the tablet 5.

[0126] As described above, according to this third embodiment, the trigger does not occur simply because the tablet 5 is not contained in the pocket 2. The trigger only occurs when the tablet 5 is outside the pocket 2, such as when the tablet 5 has popped out of the pocket 2. Therefore, based on the data at the time of the abnormality and the video data before and after, it is possible to more intensively check for errors in filling or transporting the tablet 5 that caused the situation where the tablet 5 was outside the pocket 2, as well as check for any residue of the popped-out tablet 5 on the PTP packaging machine 10.

[0127] Furthermore, the embodiment is not limited to the description above, and may be implemented as follows, for example. Of course, other applications and modifications not exemplified below are also possible.

[0128] (a) In the above embodiment, the camera 52 is a two-dimensional camera that takes two-dimensional images of the tablet 5, etc., but a three-dimensional camera that can obtain video data consisting of 3D image data [for example, a TOF (Time-of-Flight) camera] may be used as the camera 52.

[0129] Furthermore, as shown in Figure 31, both a first camera 52a consisting of a two-dimensional camera and a second camera 52b consisting of a three-dimensional camera may be provided, and the two-dimensional video data obtained by the first camera 52a may be stored separately in the ring buffer 61a, and the three-dimensional video data obtained by the second camera 52b may be stored separately in the ring buffer 61b. When the trigger generation unit 62 generates a trigger, the system may extract and save target data for a predetermined time before and after the trigger from the ring buffer 61a, 61b that stores the video data that caused the trigger. Alternatively, when the trigger generation unit 62 generates a trigger, the system may extract and save target data from both ring buffers 61a, 61b. Therefore, for example, when an abnormality is determined based on the video data obtained by the second camera 52b, not only the video data obtained by the second camera 52b but also the video data obtained by the first camera 52a may be saved as target data.

[0130] (b) In the above embodiment, the camera 52 is configured to photograph both the tablets 5 filled in the pocket 2 and the tablets 5 being transported together with the container film 3. In contrast, the camera 52 may be configured to photograph only one of the tablets 5 filled in the pocket 2 or the tablets 5 being transported together with the container film 3. However, it is not practical to photograph only the tablets 5 filled in the pocket 2, and if the camera attempts to photograph the tablets 5 filled in the pocket 2, the tablets 5 being transported together with the container film 3 will usually also be photographed.

[0131] Alternatively, multiple cameras 52 may be installed along the transport path of the container film 3. One camera 52 may capture the scene of filling the pocket portion 2 with tablets 5, while the other cameras may capture the scene of the tablets 5 being transported along with the transport of the container film 3.

[0132] (c) If the abnormality determination unit 62c determines that the "abnormal part" is outside the container film 3 (for example, that the tablet 5 is on the PTP packaging machine 10), the PTP packaging machine 10 may be stopped and the presence of the "abnormal part" may be notified to prompt the operator to remove the tablet 5.

[0133] Furthermore, if the "abnormal part" is stationary outside the container film 3, in the determination process after determining that there is an abnormality in the filling and / or transport of the tablet 5 due to the presence of the "abnormal part," the "abnormal part" may be ignored, and the presence or absence of an abnormality in the filling and / or transport of the tablet 5 may be determined. In this case, the data related to the presence of this "abnormal part" will be saved only once. Of course, the determination process may also be performed without ignoring this "abnormal part" to determine whether or not there is an abnormality. In this case, the data related to the presence of this "abnormal part" will be saved each time the determination process is performed.

[0134] (d) In the above embodiment, the tablet 5 is given as the "contents," but the contents are not limited to tablets.

[0135] Furthermore, the types and shapes of tablets are not limited to the embodiments described above. For example, tablets include not only pharmaceutical tablets but also tablets used for food and drink. In addition, tablets include uncoated tablets, sugar-coated tablets, film-coated tablets, enteric-coated tablets, and gelatin-coated tablets, as well as various types of capsule tablets such as hard capsules and soft capsules.

[0136] Furthermore, regarding the shape of the tablet, it may not only be circular in plan view, but may also be polygonal in plan view, elliptical in plan view, or oval in plan view, etc.

[0137] (e) The configuration of the PTP sheet to be manufactured is not limited to the above embodiment. For example, the arrangement and number of pockets 2 in one PTP sheet unit are not limited in any way to the above embodiment.

[0138] Furthermore, in the above embodiment, the PTP film 6 has a configuration in which a number of pocket portions 2 corresponding to one sheet are arranged along its width direction. However, it is not limited to this configuration, and for example, a configuration in which a number of pocket portions 2 corresponding to multiple sheets are arranged along its width direction may also be used.

[0139] Furthermore, in the above embodiment, the technical concept of the present invention is applied to a PTP packaging machine 10 that manufactures PTP sheets 1, but the technical concept of the present invention may also be applied to a blister packaging machine that manufactures blister sheets other than PTP sheets 1.

[0140] 1...PTP sheet (blister sheet), 2...Pocket section, 3...Container film, 4...Cover film, 5...Tablet (contents), 10...PTP packaging machine (blister packaging machine), 50...Anomaly data recording system, 51...Encoder, 52...Camera (video recording means), 61...Ring buffer, 62...Trigger generation unit (trigger generation means), 62a...Reconstructed image data acquisition unit (reconstructed image data acquisition means), 62c...Anomaly determination unit (anomaly determination means), 63...Target data storage unit (target data storage means), 63a...Chapter mark assignment unit (chapter mark assignment means), 63b...Anomaly area mark setting unit (anomaly area mark setting means), 191...Encoder unit (encoding unit), 192...Decoder unit (decoding unit), 200...AI model with tablet (identification means for presence of contents), 201...AI model without tablet (identification means for absence of contents).

Claims

1. Applicable to a blister packaging machine for manufacturing a blister sheet, which is formed by attaching a cover film to a container film filled with contents in a pocket portion so as to close the opening of the pocket portion, and relating to contents filled in the pocket portion and transported along with the transport of the container film, an abnormal time data recording system for recording video data related to abnormalities in the filling and / or transport of contents, comprising: a video shooting means that photographs at least the contents filled in the pocket portion and / or the contents transported together with the container film and obtains video data consisting of a plurality of still frame image data stored in chronological order; a ring buffer that stores the video data obtained by the video shooting means; a trigger generating means that generates a trigger when predetermined conditions are met; an encoder that can generate a phase related to the transport of contents; and a target data storage means that, when the trigger generating means generates a trigger, extracts and stores target data for a predetermined time before and after the occurrence of the trigger from the video data stored in the ring buffer, wherein the trigger generating means An abnormal data recording system comprising: an encoding unit for extracting feature quantities from input image data and a decoding unit for reconstructing image data from the feature quantities, wherein the neural network is trained as training data to generate image data corresponding to the still frame image data when a predetermined phase is generated by the encoder, wherein the pocket contains contents and there are no abnormalities related to the filling and transport of the contents; a reconstructed image data acquisition means capable of acquiring reconstructed image data as reconstructed image data by inputting inspection image data, which is the still frame image data from the video recording means obtained by the encoder when a predetermined phase is generated, into the contents identification means; and an abnormality determination means for determining whether there are any abnormalities related to the filling and / or transport of the contents by comparing the reconstructed image data and the inspection image data, wherein the system is configured to generate the trigger when the abnormality determination means determines that there is an abnormality.

2. The trigger generating means comprises a neural network having an encoding unit for extracting feature quantities from input image data and a decoding unit for reconstructing image data from the feature quantities, and is trained to generate an empty content identification means by training only image data corresponding to the still frame image data when a predetermined phase is generated by the encoder, and which is in a state where there is no content; the reconstructed image data acquisition means is capable of acquiring, as first reconstructed image data, the image data reconstructed by inputting the inspection image data to the content identification means, and as second reconstructed image data, the image data reconstructed by inputting the inspection image data to the empty content identification means; and the abnormality determination means is configured to determine whether or not there is an abnormality related to the filling and / or transport of content based on a region showing the difference between the first reconstructed image data and the inspection image data obtained by comparing the said data and a region showing the difference between the second reconstructed image data and the inspection image data obtained by comparing the said data. This is the abnormality data recording system according to claim 1.

3. The abnormal data recording system according to claim 1, characterized in that a plurality of contents presence identification means are provided, each of the plurality of contents presence identification means corresponds to each of the plurality of phases generated by the encoder, and the reconstructed image data acquisition means is configured to input the inspection image data in the same phase as the phase to which the contents presence identification means corresponds to, and acquire the reconstructed image data as the reconstructed image data.

4. The abnormal data recording system according to claim 1, further comprising a chapter marking means for assigning a chapter mark to a location in the target data corresponding to the timing at which the trigger generating means assigned a trigger.

5. The abnormal data recording system according to claim 1, further comprising an abnormal area mark setting means for providing an abnormal area mark indicating an area determined to be abnormal by the abnormality determination means to the static frame image data constituting the target data.

Images