Information processing device, information processing method, information processing program, learning method, and learned model

Abstract

The information processing device is a device for detecting a defective inspection object using a collection of image data of an inspection object without defects. The device includes an image restoration unit, a determination unit, and an output unit. The image restoration unit generates a restored image (Ir) that restores a hidden part from image data (Ih) in which a part of the inspection image is hidden, and the inspection image is an image obtained by photographing an inspection object whose presence or absence of defects is uncertain. The determination unit determines whether there are defects by comparing the restored image (Ir) with the inspection image. The output unit outputs the determination result. In addition, the image restoration unit completes learning through deep learning so that a restored image (Ir) that restores a hidden part can be generated with high precision from image data in which a part of each of a plurality of learning images is hidden, and the learning image is an image obtained by photographing an inspection object without defects. Thus, it is possible to use images of a plurality of inspection objects without defects that can be easily obtained as learning data to perform machine learning for detecting defective inspection objects.

Description

Technical Field

[0001] The present invention relates to an information processing device, an information processing method and an information processing program that can use image data of a normal inspection object without defects to learn and detect an abnormal inspection object with defects, a learning method performed during the learning, and a learned model. Background Art

[0002] Conventionally, there is known a technique for detecting an abnormal inspection object having a defect using image processing. In recent years, in particular, the introduction of a technique using machine learning has been advancing. For example, Patent Document 1 describes a defect detection technique using machine learning.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2018-81629 Summary of the invention

[0006] Problems to be solved by the invention

[0007] Patent document 1 discloses a determination system 301 that can use machine learning to determine the presence or absence of a defective portion Df1 contained in an image of an object. The determination system 301 includes a determination device 101, an accumulation device 131, and a learning device 151. In addition, among the multiple images accumulated in the accumulation device 131, 500 images of the object containing the defective portion Df1, namely defective images Sng, and 500 images of the object not containing the defective portion Df1, namely qualified images Sg, are selected, and each of them is divided into 63 partial images. In addition, when the defective portion Df1 is included in the partial image, a trajectory Tr1 is drawn for the defective portion Df1, and a label display related to the presence or absence of the trajectory Tr1 is performed. Next, the learning device 151 uses the multiple partial images and the label display to perform machine learning. Furthermore, the model that has completed the machine learning is imported into the determination device 101. If image data is input to the model, it is determined whether the defective portion Df1 is included in the image data, and the determination result is output.

[0008] However, in the case of machine learning for detecting defective and abnormal inspection objects, at least thousands to millions of images of multiple inspection objects are required as learning data. On the other hand, in the manufacturing process of industrial products, defects do not occur frequently, and it is actually difficult to obtain thousands to millions of images of defective inspection objects. In addition, the types and states of defects are diverse, including unknown types and states, and it is even more difficult to obtain images containing defects of all types and states.

[0009] The present invention has been proposed in view of such a situation, and an object thereof is to provide a technology capable of detecting an abnormal inspection object having a defect by performing machine learning using images of a plurality of normal inspection objects without defects that can be easily acquired.

[0010] Means for solving problems

[0011] In order to solve the above-mentioned problems, the first invention of the present application is an information processing device that uses a collection of image data of normal inspection objects to detect defective abnormal inspection objects, and the information processing device comprises: an image restoration unit, which generates a restored image that restores a hidden part from image data in which a part of the inspection image is hidden, and the inspection image is an image obtained by photographing an inspection object whose normal or abnormality is uncertain; a judgment unit, which compares the restored image with the inspection image to determine whether the inspection object is normal or abnormal; and an output unit, which outputs the judgment result of the judgment unit, and the image restoration unit completes learning through deep learning so as to be able to generate a restored image that restores the hidden part with high precision from image data in which a part of each of multiple learning images is hidden, and the learning image is an image obtained by photographing a normal inspection object.

[0012] The second invention of the present application is that in the information processing device of the first invention, the image restoration unit generates a plurality of restored images while sequentially changing the position of the hidden part in the image data, and the determination unit determines whether the inspection object is normal or abnormal by comparing the plurality of restored images with the inspection image respectively.

[0013] The third invention of the present application is the information processing device of the second invention, and when the difference between the restored image and the inspection image is greater than a predetermined allowable value, the judgment unit determines the position of the hidden part as the position of the defect, and the output unit also outputs information related to the position of the defect.

[0014] The fourth invention of the present application is that in the information processing device of any one of the first to third inventions, the image restoration unit performs the following processing: encoding processing, extracting features from the inspection image to generate latent variables; and decoding processing, generating the restored image from the latent variables.

[0015] The fifth invention of the present application is the information processing device of the fourth invention, wherein in the learning, the image restoration unit adjusts the parameters of the encoding process and the decoding process by a convolutional neural network.

[0016] The sixth invention of the present application is an information processing method for detecting defective and abnormal inspection objects using a set of image data of normal inspection objects, the method comprising the following steps: a) learning the following processing through deep learning: generating a restored image that restores the hidden part from image data in which a part of each of a plurality of learning images is hidden, the learning image being an image obtained by photographing a normal inspection object; b) comparing the restored image with the inspection image to determine whether the inspection object is normal or abnormal, the restored image being an image restored from image data in which a part of the inspection image is hidden using the processing learned in the step a), the inspection image being an image obtained by photographing an inspection object whose normal or abnormality is uncertain; and c) outputting the determination result of the step b).

[0017] The seventh invention of the present application is an information processing program that uses a set of image data of normal inspection objects to detect defective abnormal inspection objects, and the information processing program enables a computer to perform the following processing: a) image restoration processing, generating a restored image that restores a part of the inspection image from image data in which a part of the inspection image is hidden, and the inspection image is an image obtained by photographing an inspection object that is uncertain whether it is normal or abnormal; b) judgment processing, comparing the restored image with the inspection image to determine whether the inspection object is normal or abnormal; and c) output processing, outputting the judgment result of the judgment processing, in which learning is completed by deep learning so that a restored image that restores the hidden part can be generated with high precision from image data in which a part of each of multiple learning images is hidden, and the learning image is an image obtained by photographing a normal inspection object.

[0018] The eighth invention of the present application is a learning method, which uses deep learning to learn the following processing in order to detect defective and abnormal inspection objects: from image data in which a part of each of multiple learning images is hidden, a restored image that restores the hidden part is generated, and the learning image is an image obtained by photographing a normal inspection object.

[0019] The ninth invention of the present application is to learn a learned model through deep learning for the following processing: in order to detect a defective abnormal inspection object, a restored image in which a hidden part is restored is generated from image data in which a part of each of multiple learning images is hidden, and the learning image is an image obtained by photographing a normal inspection object.

[0020] A tenth invention of the present application is the information processing device according to any one of the first to fifth inventions, wherein the inspection object is a tablet.

[0021] Effects of the Invention

[0022] According to the first to tenth inventions of the present application, by performing machine learning using a plurality of images of normal inspection objects without defects that can be easily acquired, it is possible to detect abnormal inspection objects with defects. Thus, various defects including unknown defects in the inspection objects can be detected with high accuracy.

[0023] In particular, according to the third invention of the present application, an operator can easily reconfirm defects in the inspection image or the inspection object body by visual inspection based on information related to the defect position, thereby further improving the detection accuracy of defective inspection objects.

[0024] In particular, according to the fourth or fifth invention of the present application, even when the position of the inspection object in the inspection image is slightly offset, a defective inspection object can be detected with high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a diagram showing the structure of a tablet printing device.

[0026] Figure 2 This is a three-dimensional view of the area near the conveyor roller.

[0027] Figure 3 This is a bottom view of the nozzle.

[0028] Figure 4 This is a stereogram of the vicinity of the inspection camera.

[0029] Figure 5 This is a block diagram showing the connection between the control unit and each unit in the tablet printing device.

[0030] Figure 6 This is a block diagram conceptually showing a part of the functions of the control unit in the tablet printing device.

[0031] Figure 7 FIG. 1 is a diagram showing an example of a learning image obtained by photographing a normal tablet.

[0032] Figure 8 This is a schematic diagram showing a method of generating a restored image from image data in which a portion of a learning image is hidden. The learning image is an image of a normal tablet.

[0033] Fig. 9 This is a schematic diagram showing a method of generating a restored image from image data in which a portion of a test image is hidden. The test image is an image of a tablet that is uncertain whether it is normal or abnormal.

[0034] Fig.10This is a schematic diagram showing a method of generating a restored image from image data in which a portion of a test image is hidden. The test image is an image of a tablet that is uncertain whether it is normal or abnormal. DETAILED DESCRIPTION

[0035] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In one embodiment of the present invention, a tablet as a pharmaceutical product is described as an example of an inspection object. Then, an apparatus, method, and program for inspecting the tablet for defects such as dirt and scratches after recording an image such as a product name on the surface of the tablet by inkjet method, and detecting abnormal tablets with defects will be described.

[0036] <1. Overall structure of tablet printing device>

[0037] Reference Figure 1 , the overall structure of a tablet printing apparatus 1 according to one embodiment of the present invention is described. The tablet printing apparatus 1 includes an information processing apparatus 200 described later for detecting defects of a tablet 9 . Figure 1 1 is a diagram showing the structure of a tablet printing device 1. The tablet printing device 1 is a device that prints images such as product names, product codes, company names, and logos on the surfaces of the tablets 9 by inkjet printing for the purpose of product identification while conveying a plurality of tablets 9. The tablets 9 of this embodiment have a disc shape (see the following). Figure 4 However, the shape of the tablet 9 may be another shape such as an elliptical shape. In the following description, the direction in which the plurality of tablets 9 are conveyed is referred to as a "conveying direction", and the direction perpendicular and horizontal to the conveying direction is referred to as a "width direction".

[0038] In addition, a groove-shaped cut line 90 for dividing the tablet 9 into half is formed on the tablet 9. Hereinafter, the surface of the tablet 9 where the cut line 90 is formed is referred to as a "cut line surface". The cut line 90 passes through the center of the cut line surface and extends straight to both ends of the cut line surface. In addition, in the present embodiment, it is assumed that the cut line 90 is formed only on one side of the upper surface and the lower surface of the tablet 9 forming a disc shape. That is, in the present embodiment, only one side of the upper surface and the lower surface of the tablet 9 is the cut line surface. However, the cut line 90 may be formed on both sides of the upper surface and the lower surface of the tablet 9 forming a disc shape. That is, the cut line 90 may be formed on both the front and back sides of the tablet 9. Moreover, in the present embodiment, the product name and the like are printed only on the surface of the tablet 9 opposite to the cut line surface along the direction of the cut line 90 located on the back side. However, the printing position in the tablet 9 is not limited to this.

[0039] like Figure 1As shown, the tablet printing device 1 of the present embodiment includes a hopper 10, a feeder unit 20, a conveyor drum 30, a first printing unit 40, a second printing unit 50, a conveyor belt 60, and a control unit 70. The hopper 10, the feeder unit 20, the conveyor drum 30, the first conveyor belt 41 of the first printing unit 40, the second conveyor belt 51 of the second printing unit 50, and the conveyor belt 60 form a conveying mechanism for conveying tablets 9 along a predetermined conveying path.

[0040] The hopper 10 is a feeding part for collecting a plurality of tablets 9 into the device at a time. The hopper 10 is arranged at the uppermost part of the housing 100 of the tablet printing device 1. The hopper 10 has an opening 11 located on the upper surface of the housing 100 and a funnel-shaped inclined surface 12 that gradually converges downward. The plurality of tablets 9 fed into the opening 11 flow into the straight feeder 21 along the inclined surface 12.

[0041] The feeder 20 is a mechanism for conveying the plurality of tablets 9 put into the hopper 10 to the conveying roller 30. The feeder 20 of this embodiment includes a straight feeder 21, a rotary feeder 22, and a supply feeder 23. The straight feeder 21 includes a plate-shaped vibration trough 211. The plurality of tablets 9 supplied from the hopper 10 to the vibration trough 211 are conveyed to the rotary feeder 22 side due to the vibration of the vibration trough 211. The rotary feeder 22 includes a disc-shaped rotating table 221. The plurality of tablets 9 falling from the vibration trough 211 to the upper surface of the rotating table 221 are gathered near the outer periphery of the rotating table 221 due to the centrifugal force generated by the rotation of the rotating table 221.

[0042] The supply feeder 23 has a plurality of cylindrical portions 231 extending vertically downward from the outer peripheral portion of the rotating table 221 to the conveyance drum 30 . Figure 2 3 is a perspective view of the vicinity of the conveying roller 30. Figure 2 As shown, a plurality of cylindrical portions 231 are arranged parallel to each other. Figure 2 In the example of , eight cylindrical parts 231 are arranged. The plurality of tablets 9 conveyed to the outer periphery of the rotating table 221 are respectively supplied to any one of the plurality of cylindrical parts 231 and fall into the cylindrical part 231. Then, the plurality of tablets 9 are stacked in each cylindrical part 231. In this way, the plurality of tablets 9 are dispersedly supplied to the plurality of cylindrical parts 231, and are arranged in a plurality of conveying columns. Then, the plurality of tablets 9 in each conveying column are sequentially supplied to the conveying roller 30 starting from the tablet at the lower end.

[0043] The conveyor roller 30 is a mechanism for delivering a plurality of tablets 9 from the supply feeder 23 to the first conveyor belt 41. The conveyor roller 30 has a substantially cylindrical outer peripheral surface. The conveyor roller 30 is driven by a motor to rotate around a rotation axis extending in the width direction. Figure 1 as well as Figure 2 Rotate in the direction of the arrow. Figure 2 As shown in the figure, a plurality of holding portions 31 are provided on the outer peripheral surface of the conveying roller 30. The holding portion 31 is a recessed portion recessed inward from the outer peripheral surface of the conveying roller 30. A plurality of holding portions 31 are arranged along the circumferential direction on the outer peripheral surface of the conveying roller 30 at positions in the width direction corresponding to the plurality of conveying rows mentioned above. In addition, a suction hole 32 is provided at the bottom of each holding portion 31.

[0044] A suction mechanism is provided inside the conveying roller 30. When the suction mechanism is operated, a negative pressure lower than the atmospheric pressure is generated in each of the plurality of adsorption holes 32. The holding portion 31 uses the negative pressure to adsorb and hold the tablets 9 supplied from the supply feeder 23 one by one. In addition, an air supply mechanism is provided inside the conveying roller 30. The air supply mechanism partially blows pressurized gas from the inner side of the conveying roller 30 to the side of the first conveying conveyor belt 41 described later. As a result, the adsorption state of the tablets 9 is maintained in the holding portion 31 that is not opposite to the first conveying conveyor belt 41, and the adsorption of the tablets 9 is released only in the holding portion 31 that is opposite to the first conveying conveyor belt 41. In this way, the conveying roller 30 adsorbs and holds the plurality of tablets 9 supplied from the supply feeder 23 while rotating, and can deliver these tablets 9 to the first conveying conveyor belt 41.

[0045] A first state detection camera 33 is provided at a position facing the outer peripheral surface of the conveying roller 30. The first state detection camera 33 is a photographing unit that photographs the state of the tablet 9 held by the conveying roller 30. The first state detection camera 33 photographs the tablet 9 conveyed by the conveying roller 30, and sends the obtained image to the control unit 70. The control unit 70 detects whether there is a tablet 9 on each holding unit 31, the front and back surfaces of the tablet 9 held by the holding unit 31, and the direction of the cut line 90 based on the received image.

[0046] The first printing unit 40 is a processing unit for printing an image on one side of the tablet 9. Figure 1 As shown, the first printing unit 40 includes a first transport conveyor belt 41 , a second state detection camera 42 , a first head unit 43 , a first inspection camera 44 , and a first fixing unit 45 .

[0047] The first conveyor belt 41 includes a pair of first pulleys 411 and an annular first conveyor belt 412 mounted between the pair of first pulleys 411. The first conveyor belt 412 is arranged so that a portion thereof is close to and opposite to the outer peripheral surface of the conveyor roller 30. One of the pair of first pulleys 411 is rotated by the power obtained from the motor. As a result, the first conveyor belt 412 is rotated to Figure 1 and Figure 2At this time, the other of the pair of first pulleys 411 rotates with the rotation of the first conveyor belt 412.

[0048] like Figure 2 As shown, a plurality of retaining portions 413 are provided on the first conveyor belt 412. The retaining portion 413 is a recessed portion that is recessed from the outer surface of the first conveyor belt 412 to the inner side. At the width direction positions corresponding to the plurality of conveyor columns, the plurality of retaining portions 413 are arranged along the conveying direction. That is, the plurality of retaining portions 413 are arranged at intervals in the width direction and the conveying direction. The intervals in the width direction of the plurality of retaining portions 413 in the first conveyor belt 412 are equal to the intervals in the width direction of the plurality of retaining portions 31 in the conveying roller 30.

[0049] At the bottom of each holding portion 413, an adsorption hole 414 is provided. In addition, the first conveyor belt 41 has a suction mechanism on the inner side of the first conveyor belt 412. When the suction mechanism is operated, a negative pressure lower than the atmospheric pressure is generated in each of the plurality of adsorption holes 414. The holding portion 413 uses the negative pressure to adsorb and hold the tablets 9 delivered from the conveyor roller 30 one by one. Thus, the first conveyor belt 41 transports the plurality of tablets 9 while holding them in a state of being arranged in a plurality of conveying rows spaced apart in the width direction. In addition, an air supply mechanism is provided on the first conveyor belt 412. When the air supply mechanism is operated, the adsorption holes 414 in the holding portion 413 opposite to the second conveyor belt 51 described later become a positive pressure higher than the atmospheric pressure. Thus, the adsorption of the tablets 9 on the holding portion 413 is released, and the tablets 9 are delivered from the first conveyor belt 41 to the second conveyor belt 51. In addition, among the plurality of tablets 9 conveyed to the first conveyor belt 412, there are a mixture of tablets 9 held by the holding portion 413 from the secant surface side and tablets 9 held by the holding portion 413 from the surface side opposite to the secant surface. Moreover, when each tablet 9 is delivered from the first conveyor belt 41 to the second conveyor belt 51, the front and back sides are reversed.

[0050] The second state detection camera 42 is a photographing unit that photographs the state of the tablet 9 held on the first conveyor belt 41 on the upstream side of the conveying direction of the first nozzle unit 43. The first state detection camera 33 and the second state detection camera 42 photograph the surfaces on the opposite sides of the tablet 9. The image obtained by the second state detection camera 42 is sent from the second state detection camera 42 to the control unit 70. The control unit 70 detects whether there is a tablet 9 on each holding unit 413, the front and back surfaces of the tablet 9 held on the holding unit 413, and the direction of the cut line 90 based on the received image.

[0051] The first nozzle unit 43 is an inkjet nozzle unit that ejects ink droplets onto the upper surface of the tablet 9 transported by the first conveyor belt 41. The first nozzle unit 43 has four first nozzles 431 arranged along the transport direction. The four first nozzles 431 eject ink droplets of different colors onto the upper surface of the tablet 9 held by the holding portion 413 from the secant surface side among the plurality of tablets 9. For example, the four nozzles 431 eject ink droplets of cyan, magenta, yellow, and black. By overlapping the monochromatic images formed by these colors, a multicolor image is printed on the surface of the tablet 9. In addition, the ink ejected from each first nozzle 431 uses edible ink made of raw materials approved by the Japanese Pharmacopoeia, Food Sanitation Law, etc.

[0052] Figure 3 FIG. 4 is a bottom view of a first nozzle 431. Figure 3 In FIG. 4 , the first conveyor belt 412 and the plurality of tablets 9 held on the first conveyor belt 412 are indicated by double-dashed lines. Figure 3 As shown in the enlarged view in FIG. 1 , a plurality of nozzles 430 capable of ejecting ink droplets are provided on the lower surface of the first nozzle 431. In the present embodiment, a plurality of nozzles 430 are arranged two-dimensionally along the conveying direction and the width direction on the lower surface of the first nozzle 431. The nozzles 430 are arranged at staggered positions in the width direction. In this way, if the plurality of nozzles 430 are arranged two-dimensionally, the positions of the nozzles 430 in the width direction can be made close to each other. However, the plurality of nozzles 430 may also be arranged in a row along the width direction.

[0053] The ink droplets are ejected from the nozzle 430 by, for example, a so-called piezoelectric method, that is, a voltage is applied to a piezoelectric element to deform it, thereby pressurizing the ink in the nozzle 430 and ejecting it. However, the ink droplets may be ejected by a so-called thermal method in which a heater is energized to heat and expand the ink in the nozzle 430 and eject it.

[0054] Figure 4 This is a stereoscopic view near the first inspection camera 44. The first inspection camera 44 is a photographing unit for confirming the quality of printing performed by the first nozzle unit 43 and the presence or absence of defects in the tablet 9. The first inspection camera 44 photographs the upper surface of the tablet 9 conveyed to the first conveyor belt 412 on the downstream side of the conveying direction closer to the first nozzle unit 43. In addition, the first inspection camera 44 sends the obtained image to the control unit 70. Based on the received image, the control unit 70 checks whether there are defects such as scratches, dirt, offset of the printing position, or missing dots on the upper surface of each tablet 9. The detection method of these defects will be described in detail later.

[0055] In addition, in the present embodiment, eight first inspection cameras 44 are arranged at positions corresponding to eight tablets 9 arranged in the width direction on the first conveyor belt 412. Each first inspection camera 44 photographs one tablet 9 in the width direction. In addition, each first inspection camera 44 sequentially photographs a plurality of tablets 9 conveyed in the conveying direction. However, the arrangement space of the eight first inspection cameras 44 may be considered, and they may be arranged at positions staggered from each other in the conveying direction.

[0056] The first fixing unit 45 is a mechanism for fixing the ink ejected from the first nozzle unit 43 to the tablet 9. In the present embodiment, the first fixing unit 45 is arranged on the downstream side of the conveying direction closer to the first inspection camera 44. However, the first fixing unit 45 may be arranged between the first nozzle unit 43 and the first inspection camera 44. For example, a hot air drying type heater that blows hot air to the tablet 9 conveyed by the first conveying conveyor belt 41 is used for the first fixing unit 45. The ink attached to the surface of the tablet 9 is dried by the hot air and fixed to the surface of the tablet 9.

[0057] The second printing unit 50 is a processing unit for printing an image on the other side of the tablet 9 after printing by the first printing unit 40. Figure 1 As shown, the second printing unit 50 includes a second transport conveyor belt 51 , a third state detection camera 52 , a second head unit 53 , a second inspection camera 54 , a second fixing unit 55 , and a defective product collecting unit 56 .

[0058] The second transport conveyor 51 transports the plurality of tablets 9 delivered from the first transport conveyor 41 while holding them. The third state detection camera 52 photographs the plurality of tablets 9 transported by the second transport conveyor 51 on the upstream side of the second nozzle unit 53 in the transport direction. The second nozzle unit 53 ejects ink droplets onto the upper surface of the tablets 9 transported by the second transport conveyor 51. The second inspection camera 54 photographs the plurality of tablets 9 transported by the second transport conveyor 51 on the downstream side of the second nozzle unit 53 in the transport direction. The second fixing unit 55 fixes the ink ejected from each nozzle 531 of the second nozzle unit 53 on the tablets 9.

[0059] The second conveyor belt 51 , the third state detection camera 52 , the second nozzle unit 53 , the second inspection camera 54 and the second fixing section 55 have the same structure as the first conveyor belt 41 , the second state detection camera 42 , the first nozzle unit 43 , the first inspection camera 44 and the first fixing section 45 described above.

[0060] The defective product recovery unit 56 recovers the tablets 9 determined to be defective based on the captured images Ip obtained from the first inspection camera 44 and the second inspection camera 54. The defective product recovery unit 56 includes an air supply mechanism and a recovery box 561 arranged on the inner side of the second conveyor belt 51. When the tablets 9 determined to be defective are conveyed to the defective product recovery unit 56, the air supply mechanism blows pressurized gas from the inner side of the second conveyor belt 51 toward the tablets 9. As a result, the tablets 9 fall off the second conveyor belt 51 and are recovered by the recovery box 561.

[0061] The unloading conveyor 60 is a mechanism for unloading a plurality of tablets 9 determined as good products to the outside of the housing 100 of the tablet printing device 1. The upstream end of the unloading conveyor 60 is located below the second pulley 511 of the second conveying conveyor 51. The downstream end of the unloading conveyor 60 is located outside the housing 100. The unloading conveyor 60 uses, for example, a belt conveying mechanism. The plurality of tablets 9 that have passed through the defective product recovery unit 56 are released from the second conveying conveyor 51 to the upper surface of the unloading conveyor 60 due to the removal of the suction of the adsorption holes. Then, the plurality of tablets 9 are unloaded to the outside of the housing 100 by the unloading conveyor 60.

[0062] The control unit 70 controls the operation of each unit in the tablet printing apparatus 1 . Figure 5 70 is a block diagram showing the connection between the control unit 70 and each unit in the tablet printing device 1. Figure 5 As conceptually shown in FIG. 7 , the control unit 70 is composed of a computer including a processor 701 such as a CPU, a memory 702 such as a RAM, a storage device 703 such as a hard disk drive, a receiving unit 704, and a sending unit 705. The storage device 703 stores a computer program P and data D for executing the printing process and inspection of the tablet 9. However, the receiving unit 704 and the sending unit 705 may be provided separately from the control unit 70.

[0063] In addition, the computer program P is read from the storage medium M storing the program P, and is stored in the storage device 703 of the control unit 70. Examples of the storage medium M include a CD-ROM, a DVD-ROM, a flash memory, etc. However, the program P may be input to the control unit 70 via a network.

[0064] In addition, if Figure 5As shown, the control unit 70 is connected to the above-mentioned straight feeder 21, rotary feeder 22, conveying roller 30 (including a motor, a suction mechanism and an air supply mechanism), the first state detection camera 33, the first conveying conveyor belt 41 (including a motor, a suction mechanism and an air supply mechanism), the second state detection camera 42, the first nozzle unit 43 (including a plurality of nozzles 430 of each first nozzle 431), the first inspection camera 44, the first fixing unit 45, the second conveying conveyor belt 51, the third state detection camera 52, the second nozzle unit 53 (including a plurality of nozzles 430 of each second nozzle 531), the second inspection camera 54, the second fixing unit 55, the defective product recovery unit 56 and the unloading conveyor belt 60 via the receiving unit 704 and the sending unit 705 in a manner that can perform wired communication such as Ethernet (registered trademark) and wireless communication such as Bluetooth (registered trademark) or Wi-Fi (registered trademark).

[0065] When the control unit 70 receives information from each unit via the receiving unit 704, it temporarily reads the computer program P and data D stored in the storage device 703 into the memory 702, and the processor 701 performs calculation processing based on the computer program P and data D. In addition, the control unit 70 issues instructions to each unit via the sending unit 705, thereby controlling the operation of each unit. In this way, each process for a plurality of tablets 9 is performed.

[0066] <2. Data processing in the control unit>

[0067] Figure 6 1 is a block diagram conceptually showing a part of the functions of the control unit 70 in the tablet printing device 1. Figure 6 As shown, the control unit 70 of this embodiment includes an angle recognition unit 71, a nozzle control unit 72, and an inspection unit. These functions are realized by temporarily reading the computer program P and data D stored in the storage device 703 into the memory 702, and the processor 701 performs calculations based on the computer program P and data D. In addition, the function of the inspection unit is realized by the information processing device 200 composed of a part or all of the mechanical elements of the control unit 70. The information processing device 200 is installed with a learned learning model generated in advance by machine learning.

[0068] The angle recognition unit 71 has a function of recognizing the rotation angle (direction of the secant line 90) of each tablet 9 being conveyed. The angle recognition unit 71 obtains the images captured by the first state detection camera 33 and the second state detection camera 42, and recognizes the rotation angle of each tablet 9 conveyed by the first conveyor belt 41 based on the captured images. In addition, the angle recognition unit 71 obtains the images captured by the third state detection camera 52, and recognizes the rotation angle of each tablet 9 conveyed by the second conveyor belt 51 based on the captured images.

[0069] As described above, in the present embodiment, the product name and the like are printed along the direction of the secant line 90 located on the back side only on the surface opposite to the secant line surface of the tablet 9. Therefore, the angle recognition unit 71 recognizes the rotation angle (the direction of the secant line 90) when each tablet 9 passes through the first nozzle unit 43 based on the captured images obtained from the first state detection camera 33 and the second state detection camera 42. Similarly, the angle recognition unit 71 recognizes the rotation angle (the direction of the secant line 90) when each tablet 9 passes through the second nozzle unit 53 based on the captured images obtained from the third state detection camera 52.

[0070] In addition, the front and back surfaces of the plurality of tablets 9 being conveyed are not fixed. Figure 4 As shown, there is a case where the tablets 9 held by the holding part 413 from the secant surface side and the tablets 9 held by the holding part 413 from the surface side opposite to the secant surface are mixed and conveyed. In such a case, the angle recognition unit 71 may recognize the rotation angle of some tablets 9 when passing through the first nozzle unit 43 based on the captured image obtained from the first state detection camera 33, and recognize the rotation angle of other tablets 9 when passing through the first nozzle unit 43 based on the captured image obtained from the second state detection camera 42. In addition, the rotation angle of some tablets 9 when passing through the second nozzle unit 53 may be recognized based on the captured image obtained from the third state detection camera 52, and the rotation angle of other tablets 9 when passing through the second nozzle unit 53 may be recognized based on the captured image obtained from the second state detection camera 42.

[0071] The nozzle control unit 72 has a function for controlling the operation of the first nozzle unit 43 and the second nozzle unit 53. Figure 6 As shown, the nozzle control unit 72 includes a first storage unit 721. The function of the first storage unit 721 is realized by, for example, the storage device 703 described above. The first storage unit 721 stores printed image data D1 including information related to an image printed on the tablet 9. The image is a product name, a product code, a company name, a logo, etc., and is formed of, for example, a character string including letters and numbers (see Figure 4 and the following Figure 7 ). However, the image may be a mark or illustration other than a character string. And, the image is printed along the cut line 90 located on the back side on the surface opposite to the cut line surface of the tablet 9. However, the image may be printed on the cut line surface of the tablet 9 along the cut line 90. The printed image data D1 also includes such information specifying the printing position and printing direction of the image on the tablet 9.

[0072] When printing on the surface of the tablet 9 as a product, the head control unit 72 reads the print image data D1 from the first storage unit 721. In addition, the head control unit 72 rotates the read print image data D1 according to the rotation angle recognized by the angle recognition unit 71. Furthermore, the head control unit 72 controls the first head 431 and the second head 531 based on the rotated print image data D1. Thus, the image represented by the print image data D1 is printed on the surface of the tablet 9 along the cut line 90.

[0073] The functions of the inspection unit will be described in detail later.

[0074] <3. About Information Processing Device 200 >

[0075] Next, the structure of the information processing device 200 will be described. As described above, the function of the inspection unit in the control unit 70 is realized by the information processing device 200 which is composed of a part or all of the mechanical elements of the control unit 70. The information processing device 200 is installed with a learned learning model generated in advance by machine learning. The information processing device 200 is a device that can inspect the tablet 9 for defects such as damage and detect defective abnormal tablets 9. Figure 6 As shown, the information processing apparatus 200 includes an image restoration unit 201, a determination unit 202, and an output unit 203 as functions.

[0076] First, the steps of generating a learning model installed in the information processing device 200 through machine learning are described. Figure 6 The learning process is conceptually illustrated by the dotted line in FIG. Figure 7 ). Specifically, on the downstream side closer to the conveying direction than the first nozzle unit 43, the first inspection camera 44 takes multiple images of the tablets 9 without defects such as scars among the tablets 9 conveyed to the first conveyor belt 412. Then, a plurality of images of the upper surface of the tablets 9 taken are prepared as learning images (learning images Io) of normal tablets 9. In this embodiment, 1,000 learning images Io are prepared. In addition, the machine learning itself is usually implemented outside the tablet printing device 1. The plurality of learning images Io are input to the image restoration unit 201.

[0077] When the learning image Io is input to the image restoration unit 201, the image restoration unit 201 divides each learning image Io into a plurality of regions (see Figure 8). In the present embodiment, the image Io is divided into 16 regions (region S1 to region S16) with 4 vertical regions and 4 horizontal regions. Furthermore, the number of divided learning images Io is not limited thereto. In addition, in the present embodiment, the sizes of the divided regions S1 to region S16 are equal to each other. However, the learning image Io may also be divided into a plurality of regions of different sizes.

[0078] Next, the image restoration unit 201 generates image data Ih in which one of the regions S1 to S16 of each learning image Io is hidden. Figure 8 In the upper part of the figure, as an example, the image data Ih in which the area S2 among the areas S1 to S16 of the learning image Io is hidden is illustrated. In addition, the image restoration unit 201 of the present embodiment generates 16 pieces of image data Ih for each learning image Io while hiding one of the areas S1 to S16 in order from the area S1. The image restoration unit 201 generates 16 pieces of image data Ih for each of the 1000 learning images Io, that is, a total of 16000 pieces of image data Ih. However, the image restoration unit 201 may also use a random generator to randomly hide one of the areas S1 to S16 and generate a predetermined number of image data Ih for each learning image Io.

[0079] Next, the image restoration unit 201 performs a learning process based on deep learning so as to generate a restored image Ir that restores a hidden part from each image data Ih with high accuracy. Specifically, the image restoration unit 201 uses the original learning image Io for generating each image data Ih as training data, and performs machine learning on the learning model X (a, b, c...) involved in the image restoration process for generating the restored image Ir with high accuracy. Here, the training data is the correct data. In addition, as an example, Figure 8 The diagram shows a state in which a restored image Ir in which the hidden region S2 is restored with high accuracy is generated from the image data Ih.

[0080] At this time, the image restoration unit 201 repeatedly performs encoding processing for extracting features from the image data Ih to generate latent variables and decoding processing for generating restored images Ir from latent variables through a convolutional neural network. As a convolutional neural network, for example, U-Net or Fusion Net can be cited. Then, using the error back propagation method or the gradient descent method, etc., the parameters of the encoding processing and the decoding processing are adjusted and updated and saved so as to minimize the difference in pixel values ​​between the restored image Ir after the decoding processing and the original learning image Io that generated the image data Ih before the encoding processing. Here, the parameters of the encoding processing and the decoding processing represent multiple parameters a, b, c... in the learning model X (a, b, c...). In addition, the image restoration unit 201 can use each image data Ih for one learning or multiple learning.

[0081] However, the method of performing machine learning on the image restoration process to generate the restored image Ir with high precision is not limited to this. For example, in addition to the learning model X (a, b, c...) for generating the restored image Ir, the image restoration unit 201 may also have a learning model Y (p, q, r...) for comparing the generated restored image Ir with the learning image Io to determine which one is the original image. Moreover, it may also have an adversarial generation network that uses the error back propagation method based on the generation result of the learning model X (a, b, c...) and the determination result of the learning model Y (p, q, r...) to alternately perform machine learning while making the learning model X (a, b, c...) and the learning model Y (p, q, r...) compete with each other. As an adversarial generation network, for example, GANs or pix 2pix can be cited.

[0082] As described above, when the machine learning is completed, the learned learning model X (a, b, c ...) is installed in the information processing device 200. Moreover, the tablet printing device 1 can use the learning model X (a, b, c ...) to perform defect detection of the tablet 9. When performing defect detection of the tablet 9, first, the information processing device 200 in the tablet printing device 1 obtains the captured image Ip of the tablet 9 conveyed to the first conveyor belt 412 on the downstream side of the conveying direction of the first nozzle unit 43 from the first inspection camera 44. In addition, the captured image Ip of the tablet 9 conveyed to the second conveyor belt 512 on the downstream side of the conveying direction of the second nozzle unit 53 is obtained from the second inspection camera 54. Then, the captured image Ip is rotated according to the rotation angle recognized by the angle recognition unit 71 to generate the inspection image Ii. The inspection image Ii is an image of the tablet 9 that is unclear whether it has a defect, that is, it is unclear whether it is normal or abnormal. In addition, in the following description, it is assumed that there is a defect De at the position of the inspection image Ii of the tablet 9 located in the area S15 described later. In the present embodiment, damage is assumed as the defect De, but the defect De may also be contamination caused by ink, a shift in the printing position, or dot missing.

[0083] Next, the image restoration unit 201 divides each inspection image Ii into 16 regions (regions S1 to S16) with 4 vertical regions and 4 horizontal regions as in learning. Next, the image restoration unit 201 generates image data Ih in which one region of regions S1 to S16 of each inspection image Ii is hidden. Fig. 9 as well as Fig.10 The diagrams show how a restored image Ir is generated from image data Ih with high accuracy to restore a hidden region. Fig. 9, a situation in which a restored image Ir (hereinafter referred to as "restored image Ir1") that restores the region S1 is generated with high accuracy from image data Ih (hereinafter referred to as "image data Ih1" for ease of explanation) that hides the region S1 is illustrated. Fig.10 , the diagram shows a situation where a restored image Ir (hereinafter referred to as "restored image Ir15") that restores the region S15 with high accuracy is generated from image data Ih (hereinafter referred to as "image data Ih15" for ease of explanation) that hides the region S15. In addition, since the region S15 is hidden, the image restoration unit 201 cannot recognize the defect De, but for ease of explanation, Fig.10 In the image data Ih, the defect De is displayed in white.

[0084] Next, the image restoration unit 201 performs encoding processing for extracting features from the image data Ih that hides a part of the inspection image Ii and generating latent variables, and decoding processing for generating a restored image Ir from the latent variables, in the same manner as during learning, and uses the learning model X (a, b, c...) learned during learning to sequentially change the position of the hidden part and generate multiple restored images Ir from the image data Ih.

[0085] Specifically, the image restoration unit 201 first generates a restored image Ir1 that restores the region S1 based on the image data Ih1 in which the region S1 is hidden from the inspection image Ii, using the learning model X (a, b, c ...), and outputs it to the determination unit 202. Next, the image restoration unit 201 generates a restored image Ir2 that restores the region S2 based on the image data Ih2 in which the region S2 is hidden from the inspection image Ii, using the learning model X (a, b, c ...), and outputs it to the determination unit 202. The image restoration unit 201 repeatedly performs such restoration processing while sequentially changing the position of the hidden part. After a while, the image restoration unit 201 generates a restored image Ir15 that restores the region S15 based on the image data Ih15 in which the region S15 is hidden from the inspection image Ii, using the learning model X (a, b, c ...), and outputs it to the determination unit 202. Finally, the image restoration unit 201 generates a restored image Ir16 in which the region S16 is restored based on the image data Ih16 in which the region S16 is hidden from the inspection image Ii, using the learning model X(a, b, c . . . ), and outputs it to the determination unit 202 .

[0086] Here, as described above, the learning model X (a, b, c, ...) learned during learning is a model in which parameters are adjusted for generating a restored image Ir in which a hidden portion is restored from image data Ih in which a portion of an image obtained by photographing a normal tablet 9 without a defect De is hidden. Fig. 9As shown, when the image restoration unit 201 generates the restored image Ir1 using the learning model X (a, b, c ...) based on the image data Ih1 in which the region S1 without the defect De in the inspection image Ii is hidden, the restored image Ir1 including the region S1 without the defect De is restored with high accuracy at the position without the defect De in the inspection image Ii. Fig.10 As shown in FIG. 1 , when the image restoration unit 201 generates the restored image Ir15 using the learning model X (a, b, c, ...) based on the image data Ih15 in which the region S15 having the defect De in the inspection image Ii is hidden, the image restoration unit 201 cannot recognize the defect De. Therefore, although the defect De exists in the region S15 in the inspection image Ii, the image restoration unit 201 generates the restored image Ir15 without the defect De without recognizing the existence of the defect De.

[0087] Next, when a plurality of restored images Ir are sequentially input from the image restoration unit 201, the determination unit 202 determines whether the tablet 9 is a normal tablet without defect De or an abnormal tablet with defect De by comparing the plurality of restored images Ir with the inspection image Ii, and outputs the determination result Dr to the output unit 203. Specifically, the determination unit 202 first compares the restored image Ir1 generated by the image restoration unit 201 with the inspection image Ii, and determines whether the difference in pixel value between the restored image Ir1 and the inspection image Ii is greater than a predetermined allowable value. Next, the determination unit 202 compares the restored image Ir2 generated by the image restoration unit 201 with the inspection image Ii, and determines whether the difference in pixel value between the restored image Ir2 and the inspection image Ii is greater than a predetermined allowable value. The determination unit 202 performs such determination processing on all restored images Ir. Soon, the determination unit 202 compares the restored image Ir15 generated by the image restoration unit 201 with the inspection image Ii, and determines whether the difference in pixel values ​​between the restored image Ir15 and the inspection image Ii is greater than a predetermined allowable value. Finally, the determination unit 202 compares the restored image Ir16 generated by the image restoration unit 201 with the inspection image Ii, and determines whether the difference in pixel values ​​between the restored image Ir16 and the inspection image Ii is greater than a predetermined allowable value.

[0088] As described above, there is no defect De in the restored image Ir15 generated by the image restoration unit 201. On the other hand, there is a defect De at a position located in the area S15 in the inspection image Ii. Therefore, the difference in pixel values ​​between the restored image Ir15 and the inspection image Ii is different from other comparison results and becomes a significantly larger value. In this way, when the difference between the restored image Ir and the inspection image Ii is greater than a predetermined allowable value, the determination unit 202 determines the hidden position in the image data Ih that is the source of the restored image Ir as the position where the defect De exists. Then, the determination unit 202 outputs the determination result Dr involving the presence or absence of the defect De and the position of the defect De to the output unit 203.

[0089] In addition, when a plurality of restored images Ir are input from the image restoration unit 201, the determination unit 202 may join together the restored images of the hidden areas in the image data Ih which are the sources of each of the plurality of restored images Ir, and on this basis, compare them with the inspection image Ii as a whole to determine whether the difference in pixel values ​​is greater than a predetermined allowable value.

[0090] Through the above, the tablets 9 conveyed to the first conveyor belt 41 and the tablets 9 conveyed to the second conveyor belt 51 are judged whether they have defects De, and the inspection of all tablets 9 is completed. When the judgment result Dr is input from the judgment unit 202, the output unit 203 outputs information related to the presence or absence of defects De in the tablets 9 and the position of the defects De to the monitor or speaker, etc., and sends information related to the tablets 9 with defects De to the defective product recovery unit 56 to recover them. In addition, when the judgment unit 202 judges that there are no defects De in the tablets 9, the output unit 203 may further display this meaning.

[0091] As described above, in this embodiment, by performing machine learning using images of normal tablets 9 without defects De that can be easily acquired in large numbers, abnormal tablets 9 with defects De can be detected. Thus, various defects De including unknown defects in tablets 9 can be detected with high accuracy.

[0092] In addition, the presence or absence of a defect De and information on the position of the defect De are output from the output unit 203. Thus, the operator or the like can easily recheck the tablet 9 determined to have a defect De using the information on the position of the defect De. Thus, the detection accuracy of the tablet 9 having a defect De can be further improved.

[0093] In addition, the image restoration unit 201 of the present embodiment repeatedly performs encoding processing for extracting features from the image data Ih to generate latent variables and decoding processing for generating a restored image Ir from the latent variables by using a convolutional neural network. Therefore, even if the position of the tablet 9 in the inspection image Ii or the learning image Io is slightly deviated, or if the inspection image Ii or the learning image Io contains some noise, the tablet 9 having the defect De can be detected with high accuracy.

[0094] <4. Modifications>

[0095] As mentioned above, although the main embodiment of this invention was described, this invention is not limited to the said embodiment.

[0096] In the above-described embodiment, the image of the upper surface of the tablet 9 after the tablet 9 is printed is used to perform learning and detection of defects De in the tablet 9. However, the image of the tablet 9 before the tablet 9 is printed may be used to perform learning and detection of defects De in the tablet 9. In addition, the image obtained by photographing the tablet 9 from an oblique direction may be used to perform learning and detection of defects De in the tablet 9. Thus, not only the surface and back of the tablet 9 can be detected, but also defects De existing on the side of the tablet 9 can be detected.

[0097] In the above-mentioned embodiment, the learning model X (a, b, c ...) that has completed machine learning in advance outside the tablet printing device 1 is installed in the information processing device 200 to detect the defects De in the tablet 9. However, it is also possible to perform machine learning in a state where the learning model X (a, b, c ...) is installed in the information processing device 200 in the tablet printing device 1, and then directly detect the defects De in the tablet 9.

[0098] In the above-mentioned embodiment, as an example of the inspection object, a tablet 9 as a pharmaceutical product is used. Furthermore, the information processing device 200 of the above-mentioned embodiment determines the presence or absence of defects De such as damage, contamination, offset of the printing position, or missing dots in the tablet 9 as the inspection object, and the position of the defect De. However, the inspection object may be a substrate such as a film or paper, or a printed substrate, etc., which is printed in various printing devices, or a component used in various devices, etc. That is, the inspection object can be an object that has a roughly constant appearance under normal circumstances. Furthermore, the information processing device 200 may also determine the presence or absence of defects De in the appearance of the inspection object, and the position of the defect De.

[0099] That is, the information processing device of the present invention is an information processing device that detects defective and abnormal inspection objects using a collection of image data of normal inspection objects, and the information processing device comprises: an image restoration unit that generates a restored image that restores the hidden part from image data of a part of the inspection image that is hidden, wherein the inspection image is an image obtained by photographing an inspection object that is uncertain whether it is normal or abnormal; a determination unit that compares the restored image with the inspection image to determine whether the inspection object is normal or abnormal; and an output unit that outputs the determination result of the determination unit, wherein the image restoration unit completes learning through deep learning so that it can generate a restored image that restores the hidden part with high accuracy from image data of a part of each of a plurality of learning images that is hidden, wherein the learning image is an image obtained by photographing a normal inspection object. In addition, when the image restoration unit completes learning, it is sufficient to adjust the parameters of the encoding process and the decoding process, for example, through a convolutional neural network.

[0100] In addition, the information processing method of the present invention is an information processing method for detecting defective abnormal inspection objects using a set of image data of normal inspection objects, and the method has the following steps: a) learning the following processing through deep learning: generating a restored image in which a hidden part is restored from image data in which a part of each of a plurality of learning images is hidden, and the learning image is an image obtained by photographing a normal inspection object; b) comparing the restored image with the inspection image to determine whether the inspection object is normal or abnormal, and the restored image is an image restored from image data in which a part of the inspection image is hidden using the processing learned in step a), and the inspection image is an image obtained by photographing an inspection object whose normal or abnormality is uncertain; and c) outputting the determination result of step b).

[0101] In addition, the information processing program executed by the information processing device of the present invention is an information processing program for detecting defective abnormal inspection objects using a collection of image data of normal inspection objects, and the program enables a computer to execute the following processing: a) image restoration processing, generating a restored image in which a hidden part is restored from image data in which a part of the inspection image is hidden, wherein the inspection image is an image obtained by photographing an inspection object whose normal or abnormality is uncertain; b) judgment processing, comparing the restored image with the inspection image to determine whether the inspection object is normal or abnormal; and c) output processing, outputting the judgment result of the judgment processing, wherein learning is performed through deep learning in the image restoration processing so that a restored image in which a hidden part is restored can be generated with high precision from image data in which a part of each of multiple learning images is hidden, wherein the learning image is an image obtained by photographing a normal inspection object.

[0102] In addition, in the present invention, in order to detect defective and abnormal inspection objects, it is sufficient to learn the following processing through deep learning: generate a restored image in which the hidden part is restored from image data in which a part of the learning image is hidden, and the learning image is an image obtained by photographing a normal inspection object.

[0103] In addition, in the present invention, in order to detect defective and abnormal inspection objects, it is sufficient to have a learned model, and the learned model learns the following processing through deep learning: in order to detect defective and abnormal inspection objects, a restored image in which the hidden part is restored is generated from image data in which a part of each of multiple learning images is hidden, and the learning images are images obtained by photographing a normal inspection object.

[0104] Thus, by performing machine learning using a plurality of images of normal inspection objects without defects that can be easily acquired, it is possible to detect abnormal inspection objects with defects. Thus, various defects including unknown defects in the inspection object can be detected with high accuracy.

[0105] In the above-described embodiment, four heads are provided in each of the first printing unit 40 and the second printing unit 50. However, the number of heads included in each of the printing units 40 and 50 may be 1 to 3, or may be 5 or more.

[0106] In addition, the detailed structure of the tablet printing device 1 may be different from the respective drawings of the present application. In addition, the respective elements in the above-mentioned embodiment and modified examples may be appropriately combined within a range that does not cause inconsistency.

[0107] Explanation of symbols

[0108] 1 Tablet printing device

[0109] 9 tablets

[0110] 10 Hopper

[0111] 20 Feeding Department

[0112] 30 conveyor roller

[0113] 33 First state detection camera

[0114] 40 First Printing Department

[0115] 41 First conveyor belt

[0116] 42 Second state detection camera

[0117] 43. First nozzle unit

[0118] 44 First Inspection Camera

[0119] 45 First Fixing Section

[0120] 50 Second Printing Department

[0121] 51 Second conveyor belt

[0122] 52 Third state detection camera

[0123] 53 Second nozzle unit

[0124] 54 Second inspection camera

[0125] 55 Second fixing section

[0126] 56 Defective Product Recycling Department

[0127] 60 Moving out of the conveyor belt

[0128] 70 Control Department

[0129] 71 Angle recognition unit

[0130] 90 Secant

[0131] 100 Shell

[0132] 200 Information processing device

[0133] 201 Image Restoration Department

[0134] 202 Judgment Department

[0135] 203 Output

[0136] 411 First pulley

[0137] 412 First Conveyor Belt

[0138] 431 First Nozzle

[0139] 511 Second pulley

[0140] 512 Second conveyor belt

[0141] 531 Second nozzle

[0142] 561 Recycling Bin

[0143] 701 Processor

[0144] 702 Memory

[0145] 703 Storage Device

[0146] 704 Receiving Department

[0147] 705 Sending Department

[0148] D Data

[0149] D1 Printing image data

[0150] Defects

[0151] Dr's judgment result

[0152] Ih Image data

[0153] Ii Check Image

[0154] Io Learning Image

[0155] Ip capture image

[0156] Ir restored image

[0157] P Computer Program

[0158] X Learning Model

[0159] Y learning model.

Claims

1. An information processing device, which uses a tablet having a cut line as an inspection object and detects a defective abnormal inspection object using a collection of image data of a normal inspection object, characterized in that: The information processing device comprises: an image restoration unit that generates a restored image in which a part of an inspection image is restored from image data in which a part of the inspection image is hidden, wherein the inspection image is an image obtained by photographing an inspection object whose normality or abnormality is uncertain from an oblique direction in which a side surface of the inspection object can be visually confirmed; a determination unit that compares the restored image with the inspection image to determine whether the inspection object is normal or abnormal; as well as an output unit that outputs a determination result of the determination unit, The image restoration unit completes learning through deep learning so as to generate a restored image that restores the hidden part with high accuracy from image data in which a part of each of a plurality of learning images is hidden, wherein the learning images are images obtained by photographing a normal inspection object from an oblique direction in which the side of the inspection object can be visually confirmed, The secant line passes through the center of a secant surface forming the secant line in the inspection object and extends straight to both ends of the secant surface, and the position of the secant line can be identified in an oblique direction from the end of the secant line appearing on the side of the inspection object.

2. The information processing device according to claim 1, characterized in that The image restoration unit generates a plurality of the restored images while sequentially changing the position of the hidden portion in the image data. The determination unit determines whether the inspection object is normal or abnormal by comparing each of the plurality of restored images with the inspection image.

3. The information processing device according to claim 2, characterized in that: When the difference between the restored image and the inspection image is greater than a predetermined allowable value, the determination unit determines the position of the hidden portion as the position of the defect, The output unit further outputs information related to the position of the defect.

4. The information processing device according to any one of claims 1 to 3, characterized in that: The image restoration unit performs the following processing: an encoding process of extracting features from the inspection image to generate latent variables; and A decoding process generates the restored image from the latent variables.

5. The information processing device according to claim 4, characterized in that In the learning, the image restoration unit adjusts parameters of the encoding process and the decoding process by using a convolutional neural network.

6. An information processing method, which uses a tablet having a cut line as an inspection object and uses a collection of image data of a normal inspection object to detect a defective abnormal inspection object, characterized in that: The information processing method comprises the following steps: a) learning by deep learning the following processing: generating a restored image in which the hidden part is restored from image data in which a part of each of a plurality of learning images is hidden, wherein the learning images are images obtained by photographing a normal inspection object from an oblique direction in which the side of the inspection object can be visually confirmed; b) comparing a restored image restored from image data in which a portion of the inspection image is hidden using the processing learned in step a) with an inspection image, wherein the inspection image is an image obtained by photographing the inspection object whose normality or abnormality is uncertain from an oblique direction in which the side of the inspection object can be visually confirmed, to determine whether the inspection object is normal or abnormal; c) outputting the determination result of step b), The secant line passes through the center of a secant surface forming the secant line in the inspection object and extends straight to both ends of the secant surface, and the position of the secant line can be identified in an oblique direction from the end of the secant line appearing on the side of the inspection object.

7. The information processing method according to claim 6, characterized in that: In the step a), a plurality of restored images are generated while sequentially changing the position of the hidden portion in the image data. In the step b), the plurality of restored images are compared with the inspection image to determine whether the inspection object is normal or abnormal.

8. The information processing method according to claim 7, characterized in that: In the step b), when the difference between the restored image and the inspection image is greater than a predetermined allowable value, the position of the hidden portion is determined as the position of the defect, In the step c), information related to the position of the defect is also output.

9. The information processing method according to any one of claims 6, 7 and 8, characterized in that: In step a), the following processing is performed: an encoding process of extracting features from the inspection image to generate latent variables; and A decoding process generates the restored image from the latent variables.

10. The information processing method according to claim 9, characterized in that: In the step a), in the learning, the parameters of the encoding process and the decoding process are adjusted by a convolutional neural network.

11. A deep learning method, characterized in that: Using tablets with cut lines as inspection objects, in order to detect defective and abnormal inspection objects, the following processing is learned through deep learning: generating a restored image in which the hidden part is restored from image data in which a part of each of a plurality of learning images is hidden, wherein the learning images are images obtained by photographing a normal inspection object from an oblique direction in which the side of the inspection object can be visually confirmed, The secant line passes through the center of a secant surface forming the secant line in the inspection object and extends straight to both ends of the secant surface, and the position of the secant line can be identified in an oblique direction from the end of the secant line appearing on the side of the inspection object.

Images