Methods for creating learning models, learning models, testing equipment, testing methods, and computer-readable recording materials containing computer programs.

VN10062688BActive Publication Date: 2026-10-30HASHIMA
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
VN1202401778
Authority / Receiving Office
VN · VN
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-08-13
Publication Date
2026-10-30
Estimated Expiration
2041-08-13

AI Technical Summary

Technical Problem

Conventional foreign object inspection methods using X-rays face challenges in accurately detecting foreign substances with small differences in pixel values between normal and abnormal areas, leading to decreased accuracy and inability to inspect small foreign objects effectively.

Method used

A learning model generation method that involves irradiating objects with electromagnetic waves, acquiring and processing images to identify blobs, generating mask and background images, and creating a heat map to highlight the position and shape of anomalies, allowing for high-accuracy detection of foreign objects even with small differences in pixel values.

Benefits of technology

This approach enables high-accuracy detection of foreign objects, reducing fatigue and accidents by automating the inspection process, contributing to improved health and economic outcomes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure VN1202401778_0
    Figure VN1202401778_0
Patent Text Reader

Abstract

The invention proposes a method and device capable of detecting anomalies in an object under test with high accuracy. The method for creating a learning model includes the following steps: projecting electromagnetic waves onto the object under test; acquiring images based on the electromagnetic waves transmitted through the object under test; identifying the color spot within the object under test based on the pixel values ​​of the corresponding pixels in the acquired processed image by subjecting the image to image processing; acquiring a masked image provided at the position corresponding to the identified color spot, masked in a shape corresponding to the identified color spot, and anomaly information including compensation information for the masked portion in the processed image and information related to the anomaly of the object under test, including the color spot indicator as the type of object corresponding to the color spot; collecting teaching data including the masked image and anomaly information;Acquire a heat map image in which the location and shape of the colored spot are displayed as a heat map on the processed image based on teaching data when the image is acquired via electromagnetic waves passing through the test object after irradiating the test object with electromagnetic waves as input; and create a learning model combining the heat map image with the processed image to produce a test image that includes the heat map image.
Need to check novelty before this filing date? Find Prior Art

Description

Learning model generation method, learning model, inspection device, inspection method, and computer program

[0001] The present invention relates to object detection, and in particular to a method for generating a learning model, a learning model, an inspection device, an inspection method, and a computer program for inspecting clothing accessories, food, industrial products, etc. as inspection objects and detecting foreign objects mixed in the inspection objects.

[0002] Conventionally, there have been known devices capable of non-destructive, non-contact, and high-speed foreign substance inspection, detection, and detection using inspection waves such as X-rays, terahertz waves, infrared rays, and visible light for inspecting clothing accessories, food, industrial products, and the like, and for inspecting packages in which contents are stored in packaging material (see, for example, Patent Documents 1 and 2). Such devices irradiate an inspection wave such as an electromagnetic wave onto an object to be inspected, measure the amount of the inspection wave that has transmitted through the object to be inspected, and obtain an inspection image of the object to be inspected as an inspection image.

[0003] Inspection using X-rays, for example, involves measuring the density of the object and the density of the foreign object from an X-ray image, setting a predetermined density threshold for foreign object detection, and then determining the presence or absence of foreign objects based on the area and shape of anomalous areas with density levels exceeding the threshold within the density distribution of the acquired X-ray image. The presence or absence of abnormalities in the shape of the object under inspection was previously determined based on the area and perimeter of blobs (lumps) in the inspection image. The area of ​​a blob is calculated by counting the pixels that make up the blob, and the perimeter is calculated based on the arrangement of background pixels surrounding the blob. In general, the amount of X-ray penetration correlates with the density of the object being irradiated with X-rays: the lower the density of the object, the greater the X-ray penetration, and the higher the density of the object, the lower the X-ray penetration. Therefore, in principle, the greater the density difference between the object under inspection and the foreign object, the greater the density difference between the object and the foreign object in the X-ray image, enabling reliable foreign object detection.

[0004] JP 2018-138899 A Japanese Patent No. 5876116 A

[0005] As described above, conventional foreign body inspection methods determine the presence or absence of foreign bodies by comparing with a threshold value. However, depending on the type of object being inspected and the type of foreign body, the difference in pixel values ​​between normal areas and foreign bodies may be small, resulting in problems such as an inability to inspect the object or a decrease in the accuracy of foreign body detection.

[0006] The present invention has been made in consideration of the above circumstances, and aims to provide a method for generating a learning model, a learning model, an inspection device, an inspection method, and a computer program that can detect abnormalities in an object to be inspected with high accuracy.

[0007] A method for generating a learning model according to one aspect of the present invention includes the steps of: irradiating an object under inspection with electromagnetic waves; acquiring an image according to the electromagnetic waves transmitted through the object under inspection; identifying blobs contained in the object under inspection based on pixel values ​​of pixels in a processed image obtained by image processing the image; and acquiring a mask image masked to a position and shape corresponding to the identified blob; a background image which is the processed image that does not include the blob; and anomaly information including offset information of the masked portion in the processed image and information regarding an anomaly in the object under inspection having a blob label which is a type of object corresponding to the blob. the step of irradiating the object to be inspected with electromagnetic waves, and when an image acquired in response to the electromagnetic waves transmitted through the object to be inspected is input, acquiring a heat map image in which the position and shape of the blob are displayed as a heat map on the processed image based on the training data; and generating a learning model that outputs an inspection image including the heat map image by synthesizing the heat map image with the processed image, wherein the background image is an image with a shape different from that of the blob synthesized at an arbitrary position on the processed image.

[0008] A learning model according to one aspect of the present invention irradiates an object under inspection with electromagnetic waves, acquires an image in response to the electromagnetic waves transmitted through the object under inspection, identifies blobs contained in the object under inspection based on pixel values ​​of pixels in a processed image obtained by image processing the image, and includes an input layer to which are input a mask image masked to a position and shape corresponding to the identified blob, a background image which is the processed image not including the blob, and anomaly information including offset information of the masked portion in the processed image and information regarding an anomaly in the object under inspection having a blob label which is a type of object corresponding to the blob; and a heat map to display the position and shape of the blob on the processed image. The computer system includes an output layer that acquires a heat map image displayed as a map and synthesizes the heat map image with the processed image to output an inspection image including the heat map image, and an intermediate layer whose parameters are learned based on the mask image and the anomaly information, and when the mask image and the anomaly information are input to the input layer, the computer is caused to function to output the inspection image including the heat map image from the output layer after calculation by the intermediate layer, and when the background image is input to the input layer, the computer is caused to function to output the inspection image not including the heat map image from the output layer after calculation by the intermediate layer.

[0009] An inspection device according to one aspect of the present invention includes an irradiation unit that irradiates an object under inspection with electromagnetic waves; an image generation unit that generates an image in accordance with the electromagnetic waves that have passed through the object under inspection; an image processing unit that performs image processing on the image and acquires a processed image; an object identification unit that identifies blobs contained in the object under inspection based on pixel values ​​of pixels in the processed image; a mask image that is masked to a position and shape corresponding to the identified blob; a background image that is the processed image that does not include the blob; and anomaly information that includes offset information of the masked portion in the processed image and information regarding an anomaly in the object under inspection that has a blob label that is a type of object corresponding to the blob. as training data; an anomaly information acquisition unit that inputs the mask image and anomaly information acquired from the acquisition unit into a learning model that outputs information about an abnormality in the object to be inspected when an image acquired in response to the electromagnetic waves that have passed through the object to be inspected is input, and acquires a heat map image in which the position and shape of the blob are displayed as a heat map on the processed image; and an output unit that outputs an inspection image including the heat map image by combining the heat map image with the processed image, wherein the background image is an image with a shape different from the blob that has been combined at an arbitrary position on the processed image.

[0010] An inspection method according to one aspect of the present invention includes the steps of: irradiating an object under inspection with electromagnetic waves; acquiring an image in accordance with the electromagnetic waves transmitted through the object under inspection; identifying blobs contained in the object under inspection based on pixel values ​​of pixels in a processed image obtained by image processing the image; acquiring a mask image masked at a position and shape corresponding to the identified blob; a background image of the processed image that does not include the blob; and acquiring anomaly information including offset information of the masked portion in the processed image and information regarding an anomaly in the object under inspection having a blob label that is a type of object corresponding to the blob. The method causes a computer to execute a process including the steps of: acquiring training data including a mask image, the background image, and the abnormality information; irradiating the object to be inspected with electromagnetic waves, and when an image acquired in response to the electromagnetic waves that have passed through the object to be inspected is input, acquiring a heat map image that displays the position and shape of the blob as a heat map on the processed image based on the training data; and generating a learning model that outputs an inspection image including the heat map image by combining the heat map image with the processed image, wherein the background image is an image with a shape different from the blob that has been combined at an arbitrary position on the processed image.

[0011] A computer program according to one aspect of the present invention includes the steps of: irradiating an object under inspection with electromagnetic waves; acquiring an image in accordance with the electromagnetic waves transmitted through the object under inspection; identifying blobs contained in the object under inspection based on pixel values ​​of pixels in a processed image obtained by image processing the image; and acquiring a mask image masked at a position and shape corresponding to the identified blob, a background image which is the processed image not including the blob, and anomaly information including offset information of the masked portion in the processed image and information regarding an anomaly in the object under inspection having a blob label which is a type of object corresponding to the blob. The method causes a computer to execute a process including the steps of: acquiring training data including the mask image, the background image, and the abnormality information; irradiating the object to be inspected with electromagnetic waves, and when an image acquired in response to the electromagnetic waves transmitted through the object to be inspected is input, acquiring a heat map image in which the position and shape of the blob are displayed as a heat map on the processed image based on the training data; and generating a learning model that outputs an inspection image including the heat map image by synthesizing the heat map image with the processed image, wherein the background image is an image with a shape different from the blob synthesized at an arbitrary position on the processed image.

[0012] According to the present invention, even when the difference in pixel values ​​between normal and abnormal portions in an inspection image of an object to be inspected is small, when the shape of the foreign object is unique, or when the size of the foreign object is extremely small, it is possible to distinguish between normal and abnormal portions with high accuracy and identify abnormal conditions such as foreign object contamination. Therefore, information regarding the presence or absence of abnormalities, such as the presence or absence of foreign objects, can be detected with high accuracy. Furthermore, the present invention frees users from long, repetitive, and monotonous tasks that require intense concentration, thereby reducing fatigue and accidents associated with fatigue, and leading to a reduction in health damage caused by foreign object contamination accidents, thereby contributing to Goal 3 "Good health and well-being" and Goal 8 "Decent work and economic growth" of the United Nations-led Sustainable Development Goals (SDGs).

[0013] FIG. 1 is a schematic diagram showing the configuration of an anomaly inspection system according to a first embodiment of the present invention. FIG. 2 is a block diagram showing the configuration of an anomaly inspection system according to a first embodiment of the present invention. FIG. 3 is a schematic diagram showing a known example of a Mask R-CNN convolutional neural network. FIG. 4 is a schematic diagram showing an example of a record layout of an inspection image DB151 according to a first embodiment of the present invention. FIG. 5 is a schematic diagram relating to the generation process of an anomaly detection model according to a first embodiment of the present invention. FIG. 6 is a flowchart relating to the generation process of an anomaly detection model according to a first embodiment of the present invention. FIG. 7 is a flowchart showing the processing procedure for an anomaly detection process by the anomaly inspection system according to a first embodiment of the present invention. FIG. 8 is a schematic diagram showing a processed image of an object to be inspected. FIG. 9 is a schematic diagram showing a mask image of an object to be inspected. FIG. 10 is a schematic diagram showing a processed image of an object to be inspected that does not contain a foreign object. FIG. 11 is a schematic diagram showing a marking image for performing background image processing. FIG. 12 is a schematic diagram showing a processed image after background image processing has been performed. FIG. 13 is a schematic diagram showing a processed image of an object to be inspected that contains a foreign object. FIG. 14 is a block diagram showing the configuration of an anomaly inspection system according to a second embodiment of the present invention. 10 is a schematic diagram relating to conditions for generating a relearning process according to a second embodiment of the present invention;FIG. 11 is a flowchart illustrating a processing procedure for a relearning process according to a second embodiment of the present invention;FIG.

[0014] Hereinafter, embodiments of the present invention will be described with reference to the drawings. For convenience of explanation, the same reference numerals in each drawing indicate the same or equivalent parts unless otherwise specified, and elements having similar structures and / or functions are intended, and unnecessary detailed explanations may be omitted. Furthermore, duplicated explanations will be appropriately simplified or omitted. In addition, in this embodiment, the symbol "to" may be used to indicate a numerical range, and the numerical values ​​written before and after "to" are included in the numerical range. (Embodiment 1) [Abnormality inspection system]

[0015] FIG. 1 is a schematic diagram showing the configuration of an abnormality inspection system 10 according to a first embodiment of the present invention, and FIG. 2 is a block diagram showing the configuration of the abnormality inspection system 10. The abnormality inspection system 10 includes an information processing device 1 and an X-ray inspection device 2. In this first embodiment, the abnormality inspection system 10 is described, in which inspection objects 4 are placed in a dispersed state on a conveyor belt 61 of a conveyor unit (conveyor) 6 of the X-ray inspection device 2, and abnormalities are detected based on images of the inspection objects 4 captured using X-rays. Examples of the inspection objects 4 include shoes, bags, clothing, and food. Note that the inspection object 4 is not limited to the above-listed objects such as shoes, and can be any object whose shape can be detected by an image. Furthermore, the inspection object 4 is not limited to the above-listed objects such as shoes, and can be any object stored in packaging or placed on a tray. The information processing device 1 and the X-ray inspection device 2 can be configured separately or integrated. [X-ray Inspection Device]

[0016] The X-ray inspection apparatus 2 includes an irradiation unit 3, a conveyor belt 60, a light-emitting unit 24, a light-receiving unit 25, a controller 27, and a display unit 22. The display unit 22 includes a first display unit 41 and a second display unit 42. The X-ray inspection apparatus 2 also includes a lower housing 23 having a horizontally elongated rectangular parallelepiped shape, and an upper housing 21 that is provided on the lower housing 23 and is smaller than the lower housing 23. A conveyor unit 6 is provided within the lower housing 23. The conveyor unit 6 includes a conveyor belt 60, a conveyor belt drive unit 61, and a conveyor belt control unit 62. The conveyor belt control unit 62 receives commands from the control unit 11, including the distance and speed to convey the object 4, and controls the operation of the conveyor belt drive unit 61 based on the commands.

[0017] The upper housing 21 houses the irradiation unit 3. The irradiation unit 3 includes an irradiation body 31 and an irradiation control unit 32. The irradiation control unit 32 receives commands from the control unit 11, including detailed settings such as the X-ray irradiation time and X-ray intensity, and controls the X-rays irradiated from the irradiation body 31 based on the commands. The irradiation unit 3 can also be controlled to irradiate the object 4 with X-rays at any angle (e.g., 0°, 25°, or 45° relative to the X, Y, and Z coordinate axes shown in FIG. 1). The presence or absence of foreign matter can be more accurately confirmed by X-ray irradiation images obtained by irradiation at different angles. A display unit 22 is provided on the front of the upper housing 21. The display unit 22 displays X-ray images of the object 4 generated by the image generation unit 111, which will be described later. The display unit 22 is a display device such as an LCD (Liquid Crystal Display), and the first display unit 41 and the second display unit 42 included in the display unit 22 may be divided into separate screens within the same display device, or may be separate display devices. The upper housing 21 is provided with a detection unit 7. The detection unit 7 is called a TDI (Time Delay Integration) camera or TDI sensor, and includes a scintillator that emits fluorescence in response to the intensity of X-rays, and a number of photodiodes that detect the fluorescence emitted by the scintillator.

[0018] In a TDI camera, photodiodes are lined up linearly in the X direction, which is perpendicular to the Y direction, which is the direction of movement of the object under inspection, to form one detection line, and multiple such detection lines are provided side by side. The multiple detection lines are arranged parallel to the Y direction, which is the direction of movement of the object under inspection. Note that multiple photodiodes are regularly lined up in the X and Y directions within the housing of one TDI camera, and the photodiodes lined up in the X direction form one detection line.

[0019] First, during inspection, the user places the object under inspection 4 on the conveyor belt 60 and operates the controller 27 to convey the object under inspection 4 until it is positioned below the irradiation unit 3. When the object under inspection 4 is conveyed below the X-ray irradiation unit 3, the irradiation unit 3 irradiates X-rays toward the object under inspection 4. The irradiated X-rays pass through the object under inspection 4 and reach the light-emitting unit 24 located below the conveyor belt 60. The light-emitting unit 24 emits light in accordance with the amount of X-rays that have reached it, and the light-receiving unit 25 receives the emitted light. Then, in the X-ray inspection apparatus 2, the image generation unit 111 in the control unit 11 generates an X-ray irradiation image from the light received by the light-receiving unit 25 and displays it on the display unit 22.

[0020] The user visually checks the X-ray irradiation image displayed on the display unit 22 to determine whether or not any foreign matter has been mixed into the inspection object 4. The basic configuration of this X-ray inspection device 2 is the same as that of conventionally known X-ray inspection devices, and is not limited to this embodiment.

[0021] Next, FIG. 2 shows an outline of a block diagram illustrating the relationship between the components of the X-ray inspection apparatus 2 according to the present invention. The X-ray inspection apparatus 2 is centrally managed by a control unit 11, and the components included in the X-ray inspection apparatus 2 are controlled based on instructions from the control unit 11. The control unit 11 includes an image generation unit 111, an image processing unit 112, an object identification unit 113, an acquisition unit 114, an abnormality information acquisition unit 115, an output unit 116, a control and display unit 117, and a discrimination unit 118. The control unit 11 is implemented in an existing computer (equipped with a CPU, a storage device, etc.). The control unit 11, the image generation unit 111, the image processing unit 112, the object identification unit 113, the acquisition unit 114, the abnormality information acquisition unit 115, the output unit 116, the control and display unit 117, and the discrimination unit 118 are realized on software.

[0022] The conveyor belt 60 includes a conveyor belt drive unit 61 and a conveyor belt control unit 62. The conveyor belt control unit 62 receives commands from the control unit 11, including the distance and speed for conveying the inspection object 4, and controls the operation of the conveyor belt drive unit 61 based on the commands.

[0023] The light emitting unit 24 emits light upon receiving X-rays that are irradiated from the irradiator 31 and have passed through the inspection object 4. The light receiving unit 25 receives the emitted light and transmits it to the image generating unit 111.

[0024] The image generating unit 111 generates an X-ray irradiation image obtained by viewing through the inspection object 4 from the light transmitted from the light receiving unit 25. It is also possible to generate a black and white image according to the amount of transmitted light, and then generate a colored X-ray irradiation image by adding color according to the density value of black and white, etc.

[0025] Existing processing methods can be used for this image processing, and it is possible to change the processing applied to the image as appropriate so that the user can easily find foreign matter mixed in the inspection object 4. For example, it is possible to change all kinds of setting information related to image processing, such as changing the color applied to the image, changing the brightness or luminance, changing the resolution, or changing the image processing software.

[0026] The data file generation unit 220 (not shown) generates a data file that stores the X-ray irradiation image generated by the image generation unit 111 together with inspection information that includes at least any of the equipment setting information of the X-ray inspection device 2 at the time of inspection, image generation processing information, detected object information, and user information.

[0027] The data file generated by the data file generator 220 is saved in a format that can be read by other image display devices, such as JPEG or MPEG format, so that X-ray irradiation images can be displayed in the same manner as before on existing image display devices (such as computers or tablet terminals) that have image software that can read these formats.

[0028] Furthermore, the auxiliary storage unit 15, which will be described in detail below, can store data files generated by the data file generation unit 220. This auxiliary storage unit 15 may be a main storage unit (such as RAM) provided in the control unit 11, or an external storage medium (such as a CD or DVD).

[0029] The data reading unit 221 (not shown) can read data files stored in the auxiliary storage unit 15. The inspection information read by the data reading unit 221 is transmitted to the control unit 11, and the control unit 11 sends a command to the irradiation unit 3, the conveyor belt 60, or the image generation unit 111. [Information Processing Device]

[0030] The information processing device 1 is a computer capable of various information processing and information transmission / reception. Computers are classified into personal computers (PCs), workstations, server computers, mainframes, supercomputers, ultracomputers, minicomputers, office computers, pocket computers, microcomputers, personal digital assistants (PDAs), and sequencers (PLCs: programmable logic controllers) depending on their intended use, specifications, performance, etc., but all of these can be used as information processing devices. The X-ray inspection device 2 is connected to the information processing device 1 via a network N such as a LAN or the Internet, and is capable of communication with the information processing device 1. In addition, a cloud computer connected to the information processing device 1 via the network N such as the Internet can also execute the processing of the information processing device 1. In this embodiment, the information processing device 1 is a personal computer, and the following description will be given.

[0031] The information processing device 1 acquires an image of the object to be inspected 4 and performs a process of detecting the presence or absence of an abnormality in the object to be inspected 4 based on the image. In this embodiment, the information processing device 1 detects the presence or absence of an abnormality using an anomaly detection model 152 (described later) that has been trained to detect (identify) an abnormality in the object to be inspected 4 from an image by machine learning. The anomaly detection model 152 is expected to be used as a program module that is part of artificial intelligence software. The information processing device 1 inputs a processed image 50, which is an image processed based on an image of the object to be inspected 4 captured using the X-ray inspection device 2, to the anomaly detection model 152 and obtains, as an output, a classification result indicating the presence or absence of an abnormality in the object to be inspected 4. In accordance with instructions from the anomaly detection model 152, the control unit 11 of the information processing device 1 performs calculations on the mask image 51 and / or the background image 52, etc. input to the input layer, and operates to output a classification result indicating the presence or absence of an abnormality. The presence or absence of an abnormality corresponds to the presence or absence of an abnormality (deformation) in the shape of the object to be inspected 4 and the presence or absence of a foreign object, etc. In the following, in this embodiment, a case will be described in which the type of abnormality in the inspection object 4 is the presence or absence of a foreign object, and the information processing device 1 inputs a mask image 51 and / or a background image 52, etc., into the anomaly detection model 152. If a foreign object is found in the inspection object 4, the information processing device 1 acquires the position, shape, etc. of the foreign object as a heat map image 53. Next, the information processing device 1 outputs an inspection image 54 by combining the heat map image 53 with the processed image 50. The information processing device 1 selects whether or not to return the inspection object 4 to the manufacturer based on the acquired abnormality information such as the heat map. Here, the abnormality information includes information about the abnormality in the inspection object, such as offset information of the heat map portion in the inspection image 54 including the heat map image 53 at a position and shape corresponding to a blob identified as a foreign object in the processed image 50, and a blob label, which is the type of object corresponding to the blob.

[0032] The information processing device 1 includes a control unit 11, a main memory unit 12, a communication unit 13, an input unit 14, and an auxiliary memory unit 15. The control unit 11 has one or more arithmetic processing devices such as a central processing unit (CPU), a micro-processing unit (MPU), a graphics processing unit (GPU), etc., and performs various information processing, control processing, etc. related to the information processing device 1 by reading and executing a program 153 stored in the auxiliary memory unit 15. Each functional unit in FIG. 2 is executed by the control unit 11 operating based on the program 153.

[0033] The control unit 11 includes, as functional units, an image generation unit 111, an image processing unit 112, an object identification unit 113, an acquisition unit 114, an abnormality information acquisition unit 115, an output unit 116, a control and display unit 117, and a discrimination unit 118. The image generation unit 111 generates an X-ray image based on X-rays that have passed through the object under inspection 4 and been detected by the detection unit 7. The image processing unit 112 performs image processing on the X-ray image using a smoothing filter, a feature extraction filter, etc., to generate a processed image 50. The object identification unit 113 performs blob analysis of the object under inspection 4 based on the processed image 50 to identify blobs. The acquisition unit 114 includes a specific image acquisition unit 124 and a specific information acquisition unit 134. The specific image acquisition unit 124 acquires a mask image 51 masked to a position and shape corresponding to the blob identified by the object identification unit 113. Here, the mask image 51 is an image in which the portion of the foreign object to be detected is masked (filled in white or black, and the rest is filled in the opposite color). The specific image acquisition unit 124 also acquires a background image, which is the processed image 50 that does not include blobs. A method for generating a background image will be described in detail below. The specific information acquisition unit 134 acquires anomaly information including offset information of the masked portion in the processed image 50 and information about anomalies in the object under inspection 4 having a blob label, which is a type of object corresponding to a blob. The mask image 51, background image 52, and anomaly information are input to the anomaly detection model 152 as training data. The anomaly information acquisition unit 115 irradiates the object under inspection 4 with electromagnetic waves, and when an image acquired in response to the electromagnetic waves transmitted through the object under inspection 4 is input, inputs the mask image 51 and anomaly information acquired by the acquisition unit 114 into a learning model that outputs information about anomalies in the object under inspection 4, thereby acquiring a heat map image 53 in which the positions and shapes of blobs are displayed as a heat map on the processed image 50. The output unit 116 combines the heat map image 53 acquired by the abnormality information acquisition unit 115 onto the processed image 50 and outputs an inspection image 54 including the heat map image 53 .

[0034] The control display unit 117 displays setting input information, setting change information, setting confirmation information, etc. for the various functional units constituting the anomaly inspection system described in Fig. 2. The control display unit 117 can also add, delete, and edit data and programs contained in the inspection image DB 151, the anomaly detection model 152, and the program 153 provided in the auxiliary storage unit 15. The discrimination unit 118 selects whether or not to return the inspection object 4 to the manufacturer, stops the conveying unit 6, and marks the inspection object 4 with a mark indicating whether or not an anomaly has been detected, based on the inspection image 54 including the heat map image 53 of the inspection object 4 and the anomaly information.

[0035] The main memory unit 12 is a temporary storage area such as a static random access memory (SRAM), a dynamic random access memory (DRAM), or a flash memory, and temporarily stores data necessary for the control unit 11 to execute arithmetic processing. The communication unit 13 is a communication module for performing communication-related processing, and transmits and receives information to and from the outside.

[0036] The auxiliary storage unit 15 is a large-capacity memory, a hard disk, or the like, and stores a program 153 and other data required for the control unit 11 to execute processing. The auxiliary storage unit 15 also stores an inspection image DB 151 and the anomaly detection model 152. The inspection image DB 151 is a database that stores information on the inspection object 4, which is the target of anomaly detection.

[0037] The program 153 stored in the auxiliary storage unit 15 may be provided from a recording medium on which the program 153 is readably recorded. The recording medium is, for example, a portable memory such as a USB (Universal Serial Bus) memory, an SD (Secure Digital) card, a micro SD card, or a CompactFlash (registered trademark). The program 153 recorded on the recording medium is read from the recording medium using a reading device (not shown) and stored in the auxiliary storage unit 15. Furthermore, if the information processing device 1 includes a communication unit capable of communicating with an external communication device, the program 153 stored in the auxiliary storage unit 15 may be provided by communication via the communication unit.

[0038] The auxiliary storage unit 15 may be an external storage device connected to the information processing device 1. The information processing device 1 may also be a multi-computer consisting of multiple computers, or a virtual machine virtually constructed by software.

[0039] FIG. 3 is a schematic diagram showing a mask image 51 masked to a position and shape corresponding to a blob identified by the object identification unit 113. The object identification unit 113 of the control unit 11 detects blobs by pattern matching, edge detection, R-CNN (Regions with Convolutional Neural Networks), or the like. The mask image 51 is shown with the masked portion in white and the other background in black. The object identification unit 113 according to the present invention uses the architecture of Mask R-CNN (Mask Regions with Convolutional Neural Networks). [Mask R-CNN]

[0040] The Mask R-CNN network is known, and its architecture is shown in Figure 3. Referring to Figure 3, Mask R-CNN is a CNN divided into two sets, and the processing is divided into stage 1 and stage 2.

[0041] "Stage 1" allows pre-processing of the input image and essentially comprises a first Feature Pyramid Network (FPN) sub-network, the operation of which is described in more detail below.

[0042] "Stage 2" completes the detection and terminates (producing the desired output, i.e., a segmentation mask of the detected target elements and / or one or more detection boxes and / or classes).

[0043] Stage 1 of Mask R-CNN also includes a third subnetwork of the region proposal network (RPN) type, which is also a detection network, and a cropping module (ROI align, where ROI stands for "region of interest"). The FPN identifies potential regions of interest in the feature maps of the FPN output (i.e., those likely to contain target elements), and the cropping module "realigns" the feature maps according to the coordinates of these regions of interest to facilitate the operation of the detection network.

[0044] The CNN also includes at least a first FPN type network and a second detection network type sub-network, and optionally a third RPN type sub-network and a trimming module.

[0045] Here, the FPN (first sub-network) is the most important part of the Mask R-CNN network.

[0046] FPNs differ in that they are composed of ascending branches ("bottom-up") and descending branches ("top-down"), as well as lateral connections between the ascending and descending branches.

[0047] The ascending branch, also known as the backbone of the network as a whole, is a conventional feature extraction network, which can be of many types, in particular a conventional CNN (direct succession block of convolutional layers CONV, batch normalization layer BN, and nonlinear layer NL). The backbone extracts from an input image multiple initial feature maps that represent the input image at different scales. More precisely, the backbone consists of multiple successive convolutional blocks, whereby a first block generates a first initial feature map from the input image, then a second block generates a second initial feature map for the first initial feature map, and so on.

[0048] Conventionally, for convolutional neural networks, it is understood that each successive map gets smaller in scale (i.e., as resolution decreases, the feature maps get "smaller" and therefore less detailed), but with increasing semantic depth because increasingly higher levels of structure are being captured in the image. Specifically, the initial feature maps increase in number of channels as their size decreases.

[0049] In practice, a pooling layer is placed between two blocks to reduce the size by a factor of 2, and the number of filters in the convolutional layer used (typically 3x3 convolutions) is increased (preferably doubled) from one block to the other, so that, for example, in the 5-level case, there are consecutive channel numbers of 32, 64, 128, 256, and 512, and consecutive map sizes (for a 512x512 input image) of 512x512, 256x256, 128x128, 64x64, and 32x32. [Process for Generating a Learning Model]

[0050] FIG. 4 is a schematic diagram showing an example of a record layout of the inspection image DB 151. The inspection image DB 151 includes a mask image ID column, an offset information column, a blob label column, and a mask image column. The mask image ID indicates information for identifying the mask image 51. The offset information (X, Y, W, H) indicates coordinate information regarding the position of the mask portion on the mask image 51 of the inspection object 4 placed on the conveyor belt 60 in the X-ray irradiation area. Here, X and Y indicate information regarding the X- and Y-coordinates, and W and H indicate information regarding the width and height of the mask image 51. The blob label (foreign matter type label) indicates the type of abnormality corresponding to the mask portion, such as a scratch, iron ball, needle, or dirt. The inspection image DB 151 also stores a background image 52, which is a processed image 50 that does not include blobs. Furthermore, the inspection image DB 151 records the inspection image 54 including the heat map image 53 as an image containing foreign matter, and records the inspection image 54 not including the heat map image 53 as an image not containing foreign matter (background image).

[0051] Fig. 5 shows a schematic diagram relating to the process of generating the anomaly detection model 152. Fig. 5 conceptually shows the process of generating the anomaly detection model 152 by performing machine learning.

[0052] The information processing device 1 constructs (generates) a neural network as an anomaly detection model 152 that inputs the processed image 50, etc., and outputs information indicating anomalies in the shape of the object 4 and the presence or absence of foreign matter by learning image features of anomalies in the processed image 50 of the object 4. The neural network is a MASK R-CNN (Mask Regions with Convolutional Neural Network), and has an input layer that receives inputs of the mask image 51 and / or the background image 52, etc., an output layer that outputs a discrimination result as to whether an anomaly is present, and an intermediate layer that extracts image features of the mask image 51 and / or the background image 52, etc.

[0053] The input layer has multiple neurons that receive input of pixel values ​​of each pixel included in the inspection image 54 and pass the input pixels to the intermediate layer. The intermediate layer has multiple neurons that extract image features of the inspection image 54 and pass the extracted image features to the output layer. In FIG. 6 , the number of intermediate layers is three, but this is not limited to this. For example, if the anomaly detection model 152 is a CNN, the intermediate layer has a configuration in which convolution layers that convolve the pixel values ​​of each pixel input from the input layer and pooling layers that map the pixel values ​​convolved in the convolution layers are alternately connected, thereby compressing the pixel information of the inspection image 54 and finally extracting image features. In FIG. 6 , the convolution layers and pooling layers are not shown. The output layer has three neurons that output an identification result identifying an abnormality in the inspection object 4 and identifies the presence or absence of an abnormality in the shape of the inspection object 4 and the presence or absence of a foreign object based on the image features output from the intermediate layer. The first neuron outputs a probability value of the normality of the object 4, the second neuron outputs a probability value of the shape abnormality of the object 4, and the third neuron outputs a probability value of the presence of a foreign object in the object 4. Note that there may be three or more types of abnormality.

[0054] In this embodiment, the anomaly detection model 152 is described as a CNN; however, the anomaly detection model 152 is not limited to a CNN, and trained models constructed with other learning algorithms, such as neural networks other than CNN, SVMs (Support Vector Machines), Bayesian networks, and regression trees, can also be applied.

[0055] The information processing device 1 performs learning using training data in which a plurality of mask images 51 are associated with information indicating an abnormality of the object to be inspected 4 in each mask image 51. Here, the information indicating the above is the presence or absence of an abnormality determined on the processed image 50 of the object to be inspected 4, and, if an abnormality is determined to exist, is information regarding an abnormality in the shape of the object to be inspected 4 and the presence or absence of a foreign substance, etc. For example, as shown in Fig. 5 , the training data is data in which a mask image 51 of the object to be inspected 4 is labeled with the ID of the mask image 51, offset information, and a blob label, and / or data such as a background image 52.

[0056] The information processing device 1 inputs the mask image 51 and / or the background image 52, etc., which are teacher data, to the input layer, undergoes arithmetic processing in the intermediate layer, and obtains from the output layer a classification result indicating the presence or absence of an abnormality in the inspection object 4. Note that the classification result output from the output layer is an inspection image 54 including a heat map image 53 when the mask image 51 is input to the input layer, and is an inspection image 54 not including the heat map image 53 when the background image is input to the input layer.

[0057] The information processing device 1 compares the classification result output from the output layer with the information labeled for the mask image 51 in the training data, i.e., the correct value, and optimizes the parameters used in the calculation process in the intermediate layer so that the output value from the output layer approaches the correct value. The parameters include, for example, the weights (coupling coefficients) between neurons and the coefficients of the activation functions used in each neuron. There are no particular limitations on the method for optimizing the parameters, and the information processing device 1, for example, optimizes various parameters using the backpropagation algorithm.

[0058] The information processing device 1 performs the above processing on each inspection image 54 included in the training data to generate an anomaly detection model 152. When the information processing device 1 acquires a processed image 50 of the inspection object 4, it uses the anomaly detection model 152 to detect the presence or absence of a foreign object in the inspection object 4.

[0059] 6 is a flowchart showing an example of the processing procedure for generating the anomaly detection model 152 by the control unit 11. The control unit 11 acquires, as training data, data in which a plurality of mask images 51 of the object to be inspected 4 are associated with information indicating the anomaly of the object to be inspected 4 in each processed image 50, and / or data that is a background image 52 of the object to be inspected 4 without anomaly (S11).

[0060] The control unit 11 uses the training data to generate an anomaly detection model 152 (trained model) that outputs information on the presence or absence of an anomaly in the object 4 when a mask image 51 and / or a background image 52 of the object 4 is input (S12). Specifically, the control unit 11 inputs the training data, that is, the mask image 51 and / or the background image 52, to the input layer of the neural network, and acquires, from the output layer, a classification result that identifies the presence or absence of an anomaly in the shape of the object 4 in the processed image 50 and the presence or absence of a foreign object mixed in the object 4. The control unit 11 compares the acquired classification result with the correct value of the training data (information labeled for the mask image 51 or information indicating no anomaly in the background image 52) and optimizes parameters (weights, etc.) used in the calculation process in the intermediate layer so that the classification result output from the output layer approaches the correct value. Note that the anomaly detection model 152 can be optimized and its accuracy improved by increasing the number and types of mask images 51 and background images 52 and by increasing the number of training runs. The control unit 11 stores the generated anomaly detection model 152 in the auxiliary storage unit 15, and the series of processes ends. Note that, although the learning model generation process in this embodiment can use only the mask image 51 as training data, by having the anomaly detection model 152 learn, as a background image, blobs and the like that exist in the processed image 50 of the object to be inspected 4 and that closely resemble foreign objects but do not actually constitute foreign objects, the learning effect of the anomaly detection model 152 is improved, and the accuracy of the anomaly detection process by the anomaly inspection system 10 is further improved. [Anomaly Detection Process]

[0061] FIG. 7 is a flowchart illustrating an example of the procedure for an anomaly detection process performed by the anomaly inspection system 10. The irradiation unit 3 of the X-ray inspection device 2 irradiates the object 4 with X-rays. Based on the X-rays detected by the detection unit 7, the image generation unit 111 of the control unit 11 in the information processing device 1 generates an X-ray irradiation image (S21). The image processing unit 112 of the control unit 11 performs image processing on the X-ray image using a smoothing filter, a feature extraction filter, etc., to generate a processed image 50 (S22). The object identification unit 113 of the control unit 11 performs blob analysis of the object 4 based on the processed image 50 and identifies blobs (S23). The acquisition unit 114 of the control unit 11 acquires a mask image 51 and anomaly information, etc., based on the processed image 50 generated in S22 and the blobs identified in S23 (S24). The anomaly information acquisition unit 115 of the control unit 11 acquires a heat map image 53 based on the mask image 51, etc., acquired in S24 (S25). The output unit 116 of the control unit 11 combines the heat map image 53 onto the processed image 50, and outputs the inspection image 54 including the heat map image 53 (S26), thereby completing the process.

[0062] As described above, according to this embodiment, even when the difference in pixel values ​​between normal and abnormal portions in the processed image 50 of the object to be inspected 4 is small, normal and abnormal portions can be accurately distinguished. Therefore, information on the presence or absence of abnormalities, such as abnormalities in the shape of the object to be inspected 4 and the presence or absence of foreign matter mixed into the object to be inspected 4, can be accurately detected. Furthermore, by having the anomaly detection model 152 learn, as a background image, blobs and the like present in the processed image 50 of the object to be inspected 4 that closely resemble foreign matter but do not actually constitute foreign matter, the accuracy of the anomaly detection process by the anomaly inspection system 10 can be further improved. Therefore, if the accuracy of the anomaly inspection system 10 is further improved, the need for visual anomaly detection by the user (operator) can be reduced or eliminated, thereby improving the efficiency of the anomaly detection process. [Image]

[0063] Various images used in the anomaly detection process of the present invention will now be described. FIGS. 8A, 8B, and 8C are schematic diagrams showing a processed image 50 of an object under inspection 4. The processed image 50 is generated by applying image processing to an X-ray irradiation image. FIGS. 9A, 9B, and 9C are schematic diagrams showing masked images of the object under inspection obtained by applying masking processing to blobs identified in the processed image 50. In the masking processing, areas corresponding to the blobs are colored white, and areas other than the blobs are colored black. FIG. 10 is a schematic diagram showing a processed image 50 of an object under inspection in which no blobs are identified by the anomaly detection process or visual inspection by a user (inspector) and it is determined that no foreign matter is present. The masked image is subjected to predetermined image processing, and the image is used as a background image 52 for learning by the anomaly detection model 152. Note that even if a blob is identified in the image, if the blob is not determined to be a foreign matter, it is processed and learned as a background image 52 in the same manner as described above.

[0064] Next, FIGS. 11A and 11B are schematic diagrams showing marking images for performing background image processing. A marking log for converting a normal processed image 50 of the inspection object 4 into a background image is displayed in the upper left corner of each of FIGS. 11A and 11B. The marking logo is created in a shape completely different from various types of blobs expected so that it will not be identified as a blob during foreign object detection processing or by a user's visual inspection. In FIG. 11A, the marking logo is displayed in white, and the area other than the marking logo is displayed in black. Note that FIG. 11B is a negative-positive inverted image of FIG. 11A, which is used appropriately depending on the two-tone level of the processed image 50. The anomaly detection model 152 learns all images containing the marking logo as background images. By learning the background image 52, even if a blob is identified in the processed image, it is possible to accurately determine whether the blob is a foreign object. Next, FIG. 12 is a schematic diagram showing a background image 52 obtained by combining a normal image, which is the processed image 50 without any foreign matter as shown in FIG. 10, with the marking image as shown in FIG. 11.

[0065] Next, FIG. 13 is a schematic diagram showing a processed image 50 of an object 4 containing a foreign substance. Foreign substances are shown in the circular and elliptical dotted lines. Next, FIGS. 14(a), (b), and (c) are schematic diagrams showing image processing performed in advance when foreign substance detection processing is performed on the processed image 50 of an object 4 containing multiple foreign substances. The processed image 50 of the object 4 containing multiple foreign substances is duplicated in a number equal to the number of foreign substances, with each image showing a different foreign substance. FIG. 14(a) shows a processed image containing two foreign substances. To improve the accuracy of the foreign substance inspection processing, the processed image 50 shown in FIG. 14(a) is duplicated into processed images 50 shown in FIG. 14(b) and FIG. 14(c), which each contain a single foreign substance. Then, a mask image 51 is generated for the processed images 50 shown in FIG. 14(b) and FIG. 14(c).

[0066] Next, (a) and (b) of FIG. 15 are schematic diagrams showing inspection images of an inspection object after foreign object detection processing has been performed. FIG. 15(a) is a processed image 50 before the foreign object detection processing, and FIG. 15(b) is an inspection image 54, which is the processed image 50 before the foreign object detection processing. After the foreign object detection processing has been performed, FIG. 15(b) displays a heat map in the foreign object portion of the processed image 50, clearly indicating the presence, approximate shape, and location of the foreign object. In detail, the inspection image 54 of FIG. 15(b) is obtained by combining the processed image 50 of FIG. 15(a) with a heat map image generated based on anomaly information obtained by the foreign object detection processing. Next, (a) and (b) of FIG. 16 are schematic diagrams showing an example of an inspection object to be inspected. As can be seen from FIG. 16(a), foreign objects in the shape of an iron ball can be detected in various sizes, such as 0.8 mm, 1.0 mm, 1.2 mm, 1.5 mm, 2.0 mm, and 2.5 mm. FIG. 16(b) shows an example of a foreign object other than an iron ball. As can be seen from FIG. 16(b), foreign object detection targets include safety pins, staples, broken needles, hair ties, and lighters. Note that foreign objects that can be photographed can be detected by registering them as foreign objects in the anomaly detection model 152 and having it learn. (Embodiment 2) [Relearning]

[0067] 17 is a block diagram showing the configuration of an abnormality inspection system 20 according to embodiment 2. The abnormality inspection system 20 according to embodiment 2 has the same configuration and performs the same processing as the abnormality inspection system 10 according to embodiment 1, except that the control unit 11 has a relearning unit 119 as a functional unit. The same parts as those in FIG. 2 are designated by the same reference numerals, and detailed description thereof will be omitted.

[0068] In the foreign matter detection process of the present invention, the first display unit 41 displays the processed image 50, and a user (operator) visually checks for the presence or absence of foreign matter in the processed image 50. The second display unit 42 displays the inspection image 54 after the foreign matter detection process by the anomaly inspection system 20, and the user visually checks for the presence or absence of a heat map on the inspection image 54.

[0069] In this case, a problem occurs when the user's visual judgment of foreign matter contamination differs from the judgment made by the anomaly inspection system 20 through foreign matter detection processing. Here, we assume that the user's visual judgment is correct. FIG. 18 is a schematic diagram summarizing cases in which the user's judgment differs from the detection result of the anomaly inspection system 20. As can be seen from FIG. 18, problems arise when the anomaly inspection system 20 detects that no foreign matter is present when the user visually judges that a foreign matter is present, and when the anomaly inspection system 20 detects that a foreign matter is present when the user visually judges that no foreign matter is present. That is, the processed image 50 is re-learned in the following two cases (Pattern 1) and (Pattern 2). (Pattern 1) The anomaly inspection system 20 according to the present invention does not display a heat map on the inspection image 54 of the inspection object 4, but the user visually judges that a foreign matter is present. (Pattern 2) The anomaly inspection system 20 according to the present invention displays a heat map on the inspection image 54 of the inspection object 4, but the user visually judges that no foreign matter is present.

[0070] When a discrepancy occurs between the output result of the learning model according to the present invention and the visual judgment result of the user, the re-learning unit 119 visually checks the processed image 50 again to determine whether the inspection object 4 is in a normal or abnormal state. The re-learning unit 119 re-learns the anomaly detection model 152 using training data that associates the inspection image 54 with a judgment of the pass / fail of the anomaly detection model 152 input by the user. Here, the second display unit 42 includes an OK button 70 (not shown), which is a first operation button, and an NG button 71 (not shown), which is a second operation button. When the above-described (Pattern 1) and (Pattern 2) states occur in the images displayed on the first display unit 41 and the second display unit 42, respectively, the control unit 11 detects whether the OK button 70 or the NG button 71 is pressed by the user, associates the inspection image 54 with the presence or absence of an abnormal state, for example, the presence or absence of foreign matter contamination, and identifies the inspection image 54 as data to be re-learned. The second display unit 42 further includes a selection button 72 for inputting the type of abnormality in the inspection object 4. In the above-described (Pattern 1), the user can select and input the type of abnormality, such as the type of foreign substance, contained in the image displayed on the second display unit 42 by pressing the selection button in addition to pressing the NG button 71. The selected and input type of abnormality is further associated with the inspection image 54 in addition to the presence or absence of an abnormal state. In the above-described (Pattern 1) and (Pattern 2), accurate information is added to the image that was misjudged, and after re-learning using the anomaly detection model 152, the second display unit displays a heat map image based on the type of abnormality on the abnormal image when a similar abnormality occurs. That is, the re-learning unit 119 re-learns the anomaly detection model 152 for the inspection image 54 that has been re-classified by the discrimination unit 118. The re-learning of the anomaly detection model 152 by the re-learning unit 119 is performed, for example, overnight by automatic learning or the like. The correct determination result of the corresponding processed image 50 is saved and accumulated in the inspection image DB 151 and is used in the inspection of a new object 4 to be inspected.If the user's judgment of foreign matter contamination differs from the judgment of foreign matter contamination made by the anomaly inspection system, the anomaly detection model 152 must be retrained on the target processed image so that the anomaly inspection system does not make similar mistakes, false detections, or false judgments in the future.

[0071] FIG. 19 is a flowchart showing an example of a processing procedure of the re-learning process by the control unit 11 of the information processing device 1.

[0072] The control unit 11 accepts a pass / fail judgment of the foreign object detection judgment by the anomaly inspection system 20 for the selected inspection image 54 based on the user's pressing of the OK button 70 or NG button 71 provided on the second display unit 42 (S31). If the foreign object detection result by the anomaly inspection system 20 differs from the user's visual judgment, the user presses the NG button 71. In the case of pattern 1, the user presses the NG button 71 to acquire a mask image or the like from the inspection image 54 (S31), which is used as training data for training the anomaly detection model 152, generating a new anomaly detection model 152 (S32), which is stored in the inspection image DB 151. In the case of pattern 2, the user presses the NG button 71 to combine the inspection image 54 with a masking image to acquire a background image 52 (S31), which is used as training data for training the anomaly detection model 152, which is generated a new anomaly detection model 152 (S32), which is stored in the inspection image DB 151. If the result of the foreign object determination by the user's visual inspection is the same as the result of the foreign object detection process by the anomaly inspection system 20, the user can press the OK button 70 to end the foreign object detection process. Through these re-learning processes, the anomaly detection model 152 re-learns as the anomaly detection process continues, enabling more accurate and precise anomaly detection.

[0073] The embodiments disclosed herein are illustrative in all respects and should not be considered limiting. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the meaning and scope of the claims. For example, the object to be inspected is not limited to clothing or food, but may also be packaging, industrial products, etc. The electromagnetic waves irradiated onto the object to be inspected 4 are not limited to X-rays, but may also be terahertz waves, infrared rays, visible light, etc. Furthermore, in the processes or operations described above, the processes or operations can be freely modified as long as no inconsistencies in the processes or operations occur, such as the use of data that should not yet be available in a certain step. Furthermore, the embodiments described above are merely examples for explaining the present invention, and the present invention is not limited to these examples. The present invention can be embodied in various forms without departing from the spirit and scope of the invention.

[0074] REFERENCE SIGNS LIST 1 Information processing device 2 X-ray inspection device 3 Irradiation unit 4 Inspection object 6 Transport unit (conveyor) 7 Detection unit 10, 20 Anomaly inspection system 11 Control unit 12 Memory unit 13 Communication unit 14 Input unit 15 Auxiliary memory unit 21 Small upper housing 21 Upper housing 22 Display unit 23 Lower housing 24 Light emitting unit 25 Light receiving unit 27 Controller 31 Irradiation body 32 Irradiation control unit 41 First display unit 42 Second display unit 50 Processed image 51 Mask image 52 Background image 53 Heat map image 54 Inspection image 60 Conveyor belt 61 Conveyor belt drive unit 62 Conveyor belt control unit 70 OK button (first operation button) 71 NG button (second operation button) 72 Selection button 111 Image generation unit 112 Image processing unit 113 Object identification unit 114 Acquisition unit 115 Abnormality information acquisition unit 116 Output unit 117 Control display unit 118 Discrimination unit 119 Learning unit 124 Specific image acquisition unit 134 Specific information acquisition unit 152 Abnormality detection model 153 Program 220 Data file generation unit 221 Data reading unit DB 151 Inspection image N Network

Claims

1. The method includes the steps of: irradiating an object under inspection with electromagnetic waves; acquiring an image in accordance with the electromagnetic waves transmitted through the object under inspection; identifying blobs contained in the object under inspection based on pixel values ​​of pixels in a processed image obtained by image processing the image; acquiring a mask image masked to a position and shape corresponding to the identified blob, a background image which is the processed image not including the blob, and anomaly information including offset information of the masked portion in the processed image and information regarding an anomaly in the object under inspection having a blob label which is a type of object corresponding to the blob; acquiring training data including the mask image, the background image, and the anomaly information; irradiating the object under inspection with electromagnetic waves, and when an image acquired in accordance with the electromagnetic waves transmitted through the object under inspection is input, acquiring a heat map image in which the positions and shapes of the blobs are displayed as a heat map on the processed image based on the training data; and generating a learning model which outputs an inspection image including the heat map image by combining the heat map image with the processed image. A method for generating a learning model, characterized in that the background image is an image with a shape different from that of the blob, which is synthesized at any position of the processed image.

2. The method for generating a learning model described in claim 1, characterized in that the electromagnetic waves are X-rays, and the method includes the steps of identifying the blob from an image acquired in response to the X-rays and generating a mask image masked to a position and shape corresponding to the blob, displaying the generated mask image, and identifying the abnormality information of the object to be inspected contained in the mask image.

3. A method for generating a learning model according to claim 1 or 2, characterized in that it comprises the steps of: acquiring the mask image of the object to be inspected and the anomaly information of the object to be inspected; and re-learning the learning model based on the mask image and the anomaly information.

4. An input layer receives as input a mask image masked to a position and shape corresponding to the identified blob, a background image that is the processed image and does not include the blob, and anomaly information including offset information of the masked portion in the processed image and information regarding an anomaly in the object having a blob label that is a type of object corresponding to the blob; an output layer receives a heat map image that displays the positions and shapes of the blobs as a heat map on the processed image, and combines the heat map image with the processed image to output an inspection image including the heat map image; and an intermediate layer in which parameters are trained based on the mask image and the anomaly information, and causes the computer to function such that when the mask image and the anomaly information are input to the input layer, the inspection image including the heat map image is output from the output layer after calculation by the intermediate layer; A learning model that causes a computer to function such that when the background image is input to the input layer, the inspection image that does not include the heat map image is output from the output layer after calculation by the intermediate layer. an image generation unit that generates an image in accordance with the electromagnetic waves that have passed through the object to be inspected; an image processing unit that performs image processing on the image and acquires a processed image; an object identification unit that identifies blobs contained in the object to be inspected based on pixel values ​​of pixels in the processed image; an acquisition unit that acquires, as training data, a mask image masked to a position and shape corresponding to the identified blob, a background image that is the processed image that does not include the blob, and abnormality information that includes offset information of the masked portion in the processed image and information about an abnormality in the object to be inspected that has a blob label that is a type of object corresponding to the blob; an abnormality information acquisition unit that inputs the mask image and abnormality information acquired by the acquisition unit into a learning model that outputs information about an abnormality in the object to be inspected when an image acquired in accordance with the electromagnetic waves that has passed through the object to be inspected is input, and acquires a heat map image that displays the positions and shapes of the blobs as a heat map on the processed image; and an output unit that outputs an inspection image including the heat map image by combining the heat map image with the processed image, An inspection device in which the background image is an image having a shape different from that of the blob superimposed at an arbitrary position on the processed image.

6. The inspection device according to claim 5, wherein the electromagnetic waves are X-rays, and the image generating unit generates an image based on the X-rays.

7. An inspection device as described in claim 6, comprising a display, wherein the display unit displays a first display unit that outputs an image based on an image acquired after the X-ray irradiation and a second display unit that outputs a normal image that does not include the heat map image and is an image showing a normal state of the object to be inspected, or an abnormal image that includes the heat map image and is an image showing an abnormal state of the object to be inspected, side by side on the display, and the display unit has a first operation button and a second operation button that accepts input of normal information or abnormal information for the object to be inspected by a user.

8. The inspection device described in claim 7, characterized in that it is provided with a re-learning unit that re-learns the learning model based on the image displayed on the second display unit and the normal information or the abnormal information received by the first operation button and the second operation button.

9. The inspection device described in claim 7 or 8, characterized in that the second display unit has a selection button that accepts input of the type of abnormality in the object to be inspected, and when the selection button accepts the input, the second display unit displays the heat map image based on the type of abnormality on the abnormality image.

10. An inspection device as described in any one of claims 5 to 9, characterized in that it comprises a second acquisition unit that irradiates the object to be inspected with electromagnetic waves, acquires an image in accordance with the electromagnetic waves that have passed through the object to be inspected, identifies the blob by applying an image processing algorithm to the image to perform image processing, and acquires information regarding an abnormality in the object to be inspected based on a masked image that is masked to a position and shape corresponding to the identified blob, and the abnormality information acquisition unit inputs the heat map image in accordance with the information acquired by the second acquisition unit.

11. Irradiating an object under inspection with electromagnetic waves; acquiring an image in accordance with the electromagnetic waves transmitted through the object under inspection; identifying blobs contained in the object under inspection based on pixel values ​​of pixels in a processed image obtained by image processing the image; acquiring a mask image masked to a position and shape corresponding to the identified blob, a background image which is the processed image not including the blob, and anomaly information including offset information of the masked portion in the processed image and information regarding an anomaly in the object under inspection having a blob label which is a type of object corresponding to the blob; acquiring training data including the mask image, the background image, and the anomaly information; irradiating the object under inspection with electromagnetic waves, and when an image acquired in accordance with the electromagnetic waves transmitted through the object under inspection is input, acquiring a heat map image which displays the positions and shapes of the blobs as a heat map on the processed image based on the training data; and generating a learning model which outputs an inspection image including the heat map image by combining the heat map image with the processed image, An inspection method characterized in that the background image is an image having a shape different from that of the blob, which is synthesized at an arbitrary position on the processed image.

12. Irradiating an object under inspection with electromagnetic waves; acquiring an image in accordance with the electromagnetic waves transmitted through the object under inspection; identifying blobs contained in the object under inspection based on pixel values ​​of pixels in a processed image obtained by image processing the image; acquiring a mask image masked to a position and shape corresponding to the identified blob, a background image which is the processed image not including the blob, and anomaly information including offset information of the masked portion in the processed image and information regarding an anomaly in the object under inspection having a blob label which is a type of object corresponding to the blob; acquiring training data including the mask image, the background image, and the anomaly information; Irradiating the object under inspection with electromagnetic waves, and when an image acquired in accordance with the electromagnetic waves transmitted through the object under inspection is input, acquiring a heat map image which displays the positions and shapes of the blobs as a heat map on the processed image based on the training data; and generating a learning model which outputs an inspection image including the heat map image by combining the heat map image with the processed image, A computer program characterized in that the background image is an image having a shape different from that of the blob, which is synthesized at an arbitrary position of the processed image.