Alarm output device, alarm output method and non-transient recording medium

Abstract

The alarm output device maintains: image information, each representing one or more inspection objects included in the subject of the determination target image; alarm object information, representing the alarm objects; and co-occurrence information, representing alarm co-occurrence objects that are pre-set as objects included in the image at the same time as the alarm objects; an alarm is output when it is determined that at least one of the one or more inspection objects meets the alarm object and satisfies the specified conditions, and when it is determined that although none of the one or more inspection objects meets the alarm object, at least one of the one or more inspection objects meets at least one alarm co-occurrence object.

Description

[0001] This application claims priority based on Japanese Patent Application No. 2020-028448 filed on February 21, 2020, the entire contents of which are incorporated herein by reference. Technical Field

[0002] The invention relates to an alarm output device, an alarm output method and a non-transient recording medium. Background Art

[0003] As the risk of terrorism increases, the demand for using X-ray inspection equipment in cargo inspection is increasing. In cargo inspection using X-rays, inspectors usually view X-ray images to confirm whether specific items are mixed in. In order to accurately determine whether specific items are mixed in X-ray images, inspectors need to receive at least a high level of training or gain long-term practical experience. As a result, inspectors with immature inspection skills may miss the mixing of specific items during inspection. Therefore, in order to reduce the risk of inspectors missing, there is a mechanism in the X-ray inspection device that displays an alarm for areas with a high density of specific materials.

[0004] As background technology in this technical field, there is Japanese Patent Publication No. 2009-042134 (Patent Document 1). In the publication, it is recorded that: "It is equipped with: an X-ray irradiation device 14, which irradiates the inspection object 5 with multiple intensities of incident X-rays 7 from two or more different directions; an X-ray detection device 16, which detects the intensity of the transmission X-rays 8 that pass through the inspection object, and detects the thickness x of the inspection object based on multiple X-ray transmission images; and a computing device 18, which calculates the atomic number Z and electron density ρ of the inspection object based on the multiple intensities, the intensity of the transmission X-rays and the thickness, and identifies the material of the inspection object based on this. For at least one of the proportional constants A and B, the linear approximation of the attenuation coefficient of the X-ray is used in the material with atomic numbers of 4 to 30." (Refer to the abstract of the specification).

[0005] Prior art literature

[0006] Patent Literature

[0007] Patent Document 1: Japanese Patent Application Publication No. 2009-042134 Summary of the invention

[0008] Problems to be solved by the invention

[0009] In the technology described in Patent Document 1, whether the inspection object is an alarm object is determined based on the material, density, and shape of the inspection object in the image. However, in the case where multiple inspection objects overlap, the technology described in Patent Document 1 determines that the material density is higher than the original material density of each of the multiple inspection objects, and there is a possibility that the alarm object cannot be found. In addition, in the case where the characteristics of the alarm object itself are difficult to present in the image information, the technology described in Patent Document 1 may miss the alarm object.

[0010] Therefore, an object of the present invention is to provide a method for determining that an alarm object may exist even if the alarm object itself cannot be directly detected, thereby reducing the risk of an inspector's omission.

[0011] Means used to solve problems

[0012] In order to solve the above-mentioned problems, a technical solution of the present invention adopts the following structure. An alarm output device includes a processor and a memory connected to the output device; the memory stores: image information, each indicating one or more inspection objects included in the subject of the determination target image; alarm object information, indicating the alarm objects; and co-occurrence information, indicating alarm co-occurrence objects that are preset as objects included in the image at the same time as the alarm objects; the processor refers to the alarm object information to determine whether at least one of the one or more inspection objects meets the alarm object; when it is determined that none of the one or more inspection objects meets the alarm object, the processor refers to the co-occurrence information to determine whether at least one of the one or more inspection objects meets at least one of the alarm co-occurrence objects; when it is determined that at least one of the one or more inspection objects meets the alarm object and satisfies a specified condition, and when it is determined that although none of the one or more inspection objects meets the alarm object, at least one of the one or more inspection objects meets at least one of the alarm co-occurrence objects, the processor outputs an alarm to the output device.

[0013] Effects of the Invention

[0014] According to one aspect of the present invention, even if the alarm target object itself cannot be directly detected, it is determined that the alarm target object may exist, thereby reducing the risk of the inspector's omission.

[0015] Objects, structures, and effects other than those described above will become apparent from the following description of the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a block diagram showing a configuration example of the co-occurrence information determination system of the first embodiment.

[0017] Figure 2 This is a block diagram showing a configuration example of an image transmission terminal according to the first embodiment.

[0018] Figure 3 This is a block diagram showing a configuration example of the image determination server according to the first embodiment.

[0019] Figure 4 This is an example of the prohibited object management DB of the first embodiment.

[0020] Figure 5 This is an example of the co-occurrence object management DB of the first embodiment.

[0021] Figure 6 This is an example of the relevance DB of the first embodiment.

[0022] Figure 7 This is a block diagram showing a configuration example of the inspector terminal according to the first embodiment.

[0023] Figure 8 This is a flowchart showing an example of the alarm output determination process of the first embodiment.

[0024] Fig. 9 This is an example of a display screen of a warning that is displayed when it is determined that a prohibited object (with a high probability) and a co-occurring object of the prohibited object exist in an X-ray image in the first embodiment.

[0025] Fig.10 This is an example of a display screen for calling attention that is displayed when it is determined that a prohibited object (with a low probability) and a co-occurring object of the prohibited object exist in an X-ray image in the first embodiment.

[0026] Fig.11 This is an example of a display screen for calling attention that is displayed when it is determined that a prohibited object (of which the probability is low) exists in an X-ray image (but there is no co-occurring object) in the first embodiment.

[0027] Fig.12 This is an example of a display screen that is displayed when it is determined that there is no prohibited object in the X-ray image, but it is determined that there is a co-occurring object of the prohibited object in the X-ray image according to the first embodiment. DETAILED DESCRIPTION

[0028] Hereinafter, an embodiment of the present invention will be described in detail based on the accompanying drawings. In this embodiment, the same reference numerals are assigned to the same configuration in principle, and repeated descriptions are omitted. In addition, this embodiment is only an example for implementing the present invention, and it should be noted that it does not limit the technical scope of the present invention.

[0029] Example 1

[0030] Figure 1The co-occurrence information determination system includes, for example, one or more X-ray inspection apparatuses 101 , one or more image transmission terminals 102 , an image determination server 103 (co-occurrence information determination apparatus), and one or more inspector terminals 104 .

[0031] Each device included in the co-occurrence information determination system is connected via a network such as the Internet or an intranet or a display cable and can communicate with each other. In particular, the X-ray inspection device 101 is connected to the image transmission terminal 102, the image transmission terminal 102 is connected to the image determination server 103, and the image determination server 103 is connected to the inspector terminal 104. In addition, the communication method used by the co-occurrence information determination system may be any communication method as long as data can be transferred.

[0032] The X-ray inspection device 101 is, for example, a device that irradiates an object with X-rays to measure the amount of X-ray transmission, and can be used as a cargo inspection device in an airport, etc. The image transmission terminal 102 divides a series of X-ray images photographed by the X-ray inspection device 101, for example, into units of articles, and transmits them to the image determination server 103.

[0033] The image determination server 103 determines whether the object (cargo) of the X-ray image received from the image transmission terminal 102 is safe. The inspector terminal 104 is a terminal used by inspectors who check X-ray images, and displays output results of the image determination server 103 or receives input to the image determination server 103, for example.

[0034] The X-ray inspection apparatus 101 and the image transmission terminal 102 are installed in the required number at the site to be inspected for goods (for example, at a position where a conveyor for inspection of goods in a security inspection area can be photographed). Similarly, the image transmission terminal 102 is installed in the required number at the site to be inspected for goods.

[0035] exist Figure 1 In the example of , station A and station B are stations for cargo inspection. The inspector uses the inspector terminal 104 to visually check the X-ray image of the cargo photographed by the X-ray inspection device 101 and finally determines whether the cargo is safe.

[0036] When the image determination server 103 determines that a prohibited object or a co-occurring object exists in the subject of the X-ray image, the inspector terminal 104 feeds back the message to the inspector using the display device 704. In addition, the feedback is not limited to the display on the display device 704, and any other method may be used, such as a feedback mechanism (such as a light) provided by the X-ray inspection device 101 or a mechanism for stopping a belt conveyor that transports goods. The inspector terminal 104 is, for example, a computer such as a PC (Personal Computer) or a tablet terminal.

[0037] In addition, any two or more of the X-ray inspection apparatus 101, the image transmission terminal 102, the image determination server 103, and the inspector terminal 104 may be integrated. In particular, for example, a computer (e.g., a PC) attached to the X-ray inspection apparatus 101 may be provided, and the computer may have the functions of the image determination server 103 and the inspector terminal 104.

[0038] Figure 2 1 is a block diagram showing an example of the configuration of the image transmission terminal 102. The image transmission terminal 102 is configured by, for example, a computer including a CPU (Central Processing Unit) 210, a memory 202, an input device 203, a display device 204, a communication device 205, and an auxiliary storage device 210, which are mutually connected by an internal communication line such as a bus.

[0039] The CPU 201 includes a processor and executes a program stored in the memory 202. The memory 202 includes a ROM as a nonvolatile storage element and a RAM as a volatile storage element. The ROM stores a fixed program (such as BIOS) and the like. The RAM is a high-speed and volatile storage element such as a DRAM (Dynamic Random Access Memory) and temporarily stores the program executed by the processor and the data used when the program is executed.

[0040] The auxiliary storage device 210 is a large-capacity non-volatile storage device such as a magnetic storage device (HDD) or a flash memory (SSD), and stores an OS (Operating System) 211, a program executed by the CPU 201, and data used when the program is executed. That is, the program is read from the auxiliary storage device 210, loaded into the memory 202, and executed by the CPU 201.

[0041] The image transmission terminal 102 may also have an input interface and an output interface. The input interface is an interface connected to an input device 203 such as a keyboard or a mouse, and receives input from an operator. The output interface is an interface connected to a display device 204 such as a display device or a printer, and outputs the execution result of the program in a form recognizable to the operator. The communication device 205 is a network interface device that controls communication with other devices according to a prescribed protocol.

[0042] The program executed by the CPU 201 is provided to the computer via a removable medium (CD-ROM, flash memory, etc.) or a network, and is stored in a non-volatile auxiliary storage device 210 as a non-transitory storage medium. Therefore, the image transmission terminal 102 may have an interface for reading data from a removable medium.

[0043] The CPU 201 includes, for example, an X-ray image cutout unit 212 . The X-ray image cutout unit 212 cuts a series of X-ray images received from the X-ray inspection apparatus 101 into units of articles according to, for example, a predetermined algorithm, and transmits the images to the image determination server 103 .

[0044] For example, the CPU 201 functions as the X-ray image extraction unit 212 by operating according to the X-ray image extraction program loaded into the memory 202. The relationship between the functional units included in the CPU of other devices included in the co-occurrence information determination system and the program is similar.

[0045] In this embodiment, the information used by each device included in the X-ray automatic determination system does not depend on the data structure, and can be expressed in any data structure. For example, a data structure appropriately selected from a table, a list, a database, or a queue can store the information.

[0046] In addition, each device included in the co-occurrence information determination system of the present embodiment is a computer system that is physically constructed on one computer or on multiple computers that are logically or physically constructed. It can operate in a separate thread on the same computer or on a virtual computer built on multiple physical computer resources.

[0047] Figure 31 is a block diagram showing a configuration example of the image determination server 103. The image determination server 103 is configured by, for example, a computer having a CPU 301, a memory 302, an input device 303, a display device 304, a communication device 305, and an auxiliary storage device 310. The description of the hardware of the CPU 301, the memory 302, the input device 303, the display device 304, the communication device 305, and the auxiliary storage device 310 is the same as that of the CPU 201, the memory 202, the input device 203, the display device 204, the communication device 205, and the auxiliary storage device 210, and thus is omitted.

[0048] The auxiliary storage device 210 stores an OS 311. In addition, the auxiliary storage device stores a prohibited object management DB (database) 313, a co-occurring object management DB 314, a correlation DB 315, and an X-ray image DB 316.

[0049] The prohibited object management DB 313 defines prohibited objects and easily confused items that are general objects similar to prohibited objects. Prohibited objects are objects that are objects for which alarms are output in principle, and general objects are objects that are not objects for which alarms are output in principle. In the prohibited object management DB 313, not only prohibited objects but also easily confused items are defined, and by outputting the existence of easily confused items, the inspection accuracy of the inspector can be improved.

[0050] The co-occurrence management DB 314 defines co-occurrences and whether the co-occurrences are co-occurrences of prohibited objects or co-occurrences of general objects. The co-occurrences are objects that are likely to be included in the subject of the X-ray image when the specified object is included in the X-ray image. Co-occurrences include objects that are general objects when viewed alone but are likely to exist in the X-ray image together with other prohibited objects or easily confused objects.

[0051] For example, although the push nozzle itself is not a prohibited object, the perfume bottle, which is a prohibited object, is likely to include the push nozzle. Therefore, for example, although the image determination server 103 determines that the push nozzle is included in the X-ray image, even if it is determined that the perfume bottle is not included in the X-ray image, by defining the push nozzle as a co-occurring object in the co-occurring object management DB 314, the image determination server 103 can output an alarm considering that the perfume bottle, which is a prohibited object, is likely to be included.

[0052] The X-ray image DB 316 stores, for example, X-ray images received from the image transmission terminal 102 and image determination results for the X-ray images. The image determination results for the X-ray images include, for example, the names of objects included in the X-ray images, whether the objects included in the X-ray images are prohibited objects, easily confused objects, or co-occurring objects, or the determination basis for the X-ray image determination.

[0053] The CPU 301 includes an X-ray image recognition unit 312, a prohibited object determination unit 317, and an alarm output unit 318. The X-ray image recognition unit 312 identifies the type of the object based on the material and shape of the inspection object based on the image data received from the image transmission terminal 102.

[0054] The prohibited object determination unit 317 refers to the prohibited object management DB 313 and the co-occurring object management DB 314, determines whether the determined article is a prohibited object or a co-occurring object (if it is a co-occurring object, whether it is a co-occurring object that is a prohibited object or a co-occurring object that is a general object), and sends the determination result to the alarm output unit 318. The alarm output unit 318 outputs an alarm based on the determination result of the prohibited object determination unit 317 to the inspector terminal 104.

[0055] Figure 4 This is an example of the prohibited substance management DB 313. The prohibited substance management DB 313 includes, for example, a prohibited substance ID column 3131, a prohibited article category column 3132, a classification column 3133, and an easily confused article category column 3134.

[0056] The prohibited item ID column 3131 holds an ID for identifying the prohibited item. The prohibited item category column 3132 holds the category of the prohibited item. The category column 3133 holds the category of the prohibited item. Figure 4 In the example, since "Easily confused with prohibited items" is maintained in the category column 3133 when there are easily confused items with prohibited items, and "Obviously prohibited items" is maintained in the category column 3133 when there are easily confused items with prohibited items, the category column 3133 indicates whether there are easily confused items with prohibited items.

[0057] The easily confused item category column 3134 stores the category of easily confused items when there are easily confused items that are prohibited items. In the case of easily confused items that are not prohibited items, for example, "N / A" (not applicable) is stored in the easily confused item category column 3134.

[0058] Figure 5 This is an example of the co-occurring object management DB 314. The co-occurring object management DB 314 includes, for example, a co-occurring object ID column 3141, a co-occurring object category column 3142, and a determination column 3143. The co-occurring object ID column 3141 stores an ID for identifying a co-occurring object.

[0059] The co-occurring object category column 3142 holds the category of the co-occurring object. The determination column 3143 holds a value indicating whether the co-occurring object is included in the X-ray image together with a prohibited object or with a general object (easily confused object).

[0060] Figure 6This is an example of the association DB 315. The association DB 315, for example, holds a prohibited object ID column 3151 and a co-occurring object ID column 3152. In the association DB 315, a co-occurring object corresponding to a prohibited object is defined as being included in the X-ray image at the same time as the prohibited object or an object easily confused with the prohibited object. In the case where there is no co-occurring object corresponding to the prohibited object, for example, "N / A" (not in compliance) is stored in the co-occurring object ID column 3152.

[0061] Figure 7 1 is a block diagram showing an example of the configuration of the inspector terminal 104. The inspector terminal 104 is configured by, for example, a computer including a CPU 701, a memory 702, an input device 703, a display device 704, a communication device 705, and an auxiliary storage device 710. The description of the hardware of the CPU 701, the memory 702, the input device 703, the display device 704, the communication device 705, and the auxiliary storage device 710 is the same as that of the CPU 201, the memory 202, the input device 203, the display device 204, the communication device 205, and the auxiliary storage device 210, and thus is omitted.

[0062] The auxiliary storage device 710 holds an OS 711. The CPU 701 includes an inspection screen output unit 712. The inspection screen output unit 712 outputs an X-ray image or an alarm to the display device 704 based on the output from the alarm output unit 318 of the image determination server 103, for example.

[0063] Figure 8 is a flowchart showing an example of alarm output determination processing. Figure 8 Prior to the alarm output determination process, values ​​have been registered in the prohibited object management DB 313 , the co-occurring object management DB 314 , and the relevance DB 315 .

[0064] The X-ray inspection device 101 takes an X-ray image of the cargo and transmits the X-ray image to the image transmission terminal 102. The X-ray image cutting unit 212 of the image transmission terminal 102 divides the received X-ray image into units of articles and transmits the divided X-ray images to the image determination server 103 (S801). In addition, the X-ray image recognition unit 312 registers the X-ray image obtained in step S801 in the X-ray image DB 316.

[0065] The X-ray image recognition unit 312 determines the category of the object contained in the received X-ray image, and calculates the probability of matching the determined object (S802). Specifically, for example, the X-ray image recognition unit 312 determines the category of the object based on at least one of the material (organic matter, light metal, relatively light inorganic matter or metal), material density, and shape of the object contained in the X-ray image, and calculates the probability of matching, but the determination method is not limited to this.

[0066] The prohibited object determination unit 317 searches the prohibited object management DB 313 (S803). The prohibited object determination unit 317 determines whether or not at least one of the articles specified in step S802 is included in the prohibited object class of the prohibited object management DB 313 as a result of searching the prohibited object management DB 313 (S804).

[0067] When the prohibited object determination unit 317 determines that at least one of the objects specified in step S802 is included in the prohibited object class of the prohibited object management DB 313 (S804: Yes), it determines whether the probability of matching the prohibited object calculated in step S802 is high (for example, whether it is greater than a specified value) (S805).

[0068] When the prohibited object determination unit 317 determines that at least one of the objects specified in step S802 is included in the prohibited object class of the prohibited object management DB 313 (S804: Yes), the process may not be performed in step S805 but may be transferred to step S806 described later.

[0069] In addition, when the prohibited object determination unit 317 determines in step S804 that at least one of the items determined in step S802 is included in the easily confused object category of the prohibited object management DB 313, it can also transfer to step S805. In this case, in step S805, it is determined whether the probability of matching the easily confused object calculated in step S802 is high (for example, whether it is above a specified value).

[0070] When the prohibited article determination unit 317 determines that the probability of the prohibited article calculated in step S802 is high (S805: Yes), the alarm output unit 318 outputs information indicating a warning to the inspector terminal 104 (S811), and ends the alarm output determination process.

[0071] When the prohibited object determination unit 317 determines that the probability of matching the prohibited object calculated in step S802 is not high (S805: No), it searches the co-occurring object management DB 314 (S806). The prohibited object determination unit 317 determines whether the result of searching the co-occurring object management DB 314 is that at least one of the objects determined in step S802 matches at least one co-occurring object class in the co-occurring object management DB 314 (S807).

[0072] When it is determined that at least one of the articles identified in step S802 is included in the co-occurring article class of the co-occurring article management DB 314 (S807: Yes), the prohibited article determination unit 317 refers to the determination column 3143 of the co-occurring article management DB 314 to determine whether the co-occurring article is a prohibited article co-occurring article or a general article co-occurring article (S808). Alternatively, the process of step S807 may be omitted and the process may be transferred to step S808 following the process of step S806.

[0073] In addition, in step S808, when the prohibited object determination unit 317 determines that the co-occurring object is a co-occurring object of a prohibited object, it can also refer to the association DB 315 to determine the prohibited object ID corresponding to the co-occurring object, and refer to the prohibited object management DB 313 to determine the prohibited article category and easily confused article category corresponding to the prohibited article ID.

[0074] When the prohibited object determination unit 317 determines that the co-occurring object is a co-occurring object of a prohibited object (S808: Yes), the alarm output unit 318 outputs a message indicating a call for attention with an alarm level lower than a warning to the inspector terminal 104 (S812), and the alarm output determination process ends. When the prohibited object determination unit 317 determines that the co-occurring object is a co-occurring object of a general object (S808: No), the alarm output unit 318 does not output (S813), and the alarm output determination process ends.

[0075] In addition, when the prohibited object determination unit 317 determines that the co-occurring object is a co-occurring object of a general object (S808: No), in step S812, the alarm output unit 318 may also send information indicating that the co-occurring object of the general object is included in the X-ray image (i.e., information indicating that the possibility of containing prohibited objects is very low and safe) to the inspector terminal 104.

[0076] In addition, when the prohibited object determination unit 317 determines that the co-occurring object is a co-occurring object of a general object (S808: No), it can also transfer to step S812. In this case, the alarm output unit 318 can also send attention-awaiting information to the inspector terminal 104 indicating that although the probability is low, the X-ray image may contain a prohibited object.

[0077] When the prohibited object determination unit 317 determines that the article identified in step S802 is not included in the prohibited object class of the prohibited object management DB 313 (S804: No), it searches the co-occurring object management DB 314 (S809). The prohibited object determination unit 317 determines whether the result of searching the co-occurring object management DB 314 is that at least one of the articles identified in step S802 is included in the co-occurring object class of the co-occurring object management DB 314 (S810).

[0078] When the prohibited object determination unit 317 determines that the co-occurring object is a co-occurring object of a prohibited object (S810: Yes), the alarm output unit 318 outputs information indicating a call for attention to the inspector terminal 104 (S812), and the alarm output determination process ends. When the prohibited object determination unit 317 determines that the co-occurring object is a co-occurring object of a general object (S810: No), the alarm output unit 318 does not output (S813), and the alarm output determination process ends.

[0079] In addition, when the prohibited object determination unit 317 determines that the co-occurring object is a co-occurring object of a general object (S810: No), in step S813, the alarm output unit 318 may also send information indicating that the co-occurring object of the general object is included in the X-ray image (i.e., information indicating that the possibility of containing prohibited objects is very low and it is safe) to the inspector terminal 104.

[0080] In addition, the alarm output unit 318 transmits the X-ray image received from the image transmission terminal 102 to the inspector terminal 104 when performing any of the processes of step S811 to step S813. In addition, the alarm output unit 318 may output a specific determination result to the inspector terminal 104 when performing any of the processes of step S811 to step S813. For example, whether a prohibited object is included in the X-ray image and the probability thereof, whether an easily confused object is included in the X-ray image, whether a co-occurring object is included in the X-ray image, whether the co-occurring object is a co-occurring object of a prohibited object or a common object of a general object when a co-occurring object is included in the X-ray image, and coordinate information indicating the positions of prohibited objects or co-occurring objects when the X-ray image contains them are examples of the above-mentioned specific determination results.

[0081] Alternatively, when the prohibited object determination unit 317 determines that the probability of the prohibited object calculated in step S802 being a high probability (S805: Yes), the co-occurring object management DB 314 is further searched, and when it is determined that there are still co-occurring objects in the X-ray image, the alarm output unit 318 includes the item category of the co-occurring object and information indicating whether the co-occurring object is a prohibited object or a general object in the information indicating the warning in step S811.

[0082] In addition, when the prohibited object determination unit 317 determines in step S804 that the X-ray image does not contain prohibited objects but contains easily confused objects, in steps S811 to S813, information indicating that easily confused objects are contained may be sent to the inspector terminal 104. Thus, the inspector terminal 104 can display that easily confused objects are contained in the X-ray image, and the inspector is less likely to mistakenly determine that easily confused objects are prohibited objects, thereby improving inspection efficiency.

[0083] Figures 9 to 12 This is an example of a screen displayed by the inspection screen output unit 712 on the display device 704 based on the output from the image determination server 103. The user (inspector) of the inspector terminal 104 performs a visual inspection based on the displayed co-occurrence information and makes a final determination on whether there are prohibited items in the goods to be inspected. The inspector terminal 104 also highlights the X-ray image and the pixel areas of the prohibited items and co-occurrence items to assist the visual confirmation by the inspector.

[0084] Fig. 9 This is an example of a warning display screen displayed when the prohibited object determination unit 317 determines that there are prohibited objects (with a high probability) and co-occurring objects of prohibited objects in the X-ray image. Fig. 9 The display screen 900 is an example of a case where the process in step S805 is performed and the process in step S811 is performed.

[0085] in particular, Fig. 9 The display screen 900 is an example of a case where the prohibited object determination unit 317 determines that the probability of the prohibited object calculated in step S802 being a prohibited object is high (S805: Yes), and further performs a search of the co-occurring object management DB 314.

[0086] Figures 9 to 12 The display screen 900 includes, for example, an X-ray image display area 901, a message display area 902, an OK button 903, and an NG button 904. The inspector terminal 104 displays the X-ray image received from the image determination server 103 in the X-ray image display area 901. In addition, when it is determined that a prohibited object or a co-occurring object is included in the X-ray image, only the target pixel area of ​​the prohibited object or the co-occurring object may be cut out and enlarged and displayed in the X-ray image display area 901. In the message display area 902, a message corresponding to the alarm or the like received by the inspector terminal 104 from the alarm output unit 318 is displayed.

[0087] The OK button 903 is a button selected when the inspector visually confirms that the X-ray image does not contain prohibited objects. The NG button 904 is a button selected when the inspector visually confirms that the X-ray image contains prohibited objects. In addition, the inspector can associate information indicating that either the OK button 903 or the NG button 904 is selected for the X-ray image with the X-ray information and save it in the X-ray image DB 316.

[0088] exist Fig. 9In the example of , since it is determined that a prohibited object (of which the probability is high) and a co-occurring object exist in the X-ray image, the coordinate information of the prohibited object contained in the X-ray image area and the coordinate information of the co-occurring object associated with the prohibited object are displayed in addition to the X-ray image in the X-ray image display area 901. Fig. 9 In the example of , since it is determined that a prohibited object (of which the probability is high) and a co-occurring object exist in the X-ray image, a warning message is displayed in the message display area 902 .

[0089] Fig.10 This is an example of a display screen 900 for calling attention to the presence of a prohibited object (of which the probability is low) and a co-occurrence object of the prohibited object in the X-ray image by the prohibited object determination unit 317. Fig.10 The display screen 900 is an example in which the process of step S812 is performed after the process of step S805 to step S808.

[0090] exist Fig.10 In the example, since it is determined that a prohibited object (of which the probability is low) and co-occurring objects of the prohibited object exist in the X-ray image, in addition to the X-ray image, the coordinate information of the prohibited object contained in the X-ray image area and the coordinate information of the co-occurring objects associated with the prohibited object are displayed in the X-ray image display area 901.

[0091] In addition, Fig.10 In the example of , an attention-calling message is displayed in the message display area 902. Fig.10 In the example, the prohibited object determination unit 317 determines that although a prohibited object exists in the X-ray image, the probability is low. However, since it is determined that a co-occurrence object of the prohibited object exists in the X-ray image, "Suspected prohibited object (likelihood: high)" is displayed in the attention-calling message.

[0092] Fig.11 This is an example of a display screen 900 for calling attention to the presence of a prohibited object (of which the probability is low) in an X-ray image (but there is no co-occurring object). Fig.11 The display screen 900 is an example of a case where it is determined in step S807 that there are no co-occurring objects.

[0093] exist Fig.11 In the example, since it is determined that although a prohibited object exists in the X-ray image (with a low probability), there is no co-occurring object in the X-ray image, in addition to the X-ray image, the coordinate information of the prohibited object contained in the X-ray image area is displayed in the X-ray image display area 901.

[0094] In addition, Fig.11In the example of , an attention-calling message is displayed in the message display area 902. Fig.11 In the example, since the prohibited object determination unit 317 determines that although a prohibited object exists in the X-ray image, its probability is low, and further determines that there is no co-occurring object in the X-ray image, "Suspected prohibited object (likelihood: low)" is displayed in the attention-calling message.

[0095] Fig.12 This is an example of a display screen 900 displayed when the prohibited object determination unit 317 determines that there is no prohibited object in the X-ray image, but determines that there is a co-occurring object of the prohibited object in the X-ray image. Fig.12 The display screen 900 is an example of a case where it is determined in step S804 that it is not a prohibited object, but it is determined in step S810 that there is a co-occurring object related to the prohibited object.

[0096] exist Fig.12 In the example of , since it is determined that there is no prohibited object in the X-ray image, but there is a co-occurrence of the prohibited object in the X-ray image, in addition to the X-ray image, the coordinate information of the co-occurrence of the prohibited object contained in the X-ray image area is displayed in the X-ray image display area 901. Fig.11 In the example of FIG. 9 , a message calling attention to the presence of a co-occurring object in the X-ray image is displayed in the message display area 902 .

[0097] In addition, Figures 9 to 12 In the X-ray image display area 901, for the purpose of explanation, the items determined as co-occurring items are displayed with double lines, and the items determined as prohibited items are displayed with dotted lines and frames. However, the frame color may be set to a different color from the prohibited items, or the color of the X-ray image itself determined as prohibited items or co-occurring items may be changed to display them, and there is no limitation on the display method. In addition, the names of prohibited items, co-occurring items, and prohibited items associated with co-occurring items may be displayed together in the X-ray image display area 901 or the message display area 902, so as to support the inspector's interpretation.

[0098] In addition, Figures 9 to 12 In the display screen 900, a message may be displayed in a pop-up window, or an alarm or a warning may be issued using a mechanism such as a lamp or a speaker attached to the X-ray inspection device 101, etc., and the method of expressing the alarm is not limited. In addition, the probability of conformity with the prohibited object may be displayed as a score on the display screen 900, and the display shape and shape of the frame in the X-ray image display area 901, the light color of the lamp, and the volume output from the speaker may be changed or emphasized according to the score.

[0099] In addition, although the illustration is omitted, when it is determined in step S813 that no output is performed (i.e., when it is determined that there are neither prohibited objects nor co-occurring objects of prohibited objects in the X-ray image), the inspection screen output unit 712 can display the X-ray image on the display device 704 without displaying an alarm.

[0100] The image determination server 103 of this embodiment determines the content of the alarm not only based on whether a prohibited object exists in a pixel area of ​​an X-ray image, but also based on whether a co-occurring object exists.

[0101] Therefore, even if the image determination server 103 cannot correctly determine the existence of the prohibited object itself, it can determine the existence of the prohibited object based on the co-occurring objects associated with the prohibited object, so the determination accuracy is improved and the alarm can be output more accurately, thereby further reducing the risk of inspectors' omissions.

[0102] Furthermore, since the display screen 900 displays the prohibited objects and the co-occurring objects in a form that the inspector can easily recognize, the inspector can reduce the pixel area to be visually inspected, thereby improving the quality of the visual inspection performed by the inspector.

[0103] In addition, the present invention is not limited to the above-mentioned embodiments, and includes various modified examples. For example, the above-mentioned embodiments are described in detail in order to explain the present invention in an easy-to-understand manner, and are not limited to necessarily having all the structures described. In addition, a part of the structure of a certain embodiment can also be replaced by the structure of other embodiments, and in addition, the structure of other embodiments can also be added to the structure of a certain embodiment. In addition, with respect to a part of the structure of each embodiment, other structures can be added, deleted, or replaced.

[0104] For example, in the above-mentioned embodiment, an example of an inspection device that photographs X-ray images of goods is described, but the image is not limited to that obtained by X-ray irradiation, as long as it can be recognized as an image, and further includes a form in which the composition, density, and shape of a specific material can be detected by the device based on the image. In addition, the subject is not limited to goods. For example, this embodiment can also be used for detecting crimes based on images photographed by security cameras or discovering lesions based on images photographed by MRI (Magnetic Resonance Imaging) devices.

[0105] In addition, the above-mentioned various structures, functions, processing units, processing mechanisms, etc. can also be implemented by hardware by designing a part or all of them, for example, using integrated circuits. In addition, the above-mentioned various structures, functions, etc. can also be implemented by software by interpreting and executing programs that implement various functions through a processor. The information of programs, tables, files, etc. that implement various functions can be placed in a recording device such as a memory or a hard disk, SSD (Solid State Drive), or a recording medium such as an IC card, SD card, DVD, etc.

[0106] Furthermore, the control lines and information lines are shown as those considered necessary for explanation, and do not necessarily show all the control lines or information lines on the product. In practice, it can be considered that almost all components are connected to each other.

Claims

1. An alarm output device, including a processor and a memory, connected to an output device; The above memory holds: The image information respectively indicates one or more inspection objects included in the subject of the determination target image; Alarm object information, indicating the alarm object; as well as The co-occurrence information at least indicates an alarm co-occurrence, wherein the alarm co-occurrence is an object that is preset to be included in the image at the same time as the alarm target object; The above processor executes: Referring to the alarm object information, determining whether at least one of the one or more inspection objects meets the alarm object; When it is determined that none of the one or more inspection objects meets the alarm object, referring to the co-occurrence information, it is determined whether at least one of the one or more inspection objects meets at least one of the alarm co-occurrence objects; When it is determined that at least one of the above-mentioned one or more inspection objects meets the above-mentioned alarm object and satisfies the specified conditions, or when it is determined that although none of the above-mentioned one or more inspection objects meets the above-mentioned alarm object, at least one of the above-mentioned one or more inspection objects meets at least one of the above-mentioned alarm co-occurrence objects, an alarm is output to the above-mentioned output device.

2. The alarm output device according to claim 1, The image information indicates the probability that each of the one or more inspection objects is included in the photographed object; The predetermined condition is that the probability that the at least one inspection object that corresponds to the alarm target object corresponds to the alarm target object is greater than or equal to a predetermined value.

3. The alarm output device according to claim 2, The above processor executes: outputting a first alarm to the output device when the probability that the at least one inspection object determined to be the alarm object is the alarm object is greater than or equal to the predetermined value; When it is determined that although the probability that the above-mentioned one or more inspection objects meet the above-mentioned alarm objects is less than the above-mentioned specified value, at least one of the above-mentioned one or more inspection objects meets at least one of the above-mentioned alarm co-occurrence objects, a second alarm with a lower alarm level than the above-mentioned first alarm is output to the above-mentioned output device.

4. The alarm output device according to claim 2, The processor does not output the alarm to the output device when it is determined that none of the one or more inspection objects matches the alarm object and any of the alarm co-occurring objects.

5. The alarm output device according to claim 2, The processor does not output the alarm to the output device when it is determined that the probability that the one or more inspection objects match the alarm object is less than the predetermined value and none of the one or more inspection objects matches any of the alarm co-occurrence objects.

6. The alarm output device according to claim 2, The co-occurring object information further indicates general co-occurring objects, wherein the general co-occurring objects are objects that are preset to be included in the image together with the general objects, and the general objects are objects that are not alarm targets; The above processor executes: Referring to the co-occurring object information, determining whether at least one of the one or more inspection objects corresponds to the general co-occurring object; When it is determined that although the above-mentioned one or more inspection objects do not conform to the above-mentioned alarm objects and any of the above-mentioned alarm co-occurring objects, at least one of the above-mentioned one or more inspection objects conforms to the above-mentioned general co-occurring objects, information indicating that the above-mentioned one or more inspection objects contain easily confused objects will be output to the above-mentioned output device.

7. The alarm output device according to claim 6, outputting a first alarm to the output device when the probability that the at least one inspection object determined to be the alarm object is the alarm object is greater than a predetermined value; When the probability that the above-mentioned one or more inspection objects match the above-mentioned alarm objects is less than the above-mentioned specified value, and the above-mentioned one or more inspection objects do not match any of the above-mentioned alarm co-occurrence objects and the above-mentioned general co-occurrence objects, a third alarm with a lower alarm level than the above-mentioned first alarm is output to the above-mentioned output device.

8. The alarm output device according to claim 1, The alarm object information further indicates easily confused objects, which are objects that are pre-set to be similar to the alarm object; The processor refers to the alarm object information and, when determining that at least one of the one or more inspection objects corresponds to the easily confused object, outputs information indicating that the one or more inspection objects contain the easily confused object to the output device.

9. An alarm output method, which is an alarm output method of an alarm output device, The alarm output device includes a processor and a memory connected to the output device; The above memory holds: The image information respectively indicates one or more inspection objects included in the subject of the determination target image; Alarm object information, indicating the alarm object; as well as The co-occurrence information at least indicates an alarm co-occurrence, wherein the alarm co-occurrence is an object that is preset to be included in the image at the same time as the alarm target object; In the above alarm output method, The processor refers to the alarm object information to determine whether at least one of the one or more inspection objects meets the alarm object. The processor, when determining that none of the one or more inspection objects matches the alarm object, refers to the co-occurrence information to determine whether at least one of the one or more inspection objects matches at least one of the alarm co-occurrence objects; The processor outputs an alarm to the output device when it is determined that at least one of the one or more inspection objects meets the alarm object and satisfies the specified conditions, or when it is determined that although none of the one or more inspection objects meet the alarm object, at least one of the one or more inspection objects meets at least one of the alarm co-occurrence objects.

10. A computer-readable non-transitory recording medium storing a program for causing an alarm output device to execute an alarm output. The alarm output device includes a processor and a memory connected to the output device; The above memory holds: The image information respectively indicates one or more inspection objects included in the subject of the determination target image; Alarm object information, indicating the alarm object; as well as The co-occurrence information at least indicates an alarm co-occurrence, wherein the alarm co-occurrence is an object that is preset to be included in the image at the same time as the alarm target object; The above program causes the above processor to execute: Referring to the alarm object information, determining whether at least one of the one or more inspection objects meets the requirements of the alarm object processing; When it is determined that none of the one or more inspection objects meets the alarm object, determining whether at least one of the one or more inspection objects meets at least one of the alarm co-occurrence objects by referring to the co-occurrence object information; When it is determined that at least one of the above-mentioned one or more inspection objects meets the above-mentioned alarm object and satisfies the specified conditions, or when it is determined that although none of the above-mentioned one or more inspection objects meets the above-mentioned alarm object, at least one of the above-mentioned one or more inspection objects meets at least one of the above-mentioned alarm co-occurrence objects, an alarm is output to the above-mentioned output device.

Images