Inspection equipment, inspection system, and inspection method

Abstract

To provide an inspection device, an inspection system, and an inspection method, which are capable of accurately detecting a specific target object in an inspection target.SOLUTION: According to an embodiment, an inspection device includes an image acquisition unit and a processor. The image acquisition unit acquires captured image data including a captured image captured by irradiating an inspection target with electromagnetic waves, and physical property information representing a physical property of each portion of the captured image. The processor: detects a target object candidate, which is a candidate of a specific target object in the inspection target, based on the physical property information included in the captured image data; detects a specific portion, which is a specific part or object in the inspection target; calculates a relative relationship between the target object candidate and the specific portion; and reports a target object candidate for which the calculated relative relationship satisfies a predetermined condition.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] Embodiments of the present invention relate to an inspection apparatus, an inspection system, and an inspection method.

Background Art

[0002] Conventionally, in an inspection site, there may be a need for an inspection operation to check whether a predetermined object (hereinafter also referred to as a specific object) exists within an inspection target such as a package. When there are a large number of packages that can be inspection targets, it takes a great deal of time and effort for an inspector to open and inspect all the packages. For this reason, in an inspection site where a large number of packages are inspected, an operation of performing an open inspection on a package estimated to have a high possibility of containing a specific object by screening using various methods is desired.

[0003] In recent years, an inspection system has been proposed that detects a specific object within a package by analyzing a captured image (X-ray image) obtained by capturing the package using electromagnetic waves such as X-rays. Such an inspection system detects, for example, an object estimated to be a specific object composed of a specific substance based on whether the effective atomic number, which is the average atomic number of the substances contained in each pixel constituting the image obtained by capturing the package using X-rays, is within a threshold value. However, there is a problem that it is difficult to distinguish between a specific object and an object having physical properties similar to those of the specific object based only on the information on the effective atomic number and density obtained from the X-ray image.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] The problem that this invention aims to solve is to provide an inspection device, an inspection system, and an inspection method that can accurately detect specific objects within an object being inspected. [Means for solving the problem]

[0006] According to one embodiment, the inspection device includes an image acquisition unit and a processor. The image acquisition unit acquires captured image data, which includes an image taken by irradiating the object to be inspected with electromagnetic waves and physical property information indicating the physical properties of each part of the captured image. The processor detects candidate objects, which are candidates for a specific object in the object to be inspected, based on the physical property information contained in the captured image data, detects specific parts, which are specific parts or objects in the object to be inspected, calculates the relative relationship between the candidate objects and the specific parts, and notifies the candidate objects whose calculated relative relationship satisfies predetermined conditions. [Brief explanation of the drawing]

[0007] [Figure 1] Figure 1 is a schematic diagram showing the overall configuration of an inspection system including an inspection device according to an embodiment. [Figure 2] Figure 2 is a block diagram showing an example configuration of an information management system including an inspection device according to an embodiment. [Figure 3] Figure 3 is a block diagram showing an example of the configuration of the imaging device and the control system in the inspection device according to the embodiment. [Figure 4] Figure 4 is a block diagram showing an example configuration of a higher-level management device in an information management system including an inspection device according to an embodiment. [Figure 5] Figure 5 is a flowchart illustrating the overall flow of the inspection process using the inspection apparatus according to this embodiment. [Figure 6]Figure 6(a) shows an example of an image captured by an imaging device supplied to an inspection device according to the embodiment. Figure 6(b) shows an example of an image of a candidate object detected from an image by the inspection device according to the embodiment. Figure 6(c) shows an example of an image of a specific part detected from an image by the inspection device according to the embodiment. Figure 6(d) shows an example of a candidate object detected as a specific object based on the relative relationship between the candidate object detected by the inspection device according to the embodiment and the specific part. [Figure 7] Figure 7 is a flowchart illustrating an example of the operation of the object candidate detection process in the inspection apparatus according to the embodiment. [Figure 8] Figure 8 shows an example of a two-dimensional image generated by slicing a three-dimensional image captured by an imaging device supplied to an inspection apparatus according to the embodiment along each axis. [Figure 9] Figure 9 is a flowchart illustrating an example of the operation of the relative relationship calculation process in the inspection apparatus according to this embodiment. [Figure 10] Figure 10 shows an example of the display of inspection results for a specific object shown on a display device by the inspection apparatus according to the embodiment. [Figure 11] Figure 11 shows an example of a display in which the inspection device according to the embodiment displays the detection result of a specific object in the captured image. [Figure 12] Figure 12 shows an example of a display in which the inspection device according to the embodiment displays the detection result of a specific part in the captured image. [Figure 13] Figure 13 shows an example of how the inspection device according to the embodiment displays the relative distance between a specific part and a candidate object in a captured image. [Figure 14] Figure 14 shows an example in which the inspection device according to the embodiment displays the detection result of a specific object and also displays buttons that instruct the display of a candidate object and a specific part. [Figure 15] Figure 15 schematically shows an example in which an inspection apparatus according to the embodiment displays a binary image obtained by slicing a three-dimensional captured image at a specified location. [Modes for carrying out the invention]

[0008] The embodiments will be described below with reference to the drawings. Figure 1 is a diagram illustrating a schematic configuration example of an inspection system 1 including an inspection device 13 according to an embodiment. The inspection system 1 according to this embodiment is a system for inspecting whether or not a specific detectable object is present in the luggage to be inspected. The specific detectable object (specific object) is, for example, a hazardous item, a hazardous chemical, a drug whose handling is prohibited, or a substance whose import or export into or out of a designated area such as within a country is prohibited. Furthermore, the specific detectable object does not have to be a solid having a specific shape, but may also include substances such as liquids and powders.

[0009] In the configuration example shown in Figure 1, the inspection system 1 includes a conveyor 11, an imaging device 12, an inspection device 13, a display device 14, an operating device 15, and a speaker 16, etc. The inspection device 13 is connected to the imaging device 12, the display device 14, the operating device 15, and the speaker 16 via communication.

[0010] The conveyor belt 11 is a device for transporting the packages M to be inspected. The conveyor belt 11 transports the packages M to be inspected to the image capture position (reading position) of the imaging device 12. For example, the conveyor belt 11 transports packages M supplied by a worker. Alternatively, the conveyor belt 11 may be configured to transport packages M supplied by a robotic arm or the like.

[0011] The imaging device 12 acquires imaging data including an imaging image of the inspection target and object information indicating the physical properties of each part of the imaging image by irradiating the luggage M to be inspected with electromagnetic waves. The imaging device 12 supplies the imaging data of the luggage M to the inspection device 13. The imaging device 12 may be any device that can acquire imaging data with which the inspection device 13 can detect candidates (hereinafter also referred to as object candidates) estimated to be specific detection targets and specific parts or specific objects (hereinafter also referred to as specific parts) in the luggage M. The imaging device 12 may acquire two-dimensional image data as the imaging image, or may acquire three-dimensional image data.

[0012] The imaging device 12 is, for example, an X-ray CT imaging device. The X-ray CT imaging device as an example of the imaging device 12 acquires three-dimensional X-ray image data as an imaging image by irradiating X-rays from around the luggage M conveyed by the conveyor 11. Further, the X-ray CT imaging device as the imaging device 12 acquires imaging data including a three-dimensional X-ray image of the luggage M and object information indicating the physical properties of each constituent unit (pixel or voxel) constituting the X-ray image. The X-ray CT imaging device as the imaging device 12 supplies the imaging data acquired from the luggage M to the inspection device 13.

[0013] Note that the imaging device 12 is not limited to an X-ray CT imaging device, but in the embodiments described below, it is assumed that the imaging device 12 is an X-ray CT imaging device for the purpose of explanation.

[0014] The inspection device 13 has various functions such as a function of processing an imaging image obtained by the imaging device 12 by using electromagnetic waves to image the luggage M. For example, the inspection device 13 has a function (reception unit) of acquiring an imaging image obtained by the imaging device 12 by using electromagnetic waves to image the luggage M, and a function (transmission unit) of outputting output information based on information obtained by an inspection process described below by using an output device such as the display device 14 or the speaker 16.

[0015] Furthermore, the inspection device 13 has a function (object candidate detection unit) for detecting candidates (object candidates) that are presumed to be specific objects present in the luggage M from the captured image data acquired from the imaging device 12. Based on setting values ​​that are set according to the physical properties of the specific object, the inspection device 13 detects candidates (object candidates) that are presumed to be specific objects in the captured image of the luggage M acquired from the imaging device 12.

[0016] For example, the inspection device 13 detects potential objects based on the physical properties (density, effective atomic number, etc.) of each pixel or voxel in the captured image (X-ray image) taken by the X-ray CT scanner, which acts as the imaging device 12. A pixel corresponds to a pixel, which is the smallest unit that makes up two-dimensional image data. A voxel is the smallest unit of data that makes up three-dimensional data and represents a value in a normal grid unit. A voxel is a value that corresponds to a pixel in two-dimensional image data.

[0017] Furthermore, the inspection device 13 has a function (specific part detection unit) to detect specific parts or specific objects (hereinafter simply referred to as specific parts) from images taken of the luggage M. For example, the inspection device 13 detects specific parts, which are specific parts or specific objects, in the images taken by the imaging device 12. Specific parts are parts or objects that are often used to conceal specific objects that have been collected in advance. Specific parts may be parts or objects with a predetermined shape, or parts or objects placed in predetermined positions on the luggage. For example, specific parts could include shoes, cameras, personal computers, the walls of luggage such as trunks, or cigarettes.

[0018] The specific part detection unit of the inspection device 13 extracts images of areas or objects that are likely to be designated as specific parts from the captured images taken by the imaging device 12, and detects the specific part by recognizing the shape of the extracted images. Specifically, the specific part detection unit of the inspection device 13 can detect the specific part using semantic segmentation, which associates labels or categories with all pixels in the captured image.

[0019] However, the method for detecting specific parts applied to the specific part detection unit is not limited to the method described above. It may also be a general image recognition method, or an object recognition method such as SSD (Single Shot Detector, Single Shot MultiBox Detector) using machine learning or deep learning that takes features such as HOG (Histograms of Oriented Gradients) features extracted from an image as input.

[0020] Furthermore, the inspection device 13 has a function (relative relationship calculation unit) that evaluates whether an object candidate is likely to be a specific object (is suspected to be a specific object) based on the relative relationship, such as the distance between each object candidate and the specific part in the image captured by the imaging device 12. For example, the inspection device 13 determines that there is a high probability that the object being inspected is a specific object if the object candidate is adjacent to or included in the specific part. Alternatively, the inspection device 13 may calculate the relative distance between the object candidate and the specific part as a positional relationship, and determine that there is a high probability that the object candidate is a specific detection target if the calculated relative distance is less than or equal to a predetermined value.

[0021] The display device 14 is an output device for informing the inspector of the inspection results. The display device 14 displays guidance screens and the like in accordance with the control of the inspection device 13. As a guidance screen to be presented to the inspector, the display device 14 displays guidance screens that show the results of the inspection processing on the captured image of the luggage M. For example, the display device 14 displays an image that clearly identifies the target object candidate and specific parts in the image captured by the camera 12 generated by the inspection device 13.

[0022] The operating device 15 generates an operation signal in response to the operator's input and supplies the operation signal to the inspection device 13. The operating device 15 is composed of an operation device such as a keyboard and a pointing device. Alternatively, the operating device 15 may be composed of a touch panel or the like on the display screen of the display device 14.

[0023] Speaker 16 is an output device for informing inspectors of inspection results and other information by voice. Speaker 16 outputs voice guidance to inform inspectors of the results of the inspection process and other information.

[0024] Furthermore, the inspection device 13 has a function to perform an alerting process that notifies (issues an alert) the inspector of information such as potential target objects. For example, the inspection device 13 displays on the display device 14 the potential target objects that are determined to be highly likely to be specific target objects, along with the captured image taken by the camera device 12 of the package M being transported by the conveyor 11.

[0025] Furthermore, the inspection device 13 may display all potential objects in the captured image of the package M on the display device 14, and may also display object candidates that are determined to be highly likely to be specific objects using a different color or mark from other candidates. In addition, the inspection device 13 may display images on the display device 14 that clearly indicate the potential objects and the specific parts in the captured image of the package M. Furthermore, the inspection device 13 may switch the content displayed on the display device 14 in response to instructions from the worker via the operation device 15.

[0026] Next, the configuration of the information management system 100, which includes the inspection system 1 according to the embodiment, will be described. Figure 2 shows an example configuration of an information management system 100 including an inspection system 1 according to an embodiment. As shown in Figure 2, the information management system 100 has a higher-level management device 101 that is connected to the inspection devices 13 of the inspection system 1 installed in each inspection area. The higher-level management device 101 functions as an information management device that collects data from the inspection devices in each inspection system 1 and supplies data to each inspection device.

[0027] The higher-level management device 101 is composed of a computer, such as a server. The higher-level management device 101 has a storage device that stores information related to the inspections performed by each inspection system 1. The higher-level management device 101 may also have an interface that connects to the server device that stores the inspection information.

[0028] The higher-level management device 101 acquires information from the inspection devices 13 in each inspection system 1. The higher-level management device 101 stores and aggregates the information acquired from the inspection devices 13 in each inspection system 1. The higher-level management device 101 supplies information to the inspection devices 13 in each inspection system 1. For example, the higher-level management device 101 distributes setting values ​​used for inspection processing to the inspection devices 13 in each inspection system 1. The higher-level management device 101 may also distribute update data for the programs that each inspection device 13 uses to execute inspection processing.

[0029] Next, the configuration of the control system of the inspection device 13 in the inspection system 1 according to the embodiment will be described. Figure 3 is a block diagram showing an example of the configuration of the control system of the inspection device 13 in the inspection system 1 according to the embodiment. As shown in Figure 3, the imaging device 12 includes an imaging unit 21, a processing unit 22, and an output unit 23. The imaging unit 21 irradiates an object to be inspected, such as luggage M, with electromagnetic waves such as X-rays to capture an image. For example, if the imaging device 12 is an X-ray CT scanner, the imaging unit 21 acquires three-dimensional X-ray image data by irradiating X-rays from around the luggage M being transported by the conveyor belt 11.

[0030] The processing unit 22 includes a processor and various types of memory, and the processor executes various processes by running programs stored in the memory. For example, the processing unit 22 processes the image captured by the imaging unit 21 when it irradiates electromagnetic waves to generate captured image data that includes the captured image and physical property information indicating the physical properties of the constituent units (pixels or voxels) that make up the captured image.

[0031] The output unit 23 is an interface for outputting data such as captured image data. The output unit 23 has an interface that corresponds to the image interface 39 of the inspection device 13 and outputs captured image data to the inspection device 13. The output unit 23 may also be an input / output interface that includes an interface for inputting data such as control data from the connected inspection device 13.

[0032] Furthermore, as shown in Figure 3, the inspection device 13 includes a processor 31, ROM 32, RAM 33, storage unit 34, communication unit 35, display interface (I / F) 36, operation interface (I / F) 37, voice interface (I / F) 38, and image interface (I / F) 39.

[0033] The processor 31 performs arithmetic processing. The processor 31 is, for example, a CPU (Central Processing Unit). The processor 31 functions as a processing unit that performs various operations by executing programs stored in the ROM 32 or storage unit 34 using the RAM 33.

[0034] ROM32 is a read-only, non-volatile memory. ROM32 stores program data and control data, among other things. RAM33 is a volatile memory that functions as working memory. RAM33 temporarily stores data.

[0035] The memory unit 34 is a rewritable non-volatile memory. The memory unit 34 is composed of a hard disk drive (HDD), a solid-state drive (SSD), etc. The memory unit 34 stores information such as program data, setting values ​​as control data, and the results of inspection processing.

[0036] The communication unit 35 is a communication interface for communicating with the higher-level management device 101. The processor 31 communicates with the higher-level management device 101 via the communication unit 35. The processor 31 sends data such as processing results to the higher-level management device 101 and receives data from the higher-level management device 101 via the communication unit 35.

[0037] The display interface 36 is an interface for connecting to the display device 14, which acts as an output device. The display interface 36 only needs to be compatible with the interfaces provided by the display device 14. The processor 31 controls the content to be displayed on the display device 14 via the display interface 36.

[0038] The operation interface 37 is an interface for connecting to the operating device 15. The operation interface 37 only needs to correspond to the interfaces provided by the operating device 15. The processor 31 acquires information input by the operating device 15 via the operation interface 37.

[0039] The audio interface 38 is an interface for connecting to the speaker 16, which acts as an output device. The audio interface 38 only needs to be compatible with the interface provided by the speaker 16. The processor 31 causes the speaker 16 to output sound via the audio interface 38.

[0040] The image interface 39 is an interface for connecting to the imaging device 12. The image interface 39 is an image acquisition unit (image acquisition interface) for acquiring images from the imaging device 12. The image interface 39 only needs to be compatible with the interface provided by the imaging device 12, such as an X-ray CT scanner. The processor 31 acquires the images (X-ray images) taken by the X-ray CT scanner, which is the imaging device 12, via the image interface 39. The processor 31 may also control the imaging operation of the imaging device 12 on the luggage M via the image interface 39.

[0041] Next, the configuration of the higher-level management device 101 in the information management system 100, which includes the inspection system 1 according to the embodiment, will be described. Figure 4 is a block diagram showing an example configuration of a higher-level management device 101 in an information management system 100 including an inspection system 1 according to an embodiment. The higher-level management device 101 is an information management device that manages information for the entire inspection system 1. The higher-level management device 101 is a computer that communicates with the inspection devices 13 of the inspection system 1 installed in each inspection area. The higher-level management device 101 is configured, for example, as a server device.

[0042] In the configuration example shown in Figure 4, the higher-level management device 101 includes a processor 41, ROM 42, RAM 43, storage unit 44, and communication unit 45. The processor 41 performs arithmetic processing. The processor 41 is, for example, a CPU (Central Processing Unit). The processor 41 functions as a processing unit that performs various operations by executing programs stored in the ROM 42 or storage unit 44 using the RAM 43.

[0043] ROM42 is a read-only non-volatile memory. ROM42 stores program data and control data, etc. RAM43 is a volatile memory that functions as working memory. RAM43 temporarily stores data.

[0044] The memory unit 44 is a rewritable non-volatile memory. The memory unit 44 is composed of a hard disk drive (HDD), a solid-state drive (SSD), etc. The memory unit 34 stores information such as program data, setting values ​​as control data, and data collected from each inspection device 13.

[0045] The communication unit 45 is a communication interface for communicating with the inspection device 13 in each inspection system 1. The processor 41 communicates with the inspection device 13 via the communication unit 45. The processor 41 receives data such as processing results from the inspection device 13 and transmits data to the inspection device 13 via the communication unit 45.

[0046] Next, the inspection process for inspecting the package M to be inspected in the inspection system 1 according to this embodiment will be described. Figure 5 is a flowchart illustrating the flow of the inspection process in the inspection system 1 according to the embodiment. Figures 6(a) to 6(d) are schematic diagrams showing examples of images obtained by each step of the inspection process.

[0047] In the inspection system 1 configured as shown in Figure 1, packages M to be inspected are sequentially placed onto a conveyor belt 11. The conveyor belt 11 transports the placed packages M to the position where the image capture device 12 will take the image. The imaging unit 21 of the imaging device 12 acquires an image showing the contents of the packages M by irradiating them with electromagnetic waves as they are transported by the conveyor belt 11.

[0048] The captured image can be any image data that shows the state inside the package M. For example, the X-ray CT scanner of the imaging device 12 acquires three-dimensional data showing the state inside the package M as a captured image by irradiating the package M, which is being transported to the imaging position, with X-rays. In addition, the physical property information is acquired to show the physical properties (density and effective atomic number) of each pixel or voxel that makes up the captured image.

[0049] The processing unit 22 of the imaging device 12 generates image data that includes an image of the luggage M captured by the imaging unit 21 and physical property information indicating the physical properties of each pixel or voxel in the image. When the processing unit 22 of the imaging device acquires the image data of the luggage M, it outputs the image data to the inspection device 13 via the output unit 23.

[0050] The inspection device 13 acquires captured image data from the imaging device 12 via the image interface 39 (step S101). The processor 31 executes the process of acquiring captured image data as the processing performed by the receiving unit described above. For example, the inspection device 13 acquires captured image data that includes a captured image as shown in Figure 6(a) and physical property information indicating the physical properties of each part (pixel or voxel) of the captured image. The captured image may be 2D data, 3D data, or multiple 2D image data obtained by slicing 3D data along a specific axis. In the following explanation, for the sake of simplicity, the captured image will be described as 2D image data.

[0051] The processor 31 of the inspection device 13 performs an object candidate detection process to detect object candidates as candidates for a specific object from the captured image data acquired from the imaging device 12 (step S102). The processor 31 performs the object candidate detection process as the process performed by the object candidate detection unit described above. For example, the processor 31 detects as object candidates locations (pixels) in the captured image where the physical properties are similar to or match those of the specific object, based on setting values ​​set according to the physical properties of the specific object to be detected. Figure 6(b) shows an example of object candidates detected from captured image data as shown in Figure 6(a).

[0052] Furthermore, the processor 31 of the inspection device 13 performs a specific part detection process based on the captured image data acquired from the imaging device 12, to detect specific parts or objects (specific parts) in the captured image of the luggage M that are likely to be located in the area to be detected (step S103).

[0053] The processor 31 performs a specific part detection process as a process performed by the specific part detection unit described above. For example, the processor 31 sets in advance the shape of a part or object to be detected as a specific part, and detects an image region in the captured image that has a shape similar to the shape of the specific part to be detected as a specific part. Figure 6(c) shows an example of a specific part detected from captured image data as shown in Figure 6(a).

[0054] After the processor 31 has performed the object candidate detection process and the identification part detection process, it performs a relative relationship calculation process to calculate the relative relationship between the detected object candidate and the identification part (step S104). The processor 31 performs the relative relationship detection process as a process by the relative relationship calculation unit described above. The relative relationship between the object candidate and the identification part is information for evaluating whether or not the object candidate is suspected to be the specific object to be detected. For example, the processor 31 calculates the relative distance between the object candidate and the identification part as the relative relationship.

[0055] The processor 31 calculates the relative relationship between the target object candidate and the specific part, and then detects target object candidates that are suspected to be specific objects based on the relative relationship with the specific part (step S105). For example, the processor 31 determines that a target object candidate that satisfies a predetermined condition where the relative relationship with the specific part (e.g., relative distance) is a preset value is suspected to be a specific object (detected as a specific object).

[0056] The processor 31 performs an alarm notification process (step S106) to notify (emit) the detection result of an object candidate suspected to be a specific object, based on the relative relationship between the object candidate and the identification unit, via an output device such as the display device 14 or speaker 16. For example, if the processor 31 detects an object candidate suspected to be a specific object as a result of the processing by the transmission unit described above, it performs an alarm notification process to emit the detection result via an output device such as the display device 14 or speaker 16.

[0057] For example, the processor 31 displays on the display device 14 an object candidate whose relative distance from the specific part is below a preset threshold as an object suspected to be a specific object. Figure 6(d) is a diagram showing an example of a display that shows an object candidate that has been determined to be an object suspected to be a specific object, and the specific part associated with that object candidate, based on the relative relationship between the object candidate shown in Figure 6(b) and the specific part shown in Figure 6(c).

[0058] For packages in which the detection result of a potential target object is displayed on the display device 14 due to the alarm process, an inspection is carried out by an inspector. The inspector performs the inspection of the contents of the package while referring to the detection result and other information displayed by the alarm process. The inspection system 1 may be operated to perform the inspection on all packages M, or it may be operated to perform the inspection on packages M in which a candidate target object suspected to be a specific target object has been detected.

[0059] Next, an example of the object candidate detection process by the object candidate detection unit of the inspection device 13 according to this embodiment will be described. Figure 7 is a flowchart illustrating an example of the object candidate detection process by the object candidate detection unit of the inspection device 13 according to this embodiment.

[0060] The processor 31 of the inspection device 13, acting as a target candidate detection unit, detects target candidate objects that are similar to or match the physical properties of a specific target object to be detected, based on captured image data including, for example, a captured image of the luggage M obtained from the imaging device 12 and effective atomic number and density data for each part in the captured image. Specifically, it determines whether an object is a target candidate by checking whether the effective atomic number and density of each pixel or voxel in the captured image, which is the input data, are within a threshold range determined based on the physical property values ​​of the specific target object to be detected that have been collected in advance.

[0061] In the processing example shown in Figure 7, the processor 31 of the inspection device 13 acquires the physical property value Z of the specific object to be detected (step S201). Here, the physical property value Z of the specific object may be input by the administrator or inspector using the operating device 15, or it may be acquired from an external device such as the higher-level management device 101.

[0062] When the processor 31 obtains the physical property value Z of a specific object, it sets a threshold E based on the physical property value Z of the specific object (step S202). For example, the processor 31 sets a threshold E that sets a predetermined tolerance range for the physical property value Z of the specific object.

[0063] Note that the processes of steps S201 and S202 may be replaced with a process of setting a threshold value E input by an administrator or an inspector using the operation device 15. Further, the processes of steps S201 and S202 may be such that information specifying the threshold value E is acquired from the upper management device 101, and the threshold value E acquired from the upper management device 101 is set as a set value in the detection process of the object candidate.

[0064] The processor 31 of the inspection device 13 that has set the threshold value E, which is a set value in the detection process of the object candidate, acquires imaging data including an imaging image obtained by imaging the luggage M, which is an inspection target, using electromagnetic waves such as X-rays by the imaging device 12 and object property information indicating the physical properties of each part of the imaging image (step S203). When the imaging data of the luggage M is acquired from the imaging device 12, the processor 31 executes a process of detecting an object candidate included in the imaging image (steps S204 to S207).

[0065] In the processing example shown in FIG. 7, the processor 31 determines whether the difference between the physical property value I(p) of each pixel p constituting the imaging image of the luggage M and the physical property value Z of the specific detection object is less than the threshold value E (|I(p) - Z| < E) (step S205). When |I(p) - Z| < E (step S205, YES), the processor 31 selects the pixel p in the imaging image as an object candidate (step S206). The processor 31 repeatedly executes the processes of steps S205 and S206 for all the pixels constituting the imaging image of the luggage M (steps S204, S207).

[0066] When the processes of steps S205 and S206 for all the pixels in the imaging image are completed, the processor 31 detects an image area composed of the set of pixels detected as the object candidate pixels as the object candidate (step S208). For example, the processor 31 detects, as the object candidate, an image area in which pixels having a physical property value I(p) whose difference from the physical property value Z of the specific object is less than the threshold value E (or an image area composed of a set of pixels whose relative distance is less than or equal to a predetermined distance).

[0067] According to the example of object candidate detection process described above, the inspection device detects object candidates by determining whether the difference between the physical properties of each part in the captured image obtained by irradiating with electromagnetic waves such as X-ray images, such as the effective atomic number and density data, and the physical properties of the specified object, is within a predetermined threshold range. As a result, the inspection device according to the embodiment can detect candidate specified objects present in the package even if the specified object to be detected is a substance such as a liquid or powder that does not have a specific shape.

[0068] In the object candidate detection process described above, a 2D image was processed and each pixel was determined to be an object candidate. However, if the captured image is 3D data, object candidates in the 3D captured image can be detected by determining whether each voxel is an object candidate.

[0069] Furthermore, the processor 31 of the inspection device 13 may perform the above-described object candidate detection process on multiple 2D images obtained by slicing (dividing) a 3D image as a captured image along each axis. Figure 8 shows an example of multiple 2D images obtained by slicing (dividing) 3D data as an captured image along each axis (x axis, y axis, z axis). Once multiple two-dimensional image data, as shown in Figure 8, are obtained, the processor 31 performs the object candidate detection process described above for each two-dimensional image data. After detecting object candidates in each two-dimensional image data, the processor 31 may also detect object candidates in three-dimensional space by superimposing the object candidate detection results for each two-dimensional image.

[0070] Furthermore, the object candidate detection process is not limited to the methods described above; any method that can detect object candidates using data on specific objects collected in advance is acceptable. For example, object candidate detection may be performed using a hash table or machine learning. Also, if the specific object has distinctive shape characteristics, object candidate detection may be performed using general object recognition.

[0071] Next, the relative relationship calculation process by the relative relationship calculation unit of the inspection device 13 according to this embodiment will be described in detail. The processor 31 of the inspection device 13, acting as a relative relationship calculation unit, calculates the relative relationship between the object candidate detected from the captured image and the specific unit. The relative relationship is calculated as an index value for evaluating whether the object candidate is a specific object to be detected. For example, the relative relationship is calculated as the distance (relative distance) between the object candidate and the specific unit. In this case, it can be determined that an object candidate that is close to the specific unit is likely to be a specific object.

[0072] Furthermore, in terms of relative relationships, the inclusion relationship between the target object candidate and the specific part, or the positional relationship such as whether or not they are adjacent, may also be calculated. In this case, target object candidate within the specific part, or target object candidate adjacent to the specific part, can be evaluated as suspected to be the specific object.

[0073] Furthermore, in terms of relative relationships, it is also possible to calculate physical property information (for example, average values ​​of density and effective atomic number) for a combination of the physical property information of the candidate object and the physical property information of the specific part. In this case, even if the substances constituting the specific part and the substances constituting the candidate object are mixed, it is possible to detect a candidate object that is suspected to be the specific object.

[0074] Figure 9 is a flowchart illustrating the process of calculating the relative distance between a candidate object and a specific unit, as an example of the relative relationship calculation process performed by the relative relationship calculation unit of the inspection device 13 according to this embodiment. Figure 9 illustrates an example of a process for calculating the relative distance between a candidate object and a specific part based on the distance between each pixel detected as a candidate object and each pixel forming the specific part. One example of calculating the relative distance as a relative relationship between a candidate object and a specific part is to calculate the Euclidean distance between the center position of the candidate object and the center position of the specific part. However, the distance between the center positions is greatly influenced by the shape of the candidate object or specific part. For example, if the shape of the region of the candidate object (or specific part) is L-shaped, the center position of that region may not be on the candidate object (or specific part). Therefore, in the processing example shown in Figure 9, the relative distance between the candidate object and the specific part is calculated based on the distance between each pixel of the candidate object and each pixel of the specific part.

[0075] In other words, the processor 31 of the inspection device 13 acquires information indicating the region of the object candidate T and the region of the specific part W detected in the captured image of the luggage M to be inspected (step S301). Here, the processor 31 acquires the image region of the object candidate T and the image region of the specific part W for which the correlation is calculated. However, if there are multiple object candidates or multiple specific parts in the captured image, the processor 31 calculates the correlation (relative distance) by performing the processing in steps S301 to S307 for all combinations of each object candidate and each specific part.

[0076] When information indicating the image region of a specific part W in the captured image is obtained, the processor 31 sets the total number of pixels forming the image of the specific part W as the number of loops corresponding to the specific part (step S302). Also, when information indicating the image region of a candidate object T in the captured image is obtained, the processor 31 sets the total number of pixels forming the image of the candidate object T as the number of loops corresponding to the candidate object (step S303).

[0077] The processor 31 calculates the distance between each pixel of the target candidate and each pixel of the specific unit, and stores the calculated distances between each pixel in the RAM 33 or storage unit 34 as distance information for calculating the relative distance between the specific unit W and the target candidate T (step S304). The processor 31 repeatedly performs the process of calculating the distance between each pixel of the specific unit for one pixel of the target candidate for the number of pixels in the specific unit (number of loops according to the specific unit) (step S305).

[0078] Once the calculation of the distance of each pixel in the specific part to one pixel of the target candidate is complete, the processor 31 executes a process to calculate the distance of each pixel in the specific part to the next pixel selected from the target candidate. The processor 31 repeatedly executes the process of calculating the distance of each pixel in the specific part to pixels selected sequentially from the target candidate for the total number of pixels in the target candidate (number of loops according to the target candidate) (step S306). Through the processes in steps S302 to S306 above, the processor 31 calculates the distance of all pixels in the specific part to all pixels in the target candidate.

[0079] Once the processor 31 has finished calculating the distance between each pixel of the target candidate T and each pixel of the specific part W, it calculates the relative distance as the relative relationship between the target candidate T and the specific part W based on the distances between each pixel stored in the RAM 33 or storage unit 34 (step S307). For example, the processor 31 may calculate the minimum distance from the distances calculated for each combination of each pixel of the target candidate T and each pixel of the specific part W as the relative relationship.

[0080] However, in reality, noise may be mistakenly detected as a specific part. Noise that is mistakenly detected as a specific part may be detected at a location close to the target object candidate. In such cases, if there is even a small amount of noise closer to the target object candidate than the actual specific part, the minimum distance to the target object candidate will be calculated as a value smaller than the distance to the actual specific part.

[0081] Therefore, the processor 31 may calculate the relative relationship (relative distance) between the target candidate and the specific part, and then, instead of taking the minimum value, take the percentile value. By calculating the correlation that indicates the distance between each pixel of the target candidate and each pixel of the specific part using percentile values, it becomes possible to robustly evaluate whether the target candidate is likely to be the specified object even if there is noise detected as the specific part.

[0082] In the example processing shown in Figure 9, the distance between all pixels in the specific part is calculated for all pixels in the target candidate. However, to reduce the computational load, it is not necessary to calculate the distance for all combinations. For example, the processor 31 may reduce the number of pixel combinations for which the distance is calculated by calculating the distance between combinations of pixels sampled from the target candidate and pixels sampled from the specific part. By reducing the number of pixel combinations for which the distance is calculated in the example processing shown in Figure 9, the computational load in the relative relationship calculation process can be reduced.

[0083] Furthermore, while the example of the correlation calculation process described above uses a 2D image as the processing target to calculate the distance between pixels, if the captured image is 3D data, the distance between each voxel in the target object candidate and each voxel in the specific area should be calculated. This makes it possible to calculate the relative distance as a relative relationship between the target object candidate and the specific area, even if the captured image is 3D data.

[0084] Next, the process of issuing an alarm for detection results based on the relative relationships of the inspection device 13 according to this embodiment will be described in detail. The processor 31 of the inspection device 13 performs an alerting process that notifies the detection result of a specific object based on its relative relationship via an output device. For example, the processor 31 of the inspection device 13 displays a display screen of the inspection result of the specific object on the display device 14, highlighting the image area suspected to be the specific object (the image area detected as the specific object) in the captured image of the luggage M.

[0085] Figure 10 shows an example of a display where the inspection device 13 displays the inspection result of a specific object in the package M on the display device 14 as part of the alarm processing. The example display shown in Figure 10 displays an image of luggage M captured by the camera 12, and highlights the image region of a candidate object (image region detected as a specific object) that has been determined to be a specific object based on its relative relationship with a specific part on the captured image.

[0086] Image regions suspected of containing a specific object can be highlighted in a way that makes them easily distinguishable from other image regions. For example, an image region suspected of containing a specific object may be highlighted by displaying it in a different color than other regions. Alternatively, an image region suspected of containing a specific object may be highlighted by enclosing it in a rectangle or ellipse.

[0087] Furthermore, the inspection results displayed on the display device 14 by the inspection device 13 in the alarm process may include not only the image area of ​​the object candidate that has been determined to be a specific object, but also information indicating the specific part, the object candidate, or the relative relationship.

[0088] Figure 11 shows an example of a display that shows the image region in the captured image that was determined to be suspected to be a specific object (detection result of the specific object), along with guidance indicating the object candidates detected by the object candidate detection process. The display example shown in Figure 11 shows how the image areas of potential objects other than those suspected of being specific objects are enclosed with elliptical guide lines so that inspectors can easily see them. Alternatively, the areas detected as potential objects may be enclosed in rectangles or other shapes, or they may be displayed in a specific color designated for potential object areas.

[0089] Figure 12 shows an example of a display that shows an image region in the captured image that is suspected to be a specific object, along with guidance indicating the specific part detected by the specific part detection process. The display example shown in Figure 12 shows an example where the image region detected as a specific part in the captured image is surrounded by a rectangular guide line so that the inspector can visually confirm it. Alternatively, the image region detected as a specific part may be displayed by surrounding it with an ellipse or by displaying it in a specific color set as the area of ​​the target object candidate.

[0090] Figure 13 shows an example of a display that shows the image region in the captured image that is suspected to be a specific object, along with information indicating the relative relationship (relative distance) of each candidate object to the specific part. In the display example shown in Figure 13, information (number of pixels) indicating the relative distance of each candidate object to the detected area is displayed. Alternatively, candidate objects and specific areas may be displayed in the captured image, and information such as density and effective atomic number in the image area of ​​the candidate objects and specific areas may be displayed side by side (or superimposed).

[0091] Figure 14 shows an example of a display that shows the detection results of a specific object and includes buttons to indicate the display of candidate objects and specific parts. In the example shown in Figure 14, buttons are displayed to indicate the display of all object candidates, to indicate the display of a specific part, and to indicate the selection of individual object candidates. When the button to indicate the display of object candidates is pressed by the operating device 15, the display device 14 displays images of all object candidates detected as object candidates, as shown in Figure 14.

[0092] Furthermore, when a button to indicate the display of a specific part is pressed via the operating device 15, the display device 14 displays an image of the area detected as a specific part. Also, when a button to select an individual object candidate is pressed via the operating device 15, the display device 14 displays an image of the selected object candidate along with information about that object candidate (for example, information indicating its physical properties, or information indicating its relative relationship to the specific part).

[0093] Furthermore, if the captured image is 3D data, the processor 31 of the inspection device 13 may display the captured image on the display device 14 in 3D. In this case as well, the processor 31 may display the area of ​​the object candidate detected as a specific object in a different color from other areas. Also, even when the captured image is displayed in 3D, the processor 31 may display the object candidate or specific part on the captured image in response to instructions from the inspector. Furthermore, when the captured image is displayed as a 3D image on the display device 14, the processor 31 may rotate, enlarge, reduce, or move the 3D captured image in response to operations by the inspector.

[0094] Furthermore, if the captured image is 3D data, the processor 31 may display a 2D image sliced ​​at a location (plane) specified by the inspector on the display device 14. Figure 15 schematically shows an example of displaying a binary image obtained by slicing a 3D captured image at a specified location. In the example shown in Figure 15, the inspection device 13 displays the captured image as a two-dimensional image obtained by slicing the three-dimensional image at a specified location on the display device 14, and further displays the area detected as a candidate object, a specific part, and a specific object on the captured image. The location where the three-dimensional image is sliced ​​may be a predetermined location set in advance, or it may be a location specified by the inspector using the operating device 15.

[0095] In the above description, an example of operation of the inspection system 1 in which the imaging device 12 and the inspection device 13 are configured separately was described. However, the processing of the inspection device 13 described above may also be configured to be performed by the imaging device 12. In other words, in the inspection system 1 described above, the imaging device 12 and the inspection device 13 may be realized as a single integrated device.

[0096] Furthermore, although the above example mainly described the operation of the inspection device 13, the information management system 100 shown in Figure 2 may also be configured such that the higher-level management device 101 collects information such as the results of various processing and inspection work performed by the inspection device 13. In other words, information such as the results of various processing and inspection work performed by the inspection device 13 may be transmitted from the inspection device 13 to the higher-level management device 101 and stored in the higher-level management device. This allows the higher-level management device 101 to analyze the information collected from the inspection devices 13 of the inspection system 1 located in various places and change the parameters in the various processing performed by each inspection device 13 according to the analysis results.

[0097] As a specific example, the higher-level control device may collect information from each inspection device indicating the parts or objects in which specific targets were detected during the actual inspection work, and set the parameters for the detection process of specific parts in each inspection device so that those parts or objects are detected as specific parts. Alternatively, the higher-level control device may calculate statistical quantities such as the probability (accuracy) that a candidate object detected as a specific target will be confirmed as a specific target in the actual inspection work from the information collected from each inspection device, and adjust the setting values ​​for determining a candidate object as a specific target based on its relative relationship with the specific part according to these statistical quantities.

[0098] As described above, the inspection device according to the embodiment acquires image data including an image captured by irradiating the object to be inspected with electromagnetic waves and physical property information of each part of the image. The inspection device detects the candidate object and the specific part in the captured image and notifies the candidate object that is suspected to be the specific object based on the relative relationship between the candidate object and the specific part.

[0099] As a result, according to the inspection device of this embodiment, it is possible to detect specific parts where specific objects are likely to be placed and to identify areas suspected of being specific objects based on the relative relationship between the specific parts and the candidate objects. As a result, it is possible to narrow down candidate objects that are difficult to distinguish from specific objects based solely on physical property values ​​obtained by irradiating the inspection object with electromagnetic waves, based on their relative relationship with the specific parts, and to detect areas suspected of being specific objects with high accuracy.

[0100] While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be carried out in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents. [Explanation of symbols]

[0101] 1...Inspection system, M...Luggage, 11...Conveyor, 12...Imaging device (X-ray CT scanner), 13...Inspection device, 14...Display device, 15...Operating device, 16...Speaker, 21...Imaging unit, 22...Processing unit, 23...Output unit, 31...Processor, 32...ROM, 33...RAM, 34...Storage unit, 35...Communication unit, 36...Display interface, 37...Operation interface, 38...Voice interface, 39...Image interface (Image acquisition unit), 41...Processor, 42...ROM, 43...RAM, 44...Storage unit, 45...Communication unit, 100...Information management system, 101...Higher-level management device.

Claims

1. An image acquisition unit acquires image data including an image taken by irradiating an object to be inspected with electromagnetic waves and physical property information indicating the physical properties of each part of the image, A processor that detects candidate objects which are candidates for a specific object in the object to be inspected based on the physical property information contained in the captured image data, detects a specific part which is a specific part or object in the object to be inspected, calculates the relative relationship between the candidate object and the specific part, and notifies the candidate object whose calculated relative relationship satisfies predetermined conditions, An inspection device having a

2. The processor calculates, as the relative relationship, whether the target candidate and the specific unit are adjacent or encompassing, and notifies the target candidate that is adjacent to or encompassing the specific unit. The inspection apparatus according to claim 1.

3. The processor calculates the relative distance between the target candidate and the specific unit as the relative relationship, and notifies the target candidate whose relative distance to the specific unit is less than or equal to a predetermined threshold. The inspection apparatus according to claim 1.

4. The processor calculates the distance between each component unit of the target candidate and each component unit of the specific part, and sets the minimum value of the calculated distance as the relative distance between the target candidate and the specific part. The inspection apparatus according to claim 3.

5. The processor calculates the distance between each component unit of the target candidate and each component unit of the specified part, and sets a predetermined percentile value for the calculated distance as the relative distance between the target candidate and the specified part. The inspection apparatus according to claim 4.

6. The processor causes the display device to display the location of a candidate object whose relative relationship with the specific unit satisfies the predetermined conditions, along with the captured image. The inspection apparatus according to claim 1.

7. The processor further causes the display device to display information indicating all the target object candidates detected from the captured image. The inspection apparatus according to claim 6.

8. The processor further causes the display device to display information indicating the specific part detected from the captured image. The inspection apparatus according to claim 6.

9. The aforementioned image is an X-ray image obtained by irradiating the object to be examined with X-rays. The inspection apparatus according to any one of claims 1 to 8.

10. The aforementioned captured image is three-dimensional data obtained by photographing the subject of examination using an X-ray CT scanner. The processor calculates the relative relationship between the target candidate and the specific part in the three-dimensional data as the captured image. The inspection apparatus according to claim 9.

11. In an inspection system having an imaging device and an inspection device, The aforementioned imaging device is An imaging unit that takes images by irradiating the object to be examined with electromagnetic waves, A processing unit that generates captured image data including the captured image and physical property information indicating the physical properties of each part of the captured image, It has an output unit that outputs the captured image data, The inspection device, An image acquisition unit that acquires captured image data output from the aforementioned imaging device, A processor that detects candidate objects which are candidates for a specific object in the object to be inspected based on physical property information contained in the captured image data, detects a specific part which is a specific part or object in the object to be inspected, calculates the relative relationship between the candidate object and the specific part, and notifies the candidate object whose calculated relative relationship satisfies predetermined conditions, Inspection system.

12. An inspection method for inspecting specific objects within an inspection target, The system acquires image data that includes an image taken by irradiating the object to be inspected with electromagnetic waves, and physical property information indicating the physical properties of each part of the image. Based on the physical property information contained in the aforementioned captured image data, a candidate object, which is a candidate for a specific object in the object to be inspected, is detected. The system detects a specific part or object within the subject of inspection, The relative relationship between the candidate object and the specific part is calculated, The system notifies the candidate objects whose relative relationships, calculated above, satisfy predetermined conditions. Testing method.

Images