Inspection device, inspection system, and program

Abstract

To provide an inspection device, an inspection system, and a program that can detect that a large amount of similar articles are included in inspection targets.SOLUTION: According to an embodiment, an inspection device has an image acquisition unit and a processor. The image acquisition unit acquires photographed image data including a photographed image obtained by photographing inspection targets. The processor detects attention areas in the photographed image to be candidates for articles in the inspection targets on the basis of the photographed image data, calculates the feature quantities of the detected attention areas, and when there are a predetermined number or more of attention areas in which the similarity of the calculated feature quantity is equal to or more than a predetermined value, notifies that a predetermined number or more of similar articles are present in the inspection targets.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] FIELD Embodiments of the present invention relate to an inspection device, an inspection system, and a program. [Background technology]

[0002] Conventionally, inspection sites such as customs inspect whether or not there are any items prohibited from import within luggage or other items subject to inspection. In recent years, in order to detect items intentionally concealed within luggage, inspection systems have been proposed that detect specific objects within luggage by analyzing transmission images (X-ray images) of the luggage subject to inspection taken using electromagnetic waves such as X-rays. Conventional inspection systems use the transmission images of the luggage taken using electromagnetic waves to determine whether or not a specific inspection target item that has been set in advance is present within the luggage.

[0003] However, conventional inspection systems require that specific items to be inspected, such as items that are prohibited from being imported, be preset. Furthermore, because conventional inspection systems detect items that are determined to be preset as items to be inspected, if the items detected as items to be inspected are not detected, even if a package contains a large amount of high-value-added items, the inspector will not be able to detect the package as an item to be opened and inspected. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2017-97853 Summary of the Invention [Problem to be solved by the invention]

[0005] The problem to be solved by the present invention is to provide an inspection device, an inspection system, and a program that can detect whether an inspection target contains a large number of similar items. [Means for solving the problem]

[0006] According to an embodiment, the inspection device includes an image acquisition unit and a processor. The image acquisition unit acquires captured image data including an image of an object to be inspected. The processor detects regions of interest in the captured image that are candidates for objects in the object to be inspected based on the captured image data, calculates feature amounts for each of the detected regions of interest, and notifies the inspection device that a predetermined number or more of similar objects are present within the object to be inspected if a predetermined number or more of the regions of interest have similarities in the calculated feature amounts that are equal to or greater than a predetermined value. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram schematically illustrating the overall configuration of an inspection system including an inspection device according to an embodiment. [Figure 2] FIG. 2 is a block diagram showing an example of the configuration of an information management system including an inspection device according to an embodiment. [Figure 3] FIG. 3 is a block diagram showing an example of the configuration of a control system in the imaging device and the inspection device of the inspection system according to the embodiment. [Figure 4] FIG. 4 is a block diagram showing an example of the configuration of a host management device in an information management system including an inspection device according to an embodiment. [Figure 5] FIG. 5 is a flowchart for explaining the overall flow of processing for detecting a bag containing a plurality of similar articles by the inspection device according to the embodiment. [Figure 6] FIG. 6 is a flowchart for explaining an example of the operation of a feature amount calculation process for calculating feature amounts of each region of interest in an image captured by the inspection device according to the embodiment. [Figure 7] FIG. 7 is a flowchart for explaining an example of the operation of a detection process for detecting a plurality of similar articles by the inspection device according to the embodiment. [Figure 8] FIG. 8 is a diagram schematically showing a captured image captured by a photographing device and supplied to the inspection device according to the embodiment. [Figure 9]FIG. 9 is a diagram schematically illustrating an example of feature amounts for each region of interest in an image captured by the inspection device according to the embodiment. [Figure 10] FIG. 10 is a flowchart for explaining a first modified example of the detection process for detecting a plurality of similar articles by the inspection device according to the embodiment. [Figure 11] FIG. 11 is a flowchart for explaining a second modified example of the detection process for detecting a plurality of similar articles by the inspection device according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments will be described with reference to the drawings. FIG. 1 is a diagram for schematically explaining an example of the configuration of an inspection system 1 including an inspection device 13 according to an embodiment. The inspection system 1 including the inspection device 13 according to the embodiment is a system for inspecting whether or not a specific object to be detected (specific object) is present in a piece of luggage to be inspected. The specific object may be, for example, a dangerous substance, a dangerous chemical, a drug whose handling is prohibited, or a substance whose carrying into or out of a specified area of ​​the country is prohibited.

[0009] In such an inspection system 1, the inspection device 13 according to the embodiment has a function of notifying an inspector that there are multiple similar items when the multiple items have similar features in the package. For example, the inspection device 13 according to the embodiment is expected to reduce the risk of illegally importing large quantities of high-value-added items by alerting an inspector by notifying an inspector that there are multiple similar items in the package.

[0010] 1, the inspection system 1 includes a conveyor 11, an imaging device 12, an inspection device 13, a display device 14, and an operation device 15. The inspection device 13 is communicatively connected to the imaging device 12, the display device 14, and the operation device 15.

[0011] The conveyor 11 is a device that transports the luggage M to be inspected. The conveyor 11 transports the luggage M to be inspected to a position (reading position) where an image is captured by the imaging device 12. For example, the conveyor 11 transports luggage M supplied by a worker. The conveyor 11 may also be configured to transport luggage M supplied by a robot arm or the like.

[0012] The imaging device 12 irradiates electromagnetic waves onto the baggage M to be inspected, thereby acquiring image data including an image of the object to be inspected and physical property information (e.g., density data, effective atomic number) indicating the physical properties of each part of the image. The imaging device 12 supplies the image data of the baggage M to the inspection device 13. The imaging device 12 may acquire two-dimensional image data as the image, or may acquire three-dimensional image data. The imaging device 12 may acquire image data that allows the inspection device 13 to detect the feature amounts of each item present in the baggage M.

[0013] The imaging device 12 is, for example, an X-ray CT imaging device. The X-ray CT imaging device, which is an example of the imaging device 12, obtains three-dimensional X-ray image data (3D data) as an imaging image by irradiating X-rays from around the baggage M conveyed by the conveyor 11. The X-ray CT imaging device, which is the imaging device 12, also obtains imaging image data including a three-dimensional X-ray image of the baggage M and physical property information indicating the physical properties of each constituent unit (pixel or voxel) that makes up the X-ray image. The X-ray CT imaging device, which is the imaging device 12, supplies the imaging image data obtained from the baggage M to the inspection device 13.

[0014] The inspection device 13 has various functions, such as a function to process the image of the baggage M captured by the imaging device 12 using electromagnetic waves. For example, the inspection device 13 has a function (receiving unit) of acquiring an image of the luggage M photographed by the photographing device 12 using electromagnetic waves, and a function (transmitting unit) of outputting information based on the information obtained by the inspection process described below using an output device (alert unit) such as a display device 14 or a speaker.

[0015] The inspection device 13 also has a function of detecting an area (area of ​​interest) that is a candidate (item candidate) presumed to be an item present in the baggage M from the captured image data acquired from the imaging device 12. The inspection device 13 detects an area of ​​interest that is a candidate item based on the physical property information (density, effective atomic number, etc.) of each pixel or voxel in the captured image (e.g., X-ray image captured by an X-ray CT imaging device) captured by the imaging device 12. Here, a pixel corresponds to a picture element, which is the smallest unit that makes up two-dimensional image data. A voxel is the smallest unit of data that makes up three-dimensional data and represents a value in regular lattice units. A voxel is a value that corresponds to a pixel in the two-dimensional image data.

[0016] As a specific example, a sliding window method that determines an area of ​​a specific size by shifting it at regular intervals can be used as a method for detecting an area of ​​interest that is a candidate item in a captured image of the luggage M. Alternatively, a method for detecting an area of ​​interest that is a candidate item can be used that determines an area of ​​interest using semantic segmentation that associates a label or category with each pixel in the captured image of the luggage M.

[0017] Furthermore, since it is costly to perform processing to detect an area of ​​interest for all areas in the photographed image data of the luggage M obtained from the photographing device 12, processing to eliminate areas where no objects exist, such as areas that are only air, may be performed as preprocessing for the processing to detect an area of ​​interest. For example, as preprocessing, feature points in the photographed image of the luggage M may be calculated, and areas with a large number of these feature points may be used as the target for detecting an area of ​​interest.

[0018] When the inspection device 13 detects areas of interest that are candidate items within the baggage M, it calculates feature amounts for each detected area of ​​interest based on the shape, physical properties, etc. of the item, and stores the calculated feature amounts for each area of ​​interest in memory. For example, the inspection device 13 may calculate feature amounts for each area of ​​interest by using a histogram of physical property information such as density data or effective atomic number contained in the captured image data obtained from the imaging device 12.

[0019] Furthermore, when the data obtained from the imaging device 12 is point cloud or mesh data, or when it can be converted using a method such as the marching cubes method, the inspection device 13 may calculate three-dimensional features such as PFH (Point Feature Histograms) or SHOT (Signature of Histograms of Orientations). Because these three-dimensional features are features for a certain point, they may be converted into features for the entire region of interest by averaging the features calculated within the region of interest or using a representative value. Alternatively, bag-of-features may be used to convert these features into features for the entire region of interest. Rotation-invariant features may be used to perform rotation-independent detection of these features.

[0020] Furthermore, the inspection device 13 calculates the similarity between the feature amounts between each of the attention areas in the image captured by the imaging device 12. Since the attention areas are areas estimated as areas where items exist (item candidates) within the luggage M, similarity between the attention areas means that the items in the attention areas are similar. The inspection device 13 determines whether there are a predetermined number or more of attention areas where the similarity is equal to or greater than a predetermined value. This allows the inspection device 13 to determine whether there are a predetermined number or more (plural) of similar items within the luggage M.

[0021] Furthermore, the inspection device 13 may be configured to detect a region of interest (a region containing an item similar to the specific item) having a feature quantity similar to the feature quantity of a predetermined specific item. This results in detecting an item similar to the predetermined specific item within the package M. In other words, the inspection device 13 has a function of detecting an item similar to the predetermined specific item within the package M based on the photographed image data of the package M.

[0022] The display device 14 is an output device for notifying the inspector of the inspection results or warnings, etc. The display device 14 displays a guide screen, etc., according to the control of the inspection device 13. The display device 14 displays a guide screen showing the results of the inspection process on the captured image of the baggage M, as a guide screen to be presented to the inspector. For example, the display device 14 displays an image that clearly shows the candidate object and the specific part in the image captured by the imaging device 12, which is generated by the inspection device 13. Furthermore, if there are a predetermined number or more of similar items in the baggage M, the inspection device 13 displays a warning screen on the display device 14 indicating that there are a predetermined number or more of similar items.

[0023] The operation device 15 generates an operation signal according to an operation input by an inspector (operator) and supplies the operation signal to the inspection device 13. The operation device 15 is composed of operation devices such as a keyboard and a pointing device. The operation device 15 may also be composed of a touch panel or the like provided on the display screen of the display device 14.

[0024] The inspection system 1 may also be provided with a speaker that outputs audio as a means of informing the inspector. The speaker is connected to the inspection device 13 and outputs audio to inform the inspector of audio guidance according to the results of the inspection process in the inspection device 13. For example, the inspection device 13 may output an alert through the speaker when a predetermined number or more of similar items are present in the baggage M.

[0025] Next, the configuration of the information management system 100 including the inspection system 1 according to the embodiment will be described. FIG. 2 is a diagram showing an example of the configuration of an information management system 100 including the inspection system 1 according to the embodiment. 2, the information management system 100 has a host management device 101 that is communicatively connected to the inspection devices 13 of the inspection system 1 installed at each inspection site. The host management device 101 functions as an information management device that collects data from the inspection devices in each inspection system 1 and supplies data to each inspection device.

[0026] The host management device 101 is configured, for example, by a computer such as a server device. The host management device 101 includes a storage device that stores information about the tests performed by each testing system 1. The host management device 101 may also include an interface that connects to a server device that stores information about the tests.

[0027] The host management device 101 acquires information from the inspection devices 13 in each inspection system 1. The host management device 101 saves and aggregates the information acquired from the inspection devices 13 in each inspection system 1. The host management device 101 supplies information to the inspection devices 13 in each inspection system 1. For example, the host management device 101 distributes setting values ​​used in inspection processing to the inspection devices 13 in each inspection system 1. The host management device 101 may also distribute update data for programs used by each inspection device 13 to execute inspection processing.

[0028] Next, the configuration of the control system for the imaging device 12 and the inspection device 13 in the inspection system 1 according to the embodiment will be described. FIG. 3 is a block diagram showing an example of the configuration of a control system for the imaging device 12 and the inspection device 13 in the inspection system 1 according to the embodiment. As shown in FIG. 3, the photographing device 12 includes an imaging unit 21, a processing unit 22, and an output unit . The imaging unit 21 captures an image of the luggage M to be inspected. The imaging unit 21 irradiates the luggage M with electromagnetic waves such as X-rays to capture a transmission image of the luggage M. For example, if the imaging device 12 is an X-ray CT imaging device, the imaging unit 21 acquires three-dimensional X-ray image data by irradiating X-rays onto the luggage M to be inspected that is being transported by the conveyor 11.

[0029] The processing unit 22 includes a processor and various memories, and performs various processes by the processor executing programs stored in the memories. The processing unit 22 processes the image captured by the imaging unit 21 by irradiating it with electromagnetic waves, thereby generating captured image data including the captured image and physical property information indicating the physical properties of the constituent units (pixels or voxels) that make up the captured image. For example, the processing unit 22 generates captured image data including three-dimensional X-ray image data captured by the imaging unit 21 by irradiating it with X-rays, and density data and effective atomic number as physical property information indicating the physical properties of each voxel that makes up the three-dimensional X-ray image.

[0030] The output unit 23 is an interface that outputs data such as captured image data. The output unit 23 has an interface corresponding to the image interface 39 of the inspection device 13, and outputs the captured image data to the inspection device 13. The output unit 23 may also be an input / output interface that includes an interface for inputting data such as control data from the inspection device 13 connected thereto.

[0031] As shown in FIG. 3, the inspection device 13 includes a processor 31, a ROM 32, a RAM 33, a memory unit 34, a communication unit 35, a display interface (I / F) 36, an operation interface (I / F) 37, and an image interface (I / F) 39.

[0032] The processor 31 executes arithmetic processing. The processor 31 is, for example, a CPU (Central Processing Unit). The processor 31 functions as a processing unit that executes various processes by using the RAM 33 to execute programs stored in the ROM 32 or the storage unit 34.

[0033] The ROM 32 is a read-only non-volatile memory. The ROM 32 stores program data, control data, etc. The RAM 33 is a volatile memory that functions as a working memory. The RAM 33 temporarily stores data.

[0034] The storage unit 34 is a rewritable non-volatile memory. The storage unit 34 is configured by a hard disk drive (HDD), a solid state drive (SSD), etc. The storage unit 34 stores information such as program data, setting values ​​as control data, and results of inspection processing.

[0035] The communication unit 35 is a communication interface for communicating with the host management device 101. The processor 31 communicates with the host management device 101 via the communication unit 35. The processor 31 transmits data such as processing results to the host management device 101 and receives data from the host management device 101 via the communication unit 35.

[0036] The display interface 36 is an interface for connecting to the display device 14 as an output device (notification unit). The display interface 36 may be any interface that corresponds to the interface provided in the display device 14. The processor 31 controls the display content to be displayed on the display device 14 via the display interface 36. When the inspection system 1 is provided with a speaker, the inspection device 13 may have an interface for connecting to the speaker as an output device (notification unit).

[0037] The operation interface 37 is an interface for connecting to the operation device 15. The operation interface 37 may be any interface that corresponds to the interface provided in the operation device 15. The processor 31 acquires information input by the operation device 15 via the operation interface 37.

[0038] The image interface 39 is an interface for connecting to the imaging device 12. The image interface 39 is an image acquisition unit for acquiring captured image data from the imaging device 12. The image interface 39 may be any interface that corresponds to an interface provided in the imaging device 12, such as an X-ray CT device. The processor 31 acquires, via the image interface 39, captured image data including an image (e.g., an X-ray image) captured by the imaging device 12 and physical property information in the captured image (e.g., density data and effective atomic number). The processor 31 may also control the imaging operation of the imaging device 12 on the package M via the image interface 39.

[0039] Next, the configuration of the host management device 101 in the information management system 100 including the inspection system 1 according to the embodiment will be described. FIG. 4 is a block diagram showing an example of the configuration of the upper management device 101 in the information management system 100 including the inspection system 1 according to the embodiment. The host management device 101 is an information management device that manages information on the entire inspection system 1. The host management device 101 is a computer that is communicatively connected to the inspection devices 13 of the inspection system 1 that are installed at each inspection site. The host management device 101 is configured by, for example, a server device.

[0040] In the configuration example shown in FIG. 4, the upper management device 101 includes a processor 41, a ROM 42, a RAM 43, a storage unit 44, and a communication unit 45. The processor 41 executes arithmetic processing. The processor 41 is, for example, a CPU (Central Processing Unit). The processor 41 functions as a processing unit that executes various processes by using the RAM 43 to execute programs stored in the ROM 42 or the storage unit 44.

[0041] The ROM 42 is a read-only non-volatile memory. The ROM 42 stores program data, control data, etc. The RAM 43 is a volatile memory that functions as a working memory. The RAM 43 temporarily stores data.

[0042] The storage unit 44 is a rewritable non-volatile memory. The storage unit 44 is configured by a hard disk drive (HDD), a solid state drive (SSD), etc. The storage unit 44 stores information such as program data, setting values ​​as control data, and data collected from each inspection device 13.

[0043] The communication unit 45 is a communication interface for communicating with the inspection device 13 in each inspection system 1. The processor 41 communicates with the inspection device 13 via the communication unit 45. The processor 41 receives data such as processing results from the inspection device 13 and transmits data to the inspection device 13 via the communication unit 45.

[0044] Next, a detection process in which the inspection device 13 detects the presence of a plurality of articles (a predetermined number or more) in the baggage M in the inspection process in the inspection system 1 according to the embodiment will be described. FIG. 5 is a flowchart for explaining the outline of the flow of the detection process for detecting a plurality of similar articles by the inspection device 13 according to the embodiment.

[0045] In an inspection system 1 configured as shown in Fig. 1, luggage M to be inspected is sequentially placed on a conveyor 11. The conveyor 11 transports the placed luggage M to a position where an image is captured by an imaging device 12. An imaging unit 21 of the imaging device 12 irradiates electromagnetic waves onto the luggage M being transported by the conveyor 11, thereby acquiring a photographed image showing the contents of the luggage M.

[0046] The captured image may be image data showing the state inside the luggage M. For example, the X-ray CT device of the imaging device 12 irradiates X-rays onto the luggage M being transported to the imaging position, thereby acquiring three-dimensional data showing the state inside the luggage M as a captured image. In addition, the acquired physical property information indicates the physical properties (density and effective atomic number) of each pixel or voxel constituting the captured image.

[0047] The processing unit 22 of the photographing device 12 generates photographed image data including a photographed image of the baggage M photographed by the imaging unit 21 and physical property information indicating the physical properties of each pixel or voxel in the photographed image. The photographed image data may be two-dimensional image data, three-dimensional image data, or a group of multiple two-dimensional image data obtained by slicing the three-dimensional image data along a specific axis. When the processing unit 22 of the photographing device acquires the photographed image data of the baggage M, it outputs the photographed image data to the inspection device 13 via the output unit 23.

[0048] The inspection device 13 acquires photographed image data of the luggage M from the photographing device 12 via the image interface 39 (step S11). The processor 31 executes processing to acquire the photographed image data as processing by the receiving unit described above. The inspection device 13 acquires photographed image data including a photographed image as a transmitted image of the inside of the luggage M using electromagnetic waves and physical property information indicating the physical properties of each part (pixel or voxel) of the photographed image. In the following explanation, for simplicity, it is assumed that the photographed image is two-dimensional image data.

[0049] The processor 31 of the inspection device 13 determines an area of ​​interest that is a candidate for an item within the baggage M from the captured image data acquired from the imaging device 12 (step S12). The processor 31 detects an area of ​​interest that is a candidate for an item within the captured image of the baggage M. Here, the processor 31 may calculate feature points within the entire captured image of the baggage M and estimate an area where no item exists from the distribution of feature points, etc. In this case, the processor 31 may detect an area of ​​interest by excluding areas where no item exists in the captured image. This can speed up the detection of an area of ​​interest that is a candidate for an item within the captured image of the baggage M.

[0050] When the processor 31 of the inspection device 13 detects a region of interest in the captured image of the baggage M, it performs a process of calculating a feature amount for each region of interest (step S13). For example, the processor 31 calculates density data or effective atomic number in the region of interest as the feature amount for each region of interest. Furthermore, if the captured image is a three-dimensional image, the processor 31 may calculate the feature amount for the entire region of interest based on the three-dimensional feature amount of the region of interest calculated by PFH or SHOT as described above.

[0051] FIG. 6 is a flowchart for explaining an example of a feature amount calculation process performed by the inspection device 13 to calculate a feature amount for each region of interest. The processor 31 of the inspection device 13 sequentially designates each region of interest detected from the captured image data and calculates the feature amount. Fig. 6 shows that the processes of steps S22-33 are executed for all regions of interest extracted from the captured image.

[0052] That is, the processor 31 designates the ith region of interest Ai by incrementing a variable i, which has an initial value of 1 (step S21). After designating the region of interest Ai, the processor 31 calculates the feature amount Fi of the region of interest Ai in the captured image data (step S22). After calculating the feature amount Fi of the region of interest Ai, the processor 31 stores the calculated feature amount Fi in the RAM 33 or the storage unit 34 (step S23). If there is a region of interest for which the feature amount has not been calculated (step S24), the processor 31 returns to S21 and executes the above-described process again, and if there is no region of interest for which the feature amount has not been calculated, the feature amount calculation process ends.

[0053] The processor 31 of the inspection device 13 calculates the feature amounts of each attention area in the captured image of the baggage M, and then performs a process of detecting multiple similar items by detecting similar items (similar items) based on the similarity between the feature amounts of each attention area (similarity between the feature amounts) (step S14). For example, the processor 31 calculates the similarity between the feature amounts of each attention area, and detects similar items in the inspection target by determining that similar items exist in attention areas with similar feature amounts. When similar items are detected, the processor 31 determines whether a predetermined number or more of similar items exist based on whether the number of similar items is equal to or greater than a predetermined number. Furthermore, if the processor 31 determines that a predetermined number or more of similar items exist, it notifies the user that multiple similar items exist in the baggage to be inspected.

[0054] FIG. 7 is a flowchart for explaining an example of the operation of a detection process for detecting a plurality of similar articles (a detection process for a plurality of similar articles) performed by the inspection device 13 according to the embodiment. The processor 31 of the inspection device 13 sequentially designates the regions of interest detected from the captured image data and calculates similar regions of interest (features). Fig. 7 shows that the processes of steps S32-34 are executed for all regions of interest extracted from the captured image.

[0055] First, processor 31 specifies one area of ​​interest to be subjected to similarity determination (step S31). Processor 31 sequentially specifies each area of ​​interest extracted from the captured image of package M. For example, processor 31 defines a variable i with the number of areas of interest in the captured image set to a maximum value I, and specifies the i-th area of ​​interest by incrementing variable i (i=i+1).

[0056] When processor 31 designates an area of ​​interest Ai to be subjected to similarity determination, it calculates the similarity between the feature amount Fi of area of ​​interest Ai and the feature amounts of other areas of interest (step S32). The similarity between areas of interest can be determined by calculating the distance between the vectors of the feature amounts of the areas of interest to be subjected to similarity calculation or by a correlation coefficient. After calculating the similarity between the feature amount Fi of area of ​​interest Ai and other areas of interest, processor 31 determines whether the number of areas of interest whose similarity to the designated area of ​​interest Ai (feature amount Fi) is equal to or greater than a predetermined value is equal to or greater than a predetermined number (step S33).

[0057] If there are a predetermined number or more of attention areas whose similarity to attention area Ai is equal to or greater than a predetermined value (YES in step S33), processor 31 notifies that there are a predetermined number or more (plurality) of similar items in package M (step S34). As a result, attention areas whose similarity is equal to or greater than a predetermined value are regarded as similar items (similar items), and it is possible to notify that there are a predetermined number or more of these similar items in package M. Here, if the type or name of an item in attention area Ai can be estimated from features or the like, it is also possible to notify the type or name of an item determined to be present in a predetermined number or more (plurality).

[0058] If there are not a predetermined number of areas of interest with a similarity greater than or equal to a predetermined value (step S33, NO), or if there are areas of interest with an uncalculated similarity after notifying that there are a predetermined number of similar items (step S35), processor 31 returns to S31 and executes the above-described processing again, and if there are no areas of interest with an uncalculated similarity, ends the processing of detecting multiple similar items.

[0059] Here, the above-mentioned process of detecting a plurality of similar articles will be explained using a specific example. FIG. 8 is a diagram showing an example in which an attention area is detected in a captured image captured by the imaging device 12 and supplied to the inspection device 13. In FIG. In the example of the captured image shown in Figure 8, it is assumed that attention areas A1 to A6 are extracted from the captured image of luggage M captured by the photographing device 12, and similar items (e.g., wristwatches) are present in attention areas A1, A2, A4 and A6, while similar items are not present in attention areas A3 and A5.

[0060] FIG. 9 is a diagram schematically illustrating an example of feature amounts for each attention area in the photographed image illustrated in FIG. In the example shown in FIG. 9, feature quantities F1 to F6 are calculated for the regions of interest A1 to A6 shown in FIG. 8. In the example shown in FIG. 9, information indicating physical properties such as density data and effective atomic number is calculated as a histogram distribution as the feature quantity for each region of interest. In the example shown in FIG. 9, feature quantities F1, F2, F4, and F6 for regions of interest A1, A2, A4, and A6 have similar values ​​(similarity is equal to or greater than a predetermined value). These represent feature quantities for similar items present in regions of interest A3 and A5, respectively.

[0061] 9, the feature F3 for the region of interest A3 has a value that is significantly different from the feature values ​​of the other regions of interest (the degree of similarity is less than a predetermined value), and the feature F5 for the region of interest A5 also has a value that is significantly different from the feature values ​​of the other regions of interest (the degree of similarity is less than a predetermined value). This indicates that no item similar to the item present in the region of interest A3 exists in the other regions of interest (regions of interest other than the region of interest A3), and no item similar to the item present in the region of interest A5 exists in the other regions of interest (regions of interest greater than or equal to the region of interest A5).

[0062] 9 is obtained, the inspection device 13 can determine that similar items with similarities equal to or greater than a predetermined value are present in the attention areas A1, A2, A4, and A6. In this case, the inspection device 13 can notify the inspector that there are four similar items in the baggage M. The inspection device 13 can also notify the inspector of the areas (attention areas) in the baggage M where the similar items are present.

[0063] According to the above processing, the inspection device of the embodiment calculates the features of multiple attention areas divided into the image of the luggage to be inspected, detects attention areas where similar items exist based on the similarity between the features of each attention area, and if a predetermined number or more of attention areas where similar items exist are detected, notifies the user that a predetermined number or more of similar items exist.

[0064] This allows the inspection device according to the embodiment to detect and report the presence of a predetermined number or more of similar items in a package without the need to register the items (products) to be reported in advance. As a result, the inspection device according to the embodiment can report packages containing large quantities of high-value-added products such as watches that are similar items, making it easier to detect smuggled items.

[0065] Next, a first modified example of the above-described process by the inspection device 13 according to the embodiment will be described. The first modification is a process for detecting similar items in the inspection device 13, in which an inspector is notified when there are a predetermined number or more of items similar to a preset specific item. The preset specific item is an item that is subject to inspection, such as a large quantity of imports, and the characteristic quantities of the specific item are stored in a memory unit or the like.

[0066] FIG. 10 is a flowchart for explaining a first modified example of the process of detecting similar articles by the inspection device 13. The process shown in Fig. 10 is, for example, a process performed as the similar item detection process shown in Fig. 6. The processor 31 of the inspection device 13 detects items similar to the specific item by calculating the similarity between each of the predetermined specific items and each attention area detected from the captured image data. Fig. 10 shows that the process of steps S42-44 is performed for all of the predetermined specific items.

[0067] First, processor 31 specifies a preset specific item to be subjected to similarity determination (step S41). If multiple specific items are set, processor 31 specifies the specific items in order. For example, processor 31 defines a variable j whose maximum value is J, and specifies the jth specific item by incrementing variable j (j=j+1) (step S41).

[0068] When processor 31 designates a specific item to be subjected to similarity determination, processor 31 calculates the similarity between the feature amount of the specific item and the feature amount of each attention area in the captured image (step S42). After calculating the similarity between the feature amount of the specific item and the feature amount of all attention areas in the captured image, processor 31 determines whether the number of attention areas whose similarity to the specific item is equal to or greater than a predetermined value is equal to or greater than a predetermined number (step S43).

[0069] If there are a predetermined number or more of attention areas whose similarity to the specific item is equal to or greater than a predetermined value (YES in step S43), the processor 31 notifies the user that there are a predetermined number or more (plural) of items similar to the specific item in the package M (step S44). This allows the inspection device 13 to notify the user that there are a predetermined number or more of items (items similar to the specific item) whose similarity to the specific item is equal to or greater than a predetermined value in the package M. Here, if the type or name of the specific item is set, the type or name of the specific item that is determined to be present in a predetermined number or more (plural) may also be notified.

[0070] If there are not a predetermined number of attention areas with a similarity to the specific item that is equal to or greater than a predetermined value (step S43, NO), or if there are specific items for which similarity determination has not been performed after notifying that there are a predetermined number of similar items, that is, if J=j is not true (step S45), processor 31 returns to step S41 and executes the above-described process again. Also, if there are not specific items for which similarity determination has not been performed, that is, if J=j is true (step S45), processor 31 ends the process of detecting similar items to the specific item.

[0071] The processor 31 of the inspection device 13 may perform the process shown in Fig. 10 described above in combination with the process shown in Fig. 7 described above. For example, the processor 31 may perform the process shown in Fig. 7 after performing the process shown in Fig. 10. This enables the inspection device 13 to notify the inspector when a first predetermined number or more of a preset specific item is present, or when a second predetermined number or more of a non-preset similar item is present (which may be the same as the first predetermined number, or may be a different number).

[0072] According to the first variant example described above, in addition to the above-described processing, the inspection device of the embodiment pre-registers the features of a specific item that is to be found as an item similar to a predetermined number or more, detects an area of ​​interest with features similar to the features of the specific item, and issues an alert when a predetermined number or more of items similar to the specific item are present. As a result, the inspection device according to the first variant of the embodiment can detect and notify not only similar items having similar features in the area of ​​interest, but also the presence of a predetermined number or more of items having similar features to the features of a specific item registered in advance.

[0073] Next, a second modification of the above-described process by the inspection device 13 according to the embodiment will be described. The second modification involves performing a process of notifying an inspector when there are a predetermined number or more of articles similar to a preset specific article as a similar article detection process in the inspection device 13. The preset specific article is an article that is subject to inspection, such as a large quantity of imports, and the characteristic quantities of the specific article are stored in a memory unit or the like.

[0074] FIG. 11 is a flowchart for explaining a second modified example of the process of detecting similar articles by the inspection device 13 according to the embodiment. The process shown in Fig. 11 is, for example, a process performed as the similar article detection process shown in Fig. 6. The processor 31 of the inspection device 13 sequentially specifies the areas of interest detected from the captured image data and calculates similar areas of interest (features), similar to the operation example shown in Fig. 7. Fig. 11 shows that the process of steps S52-55 is performed for each of all the predetermined specific articles.

[0075] First, processor 31 specifies one area of ​​interest to be subjected to similarity determination (step S51). Processor 31 sequentially specifies each area of ​​interest extracted from the captured image of package M. For example, processor 31 defines a variable i with the number of areas of interest in the captured image set to a maximum value I, and specifies the i-th area of ​​interest by incrementing variable i (i=i+1).

[0076] When the processor 31 designates the attention area Ai to be subjected to similarity determination, the processor 31 determines whether the similarity between the feature amount of the pre-set excluded item and the feature amount of the designated attention area Ai is equal to or greater than a predetermined value (step S52). That is, the processor 31 determines whether the item in the designated attention area Ai is an excluded item (similar to an excluded item).

[0077] If the feature amount of the excluded item and the feature amount of the attention area Ai are greater than or equal to a predetermined value (step S52, YES), the processor 31 proceeds to processing the next attention area without calculating the similarity between the attention area Ai and other attention areas, that is, without determining whether there are any items in other attention areas that are similar to the item in the attention area Ai.

[0078] If the feature amount of the excluded item and the feature amount of the attention area Ai are not equal to or greater than a predetermined value (step S52, NO), processor 31 calculates the similarity between the feature amount Fi of the attention area Ai and the feature amounts of the other attention areas (step S53). After calculating the similarity between the feature amount Fi of the attention area Ai and the other attention areas, processor 31 determines whether the number of attention areas whose similarity to the specified attention area Ai (feature amount Fi) is equal to or greater than a predetermined value is equal to or greater than a predetermined number (step S54).

[0079] If there are a predetermined number or more of attention areas whose similarity to attention area Ai is equal to or greater than a predetermined value (YES in step S54), processor 31 notifies that there are a predetermined number or more (plurality) of similar items in package M (step S55). This makes it possible to determine attention areas other than excluded items whose similarity is equal to or greater than a predetermined value as similar items (similar items), and to notify that there are a predetermined number or more of these similar items in package M. Here, if the type or name of an item in attention area Ai can be estimated from features or the like, the type or name of the item determined to be present in a predetermined number or more (plurality) may also be notified.

[0080] If there are not a predetermined number of attention areas with a similarity equal to or greater than a predetermined value (step S54, NO), or after notifying that there are a predetermined number of similar items, or if there are attention areas with an uncalculated similarity (step S56), processor 31 returns to step S51 and executes the above-described process again. Also, if there are no attention areas with an uncalculated similarity (step S56), processor 31 ends the similar item detection process.

[0081] The processor 31 of the inspection device 13 may combine the processing of Fig. 11 described above with the processing of Fig. 10 described above. For example, the processor 31 may perform the processing shown in Fig. 11 after performing the processing shown in Fig. 10. This enables the inspection device 13 to notify the inspector when a first predetermined number or more of preset specific items are present, and when a second predetermined number or more of similar items that are not preset, excluding excluded items, are present (which may be the same as the first predetermined number, or may be a different number).

[0082] According to the second variant example described above, in addition to the above-mentioned processing, the inspection device of the embodiment pre-registers the features of specific items (excluded items) to be excluded from targets for detection as similar items, excludes attention areas with features similar to the features of the excluded items from the targets for determining similar items, and issues an alert if a predetermined number or more of similar items exist outside of attention areas whose similarity to the features of the excluded items is equal to or greater than a predetermined value.

[0083] As a result, the inspection device according to the second modification of the embodiment can detect a predetermined number or more of similar items by excluding the area of ​​interest where similar items to the excluded item are located, thereby speeding up the process of detecting similar items. As a result, specific items that are not to be detected among the items that are likely to detect many similar items can be registered as excluded items, thereby realizing efficient processing.

[0084] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, and are also included in the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0085] 1...inspection system, M...baggage, 11...conveyor, 12...imaging device (X-ray CT device), 13...inspection device, 14...display device, 15...operation device, 21...imaging unit, 22...processing unit, 23...output unit, 31...processor, 32...ROM, 33...RAM, 34...memory unit, 35...communication unit, 36...display interface, 37...operation interface, 39...image interface (image acquisition unit).

Claims

1. an image acquisition unit that acquires captured image data including a captured image of an inspection object; a processor that detects areas of interest in the captured image that are candidates for items in the inspection object based on the captured image data, calculates feature amounts for each of the detected areas of interest, and notifies that a predetermined number or more of similar items exist within the inspection object when there are a predetermined number or more areas of interest whose calculated feature amounts have similarities equal to or greater than a predetermined value; An inspection device having the above structure.

2. the image acquisition unit acquires photographed image data including a photographed image as a transmitted image through the baggage, the photographed image being obtained by irradiating the baggage as the inspection target with an electromagnetic wave; The inspection device according to claim 1 .

3. the image acquisition unit acquires photographed image data including the photographed image and physical property information indicating physical properties of each portion of the photographed image; the processor detects an area of ​​interest that is a candidate for an item in the baggage to be inspected based on physical property information included in the captured image data; The inspection device according to claim 2 .

4. the processor calculates a feature amount for each region of interest in the captured image based on physical property information included in the captured image data; The inspection device according to claim 3 .

5. the processor calculates a histogram indicating physical property information of each region of interest in the captured image as a feature amount of each region of interest; The inspection device according to claim 4.

6. the processor notifies the user that a predetermined number or more of articles similar to the specific article are present within the inspection target when a predetermined number or more of attention areas have feature amounts whose similarity to the feature amount of the specific article is equal to or greater than a predetermined value. The inspection device according to claim 1 .

7. the processor excludes attention areas having feature amounts whose similarity to the feature amounts of the specific article is equal to or greater than a predetermined value, and determines whether there are a predetermined number or more attention areas whose similarity is equal to or greater than a predetermined value. The inspection device according to claim 1 .

8. An inspection system including an imaging device and an inspection device, The imaging device is an imaging unit that irradiates an electromagnetic wave onto an object to be inspected and captures an image; a processing unit that generates captured image data including a captured image captured by the imaging unit and physical property information indicating physical properties of each part of the captured image; an output unit that transmits the photographed image data generated by the processing unit to the inspection device; The inspection device includes: an image acquisition unit that acquires photographed image data of the inspection object from the photographing device; a processor that detects areas of interest that are candidates for articles in the inspection object based on physical property information included in the photographed image data, calculates feature amounts for each of the detected areas of interest, and notifies that a predetermined number or more of similar articles exist within the inspection object when there are a predetermined number or more areas of interest where the similarity of the calculated feature amounts is a predetermined value or more; An inspection system having:

9. The processor of the inspection device Acquiring photographed image data including a photographed image of the inspection object by an image acquisition unit; detecting an area of ​​interest in the photographed image that is a candidate for an item in the inspection target based on the photographed image data; Calculating a feature amount of each region of interest in the captured image; If there are a predetermined number or more attention areas in the captured image where the similarity of the calculated feature amount is equal to or greater than a predetermined value, it is notified that there are a predetermined number or more similar articles within the inspection object. A program that makes it happen.

Images