X-ray inspection system

Abstract

To accurately estimate the type of a cargo.SOLUTION: An X-ray inspection system 100 includes: an imaging unit 400 which images a cargo to be inspected with an X-ray and outputs an X-ray image of the subject; a first image resizing unit 582 which resizes the X-ray image on the basis of the dimension of the cargo included in custom declaration information; a first image division unit 584 which divides the resized X-ray image into multiple partial X-ray images having a predetermined image size; and an estimation unit 586 which estimates the type of the cargo included in at least one of the multiple partial X-ray images, using a learning model built on the basis of a learning X-ray image related to the cargo.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present disclosure relates to an X-ray inspection system.

Background Art

[0002] In customs and the like, an X-ray inspection apparatus is used to image an X-ray image of goods through a container in order to inspect the goods in the container undergoing import / export inspection. In this X-ray inspection apparatus, the captured X-ray image is input into a pre-trained learning model, and the type of goods in the container can be obtained as an output.

[0003] For example, Patent Document 1 discloses a technique of imaging an X-ray image of goods through a container using an X-ray inspection apparatus, inputting the captured X-ray image into a classifier, and obtaining the type of goods stored in the container as an output.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] By the way, since the container for storing goods is very large compared to the goods, the X-ray image of the container is inevitably much larger than the X-ray image of the goods alone. When training a learning model using such an X-ray image of the container, it is difficult to improve the training accuracy because the X-ray image is very large. Therefore, a countermeasure has been taken to improve the training accuracy of the learning model by, for example, dividing the X-ray image as described in Patent Document 1 and inputting a smaller X-ray image into the learning model.

[0006] However, while dividing an X-ray image into smaller segments improves the detection accuracy of smaller cargo, it lowers the detection accuracy of larger cargo. Conversely, dividing an X-ray image into larger segments improves the detection accuracy of larger cargo, but lowers the detection accuracy of smaller cargo. Thus, in containers containing cargo of various sizes, it has been difficult to accurately estimate the type of cargo inside a container from X-ray images of the container itself.

[0007] Therefore, the purpose of this disclosure is to provide an X-ray inspection system capable of accurately estimating the type of cargo. [Means for solving the problem]

[0008] To solve the above problems, the X-ray inspection system of this disclosure is An imaging unit that images the cargo to be inspected using X-rays and outputs an X-ray image of the cargo to be inspected, A first image resizing unit that changes the size of the X-ray image based on the size of the cargo included in the customs declaration information, A first image division unit divides the X-ray image whose size has been changed into a plurality of partial X-ray images having a predetermined image size, An estimation unit that estimates the type of cargo depicted in at least one of the plurality of partial X-ray images, using a learning model constructed based on training X-ray images of the cargo; It is equipped with.

[0009] The predetermined image size may be the size of the training image used to generate the learning model.

[0010] A customs database where customs declaration information including information on the type and size of one or more types of the aforementioned goods is stored, An X-ray image database where training X-ray images obtained by imaging cargo that is the subject of training using X-rays are stored, A second image resizing unit that changes the size of the learning X-ray image based on the size of the cargo included in the customs declaration information, A second image division unit divides the resized learning X-ray image into a plurality of partial learning X-ray images having a predetermined image size, A learning unit constructs a learning model of the cargo to be learned based on the aforementioned partial learning X-ray images, It may be provided.

[0011] The system further comprises an association unit that associates the type of cargo to be learned, as depicted in the learning X-ray image, with the learning X-ray image, based on the customs declaration information and user input. The aforementioned learning unit, The X-ray images used for partial learning may be input to the learning model, and the parameters of the learning model may be modified so that the type of cargo to be learned, output from the learning model, matches the type of cargo associated with the X-ray images used for partial learning.

[0012] If the size of the goods included in the customs declaration information is greater than a predetermined first standard value, the image resizing unit may reduce the size of the X-ray image.

[0013] If the size of the cargo included in the customs declaration information is less than a predetermined second standard value, the image resizing unit may enlarge the size of the X-ray image by adding a margin area around the X-ray image. [Effects of the Invention]

[0014] According to this disclosure, the type of cargo can be accurately estimated. [Brief explanation of the drawing]

[0015] [Figure 1] Figure 1 is a schematic diagram showing an X-ray inspection apparatus according to the first embodiment of the present disclosure. [Figure 2] Figure 2 is a block diagram showing the configuration of the image processing apparatus according to the same embodiment. [Figure 3]FIG. 3 is a functional block diagram for explaining the functions of the inference module and the learning module according to the embodiment. [Figure 4] FIG. 4 is an explanatory diagram showing image processing for reducing and dividing a learning X-ray image according to the embodiment. [Figure 5] FIG. 5 is an explanatory diagram showing image processing for enlarging and dividing a learning X-ray image according to the embodiment. [Figure 6] FIG. 6 is an explanatory diagram showing image processing for dividing a learning X-ray image at the same magnification according to the embodiment. [Figure 7] FIG. 7 is a schematic configuration diagram showing an example of a learning model according to the embodiment. [Figure 8] FIG. 8 is a flowchart showing an X-ray inspection method according to the embodiment. MODE FOR CARRYING OUT THE INVENTION

[0016] Embodiments of the present disclosure will be described below with reference to the accompanying drawings. The dimensions, materials, and other specific numerical values shown in the embodiments are merely examples for ease of understanding and do not limit the present disclosure unless otherwise specified. In the present specification and drawings, elements having substantially the same functions and configurations are denoted by the same reference numerals to omit redundant descriptions, and elements not directly related to the present disclosure are not shown.

[0017] [1. Overall Configuration of X-ray Inspection System] First, referring to FIG. 1, the overall configuration of an X-ray inspection system 100 according to the first embodiment of the present disclosure will be described. FIG. 1 is a schematic diagram showing the X-ray inspection system 100 according to the present embodiment.

[0018] [[ID=三十一]] As shown in Figure 1, the X-ray inspection system 100 according to this embodiment is a system for capturing an X-ray image of the object to be inspected and inspecting the contents of the object based on the X-ray image. The X-ray inspection system 100 may be, for example, a large-scale X-ray inspection system installed at customs offices, etc., for inspecting goods undergoing import and export inspection. However, the X-ray inspection system 100 is not limited to this example, and can be applied to various other inspection systems as long as they are systems that use X-ray images to inspect the object to be inspected.

[0019] The X-ray inspection system 100 comprises a customs database 200, an X-ray image database 300, an X-ray imaging device 400, and an image processing device 500. The customs database 200, the X-ray image database 300, and the image processing device 500 are interconnected and can communicate with each other via a wired or wireless network or wiring. The X-ray imaging device 400 and the image processing device 500 are also interconnected and can communicate with each other via a wired or wireless network or wiring.

[0020] The customs database 200 stores customs declaration information, including cargo information, for goods 412 contained in container 410 that are subject to inspection. The customs declaration information is created based on the customs declaration form. The customs declaration information includes cargo information regarding the type and size of one or more types of goods 412 subject to inspection. Specifically, the customs declaration information includes cargo information such as image information, name, quantity, and size of goods 412.

[0021] The X-ray image database 300 stores training X-ray images for building a learning model used to estimate the type of cargo 412 inside the container 410 that is to be inspected. Training X-ray images can be obtained by setting the cargo 412 contained in the container 410 as the training target and imaging the cargo 412 set as the training target with the X-ray imaging device 400. Specifically, training X-ray images can be obtained by placing the cargo 412 as the training target inside the container 410 and imaging the cargo 412 inside the container 410 using X-rays 420 with the X-ray imaging device 400. The training X-ray images may also be stored in the customs database 200 as image information of cargo 412 in the customs declaration information and associated with the customs declaration information. Furthermore, although the X-ray image database 300 in this embodiment stores training X-ray images captured by the X-ray imaging device 400, it is not limited to this and may also store training X-ray images acquired from outside the X-ray inspection system 100. For example, the X-ray image database 300 may acquire and store training X-ray images captured by other X-ray imaging devices located outside the X-ray inspection system 100.

[0022] The X-ray imaging device 400 is an example of the imaging unit of this disclosure. The X-ray imaging device 400 images the object to be inspected using X-rays 420. The X-ray imaging device 400 images the cargo 412 contained in the container 410, which is the object to be inspected, using X-rays 420 to generate an X-ray image of the cargo 412. The X-ray image may be, for example, a grayscale image that represents the object to be imaged (e.g., cargo 412) in shades of black and white. The X-ray imaging device 400 outputs the X-ray image of the object to be inspected obtained by the imaging process to the image processing device 500. The X-ray imaging device 400 also outputs a training X-ray image of the cargo 412, which is the learning target, to the image processing device 500.

[0023] The image processing device 500 is an information processing device that performs various image processing and various calculation processing on X-ray images. As will be described in detail later, the image processing device 500 can construct a learning model for estimating the type of cargo 412 to be inspected that is captured in the X-ray image, based on the customs declaration information stored in the customs database 200 and the training X-ray images stored in the X-ray image database 300. Furthermore, the image processing device 500 can estimate the type of cargo 412 to be inspected from the X-ray image based on the constructed learning model.

[0024] The X-ray inspection system 100, which includes the customs database 200, X-ray image database 300, X-ray imaging device 400, and image processing device 500, can accurately estimate the type of cargo 412 to be inspected contained in container 410. Therefore, users (e.g., customs inspectors) can easily determine the exact type of cargo 412 in container 410 to be inspected using the X-ray inspection system 100. Thus, users do not need to identify cargo 412 by visually observing X-ray images, significantly reducing the burden on the user's inspection work.

[0025] [2. Regarding cargo] Next, with reference to Figure 1, the cargo 412 that is the subject of inspection by the X-ray inspection system 100 according to this embodiment will be described.

[0026] As shown in Figure 1, the cargo 412 is the object of inspection by the X-ray inspection system 100. The cargo 412 can be of various types and sizes, and at least one cargo 412 is contained in the container 410. The cargo 412 is set as the object of learning when constructing a learning model. In this embodiment, the cargo 412 is contained inside the transport container 410. The cargo 412 can be any item to be transported, for example, the cargo 412 may be various items such as products, parts, materials, machinery and equipment, or food products. In addition, although the cargo 412 to be inspected in this embodiment is stored contained in the container 410, it is not limited to this example. The cargo 412 may be stored exposed without being contained in a transport container such as the container 410.

[0027] [3. About X-ray imaging equipment] Next, with reference to Figure 1, the configuration of the X-ray imaging apparatus 400 according to this embodiment will be described.

[0028] As shown in Figure 1, the X-ray imaging device 400 (corresponding to the "imaging unit") is a device for imaging cargo 412 contained in a container 410 and generating an X-ray image or training X-ray image of the cargo 412. The X-ray imaging device 400 comprises an X-ray irradiation unit 430, an X-ray detection unit 440, an X-ray image generation unit 450, a controller 460, and a transport device (not shown) for transporting the container 410.

[0029] The X-ray irradiation unit 430 and the X-ray detection unit 440 are arranged opposite each other with a predetermined distance between them. In the example shown in Figure 1, the X-ray irradiation unit 430 and the X-ray detection unit 440 are arranged opposite each other in the vertical direction. The X-ray irradiation unit 430 includes an X-ray source that generates X-rays 420 and an X-ray irradiation device for the X-rays 420. The X-ray irradiation unit 430 irradiates the X-ray detection unit 440 with X-rays 420 in a predetermined direction (for example, the Z direction in Figure 1). The X-rays 420 pass through the object to be imaged and head towards the X-ray detection unit 440. The X-ray detection unit 440 is composed of, for example, a line sensor for X-ray detection. The X-ray detection unit 440 detects the X-rays 420 that have been irradiated from the X-ray irradiation unit 430 and passed through the object to be imaged. The X-ray detection unit 440 outputs an imaging signal representing the detected X-rays 420.

[0030] The X-ray image generation unit 450 generates an X-ray image of the object to be imaged based on the imaging signal of the X-rays 420 detected by the X-ray detection unit 440. The X-ray image generation unit 450 includes, for example, an AD conversion element and an image memory. The AD conversion element performs AD conversion on the imaging signal output from the X-ray detection unit 440. The image memory stores the AD-converted imaging signal pixel by pixel.

[0031] The transport device consists of a belt conveyor and the like, and transports the container 410 containing the cargo 412 in a predetermined direction (for example, the X direction in Figure 1). The controller 460 controls each part of the X-ray imaging device 400, including the X-ray irradiation unit 430, the X-ray detection unit 440, the X-ray image generation unit 450, and the transport device.

[0032] With the X-ray imaging device 400 configured in this way, X-rays 420 can be scanned over the cargo 412 to be inspected, and an X-ray image of the cargo 412 can be acquired. The X-ray imaging operation by such an X-ray imaging device 400 will be described below.

[0033] The X-ray imaging device 400 operates the transport device with the container 410 placed on it. As a result, the container 410 moves relative to the X-ray irradiation unit 430 and the X-ray detection unit 440 in the scanning direction of the X-rays 420 (X direction in Figure 1), passing between the X-ray irradiation unit 430 and the X-ray detection unit 440. At this time, the X-ray irradiation unit 430 irradiates the container 410, which is the object to be imaged, with X-rays 420. The X-ray detection unit 440 detects the X-rays 420 that have passed through the container 410 and the cargo 412 inside it from the X-ray irradiation unit 430. The X-ray image generation unit 450 generates an X-ray image of the cargo 412 based on the X-rays 420 detected by the X-ray detection unit 440.

[0034] In this embodiment, an X-ray image is generated of the cargo 412 inside the container 410, taken from above. This X-ray image is a two-dimensional image of the cargo 412 projected onto the XY plane, and corresponds to an image of the cargo 412 viewed through from above. However, the imaging direction of the X-ray image is not limited to the vertical direction. For example, the X-ray image may be an image of the cargo 412 taken from below (Z direction). Alternatively, the X-ray image may be a side view (Y direction) image of the cargo 412 taken from the right or left side. Alternatively, the X-ray image may be a front view (X direction) image of the cargo 412 taken from the front or back. Furthermore, multiple X-ray images of the cargo 412 taken from multiple directions may be generated.

[0035] Furthermore, the X-ray imaging device 400 generates an X-ray image of the cargo 412 to be inspected in the manner described above. The X-ray imaging device 400 outputs the generated X-ray image to the image processing device 500. In the case where cargo 412 is a training object, a training X-ray image is generated in the same manner as when it is an inspection object, and the generated training X-ray image is output to the image processing device 500.

[0036] Here, we will describe an example of the specifications of an X-ray image captured by the X-ray imaging apparatus 400 according to this embodiment.

[0037] The X-ray image according to this embodiment is, for example, a grayscale image that represents the imaged object being inspected using shades of black and white. A grayscale image does not contain color information, but only brightness information. The grayscale image may be, for example, an image in which the brightness value (pixel value) of one pixel is represented by 8 bits. The gradation of this 8-bit grayscale image is 256 levels (brightness value: 0 to 255). A brightness value of 0 represents black, and a brightness value of 255 represents white. By using a typical 256-level grayscale image, the characteristic image processing of this embodiment can be suitably realized. However, the gradation of the X-ray image may be other than 256 levels.

[0038] [4. Configuration of the image processing device] Next, the configuration of the image processing apparatus 500 according to this embodiment will be described with reference to Figure 2. Figure 2 is a block diagram showing the configuration of the image processing apparatus 500 according to this embodiment.

[0039] [4.1. Hardware configuration of the image processing unit] First, the hardware configuration of the image processing device 500 according to this embodiment will be described. Figure 2 shows an example of the hardware configuration of the image processing device 500 according to this embodiment.

[0040] As shown in Figure 2, the image processing device 500 includes a processor 510, memory 520, storage 530, communication device 540, input device 550, output device 560, and bus 570.

[0041] The processor 510 is an arithmetic processing unit installed in a computer. The processor 510 may consist of, for example, a CPU (Central Processing Unit), but it may also consist of other microprocessors. Furthermore, the processor 510 may consist of one or more processors. The processor 510 executes programs stored in the memory 520 or other storage media. This enables the execution of various processes in the image processing device 500.

[0042] The program is a computer program that includes instructions to be executed by a computer. The program may be provided to the image processing device 500, for example, by distribution from an external device via a communication network. Alternatively, the program may be provided to the image processing device 500 via a non-transitory computer-readable medium. By installing the program in the image processing device 500, the image processing device 500 becomes capable of realizing various functions defined by the program.

[0043] Memory 520 is a storage medium that stores programs and various other data. Memory 520 includes, for example, RAM (Random Access Memory) and ROM (Read Only Memory). ROM is a non-volatile memory that stores programs used by the processor 510 and data necessary to run those programs. RAM is a volatile memory that temporarily stores data such as variables, arithmetic parameters, and calculation results used in processes executed by the processor 510. Programs stored in ROM are read into RAM and executed by the processor 510, such as the CPU.

[0044] The storage device 530 is a storage device for storing various types of information and data. The storage device 530 includes, for example, a recording medium such as semiconductor memory, a hard disk, or an optical disc, and a drive for reading data from or writing data to the recording medium. The storage device 530 can store a larger amount of data than the memory device 520. The storage device 530 may be internal storage built into the image processing device 500, or it may be external storage connected via the external input / output terminals of the image processing device 500. Alternatively, the storage device 530 may be online storage connected via a communication network.

[0045] The communication device 540 is a device for communicating with an external device connected to the image processing device 500 by wire or wireless connection. The communication device 540 establishes a communication connection with the external device according to a predetermined protocol and transmits and receives various information and data with the external device.

[0046] The input device 550 is a device used by a user to input information into the image processing device 500. The input device 550 includes, for example, a touch sensor, keyboard, keypad, mouse, remote controller, button, switch, or dial. The input device 550 may also include an input device for voice input, such as a microphone or a voice recognition module. The input device 550 may also include a remote control module that receives user input from a remote device for remotely operating the image processing device 500. When the input device 550 receives an input operation from the user, it transmits an input signal corresponding to the input operation to the processor 510.

[0047] The output device 560 is a device for outputting information and data to the outside of the image processing device 500. The output device 560 includes a display device that displays information such as text, graphics, and images, and an audio output device that outputs sound. The display device comprises a display screen and an image display module. The display may be, for example, a liquid crystal display (LCD), a plasma display (PDP), an organic light-emitting diode (OLED), or a cathode ray tube (CRT). The display device may also be a touch panel with a touch sensor on the display screen. The audio output device comprises a speaker and an audio output module.

[0048] Bus 570 interconnects the processor 510, memory 520, storage 530, communication device 540, input device 550, and output device 560. This allows various types of information and data to be transmitted and received between these devices.

[0049] [4.2. Functional Configuration of Image Processing Devices] Next, the functional configuration of the image processing apparatus 500 according to this embodiment will be described with reference to Figures 2 to 7.

[0050] As shown in Figure 2, the image processing device 500 comprises an inference module 580 and a learning module 590. As shown in Figure 3, the inference module 580 comprises a first image resizing unit 582, a first image splitting unit 584, and an estimation unit 586. The learning module 590 comprises an association unit 592, a second image resizing unit 594, a second image splitting unit 596, and a learning unit 598. The processor 510 of the image processing device 500 performs calculations based on a program, thereby realizing each of these functional units of the image processing device 500.

[0051] The image processing device 500 of this embodiment functions in two main parts: a learning part and an inference part. Below, the learning part will be described first, followed by the inference part.

[0052] (Learning section) The learning phase involves building a learning model that is used to estimate the type of cargo 412 based on customs declaration information and training X-ray images.

[0053] Figure 3 is a functional block diagram illustrating the functions of the inference module 580 and the learning module 590 according to this embodiment. As shown in Figure 3, first, the association unit 592 acquires a learning X-ray image from the X-ray image database 300. As described above, the learning X-ray image is, for example, an X-ray image taken by the X-ray imaging device 400 that shows the cargo 412, which is the target of learning, inside the container 410, and is an X-ray image that shows a known cargo 412.

[0054] The type of cargo 412 captured in the training X-ray image may be known by the user (e.g., an inspector at customs, etc.) when the X-ray imaging device 400 is used for imaging, or cargo information such as the location, type, and size of cargo 412 may be associated with and stored in the training X-ray image.

[0055] Furthermore, the association unit 592 obtains customs declaration information from the customs database 200. At this time, the association unit 592 obtains customs declaration information that includes cargo information about the cargo 412 inside the container 410 when the training X-ray image was captured. The association unit 592 then extracts cargo information regarding the type and size of the cargo 412 from the customs declaration information. The association unit 592 may also obtain training X-ray images and customs declaration information from other X-ray image databases and other customs databases located outside the X-ray inspection system 100.

[0056] Next, the association unit 592 displays the acquired training X-ray images and customs declaration information on the display device of the output device 560. At this time, the user checks the training X-ray images and customs declaration information displayed on the display device and understands the position, type, and size of each cargo 412 to be studied that is captured in the training X-ray images.

[0057] The user uses the input device 550 to add cargo information to the training X-ray image, including the location, type, and size of each cargo 412 that they have identified as the target of training. For example, the user can add annotations such as bounding boxes to the location of each cargo 412 that appears in the training X-ray image.

[0058] The association unit 592 processes cargo information regarding the location, type, and size of each cargo 412 to be learned, which has been added by user input, to the training X-ray image. In this way, the association unit 592 processes cargo information regarding the location, type, and size of each cargo 412 that is the target of learning and is included in the training X-ray image, based on the training X-ray image, customs declaration information, and user input, to associate the training X-ray image with the training X-ray image. The training X-ray image with associated cargo information is stored in the storage 530. However, the association unit 592 may output the training X-ray image with associated cargo information to the second image resizing unit 594.

[0059] Next, the second image resizing unit 594 acquires a training X-ray image associated with cargo information from the association unit 592 or the storage 530. The second image resizing unit 594 also acquires customs declaration information from the association unit 592. Alternatively, the second image resizing unit 594 may acquire customs declaration information from the customs database 200.

[0060] The second image resizing unit 594 resizes the training X-ray image based on the size of each cargo 412 included in the customs declaration information. Specifically, the second image resizing unit 594 resizes the training X-ray image if the size of each cargo 412 included in the customs declaration information is greater than a predetermined first reference value or less than a predetermined second reference value. The predetermined second reference value is a value smaller than the predetermined first reference value. For example, the predetermined second reference value is half the value of the predetermined first reference value. However, it is not limited to this, and the predetermined second reference value may be equal to the predetermined first reference value.

[0061] In detail, the second image resizing unit 594 reduces the size of the training X-ray image if the size of each cargo 412 included in the customs declaration information is greater than a predetermined first standard value. Conversely, the second image resizing unit 594 enlarges the size of the training X-ray image if the size of each cargo 412 included in the customs declaration information is less than a predetermined second standard value.

[0062] Figure 4 is an explanatory diagram showing the image processing for reducing and dividing the training X-ray image according to this embodiment. In Figure 4, the first cargo 412A is captured in the training X-ray image 600A. Here, the size of the first cargo 412A is assumed to be larger than a predetermined first reference value.

[0063] First, the second image resizing unit 594 determines whether the size of the first cargo 412A included in the customs declaration information is greater than a predetermined first reference value. Next, if the second image resizing unit 594 determines that the size of the first cargo 412A is greater than a predetermined first reference value, it generates a reduced X-ray image 700 by reducing the size of the training X-ray image 600A, as shown in Figure 4.

[0064] Here, the predetermined first reference value is the size of cargo 412 such that the entire cargo 412 is captured within a predetermined image size in the X-ray image of cargo 412 or the training X-ray image of cargo 412. The predetermined image size is the size of the training image used to construct the training model described later, and is, for example, an image size with 224px × 224px pixels. In other words, the predetermined first reference value is, for example, the size of cargo 412 such that the entire cargo 412 is captured within an image size with 224px × 224px pixels.

[0065] In Figure 4, a predetermined image size is represented by an image size frame S indicated by a dashed line. As shown in Figure 4, the size of the first cargo 412A in the training X-ray image 600A is larger than the image size frame S representing the predetermined image size, and the entirety of the first cargo 412A does not fit within the frame of the image size frame S.

[0066] The second image resizing unit 594, if it determines that the size of the first cargo 412A is larger than a predetermined first reference value, calculates a reduction ratio so that the size of the first cargo 412A becomes the predetermined first reference value, multiplies the calculated reduction ratio by the training X-ray image 600 to generate a reduced X-ray image 700.

[0067] Furthermore, the learning X-ray image 600A is associated with cargo information regarding the position, type, and size of each cargo 412 that is the subject of learning, by the association unit 592. The second image size changing unit 594 may determine whether the size of the first cargo 412A is larger than a predetermined image size based on the cargo information regarding the size of the first cargo 412A associated with the learning X-ray image 600A.

[0068] In Figure 4, the cargo size frame 610A, which represents the size of the first cargo 412A associated with the training X-ray image 600A, is shown by a dashed line. As shown in Figure 4, the size of the cargo size frame 610A is larger than the image size frame S, which represents a predetermined image size.

[0069] The second image resizing unit 594, if it determines that the size of the cargo size frame 610A is larger than a predetermined image size, calculates a reduction ratio so that the size of the cargo size frame 610A becomes the predetermined image size, and multiplies the calculated reduction ratio by the training X-ray image 600A to generate a reduced X-ray image 700.

[0070] In this way, the second image resizing unit 594 determines whether the size of each cargo 412, which is the learning target and is captured in the learning X-ray image, is larger than a predetermined first reference value or a predetermined image size. If the second image resizing unit 594 determines that the size of the first cargo 412A captured in the learning X-ray image 600A is larger than a predetermined first reference value or a predetermined image size, it reduces the size of the learning X-ray image 600A and generates a reduced X-ray image 700. This makes it possible to fit the first cargo 412A captured in the learning X-ray image 600A within the predetermined image size.

[0071] Figure 5 is an explanatory diagram showing the image processing for enlarging and dividing the training X-ray image according to this embodiment. In Figure 5, the second cargo 412B is visible in the training X-ray image 600B. Here, the size of the second cargo 412B is assumed to be smaller than a predetermined second reference value.

[0072] First, the second image resizing unit 594 determines whether the size of the second cargo 412B included in the customs declaration information is less than a predetermined second standard value. Next, if the second image resizing unit 594 determines that the size of the second cargo 412B is less than a predetermined second standard value, it generates an enlarged X-ray image 800 by enlarging the size of the training X-ray image 600B, as shown in Figure 5.

[0073] Furthermore, the learning X-ray image 600B is associated with cargo information regarding the position, type, and size of each cargo 412 that is the subject of learning, by the association unit 592. The second image size changing unit 594 may determine whether the size of the second cargo 412B is less than a predetermined image size based on the cargo information regarding the size of the second cargo 412B associated with the learning X-ray image 600B.

[0074] In Figure 5, the cargo size frame 610B, which represents the size of the second cargo 412B associated with the training X-ray image 600B, is shown by a dashed line. As shown in Figure 5, the size of the cargo size frame 610B is smaller than the image size frame S, which represents a predetermined image size.

[0075] Here, if it is determined that the size of the cargo size frame 610B is less than the predetermined image size, it is also possible to calculate a magnification factor so that the size of the cargo size frame 610B becomes the predetermined image size, and multiply the calculated magnification factor by the training X-ray image 600B to generate an enlarged X-ray image. However, in that case, the image will become coarse and blurry due to the image enlargement process, which may reduce the detection accuracy of the training model described later.

[0076] Therefore, as shown in Figure 5, the second image resizing unit 594 generates an enlarged X-ray image 800 by providing a margin area 810 around the learning X-ray image 600B. The width of this margin area 810 is set to a width such that the image size frame S, which represents a predetermined image size, fits within the enlarged X-ray image 800 when the center of the second cargo 412B in the learning X-ray image 600B is aligned with the center of the image size frame S.

[0077] In this way, the second image resizing unit 594 determines whether the size of each cargo 412, which is the learning target and is captured in the learning X-ray image, is less than a predetermined second reference value or a predetermined image size. If the second image resizing unit 594 determines that the size of the second cargo 412B captured in the learning X-ray image 600B is less than a predetermined second reference value or a predetermined image size, it generates an enlarged X-ray image 800 by providing a margin area 810. This makes it possible to generate an enlarged X-ray image 800 without performing image enlargement processing that multiplies the learning X-ray image 600B by an enlargement magnification factor.

[0078] The second image resizing unit 594 outputs the reduced X-ray image 700 or the enlarged X-ray image 800, thus reduced in size, to the second image splitting unit 596. If the size of each cargo 412 to be studied is less than or equal to a predetermined first reference value and greater than or equal to a second reference value, the second image resizing unit 594 outputs a full-size training X-ray image to the second image splitting unit 596 without changing the size of the training X-ray image. Alternatively, if the size of each cargo 412 to be studied is a predetermined image size, the second image resizing unit 594 outputs a full-size training X-ray image to the second image splitting unit 596 without changing the size of the training X-ray image.

[0079] The second image splitting unit 596 divides the training X-ray image acquired from the second image resizing unit 594 into a plurality of partial training X-ray images having a predetermined image size. Specifically, returning to Figure 4, the second image splitting unit 596 divides the reduced-size X-ray image 700 into a plurality of partial training X-ray images 710 having a predetermined image size. These partial training X-ray images 710 have a size equal to the image size frame S having a predetermined image size, and the entirety of the first cargo 412A captured in the partial training X-ray images 710 is contained within the partial training X-ray images 710.

[0080] Returning to Figure 5, the second image division unit 596 divides the enlarged X-ray image 800 into a plurality of partial learning X-ray images 820 having a predetermined image size. Each of these partial learning X-ray images 820 has the same size as the image size frame S, and the entirety of the second cargo 412B captured in the partial learning X-ray image 820 is contained within the partial learning X-ray image 820.

[0081] Figure 6 is an explanatory diagram showing the image processing for dividing the training X-ray image into equal-size sections according to this embodiment. In Figure 6, the third cargo 412C is visible in the training X-ray image 600C. Here, the size of the third cargo 412C is assumed to be less than or equal to a predetermined first reference value and greater than or equal to a predetermined second reference value. Alternatively, the size of the third cargo 412C is assumed to be equal to a predetermined image size.

[0082] In Figure 6, the cargo size frame 610C, which represents the size of the third cargo 412C associated with the training X-ray image 600C, is shown by a dashed line. As shown in Figure 6, the size of the cargo size frame 610C is equal to the image size frame S, which represents a predetermined image size.

[0083] As shown in Figure 6, the second image division unit 596 divides the 1:1 learning X-ray image 600C into a plurality of partial learning X-ray images 620 having a predetermined image size. Each of these partial learning X-ray images 620 has the same size as the image size frame S, and the entirety of the third cargo 412C captured in the partial learning X-ray image 620 is contained within the partial learning X-ray image 620.

[0084] The second image splitting unit 596 divides the training X-ray image into a plurality of partial training X-ray images having a predetermined image size, while shifting the splitting positions within the training X-ray image by a predetermined amount. Parts of the plurality of partial training X-ray images may overlap. The second image splitting unit 596 outputs the plurality of partial training X-ray images to the learning unit 598. The second image splitting unit 596 may also extract feature quantities for each cargo 412 within the training X-ray image. The second image splitting unit 596 may also split the training X-ray image into partial training X-ray images of a predetermined image size from which the feature quantities can be obtained. Image processing such as horizontal inversion, vertical inversion, rotation, and contrast processing is performed on the partial training X-ray images output from the second image splitting unit 596 by an image processing unit (not shown). The learning unit 598 receives the partial training X-ray images processed by the image processing unit. However, the image processing by the image processing unit may be performed before the image splitting by the second image splitting unit 596.

[0085] The learning unit 598 acquires multiple X-ray images for partial learning from the second image segmentation unit 596 and constructs a learning model based on the multiple X-ray images for partial learning. The learning model is, for example, a deep learning model in image analysis. The learning model has, for example, a neural network (NN). The neural network is a convolutional neural network (CNN) trained using supervised learning. However, a neural network other than a convolutional neural network may be used. Also, a learning model other than a neural network may be used.

[0086] A learning model is constructed, for example, by a combination of the structure of a neural network and parameters representing the strength of the connections between each neuron. Each connection between neurons has a single parameter coefficient. Each parameter is adjustable. The internal state of the learning model is represented by a set of numerical values ​​called internal variables, which are combinations of the neural network structure and the parameters between each neuron.

[0087] Figure 7 is a schematic diagram showing an example of a learning model according to this embodiment. As shown in Figure 7, the learning model receives multiple partial learning X-ray images as input, and each neuron N1, N2, N3, N4, N5, N M-1 , N M Through this process, a value representing the type of each cargo item (412) is output as output data.

[0088] The learning model is constructed based on training X-ray images to which cargo information regarding the position, type, and size of each cargo 412 is associated by the association unit 592. The learning model is then input to the learning model by the second image segmentation unit 596, which divides the X-ray images into partial training images. At this point, the values ​​of the learning model's internal variables are set so that the error between the type of each cargo 412 as output data of the learning model and the type of each cargo 412 associated with the training X-ray images is minimized.

[0089] In this way, the learning unit 598 inputs the partial learning X-ray image into the learning model and modifies the parameters of the learning model so that the type of cargo 412 to be learned output from the learning model matches the type of cargo 412 associated with the partial learning X-ray image. In this way, the learning model is constructed based on the partial learning X-ray image. Since the size of the partial learning X-ray image used to construct the learning model is formed to be the same regardless of the size of the cargo 412, the detection accuracy of the learning model can be improved. In addition, even if a large cargo 412 is captured in the partial learning X-ray image, the size reduction process is performed so that the entire cargo 412 is captured, thereby improving the detection accuracy of the learning model. Furthermore, since the partial learning X-ray image is not subjected to image enlargement processing due to the magnification factor, the blurring and coarseness of the image caused by image enlargement processing is suppressed, and the detection accuracy of the learning model can be improved.

[0090] (Inference part) The inference part uses the learned model constructed in the learning part to estimate the type of cargo 412 that is visible in the X-ray image of the object being inspected.

[0091] First, the first image resizing unit 582 acquires an X-ray image of the cargo 412 to be inspected inside the container 410, which is captured by the X-ray imaging device 400 shown in Figure 1. The first image resizing unit 582 also acquires customs declaration information from the customs database 200. Then, the first image resizing unit 582 extracts cargo information regarding the type and size of the cargo 412 from the customs declaration information.

[0092] The first image resizing unit 582, similar to the learning part described above, resizes the X-ray image based on the size of the cargo 412 included in the customs declaration information. Specifically, if the size of the cargo 412 to be inspected included in the customs declaration information is larger than a predetermined first reference value or a predetermined image size, the first image resizing unit 582 reduces the size of the X-ray image. If the size of the cargo to be inspected included in the customs declaration information is less than a predetermined second reference value or a predetermined image size, the first image resizing unit 582 enlarges the size of the X-ray image by adding a margin area 810 around the X-ray image as shown in Figure 4. The process of resizing the X-ray image by the first image resizing unit 582 is the same as the process of resizing the learning X-ray image by the second image resizing unit 594 described above, so a detailed explanation is omitted. The X-ray image resized by the first image resizing unit 582 or the original X-ray image is output to the first image splitting unit 584.

[0093] The first image splitting unit 584, similar to the learning part described above, splits an X-ray image of the same size or an X-ray image with a changed size into a plurality of partial X-ray images having a predetermined image size, as shown in Figures 4 to 6. The process of splitting the X-ray image into partial X-ray images by the first image splitting unit 584 is the same as the process of splitting the learning X-ray image into partial learning X-ray images by the second image splitting unit 596 described above, so a detailed explanation is omitted. The partial X-ray images split by the first image splitting unit 584 are output to the estimation unit 586. Image processing such as horizontal inversion, vertical inversion, rotation, and contrast processing is performed on the partial X-ray images output from the first image splitting unit 584 by an image processing unit (not shown). The partial X-ray images processed by the image processing unit are input to the estimation unit 586. However, the image processing by the image processing unit may be performed before the image splitting by the first image splitting unit 584.

[0094] The estimation unit 586 uses a learning model constructed in the learning part based on the training X-ray images to estimate the type of cargo depicted in the divided partial X-ray images. In other words, the estimation unit 586 uses a learning model constructed based on the training X-ray images of cargo 412 to estimate the type of cargo 412 depicted in at least one of the multiple partial X-ray images. Specifically, the estimation unit 586 inputs the partial X-ray images into the learning model constructed by the learning unit 598. At this time, the learning model classifies each pixel of the partial X-ray image and outputs a value representing the type of cargo 412 as output data. The size of the partial X-ray images input to the learning model is formed to be the same size regardless of the size of the cargo 412. Furthermore, since the size of the partial X-ray images is the same as the size of the partial training X-ray images used when constructing the learning model, the estimation accuracy of the type of cargo 412, which is the output of the learning model, can be improved. Furthermore, in partial X-ray images, even if a large cargo 412 is visible, the size is reduced so that the entire cargo 412 is captured, thereby improving the accuracy of the learned model's output for estimating the type of cargo 412. In addition, since partial X-ray images are not subjected to image magnification processing, the blurring and coarseness that can occur due to image magnification processing is suppressed, further improving the accuracy of the learned model's output for estimating the type of cargo 412.

[0095] [5. X-ray examination method] Next, with reference to Figure 8, the X-ray inspection method using the X-ray inspection system 100 according to this embodiment will be described in detail. Figure 8 is a flowchart of the X-ray inspection method according to this embodiment.

[0096] As shown in Figure 8, in the learning part, the association unit 592 acquires X-ray images for learning from the X-ray image database 300 (S10). The association unit 592 also acquires customs declaration information from the customs database 200 (S20). Then, based on user input, the association unit 592 associates cargo information regarding the location, type, and size of each cargo 412 to be learned, as contained in the customs declaration information, with the X-ray images for learning (S30).

[0097] Next, the second image resizing unit 594 resizes the training X-ray image based on the size of the cargo 412 included in the customs declaration information (S40). This resizing process has been explained in detail above, so a detailed explanation is omitted here.

[0098] The second image splitting unit 596 splits the resized training X-ray image into a plurality of partial training X-ray images having predetermined image sizes (S50). This image splitting process has been explained in detail above, so a detailed explanation is omitted here.

[0099] Then, the learning unit 598 constructs a learning model based on the X-ray images used for partial learning (S60). The process of constructing this learning model has been explained in detail above, so a detailed explanation is omitted here.

[0100] Next, in the inference part, the first image resizing unit 582 acquires an X-ray image of the cargo 412 to be inspected, which is captured by the X-ray imaging device 400 (S70). The first image resizing unit 582 also acquires customs declaration information from the customs database 200 (S80).

[0101] The first image resizing unit 582 resizes the X-ray image based on the size of the cargo 412 included in the customs declaration information (S90). This resizing process is the same as the resizing process in the learning part and has been explained in detail above, so a detailed explanation is omitted here. Note that "X-ray image for learning" in the resizing process in the learning part can be read as "X-ray image".

[0102] The first image splitting unit 584 splits the resized X-ray image into a plurality of partial X-ray images having predetermined image sizes (S100). This image splitting process is the same as the image splitting process in the learning part and has been explained in detail above, so a detailed explanation is omitted here. Note that the "X-ray image for partial learning" in the image splitting process in the learning part can be read as "partial X-ray image".

[0103] The estimation unit 586 uses the learning model constructed by the learning unit 598 to estimate the type of cargo 412 shown in the divided partial X-ray images (S110). This estimation process has been explained in detail above, so a detailed explanation is omitted here.

[0104] [6. Summary] The X-ray inspection system 100 according to this embodiment has been described in detail above. According to this embodiment, the size of the X-ray image is changed based on the size of the cargo 412 to be inspected included in the customs declaration information, and the resized X-ray image is divided into a plurality of partial X-ray images having a predetermined image size. Then, the type of cargo 412 to be inspected is estimated by inputting the plurality of partial X-ray images into a learning model.

[0105] This allows the partial X-ray image input to the learning model to include the entire cargo 412, even when cargo 412 of various sizes are contained within the container 410. Furthermore, regardless of the size of the cargo 412, the partial X-ray image input to the learning model can be made the same size as the partial training X-ray image used to construct the learning model. As a result, the accuracy of the estimation of the cargo type 412, which is the output of the learning model, can be improved, and the type of cargo 412 inside the container 410 can be accurately estimated from the X-ray image of the container 410.

[0106] Furthermore, according to this embodiment, based on customs declaration information and user input, the type of each cargo to be trained that is depicted in the training X-ray image is associated with the training X-ray image. In addition, the size of the training X-ray image is changed based on the size of the cargo 412 to be trained included in the customs declaration information, and the resized training X-ray image is divided into a plurality of partial training X-ray images having predetermined image sizes. Then, a training model is constructed based on the plurality of partial training X-ray images. At this time, the training model is constructed by changing the parameters of the training model so that the cargo type, which is the output when the partial training X-ray image is input to the training model, matches the type associated with the partial training X-ray image.

[0107] This allows the X-ray images used for partial training input to the training model to include the entire cargo 412, even when cargo 412 of various sizes are contained within the container 410. Furthermore, the X-ray images used for partial training input to the training model can be made the same size regardless of the size of the cargo 412. As a result, the detection accuracy of the training model can be improved.

[0108] Furthermore, according to this embodiment, if the size of the cargo included in the customs declaration information is larger than a predetermined first standard value or a predetermined image size, the size of the X-ray image is reduced. As a result, even if a large cargo 412 is visible in the partial X-ray image, the size reduction process is performed so that the entire cargo 412 is visible, thereby improving the accuracy of the estimation of the type of cargo 412, which is the output of the learning model.

[0109] Furthermore, according to this embodiment, if the size of the cargo included in the customs declaration information is less than a predetermined second standard value or less than a predetermined image size, the size of the X-ray image is enlarged by adding a margin area around the X-ray image. As a result, partial X-ray images are not subjected to image enlargement processing due to the magnification factor, which suppresses the image becoming coarse and blurry due to image enlargement processing and improves the estimation accuracy of the cargo type 412, which is the output of the learning model.

[0110] While embodiments of this disclosure have been described above with reference to the attached drawings, it goes without saying that this disclosure is not limited to such embodiments. It will be obvious to those skilled in the art that various modifications or alterations can be conceived within the scope of the claims, and these will naturally also fall within the technical scope of this disclosure.

[0111] The series of processes performed by each device, such as the image processing apparatus 500 according to this embodiment, may be implemented using software, hardware, or a combination of software and hardware. The program constituting the software may be pre-stored in, for example, a non-transitory storage medium provided inside or outside each device. The program may then be read from, for example, a non-transitory storage medium (e.g., ROM) to a temporary storage medium (e.g., RAM) and executed by a processor such as a CPU.

[0112] It is possible to create programs to implement each of the above-mentioned devices and install them on the computers of each device. The processor executes the programs stored in memory, thereby carrying out the processing of each of the above-mentioned functions. At this time, the program may be divided and executed by multiple processors, or it may be executed by a single processor. Alternatively, each of the above-mentioned devices may be implemented through cloud computing, which uses multiple computers interconnected by a communication network.

[0113] The program may be provided to and installed on each device's computer via a communication network from an external device. Alternatively, the program may be stored on a non-transitory computer-readable medium and provided to and installed on each device's computer via that medium.

[0114] Furthermore, according to this embodiment, a program for executing the processing of each function of each of the above-mentioned devices can be provided. In addition, a non-temporary recording medium that can be read by a computer and on which the program is stored can also be provided. The non-temporary recording medium may be a disk-type recording medium such as an optical disk, magnetic disk, or magneto-optical disk, or it may be a semiconductor memory such as a flash memory or USB memory. [Explanation of symbols]

[0115] 100 X-ray inspection systems 200 Customs Database 300 X-ray image databases 400 X-ray imaging device 410 containers 412 Cargo 420 X-ray 430 X-ray irradiation section 440 X-ray detection unit 450 X-ray image generation section 460 Controller 500 Image Processing Devices 510 Processor 520 memory 530 storage 540 Communication equipment 550 Input Device 560 Output device 570 Bus 580 Inference Modules 582 First image resizing section 584 First image division section 586 Estimation Department 590 Learning Modules 592 Association section 594 Second image resizing section 596 Second image division section 598 Learning Department

Claims

1. An imaging unit that images the cargo to be inspected using X-rays and outputs an X-ray image of the cargo to be inspected, A first image resizing unit that changes the size of the X-ray image based on the size of the cargo included in the customs declaration information, A first image division unit divides the X-ray image whose size has been changed into a plurality of partial X-ray images having a predetermined image size, An estimation unit that estimates the type of cargo depicted in at least one of the multiple partial X-ray images, using a learning model constructed based on training X-ray images of the cargo; An X-ray inspection system equipped with [the following features].

2. The predetermined image size is the size of the training image used to construct the learning model. The X-ray inspection system according to claim 1.

3. A customs database where customs declaration information including information on the type and size of one or more types of the aforementioned goods is stored, An X-ray image database where training X-ray images obtained by imaging cargo that is the subject of training using X-rays are stored, A second image resizing unit changes the size of the learning X-ray image based on the size of the cargo included in the customs declaration information, A second image division unit divides the resized learning X-ray image into a plurality of partial learning X-ray images having a predetermined image size, A learning unit constructs a learning model of the cargo to be learned based on the aforementioned X-ray images for partial learning, Equipped with, The X-ray inspection system according to claim 1.

4. The system further includes an association unit that associates the type of cargo to be learned, as depicted in the learning X-ray image, with the learning X-ray image, based on the customs declaration information and user input. The aforementioned learning unit, The X-ray images used for partial training are input to the training model, and the parameters of the training model are modified so that the type of cargo to be trained, output from the training model, matches the type of cargo associated with the X-ray images used for partial training. The X-ray inspection system according to claim 3.

5. If the size of the goods included in the customs declaration information is greater than a predetermined first standard value, the image resizing unit reduces the size of the X-ray image. The X-ray inspection system according to claim 1.

6. If the size of the cargo included in the customs declaration information is less than a predetermined second standard value, the image resizing unit enlarges the size of the X-ray image by adding a margin area around the X-ray image. The X-ray inspection system according to claim 1.

Images