Equipment and method for inspecting personal belongings

Abstract

This invention provides a personal belongings inspection device and inspection method that can obtain images capable of detecting personal belongings protruding from the human body. [Solution] The sensor image detected by the terahertz sensor 101 is used to determine the foreground and background regions from the camera image acquired by the visible light camera 102 in the region corresponding to the sensor image. The sensor image is then binarized into detected and undetected values. The pixels in the background region are replaced with detected values, and the pixels in the foreground region are replaced with the pixel values ​​from the original binarized image and combined. This process yields an image of the shadow of the object, which is a composite of a binarized image of the object extending beyond the human body in the background region and a binarized image of an object blocking the terahertz waves emitted from the human body in the foreground region.

Description

Technical Field

[0006] , , ,

[0005] , , ,

[0001] The present invention relates to a possession inspection device and a possession inspection method for inspecting possessions using terahertz waves radiated from the human body.

Background Art

[0002] Patent Document 1 describes a monitoring system that acquires a terahertz image and a visualization image at a predetermined position, detects a person to be detected from the visualization image, and detects an object carried at the position of the person in the terahertz image subjected to image processing.

Prior Art Documents

Patent Documents

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, since the technology of this monitoring system detects the presence of a possession by the possession blocking electromagnetic waves emitted from the human body, no shadow of the possession can be formed when there is no human body serving as an electromagnetic wave source behind the possession. For example, this applies to a situation where a person to be detected is pulling a luggage such as a carry bag or carrying a bag in hand. Therefore, in hand luggage inspection using this type of technology, there is a problem that a possession protruding from the human body cannot be detected.

[0005] The present invention has been made in view of the above circumstances, and an object thereof is to provide a possession inspection device and an inspection method capable of obtaining an image capable of detecting a possession protruding from the human body.

Means for Solving the Problems

[0006] Note: There seem to be some incorrect or repeated tags in the original text which are retained as per the instructions. For example, <000001X> and <000002X> which might be errors in the original but are translated as is. Also, the number of lines in the translation is the same as the original.The present invention provides a personal belongings inspection device and personal belongings inspection method, characterized in that, with respect to a sensor image detected by a terahertz sensor, the foreground region and background region are determined from a camera image acquired by a visible light camera in the region corresponding to the sensor image, the sensor image is binarized into detected values ​​and undetected values, the pixels of the background region are replaced with the detected values, and the pixels of the foreground region are synthesized using the pixel values ​​of the original binarized image. [Effects of the Invention]

[0007] In this invention, objects under clothing are detected by binarizing the sensor image detected by a terahertz sensor into detected and undetected values, and objects protruding from the human body are detected by image processing of the background region of the sensor image using information from the visible light image. Therefore, according to the present invention, it is possible to provide a personal belongings inspection device and inspection method that can obtain an image capable of detecting personal belongings protruding from the human body. [Brief explanation of the drawing]

[0008] [Figure 1] This is a schematic diagram illustrating the system configuration of a personal belongings inspection device according to an embodiment of the present invention. [Figure 2] This is a functional block diagram showing a personal belongings inspection device according to an embodiment of the present invention. [Figure 3] Figures 1 and 2 are perspective views showing the imaging direction and imaging range of the terahertz imaging unit. [Figure 4] Figures 1 and 2 are plan views showing the imaging direction and range of the visible light imaging unit and the terahertz imaging unit, respectively. [Figure 5] This is a plan view illustrating the area of ​​the terahertz imaging unit that changes as a person being inspected moves through the passageway of the personal belongings inspection device. [Figure 6] This is a frontal view image generated by combining a left-front image and a right-front image, showing the case where there are no items being carried under the clothes. [Figure 7]This is a frontal view image generated by combining a left-front image and a right-front image, showing the case where there are items being carried under the clothing. [Figure 8] This is a flowchart showing a method for inspecting personal belongings according to an embodiment of the present invention. [Figure 9] Figure 8 is a flowchart showing a specific example of foreground / background recognition processing. [Figure 10] Figure 8 is a flowchart showing a specific example of image integration processing. [Figure 11] This figure shows examples of images taken by the terahertz imaging unit and the visible light imaging unit. [Figure 12] This figure shows an example of images combined using the terahertz imaging unit. [Figure 13] This diagram illustrates the determination of foreground and background regions using images captured by the visible light imaging unit. [Figure 14] This diagram illustrates the processing applied to each pixel of a binarized image. [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described below with reference to the drawings. Figure 1 shows the system configuration of a personal belongings inspection device according to an embodiment of the present invention. The personal belongings inspection device 1 is installed indoors or outdoors in places where safety and security must be ensured, such as airports, train stations, public facilities, schools, concert halls, theaters, exhibitions, convention centers, or commercial facilities.

[0010] This personal belongings inspection device 1 consists of photographic units 11-1, 11-2, 11-3, and 11-4 arranged on either side of the passage 3 through which the person being inspected 2 passes, inspection unit 12-1 positioned between photographic units 11-1 and 11-3, and inspection unit 12-2 positioned between photographic units 11-2 and 11-4. Near the entrance side of the personal belongings inspection device 1, a human detection unit (human motion sensor) 4 is installed to detect when the person being inspected 2 approaches the personal belongings inspection device 1. Here, an example is shown in which the human detection unit 4 is installed on photographic unit 11-4, but it may also be installed on photographic unit 11-3, or it may be installed independently of the personal belongings inspection device 1. The person being inspected 2 is a person who is about to enter a specific place and passes through the personal belongings inspection device 1 along the direction of travel A indicated by the arrow.

[0011] Each of the imaging units 11-1 to 11-4 is equipped with an imaging unit 100 for imaging the subject 2, and is housed in the housing cases HC1a to HC4a of each unit. The imaging unit 100 comprises a terahertz imaging unit (terahertz sensor) 101 and a visible light imaging unit (visible light camera) 102. Inspection unit 12-1 is a processing unit 110 for processing images captured by imaging units 11-1 and 11-3, and inspection unit 12-2 is a processing unit 110 for processing images captured by imaging units 11-2 and 11-4. These processing units 110, 110 are housed in the housing cases HC1b and HC2b of each unit 12-1 and 12-2. Each imaging unit 11-1 to 11-4 and each inspection unit 12-1 and 12-2 are connected wirelessly or by wire.

[0012] The housing cases HC1a to HC4a, HC1b, HC2b are hard exteriors or casings that protect against causes of malfunctions such as physical damage, dust, water droplets, or electromagnetic interference. The housing cases HC1a to HC4a, HC1b, HC2b are durable and are formed of materials (such as metal, reinforced glass, etc.) that reflect the surrounding installed environment. The housing cases HC1a, HC3a, HC1b of these imaging units 11-1, 11-3 and inspection unit 12-1, and the housing cases HC2a, HC4a, HC2b of the imaging units 11-2, 11-4 and inspection unit 12-2 are arranged to face each other with the passage 3 therebetween, and the inspection target person 2 passes through the passage 3 to conduct a possession inspection.

[0013] The inspection units 12-1, 12-2 also function as computers in which software (possession inspection software) related to the possession inspection device 1 accessible by the administrator is installed. The possession inspection software is software that inspects whether the inspection target person 2 is in possession of dangerous goods or prohibited items (such as dangerous goods). Dangerous goods are items that may pose a danger to human health or safety, such as weapons (guns, knives, etc.), explosives (gunpowder, gasoline, etc.). Prohibited items are items whose entry into a specific place is prohibited by rules such as laws or regulations, such as food and drink, imaging equipment (equipment such as cameras and videos), and communication terminals (smartphones, tablets, or PCs, etc.).

[0014] FIG. 2 is a functional block diagram showing a possession inspection device according to an embodiment of the present invention. Each of the imaging units 11-1 to 11-4 is provided with an imaging unit 100. The imaging unit 100 is composed of a terahertz imaging unit (terahertz sensor) 101 and a visible light imaging unit (visible light camera) 102. Instead of a visible light camera, a line scan camera can be used for the visible light imaging unit 102. Each of the inspection units 12-1, 12-2 is provided with a processing unit 110. The processing unit 110 includes a control device 111 and an arithmetic device 112.

[0015] The sensor images detected by the terahertz imaging units 101 of the imaging units 11-1 and 11-3, and the camera images obtained by the visible light imaging unit 102 for the regions corresponding to these sensor images are input to the control device 111 of the inspection unit 12-1. Also, the sensor images detected by the terahertz imaging units 101 of the imaging units 11-2 and 11-4, and the camera images obtained by the visible light imaging unit 102 for the regions corresponding to these sensor images are input to the control device 111 of the inspection unit 12-2. When the control device 111 of the inspection unit 12-2 detects the inspection target person 2 by the human detection unit 4, it causes the arithmetic device 112 to execute an operation for detecting the belongings. Also, it communicates with the control device 111 of the inspection unit 12-1 and causes the arithmetic device 112 to execute an operation for detecting the belongings. Then, the arithmetic results of the inspection units 12-1 and 12-2 are displayed on the display device 120. The administrator monitors the video displayed on the display device 120 to inspect whether the inspection target person 2 is carrying dangerous items or the like.

[0016] FIG. 3 shows the shooting direction and shooting range of the terahertz imaging unit 101 in FIGS. 1 and 2. Also, FIG. 4 shows the shooting direction and shooting range of the visible light imaging unit 102 in the system of FIG. 1, and the shooting direction and shooting range of the terahertz imaging unit 101. Here, the imaging unit 11-1 is exemplified, but the shooting directions and shooting ranges of the imaging units 100 of the other imaging units 11-2 to 11-4 are the same.

[0017] The terahertz imaging unit 101 faces in a diagonal direction with respect to the traveling direction of the inspection target person 2 and performs scanning in the vertical direction. That is, the intensity of the radio wave in the vertical direction at a certain moment is acquired. Since the terahertz imaging unit 101 is a sensor, the viewing angle is 1 to 10°, and the fan-shaped region scanned in the vertical direction at this viewing angle (see FIG. 3) is the shooting range R1. On the other hand, since the visible light imaging unit 102 is a camera (digital camera), its shooting range R2 is wide and the viewing angle is 45 to 60°. Therefore, the shooting range R2 completely covers the shooting range Rl (see FIG. 4).

[0018] As person 2 passes through the baggage inspection device 1 in the direction of travel A, the imaging unit 11-4, located to the right of person 2, photographs person 2 from the right rear, and imaging unit 11-2 photographs person 2 from the right front. Additionally, imaging unit 11-3, located to the left of person 2 in the direction of travel A, photographs person 2 from the left rear, and imaging unit 11-1 photographs person 2 from the left front. The background of the captured image will be the area around the surface of the imaging units facing each other at an oblique angle.

[0019] Therefore, the image captured by the imaging unit 11-4 will show the area around imaging unit 11-1, and the image captured by imaging unit 11-1 will show the area around imaging unit 11-4. Similarly, the image captured by imaging unit 11-3 will show the area around imaging unit 11-2, and the image captured by imaging unit 11-2 will show the area around imaging unit 11-3.

[0020] As subject 2 walks toward direction A, the area of ​​subject 2 captured by the terahertz imaging unit 101 changes as the subject moves, as shown in Figures 5(a) to 5(c). First, at the position shown in Figure 5(a), the right half of subject 2's body is captured far away (small). This is because the distance from the terahertz imaging unit to subject 2's right half is large. Next, at the position shown in Figure 5(b), the center of subject 2's body is captured. Subsequently, at the position shown in Figure 5(c), subject 2's left shoulder is captured close up (large). This is because the distance from the terahertz imaging unit to subject 2's left shoulder is small. Thus, because the imaging range (scan range) of the terahertz imaging unit 101 is fixed and narrow, images of the subject 2 taken from an oblique front angle are obtained over time.

[0021] While the imaging unit 11-1 has been described, imaging units 11-2 to 11-4 are similar, and similar images can be obtained from these imaging units as well. The acquired image information is input to the examination units 12-1 and 12-2, where image synthesis is performed. By combining the left front image from imaging unit 11-1 and the right front image from imaging unit 11-2 to create a view from the front, images in the frontal direction, as shown in Figures 6 and 7, are obtained. Figure 6 is an image of subject 2 when they have no possessions, and Figure 7 is an image when they have possessions under their clothing from their right abdomen to their right chest, as indicated by the ellipse. On the other hand, by combining the left rear image from imaging unit 11-3 and the right rear image from imaging unit 11-4 to create a view from the rear, an image in the rear direction is obtained.

[0022] Next, the detection of personal belongings protruding from the human body in the personal belongings inspection device 1 configured as described above will be explained in detail with reference to Figures 8 to 14. Figure 8 is a flowchart of a personal belongings inspection method according to an embodiment of the present invention. When the person detection unit 4 detects that the person to be inspected 2 has approached the entrance side of the personal belongings inspection device 1 or entered the personal belongings inspection device 1 (step ST1), a terahertz image is acquired by the terahertz imaging unit 101 of the imaging units 11-1 to 11-4 (step ST2). This image is a single-line image captured by the terahertz sensor, as shown in Figure 11(a). If the person detection unit 4 does not detect the person to be inspected 2, it remains in a standby state until the person to be inspected 2 is detected.

[0023] In the next step, ST3, the acquired terahertz images are integrated. Subsequently, the visible light imaging unit 102 captures an image, and a camera image as shown in Figure 11(b) is acquired. From this camera image, an image synchronized with the terahertz image (line scan image) is extracted (step ST4), as shown in Figure 11(c). After that, the acquired line scan camera images are integrated (step ST5). In the next step, ST6, if the subject 2 is continuously detected by the human detection unit 4, the process returns to step ST2. That is, as long as the subject 2 is detected by the human detection unit 4, the operation of steps ST2 to ST5 is repeated.

[0024] As a result, terahertz images are obtained by the terahertz imaging unit 101, capturing one line at a time as time progresses with the progress of the subject 2 during the examination. In addition, synchronized with the terahertz images from the terahertz imaging unit 101, camera images are obtained by extracting one vertical line at a time from the camera images captured by the visible light imaging unit 102 in the same direction as the imaging direction of the terahertz images. When these extracted lines are arranged in the same time series as the sensor images, an image like the one shown in Figure 12 is obtained.

[0025] Then, in step ST6, when the person detection unit 4 no longer detects the person being inspected 2, the processing units 110 of inspection units 12-1 and 12-2 perform background / foreground identification processing to identify the human body region (foreground) and the background region (step ST7). The background image is formed by extracting the edges of the camera image (composite of line scan camera images) created in steps ST2 to ST5 and stretching it. As shown in Figure 13(a), when the foreground region (person + luggage) and the background region are determined using the camera image, the background and foreground are identified as shown in Figure 13(b). In this identified camera image, it is clear that the person is pulling luggage such as a carry-on bag, but it is not clear whether or not there are belongings under the clothing.

[0026] On the other hand, the terahertz image obtained by acquiring and accumulating each line is as shown in Figure 13(c). In the following step ST8, this terahertz image is binarized to obtain an image represented by "0" / "1", where areas brighter than the threshold are white (=1) and dark areas are black (=0). As a result, an image is generated in which the human body is represented as white (=1), the shadows of possessions as black (=0), and the background as black (=0), as shown in Figure 13(d). In this terahertz image, possessions under clothing can be recognized, but the carry bag is not visible. Subsequently, the camera image generated in step ST7, which has been processed to identify the human body region and the background region, is combined with the binarized terahertz image from step ST8 (step ST9).

[0027] Figure 9 is a flowchart illustrating a specific example of the foreground / background recognition process (ST7) in Figure 8. In the foreground / background recognition process, the camera image data is stretched at the edges (ST11). Next, an absolute difference image is generated (ST12). Then, pixels whose absolute value is greater than a threshold are designated as the foreground region (ST13). In other words, the difference in brightness between one line of the camera image and the background image is obtained, and the region with a large difference is designated as the foreground region, and the rest as the background region.

[0028] Figure 10 is a flowchart illustrating a specific example of the image integration process (ST9) in Figure 8. In the image integration process, the upper left pixel is examined in the background and foreground identification results of the binarized terahertz image (ST21). It is determined whether the pixel is in the background region or not (ST22), and if it is determined to be in the background region, the pixel is replaced with "white" (=1) (ST23). At this time, the pixel value of the original binarized image ("0" or "1") is retained for the foreground region.

[0029] If it is determined that a pixel is not in the background area, or if a pixel has been replaced with "white", it is determined whether processing of all pixels is complete (ST24). If processing of all pixels is not complete, attention is paid to the next pixel in the binarized terahertz image (ST25), and the process returns to step ST22 to replace pixels in the background area with "white" until processing of all pixels is complete (ST23). Once processing of all pixels is complete, the image integration process is terminated.

[0030] Thus, by using the foreground and background region information (region determination result) generated in step ST7 as shown in Figure 14(b), and performing image processing on the binarized image shown in Figure 14(a), the human body becomes white (=1), the shadows of possessions become black (=0), and the background is inverted from black to white (=1), leaving the shadows of possessions as shown in Figure 14(c). The shadows of possessions represent both possessions under clothing and possessions that extend beyond the human body.

[0031] As described above, in this invention, the terahertz image (sensor image) detected by the terahertz imaging unit is binarized into detected and undetected values ​​to detect objects under clothing, and the background region is processed using visible light image information to detect objects protruding from the human body. Therefore, according to this invention, an image capable of detecting objects protruding from the human body can be obtained.

[0032] The circuit configurations and control procedures described in the above embodiments are merely schematic representations to the extent that the present invention can be understood and implemented. Therefore, the present invention is not limited to the embodiments described, and can be modified in various forms as long as it does not deviate from the scope of the technical idea set forth in the claims.

[0033] For example, Figures 1 and 2 show an example with four imaging units and two inspection units, but the number and arrangement of imaging and inspection units are not limited to this example.

[0034] Furthermore, in the system shown in Figure 1, the overall shape of the housing case is a concave shape formed by combining multiple rectangular housing cases, with a passageway in between them, but the system is not limited to this shape. For example, it could be a configuration where a single rectangular housing case forms the middle of the passageway.

[0035] The shape of the imaging unit housing is not limited to a columnar shape; it may be spherical, polyhedral, or any other shape. Furthermore, the hardware configuration of the inspection unit may differ from that exemplified in the embodiment. For example, the correspondence between the imaging unit and the inspection unit is not limited to that exemplified in the embodiment.

[0036] In this embodiment, one inspection unit processes images from two imaging units, but one inspection unit may process images from all four imaging units. Alternatively, the inspection units may correspond one-to-one with the imaging units and be housed in the same enclosure.

[0037] Furthermore, the inspection unit may be a server on a computer network. This server may be a physical server or a virtual server (so-called cloud). [Explanation of symbols]

[0038] 1...Personal belongings inspection device, 2...Person being inspected, 3...Passageway, 4...Person detection unit (human presence sensor), 11-1~11-4...Shooting unit, 12-1,12-2...Inspection unit, 100...Imaging unit, 101...Terahertz imaging unit (terahertz sensor), 102...Visible light imaging unit (visible light camera), 110...Processing unit, 111...Control device, 112...Calculation unit, 120...Display device

Claims

1. A personal belongings inspection device that, based on a sensor image detected by a terahertz sensor, determines a foreground and background region from a camera image acquired by a visible light camera in the region corresponding to the sensor image, performs a process to binarize the sensor image into detected and undetected values, replaces the pixels in the background region with the detected values, and synthesizes the pixels in the foreground region using the pixel values ​​from the original binarized image.

2. The personal belongings inspection device according to claim 1, characterized in that it acquires a camera image with the visible light camera in synchronization with the terahertz sensor, extracts one line in the same direction as the imaging direction of the terahertz sensor, and arranges the extracted line in the same time series as the sensor image to obtain a background image.

3. The personal belongings inspection device according to claim 2, characterized in that it acquires the difference in brightness between one line of the camera image and the background image, and designates the area with a large difference as the foreground area and the rest as the background area.

4. A method for inspecting personal belongings, comprising: determining a foreground and background region from a camera image acquired by a visible light camera in the region corresponding to the sensor image detected by a terahertz sensor; performing a process to binarize the sensor image into detected and undetected values; replacing the pixels in the background region with the detected values; and synthesizing the pixels in the foreground region using the pixel values ​​from the original binarized image.

Images