Hazardous materials detection system

Abstract

To allow an inexperienced or unskilled user to associate a position of a scan image with a position of a captured image.SOLUTION: A first acquisition unit 111 acquires scan signals from a detector 2. A generation unit 112 generates a scan image from the acquired scan signals. A first extraction unit 113 extracts a person region from the generated scan image. A detection unit 114 detects a hazardous material shown in the scan image. A correction unit 115 corrects distortion of the scan image. A second acquisition unit 116 acquires a captured image of a person Q from an optical camera 3. A second extraction unit 117 reads a trained model 123 from a memory 12 and uses feature quantities or the like described in the model, to extract a person region from the captured image acquired by the second acquisition unit 116. A display control unit 118 causes a display unit 15 to display the scan image and the captured image in a mode so that a user can identify a correspondence relation between the positions of the person regions extracted in the scan image and the captured image, respectively.SELECTED DRAWING: Figure 8

Description

Technical Field

[0001] The present invention relates to a display device that displays a person's belongings by electromagnetic waves and a technology of a dangerous object detection system that detects dangerous objects from a person's belongings.

Background Art

[0002] As a method for conducting a belongings inspection, there is a method of inspecting for the presence or absence of dangerous objects hidden under clothes using electromagnetic waves. For example, Patent Document 1 discloses a technique for visualizing dangerous objects or the like carried by a subject passing through a gate area as a two-dimensional semi-transparent image based on measurement data obtained from an inspection unit.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, even if a belongings inspection device discovers a dangerous object by a semi-transparent image or the like, it is the staff at the belongings inspection site who visually checks the display of the semi-transparent image or the like. Since the display of the semi-transparent image or the like is not an image based on visible light, it may be difficult for the staff who are looking at the subject with the naked eye under visible light to associate the appearance of the subject with the display of the semi-transparent image or the like.

[0005] Also, in the technology described in Patent Document 1 and the like, the inspection unit repeatedly acquires elongated line-shaped detection images in the Z direction (vertical, height direction), and connects these in the Y direction (lateral direction) in which the subject moves to generate an image corresponding to the front or side of the subject.

[0006] * In the translation of "Japanese Patent Application Laid-Open No. 202*-115107", the asterisk is used to represent the original character in the Japanese patent number which is not provided in the original text. You may need to replace it with the correct number according to the actual patent number.Images obtained by inspection units that scan in one direction are distorted in areas closer to the edges of a single scan line. Due to this distortion, it becomes even more difficult to match the semi-transparent image displayed by the inspection unit with the appearance of the subject as seen by the naked eye, and the staff may not be able to determine where dangerous materials are hidden on the subject.

[0007] One of the objectives of the present invention is to enable users (staff) without experience or skills to associate the position of scanned images with the position of captured images. [Means for solving the problem]

[0008] This invention combines a scanned image generated by scanning space by changing the wave receiving direction with a captured image taken by an optical camera. The correspondence between the positions of the regions representing the extracted objects in each case. User is special In a determinable manner 、 Each display device A hazardous materials detection system that has the following capabilities: it receives radio waves of two or more different frequencies, generates a scan image for each, detects different types of hazardous materials among the target objects based on each of the scan images, and displays the location of the hazardous materials on the display device. , is provided as the first embodiment.

[0009] The first aspect Hazardous materials detection system According to this, it is possible to enable even users without experience or skills to associate the position of scanned images with the position of captured images. Furthermore, with this embodiment of the hazardous materials detection system, the user can identify different types of hazardous materials detected at two or more frequencies.

[0010] The first aspect Hazardous materials detection system In a second embodiment, a configuration may be adopted in which the scanned image is corrected so that the position of the object shown by the scanned image matches the position of the object shown by the captured image.

[0011] The second aspect Hazardous materials detection system According to this, the distortion of the scanned image relative to the captured image becomes less noticeable. In the hazardous materials detection system of the second embodiment, the scanned image is generated by scanning the receiving direction in the height direction, and a third embodiment may be adopted in which distortion caused by scanning the receiving direction in the height direction in the scanned image is corrected by performing a conversion that multiplies the receiving direction by a numerical value corresponding to the angle.

[0012] The first or second form Hazardous materials detection system In this configuration, the scanned image and the captured image are displayed in a way that allows the user to compare them. 4 This may be adopted as one of the following forms.

[0013] According to the 4 aspect of Hazardous materials detection system , the user can compare the scanned image and the photographed image.

[0016] In the dangerous object detection system according to the 1 aspect, a configuration of extracting a region indicating a person from the photographed image and displaying the position of the region on the display device may be adopted as the fifth aspect.

[0017] According to the dangerous object detection system of the fifth aspect, the user can confirm the region indicating a person in the photographed image.

[0018] In the dangerous object detection system according to the 5 aspect, The system includes a first extraction unit that binarizes the scanned image and extracts a region representing a person from the scanned image by comparing the number of consecutive pixels in the vertical direction and the number of consecutive pixels in the horizontal direction with thresholds, respectively, and a second extraction unit that extracts a region representing a person from the captured image using a trained model. , a configuration may be adopted as the sixth aspect.

[0019] According to the dangerous object detection system of the sixth aspect, the user can People are extracted based on the number of consecutive pixels and the trained model. confirm.

Brief Description of Drawings

[0020] [Figure 1] Block diagram showing the configuration of the dangerous object detection system 9 according to an embodiment of the present invention. [Figure 2] Plan view showing the arrangement of the detector 2 and the optical camera 3. [Figure 3] Diagram for explaining the detection region Ra of the detector 2. [Figure 4] Diagram for explaining the distortion caused by capturing the detection object around the axis F. [Figure 5] Diagram showing an example of the configuration of the display device 1. [Figure 6] Diagram showing an example of the configuration of the extraction reference table 121. [Figure 7] Diagram showing an example of the configuration of the detection reference table 122. [Figure 8] Diagram showing an example of the functional configuration of the display device 1. [Figure 9] Flow chart showing an example of the operation flow of the display device 1. [Figure 10] A diagram showing an example of binarization of scanned images. [Figure 11] A diagram illustrating the process of extracting human areas by processing scanned images. [Figure 12] This figure shows an example of displaying the extracted human region overlaid on a scanned image. [Figure 13] This figure shows an example of displaying the extracted human region overlaid on a captured image. [Figure 14] A diagram illustrating the corrections in the modified example. [Modes for carrying out the invention]

[0021] <Embodiment> <Configuration of the hazardous materials detection system> In the following diagrams, the space in which each component is arranged is represented as an XYZ right-handed coordinate system. Among the coordinate symbols shown in the diagrams, the symbol of a dot inside a circle represents an arrow pointing from the back of the page to the front. The direction along the X-axis in space is called the X-axis direction. Within the X-axis direction, the direction in which the X component increases is called the +X direction, and the direction in which the X component decreases is called the -X direction. The Y and Z components are defined similarly to the X component.

[0022] Figure 1 is a block diagram showing the configuration of a hazardous materials detection system 9 according to an embodiment of the present invention. The hazardous materials detection system 9 shown in Figure 1 includes a display device 1, a detector 2, and an optical camera 3. The display device 1 is connected to the detector 2 and the optical camera 3 via wired or wireless communication. In the hazardous materials detection system 9, the number of display devices 1, detectors 2, and optical cameras 3 may be one or multiple.

[0023] Figure 2 is a plan view showing the arrangement of the detectors 2 and optical cameras 3 in the hazardous materials detection system 9. The hazardous materials detection system 9 shown in Figure 2 has four detectors 2 and four optical cameras 3.

[0024] In Figure 2, the -Z direction is downward, i.e., the direction of gravity; the X-axis direction is the direction of the width of the path Pa; and the Y-axis direction is the direction along the path Pa. The hazardous material detection system 9 is a system that inspects the belongings of person Q walking in the +Y direction along the path Pa.

[0025] Four detectors 2 are positioned, two on each side of the path Pa. The four detectors 2 receive electromagnetic waves such as terahertz waves emitted in all four directions when person Q and the objects that person Q is carrying (referred to as "possessions") are located near the center of the Y-axis direction of the path Pa. The path through which these electromagnetic waves propagate is called the propagation path D. Person Q is the person subject to inspection by the detectors 2 of the hazardous materials detection system 9, i.e., the person being inspected.

[0026] From the perspective of person Q moving in the +Y direction along the path Pa, the detector 2 on the right front and the detector 2 on the left rear are facing each other, and the other detector 2 is positioned on the extension of the electromagnetic wave receiving region of the other detector 2. The relationship between the detector 2 on the left front and the detector 2 on the right rear from the perspective of person Q moving in the +Y direction along the path Pa is similar. However, the opposing arrangement of each detector 2 is not a mandatory configuration.

[0027] Each of the four detectors 2 scans person Q and their belongings in the Z-axis direction. As person Q walks (moves) along path Pa in the +Y direction, person Q and their belongings pass through the propagation path D of the electromagnetic waves received by each of the four detectors 2. As a result, the four detectors 2 scan person Q and their belongings in the Y-axis direction and the X-axis direction (i.e., horizontal direction).

[0028] The optical cameras 3 are positioned one at a time near each of the four detectors 2, and each is configured to capture images of person Q from the extension of the propagation path D.

[0029] <Detector Configuration> Figure 3 is a diagram illustrating the detection region Ra of detector 2. Figure 3 shows the shape of detector 2 as viewed from a direction intersecting both the Z-axis direction and the propagation path D shown in Figure 2, and the shape of the detection region Ra detected by detector 2. Detector 2 comprises a radome 21, an optical system 22, a filter 23, and a sensor 24. The optical system 22 and the sensor 24 are connected to the display device 1 via wired or wireless communication and are controlled by the display device 1.

[0030] The radome 21 is a plate-shaped component and is made of a resin material that is relatively permeable to electromagnetic waves, such as polyethylene, polypropylene, polytetrafluoroethylene, or polymethylpentene. The radome 21 allows electromagnetic waves arriving from outside the housing of the detector 2 to pass through to the inside while protecting the inside from dust and other debris. These electromagnetic waves are, for example, terahertz waves with a frequency of 100 GHz or more and less than 10 THz.

[0031] The optical system 22, filter 23, and sensor 24 constitute a sensor unit U. This sensor unit U is, for example, a rectangular prism with a width of 200 millimeters, a depth of 300 millimeters, and a height of 400 millimeters.

[0032] The optical system 22 shown in Figure 3 includes, for example, a polygon mirror and a focusing mirror. The polygon mirror is a polygonal mirror that rotates around the axis F shown in Figure 3, and is, for example, a square pyramid composed of mirror surfaces. Electromagnetic waves emitted from person Q and propagating along the propagation path D pass through the radome 21, are reflected by the polygon mirror, and are collected by the focusing mirror.

[0033] As the polygon mirror rotates, its reflective surface changes angle, so the range in which the optical system 22 receives (detects) electromagnetic waves becomes a sector-shaped detection region Ra centered on axis F, as shown in Figure 3. In other words, as the polygon mirror rotates around axis F, the detector 2 performs scanning in the height direction (Z-axis direction).

[0034] The focusing mirrors that make up the optical system 22 are mirrors that reflect electromagnetic waves collected by the polygon mirrors and guide them to the filter 23 and sensor 24. The focusing mirrors are, for example, parabolic mirrors.

[0035] The detection region Ra is the area in which electromagnetic waves can be detected by detector 2. As shown in Figure 3, the detection region Ra is within an angle range of approximately 100 degrees around axis F. The thickness of the detection region Ra is several centimeters.

[0036] Here, the detection target position Ta shown in Figure 3 is the position where the object to be detected is located, and it extends in the height direction. Since the detection region Ra scans in the height direction in the wave receiving direction with axis F as its center, the distance between axis F and the detection target position Ta varies depending on its height. That is, when the detection target position Ta is at the same height as axis F, the object to be detected is closest to axis F, and the further away the object to be detected is as the height differs from axis F. Therefore, for example, if pixels of the scanned image are generated at equal angles, the scanned image will be distorted as you move upwards or downwards in the height direction.

[0037] Figure 4 is a diagram illustrating the distortion caused by capturing the object to be detected around axis F. The position Z of the detected object Ta, which extends in the height direction at a horizontal distance r from axis F, is expressed as the tangent of the angle θ by the following equation (1).

[0038]

number

[0039] The optical system 22 divides the angular range of the detection area Ra into multiple sections and generates pixels of the scanned image for each divided section. For example, the optical system 22 divides the angular range of the detection area Ra, which is 100°, into 228 equal sections. That is, the optical system 22 divides the angular range of the detection area Ra into 228 sections with equal angles. In this case, the angles around the axis F of the divided sector are adjusted to be equal for all sections, so the small angle Δθ for one section is 100° / 228 ≈ 0.439°.

[0040] Here, the height of the image captured by the sensor unit U with respect to the small angle Δθ which is the field of view of this one section, is expressed by the following equation (2).

[0041]

number

[0042] This magnitude ΔZ is small as θ approaches 0 and large as θ moves away from 0. On the other hand, Δy is the circumference-arc length corresponding to the small angle Δθ in a circle with radius r. This Δy is expressed by the following equation (3), which includes the horizontal distance r and the small angle Δθ.

[0043]

number

[0044] Since both the horizontal distance r and the small angle Δθ are constant, Δy is also constant. In other words, the Δy shown in Figure 4 has a common size regardless of the scanning angle. Then, ΔZ is reduced by the sensor unit U so that it fits within the common size Δy shown in Figure 4.

[0045] In other words, the actual size ΔZ of an object within the field of view per section increases as it is positioned higher or lower than axis F at the detection target position Ta. This ΔZ is then reduced to fit within a common size Δy, causing the scanned image to become distorted.

[0046] As the small angle Δθ approaches 0, the ratio of ΔZ / Δy becomes dZ / dy, as shown in equation (4) below.

[0047]

number

[0048] Therefore, this scanned image is distorted because it is reduced in size towards the top and bottom edges. To eliminate this distortion, the scanned image is calculated as dZ / dy = (1 / cos 2 It is necessary to multiply by θ). The scanned image is corrected using this dZ / dy by, for example, the display device 1.

[0049] The values ​​used for correction to restore distortion are not limited to the examples described above. For example, the scanned image may be corrected by multiplying the size of the image in the height direction at angle θ by a factor of (1 / cosθ). Also, the scanned image does not necessarily have to be corrected by applying a different factor to each divided section. For example, the scanned image may be divided into three sections: a region close to axis F, and distant regions located above and below it, and each of these sections may be corrected by applying a predetermined factor.

[0050] The filter 23 shown in Figure 3 is an optical filter, such as a polarizing filter. The filter 23 separates the electromagnetic waves guided from the optical system 22 into a first frequency band centered around 100 GHz (referred to as the low frequency band) and a second frequency band centered around 150 GHz (referred to as the high frequency band), and supplies them to the sensor 24. The electromagnetic waves in the low frequency band and the high frequency band may be supplied to the sensor 24 at different timings. In this case, the filter 23 may have multiple optical filters, each of which may be moved to a position that periodically blocks the optical path.

[0051] Sensor 24 detects low-frequency or high-frequency electromagnetic waves, such as terahertz waves, that have been collected and reflected by the optical system 22, and measured their intensity.

[0052] For example, person Q emits terahertz waves. On the other hand, if person Q is concealing an object that does not easily transmit terahertz waves, such as metal, inside their clothing, the terahertz waves emitted by person Q will be shielded by that object. Therefore, when detector 2 scans person Q, it generates a scan image in which there is a difference in the intensity of the received waves at the outline of the aforementioned object. Using this generated scan image, display device 1 can, for example, identify the shape of person Q's object.

[0053] In this way, detector 2 detects objects such as metal, explosives, ceramics, and flammable liquids that person Q has hidden inside their clothing.

[0054] <Display device configuration> Figure 5 shows an example of the configuration of the display device 1. The display device 1 includes a processor 11, memory 12, interface 13, and display unit 15.

[0055] Memory 12 includes RAM (Random Access Memory), ROM (Read Only Memory), a solid-state drive, a hard disk drive, etc., and stores computer programs (hereinafter simply referred to as "programs").

[0056] Furthermore, the memory 12 shown in Figure 5 stores the extraction criteria table 121, the detection criteria table 122, and the trained model 123. These are used when the processor 11 executes processing.

[0057] For example, the trained model 123 is a model that represents the characteristics of a human region, constructed by training the model using photographic images as training data, where information about the presence or absence of a region indicating a person (hereinafter also referred to as "human region") is associated with the location of that region if it exists. By referring to the trained model 123, the processor 11 extracts the human region from the image captured by the optical camera 3.

[0058] The processor 11 controls the display device 1 by reading and executing a program from the memory 12. The processor 11 also controls the detector 2 and the optical camera 3 via the interface 13. The processor 11 is, for example, a CPU (Central Processing Unit). Alternatively, the processor 11 may be, for example, an FPGA (Field Programmable Gate Array), or may include an FPGA. Furthermore, this processor may have an ASIC (Application Specific Integrated Circuit) or other programmable logic device, and control may be performed by these.

[0059] Interface 13 is a communication circuit that connects the display device 1 to other devices via wired or wireless means, enabling communication between them.

[0060] The display unit 15 has a display screen such as a liquid crystal display and displays an image under the control of the processor 11.

[0061] <Structure of the extraction criteria table> Figure 6 shows an example of the structure of the extraction criteria table 121. The extraction criteria table 121 is a table that describes the parameters used as criteria when extracting the human region described above from the scanned image. The extraction criteria table 121 shown in Figure 6 stores the following items: "vertical consecutive count," "horizontal consecutive count," "vertical exception condition," and "horizontal exception condition," and stores the corresponding values ​​for each.

[0062] <Structure of the detection criteria table> Figure 7 shows an example of the structure of the detection criteria table 122. The detection criteria table 122 is a table that stores the types of objects to be detected as hazardous materials and their detection patterns, associated with each frequency band of the scanned image.

[0063] Electromagnetic waves such as terahertz waves have different penetration characteristics through objects depending on the frequency band. For example, scanning images generated by high-frequency electromagnetic waves are more likely to detect powdery objects than scanning images generated by low-frequency electromagnetic waves. On the other hand, scanning images generated by low-frequency electromagnetic waves are more likely to detect the shape of metals and other materials than scanning images generated by high-frequency electromagnetic waves.

[0064] The detection criteria table 122 stores, for each frequency band such as high-frequency band or low-frequency band, the material of the object to be detected and the pixel pattern when it is detected. The processor 11 refers to this detection criteria table 122 to detect hazardous materials from the scanned image.

[0065] <Functional configuration of the display device> Figure 8 shows an example of the functional configuration of the display device 1. The processor 11 of the display device 1 functions as a first acquisition unit 111, a generation unit 112, a first extraction unit 113, a detection unit 114, a correction unit 115, a second acquisition unit 116, a second extraction unit 117, and a display control unit 118 by reading and executing a program stored in the memory 12.

[0066] The first acquisition unit 111 acquires the scanning signal obtained by the detector 2 scanning space while changing the wave receiving direction.

[0067] The generation unit 112 generates, for example, elongated strip-shaped images from the scanning signal acquired by the first acquisition unit 111, and then generates a scanning image by arranging these multiple strip-shaped images in sequence over time. The scanning image thus generated is an example of a scanning image generated by scanning space due to changes in the wave reception direction.

[0068] The first extraction unit 113 reads the extraction criteria table 121 from the memory 12 and uses the parameters described therein to extract the human region from the scanned image generated by the generation unit 112.

[0069] The detection unit 114 reads the detection criteria table 122 from the memory 12 and uses the parameters described therein to detect the hazardous material shown in the scanned image.

[0070] The correction unit 115 corrects distortion caused by capturing an object located at the detection target position Ta centered on the axis F by performing a conversion on the scanned image using, for example, the numerical values ​​dZ / dy mentioned above.

[0071] The second acquisition unit 116 acquires the captured image of person Q from the optical camera 3. This captured image acquired by the second acquisition unit 116 is an example of an image captured by the optical camera.

[0072] The second extraction unit 117 reads the trained model 123 from the memory 12 and uses the features described in it to extract the human region from the captured image acquired by the second acquisition unit 116.

[0073] The display control unit 118 displays the scanned image and the captured image on the display unit 15 in a manner that allows the user to identify the corresponding positions of the human regions extracted in each image. In other words, the display device 1, which has a processor 11 that functions as a display control unit 118, is an example of a display device that displays the scanned image and the captured image taken by the optical camera in a manner that allows the user to identify the corresponding positions.

[0074] <Display device operation> Figure 9 is a flowchart showing an example of the operation flow of the display device 1. The processor 11 of the display device 1 executes the processing of the scanned image from step S101 to step S107 and the processing of the captured image from step S201 to step S202 in parallel.

[0075] <Processing of scanned images> The processor 11 of the display device 1 determines whether or not it has acquired a scanning signal from the detector 2 (step S101). While it is determined that no scanning signal has been acquired (step S101; NO), the processor 11 continues this determination.

[0076] On the other hand, if it is determined that a scan signal has been acquired (step S101; YES), the processor 11 generates a scan image from the scan signal by processing the acquired scan signals, such as arranging them in chronological order (step S103).

[0077] Then, the processor 11 binarizes the generated scan image (step S104) and extracts the human region from this binarized scan image according to the extraction criteria table 121 described above (step S105).

[0078] Figure 10 shows an example of the binarization of a scanned image. Figure 10(a) shows the scanned image before binarization. By comparing each pixel of this scanned image with a predetermined threshold, the scanned image is binarized as shown in Figure 10(b).

[0079] Figure 11 is a diagram illustrating the process of extracting the human region by processing a binarized scan image. The binarized scan image shown in Figure 10(b) is processed to obtain the image shown in Figure 11(a) by gathering the white pixels at the bottom.

[0080] The first extraction unit 113, implemented by the processor 11, reads the vertical continuity number L1 described in the extraction criterion table 121. The first extraction unit 113 then compares this vertical continuity number L1 as a threshold to the vertical pixel group of the processed image shown in Figure 11(a), thereby identifying the width W of the range where the number of vertically consecutive white pixels exceeds L1 and its position.

[0081] Furthermore, the binarized scanned image shown in Figure 10(b) is processed to obtain the image shown in Figure 11(b) by shifting the white pixels to the right.

[0082] The first extraction unit 113 reads the number of consecutive horizontal pixels L2 described in the extraction criteria table 121. Then, the first extraction unit 113 compares this number of consecutive horizontal pixels L2 as a threshold to the horizontal pixel group of the processed image shown in Figure 11(b), thereby identifying the height H and position of the range where the number of consecutive white pixels in the horizontal direction exceeds L2. These identified width W, height H, and their positions are used to extract the human region from the scanned image.

[0083] Returning to the explanation of the flowchart in Figure 9, when the processor 11 extracts the human area, it attempts to detect hazardous materials within the human area, for example, according to the detection criteria table 122 described above (step S106). If hazardous materials are detected in the scanned image, the processor 11 identifies the area in the scanned image where the hazardous materials are presumed to be present (referred to as the hazardous material area).

[0084] The processor 11 enlarges or reduces the scanned image in the height direction to correct the distortion of the scanned image caused by capturing the object to be detected by the detection area Ra (step S107).

[0085] <Processing of captured images> Meanwhile, the processor 11 performs processing on the captured image in parallel with the processing on the scanned image described above. The processor 11 determines whether or not it has acquired a captured image from the optical camera 3 (step S201). As long as it determines that no captured image has been acquired (step S201; NO), the processor 11 continues this determination.

[0086] On the other hand, if it determines that an image has been captured (step S201; YES), the processor 11 reads the trained model 123 from memory 12 and extracts the human region from the acquired image (step S202).

[0087] Once both steps S107 and S202 are completed, the processor 11 displays on the display unit 15 the corresponding positions of the human region extracted in the scanned image and the human region extracted in the captured image, respectively, in a manner that allows the user to identify them.

[0088] Figure 12 shows an example of displaying the extracted human region overlaid on the scanned image. The processor 11 overlays the frame R0 indicating the extracted human region onto the scanned image before it is binarized and displays it on the display unit 15. This allows the user to understand that the area inside frame R0 in the scanned image is the human region.

[0089] Furthermore, the processor 11 overlays a frame R1 indicating the identified hazardous material area onto the scanned image and displays it on the display unit 15. This allows the user to understand that the area inside frame R1 in the scanned image is the hazardous material area.

[0090] In other words, the hazardous materials detection system 9 having a display device 1 is an example of a hazardous materials detection system that has a display device, detects hazardous materials based on a scanned image, and displays the location of the hazardous materials on the display device.

[0091] Figure 13 shows an example of displaying the extracted human region overlaid on the captured image. The processor 11 overlays a frame R2 indicating the human region, extracted using the trained model 123, onto the captured image taken by the optical camera 3 and displays it on the display unit 15. This allows the user to understand that the area inside frame R2 in the scanned image is the human region.

[0092] In other words, the hazardous material detection system 9 having a display device 1 is an example of a hazardous material detection system that extracts a region indicating a person from a captured image and displays the location of that region on the display device.

[0093] Furthermore, the processor 11 overlays a frame R3 indicating the identified hazardous material area onto the captured image and displays it on the display unit 15. The position and size of this frame R3 indicating the hazardous material area may be determined by applying the relative position and size of frame R1 indicating the hazardous material area to frame R0 indicating the human area extracted in the scanned image, to frame R2 indicating the human area extracted in the captured image.

[0094] The processor 11 causes the display unit 15 to display the scanned image shown in Figure 12 and the captured image shown in Figure 13, so that the user can compare their corresponding positions and understand their relationship. In other words, the display device 1 having this processor 11 is an example of a display device that displays scanned images and captured images in a way that allows the user to compare them.

[0095] For example, the processor 11 may display the scanned image and the captured image side by side on the display unit 15. Alternatively, the processor 11 may display the scanned image and the captured image alternately so that frame R0 and frame R2 are displayed in the same position. Alternatively, the processor 11 may display the scanned image and the captured image superimposed so that frame R0 and frame R2 are displayed in the same position. When displaying the scanned image and the captured image superimposed, the transmittance of at least one of them can be adjusted so that both can be viewed simultaneously.

[0096] By performing the process described above, the hazardous materials detection system 9 can enable even inexperienced or unskilled users (personnel) to recognize the correspondence between the location of the scanned image and the location of the captured image.

[0097] The configurations, shapes, sizes, and arrangements described in the above embodiments are merely schematic representations to the extent that the present invention can be understood and implemented. Therefore, the present invention is not limited to the described embodiments and can be modified in various forms as long as it does not deviate from the scope of the technical idea set forth in the claims.

[0098] <Variation> The above describes the embodiment, but the contents of this embodiment can be modified as follows. Furthermore, the following modifications may be combined.

[0099] <1> In the embodiment described above, the detector 2 of the hazardous materials detection system 9 had a filter 23, which is an optical filter such as a polarizing filter that selectively transmits low-frequency and high-frequency electromagnetic waves from the electromagnetic waves collected by the optical system 22. However, the detector 2 does not necessarily have to have a filter 23.

[0100] Furthermore, in the hazardous materials detection system 9, the display device 1 may separate the electromagnetic waves received by the detector 2 into frequency bands instead of the filter 23 of the detector 2. In this case, the processor 11 shown in Figure 8 may function as a filter unit 110. The filter unit 110 shown in Figure 8 may separate the scanning signal acquired by the first acquisition unit 111 into frequency bands. The filter unit 110 may select the frequency band of the scanning signal using, for example, a discrete Fourier transform.

[0101] In this case, as shown in the flowchart in Figure 9, the processor 11 only needs to apply a band filter to the scan signal acquired in step S101 (step S102). Then, in step S103, the processor 11 only needs to generate different scan images for each frequency band of the separated scan signal.

[0102] When electromagnetic waves of different frequency bands are received by the filter 23 of the detector 2, or by the filter unit 110 implemented by the processor 11, and a scan image is generated from each, the hazardous material detection system 9 detects different types of hazardous materials from each of the electromagnetic waves of different frequency bands. This is because the types of hazardous materials that are easily detected also change as the frequency band changes. As described above, the hazardous material detection system 9, which detects hazardous materials by receiving electromagnetic waves of multiple different frequency bands, is an example of a hazardous material detection system that receives two or more radio waves of different frequencies, generates a scan image from each, and detects different types of hazardous materials based on each of those scan images.

[0103] <2> In the embodiment described above, the processor 11, which functions as a correction unit 115, corrected the distortion caused by capturing an object located at the detection target position Ta with respect to axis F, but a different correction may be performed. For example, the detection target position Ta may not be a position extending in the height direction, but rather a position on a spherical surface at a certain distance from the focal point of the optical system of the optical camera 3.

[0104] Figure 14 is a diagram illustrating the correction in the modified example. In Figure 14, the focal point F3 is the focal point in the optical system of the optical camera 3. In this modified example, the detection target position Ta lies on a sphere at a distance of radius r3 from the focal point F3. The optical camera 3 captures an object on this sphere and generates a planar image. The processor 11 can correct the scan signal obtained by scanning the detection region Ra by changing the reception direction around axis F to match the detection target position Ta located on the sphere as described above.

[0105] In short, the display device 1 only needs to correct the scanned image so that the position of the object shown in the scanned image matches the position of the object shown in the captured image. That is, this display device 1 is an example of a display device that corrects its scanned image so that the position of the object shown in the scanned image matches the position of the object shown in the captured image.

[0106] Furthermore, the captured image taken by the optical camera 3 may be corrected for distortion originating from the optical system, such as the lens. In this case, the display device 1 should correct the scanned image so that the position of the object shown by the scanned image matches the position of the object shown by the corrected captured image.

[0107] <3> In the modified example described above, the detector 2 or optical camera 3 does not have a processor or memory, but it may have a processor and memory. In this case, at least a part of the functions realized by the processor 11 and memory 12 of the display device 1 described above may be realized by the processor or memory of the detector 2 or optical camera 3. [Explanation of symbols]

[0108] 1...Display device, 11...Processor, 110...Filter unit, 111...First acquisition unit, 112...Generation unit, 113...First extraction unit, 114...Detection unit, 115...Correction unit, 116...Second acquisition unit, 117...Second extraction unit, 118...Display control unit, 12...Memory, 121...Extraction criterion table, 122...Detection criterion table, 123...Trained model, 13...Interface, 15...Display unit, 2...Detector, 21...Radome, 22...Optical system, 23...Filter, 24...Sensor, 3...Optical camera, 9...Hazardous material detection system, F...Axis, F3...Focus, L1...Vertical consecutive number, L2...Horizontal consecutive number, R0...Frame, R1...Frame, R2...Frame, R3...Frame, r...Horizontal distance, r3...Radius.

Claims

1. The system includes a display device that shows a scanned image generated by scanning space due to a change in the wave reception direction and a captured image taken with an optical camera, in a manner that allows the user to identify the correspondence between the positions of the regions representing the extracted objects in each image. The system receives two or more radio waves of different frequencies, generates a scan image for each, detects different types of hazardous materials among the target objects based on each of the scan images, and displays the location of the hazardous materials on the display device. Hazardous materials detection system.

2. The scanned image is corrected so that the position of the object shown in the scanned image matches the position of the object shown in the captured image. The hazardous materials detection system according to claim 1.

3. The scanned image is generated by scanning the receiving direction in the height direction, By performing a conversion that multiplies the receiving direction by a numerical value corresponding to the angle, the distortion caused by scanning the receiving direction in the scanned image in the height direction is corrected. The hazardous materials detection system according to claim 2.

4. The display device displays the scanned image and the captured image in a way that allows the user to compare them. The hazardous materials detection system according to claim 1 or 2.

5. The system extracts a region representing a person from the captured image and displays the location of that region on the display device. The hazardous materials detection system according to claim 1.

6. A first extraction unit that binarizes the scanned image and compares the number of consecutive pixels in the vertical direction and the number of consecutive pixels in the horizontal direction with thresholds to extract a region representing the person from the scanned image, The system includes a second extraction unit that extracts a region representing a person from the captured image using a trained model, The hazardous materials detection system according to claim 5.

Images