Alignment marker and camera system

Abstract

To provide an alignment marker that can increase the accuracy of an alignment when a terahertz camera and a visualization camera are aligned with each other.SOLUTION: The alignment marker is an alignment marker for making a terahertz camera unit for acquiring an image formed by a terahertz wave and a visualization camera unit for acquiring an image formed by visible light in alignment with each other, and includes a background unit formed of a first material and a pattern unit formed of a second material and having an irregular shape which makes a terahertz wave not mirror-reflected.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an alignment marker for aligning a terahertz camera with a visible light camera. [Background technology]

[0002] Inspection techniques using terahertz waves, which are defined as electromagnetic waves having a frequency of 30 GHz or more and 30 THz or less, have been proposed. Patent Document 1 discloses a method for inspecting for prohibited drugs enclosed in a sealed letter without opening it, by utilizing the absorption spectrum of prohibited drugs such as narcotics in the terahertz band. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2004-286716 Summary of the Invention [Problem to be solved by the invention]

[0004] The terahertz image formed by terahertz waves used in the method of Patent Document 1 is suitable for detecting dangerous objects. However, a visible image formed by visible light is more suitable for grasping the characteristics of an object to be inspected. Therefore, it is preferable to realize a configuration in which the terahertz image and the visible image are superimposed and displayed so that the object to be inspected and the dangerous object can be visually recognized at the same time.

[0005] A conventional method uses alignment markers to superimpose and display multiple images acquired by multiple cameras. However, if a conventional alignment marker is used to align a terahertz camera that acquires terahertz images with a visible light camera that acquires visible images, the marker image cannot be accurately captured in the terahertz image, resulting in reduced alignment accuracy.

[0006] An object of the present invention is to provide an alignment marker that can improve the alignment accuracy when aligning a terahertz camera with a visible light camera. [Means for solving the problem]

[0007] An alignment marker according to one aspect of the present invention is an alignment marker for aligning a terahertz camera unit that acquires an image formed by terahertz waves with a visible light camera unit that acquires an image formed by visible light, The device has a first layer that transmits terahertz waves and is used for aligning the visible light camera unit, and a second layer that is used for aligning the terahertz camera unit, and the second layer is made of a first material that absorbs the terahertz waves. background part and a second material that reflects terahertz waves. With pattern part The pattern portion has an uneven shape that prevents specular reflection of the terahertz wave. It is characterized by: [Effects of the Invention]

[0008] According to the present invention, it is possible to provide an alignment marker that can improve the alignment accuracy when aligning a terahertz camera with a visible light camera. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is an explanatory diagram of a configuration of a camera system according to an embodiment of the present invention. [Figure 2] 10 is a flowchart illustrating a registration method. [Figure 3] FIG. 10 is a diagram illustrating an example of an alignment marker. [Figure 4] 1A and 1B are diagrams illustrating the state of reflection of terahertz waves. [Figure 5] FIG. 10 is a diagram showing another example of an alignment marker. [Figure 6] FIG. 10 is a diagram showing another example of an alignment marker. [Figure 7] FIG. 10 is a diagram illustrating an example of image processing. DETAILED DESCRIPTION OF THE INVENTION

[0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are used to designate the same components, and redundant explanations will be omitted.

[0011] 1A and 1B are explanatory diagrams illustrating the configuration of a camera system according to an embodiment of the present invention. Fig. 1A is a diagram illustrating the camera system as seen from the side. Fig. 1B is a diagram illustrating an illumination unit 110 as seen from above.

[0012] The camera system includes an illumination unit 110, a terahertz camera unit 120, a visible camera unit 130, and a processor (control unit) 170. The illumination unit 110, the terahertz camera unit 120, and the visible camera unit 130 are arranged facing an inspection object 160. In this embodiment, the illumination unit 110 is arranged under a floor 140, and the terahertz camera unit 120 and the visible camera unit 130 are arranged side by side on a ceiling 150. Alternatively, the illumination unit 110 may be arranged on the ceiling 150, and the terahertz camera unit 120 and the visible camera unit 130 may be arranged under the floor 140. Alternatively, at least one of the illumination unit 110, the terahertz camera unit 120, and the visible camera unit 130 may be arranged on a side wall. The floor 140 and the ceiling 150 are preferably made of a material that is transparent to terahertz waves, such as polyethylene.

[0013] The illumination unit 110 emits terahertz waves. As shown in FIG. 1(b), the illumination unit 110 includes a plurality of illumination elements 111 arranged two-dimensionally on the xy plane. In FIG. 1(b), the plurality of illumination elements 111 are arranged in six rows and five columns along the x-axis direction and the y-axis direction. The number of illumination elements 111 may be multiple or may be one.

[0014] The terahertz camera unit 120 acquires a terahertz image 121 formed by terahertz waves. The visible light camera unit 130 acquires a visible image 131 formed by visible light.

[0015] The terahertz waves emitted from the illumination unit 110 are specularly reflected by the inspection object 160 and enter the terahertz camera unit 120. The inspection object 160 is usually a human, but may also be a non-human animal or a robot. Because terahertz waves can pass through fabrics and the like, the processor 170 can detect dangerous objects concealed under clothing based on the terahertz image 121 acquired by the terahertz camera unit 120.

[0016] FIG. 2 is a flowchart showing a method for aligning the terahertz camera unit 120 and the visible light camera unit 130.

[0017] In step S11, the processor 170 detects the alignment markers in the visible image 131 by, for example, a pattern matching method or a method using a neural network. As preprocessing for this step, the processor 170 may perform noise removal processing on the visible image 131. For example, a bilateral filter or a median filter may be used for the noise removal processing. In addition, in this embodiment, as will be described later, metal is used for the alignment markers, and therefore, the detection accuracy of the alignment markers may be reduced due to metallic luster. Therefore, the influence of metallic luster may be reduced by using a polarizing filter or performing image processing such as dodging.

[0018] In step S12, processor 170 detects alignment markers in terahertz image 121 using the same method as in step S11. As preprocessing for this step, processor 170 may perform binarization or smoothing on terahertz image 121. This can suppress illumination unevenness and noise, allowing for more optimal detection of alignment markers. For example, a bilateral filter, a median filter, or the like may be used for the smoothing process.

[0019] In steps S11 and S12, processor 170 specifically obtains a rectangle that indicates the periphery of the alignment marker.

[0020] In step S13, processor 170 obtains the coordinates of the four corners of the alignment marker detected in steps S11 and S12.

[0021] In step S14, the processor 170 calculates a projection transformation matrix H that satisfies the following equation (1) as a parameter for alignment. In this embodiment, the position of the terahertz image 121 is aligned with the position of the visible image 131. Note that x vi and y vi indicates the ith coordinate of the four corners of the alignment marker in the visible image 131, and x ti and y ti indicates the ith coordinate of the four corners of the alignment marker of the terahertz image 121.

[0022]

number

[0023] In step S15, the processor 170 applies the projective transformation matrix H calculated in step S14 to the terahertz image 121 to perform alignment.

[0024] In the camera system of this embodiment, alignment parameters (projection transformation matrix H) for the terahertz image 121 and the visible image 131 are acquired in advance using alignment markers. When actually capturing an image of the inspection object 160, the alignment parameters acquired in advance are used to perform alignment, thereby enabling the terahertz image 121 and the visible image 131 to be superimposed and displayed. By acquiring the alignment parameters in advance, it is not necessary to capture an image of the alignment marker simultaneously with the inspection object 160, thereby improving the convenience of the inspection.

[0025] The camera system of this embodiment may also be configured as a so-called walk-through configuration in which inspection is performed while the inspection object 160 is moved in the y-axis direction in Fig. 1. In this case, the alignment parameters differ depending on the position of the inspection object 160 in the y-axis direction, so alignment parameters are acquired at multiple positions by repeating the processes of steps S11 to S15 while moving the alignment marker in the y-axis direction. This allows the terahertz image 121 and the visible image 131 to be superimposed and displayed even in the walk-through configuration, without capturing the alignment marker simultaneously with the inspection object 160.

[0026] In this embodiment, the position of the terahertz image 121 is aligned with the position of the visible image 131 using equation (1), but it is preferable to align the position of the visible image 131 with the position of the terahertz image 121. This is because terahertz waves have a longer wavelength than visible light, making it difficult to increase the resolution of the terahertz image 121. By performing projective transformation processing on the visible image 131, which has a relatively high resolution, rather than performing projective transformation processing on the terahertz image 121, which has a relatively low resolution, the effect of resolution degradation due to the projective transformation processing can be suppressed.

[0027] Furthermore, since the scale of calculation processing increases when the visible image 131, which has a relatively high resolution, is used as is, preprocessing such as resolution conversion may be performed on the visible image 131. In this case, it is preferable to reduce the scale of calculation processing by performing resolution conversion on the visible image 131 when detecting the alignment marker, and not to perform resolution conversion on the visible image 131 when photographing the inspection object 160, thereby preventing degradation of the resolution of the visible image 131. This is because, although information on the image itself is not necessary during alignment, it is necessary to accurately grasp the characteristics of the inspection object 160 from the visible image 131 when photographing the inspection object 160.

[0028] 3A and 3B are diagrams showing an example of an alignment marker 200 used for alignment in the camera system of this embodiment. Fig. 3A is a front view of the alignment marker 200. Fig. 3B is a cross-sectional view taken along line A-A' in Fig. 3A.

[0029] Alignment marker 200 has pattern portion 201 made of a reflective material such as metal that reflects terahertz waves, and background portion 202 made of an absorptive material such as urethane that absorbs terahertz waves. Here, "absorbs terahertz waves" means that the absorption rate of terahertz waves irradiated by illumination unit 110 is 50% or more. Furthermore, "reflects terahertz waves" means that the reflectance rate of terahertz waves irradiated by illumination unit 110 is 50% or more. The shape of alignment marker 200 is not limited to the shape shown in FIG. 3 and may be any shape that facilitates extraction of feature points. However, to prevent erroneous detection of brightness differences due to uneven illumination by illumination unit 110, alignment marker 200 is preferably rectangular.

[0030] Furthermore, in alignment marker 200, pattern portion 201 has an uneven shape that does not specularly reflect terahertz waves. Specifically, the surface roughness RMS of pattern portion 201 is 0.1 to 10 times the wavelength of the terahertz waves irradiated by illumination unit 110. For example, if the wavelength of the terahertz waves irradiated by illumination unit 110 is 0.5 THz, which is appropriate when the inspection target is a human, the surface roughness of pattern portion 201 is set to 60 μm to 6 mm. In this way, by forming an uneven shape in pattern portion 201, it is possible to improve the detection accuracy when detecting the image of alignment marker 200 from terahertz image 121.

[0031] It is preferable that the surface roughness RMS of the pattern portion 201 is 0.5 to 2 times the wavelength of the terahertz waves irradiated by the illumination portion 110. For example, when the wavelength of the terahertz waves irradiated by the illumination portion 110 is 0.5 THz, which is suitable when the inspection target is a human, it is preferable that the surface roughness of the pattern portion 201 is set to 0.3 to 1.2 mm.

[0032] Fig. 4 is a diagram illustrating the reflection of terahertz waves. Fig. 4(a) is a diagram illustrating the reflection of terahertz waves irradiated from the illumination unit 110 when the pattern unit 201 is made of a flat metal (surface roughness of 10 μm or less) and the background unit 202 is made of urethane. Fig. 4(b) is a diagram illustrating the reflection of terahertz waves irradiated from the illumination unit 110 when the alignment marker 200 described in Fig. 3 is used. That is, in Fig. 4(a), the surface of the pattern unit 201 is smooth, and in Fig. 4(b), the surface of the pattern unit 201 is uneven.

[0033] 4(a), because terahertz waves have a long wavelength, they are not scattered by uneven shapes of 10 μm or less, and specular reflection occurs. Therefore, the light emission distribution 112 of the illumination unit 110 is reflected in the terahertz image 121. As a result, illumination unevenness occurs in the terahertz image 121, and the alignment marker 200 cannot be detected accurately.

[0034] 4(b), the pattern portion 201 has an uneven shape having a size similar to the wavelength of the terahertz wave, which can scatter the terahertz wave as well. As a result, uneven illumination of the terahertz image 121 is eliminated, and the alignment marker 200 can be detected with high accuracy.

[0035] In this embodiment, the height of the pattern portion 201 in a direction perpendicular to the plane on which the terahertz waves are incident of the alignment marker 200 (the height of the pattern portion 201) is higher than the height of the background portion 202 in the perpendicular direction (the height of the background portion 202), but the present invention is not limited to this. The height of the pattern portion 201 may be lower than the height of the background portion 202, or may be the same as the height of the background portion 202. However, it is preferable that the height of the pattern portion 201 is equal to or higher than the height of the background portion 202. An explanation will be given below.

[0036] 5A and 5B are diagrams illustrating another example of the alignment marker 200. Fig. 5A is a diagram illustrating how the terahertz waves irradiated from the illumination unit 110 are reflected when the height of the pattern unit 201 is lower than the height of the background unit 202. Fig. 5B is a diagram illustrating how the terahertz waves irradiated from the illumination unit 110 are reflected when the height of the pattern unit 201 is higher than the height of the background unit 202.

[0037] 5(a), the height of the pattern portion 201 is lower than the height of the background portion 202, and therefore a part of the pattern portion 201 is in the shadow portion 203 of the background portion 202. Since the terahertz waves are not irradiated onto the shadow portion 203 from the illumination unit 110, the image of the alignment marker 200 is distorted, and the alignment marker 200 cannot be detected with high accuracy.

[0038] 5(b), the height of the pattern portion 201 is higher than the height of the background portion 202, so part of the background portion 202 becomes a shadow portion 204 of the pattern portion 201. Because the background portion 202 is made of an absorptive material that absorbs terahertz waves, the image of the alignment marker 200 does not distort even if there is no reflected light of the terahertz waves from the shadow portion 204.

[0039] As described above, in order to improve the detection accuracy of the alignment marker 200, it is preferable that the height of the pattern portion 201 is equal to or greater than the height of the background portion 202. When there is a difference between the heights of the pattern portion 201 and the background portion 202, it is preferable to set the difference between the heights of the pattern portion 201 and the background portion 202 to be 10 times or more the wavelength of the terahertz waves irradiated by the illumination unit 110. This makes it possible to suppress scattering at the edge of the pattern portion 201.

[0040] The above configuration is designed to detect metals, which are materials that reflect terahertz waves, using visible light as well. However, as mentioned above, metals are not necessarily the optimal alignment marker material for visible light due to the influence of metallic luster and other factors.

[0041] Therefore, it is preferable to configure the alignment marker as shown in FIG. 6. The alignment marker of FIG. 6 is formed of a first layer 210 on the front side made of a material such as PACS that transmits terahertz light, and a second layer 220 on the back side made of a background portion 202 that absorbs terahertz waves and a pattern portion 201 that reflects terahertz waves. The configuration of the second layer 220 is the same as that of the alignment marker 200 of FIG. 3 and serves as an alignment marker for the terahertz camera unit 120. On the other hand, the first layer 210 serves as an alignment marker for the visible light camera unit 130 and is formed of a background portion 205 (e.g., black) that absorbs visible light and a pattern portion 206 (e.g., white) that reflects visible light. Because the pattern portion 206 is not made of metal but of a material such as PACS, there is no effect of metallic luster. FIG. 6(a) is a front view of the first layer 210. FIG. 6(b) is a front view of the second layer 220. FIG. 6(c) is a cross-sectional view taken along line AA' in FIGS. 6(a) and 6(b).

[0042] That is, visible light camera unit 130 detects the position of the alignment marker in the visible image from the pattern formed on first layer 210. On the other hand, because terahertz waves pass through the first layer, terahertz camera unit 120 detects the position of the alignment marker in the terahertz image from the pattern formed on second layer 220. In this way, by forming alignment markers for visible light and alignment markers for terahertz waves separately on two layers, it is possible to achieve both high accuracy in detecting the alignment marker from the visible image and high accuracy in detecting the alignment marker from the terahertz image, which is preferable.

[0043] 6 shows an example in which pattern portion 201 and pattern portion 206 have the same shape, but they may have different shapes. Also, a configuration may be adopted in which both a pattern portion for visible light and a pattern portion for terahertz waves are formed on background portion 202 that absorbs visible light and terahertz waves.

[0044] In this embodiment, the pattern portion 201 is made of a metal that reflects terahertz waves, and the background portion 202 is made of urethane that absorbs terahertz waves, but the present invention is not limited to this. The pattern portion 201 may also be made of an absorptive material that absorbs terahertz waves, and the background portion 202 may be made of a reflective material that reflects terahertz waves. However, reflective materials are more susceptible to brightness differences due to uneven lighting and other factors than absorbent materials, which may cause erroneous detection of the alignment marker 200. Therefore, it is preferable that the background portion 202 be made of an absorptive material. In order to avoid the influence of uneven lighting and stray light from subjects other than the alignment marker 200, it is even more preferable that the pattern portion 201 be made of a reflective material and the background portion 202 be made of an absorptive material with a larger angle of view than the terahertz camera unit 120.

[0045] When the pattern portion 201 is made of a metal that reflects terahertz waves, since metals also have a high reflectance to visible light, it is preferable that the reflectance to visible light of the background portion 202 is low. Specifically, it is preferable that the average reflectance to visible light (380 nm to 780 nm) is 20% or less.

[0046] As described above, brightness differences due to uneven illumination can cause erroneous detection of the alignment marker 200, so it is preferable to reduce uneven illumination when performing alignment. Specifically, uneven illumination can be reduced by diffusing the orientation of the illumination unit 110 using a lens compatible with terahertz waves. The lens compatible with terahertz waves can be made of a material such as PACS that transmits terahertz waves.

[0047] When photographing the inspection object rather than detecting the alignment marker 200, it is preferable to prioritize increasing the irradiation intensity of the illumination unit 110 over reducing uneven illumination of the terahertz waves. In particular, when the inspection object 160 is a human, the human's reflectivity to terahertz waves is approximately 10%, so diffusing the orientation of the illumination unit 110 to reduce uneven illumination does not result in sufficiently strong reflection. On the other hand, the alignment marker 200 can be made of a material with high reflectivity to terahertz waves (e.g., metal has a reflectivity of 50% or more). Therefore, even if the orientation of the illumination unit 110 is diffused to reduce uneven illumination, sufficient reflection strength can be obtained. Therefore, it is preferable to widen the orientation angle of the terahertz waves when detecting the alignment marker 200 compared to when photographing the inspection object 160. To control the orientation angle, a lens that diffuses the terahertz waves may be provided when detecting the alignment marker 200, or a lens that condenses the terahertz waves may be provided when photographing the inspection object 160, or both may be used. Furthermore, a so-called zoom lens configuration may be used in which a plurality of lenses compatible with terahertz waves are moved in the optical axis direction, thereby making it possible to change the orientation angle of the terahertz waves.

[0048] Furthermore, the processor 170 may change the content of image processing for the terahertz image 121 using the shape of the alignment marker 200 extracted from the visible image 131. Specifically, after detecting the shape of the alignment marker 200 in the visible image 131, image processing that makes it easier to detect the shape of the alignment marker 200 may be performed on the terahertz image 121. FIG. 7 is a diagram showing an example of image processing. In FIG. 7, edge enhancement processing is performed in a direction where a boundary between the pattern portion 201 and the background portion 202 is predicted to exist, and averaging processing is performed in a direction where no boundary is predicted to exist, in accordance with the shape of the alignment marker 200 detected in the visible image 131. When edge enhancement processing is performed, for example, a Sobel filter may be applied for each direction, and when averaging processing is performed, for example, a median filter may be applied for each direction.

[0049] Furthermore, in this embodiment, the camera system 100 includes multiple lighting elements 111 and one pair of the terahertz camera unit 120 and the visible camera unit 130, but the present invention is not limited to this. The camera system 100 may include multiple pairs of the terahertz camera unit 120 and the visible camera unit 130. For example, by including multiple pairs of the terahertz camera unit 120 and the visible camera unit 130 at different positions or with different imaging directions, terahertz images with different imaging ranges and imaging directions can be acquired, thereby improving the accuracy of detecting hazardous objects. When multiple pairs of the terahertz camera unit 120 and the visible camera unit 130 are included, an image of the alignment marker 200 can be acquired by each pair of the terahertz camera and the visible camera, and alignment can be performed.

[0050] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. [Explanation of symbols]

[0051] 120 Terahertz Camera Unit 121 Terahertz Images 130 Visible Camera Unit 131 Visible Images 200 Alignment Markers 201 Pattern Section 202 Background part

Claims

1. An alignment marker for aligning a terahertz camera unit that acquires an image formed by terahertz waves and a visible light camera unit that acquires an image formed by visible light, a first layer that transmits the terahertz wave and is used for aligning the visible light camera unit; a second layer used for aligning the terahertz camera unit; the second layer has a background portion made of a first material that absorbs the terahertz waves and a pattern portion made of a second material that reflects the terahertz waves; The pattern portion has an uneven shape that prevents specular reflection of the terahertz wave.

2. 2. The alignment marker according to claim 1, wherein the average reflectance of the background portion to visible light is 20% or less.

3. an illumination unit that irradiates terahertz waves; a terahertz camera unit for acquiring a terahertz image formed by the terahertz wave; a visible light camera unit for acquiring a visible image formed by visible light; 3. A camera system comprising: a control unit that aligns the terahertz camera unit and the visible light camera unit using the alignment marker according to claim 1.

4. 4. The camera system according to claim 3, wherein the surface roughness of the pattern portion is 0.1 to 10 times the wavelength of the terahertz wave.

5. The camera system according to claim 3 or 4, wherein the height of the alignment marker of the pattern portion in a direction perpendicular to the surface on which the terahertz waves are incident is equal to or greater than the height of the background portion in the perpendicular direction.

6. 6. The camera system according to claim 5, wherein the height of the pattern portion in the orthogonal direction is 10 times or more the wavelength of the terahertz wave compared to the height of the background portion in the orthogonal direction.

7. 7. The camera system according to claim 3, wherein the control unit aligns a position of the visible image with a position of the terahertz image by performing a projective transformation process on the visible image.

8. 8. The camera system according to claim 3, wherein the control unit performs at least one of binarization and smoothing on the terahertz image.

9. 9. The camera system according to claim 3, wherein the visible light camera section acquires the visible light image through a polarizing filter.

10. 10. The camera system according to claim 3, wherein the control unit applies a dodging process to the visible image.

11. 11. The camera system according to claim 3, wherein the control unit changes the content of image processing for the terahertz image using information on the shape of the alignment marker extracted from the visible image.

12. 12. The camera system according to claim 3, wherein the control unit uses the alignment marker to acquire parameters for aligning the terahertz image and the visible image, and aligns the terahertz camera unit and the visible camera unit using the parameters when photographing the object to be inspected.

13. The camera system according to claim 12 , wherein, when acquiring the parameters, the control unit performs resolution conversion to lower the resolution of the visible image than when photographing the inspection object.

14. 14. The camera system according to claim 12, wherein the illumination unit has an orientation angle wider when acquiring the parameters than when photographing the inspection object.

15. The camera system according to claim 14 , further comprising a lens that corresponds to the terahertz wave and controls the orientation angle.

Images