Image pickup system, control method for image pickup system, computer program product, and storage medium

Abstract

The invention discloses an image pickup system, a control method of the image pickup system, a computer program product and a storage medium. The image pickup system includes: an illumination unit configured to emit a terahertz wave or light having a frequency different from that of the terahertz wave; an imaging unit configured to image a subject irradiated with a terahertz wave or light having a frequency different from that of the terahertz wave; and a control unit configured to control the imaging unit or the illumination unit for irradiating a target area according to a positional relationship between the target area to be imaged by the imaging unit and the imaging unit for imaging the target area, the position relationship between the lighting unit and the imaging unit satisfies a predetermined condition.

Description

Technical Field

[0001] The present invention relates to a camera system, a control method of the camera system and a recording medium. Background Art

[0002] In recent years, in order to prevent crimes committed by people who conceal and carry dangerous objects, there is a need for technology to detect objects carried by people (such as dangerous objects carried by these people). As a technology, an active terahertz system using terahertz waves is known. In an active terahertz system, a subject is irradiated with terahertz waves, and the terahertz waves reflected by the subject are detected by a terahertz camera, thereby performing imaging and inspection. For example, Japanese Patent Application Laid-Open No. 2021-181925 discloses: irradiating a test object with terahertz waves, acquiring a terahertz wave image from the reflected wave, and detecting a hidden object from the acquired image.

[0003] Here, for example, when imaging is performed without the lighting unit, imaging unit, or target area to be imaged being fixed, the positional relationship between the lighting unit, imaging unit, and target area to be imaged may differ each time imaging is performed. In this case, as in the technology of Japanese Patent Application Laid-Open No. 2021-181925, when the positional relationship between the lighting unit for illuminating the target area and the imaging unit for imaging the target area does not change regardless of the positional relationship between the target area and the imaging unit, there is a case where imaging is performed without a predetermined positional relationship between the lighting unit and the imaging unit. Summary of the Invention

[0004] According to the present invention, the imaging system includes: an illumination unit configured to emit terahertz waves or light having a frequency different from that of the terahertz waves; an imaging unit configured to capture an image of a subject irradiated with the terahertz waves or the light having a frequency different from that of the terahertz waves; and a control unit configured to control the imaging unit or the illumination unit for irradiating the target area based on the positional relationship between the target area to be imaged by the imaging unit and the imaging unit for capturing the target area, so that the positional relationship between the illumination unit and the imaging unit satisfies a predetermined condition.

[0005] Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 is a diagram illustrating an example of a functional configuration of an imaging system.

[0007] Figure 2 is a diagram showing the hardware configuration of an imaging system.

[0008] Figure 3A and Figure 3B This is a schematic diagram of the usage of the camera system.

[0009] Figure 4A is a diagram showing an image obtained by photographing a target object with a visible light camera, and Figure 4B 1 is a diagram showing an image obtained by capturing an image of a target object using an imaging system.

[0010] Figure 5A and Figure 5B is a diagram showing an example of arrangement of an illumination unit and an imaging unit in an imaging system.

[0011] Figure 6A and Figure 6B is a diagram showing the positional relationship among the lighting unit, the imaging unit, and the target object.

[0012] Figure 7 This is a diagram showing a lighting unit management table.

[0013] Figure 8A and Figure 8B It is a flowchart showing the flow of control processing.

[0014] Figure 9 1 is a diagram illustrating a first modification of the arrangement of the lighting unit and the imaging unit in the imaging system.

[0015] 10A to 10D 2 is a diagram showing a second modification of the arrangement of the lighting unit and the imaging unit in the imaging system.

[0016] Figure 11 is a diagram showing the configuration of an image pickup system according to a second embodiment.

[0017] Figure 12 : is a diagram showing a lighting unit management table according to the second embodiment.

[0018] Figure 13A and Figure 13B : is a flowchart showing the flow of control processing in the second embodiment. DETAILED DESCRIPTION

[0019] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

[0020] Figure 1 This figure shows an example of the functional configuration of an imaging system 100 according to the first embodiment. The imaging system 100 according to this embodiment is a system for detecting reflected light from a target object to be imaged when terahertz waves are emitted toward the target object, and for detecting an object carried by the target object from the image obtained by the imaging. Terahertz waves are electromagnetic waves with a frequency of 30 GHz or higher and 30 THz or lower.

[0021] Terahertz waves have different properties from visible light, such as higher linearity and transparency. The imaging system 100 according to this embodiment detects a target object in a non-destructive and non-contact manner by irradiating the target object with terahertz waves and imaging the target object.

[0022] In the following description, the target object to be imaged may be simply referred to as the target object. The target object may also be considered as the target object to be inspected using the imaging system 100. In the following description, the area to be imaged by the imaging system 100 may be referred to as the target area. In situations where the target object is located within the range of the area predetermined for imaging, such as when the imaging system 100 is imaging the target object, the target area is the area where the target object is located. Furthermore, in situations where the target object is not located within the range of the area predetermined for imaging, such as when the imaging system 100 is preparing to capture an image, the target area is the area predetermined as the area where the target object is located at the time of imaging. The predetermined range and predetermined area may be an area corresponding to the position provided for imaging, or may be the entire area included in the angle of view of the imaging unit 130, such as the front of the imaging unit 130.

[0023] The imaging system 100 includes an illumination unit 120 , an imaging unit 130 , a processing unit 140 , a display unit 150 , an acquisition unit 160 , a storage unit 170 , a detection unit 180 , and a control unit 110 .

[0024] The lighting unit 120 has a light-emitting device (not shown) that emits terahertz waves. An example of a light-emitting device is a device equipped with an antenna including a negative differential resistance element and a resonant circuit. In addition, an example of a negative differential resistance element is a resonant tunneling diode, etc. The light-emitting device can be any existing device as long as it is a device that emits terahertz waves. In addition, the lighting unit 120 can be provided with a plurality of light-emitting devices. When a plurality of light-emitting devices are provided in the lighting unit 120, the light-emitting intensity can be increased by resonantly driving the plurality of light-emitting devices, or the light-emitting devices can be arranged in an array to illuminate the entire target object with terahertz waves. In addition, the imaging system 100 according to this embodiment is provided with a plurality of lighting units 120. In the illustrated example, the imaging system 100 is provided with four lighting units 120. However, the number of lighting units 120 provided in the imaging system 100 is not limited to the example illustrated in the figure.

[0025] Image capture unit 130 detects the intensity of the terahertz waves emitted from illumination unit 120 and reflected by the target object, thereby capturing an image of the target object. Image capture unit 130 includes a detection device for detecting terahertz waves. Examples of detection devices include Schottky barrier diodes, bolometers, and MEMS resonators. The detection device can be any existing device as long as it is a device for detecting terahertz waves.

[0026] The processing unit 140 processes the imaging signal obtained by imaging the image capturing unit 130. More specifically, the processing unit 140 converts the imaging signal into an image signal through processing such as demosaicing and black level adjustment. Furthermore, when converting the imaging signal, the processing unit 140 performs image correction such as defect correction and shading correction. Furthermore, the processing unit 140 can perform image processing such as noise removal and edge extraction, as well as object recognition through image recognition.

[0027] The display unit 150 displays the image processed by the processing unit 140. In addition, the display unit 150 can display the result of the image recognition performed by the processing unit 140. In addition, when the result of the image recognition performed by the processing unit 140 is displayed on the display unit 150, if a predetermined object is image-recognized by the processing unit 140, the recognized object can be displayed on the display unit 150 in an emphasized manner.

[0028] The acquisition unit 160 acquires information from an external device of the imaging system 100 .

[0029] The storage unit 170 stores information acquired by the acquisition unit 160 and information input to the imaging system 100. The contents of the information stored in the storage unit 170 will be described in detail later.

[0030] The detection unit 180 detects the positional relationship between the lighting unit 120, the imaging unit 130, and the target object. The detection unit 180 may include a distance sensor (not shown) for measuring distances, and may use this sensor to measure the distance from the target object to the imaging unit 130 and the distance from the target object to the lighting unit 120, thereby detecting the positional relationship between the lighting unit 120, the imaging unit 130, and the target object. Furthermore, when the lighting unit 120 transmits a terahertz wave toward the target object, the detection unit 180 may identify the position of the target object and the distance from the target object to the imaging unit 130 based on the intensity of the reflected wave reflected by the target object and detected by the imaging unit 130. Furthermore, the detection unit 180 may include a position sensor (not shown) for detecting a two-dimensional or three-dimensional area where an object is located, and may use this position sensor to detect the positional relationship between the lighting unit 120, the imaging unit 130, and the target object. The position sensor may be a sensor for optically detecting the area where the object is located, or a sensor for magnetically detecting the area where the object is located.

[0031] The object to be detected by the detection unit 180 may be predetermined, or may be any object located in an area that can be detected by the detection unit 180. In addition, the detection unit 180 transmits information indicating the detection result to the control unit 110.

[0032] The control unit 110 controls the illumination unit 120, the imaging unit 130, and the detection unit 180. When the control unit 110 acquires detection results from the detection unit 180, it identifies, from the acquired results, which of the multiple illumination units 120 will cause the reflected wave from the target object to be incident on the detection device of the imaging unit 130 when the terahertz wave is emitted toward the target object. The identified illumination unit 120 is then determined as the illumination unit 120 to be used to illuminate the target object with the terahertz wave. The control unit 110 controls the switching between on and off of the illumination unit 120 and controls, for example, the intensity of light emitted by the terahertz wave when the illumination unit 120 emits the terahertz wave. Examples of the intensity of light emitted by the terahertz wave include the luminous intensity of the terahertz wave emitted from the illumination unit 120, the beam of the terahertz wave emitted from the illumination unit 120, and the illuminance at the location where the light is emitted by the terahertz wave emitted from the illumination unit 120. The control unit 110 also controls the exposure time, aperture, focus, etc. in the imaging unit 130. In addition, the control unit 110 controls whether the detection unit 180 is activated.

[0033] The processing of the control unit 110 and the processing unit 140 is implemented by a processing device such as a CPU or an ISP, etc. The control unit 110 and the processing unit 140 may be configured by the same processing device, or may be configured by separate processing devices.

[0034] In the imaging system 100, the lighting unit 120 and the imaging unit 130 may be integrally provided or independently provided as separate devices. When the lighting unit 120 and the imaging unit 130 are integrally provided, the positional relationship between the lighting unit 120 and the imaging unit 130 may be fixed. Furthermore, when the lighting unit 120 and the imaging unit 130 are independently provided, the positional relationship between the lighting unit 120 and the imaging unit 130 may be changed by changing the position of one of the lighting unit 120 and the imaging unit 130.

[0035] In the illustrated example, the camera system 100 is provided with a processing unit 140 and a display unit 150, but the present invention is not limited thereto. The processing unit 140 and the display unit 150 may be provided in a device external to the camera system 100, and the camera system 100 may transmit information obtained by capturing images with the camera unit 130 to the processing unit 140 and the display unit 150 via a network. In this case, the network connecting the camera system 100 with the processing unit 140 and the display unit 150 may be any network configured to enable transmission and reception of information. The network may be the Internet, a local area network (LAN), a wide area network (WAN), a cellular network such as LTE or 5G, a wireless network, a dedicated digital line, Bluetooth (registered trademark), Bluetooth low energy, or the like, or a combination thereof.

[0036] Figure 2 is a diagram showing the hardware configuration of the camera system 100. The camera system 100 includes a CPU 101, a storage device 102, a memory 103, an operation I / F unit 104, and a communication I / F unit 105. The CPU 101 controls the entire camera system 100. The CPU 101 loads a program from the storage device 102 into the memory 103 and executes the program to implement various controls. The storage device 102 stores an operating system (OS), programs, management data, data collected from external systems and devices, and the like. The memory 103 serves as a work area for the CPU 101. The operation I / F unit 104 outputs various types of data and execution results of the program, etc. to an output device such as a connected display, and receives input from a connected input device. The communication I / F unit 105 is a network interface for communicating with external systems and devices. Figure 2 The illustrated configuration can also be regarded as a hardware configuration of the control unit 110 in the camera system 100 .

[0037] Figure 3A and Figure 3B Schematic diagram of a usage form of the imaging system 100 . Figure 3A and Figure 3BAn example is shown in which the camera system 100 captures an image of a target object 200 as a person to be inspected and detects an object that the target object 200 is concealing.

[0038] Figure 3A An example is shown in which the camera system 100 is provided with a holding unit 210 and an inspector who inspects the target object 200 uses the camera system 100 by holding the holding unit 210. The inspector can use the camera system 100 to preferentially photograph portions of the target object 200 that are more likely to contain dangerous objects, or can capture an image of the target object 200 so that the entire target object 200 is contained within the field of view of the camera unit 130. Examples of places where such a camera system 100 is used include airport security checkpoints and places at venues where access is controlled. The inspector uses the camera system 100 to detect objects that are hidden and carried by the target object 200, thereby preventing accidents that would be caused by the introduction of dangerous objects by the target object 200 in advance. The target object 200 is not limited to a person, but can be any object that is carried, such as a bag, a carton, an envelope, etc.

[0039] like Figure 3B As shown, in the imaging system 100, when at least one of the plurality of lighting units 120 illuminates the target object 200, the imaging unit 130 detects a reflection from the target object 200. An image obtained based on the detection result of the imaging unit 130 is then processed by the processing unit 140 and displayed on the display unit 150. The inspector checks the image displayed on the display unit 150 to identify the object carried by the target object 200.

[0040] Figure 4A 1 is a diagram showing an image obtained by capturing an image of the target object 200 by a camera (hereinafter referred to as a visible light camera) that captures an image by detecting a reflected wave from the target object 200 when visible light is emitted to the target object 200 . Figure 4B is a diagram showing an image obtained by capturing an image of a target object 200 by the imaging system 100 according to this embodiment. Figure 4A and Figure 4B In the illustrated example, it is assumed that the target object 200 has a knife 310 and a bomb 320 hidden inside his / her clothes.

[0041] When the visible light camera illuminates the target object 200 with visible light, the visible light is reflected by the clothes worn by the target object 200 before reaching the knife 310 or the bomb 320. In this case, Figure 4A As shown, the knife 310 and the bomb 320 are not displayed in the image captured by the visible light camera, and it is difficult for the inspector to identify the knife 310 or the bomb 320 .

[0042] On the other hand, when the camera system 100 irradiates the target object 200 with terahertz waves, the terahertz waves pass through the clothes of the target object 200 and are reflected by the knife 310 and the bomb 320. In this case, as Figure 4B As shown, a knife 310 and a bomb 320 are displayed in an image captured by the camera system 100 , and the inspector can recognize from the image that the target object 200 is carrying the knife 310 or the bomb 320 .

[0043] Figure 5A and Figure 5B 1 is a diagram showing an example of arrangement of the lighting unit 120 and the imaging unit 130 in the imaging system 100. In the following description, Figure 5A and Figure 5B The left and right directions in can be called X directions. Figure 5A and Figure 5B The front-to-back direction in can be called the Y direction, and Figure 5A and Figure 5B The up and down direction in can be called the Z direction. Figure 5A and Figure 5B The left side in the figure can be called the upstream side in the X direction, and Figure 5A and Figure 5B The right side in can be referred to as the downstream side in the X direction. Figure 5A and Figure 5B The front side in the Y direction can be called the upstream side, and Figure 5A and Figure 5B The rear side in the Y direction can be referred to as the downstream side. Figure 5A and Figure 5B The lower side in the Z direction can be called the upstream side, and Figure 5A and Figure 5B The upper side in can be called the downstream side in the Z direction. Figure 5A and Figure 5B In the illustrated example, both the direction in which the illumination unit 120 emits light and the direction in which the imaging unit 130 faces the subject are the Y direction.

[0044] exist Figure 5A In the example shown, in the imaging system 100, four lighting units 120 and an imaging unit 130 are arranged horizontally in the X direction, with the imaging unit 130 located to the right of each lighting unit 120. In the following description, the four lighting units 120 will be referred to as a first lighting unit 120A, a second lighting unit 120B, a third lighting unit 120C, and a fourth lighting unit 120D, respectively. Furthermore, when describing the first lighting unit 120A, the second lighting unit 120B, the third lighting unit 120C, and the fourth lighting unit 120D without making any particular distinction between them, they will simply be referred to as lighting units 120.

[0045] In the illustrated example, the first lighting unit 120A, the second lighting unit 120B, the third lighting unit 120C, the fourth lighting unit 120D, and the imaging unit 130 are arranged in this order from upstream to downstream in the X direction. The distance from the imaging unit 130 to the fourth lighting unit 120D is distance X1. Furthermore, the distance from the imaging unit 130 to the third lighting unit 120C is distance X2, which is longer than distance X1. Furthermore, the distance from the imaging unit 130 to the second lighting unit 120B is distance X3, which is longer than distance X2. Furthermore, the distance from the imaging unit 130 to the first lighting unit 120A is distance X4, which is longer than distance X3.

[0046] exist Figure 5B In the illustrated example, in the imaging system 100, four lighting units 120 and the imaging unit 130 are arranged horizontally in the X direction, with two lighting units 120 located upstream and downstream of the imaging unit 130 in the X direction, respectively. The four lighting units 120 are arranged symmetrically with respect to the imaging unit 130. In the illustrated example, the first lighting unit 120A, the second lighting unit 120B, the imaging unit 130, the third lighting unit 120C, and the fourth lighting unit 120D are arranged in this order from upstream to downstream in the X direction. Furthermore, the distance from the imaging unit 130 to the second lighting unit 120B and the distance from the imaging unit 130 to the third lighting unit 120C are both distances X5. Furthermore, the distance from the imaging unit 130 to the first lighting unit 120A and the distance from the imaging unit 130 to the fourth lighting unit 120D are both distances X6, which are longer than distance X5.

[0047] exist Figure 5A and Figure 5B In the illustrated example, the positions of the four lighting units 120 and the imaging units 130 are aligned in the Y direction and the Z direction in any case.

[0048] Figure 6A and Figure 6B 1 is a diagram illustrating the positional relationship among the lighting unit 120 , the imaging unit 130 , and the target object 200 . Figure 6A and Figure 6B 1 and 2 are views of the imaging system 100 and the target object 200 as viewed from above (upstream in the Z direction). Figure 6A and Figure 6B The X, Y, and Z directions in Figure 5A and Figure 5B In addition, assuming that Figure 6A and Figure 6B The camera system 100 shown is Figure 5B The camera system 100 shown in FIG. Figure 6Aand Figure 6B In FIG. 1 , it is assumed that the target object 200 has a shape symmetrical in the X direction with respect to a position overlapping with the center of the imaging unit 130 .

[0049] In addition, Figure 6A and Figure 6B In the imaging system 100 shown, it is assumed that the directions in which the terahertz waves are emitted by the respective illumination units 120 are predetermined. More specifically, the first illumination unit 120A and the second illumination unit 120B emit terahertz waves toward the downstream side in the Y direction and the downstream side in the X direction, and the third illumination unit 120C and the fourth illumination unit 120D emit terahertz waves toward the downstream side in the Y direction and the upstream side in the X direction.

[0050] exist Figure 6A In the example shown, it is assumed that the distance (the distance in the Y direction) from each of the lighting units 120 and the imaging unit 130 to the target object 200 is distance Y1. Here, when the first lighting unit 120A emits a terahertz wave, the emitted terahertz wave is reflected by the target object 200. In this case, the incident angle a1 and the reflection angle a2 of the terahertz wave relative to the target object 200 are equal. In addition, since terahertz waves have higher linearity than visible light, etc., when the lighting unit 120 emits a terahertz wave, specular reflection occurs with the incident angle and the reflection angle being equal. In addition, since terahertz waves are less prone to diffuse reflection than visible light, etc., in order to capture an image using terahertz waves, the detection device of the imaging unit 130 needs to detect the specularly reflected terahertz wave. In the following description, when describing the reflected terahertz wave, it is assumed that the terahertz wave is the specularly reflected terahertz wave.

[0051] exist Figure 6A In the illustrated example, the terahertz wave emitted from the first illumination unit 120A and reflected by the target object 200 is incident on the detection device of the imaging unit 130. On the other hand, when the second illumination unit 120B emits a terahertz wave, the emitted terahertz wave passes downstream in the X direction relative to the imaging unit 130 after being reflected by the target object 200, and therefore does not enter the detection device of the imaging unit 130.

[0052] In this case, the first lighting unit 120A and the imaging unit 130 are in a positional relationship in which the terahertz wave emitted from the lighting unit 120 and reflected by the target object 200 is detected by the detection device of the imaging unit 130. On the other hand, the second lighting unit 120B and the imaging unit 130 are not in a positional relationship in which the terahertz wave emitted from the lighting unit 120 and reflected by the target object 200 is detected by the detection device of the imaging unit 130. Therefore, it is preferable that the control unit 110 causes the first lighting unit 120A to irradiate the target object 200 with the terahertz wave, but does not cause the second lighting unit 120B to irradiate the target object 200 with the terahertz wave.

[0053] Although not shown, the third lighting unit 120C and the imaging unit 130 are not in a positional relationship in which the terahertz wave emitted from the lighting unit 120 and reflected by the target object 200 is detected by the detection device of the imaging unit 130. On the other hand, the fourth lighting unit 120D and the imaging unit 130 are in a positional relationship in which the terahertz wave emitted from the lighting unit 120 and reflected by the target object 200 is detected by the detection device of the imaging unit 130. Therefore, it is preferable that the control unit 110 not cause the third lighting unit 120C to irradiate the target object 200 with the terahertz wave, but cause the fourth lighting unit 120D to irradiate the target object 200 with the terahertz wave.

[0054] exist Figure 6B In the example shown, it is assumed that the distance (in the Y direction) from each of the lighting unit 120 and the imaging unit 130 to the target object 200 is a distance Y2 shorter than the distance Y1. Here, when the first lighting unit 120A emits a terahertz wave, the emitted terahertz wave passes upstream in the X direction relative to the imaging unit 130 after being reflected by the target object 200, and therefore does not enter the detection device of the imaging unit 130. In addition, Figure 6B In the example shown, the distance from the lighting unit 120 and the imaging unit 130 to the target object 200 is greater than the distance from the lighting unit 120 and the imaging unit 130 to the target object 200. Figure 6A The distance in the illustrated example is short. Therefore, the position at which the terahertz wave emitted from the first illumination unit 120A and reflected by the target object 200 is incident on the imaging system 100 changes. Furthermore, when the second illumination unit 120B emits a terahertz wave, the emitted terahertz wave is reflected by the target object 200 and then incident on the detection device of the imaging unit 130.

[0055] In this case, the first lighting unit 120A and the imaging unit 130 are not in a positional relationship in which the terahertz wave emitted from the lighting unit 120 and reflected by the target object 200 is detected by the detection device of the imaging unit 130. On the other hand, the second lighting unit 120B and the imaging unit 130 are in a positional relationship in which the terahertz wave emitted from the lighting unit 120 and reflected by the target object 200 is detected by the detection device of the imaging unit 130. Therefore, it is preferable that the control unit 110 not cause the first lighting unit 120A to irradiate the target object 200 with the terahertz wave, but cause the second lighting unit 120B to irradiate the target object 200 with the terahertz wave.

[0056] Although not shown, the third lighting unit 120C and the imaging unit 130 are in a positional relationship in which the terahertz wave emitted from the lighting unit 120 and reflected by the target object 200 is detected by the detection device of the imaging unit 130. On the other hand, the fourth lighting unit 120D and the imaging unit 130 are not in a positional relationship in which the terahertz wave emitted from the lighting unit 120 and reflected by the target object 200 is detected by the detection device of the imaging unit 130. Therefore, it is preferable that the control unit 110 causes the third lighting unit 120C to irradiate the target object 200 with the terahertz wave, but does not cause the fourth lighting unit 120D to irradiate the target object 200 with the terahertz wave.

[0057] Thus, when the target object 200 is illuminated by any one of the plurality of illumination units 120, whether the terahertz wave reflected by the target object 200 is incident on the detection device of the imaging unit 130 may vary depending on the positional relationship between the imaging system 100 and the target object 200. Therefore, in this embodiment, the control unit 110 determines which of the plurality of illumination units 120 is to emit the terahertz wave based on the positional relationship between the imaging system 100 and the target object 200.

[0058] exist Figure 6A and Figure 6B In the illustrated example, it is assumed that both the terahertz wave incident on the target object 200 from the illumination unit 120 and the terahertz wave reflected by the target object 200 travel perpendicularly to the Z direction, in other words, travel horizontally to the X and Y directions. On the other hand, there are cases where the terahertz wave from the illumination unit 120 is emitted in a direction that is not horizontal to any of the X, Y, and Z directions. In such cases, the control unit 110 determines which of the multiple illumination units 120 is to emit the terahertz wave based on the positional relationship between the illumination unit 120, the imaging unit 130, and the target object 200 in a three-dimensional area defined by the coordinates in the X, Y, and Z directions.

[0059] Figure 7 1 is a diagram showing a lighting unit management table. The lighting unit management table is a table for managing the illumination of the lighting unit 120. The lighting unit management table is stored in the storage unit 170 (see FIG. 170 ) of the camera system 100. Figure 1 ). In addition, assuming Figure 7 The lighting unit management table shown indicates Figure 5A The lighting unit 120 is shown as the target information.

[0060] The contents of the lighting unit management table will be described in detail.

[0061] The "lighting unit" is information for identifying the target lighting unit 120 in the imaging system 100. In the illustrated example, the "lighting unit" is indicated as "120A," "120B," "120C," or "120D," thereby identifying which of the first to fourth lighting units 120A to 120D the information indicated in the lighting unit management table is targeted for.

[0062] The “distance” is the distance from the lighting unit 120 to the imaging unit 130 .

[0063] The "irradiation area" is a target area that meets the illumination condition. The illumination condition is a predetermined condition related to the positional relationship between the lighting unit 120 and the imaging unit 130. The control unit 110 uses the illumination condition to determine from which lighting unit 120 the target object will be illuminated with the terahertz wave. In this embodiment, the lighting unit 120 and the imaging unit 130 have a specific positional relationship, which is defined as the illumination condition. The specific positional relationship is the positional relationship between the lighting unit 120 and the imaging unit 130, wherein the terahertz wave emitted from the lighting unit 120 and reflected by the target object is incident on the detection device of the imaging unit 130. In the example shown, the "irradiation area" shows the target area that meets the illumination condition as a three-dimensional area range defined by coordinates in the X, Y, and Z directions.

[0064] The "Illumination Level" shows an indicator indicating the intensity of light emitted by the terahertz waves emitted from the illumination unit 120. A larger value shown in the "Illumination Level" indicates a higher intensity of light emitted by the terahertz waves emitted from the illumination unit 120. Similarly, as shown in the figure, the greater the distance from the imaging unit 130 to the "Illumination Unit", the higher the "Illumination Level" associated therewith.

[0065] An example of the contents written into the lighting unit management table will be described. The first lighting unit 120A, namely "120A", is associated with "X4" as the "distance", "x1, y1, z1 to xa, ya, za" as the "irradiation area", and "4" as the "irradiation level".

[0066] The information shown in the lighting unit management table may be information input to the camera system 100 by a user of the camera system 100 , or may be information recognized by the control unit 110 .

[0067] Figure 8A and Figure 8B1 is a flowchart showing the flow of control processing. The control processing is a process in which the control unit 110 of the camera system 100 controls the lighting unit 120 and the camera unit 130. In this embodiment, for example, a user of the camera system 100 (such as an inspector, etc.) operates an operation unit (not shown) of the camera system 100 to start the control processing. In addition, for example, the control processing can be started when the camera system 100 is powered on. In this embodiment, the CPU 101 (see Figure 2 ) Loads the program stored in the storage device 102 into the memory 103 and executes the program, thereby realizing each process in the control process.

[0068] The control unit 110 operates the imaging unit 130 and the detection unit 180 (step (hereinafter sometimes referred to as "S") 101). More specifically, the control unit 110 places the imaging unit 130 in a state where it can capture an image and causes the detection unit 180 to detect a target object.

[0069] The control unit 110 determines whether the detection unit 180 has detected the target object (S102). Examples of cases where the detection unit 180 has not detected the target object include a case where the target object is not within the range within which the detection unit 180 can detect the target object. While the negative result is maintained in step 102, the control unit 110 repeats the processing of step 102.

[0070] When the control unit 110 determines that the detection unit 180 has detected the target object ("Yes" in S102), the control unit 110 determines whether the positional relationship between the lighting unit 120 and the imaging unit 130 satisfies the illumination condition for each lighting unit 120 (S103). The control unit 110 determines whether the positional relationship between the lighting unit 120 and the imaging unit 130 satisfies the illumination condition based on whether the target area is included in the range of the area indicated by the "Illumination Area" in the lighting unit management table. In this case, the target area is the area detected by the detection unit 180 as the area where the target object is located.

[0071] When the control unit 110 determines that the positional relationship between the lighting unit 120 and the imaging unit 130 does not satisfy the illumination condition for any of the lighting units 120 ("No" in S103), the control unit 110 determines whether to cause the detection unit 180 to continue detecting the target object (S104). In this case, information indicating that the illumination condition is not satisfied and information for selecting whether to cause the detection unit 180 to continue detecting the target object may be displayed on the display unit 150 (see Figure 1 Then, in response to the user's operation of the operation unit (not shown) of the camera system 100, the control unit 110 may determine whether to cause the detection unit 180 to continue detecting the target object.

[0072] If the control unit 110 determines that the detection unit 180 should not continue to detect the target object ("No" in S104), the control unit 110 terminates the operation of the imaging unit 130 and the detection unit 180 (S105), and the control process ends. More specifically, the control unit 110 places the imaging unit 130 in a state in which it cannot capture images, and places the detection unit 180 in a state in which it cannot detect the target object.

[0073] Furthermore, when the control unit 110 determines to cause the detection unit 180 to continue detecting the target object (YES in S104 ), the processing from step 102 is performed again.

[0074] In addition, when the control unit 110 determines that the positional relationship between the lighting unit 120 and the imaging unit 130 satisfies the illumination condition for any lighting unit 120 ("Yes" in S103), the control unit 110 turns on the target lighting unit 120 that satisfies the illumination condition (S106). More specifically, the control unit 110 causes the lighting unit 120 located at the position that satisfies the illumination condition among the multiple lighting units 120 to emit a terahertz wave. As a result, the terahertz wave emitted from the target lighting unit 120 that satisfies the illumination condition and reflected by the target object is incident on the detection device of the imaging unit 130, and at this time the imaging unit 130 photographs the target object, thereby obtaining an image showing the object carried by the target object. At this time, the control unit 110 controls the intensity of light emitted by the terahertz wave emitted from the lighting unit 120 according to the distance from the lighting unit 120 located at the position that satisfies the illumination condition to the imaging unit 130. More specifically, the control unit 110 causes the lighting unit 120 located at the position that satisfies the illumination condition to emit a terahertz wave in the lighting unit management table (see Figure 7 ) for the lighting unit 120 to emit terahertz waves. The target lighting unit 120 that meets the illumination condition can be referred to as a target unit.

[0075] Furthermore, when there are a plurality of lighting units 120 located at positions satisfying the irradiation condition, the control unit 110 causes all lighting units 120 satisfying the irradiation condition to emit terahertz waves.

[0076] The control unit 110 determines whether the detection unit 180 detects the target object again (S107). More specifically, the control unit 110 causes the detection unit 180 to detect the target object again, and determines whether the detection unit 180 has detected the target object again.

[0077] When the detection unit 180 detects the target object again ("Yes" in S107), the control unit 110 determines whether the positional relationship between the target unit, the imaging unit 130, and the target object has changed (S108). The control unit 110 determines whether the positional relationship has changed based on whether the area detected by the detection unit 180 as the area where the target object is located is no longer included in the range of the area shown in the "illumination area" of the lighting unit management table for the lighting unit 120 as the target unit.

[0078] When the control unit 110 determines that the positional relationship among the target unit, the imaging unit 130 , and the target object has not changed (No in S108 ), the processing from step 107 is repeated.

[0079] In addition, when the control unit 110 determines that the positional relationship between the target unit, the camera unit 130 and the target object has changed ("Yes" in S108), the control unit 110 determines whether the positional relationship between the lighting unit 120 and the camera unit 130 meets the illumination condition for each lighting unit 120 (S109).

[0080] When a negative result is obtained in step 107 or step 109, the control unit 110 turns off the target unit turned on in step 106 (S110). More specifically, the control unit 110 causes the illumination unit 120 that has emitted the terahertz wave in step 106 to end emission of the terahertz wave.

[0081] The control unit 110 determines whether to continue detecting the target object with the detection unit 180 (S111). If the control unit 110 determines not to continue detecting the target object with the detection unit 180 ("No" in S111), the control unit 110 terminates the operation of the imaging unit 130 and the detection unit 180 (S112), and the control process ends. The processes in step 111 and step 112 are the same as those in step 104 and step 105, respectively.

[0082] Furthermore, when the control unit 110 determines to cause the detection unit 180 to continue detecting the target object (YES in S111 ), the processing from S102 is performed again.

[0083] Furthermore, when the control unit 110 determines that the positional relationship between the lighting unit 120 and the imaging unit 130 satisfies the illumination condition for any lighting unit 120 ("Yes" in S109), the control unit 110 turns off the target unit turned on in step 106 (S113). The processing in step 113 is the same as the processing in step 110.

[0084] The control unit 110 turns on the target unit that has newly met the illumination conditions (S114). More specifically, the control unit 110 causes the illumination unit 120, located at the position that has newly met the illumination conditions, among the multiple illumination units 120 to emit terahertz waves. As a result, even if the positional relationship between the illumination unit 120, the imaging unit 130, and the target object changes, the terahertz waves emitted from the new illumination unit 120 and reflected by the target object are incident on the imaging unit 130. After step 114, the process from step 107 onwards is repeated.

[0085] As described above, the control unit 110 controls the lighting unit 120 according to the positional relationship between the target area to be photographed by the imaging unit 130 and the imaging unit 130, so that the positional relationship between the lighting unit 120 for illuminating the target area and the imaging unit 130 for photographing the target area satisfies the illumination condition.

[0086] In this case, even if the positional relationship between the lighting unit 120, the imaging unit 130 and the target area is different each time imaging is performed, imaging can be achieved under a positional relationship that satisfies the illumination conditions between the lighting unit 120 for illuminating the target area and the imaging unit 130 for imaging the target area.

[0087] Furthermore, in this embodiment, the lighting unit 120 includes a plurality of lighting units 120 located at different positions, and the control unit 110 causes the lighting unit 120 located at a position that satisfies the irradiation condition to irradiate the target area. In this case, compared to when terahertz waves are emitted from the lighting unit 120 located at a position that does not satisfy the irradiation condition, the power consumed by the lighting unit 120 can be reduced.

[0088] In addition, when the positional relationship between the target area and the imaging unit 130 changes, the control unit 110 controls the lighting unit 120 so that the illumination condition (see FIG. 1 ) is satisfied after the change. Figure 8B In this case, even if the positional relationship between the lighting unit 120, the imaging unit 130, and the target area changes, imaging can be achieved under a positional relationship that satisfies the illumination condition between the lighting unit 120 for illuminating the target area and the imaging unit 130 for imaging the target area.

[0089] Furthermore, when the positional relationship between any lighting unit 120 and the imaging unit 130 does not satisfy the illumination condition, the control unit 110 controls the specific lighting unit 120 so that the positional relationship between the specific lighting unit 120 that has not yet illuminated the target area and the imaging unit 130 satisfies the illumination condition. An example of the specific lighting unit 120 is in step 114 of the control process (see Figure 8B) is a lighting unit 120 that newly emits terahertz waves.

[0090] In this case, even if the positional relationship between any lighting unit 120 and the imaging unit 130 does not satisfy the illumination condition, imaging can be achieved under a positional relationship satisfying the illumination condition between the lighting unit 120 for illuminating the target area and the imaging unit 130 for imaging the target area.

[0091] The positional relationship between the lighting unit 120 and the imaging unit 130 that satisfies the illumination condition is determined by the area where the lighting unit 120 is located, the area where the imaging unit 130 is located, and the target area. Therefore, the illumination condition can also be understood as a condition determined by the positional relationship between the lighting unit 120, the imaging unit 130, and the target area.

[0092] Furthermore, in this embodiment, the positional relationship satisfying the irradiation condition is determined as the relationship between three-dimensional regions, but the present invention is not limited thereto.

[0093] In the camera system 100, each lighting unit 120, camera unit 130 and target area can be located as follows: Figure 5A and Figure 5B The two-dimensional area shown (in Figure 5A and Figure 5B In the example shown, the two-dimensional area defined by the coordinates in the X and Y directions is used. In this case, the positional relationship that satisfies the illumination condition can be determined as the relationship between the two-dimensional areas. An example of the relationship between the two-dimensional areas is the relationship between the distances from the imaging unit 130 to the target area.

[0094] An example in which the positional relationship satisfying the illumination condition is determined as the relationship of the distance from the imaging unit 130 to the target area will be described. Figure 6A and Figure 6B As shown in FIG, the shorter the distance from the imaging unit 130 to the target area, the shorter the distance from the imaging unit 130 to the lighting unit 120 located at a position that satisfies the illumination condition. Therefore, based on the distance from the lighting unit 120 to the imaging unit 130, the distance from the imaging unit 130 to the target area can be determined as a positional relationship that satisfies the illumination condition. In this case, in the lighting unit management table (see Figure 7 ), for each "illumination unit", the range of the distance from the imaging unit 130 to the target area that satisfies the illumination condition can be shown in the "illumination area" according to the "distance" from the illumination unit 120 to the imaging unit 130. Then, in the control process (see Figure 8A and Figure 8B), the control unit 110 may determine that the “illumination unit” associated with the “illumination area” including the distance from the imaging unit 130 to the target object 200 is the illumination unit 120 located at a position that satisfies the illumination condition.

[0095] In addition, the positional relationship that satisfies the illumination condition is not limited to the relationship of the distance from the imaging unit 130 to the target area. Figure 6A and Figure 6B In the example shown, the difference between the distance from lighting unit 120 to the target area and the distance from imaging unit 130 to the target area may be small. Therefore, the distance from lighting unit 120 to the target area can be determined as a positional relationship that satisfies the illumination condition. Furthermore, the distance from imaging system 100 to the target area can be determined as a positional relationship that satisfies the illumination condition.

[0096] As described above, in this embodiment, the plurality of lighting units 120 include a first lighting unit 120 that is a first distance away from the imaging unit 130 and a second lighting unit 120 that is a distance longer than the first distance from the imaging unit 130. The target area includes a first target area that is a second distance away from the imaging unit 130 and a second target area that is a distance longer than the second distance from the imaging unit 130. Then, when the target area is the first target area, the control unit 110 causes the first lighting unit 120 to emit a terahertz wave, and when the target area is the second target area, the control unit 110 causes the second lighting unit 120 to emit a terahertz wave. An example of the first distance is a distance X5 (see Figure 5B ). In addition, examples of the first lighting unit 120 include Figure 5B In the example shown, the second lighting unit 120B and the third lighting unit 120C are shown. In addition, an example of the second lighting unit 120 includes Figure 5B In the example shown, the first lighting unit 120A and the fourth lighting unit 120D are shown. In addition, an example of the second distance is the distance Y2 (see Figure 6B ). In addition, an example of the first target area is Figure 6B The area where the target object 200 is located in the example shown. In addition, an example of the second target area is Figure 6A The area where the target object 200 is located in the example shown.

[0097] In this case, even if the distance from the imaging unit 130 to the target area is different each time imaging is performed, imaging is performed under a positional relationship between the lighting unit 120 for illuminating the target area and the imaging unit 130 for imaging the target area that satisfies the illumination conditions.

[0098] Furthermore, when illumination unit 120 illuminates the second target area with terahertz waves, control unit 110 controls the intensity of light emitted by the terahertz waves to be higher than when illumination unit 120 illuminates the first target area with terahertz waves. In this case, regardless of the distance from illumination unit 120, which emits terahertz waves, to imaging unit 130, unevenness in the accuracy of images obtained through imaging is suppressed, compared to when the intensity of light emitted by the terahertz waves is the same.

[0099] In addition, a plurality of lighting units 120 and imaging units 130 are arranged in one direction. In this case, it is easier to associate the lighting unit 120 that meets the illumination condition based on the positional relationship between the imaging unit 130 and the target area.

[0100] Figure 9 1 is a diagram illustrating a first modification of the arrangement of the lighting unit 120 and the imaging unit 130 in the imaging system 100 . Figure 9 1 and 2 are views of the imaging system 100 and the target object 200 as viewed from above (upstream in the Z direction). Figure 9 The X, Y, and Z directions in Figure 5A and Figure 5B The X, Y, and Z directions correspond to the

[0101] Figure 9 The arrangement of the lighting unit 120 and the camera unit 130 shown is similar to Figure 6B The common point of the arrangement shown is that the camera unit 130 is sandwiched between the first lighting unit 120A and the second lighting unit 120B and the third lighting unit 120C and the fourth lighting unit 120D in the X direction. On the other hand, in terms of the positional relationship between the lighting unit 120 and the camera unit 130 in the Y direction, Figure 9 The arrangement of the lighting unit 120 and the camera unit 130 shown is similar to Figure 6B More specifically, in Figure 9 In the example shown, the second lighting unit 120B and the third lighting unit 120C are located downstream of the imaging unit 130 in the Y direction. In addition, the first lighting unit 120A and the fourth lighting unit 120D are located downstream of the second lighting unit 120B and the third lighting unit 120C in the Y direction.

[0102] Here, the first lighting unit 120A, the second lighting unit 120B, the third lighting unit 120C, and the fourth lighting unit 120D are not arranged on a straight line relative to the imaging unit 130. In other words, a straight line L1 passing through the first lighting unit 120A and the imaging unit 130, a straight line L2 passing through the second lighting unit 120B and the imaging unit 130, a straight line L3 passing through the third lighting unit 120C and the imaging unit 130, and a straight line L4 passing through the fourth lighting unit 120D and the imaging unit 130 all intersect. Note that a straight line passing through the lighting unit 120 and the imaging unit 130 means a straight line passing through the center of the lighting unit 120 and the center of the imaging unit 130.

[0103] As mentioned above, in Figure 9 In the illustrated example, the plurality of lighting units 120 include a first lighting unit 120 and a second lighting unit 120, and the second lighting unit 120 is provided at a position different from a straight line passing through the first lighting unit 120 and the imaging unit 130. The first lighting unit 120 is Figure 9 Any one of the four lighting units 120 shown. In addition, the second lighting unit 120 is Figure 9 The illustrated lighting units 120 are lighting units 120 that are different from the first lighting unit 120 .

[0104] In this case, the emission of the lighting unit 120 corresponding to the target object 200 that does not have a linear shape can be achieved. Figure 9 In the example shown, the distance between each lighting unit 120 and the target object 200 is shortened by the degree of arrangement of the lighting units 120 corresponding to the shape of a person as the target object 200, and the accuracy of the image pickup by the image pickup system 100 is improved. Figure 9 When the plurality of lighting units 120 are arranged in an arc shape as shown, this contributes to a smaller size of the camera system 100 compared to when the plurality of lighting units 120 are arranged in a straight line.

[0105] Although not shown, Figure 5A and Figure 5B In the illustrated example, the first lighting unit 120A, the second lighting unit 120B, the third lighting unit 120C, and the fourth lighting unit 120D are arranged on a straight line relative to the imaging unit 130. In other words, a straight line passing through the first lighting unit 120A and the imaging unit 130, a straight line passing through the second lighting unit 120B and the imaging unit 130, a straight line passing through the third lighting unit 120C and the imaging unit 130, and a straight line passing through the fourth lighting unit 120D and the imaging unit 130 do not intersect.

[0106] Figure 9The lighting units 120 shown are arranged in sequence in the X direction. Figure 9 The illustrated plurality of lighting units 120 and imaging units 130 may be regarded as an example in which they are arranged in one direction.

[0107] exist Figure 5A 、 Figure 5B and Figure 9 In the illustrated example, all lighting units 120 provided in the camera system 100 are arranged in one direction, but the present invention is not limited thereto. As long as at least two of the lighting units 120 provided in the camera system 100 are arranged in the same direction as the camera unit 130, the other lighting units 120 can be arranged without being arranged in the same direction.

[0108] 10A to 10D 1 is a diagram illustrating a second modification of the arrangement of the lighting unit 120 and the imaging unit 130 in the imaging system 100 . 10A to 10D 1 and 2 are diagrams of the imaging system 100 viewed from the upstream side in the Y direction of the imaging system 100 . 10A to 10D The X, Y, and Z directions in Figure 5A and Figure 5B The X, Y, and Z directions correspond to the

[0109] exist Figure 10A In the illustrated example, the imaging system 100 is provided with multiple groups of lighting units 120 divided according to the distance from the imaging unit 130 to the lighting units 120. More specifically, the imaging system 100 is provided with a first group of lighting units 1201, a second group of lighting units 1202, and a third group of lighting units 1203. The lighting units 120 belonging to the first group of lighting units 1201 are located at a distance Z1 from the imaging unit 130. Furthermore, the lighting units 120 belonging to the second group of lighting units 1202 are located at a distance Z2 from the imaging unit 130, which is longer than the distance Z1. Furthermore, the lighting units 120 belonging to the third group of lighting units 1203 are located at a distance Z3 from the imaging unit 130, which is longer than the distance Z2. Eight lighting units 120, arranged at equal intervals in the circumferential direction of the imaging unit 130, belong to each group. Furthermore, the positions of the imaging unit 130 and the respective lighting units 120 are aligned in the Y direction.

[0110] Here, for example, when the distance from the imaging system 100 to the target area is relatively short, as in Figure 10BAs shown, the control unit 110 causes each lighting unit 120 belonging to the first lighting unit group 1201 to emit terahertz waves. When the lighting units 120 located at positions that meet the illumination conditions are the first lighting unit group 1201, it is assumed that the target area is upstream or downstream of the imaging system 100 in the Y direction. Alternatively, this area may be located between the imaging unit 130 and the first lighting unit group 1201 in the X or Z direction.

[0111] In addition, for example, when the distance from the imaging system 100 to the target object is relatively long, as in Figure 10C As shown, the control unit 110 causes each lighting unit 120 belonging to the second lighting unit group 1202 to emit terahertz waves. When the lighting units 120 located at a position that satisfies the illumination conditions are the second lighting unit group 1202, it is assumed that the target area is further away from the imaging system 100 than when the first lighting unit group 1201 emits terahertz waves. Furthermore, the target area may be an area upstream or downstream of the imaging system 100 in the Y direction, or an area between the imaging unit 130 and the second lighting unit group 1202 in the X or Z direction.

[0112] Furthermore, for example, when the distance from the camera system 100 to the target object is even longer, as in Figure 10D As shown, the control unit 110 causes each lighting unit 120 belonging to the third lighting unit group 1203 to emit terahertz waves. When the lighting units 120 located at a position that satisfies the illumination conditions are the third lighting unit group 1203, it is assumed that the target area is further away from the imaging system 100 than when the second lighting unit group 1202 emits terahertz waves. Alternatively, the target area may be an area upstream or downstream of the imaging system 100 in the Y direction, or an area between the imaging unit 130 and the third lighting unit group 1203 in the X or Z direction.

[0113] As mentioned above, in 10A to 10D In the illustrated example, each of the multiple lighting units 120 belongs to any of a plurality of groups divided according to the distance from the imaging unit 130 to the lighting unit 120. The control unit 110 then causes the lighting unit 120 belonging to a group in the plurality of groups, located at a position that satisfies the illumination conditions, to emit terahertz waves. In this case, imaging accuracy is improved compared to a configuration in which only a single lighting unit 120 emits terahertz waves.

[0114] In particular, 10A to 10D In the example shown, the lighting units 120 of various groups are arranged around the camera unit 130. In this case, the target object can be illuminated with terahertz waves from various angles, making it easier to obtain an image showing an object carried by the target object regardless of the orientation of the target object.

[0115] In the present embodiment, the control unit 110 controls so that terahertz waves are not emitted from the illumination unit 120 at a position that does not satisfy the illumination condition. However, the present invention is not limited thereto. For example, the control unit 110 may control so that terahertz waves are emitted from an illumination unit 120 that is closer to the imaging unit 130 than the illumination unit 120 that satisfies the illumination condition.

[0116] In addition, the lighting unit 120 can change the area to which the terahertz wave is emitted. In this case, for each area to which the lighting unit 120 can emit terahertz waves, the lighting unit management table (see Figure 7 ) indicates the target area that meets the irradiation conditions.

[0117] Furthermore, the target areas satisfying the illumination conditions in the plurality of illumination units 120 may at least partially overlap, or the target areas satisfying the illumination conditions may not overlap.

[0118] Furthermore, the configuration for achieving emission of terahertz waves from the illumination unit 120 under the positional relationship between the illumination unit 120 and the imaging unit 130 that satisfies the illumination conditions is not limited to the above-described example.

[0119] The imaging system 100 may be provided with multiple imaging units 130, each located at a different position. The control unit 110 identifies an imaging unit 130 located in an area that satisfies the illumination condition from among the multiple imaging units 130 as the imaging unit 130 to be used to capture an image of the target area. The control unit 110 then causes the illumination unit 120 located in the area that satisfies the illumination condition to emit terahertz waves, and causes the imaging unit 130 located in the area that satisfies the illumination condition to capture an image of the target object. Specifically, the control unit 110 may control the imaging unit 130 so that the positional relationship between the illumination unit 120 and the imaging unit 130 satisfies the illumination condition, based on the positional relationship between the target area to be captured by the imaging unit 130 and the imaging unit 130. Furthermore, if the positional relationship between the target area and the imaging unit 130 changes, the control unit 110 may control the imaging unit 130 to capture the image using the imaging unit 130 located in the changed area that satisfies the illumination condition.

[0120] Furthermore, the imaging system 100 may be provided with a plurality of lighting units 120, each located at a different position, and may also be provided with a plurality of imaging units 130, each located at a different position. The control unit 110 may then determine, from among the plurality of lighting units 120 and the plurality of imaging units 130, a lighting unit 120 and an imaging unit 130 located in an area that satisfies the illumination conditions as the lighting unit 120 and the imaging unit 130 to be used for imaging the target object.

[0121] Next, an image pickup system 100 according to a second embodiment will be described.

[0122] The imaging system 100 according to the second embodiment shares the same characteristics as the imaging system 100 according to the first embodiment in that the illumination unit 120 is controlled so that the positional relationship between the illumination unit 120, the imaging unit 130, and the target area satisfies the illumination condition. On the other hand, the imaging system 100 according to the second embodiment differs from the imaging system 100 according to the first embodiment in the method used to control the illumination unit 120 to satisfy the illumination condition. Furthermore, the imaging system 100 according to the first embodiment controls the illumination unit 120 for emitting terahertz waves so that terahertz waves are emitted from the illumination unit 120 located at a position satisfying the illumination condition among the plurality of illumination units 120. In contrast, the imaging system 100 according to the second embodiment moves the illumination unit 120 to a position satisfying the illumination condition and then emits terahertz waves from the moved illumination unit 120. In other words, while the positional relationship between each illumination unit 120 and the imaging unit 130 is fixed in the first embodiment, the positional relationship between the illumination unit 120 and the imaging unit 130 is not fixed in the second embodiment.

[0123] Figure 11 2 is a diagram showing a configuration of an imaging system 100 according to a second embodiment. In addition, in the second embodiment, configurations different from those in the first embodiment will be described, and descriptions of configurations the same as those in the first embodiment will be omitted. Figure 11 The X, Y, and Z directions in Figure 5A and Figure 5B The X, Y, and Z directions correspond to the

[0124] like Figure 11 As shown in FIG. 1 , the imaging system 100 according to the second embodiment is provided with an illumination unit 120 and an imaging unit 130. Furthermore, the illumination unit 120 is provided with a driving unit 121 for driving the illumination unit 120, and the control unit 110 can move the illumination unit 120 within the range of a region R in the imaging system 100 by driving the driving unit 121. In this manner, the illumination unit 120 can be positioned in a region including the periphery of the imaging unit 130 by moving in the X and Z directions.

[0125] Furthermore, in this embodiment, the control unit 110 manages the position of the lighting unit 120 and the position of the camera unit 130. More specifically, the control unit 110 manages the position of the lighting unit 120 in the camera system 100, the position of the camera unit 130 in the camera system 100, and the positional relationship between the lighting unit 120 and the camera unit 130. The positional relationship between the lighting unit 120 and the camera unit 130 also includes the distance from the lighting unit 120 to the camera unit 130. The control unit 110 manages the positions of the lighting unit 120 and the camera unit 130 by acquiring information indicating the positions of the lighting unit 120 and the camera unit 130. The information indicating the positions of the lighting unit 120 and the camera unit 130 may be generated by the lighting unit 120 and the camera unit 130, or may be acquired by the acquisition unit 160 as information input by a user of the camera system 100 and stored in the storage unit 170.

[0126] The region where the illumination unit 120 moves is not limited to the range shown in the figure. The illumination unit 120 can move in the Y direction in the imaging system 100.

[0127] Figure 12 : is a diagram showing a lighting unit management table according to the second embodiment.

[0128] Figure 12 The lighting unit management table shown indicates "range" and "irradiation level"

[0129] The “range” is a range of the distance D which is the distance from the illumination unit 120 to the imaging unit 130 when the illumination unit 120 emits the terahertz wave.

[0130] “Illumination level” shows an index indicating the intensity of light emitted by the terahertz wave emitted from the illumination unit 120 .

[0131] In the lighting unit management table according to the second embodiment, the "range" and the "irradiation level" are associated with each other so that the intensity of light emitted by the terahertz wave emitted from the lighting unit 120 increases as the distance D from the lighting unit 120 to the imaging unit 130 increases. When the control unit 110 causes the lighting unit 120 to emit the terahertz wave, the control unit 110 causes the lighting unit 120 to emit the terahertz wave at the "irradiation level" corresponding to the "range" to which the distance D from the lighting unit 120 to the imaging unit 130 belongs, based on the lighting unit management table.

[0132] Figure 13A and Figure 13B : is a flowchart showing the flow of control processing in the second embodiment.

[0133] The control unit 110 operates the imaging unit 130 and the detection unit 180 (S201), and determines whether the detection unit 180 has detected the target object (S202). The processing in step 201 and the processing in step 202 are respectively the same as Figure 8A and Figure 8B The process in step 101 in the control process shown is the same as the process in step 102. In addition, while the negative result is maintained in step 202, the control unit 110 repeats the process in step 202.

[0134] When control unit 110 determines that detection unit 180 has detected the target object ("Yes" in S202), control unit 110 identifies the area that satisfies the illumination condition as the position of illumination unit 120 based on the positional relationship between the target object and imaging unit 130 (S203). In other words, control unit 110 identifies the area in which illumination unit 120 must be located to satisfy the illumination condition based on the positional relationship between the target object and imaging unit 130. Control unit 110 identifies the positional relationship between the target object and imaging unit 130 based on the position of the target object detected by detection unit 180, and identifies the area that satisfies the illumination condition as the position of illumination unit 120 based on the identified positional relationship.

[0135] The control unit 110 determines whether the area of ​​the position of the lighting unit 120 identified as satisfying the illumination condition is included in the range in which the lighting unit 120 can move (S204). More specifically, the control unit 110 determines whether the area identified in step 203 is included in the range in which the lighting unit 120 can move (S204). Figure 11 In the region R shown, a determination is made in step 204 .

[0136] When the control unit 110 determines that the area of ​​the lighting unit 120 that satisfies the illumination condition is not included in the movable range of the lighting unit 120 ("No" in S204), the control unit 110 determines whether to make the detection unit 180 continue to detect the target object (S205). When the control unit 110 determines not to make the detection unit 180 continue to detect the target object ("No" in S205), the control unit 110 ends the operation of the imaging unit 130 and the detection unit 180 (S206). The processing in step 205 and the processing in step 206 are respectively the same as Figure 8A and Figure 8B The processing in step 104 and the processing in step 105 in the control processing shown are the same.

[0137] Furthermore, when the control unit 110 determines to cause the detection unit 180 to continue detecting the target object 200 (YES in S205 ), the processing from S202 is performed again.

[0138] Furthermore, when the control unit 110 determines that the area of ​​the lighting unit 120 that satisfies the illumination condition is included in the range in which the lighting unit 120 is movable (YES in S204 ), the control unit 110 proceeds to the next step.

[0139] The control unit 110 determines whether the area of ​​the lighting unit 120 that meets the illumination condition is the area where the lighting unit 120 is currently located ( S207 ).

[0140] When the control unit 110 determines that the area of ​​the lighting unit 120 that meets the illumination condition is not the area where the lighting unit 120 is currently located (No in S207), the control unit 110 moves the lighting unit 120 to the area that meets the illumination condition (S208).

[0141] When a positive result is obtained in step 207, or after step 208, the control unit 110 turns on the lighting unit 120 as the target unit that meets the irradiation condition, so that the lighting unit 120 emits the terahertz wave (S209). Figure 12 According to the lighting unit management table shown, the control unit 110 causes the lighting unit 120 to emit terahertz waves at an “irradiation level” corresponding to the “distance” from the lighting unit 120 to the imaging unit 130 .

[0142] The control unit 110 determines whether the detection unit 180 detects the target object again (S210). The processing in step 210 is the same as Figure 8A and Figure 8B The processing in step 107 in the control processing shown is the same.

[0143] When the detection unit 180 has detected the target object again (YES in S210 ), the control unit 110 newly identifies an area satisfying the illumination condition as the position of the lighting unit 120 based on the positional relationship between the target object and the imaging unit 130 ( S211 ).

[0144] The control unit 110 determines whether the area newly identified as the position of the lighting unit 120 that satisfies the illumination condition has changed from the area identified in step 203 ( S212 ).

[0145] When the area of ​​the lighting unit 120 that satisfies the illumination condition has not changed (No in S212 ), the processing from step S210 is repeated.

[0146] In addition, when the control unit 110 determines that the area of ​​the lighting unit 120 that meets the illumination condition has changed ("Yes" in S212), the control unit 110 determines whether the new area that meets the illumination condition is included in the movable range of the lighting unit 120 (S213).

[0147] If the control unit 110 determines that the new area that satisfies the irradiation condition is not included in the movable range of the lighting unit 120 (No in S213), the control unit 110 turns off the target unit turned on in step 209 (S214). More specifically, the control unit 110 ends the emission of the terahertz wave by the lighting unit 120.

[0148] The control unit 110 determines whether to continue detecting the target object with the detection unit 180 (S215). If the control unit 110 determines not to continue detecting the target object with the detection unit 180 ("No" in S215), the control unit 110 terminates the operation of the imaging unit 130 and the detection unit 180 (S216), and the control process ends. The processes in step 215 and step 216 are the same as the processes in step 205 and step 206, respectively.

[0149] Furthermore, when the control unit 110 determines to cause the detection unit 180 to continue detecting the target object (YES in S215 ), the processing from S202 is performed again.

[0150] Furthermore, when the control unit 110 determines that the new area that satisfies the illumination condition is included in the movable range of the lighting unit 120 ("Yes" in S213), the control unit 110 moves the lighting unit 120 to the new area that satisfies the illumination condition (S217). In this way, even when the positional relationship between the lighting unit 120 and the imaging unit 130 no longer satisfies the illumination condition due to a change in the area of ​​the lighting unit 120 that satisfies the illumination condition (see S212), the lighting unit 120 is moved to the new area that satisfies the illumination condition.

[0151] Thereafter, the processing from step 210 is repeated.

[0152] As described above, in this embodiment, the control unit 110 moves the lighting unit 120 so that the positional relationship between the lighting unit 120 and the imaging unit 130 satisfies the illumination condition, based on the positional relationship between the target area to be imaged by the imaging unit 130 and the imaging unit 130. With this configuration, even if the positional relationship between the lighting unit 120, the imaging unit 130, and the target area is different each time imaging is performed, imaging can be performed with the positional relationship between the lighting unit 120 for illuminating the target area and the imaging unit 130 for capturing the target area satisfying the illumination condition.

[0153] In addition, when the positional relationship between the target area and the imaging unit 130 changes, the control unit 110 controls the lighting unit 120 so that the illumination condition (see FIG. 1 ) is satisfied after the change. Figure 13B 212 to 217 in FIG.

[0154] Furthermore, when the positional relationship between the lighting unit 120 and the imaging unit 130 does not satisfy the illumination condition, the control unit 110 controls the specific lighting unit 120 so that the positional relationship between the specific lighting unit 120 that has not yet illuminated the target area and the imaging unit 130 satisfies the illumination condition. Here, the "specific lighting unit 120 that has not yet illuminated the target area" also includes the lighting unit 120 that illuminated the target area before the change with the terahertz wave when the target area has changed (see S212).

[0155] In the present embodiment, the configuration for achieving emission of terahertz waves from the illumination unit 120 under the positional relationship between the illumination unit 120 and the imaging unit 130 that satisfies the illumination condition is not limited to the above-described example.

[0156] In the imaging system 100, the imaging unit 130 may be configured to be movable. In this case, the control unit 110 identifies an area that satisfies the illumination conditions as the position of the imaging unit 130 based on the positional relationship between the illumination unit 120 and the target area, moves the imaging unit 130 to the identified area, and then causes the illumination unit 120 to emit terahertz waves. In this way, the control unit 110 can move the imaging unit 130 based on the positional relationship between the target area to be imaged by the imaging unit 130 and the imaging unit 130, so that the positional relationship between the illumination unit 120 and the imaging unit 130 satisfies the illumination conditions.

[0157] Furthermore, both the lighting unit 120 and the imaging unit 130 may be movable. In this case, the control unit 110 identifies the areas satisfying the illumination conditions as the positions of the lighting unit 120 and the imaging unit 130 based on the target area, moves the lighting unit 120 and the imaging unit 130 to the identified areas, and then causes the lighting unit 120 to emit terahertz waves.

[0158] In addition, in this embodiment, the control unit 110 identifies the positional relationship between the target object and the imaging unit 130 based on the position of the target object detected by the detection unit 180, but the present invention is not limited thereto. The detection unit 180 may detect the position of the target object, identify the positional relationship between the target object and the imaging unit 130 based on the detection result, and transmit information indicating the identified positional relationship to the control unit 110.

[0159] Furthermore, in the first and second embodiments, the illumination condition was described as the positional relationship between the illumination unit 120 and the imaging unit 130, in which the terahertz wave emitted from the illumination unit 120 and reflected by the target object is incident on the detection device of the imaging unit 130. However, the present invention is not limited to this. For example, the illumination condition may be the positional relationship between the illumination unit 120 and the imaging unit 130, in which the terahertz wave emitted from the illumination unit 120 and reflected by the target object is incident on a predetermined range relative to the detection device of the imaging unit 130. The predetermined range may be any range, but for example, is a range with a radius of 10 cm centered on the detection device.

[0160] Furthermore, the imaging system 100 according to the first and second embodiments has been described as having the configuration including the holding unit 210 , but the present invention is not limited thereto.

[0161] The camera system 100 may be configured without the holding unit 210 and may be fixed to a predetermined position. In this case, the camera system 100 may be a surveillance camera for controlling the lighting unit 120 and the camera unit 130 so as to satisfy illumination conditions based on the positional relationship between a moving target object such as a person and the camera unit 130.

[0162] Furthermore, the camera system 100 according to the first and second embodiments may be a mobile camera mounted on a moving object such as a car or drone. Furthermore, the camera system 100 may be a wearable camera mounted on an object worn by a person, such as a head-mounted display (HMD) or smart glasses. Thus, even when the camera system 100 is moving, the camera system 100 can control the lighting unit 120 and the camera unit 130 based on the positional relationship between the target area and the camera unit 130 so that the illumination conditions are met.

[0163] Furthermore, the imaging system 100 according to the first and second embodiments is configured so that the position of the target object is recognized by the detection unit 180 , but the present invention is not limited thereto.

[0164] Information indicating the position of the target object may be transmitted to the camera system 100, acquired by the acquisition unit 160, and then stored in the storage unit 170. The control unit 110 may then recognize the position of the target object from the information stored in the storage unit 170.

[0165] Furthermore, while the illumination unit 120 of this embodiment has been described as emitting terahertz waves, the present invention is not limited thereto. The illumination unit 120 may also emit light having a frequency different from the terahertz wave. Even in this case, the control unit 110 can control the illumination unit 120 and the imaging unit 130 based on the positional relationship between the target object and the imaging unit 130 to satisfy the illumination conditions.

[0166] The embodiments of the present invention may also be implemented by reading and executing computer-executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be more fully referred to as a "non-transitory computer-readable storage medium") to perform one or more functions in the above-described embodiments and / or a computer of a system or device including one or more circuits (e.g., an application-specific integrated circuit (ASIC)) for performing one or more functions in the above-described embodiments, and by the following method, wherein the computer of the system or device performs the above-described method by, for example, reading and executing computer-executable instructions from a storage medium to perform one or more functions in the above-described embodiments and / or controlling the one or more circuits to perform one or more functions in the above-described embodiments. The computer may include one or more processors (e.g., a central processing unit (CPU), a microprocessing unit (MPU)), and may include a network of separate computers or separate processors to read and execute computer-executable instructions. For example, these computer-executable instructions may be provided to the computer from a network or a storage medium. The storage medium may include, for example, a hard disk, a random access memory (RAM), a read-only memory (ROM), a storage portion of a distributed computing system, an optical disk such as a compact disk (CD), a digital versatile disk (DVD), or a Blu-ray disk (BD). TM etc.), one or more of a flash memory device and a memory card, etc.

[0167] Other embodiments

[0168] The embodiments of the present invention can also be implemented by the following method, that is, software (including computer program products of computer programs) that perform the functions of the above-mentioned embodiments is provided to a system or device through a network or various storage media, and a computer (central processing unit (CPU), microprocessing unit (MPU)) of the system or device reads and executes the computer program.

[0169] While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

[0170] This application claims the benefit of Japanese Patent Application No. 2024-41952, filed March 18, 2024, which is hereby incorporated by reference herein in its entirety.

Claims

1. A camera system comprising: an illumination unit configured to emit a terahertz wave or light having a frequency different from the terahertz wave; an imaging unit configured to capture an image of a subject irradiated with a terahertz wave or light having a frequency different from that of the terahertz wave; as well as A control unit is configured to control the imaging unit or the lighting unit for illuminating the target area based on the positional relationship between the target area to be photographed by the imaging unit and the imaging unit for photographing the target area, so that the positional relationship between the lighting unit and the imaging unit satisfies a predetermined condition.

2. The camera system according to claim 1, in, The lighting unit includes a plurality of lighting units located at different positions, and The control unit causes a lighting unit located at a position satisfying the condition among the plurality of lighting units to illuminate the target area.

3. The camera system according to claim 2, in, The plurality of lighting units include a first lighting unit that is a first distance away from the imaging unit and a second lighting unit that is a longer distance from the imaging unit than the first distance. The target area includes a first target area that is a second distance away from the imaging unit and a second target area that is longer than the second distance from the imaging unit, and When the target area is the first target area, the control unit causes the first lighting unit to emit terahertz waves or light having a frequency different from terahertz waves, and when the target area is the second target area, the control unit causes the second lighting unit to emit terahertz waves or light having a frequency different from terahertz waves.

4. The camera system according to claim 1, in, The target area includes a first target area and a second target area that is farther away from the imaging unit than the first target area, and When the lighting unit illuminates the second target area with terahertz waves or light with a frequency different from the terahertz waves, the control unit controls the intensity of the light emitted by the terahertz waves or the light with a frequency different from the terahertz waves to be higher than when the lighting unit illuminates the first target area with terahertz waves or light with a frequency different from the terahertz waves.

5. The camera system according to claim 2, in, The plurality of lighting units and the imaging unit are arranged along one direction.

6. The camera system according to claim 2, in, The plurality of lighting units include a first lighting unit and a second lighting unit, and The second lighting unit is provided at a position different from a straight line passing through the first lighting unit and the imaging unit.

7. The camera system according to claim 2, in, Each lighting unit among the plurality of lighting units belongs to any group among a plurality of groups divided according to a distance from the imaging unit to the lighting unit, and The control unit causes an illumination unit belonging to a group located at a position satisfying the condition among the plurality of groups to emit a terahertz wave or light having a frequency different from the terahertz wave.

8. The camera system according to claim 1, in, When a positional relationship between the target area and the imaging unit changes, the control unit controls the lighting unit or the imaging unit so that the condition is satisfied after the change.

9. The camera system according to claim 1, in, The lighting unit includes one or more lighting units, and When the positional relationship between at least one lighting unit and the camera unit does not satisfy the condition, the control unit controls a specific lighting unit that has not yet illuminated the target area so that the positional relationship between the specific lighting unit and the camera unit satisfies the condition.

10. A method for controlling a camera system, the camera system comprising: an illumination unit for emitting a terahertz wave or light having a frequency different from that of the terahertz wave; A camera unit, used for capturing images of a subject; and a control unit for controlling the lighting unit or the camera unit, wherein the control method includes: Based on the positional relationship between the target area to be photographed by the imaging unit and the imaging unit used to photograph the target area, the imaging unit or the lighting unit used to illuminate the target area is controlled so that the positional relationship between the lighting unit and the imaging unit satisfies a predetermined condition. 11 . A computer program product comprising a program for causing a computer to execute the control method according to claim 10 . 12 . A computer-readable storage medium storing a program for causing a computer to execute the control method according to claim 10 .

Images