Door control camera, automobile, vehicle lamp, image processing device, and image processing method

Abstract

The lighting device (22) irradiates the detection light (L1). The depth direction is divided into a plurality of shooting ranges based on the shooting of the gated camera (20). The camera controller (26) controls the timing of the irradiation of the detection light (L1) based on the lighting device (22) and the timing of the exposure based on the image sensor (24) while moving the shooting range, so that the slice image (IMGs) output by the image sensor (24) in each shooting range only includes the object included in the corresponding shooting range. The image processing unit (28) generates a final image (IMGf) including the entire object of interest based on the plurality of slice images (IMGs) corresponding to the plurality of shooting ranges (RNG).

Description

Technical Field

[0001] The present invention relates to a gated camera. Background Art

[0002] Object recognition systems are used to detect the position and type of objects around the vehicle for autonomous driving and automatic control of headlight distribution. These systems consist of sensors and a processing unit that analyzes the sensor output. Sensors are selected from a variety of options, including cameras, LiDAR (Light Detection and Ranging), millimeter-wave radar, and ultrasonic sonar, depending on the application, required accuracy, and cost.

[0003] Conventional SLR cameras cannot obtain depth information, making it difficult to separate overlapping objects at different distances.

[0004] A TOF camera is a well-known camera that can obtain depth information. A TOF (Time of Flight) camera projects infrared light through a light-emitting device, measures the time it takes for the reflected light to return to an image sensor, and generates an image that converts the flight time into distance information.

[0005] As an active sensor alternative to TOF cameras, the present applicant has proposed a gated camera (Patent Documents 1 and 2). A gated camera divides the image capture range into multiple capture ranges and captures multiple images by varying the exposure timing and duration for each range. This generates slice images for each captured range of an object, each containing only the object within the corresponding capture range.

[0006] [Prior Art Literature]

[0007] [Patent Document]

[0008] Patent Document 1: Japanese Patent Application Laid-Open No. 2009-257983

[0009] Patent Document 2: International Publication No. WO2017 / 110413A1 Summary of the Invention

[0010] [Problems to be Solved by the Invention]

[0011] 1. The present inventors have studied gated cameras and have found the following problems. Figure 1 (a) Figure 1(b) is a diagram illustrating a problem that occurs in a gated camera. Figure 1 (a) is an example of a shooting scene based on a gated camera. If the depth of the shooting range is narrowed, a certain object will exist across multiple shooting ranges. Figure 1 (a) shows a case where the depth of the imaging range is set to 1.5 m and the car is photographed from the front. Figure 1 (b) means Figure 1 The slice images IMGs corresponding to a certain shooting range are obtained from the multiple slice images obtained from the shooting scene (a). i Only part of the car's bumper or hood is captured in the other sliced ​​images IMGs i+1 In the image, only a portion of the car's hood or windshield is captured. Therefore, it is difficult to identify the object based on each slice image IMGs.

[0012] This problem can be solved by increasing the depth of the imaging range to about tens of meters. However, if the depth of the imaging range is too large, there is a high probability that objects other than the object of interest will be included in a certain range, which will undermine the advantages of the gated camera.

[0013] 1. The present disclosure has been made in view of the above-mentioned problems, and one exemplary object of one aspect thereof is to provide a gated camera that can reduce the depth of an imaging range and generate an image that includes the entire object.

[0014] 2. In gated cameras, the transmission speed of images from the image sensor to the image processing unit becomes a bottleneck. Specifically, while the image sensor frame rate can be set very high, the time required for the gated camera to generate slice images of the entire imaging range (hereinafter referred to as the gated camera's capture time) is limited by the transmission speed.

[0015] The present disclosure has been made in view of the above problems, and one exemplary purpose of one aspect thereof is to provide a gated camera capable of shortening shooting time.

[0016] 3. Active sensors, such as gated cameras, illuminate an object with a probe light and detect the reflected light. Active sensors primarily consist of a light projector (illuminator) that illuminates the object with the probe light and a light sensor that detects the reflected light. Active sensors have the advantage of aligning the wavelength of the probe light with the sensor's sensitivity wavelength range, resulting in improved resistance to interference compared to passive sensors.

[0017] If a pulse light source having the same wavelength as that of the illumination device of the active sensor and emitting light at a period that is an integer multiple or an integer sub-period exists near the active sensor, the detection accuracy of the active sensor will be significantly reduced.

[0018] The present disclosure has been made in view of the above problems, and one exemplary purpose of one aspect thereof is to provide an active sensor capable of reducing the influence of surrounding pulse light sources.

[0019] [Technical solutions for solving technical problems]

[0020] An overview of several exemplary embodiments of the present disclosure is described. This overview serves as a prelude to the detailed description that will be described later. It simplifies and explains several concepts of one or more embodiments for the purpose of providing a basic understanding of the embodiments, and does not limit the breadth of the invention or disclosure. In addition, this overview is not a general overview of all contemplated embodiments, and does not limit the constituent elements that are missing from the embodiments. For convenience, "one embodiment" is sometimes used to indicate one embodiment (an embodiment, a modified example) or multiple embodiments (an embodiment, a modified example) disclosed in this specification.

[0021] 1. A gated camera in one embodiment includes: an illumination device for irradiating detection light; an image sensor; a camera controller that divides the depth direction into multiple shooting ranges, and controls the timing of irradiating the detection light of the illumination device and the timing of exposing the image sensor while moving the shooting range, so that in each shooting range, the slice image output by the image sensor only includes the objects included in the corresponding shooting range; and an image processing unit that generates a final image including the entire object of interest (OOI) based on multiple slice images corresponding to the multiple shooting ranges.

[0022] According to this aspect, it is possible to reduce the depth of the imaging range and generate a final image including the object of interest.

[0023] The final image may also contain only the object of interest. By removing the object of interest, the visual recognition when displayed on a monitor can be improved. Alternatively, subsequent processing such as object recognition of the object of interest can be facilitated.

[0024] Alternatively, the image processing unit may synthesize a plurality of slice images to generate a composite image in which each pixel has distance information, and extract a region containing the object of interest from the composite image. By referring to the distance information of the intermediate image, object separation becomes easier.

[0025] A gated camera in one embodiment includes: an illumination device for irradiating detection light; an image sensor; a camera controller that divides the depth direction into multiple shooting ranges, and controls the timing of irradiating the detection light of the illumination device and the timing of exposing the image sensor while moving the shooting range, so that in each shooting range, the slice image output by the image sensor only includes objects included in the corresponding shooting range; and an image processing unit that synthesizes multiple slice images corresponding to the multiple shooting ranges to generate a composite image with distance information for each pixel.

[0026] By referring to the distance information of the intermediate image, object separation becomes easier.

[0027] 2. One embodiment of a gated camera includes: an illumination device for emitting probe light; an image sensor; a camera controller that divides the depth direction into a plurality of imaging ranges and controls the timing of emitting probe light from the illumination device and the timing of exposing the image sensor while moving the imaging ranges so that, in each imaging range, a slice image output by the image sensor contains only objects within the corresponding imaging range; and an image processing unit that receives and processes the slice images from the image sensor. The camera controller is configured to adaptively control the amount of slice image data transmitted from the image sensor to the image processing unit.

[0028] According to this aspect, by reducing the amount of slice image data to be transmitted according to the situation, the transmission time can be shortened, and thus the imaging time can be shortened.

[0029] The camera controller can also reduce the size of the transmitted slice image by cropping a portion of the slice image through the image sensor or the interface (image transmission circuit) between the image sensor and the image processing unit. By cropping and transmitting only the required portion, the amount of data can be reduced.

[0030] Instead of or in addition to cropping, the data volume may be reduced by lowering the resolution through merging or thinning-out processing.

[0031] Since close objects move relatively quickly within the slice image, even if a cropping region is determined based on a certain point in time, there's a high probability that the object will later escape from the cropping region. On the other hand, since distant objects move relatively slowly within the slice image, they remain within the cropping region for a longer period of time. Furthermore, there's a high probability that distant objects will appear concentrated near the vanishing point. Therefore, it's possible to always transmit slice image groups corresponding to shooting range groups closer than a specified distance without cropping, while controlling whether or not to crop slice image groups corresponding to shooting range groups farther than a specified distance are cropped. This allows for the transmission of slice images containing the object of interest.

[0032] Alternatively, the gated camera transmits multiple slice images without cropping, and determines whether subsequent slice images should be cropped based on the multiple slice images that have not been cropped. Thus, the conditions of future slice images can be predicted and estimated based on the multiple slice images that have not been cropped, and whether to crop or not can be determined.

[0033] Alternatively, in the first mode, all slice images corresponding to all imaging ranges may be transmitted without cropping, and if, among the multiple slice images transmitted in the first mode, a distant slice image group corresponding to an imaging range group farther than a predetermined distance contains only a single object of interest, the mode may be switched to the second mode. In the second mode, the distant slice image group may be cropped and transmitted for the range of interest containing the object of interest.

[0034] The lighting device can also be configured to make the illumination range variable by focusing and diffusing the probe light. The camera controller can also cause the lighting device to focus the probe light when shooting the slice image that should be cut and transmitted. When shooting a distant shooting range, due to the large attenuation of the probe light and its reflected light, a single exposure cannot generate a slice image of sufficient brightness. In such a case, for the distant shooting range, multiple exposures are required, and the slice images obtained by each exposure are composited to generate a single slice image. By focusing the probe light, the brightness of the slice image obtained by a single exposure can be brightened, thereby reducing the number of exposures and the number of transmissions of the slice image. As a result, the shooting time can be shortened.

[0035] A gated camera according to one embodiment includes: an illumination device for emitting probe light, configured to vary the illumination range by focusing and diffusing the probe light; an image sensor; a camera controller for dividing the depth direction into a plurality of imaging ranges and controlling the timing of the illumination device's illumination of the probe light and the exposure timing of the image sensor while shifting the imaging ranges so that, in each imaging range, a slice image output by the image sensor contains only objects within the corresponding imaging range; and an image processing unit for receiving and processing the slice images from the image sensor. The camera controller controls the illumination range of the illumination device based on detection results from the image processing unit.

[0036] When capturing distant areas or in dense fog, the attenuation of the probe light and its reflected light increases. In these situations, a single exposure cannot produce a sufficiently bright slice image, requiring multiple exposures, each resulting in a composite slice image. Focusing the probe light brightens the slice image produced by a single exposure, reducing the number of exposures and the number of slice image transmissions. This reduces capture time.

[0037] In the first mode, the probe light may be irradiated onto the entire irradiation range. Alternatively, when the image processing unit detects an object of interest within a slice image of a predetermined imaging range in the first mode, the image processing unit may switch to the second mode, and in the second mode, when imaging the predetermined imaging range, the probe light may be irradiated onto a region of interest including the object of interest.

[0038] An active sensor according to one embodiment includes: a light emitting device that emits multiple pulses of light at uneven time intervals; and a light sensor that detects reflected light from an object at a timing synchronized with each emission of the light emitting device, and accumulates and outputs multiple detection results.

[0039] Because the photosensor is synchronized with the light-emitting device within the same active sensor, reflected light from an object enters the photosensor at the same timing, even if the emission timing varies. Consequently, the detected values ​​of reflected light are accumulated, resulting in a larger signal component. Because the emission timing of other noise light sources is not synchronized with the photosensor's exposure time, the frequency with which noise light from these sources enters the photosensor and is included in the photosensor's exposure timing can be reduced. This reduces the influence of surrounding pulsed light sources.

[0040] One sensing including one emission of light by the light emitting device and one exposure of the light sensor may be repeated in a predetermined cycle, and the emission timing of the light emitting device within the predetermined cycle may be changed every time sensing is performed.

[0041] The active sensor may be a gated camera that divides the depth direction into a plurality of ranges and changes the time difference between light emission and imaging for each range, thereby acquiring a plurality of images corresponding to the plurality of ranges.

[0042] The active sensor may also be a TOF (Time Of Flight) camera.

[0043] The active sensor may also be LIDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging).

[0044] [Effects of the Invention]

[0045] According to one embodiment, the depth of the imaging range can be reduced while generating an image that includes the entire object of interest. According to one embodiment, the imaging time of the gated camera can be shortened. According to one embodiment, the influence of surrounding active sensors can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 (a) Figure 1 (b) is a diagram illustrating a problem that occurs in a gated camera.

[0047] Figure 2 This is a block diagram of a gate control camera according to embodiment 1.

[0048] Figure 3 A diagram illustrating the photographing operation of a gated camera.

[0049] Figure 4 is a diagram showing an example of a shooting scene.

[0050] Figure 5 It means in Figure 4 All slices IMGs1~IMGs obtained from the driving scene N 3 slice images IMGs in i ~IMGs i+2 Picture.

[0051] Figure 6 3 is a diagram showing a final image IMGf generated by the image processing unit.

[0052] Figure 7 3 is a diagram illustrating a synthesized image IMGc generated by the image processing unit.

[0053] Figure 8 (a)~ Figure 8 (c) is a diagram showing a specific example of the generation process of the composite image IMGc in the image processing unit.

[0054] Figure 9 (a)~ Figure 9 (c) is a diagram illustrating the generation of a final image IMGf based on the synthesized image IMGc.

[0055] Figure 10 This is a block diagram of a gate control camera according to embodiment 2.

[0056] Figure 11 A diagram illustrating the photographing operation of a gated camera.

[0057] Figure 12 (a) Figure 12 (b) is a diagram illustrating an example of cropping processing of the slice image IMGs.

[0058] Figure 13 (a) Figure 13 (b) is a diagram illustrating the capture and transmission of a slice image.

[0059] Figure 14 is a flow chart of an example of mode control in a gated camera.

[0060] Figure 15 This is a diagram showing an example of a situation shifting to the second mode.

[0061] Figure 16 This is a block diagram of a gate control camera according to a third embodiment.

[0062] Figure 17 This is a block diagram of a gate control camera according to a fourth embodiment.

[0063] Figure 18 This is a block diagram of an active sensor according to a fifth embodiment.

[0064] Figure 19 This is a timing chart showing the relationship between the light emission timing of the light emitting device and the exposure timing of the photosensor.

[0065] Figure 20 This is the operating waveform diagram of an active sensor with a relatively high technology.

[0066] Figure 21 yes Figure 18 The action waveform of the active sensor.

[0067] Figure 22 is a block diagram of a gated camera according to an embodiment.

[0068] Figure 23 This is a diagram illustrating the operation of the gate camera.

[0069] Figure 24 (a) Figure 24 (b) is a diagram illustrating an image obtained by a gated camera.

[0070] Figure 25 This diagram shows a vehicle lamp with a built-in access control camera or active sensor.

[0071] Figure 26 This is a block diagram showing a vehicle lamp equipped with an object recognition system. DETAILED DESCRIPTION

[0072] The present invention is described below based on preferred embodiments with reference to the accompanying drawings. Identical or equivalent components, members, and processes shown in the various drawings are denoted by the same reference numerals, and duplicate descriptions are omitted as appropriate. The embodiments do not limit the invention but are merely illustrative. Not all features or combinations thereof described in the embodiments constitute essential aspects of the invention.

[0073] (Implementation Method 1)

[0074] Figure 2 2 is a block diagram of the gate camera 20 according to Embodiment 1. The gate camera 20 divides the depth direction into a plurality of N (N≧2) imaging ranges RNG1 to RNG NFor example, the depth of one imaging range is about 1 to 2 meters (for example, 1.5 meters), and N=100.

[0075] The gate control camera 20 includes a lighting device 22 , an image sensor 24 , a camera controller 26 , and an image processing unit 28 .

[0076] The lighting device 22 irradiates the detection light L1 toward the front of the vehicle in synchronization with the emission timing signal S1 supplied from the camera controller 26. The detection light L1 is preferably infrared light, but is not limited thereto and may be visible light having a predetermined wavelength.

[0077] The image sensor 24 is configured to perform exposure control synchronized with an exposure timing signal S2 supplied from the camera controller 26 and to generate slice images IMGs. The image sensor 24 is sensitive to the same wavelength as the probe light L1, and captures reflected light (return light) L2 reflected from the object OBJ, outputting the slice images IMGs.

[0078] The camera controller 26 holds the image data for each of the imaging ranges RNG1 to RNG N The camera controller 26 takes a picture of a certain shooting range RNG. i When the ith slice image IMGs is obtained, the luminescence timing signal S1 and the exposure timing signal S2 are generated based on the luminescence timing and exposure timing corresponding to the imaging range and imaging is performed. i The corresponding shooting range RNG is captured in i The objects contained in .

[0079] In this embodiment, the camera controller 26 sets the imaging range RNG i Shoot while moving (shifting) in the depth direction, for the entire shooting range RNG1~RNG N , generate slice images IMGs1~IMGs N .

[0080] Figure 3 It is a diagram for explaining the imaging operation of the gate camera 20 . Figure 3 The i-th shooting range RNG is shown i When measuring the range of interest (ROI). The lighting device 22 is synchronized with the lighting timing signal S1 and emits light during the lighting period τ1 between time t0 and t1. At the top, a graph of light is shown with the horizontal axis representing time and the vertical axis representing distance. i The distance to the nearest boundary is set to d MINi , will be in the range RNG iThe distance from the deep side boundary is set to d MAXi .

[0081] At a certain moment, the light emitted from the lighting device 22 reaches a distance d MINi The round trip time T until the reflected light returns to the image sensor 24 is MINi T MINi =2×d MINi / c. c is the speed of light.

[0082] Similarly, at a certain moment, the light emitted from the lighting device 22 reaches the distance d MAXi The round trip time T until the reflected light returns to the image sensor 24 is MAXi T MAXi =2×d MAXi / c.

[0083] To shoot the range RNG i When the camera controller 26 captures the object OBJ contained in MINi Exposure starts at time t3 = t1 + T MAXi The exposure timing signal S2 is generated in such a way that the exposure is ended at the same time. This is one exposure operation.

[0084] In the i-th shooting range RNG i When photographing, multiple exposures may be repeated (multiple exposures). In this case, the camera controller 26 may repeat the above-mentioned exposure operation multiple times at a predetermined cycle τ2.

[0085] return Figure 2 The image processing unit 28 receives input data related to the plurality of imaging ranges RNG1 to RNG N Corresponding multiple slice images IMGs1~IMGs N The image processing unit 28 processes the image based on the plurality of slice images IMGs1 to IMGs N A final image IMGf including the object of interest (OOI) is generated.

[0086] The above is the configuration of the gate camera 20. Next, its operation will be described.

[0087] Figure 4 : is a diagram showing an example of a shooting scene. In front of the gate control camera 20, there are a vehicle OBJ1 and a pedestrian OBJ2. The vehicle OBJ1 spans across multiple shooting ranges RNG. i ~RNG i+2 And exists, pedestrian OBJ2 is included in the shooting range RNG i .

[0088] Figure 5 It means in Figure 4 All slices IMGs1~IMGs obtained from the driving scene N 3 slice images IMGs in i ~IMGs i+2 Fig. Slice images IMGs i Only the corresponding shooting range RNG is captured i Therefore, the slice image IMGs i OBJ1 contains pedestrian 2 and part of vehicle OBJ1 A , slice images IMGs i+1 Contains a portion of the vehicle OBJ1 B , slice images IMGs i+2 Contains a portion of the vehicle OBJ1 C .

[0089] For example, the image processing unit 28 scans all the slice images IMGs1 to IMGs N , set the candidate of the object of interest contained therein. In this example, if the image processing unit 28 detects the slice image IMGs i If a part of the vehicle is included in , the vehicle is set as the object of interest. And all the slice images IMGs1 to IMGs N , detect other parts of OBJ1 that contain the same vehicle OBJ1 B OBJ1 C Slice images IMGs i+1 、IMGs i+2 . And, from these slice images IMGs i ~IMGs i+2 Extract the parts that constitute the same object of interest OBJ1 A ~OBJ1 C , and synthesize them. Figure 6 : is a diagram showing a final image IMGf generated by the image processing unit 28. It should be noted that the final image IMGf does not include the pedestrian OBJ2 which is not the object of interest.

[0090] The above is the operation of the gate camera 20. According to the gate camera 20, the depth of the imaging range RNG can be reduced, and the object of interest can be extracted to generate a final image IMGf including the entire object.

[0091] The gated camera 20 has the advantage of being able to remove objects other than the object of interest from the final image. Figure 4In addition to such shooting scenes, this advantage is particularly evident in specific situations such as snowfall. During snowfall, a large number of small snow particles are included in each slice image. Since the gated camera 20 can remove the snow particles from the final image IMGf, an image with excellent visual recognition can be generated.

[0092] Next, a description will be given of a specific example of the processing in the image processing unit 28. The image processing unit 28 may generate a synthesized image IMGc in the step of generating the final image IMGf, and generate the final image IMGf based on the synthesized image IMGc.

[0093] Figure 7 This figure explains the composite image IMGc generated by the image processing unit 28. The slice image IMGs and the intermediate image IMGc have X×Y pixels. The pixel in the jth row and the ith column is denoted as P ij Each pixel P of the intermediate image IMGc ij With all slice images IMGs1~IMGs N An IMGs k (k=1~N) corresponding pixel P ij The brightness value L or the brightness value L' obtained by calculating the brightness value. ij Contains the slice image IMGs that is the source of this brightness value k The identifier k can be regarded as the distance information to the object, so each pixel P of the intermediate image IMGc ij Instead of the identifier k, the distance d to the object that is the origin of the pixel may be used. In this specification, each pixel P is represented by the form [L, k]. ij .

[0094] The format of the intermediate image IMGc is not particularly limited. For example, the intermediate image IMGc may include two pieces of image data: image data representing luminance values ​​X×Y and image data X×Y representing distance information.

[0095] Figure 8 (a)~ Figure 8 (c) is a diagram showing a specific example of the generation process of the composite image IMGc in the image processing unit 28. Here, for ease of understanding, it is assumed that N=6, and the grayscale of each pixel is represented by 8 grayscales (3 bits) from 0 to 7. Figure 8 (a) shows a plurality of slice images IMGs1 to IMGs6 corresponding to the entire imaging range.

[0096] The synthesis method is not particularly limited, and examples thereof include the following methods.

[0097] Figure 8(b) shows the intermediate image IMGc obtained by the first generation method. In the first generation method, each pixel P of the intermediate image IMGc is ij The brightness value of each pixel P in all slice images IMGs1 to IMGs6 is ij The maximum value among the brightness values ​​(or the value obtained by calculating the maximum value).

[0098] Figure 8 (c) shows the intermediate image IMGc obtained by the second generation method. In the second generation method, each pixel P of the intermediate image IMGc ij The brightness value of each pixel P in all slice images IMGs1 to IMGs6 is ij The brightness value of the image slice having the brightness value of the closest preceding (i.e., closest) slice image among the brightness values ​​having the valid value (or the value obtained by calculating the brightness value of the slice image). The valid value may be non-zero or a value greater than a predetermined threshold.

[0099] The above is the description of the composite image IMGc. Figure 9 (a)~ Figure 9 (c) is a diagram illustrating the generation of a final image IMGf based on the synthesized image IMGc.

[0100] The synthesized image IMGc includes, in addition to the brightness value L, an identifier k indicating the distance to the object, making it easier to isolate the object of interest OOI. This is because an object extraction algorithm can be created based on the assumption that pixels farther from the identifier k are likely to belong to other objects, while pixels closer to the identifier k are likely to belong to the same object.

[0101] When the composite image IMGc contains multiple objects of interest OOI1 and OOI2, as shown in FIG. Figure 9 As shown in (b) of FIG. 1 , final images IMGf1 and IMGf2 may be generated for each object of interest.

[0102] Alternatively, when the composite image IMGc contains multiple objects of interest OOI1 and OOI2, as shown in FIG. Figure 9 As shown in (c), a single final image IMGf including a plurality of objects of interest may be generated.

[0103] Once the composite image IMGc is generated, the original multiple slice images IMGc1 to IMGs can be discarded. N This also has the advantage of being able to reduce memory capacity.

[0104] (Implementation Method 2)

[0105] Figure 102 is a block diagram of the gate camera 20 according to Embodiment 2. The gate camera 20 divides the depth direction into a plurality of N (N≧2) imaging ranges RNG1 to RNG N For example, the depth d of one imaging range is about 1 to 2 meters (for example, 1.5 meters), and N=100.

[0106] The gate control camera 20 includes a lighting device 22 , an image sensor 24 , a camera controller 32 , and an image processing unit 34 .

[0107] The lighting device 22 irradiates the detection light L1 toward the front of the vehicle in synchronization with the emission timing signal S1 supplied from the camera controller 32. The detection light L1 is preferably infrared light, but is not limited thereto and may be visible light having a predetermined wavelength.

[0108] The image sensor 24 is configured to perform exposure control synchronized with an exposure timing signal S2 supplied from the camera controller 32 and to generate slice images IMGs. The image sensor 24 is sensitive to the same wavelength as the probe light L1, and captures reflected light (return light) L2 reflected from the object OBJ, outputting slice images IMGs.

[0109] The camera controller 32 holds the image data for each of the imaging ranges RNG1 to RNG2. N The camera controller 32 takes a picture of a certain shooting range RNG. i When the ith slice image IMGs is obtained, the luminescence timing signal S1 and the exposure timing signal S2 are generated based on the luminescence timing and exposure timing corresponding to the imaging range and imaging is performed. i The corresponding shooting range RNG is captured in i The objects contained in .

[0110] In this embodiment, the camera controller 32 sets the imaging range RNG i Shoot while moving in the depth direction. N , generate slice images IMGs1~IMGs N .

[0111] Figure 11 It is a diagram for explaining the imaging operation of the gate camera 20 . Figure 11 The measurement of the i-th shooting range RNG is shown i The lighting device 22 is synchronized with the lighting timing signal S1 and emits light during the lighting period τ1 between time t0 and t1. At the top, a graph of light is shown with the horizontal axis representing time and the vertical axis representing distance. i The distance to the nearest boundary is set to d MINi, will be in the range RNG i The distance from the deep side boundary is set to d MAXi .

[0112] At a certain moment, the light emitted from the lighting device 22 reaches a distance d MINi The round trip time T until the reflected light returns to the image sensor 24 is MINi T MINi =2×d MINi / c. c is the speed of light.

[0113] Similarly, at a certain moment, the light emitted from the lighting device 22 reaches the distance d MAXi The round trip time T until the reflected light returns to the image sensor 24 is MAXi T MAXi =2×d MAXi / c.

[0114] To shoot the range RNG i When the camera controller 32 captures the object OBJ contained in MINi Exposure starts at time t3 = t1 + T MAXi The exposure timing signal S2 is generated in such a way that the exposure is ended at the same time. This is one exposure operation.

[0115] In the i-th shooting range RNG i When photographing, multiple exposures may be repeated (multiple exposures). In this case, the camera controller 32 may repeat the above-mentioned exposure operation multiple times at a predetermined cycle τ2.

[0116] return Figure 10 The image processing unit 34 receives the slice image IMGs from the image sensor 24. i (i=1, 2, ...N), and the received slice images IMGs i to be processed.

[0117] The camera controller 32 and the image processing unit 34 may be implemented as separate hardware or in a single hardware. For example, the camera controller 32 and the image processing unit 34 may be implemented by a combination of a microcomputer or other arithmetic processing device 30 and a software program executed thereby.

[0118] The image processor 24 and the arithmetic processing unit 30 are connected via an interface 26. The interface 26 transmits the slice images IMGs from the image sensor 24 to the image processing unit 34. The type of interface 26 is not particularly limited, and for example, MIPI (Mobile Industry Processor Interface) or USB (Universal Serial Bus) can be used.

[0119] In this embodiment, the camera controller 32 is configured to control at least one of the image sensor 24 and the interface 26 and can adaptively control the data volume of the slice images IMGs transmitted from the image sensor 24 to the image processing unit 34 .

[0120] In this embodiment, the camera controller 32 reduces the size (number of pixels) of the transmitted slice image IMGs by cropping a portion of the slice image IMGs in the image sensor 24 or the interface 26, thereby reducing the amount of data. The camera controller 32 generates a control signal S3 that specifies whether to crop. The control signal S3 may also include information about the size and position of the cropped area.

[0121] The cropping method is not particularly limited, and a portion of the slice image IMGs may be cropped in the image sensor 24. In other words, the image itself read by the image sensor 24 may be cropped. Alternatively, a portion of the slice image IMGs may be cropped in the interface 26. In other words, all pixels may be read from the image sensor 24, and a portion of the full-size image read out may be cropped.

[0122] Figure 12 (a) Figure 12 (b) is a diagram illustrating an example of a shearing process of a slice image IMGs. Figure 12 As shown in (a), when there is a high probability that the object OBJ exists only in a portion of the slice image IMGs obtained by the image sensor 24, the range (region of interest ROI) including the object OBJ is cut out and the cut slice image (cut out image) IMGs' is transmitted.

[0123] like Figure 12 As shown in (b) , when there is a high possibility that one or more objects OBJ exist within a wide range of the slice image IMGs obtained by the image sensor 24 , the original slice image IMGs is transmitted without being cut.

[0124] Because close objects move relatively quickly within a slice image, even if a cropping region is determined based on a specific time point, there's a high probability that the object will later escape the cropping region. Therefore, in this embodiment, close slice images are always transmitted without cropping. This prevents objects from being missed.

[0125] On the other hand, distant objects move relatively slowly in the slice image and therefore stay within the cropping area for a longer time. In addition, distant objects are more likely to appear concentrated near the vanishing point.

[0126] Therefore, in this embodiment, the slice image groups IMGs1 to IMGs2 corresponding to the imaging range group (10 to 100 m) closer than a predetermined distance (for example, 100 m) may be processed. Y , always transmitted without cutting, for the slice image group IMGs corresponding to the shooting range group (100 to 150m) farther than the specified distance (100m) Y+1 ~IMGs N , control whether there is shearing.

[0127] Furthermore, the "pre-transmission" slice images IMGs obtained by the image sensor 24 cannot be observed from the processing unit 30. Therefore, in this embodiment, the camera controller 32 needs to estimate the state of the current slice images IMGs based on the non-cropped slice images IMGs previously received by the image processing unit 32 and determine whether cropping has occurred.

[0128] Therefore, the gated camera 20 transmits all the slice images IMGs1 to IMGs2 without cutting. N Furthermore, it is also possible to use the uncut slice image groups IMGs1 to IMGs2 as the basis. N To determine the subsequent slice image groups IMGs1~IMGs N Thus, the state of future slice images can be predicted and estimated based on the slice images that have not been cropped, and whether or not to crop can be determined. In addition to whether or not to crop, the size and position of the cropping area can also be determined.

[0129] The above is the basic structure of the gate camera 20. Next, its operation will be described.

[0130] Figure 13 (a) Figure 13 (b) is a diagram illustrating the capture and transmission of slice images. Figure 13 In (a), the entire slice image captured by the image sensor 24 is directly transmitted for all imaging ranges.

[0131] exist Figure 13In (b), a portion of the slice image IMGs captured by the image sensor 24 in at least one capturing range RNG is cropped, and the cropped slice image IMGs' is transmitted. Since the amount of data in the cropped slice image IMGs' is smaller than that in the pre-cropped slice image IMGs, the transmission time is shortened.

[0132] The above describes the operation of the gated camera 20. Next, its advantages will be described. The imaging time of the gated camera 20 can be determined as the sum of the exposure time for all imaging ranges and the transmission time of the slice images. The gated camera 20 reduces the amount of slice image IMGs data transmitted from the image sensor 24 to the image processing unit 34 according to the situation, thereby shortening the transmission time and the imaging time of the gated camera 20.

[0133] In particular, the size of the slice image is reduced by cropping a portion of the slice image. By cropping and transmitting only the required portion, the amount of data can be reduced. Alternatively, instead of cropping, the amount of data can be reduced by merging or pixel thinning. However, in this case, the resolution of the slice image input to the image processing unit 34 is reduced, and the accuracy of the subsequent object recognition processing may be reduced. In contrast, if cropping is used, the resolution is not reduced, so there is no concern about reduced object recognition accuracy.

[0134] Next, a specific example of control of the presence or absence of shearing will be described.

[0135] The gate control camera 20 can switch between the first mode and the second mode. In the first mode, the image data of all the imaging ranges RNG1 to RNG2 is transmitted without being cut. N All corresponding slice images IMGs1~IMGs N In the second mode, the close shooting range RNG1 to RNG Y Corresponding slice image groups IMGs1 to IMGs Y Transmit without cutting, with long-distance shooting range RNG Y+1 ~RNG N Corresponding slice image groups IMGs Y+1 ~IMGs N Cut and transmitted.

[0136] Whether or not the transition to the second mode is made is based on the plurality of slice images IMGs1 to IMGs2 transmitted in the first mode. N The shooting range group RNG is farther than the specified distance Y+1 ~RNG N Corresponding distant slice image groups IMGs Y+1 ~IMGs NSpecifically, in the distant slice image group IMGs Y+1 ~IMGs N If only a single object of interest is included in the image, the mode is switched to the second mode. In the second mode, the portion including the object of interest is set as the region of interest, and only the region of interest is cut and transferred.

[0137] Figure 14 1 is a flowchart of an example of mode control in the gated camera 20. First, the first mode is set (S100). As described above, in the first mode, all slice images IMGs1 to IMGs N The image is input to the image processing unit 34 (S102).

[0138] The image processing unit 34 processes the remote slice image group IMGs Y+1 ~IMGs Y+N If only a single object of interest (OOI) is included in the image (Yes in S104), the gate camera 20 is switched to the second mode (S106). Otherwise (No in S104), the first mode is maintained and the process returns to S102.

[0139] For example, in the process S104, the image processing unit 34 may also calculate the position of the object of interest when detecting the object of interest in the long-distance slice image group IMGs. If it is determined that the object of interest is within the long-distance imaging range RNG Y+1 ~RNG N If there is no object other than the object of interest, the mode is switched to the second mode.

[0140] Figure 15 1 is a diagram showing an example of a situation in which the vehicle is shifted to the second mode. The gate control camera 20 is mounted on a vehicle 800. There is an opposing vehicle 802 in front of the vehicle 800. The opposing vehicle 802 is located in a distant range RNG. Y+1 ~RNG N The object of interest, in the distant range RNG Y+1 ~RNG N There is no object of interest other than the vehicle 802. When the gate control camera 20 detects such a situation, it shifts to the second mode.

[0141] return Figure 14 When the process shifts to the second mode (S106), the size and position of the ROI containing the object of interest detected in the process S104 are determined. During the second mode, the remote slice image group IMGs Y+1 ~IMGs N , cut ROI, transfer the slice image IMGs containing the region of interest ROI Y+1 ~IMGsN .

[0142] Furthermore, as described above, in the second mode, for the close-range slice images IMGs1 to IMGs Y , are also transferred at their original uncropped size.

[0143] While the return condition to the first mode is not satisfied (No in S108), the second mode is maintained. When the return condition to the first mode is satisfied (Yes in S108), the process returns to the first mode S100.

[0144] The return condition to the first mode may be a predetermined time or the remote shooting range RNG Y+1 ~RNG N There may be multiple objects of interest in the image, or the original object of interest may no longer exist in the long-distance shooting range RNG.

[0145] (Implementation 3)

[0146] Figure 16 This is a block diagram of a gate camera 20A according to Embodiment 3. In the gate camera 20A, the illumination device 22 is configured to vary the illumination range by focusing and diffusing the detection light L1. The camera controller 32 can adaptively control the illumination range of the illumination device 22 based on a control signal S4.

[0147] When capturing the slice image IMGs to be cut and transferred, the camera controller 32 causes the illumination device 22 to focus the probe light L1 and narrow the irradiation range.

[0148] For example, in the second mode, the camera controller 32 captures the long-distance slice image group IMGs Y+1 ~IMGs N When the probe light L1 is focused, the region of interest is intensively irradiated. This can improve the illumination of the irradiation range and obtain a clear image.

[0149] The above is the configuration of the gate camera 20A. Next, its advantages will be described.

[0150] When capturing distant imaging ranges, the attenuation of the probe light L1 and its reflected light L2 is significant. Consequently, a single exposure may not produce a sufficiently bright slice image IMGs. In such cases, multiple exposures are required for the distant imaging range, with the slice images obtained from each exposure being composited to produce a single slice image. In this embodiment, by focusing the probe light L1, the brightness of the slice image IMGs obtained from a single exposure can be increased, thereby reducing the number of exposures and the number of slice image transmissions. This reduces imaging time.

[0151] (Implementation 4)

[0152] Figure 17 This is a block diagram of a gated camera 20B according to Embodiment 4. In gated camera 20B, only the illumination range of lighting device 22 is controlled; cropping during the transmission of slice images is not performed. Control of lighting device 22 is similar to that of Embodiment 3. This gated camera 20B can also reduce the exposure flux when capturing images of distant imaging areas, enabling the acquisition of clear slice images.

[0153] Figure 18 This is a block diagram of an active sensor 70 according to Embodiment 5. The active sensor 70 is a gated camera, a ToF camera, a LIDAR, or the like, and includes a light emitting device 72 , a light sensor 4 , and a controller 76 .

[0154] The light emitting device 72 emits multiple pulses of light during one sensing operation. In this embodiment, the time interval T of the light emitting device 72 is INT Uneven. Time interval T INT It can be randomly determined in each cycle or it can be changed according to a predetermined pattern. INT It can increase monotonically or decrease monotonically in each action cycle Tc.

[0155] Light L1 emitted by light emitting device 72 is reflected by object OBJ and enters optical sensor 74. Reflected light L2 is delayed by τ relative to emitted light L1. τ corresponds to the distance z to object OBJ and is expressed by equation (1). τ is called the round-trip time of light.

[0156] τ=2×z / c…(1)

[0157] c represents the speed of light.

[0158] The optical sensor 74 controls the exposure timing and exposure time so as to detect each pulse included in the reflected light L1 in synchronization with each emission of the light emitting device 72. The emission timing of the light emitting device 72 and the exposure timing of the optical sensor 74 are controlled by the controller 74.

[0159] Due to the multiple emission of light by the light emitting device 72, the reflected light L2 from the object enters the light emitting sensor 74 multiple times. The light emitting sensor 74 integrates the reflected light received multiple times and outputs a signal according to the integrated value.

[0160] Figure 19This is a timing diagram showing the relationship between the light emission timing of light emitting device 72 and the exposure timing of light sensor 74. For example, the start time of light exposure of light sensor 74 may be delayed by a certain time Td from the start time of light emission of light emitting device 72. The method for determining time difference Td depends on the type and configuration of active sensor 70, and may be based on, for example, the round-trip time τ to the object to be imaged. Exposure time Te may be the same as or longer than the light emission time (pulse width) Tp of light emitting device 72.

[0161] One sensing operation, which includes one light emission by the light emitting device 72 and one exposure by the light sensor, may be repeated at a predetermined period Tc. In this case, the light emission timing of the light emitting device 72 within the predetermined period Tc (the time Tx from the start of the period Tc to the light emission) may also be changed for each sensing operation. The time interval T between the i-th and i+1-th light emission timings is INTi It is expressed by formula (2).

[0162] T INTi =Tc+Tx i+1 -Tx i …(2)

[0163] The above is the structure of the active sensor 70. Next, its operation will be described.

[0164] To clarify the advantages of the active sensor 70, a comparative technique is first described. In the comparative technique, the light emitting device emits light at a constant time interval, and the light sensor is also exposed at a constant time interval. Figure 20 This is the operation waveform diagram of the active sensor of the comparative technology. Now, suppose that there is another light source near the active sensor that emits pulses with a period Tc' that is an integer multiple or an integer fraction of the operation period Tc. Figure 20 In FIG. 5 , the light from the other light source is represented as disturbance light L3 . In this example, the emission period Tc′ of the disturbance light is equal to the operation period Tc of the active sensor.

[0165] If each pulse of the interference light L3 is included in the exposure period Te, it will affect the sensing of the active sensor. INT Since the exposure time is constant (Tc), the exposure timing interval is also constant (Tc).

[0166] Therefore, each pulse of the disturbance light L3 is always included in the exposure time Te, and its influence is accumulated to form a large error. Figure 20 In the figure, the error components are indicated by hatching.

[0167] Next, the operation of the active sensor 70 according to the fifth embodiment will be described. Figure 21 yes Figure 18The operation waveform diagram of the active sensor 70 is shown.

[0168] As described above, if each pulse of the disturbance light L3 is included in the exposure period Te, the sensing of the active sensor 70 is affected. INT The exposure start time is different for each operation cycle Tc. Therefore, each pulse of the disturbance light L3 is included in or not included in the exposure time Te. Figure 20 In the example shown in FIG, the first pulse of the disturbance light L3 is erroneously detected, but the subsequent pulse is not erroneously detected because it deviates from the exposure period. Therefore, the influence of the disturbance light L3 can be reduced, and high-precision detection can be performed.

[0169] Next, the purpose of the active sensor 70 will be described. One embodiment of the active sensor 70 is a gated camera.

[0170] Figure 22 2 is a block diagram of a gate control camera 20 according to an embodiment. The gate control camera 20 is divided into a plurality of N (N≧2) imaging ranges RNG1 to RNG N And shoot.

[0171] The gate control camera 20 includes an illumination device 22, an image sensor 24, a camera controller 26, and an image processing unit 28. The illumination device 22 and the image sensor 24 are connected. Figure 18 The light emitting device 72 corresponds to the image sensor 24 and Figure 18 The camera controller 26 corresponds to the optical sensor 74. Figure 18 Corresponding to the controller 76.

[0172] The lighting device 22 irradiates a plurality of pulses of probe light L1 in front of the vehicle in synchronization with a light emission timing signal S1 supplied from the camera controller 26. The probe light L1 is preferably infrared light, but is not limited thereto and may also be visible light of a predetermined wavelength. As described above, the pulses are spaced at uneven intervals.

[0173] The image sensor 24 is configured to generate a slice image IMG by performing exposure control in synchronization with an exposure timing signal S2 supplied from the camera controller 26. The image sensor 24 is sensitive to the same wavelength as the probe light L1 and captures reflected light (return light) L2 reflected from the object OBJ.

[0174] The camera controller 26 maintains the predetermined light emission timing and exposure timing for each photographing range RNG. iWhen the range is RNG1, the light emission timing signal S1 and the exposure timing signal S2 are generated based on the light emission timing and exposure timing corresponding to the range and the image is taken. N Corresponding multiple slice images IMG1~IMG N The i-th slice image IMG i The corresponding range RNG is captured in i The objects contained in .

[0175] Figure 23 It is a diagram for explaining the operation of the gate camera 20. Figure 23 Shows the measurement of the i-th range RNG i The lighting device 22 is synchronized with the lighting timing signal S1 and emits light during the lighting period τ1 between time t0 and t1. At the top, a graph of light is shown with the horizontal axis representing time and the vertical axis representing distance. i The distance to the nearest boundary is set to d MINi , will be in the range RNG i The distance from the deep side boundary is set to d MAXi .

[0176] At a certain moment, the light emitted from the lighting device 22 reaches a distance d MINi The round trip time T until the reflected light returns to the image sensor 24 is MINi T MINi =2×d MINi / c. c is the speed of light.

[0177] Similarly, at a certain moment, the light emitted from the lighting device 22 reaches the distance d MAXi The round trip time T until the reflected light returns to the image sensor 24 is MAXi T MAXi =2×d MAXi / c.

[0178] In the range RNG i When the camera controller 26 captures the object OBJ contained in MINi Exposure starts at time t3 = t1 + T MAXi The exposure timing signal S2 is generated in such a way that the exposure is ended at the same time. This is one exposure operation.

[0179] In the i-th range RNG i When photographing, light emission and exposure are repeated multiple times, and the measurement results are accumulated in the image sensor 24 .

[0180] Figure 24 (a) Figure 24(b) is a diagram illustrating an image obtained by the gate camera 20. Figure 24 In the example of (a), there is an object (pedestrian) OBJ1 in the range RNG1, and an object (vehicle) OBJ3 in the range RNG3. Figure 24 (b) shows that Figure 24 When the slice image IMG1 is captured, the image sensor is exposed only by the reflected light from the range RNG1 , so the object image OBJ1 of the pedestrian OBJ1 is captured in the slice image IMG1 .

[0181] When capturing the slice image IMG2 , the image sensor is exposed by the reflected light from the range RNG2 , and therefore no object is captured in the slice image IMG2 .

[0182] Similarly, when capturing slice image IMG3, the image sensor is exposed to the reflected light from range RNG3, so only object image OBJ3 is captured in slice image IMG3. In this way, the gated camera 20 can separate objects for each range and capture them.

[0183] The above is the operation of the gate camera 20. In this gate camera, by making the time intervals of light emission of the illumination device 22 non-uniform, the influence of the surrounding pulse light source can be reduced, and a clear image with less noise components can be obtained.

[0184] A modification example related to the third embodiment will be described.

[0185] (Variation 1)

[0186] The time interval T for the pulse light emission of the light emitting device 72 is INT The method of change is not limited to Figure 19 For example, Tx may be set constant and the operation period Tc may be changed every period.

[0187] (Variation 2)

[0188] The active sensor 70 is not limited to a gated camera, and may be a TOF (Time Of Flight) camera or a LIDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging).

[0189] Figure 25This figure shows a vehicle lamp 200 that incorporates a gate control camera 20 or an active sensor 70 (hereinafter referred to as a gate control camera). The vehicle lamp 200 includes a housing 210, an outer lens 220, high-beam and low-beam lamp units 230H / 230L, and the gate control camera 20. The lamp units 230H / 230L and the gate control camera 20 are housed in the housing 210.

[0190] Furthermore, a portion of the gate camera 20 , for example, the image sensor 24 , may be provided outside the vehicle lamp 200 , for example, on the back side of the interior mirror.

[0191] Figure 26 This is a block diagram illustrating a vehicle lamp 200 equipped with an object recognition system 10. The vehicle lamp 200 and the vehicle-side ECU 304 together constitute a lamp system 310. The vehicle lamp 200 includes a light source 202, a lighting circuit 204, and an optical system 206. Furthermore, the vehicle lamp 200 is equipped with the object recognition system 10. The object recognition system 10 includes a gate control camera 20 (or an active sensor 70) and a processing unit 40.

[0192] The calculation processing device 40 is configured to be able to obtain the information obtained by the gate control camera 20 and the plurality of imaging ranges RNG1 to RNG N Corresponding multiple slice images IMG1~IMG N To identify the type of object. The processing unit 40 includes a classifier implemented based on a prediction model generated by machine learning. The classifier algorithm is not particularly limited, and may be YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector), R-CNN (Region-based Convolutional Neural Network), SPPnet (Spatial Pyramid Pooling), Faster R-CNN, DSSD (Deconvolution–SSD), Mask R-CNN, etc., or an algorithm to be developed in the future may be used.

[0193] The processing unit 40 can be implemented by a combination of a processor (hardware) such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or a microcomputer, and a software program executed by the processor (hardware). The processing unit 40 can also be a combination of multiple processors. Alternatively, the processing unit 40 can be composed solely of hardware.

[0194] Information related to objects OBJ detected by the object recognition system 10 can also be used to control the light distribution of the vehicle lamp 200. Specifically, the lamp-side ECU 208 generates an appropriate light distribution pattern based on information related to the type and position of the object OBJ generated by the object recognition system 10. The lighting circuit 204 and the optical system 206 operate to achieve the light distribution pattern generated by the lamp-side ECU 208.

[0195] Furthermore, information on the object OBJ detected by the processing unit 40 may be transmitted to the vehicle-side ECU 304. The vehicle-side ECU may use this information for autonomous driving or driving assistance.

[0196] The present invention is described using specific sentences based on the implementation mode, but the implementation mode only represents one aspect of the principle and application of the present invention. In the implementation mode, many modifications and configuration changes can be confirmed without departing from the scope of the idea of ​​the present invention defined in the claims.

[0197] In summary, the present invention can be summarized as follows:

[0198] Solution 1. A gated camera, comprising:

[0199] an illumination device for irradiating the detection light,

[0200] Image sensor,

[0201] a camera controller that divides the depth direction into a plurality of shooting ranges, and controls the timing of irradiation of the detection light by the illumination device and the timing of exposure of the image sensor while moving the shooting range, so that the slice image output by the image sensor in each shooting range only includes the object included in the corresponding shooting range; and

[0202] The image processing unit generates a final image including the entire object of interest based on the plurality of slice images corresponding to the plurality of imaging ranges.

[0203] Solution 2. The gated camera as described in Solution 1, characterized in that:

[0204] The final image only contains the object of interest.

[0205] Solution 3. The gated camera as described in Solution 1 or 2, characterized in that:

[0206] The image processing unit synthesizes the plurality of slice images to generate a synthesized image in which each pixel has distance information, and extracts a region including the object of interest from the synthesized image.

[0207] Solution 4. A gated camera, comprising:

[0208] an illumination device for irradiating the detection light,

[0209] Image sensor,

[0210] a camera controller that divides the depth direction into a plurality of shooting ranges, and controls the timing of irradiation of the detection light by the illumination device and the timing of exposure of the image sensor while moving the shooting range, so that the slice image output by the image sensor in each shooting range only includes the object included in the corresponding shooting range; and

[0211] The image processing unit synthesizes a plurality of slice images corresponding to a plurality of imaging ranges to generate a synthesized image in which each pixel has distance information.

[0212] Solution 5. An automobile, comprising:

[0213] A gated camera as described in any one of solutions 1 to 4, and

[0214] An arithmetic processing device processes the output of the gated camera.

[0215] Solution 6. A vehicle lamp, characterized in that:

[0216] Includes a gated camera as described in any one of options 1 to 4.

[0217] Solution 7. An image processing device for a gated camera, characterized in that:

[0218] The gated camera comprises:

[0219] an illumination device for irradiating the detection light,

[0220] image sensors, and

[0221] a camera controller that divides the depth direction into a plurality of shooting ranges and controls the timing of irradiation of the detection light by the illumination device and the timing of exposure of the image sensor while moving the shooting ranges so that the slice image output by the image sensor in each shooting range only includes objects included in the corresponding shooting range;

[0222] The image processing device extracts slice images including an object of interest from among a plurality of slice images corresponding to the plurality of imaging ranges, and synthesizes the extracted slice images to generate a final image.

[0223] Solution 8. An image processing method is an image processing method in a gated camera, characterized in that:

[0224] The gated camera comprises:

[0225] an illumination device for irradiating the detection light,

[0226] image sensors, and

[0227] a camera controller that divides the depth direction into a plurality of shooting ranges and controls the timing of irradiation of the detection light by the illumination device and the timing of exposure of the image sensor while moving the shooting ranges so that the slice image output by the image sensor in each shooting range only includes objects included in the corresponding shooting range;

[0228] The image processing method comprises:

[0229] a step of extracting a slice image including an object of interest from among a plurality of slice images corresponding to the plurality of imaging ranges, and

[0230] The step of synthesizing the extracted slice images to generate the final image.

[0231] Solution 9. An image processing device for a gated camera, characterized in that:

[0232] The gated camera comprises:

[0233] an illumination device for irradiating the detection light,

[0234] image sensors, and

[0235] a camera controller that divides the depth direction into a plurality of shooting ranges and controls the timing of irradiation of the detection light by the illumination device and the timing of exposure of the image sensor while moving the shooting ranges so that the slice image output by the image sensor in each shooting range only includes objects included in the corresponding shooting range;

[0236] The image processing device synthesizes a plurality of slice images corresponding to the plurality of imaging ranges to generate a synthesized image in which each pixel has distance information.

[0237] Solution 10. An image processing method is an image processing method in a gated camera, characterized in that:

[0238] The gated camera comprises:

[0239] an illumination device for irradiating the detection light,

[0240] image sensors, and

[0241] a camera controller that divides the depth direction into a plurality of shooting ranges and controls the timing of irradiation of the probe light by the illumination device and the timing of exposure of the image sensor while moving the shooting ranges so that a slice image output by the image sensor in each shooting range only includes objects included in the corresponding shooting range;

[0242] The image processing method includes the following steps: synthesizing a plurality of slice images corresponding to the plurality of shooting ranges to generate a synthesized image in which each pixel has distance information.

[0243] Solution 11. A gated camera, comprising:

[0244] an illumination device for irradiating the detection light,

[0245] Image sensor,

[0246] a camera controller that divides the depth direction into a plurality of shooting ranges, and controls the timing of irradiation of the detection light by the illumination device and the timing of exposure of the image sensor while moving the shooting range, so that the slice image output by the image sensor in each shooting range only includes the object included in the corresponding shooting range; and

[0247] an image processing unit configured to receive the slice image from the image sensor and process the slice image;

[0248] The camera controller is configured to adaptively control the amount of data of the slice image transmitted from the image sensor to the image processing unit.

[0249] Solution 12. The gate-controlled camera according to Solution 11, characterized in that:

[0250] The camera controller reduces the size of the transmitted slice image by cropping a portion of the slice image.

[0251] Solution 13. The gated camera according to Solution 12, characterized in that:

[0252] The slice image group corresponding to the imaging range group closer than the specified distance is always transmitted without being cut;

[0253] For the slice image group corresponding to the imaging range group farther than the predetermined distance, whether or not to crop is controlled.

[0254] Solution 14. The gated camera according to Solution 12 or 13, wherein:

[0255] transmitting a plurality of slice images corresponding to the plurality of shooting ranges without cutting;

[0256] Based on the plurality of slice images that have not been cropped, it is determined whether the subsequent plurality of slice images are cropped.

[0257] Solution 15. The gated camera according to Solution 12, characterized in that:

[0258] In the first mode, all slice images corresponding to all shooting ranges are transmitted without cropping;

[0259] When the distant slice image group corresponding to the imaging range group farther than the predetermined distance transmitted in the first mode includes only a single object of interest, the mode is shifted to the second mode;

[0260] In the second mode, a target range including the target object is cut out from the distant slice image group and then transmitted.

[0261] Solution 16. The gated camera according to Solution 15, characterized in that:

[0262] The lighting device is configured to make the illumination range variable by focusing and diffusing the detection light;

[0263] The camera controller causes the illumination device to focus the probe light when capturing an image of a slice to be sheared and transmitted.

[0264] Solution 17. A gated camera, comprising:

[0265] An illumination device for irradiating probe light, wherein the illumination range is variable by focusing and diffusing the probe light.

[0266] Image sensor,

[0267] a camera controller that divides the depth direction into a plurality of shooting ranges, and controls the timing of irradiation of the detection light by the illumination device and the timing of exposure of the image sensor while moving the shooting range, so that the slice image output by the image sensor in each shooting range only includes the object included in the corresponding shooting range; and

[0268] an image processing unit configured to receive the slice image from the image sensor and process the slice image;

[0269] The camera controller controls the irradiation range of the lighting device based on the detection result of the image processing unit.

[0270] Solution 18. The gated camera as described in Solution 7, characterized in that

[0271] In the first mode, the detection light is irradiated to the entire irradiation range;

[0272] When the image processing unit detects the object of interest in the slice image within a predetermined imaging range in the first mode, the image processing unit shifts to the second mode;

[0273] In the second mode, when imaging the predetermined imaging range, the probe light is irradiated onto a region of interest including the object of interest.

[0274] Solution 19. An automobile, comprising:

[0275] A gated camera as described in any one of schemes 11 to 18, and

[0276] An arithmetic processing device processes the output of the gated camera.

[0277] Solution 20. A vehicle lamp, characterized in that:

[0278] Includes a gated camera as described in any one of options 11 to 18.

[0279] Solution 21. An active sensor, comprising:

[0280] A light emitting device that emits multiple pulses of light at uneven time intervals, and

[0281] The optical sensor detects reflected light from an object at a timing synchronized with each light emission of the light emitting device, and accumulates and outputs a plurality of detection results.

[0282] Solution 22. The active sensor according to Solution 21, characterized in that:

[0283] One time of sensing including one time of light emission by the light emitting device and one time of exposure by the light sensor is repeated in a predetermined cycle;

[0284] At each sensing time, the light emission timing of the light emitting device within the predetermined period changes.

[0285] Solution 23. The active sensor according to Solution 21 or 22, characterized in that:

[0286] The active sensor is a gated camera that divides the depth direction into a plurality of ranges and changes the time difference between light emission and imaging for each range, thereby acquiring a plurality of images corresponding to the plurality of ranges.

[0287] Solution 24. The active sensor according to Solution 21 or 22, characterized in that:

[0288] The active sensor is a TOF (Time Of Flight) camera.

[0289] Solution 25. The active sensor according to Solution 21 or 22, characterized in that:

[0290] The active sensor is LIDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging).

[0291] Solution 26. An object recognition system, comprising:

[0292] An active sensor as described in any one of schemes 21 to 25, and

[0293] A processing device capable of identifying the type of an object based on the image obtained by the active sensor.

[0294] Solution 27. A vehicle lamp, characterized in that:

[0295] Including an object recognition system as described in Option 26.

[0296] Solution 28. A vehicle, characterized in that:

[0297] Including an object recognition system as described in Option 26.

[0298] [Industrial Applicability]

[0299] The present invention relates to a gated camera.

[0300] [Explanation of Reference Numerals]

[0301] 10 Object Recognition System

[0302] 20 gated cameras

[0303] 22 lighting fixtures

[0304] 24 Image Sensor

[0305] 26 Camera Controller

[0306] 28 Image Processing Department

[0307] S1 light timing signal

[0308] S2 exposure timing signal

[0309] 40. Processing unit

[0310] 70 active sensors

[0311] 72 Lighting Device

[0312] 74 Light Sensor

[0313] 76 Controller

[0314] 200 Vehicle lighting

[0315] 210 housing

[0316] 220 outer lens

[0317] 230 Lighting unit 230

[0318] 310 lighting system

[0319] 304 Vehicle-side ECU

[0320] 200 Vehicle lighting

[0321] 202 Light Source

[0322] 204 Lighting Circuit

[0323] 206 Optical System

[0324] 310 lighting system

[0325] 304 Vehicle-side ECU

Claims

1. A gate control camera, characterized in that: include: an illumination device for irradiating the detection light, Image sensor, a camera controller for acquiring a plurality of slice images capturing only an object included in each of the plurality of imaging ranges divided in the depth direction by respectively changing the timing of irradiation of the probe light by the illumination device and the timing of exposure of the image sensor; and The image processing unit determines whether a predetermined object of interest is included across a plurality of slice images. If so, the image processing unit synthesizes the plurality of slice images to generate a final image that includes the entire object of interest and excludes an object included in only one of the plurality of slice images.

2. The door control camera according to claim 1, wherein: The predetermined object of interest is a vehicle, and the object included in only one slice image among the plurality of slice images is a pedestrian.

3. A gate control camera, characterized in that: include: an illumination device for irradiating the detection light, Image sensor, a camera controller for acquiring a plurality of slice images capturing only an object included in each of the plurality of imaging ranges divided in the depth direction by respectively changing the timing of irradiation of the probe light by the illumination device and the timing of exposure of the image sensor; and An image processing unit synthesizes the plurality of slice images to generate a composite image, and generates a final image based on the composite image, wherein Each pixel of the plurality of pixels forming the composite image includes: Brightness value information, which represents a brightness value selected according to a predetermined condition from among a plurality of brightness values ​​of corresponding pixels in the plurality of slice images; distance information indicating a distance corresponding to one of the plurality of slice images including a pixel having a selected brightness value; The final image includes at least one object of interest extracted based on the distance information.

4. A car, characterized in that: include: The gated camera according to any one of claims 1 to 3, and An arithmetic processing device processes the output of the gated camera.

5. A vehicle lamp, characterized in that: Comprising a gated camera as claimed in any one of claims 1 to 3.

6. An image processing device for a gated camera, characterized in that: The gated camera comprises: an illumination device for irradiating the detection light, image sensors, and a camera controller configured to acquire a plurality of slice images capturing only an object included in each of the plurality of imaging ranges divided in the depth direction by respectively changing the timing of irradiation of the probe light by the illumination device and the timing of exposure of the image sensor; The image processing device determines whether a predetermined object of interest is included across multiple slice images. If determined to be included, the image processing device synthesizes the multiple slice images to generate a final image that includes the entire object of interest and excludes objects included in only one slice image of the multiple slice images.

7. An image processing method is an image processing method in a gated camera, characterized in that: The gated camera comprises: an illumination device for irradiating the detection light, image sensors, and a camera controller configured to acquire a plurality of slice images capturing only an object included in each of the plurality of imaging ranges divided in the depth direction by respectively changing the timing of irradiation of the probe light by the illumination device and the timing of exposure of the image sensor; The image processing method comprises: A step of determining whether a predetermined object of interest is included across multiple slice images, and if so, synthesizing the multiple slice images to generate a final image that includes the entire object of interest and removes objects that are included in only one slice image of the multiple slice images.

8. An image processing device for a gated camera, characterized in that: The gated camera comprises: an illumination device for irradiating the detection light, image sensors, and a camera controller configured to acquire a plurality of slice images capturing only an object included in each of the plurality of imaging ranges divided in the depth direction by respectively changing the timing of irradiation of the probe light by the illumination device and the timing of exposure of the image sensor; The image processing device generates a composite image by synthesizing the plurality of slice images, and generates a final image based on the composite image. Each pixel of the plurality of pixels forming the composite image includes: Brightness value information, which represents a brightness value selected according to a predetermined condition from among a plurality of brightness values ​​of corresponding pixels in the plurality of slice images; distance information indicating a distance corresponding to one of the plurality of slice images including a pixel having a selected brightness value; The final image includes at least one object of interest extracted based on the distance information.

9. An image processing method is an image processing method in a gated camera, characterized in that: The gated camera comprises: an illumination device for irradiating the detection light, image sensors, and a camera controller for acquiring a plurality of images of only objects included in each of the plurality of imaging ranges divided in the depth direction by respectively changing the timing of irradiation of the probe light by the illumination device and the timing of exposure of the image sensor; The image processing method comprises the following steps: a step of synthesizing a plurality of slice images to generate a synthesized image, a step of generating a final image based on the synthesized image, Each pixel of the plurality of pixels forming the composite image includes: Brightness value information, which represents a brightness value selected according to a predetermined condition from among a plurality of brightness values ​​of corresponding pixels in a plurality of slice images; distance information indicating a distance corresponding to one of the plurality of slice images including a pixel having a selected brightness value; The final image includes at least one object of interest extracted based on the distance information.

Images