Electromagnetic wave detection device and ranging device

Abstract

An electromagnetic wave detection device and a ranging device are provided to reduce the deviation between the timing of image acquisition and electromagnetic wave detection. The electromagnetic wave detection device (10) includes: an image information acquisition unit (141) that acquires image information of the space where an object (ob) is located; a contour detection unit (142) that detects the contour of the object contained in the image information; an illumination system (111) that illuminates electromagnetic waves into the space; an illumination control unit (143) that controls the illumination system to illuminate electromagnetic waves in the space in a direction corresponding to the contour of the object; and a first detection unit (20) that detects the reflected wave after the electromagnetic wave is reflected by the object.

Description

[0001] This application claims priority to Japanese Patent Application No. 2020-089145 (filed on May 21, 2020), the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to electromagnetic wave detection devices and ranging devices. Background Technology

[0003] In recent years, devices have been developed that obtain information about the surrounding environment based on the detection results of multiple detectors that detect electromagnetic waves. For example, an electromagnetic wave detection device that reduces the difference in coordinate systems in the detection results of each detector is known (see Patent Document 1).

[0004] Existing technical documents

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 2018-200927 Summary of the Invention

[0007] The electromagnetic wave detection device according to the first aspect includes:

[0008] The image information acquisition unit acquires image information of the space where the object is located.

[0009] A contour detection unit detects the contour of the object contained in the image information;

[0010] An illumination system that irradiates electromagnetic waves into the space;

[0011] An illumination control unit controls the illumination system to illuminate the electromagnetic waves in the space in a direction corresponding to the outline of the object; and

[0012] The first detection unit detects the reflected wave after the electromagnetic wave is reflected by the object.

[0013] Furthermore, the ranging device according to the second aspect includes:

[0014] The image information acquisition unit acquires image information of the space where the object is located.

[0015] A contour detection unit detects the contour of the object contained in the image information;

[0016] An illumination system that irradiates electromagnetic waves into the space;

[0017] An illumination control unit controls the illumination system to irradiate the electromagnetic waves in the space in a direction corresponding to the outline of the object.

[0018] A first detection unit detects the reflected wave of the electromagnetic wave after it has been reflected by the object; and

[0019] The calculation unit calculates the distance between itself and the object based on the detection information from the first detection unit. Attached Figure Description

[0020] Figure 1 This is a structural diagram showing a schematic structure of an electromagnetic wave detection device according to one embodiment.

[0021] Figure 2 It is used for explanation Figure 1 The diagram shows the direction of electromagnetic wave propagation in the first and second states of the electromagnetic wave detection device.

[0022] Figure 3 This is a timing diagram used to illustrate distance calculations.

[0023] Figure 4 It is a diagram representing an example of an image of the space in which an object resides.

[0024] Figure 5 It means Figure 4 The outline of the image.

[0025] Figure 6 This is a diagram showing an example of a location irradiated by electromagnetic waves.

[0026] Figure 7 It is a graph representing the timing of image information output, contour detection, and illumination control.

[0027] Figure 8 This is a flowchart illustrating the processing of the control unit.

[0028] Figure 9 This is a structural diagram showing a simplified structure of a modified example of an electromagnetic wave detection device. Detailed Implementation

[0029] Figure 1 This is a structural diagram showing a schematic structure of an electromagnetic wave detection device 10 according to one embodiment. The electromagnetic wave detection device 10 is configured to include an illumination system 111, a light receiving system 110, and a control unit 14. In this embodiment, the case where the electromagnetic wave detection device 10 has one illumination system 111 and one light receiving system 110 has been described, but the illumination system 111 and the light receiving system 110 are not limited to one, and may also be a structure in which each of a plurality of light receiving systems 110 corresponds to each of a plurality of illumination systems 111.

[0030] The illumination system 111 includes an illumination section 12 and a deflection section 13. The light receiving system 110 includes an incident section 15, a separation section 16, a first detection section 20, a second detection section 17, a switching section 18, and a first post-stage optical system 19. The control section 14 includes an image information acquisition section 141, a contour detection section 142, an illumination control section 143, a light receiving control section 144, and a calculation section 145. Detailed descriptions of each functional block of the electromagnetic wave detection device 10 according to this embodiment will be provided later.

[0031] In the accompanying drawings, the dashed lines connecting the functional blocks represent the information flow of control signals or communication. The communication shown by the dashed lines can be wired or wireless. Additionally, the solid arrows represent bundles of electromagnetic waves. Furthermore, in the drawings, object ob is the subject of the electromagnetic wave detection device 10. The subject may include objects such as roads, median strips, sidewalks, roadside trees, vehicles, etc., and may also include people. Furthermore, object ob is not limited to one.

[0032] The electromagnetic wave detection device 10 can identify a subject by acquiring an image containing the subject and detecting the reflected wave after it is reflected by the subject. For example, the electromagnetic wave detection device 10 may be a driving assistance device mounted on a vehicle or the like that detects an approaching object ob while driving and notifies the driver. The electromagnetic wave detection device 10 according to this embodiment includes a calculation unit 145, which, as described below, functions as a ranging device that measures the distance to the object ob by means of the calculation unit 145.

[0033] (Irradiation system)

[0034] The irradiation system 111 irradiates electromagnetic waves into the space where the object ob is located. In this embodiment, the irradiation system 111 irradiates the electromagnetic waves irradiated by the irradiation unit 12 into the space where the object ob is located via the deflection unit 13. As another example, the irradiation system 111 may be a structure in which the irradiation unit 12 directly irradiates electromagnetic waves into the object ob.

[0035] The irradiation unit 12 irradiates at least one of infrared light, visible light, ultraviolet light, and radio waves. In this embodiment, the irradiation unit 12 irradiates infrared light. Additionally, in this embodiment, the irradiation unit 12 irradiates a narrow-width (e.g., 0.5°) beam of electromagnetic waves. Furthermore, the irradiation unit 12 irradiates electromagnetic waves in a pulsed manner. The irradiation unit 12 may be configured to include, for example, an LED (Light Emitting Diode) as the electromagnetic wave irradiation element. Alternatively, the irradiation unit 12 may be configured to include, for example, an LD (Laser Diode) as the electromagnetic wave irradiation element. The irradiation unit 12 switches and stops the electromagnetic wave irradiation based on the control of the control unit 14. Here, the irradiation unit 12 may be composed of an LED array or LD array in which multiple electromagnetic wave irradiation elements are arranged in an array, simultaneously irradiating multiple beams.

[0036] The deflector 13 causes the electromagnetic waves irradiated by the irradiation unit 12 to output in multiple different directions, thereby changing the irradiation position of the electromagnetic waves irradiating the space where the object ob is located. Outputting in multiple different directions can be achieved by the deflector 13 reflecting the electromagnetic waves from the irradiation unit 12 while changing the direction. For example, the deflector 13 scans the object ob in a one-dimensional or two-dimensional direction. Here, if the irradiation unit 12 is configured as an LD array, for example, the deflector 13 reflects all the multiple beams output from the LD array and outputs them in the same direction. That is, the irradiation system 111 has a deflector 13 relative to the irradiation unit 12, which has one or more electromagnetic wave irradiation elements.

[0037] The deflector 13 is configured such that at least a portion of the space from which the electromagnetic wave is output, i.e., the irradiation area, is included within the detection range of the electromagnetic wave in the light receiving system 110. Therefore, at least a portion of the electromagnetic wave irradiated by the deflector 13 into the space where the object ob is located is reflected by at least a portion of the object ob, and thus can be detected in the light receiving system 110. Here, the electromagnetic wave after being reflected by at least a portion of the object ob from the electromagnetic wave output from the deflector 13 is called a reflected wave.

[0038] The deflection section 13 may include, for example, a MEMS (Micro Electro Mechanical Systems) mirror, a faceted mirror, and a galvanometer mirror. In this embodiment, the deflection section 13 includes a MEMS mirror.

[0039] The deflection unit 13 can change the direction of the reflected electromagnetic wave based on the control of the control unit 14. Furthermore, the deflection unit 13 may include an angle sensor, such as an encoder, and can notify the control unit 14 of the angle detected by the angle sensor as direction information of the reflected electromagnetic wave. In this configuration, the control unit 14 can calculate the illumination position of the electromagnetic wave based on the direction information obtained from the deflection unit 13. Alternatively, the control unit 14 can also calculate the illumination position based on a drive signal input to cause the deflection unit 13 to change the direction of the reflected electromagnetic wave.

[0040] (Optical receiving system)

[0041] In the following text, "electromagnetic wave containing reflected waves" refers to an electromagnetic wave that is incident on the light receiving system 110 and contains reflected waves from the object ob. That is, in order to distinguish it from electromagnetic waves irradiated by the irradiation system 111, electromagnetic waves incident on the light receiving system 110 are sometimes referred to as "electromagnetic waves containing reflected waves". Electromagnetic waves containing reflected waves include not only reflected waves from electromagnetic waves irradiated by the irradiation system 111 after being reflected by the object ob, but also external light such as sunlight, and light from external light reflected by the object ob.

[0042] The incident section 15 is an optical system having at least one optical component, which images the object ob, which is the subject. The optical component includes at least one of, for example, a lens, a mirror, an aperture, and an optical filter.

[0043] A separation unit 16 is disposed between the incident unit 15 and a primary imaging position, which is the imaging position of the image of the object ob that is positioned away from the incident unit 15 at a predetermined position. The separation unit 16 separates the electromagnetic wave, which includes the reflected wave, into waves that travel along a first direction d1 or a second direction d2, depending on the wavelength.

[0044] In this embodiment, the separating unit 16 reflects a portion of the electromagnetic wave containing the reflected wave in the first direction d1 and transmits another portion in the second direction d2. In this embodiment, the separating unit 16 reflects visible light, such as sunlight, from the incident electromagnetic wave after it has been reflected by the object ob, in the first direction d1. Additionally, the separating unit 16 transmits infrared light, such as infrared radiation from the irradiation unit 12, from the incident electromagnetic wave after it has been reflected by the object ob, in the second direction d2. As another example, the separating unit 16 can transmit a portion of the incident electromagnetic wave in the first direction d1 and reflect another portion in the second direction d2. Furthermore, the separating unit 16 can refract a portion of the incident electromagnetic wave in the first direction d1 and refract another portion in the second direction d2. The separating unit 16 can be, for example, a semi-reflecting mirror, a beam splitter, a dichroic mirror, a cold reflector, a hot reflector, a metallic mirror, a deflecting element, or a prism.

[0045] The second detection unit 17 is disposed on the path of the electromagnetic wave traveling from the separation unit 16 along the first direction d1. The second detection unit 17 is disposed at or near the imaging position of the object ob in the first direction d1. The second detection unit 17 detects the electromagnetic wave traveling from the separation unit 16 along the first direction d1.

[0046] Alternatively, the second detection unit 17 can be configured relative to the separation unit 16 such that the first travel axis of the electromagnetic wave traveling from the separation unit 16 toward the first direction d1 is parallel to the first detection axis of the second detection unit 17. The first travel axis is the central axis of the electromagnetic wave that propagates radially from the separation unit 16 toward the first direction d1. In this embodiment, the first travel axis is an axis that extends the optical axis of the incident unit 15 to the separation unit 16 and bends in the separation unit 16 in a manner parallel to the first direction d1. The first detection axis is an axis that passes through the center of the detection surface of the second detection unit 17 and is perpendicular to the detection surface.

[0047] Furthermore, the second detection unit 17 can be configured such that the interval between the first travel axis and the first detection axis is less than or equal to a first interval threshold. Alternatively, the second detection unit 17 can be configured such that the first travel axis and the first detection axis are aligned. In this embodiment, the second detection unit 17 is configured such that the first travel axis and the first detection axis are aligned.

[0048] Furthermore, the second detection unit 17 can be configured relative to the separation unit 16 such that the first angle formed by the first travel axis and the detection surface of the second detection unit 17 is below a first angle threshold or a predetermined angle. In this embodiment, the second detection unit 17 is configured such that the first angle is 90°.

[0049] In this embodiment, the second detection unit 17 is a passive sensor. More specifically, in this embodiment, the second detection unit 17 includes an array of elements. For example, the second detection unit 17 includes an imaging element such as an image sensor or an imaging array, which captures an image of electromagnetic waves imaged on the detection surface and generates image information containing the space of the captured object ob.

[0050] In this embodiment, more specifically, the second detection unit 17 captures an image of visible light. The second detection unit 17 sends the generated image information as a signal to the control unit 14. The second detection unit 17 can capture images other than visible light, such as infrared, ultraviolet, and radio waves.

[0051] The switching unit 18 is disposed on the path of the electromagnetic wave traveling from the separation unit 16 to the second direction d2. The switching unit 18 is disposed at or near the primary imaging position of the object ob in the second direction d2.

[0052] In this embodiment, the switching unit 18 is provided at the imaging position. The switching unit 18 has an action surface as that allows electromagnetic waves that have passed through the incident unit 15 and the separation unit 16 to be incident. The action surface as is composed of a plurality of switching elements se arranged in a two-dimensional pattern. The action surface as is a surface that causes electromagnetic waves to have effects such as reflection and transmission in at least one of the first state and the second state described later.

[0053] The switching unit 18 can switch each switching element se to a first state where the electromagnetic wave incident on the action surface as travels in a third direction d3, and a second state where it travels in a fourth direction d4. In this embodiment, the first state is a first reflection state where the electromagnetic wave incident on the action surface as is reflected in a third direction d3. The second state is a second reflection state where the electromagnetic wave incident on the action surface as is reflected in a fourth direction d4.

[0054] In this embodiment, more specifically, the switching unit 18 includes a reflecting surface for reflecting electromagnetic waves in each switching element se. The switching unit 18 switches between a first reflection state and a second reflection state for each switching element se by arbitrarily changing the orientation of each reflecting surface of each switching element se.

[0055] In this embodiment, the switching unit 18 includes, for example, a DMD (Digital Micromirror Device). The DMD can switch the reflective surface relative to the operating surface as to either a +12° or -12° tilt state for each switching element se by driving the tiny reflective surface constituting the operating surface as. The operating surface as is parallel to the surface of the substrate on which the tiny reflective surface in the DMD is mounted.

[0056] Based on the control of the control unit 14, the switching unit 18 switches each switching element se to a first state and a second state. For example, as... Figure 2 As shown, the switching unit 18, by simultaneously switching a portion of the switching elements se1 to the first state, enables the electromagnetic wave incident on the switching element se1 to travel in the third direction d3, and by switching another portion of the switching elements se2 to the second state, enables the electromagnetic wave incident on the switching element se2 to travel in the fourth direction d4. More specifically, the control unit 14 detects the direction or position of the electromagnetic wave being irradiated based on the direction information from the deflection unit 13. Then, the switching element se1 corresponding to the detected irradiation direction or irradiation position of the electromagnetic wave is set to the first state, and the other switching elements se1 are set to the second state, thereby selectively causing the reflected wave from the target ob to travel in the third direction d3. The electromagnetic wave other than the reflected wave from the target ob, which has passed through the separation unit 16, travels in the fourth direction d4 and therefore will not be incident on the first detection unit 20.

[0057] like Figure 1 As shown, a first post-stage optical system 19 is provided on a third direction d3 starting from the switching unit 18. The first post-stage optical system 19 includes, for example, at least one of a lens and a mirror. The first post-stage optical system 19 images an object ob, which is an electromagnetic wave whose direction of travel has been switched in the switching unit 18.

[0058] The first detection unit 20 detects the reflected wave. The first detection unit 20 is positioned to detect the electromagnetic wave that travels from the switching unit 18 in the third direction d3 and then through the first post-stage optical system 19. The first detection unit 20 detects the electromagnetic wave that travels through the first post-stage optical system 19, that is, the electromagnetic wave that travels in the third direction d3, and outputs a detection signal.

[0059] Furthermore, the first detection unit 20, together with the switching unit 18, can be configured such that the second travel axis of the electromagnetic wave, which travels from the separation unit 16 in the second direction d2 and whose travel direction is switched by the switching unit 18 to the third direction d3, is parallel to the second detection axis of the first detection unit 20. The second travel axis is the central axis of the electromagnetic wave that propagates radially from the switching unit 18 to the third direction d3. In this embodiment, the second travel axis is an axis that extends the optical axis of the incident unit 15 to the switching unit 18 and bends in the switching unit 18 in a manner parallel to the third direction d3. The second detection axis is an axis that passes through the center of the detection surface of the first detection unit 20 and is perpendicular to the detection surface.

[0060] Furthermore, the first detection unit 20 can be configured together with the switching unit 18 such that the interval between the second travel axis and the second detection axis is less than or equal to a second interval threshold. The second interval threshold can be the same as or different from the first interval threshold. Additionally, the first detection unit 20 can be configured such that the second travel axis and the second detection axis are aligned. In this embodiment, the first detection unit 20 is configured such that the second travel axis and the second detection axis are aligned.

[0061] Furthermore, the first detection unit 20, together with the switching unit 18, can be configured relative to the separation unit 16 such that the second angle formed by the second travel axis and the detection surface of the first detection unit 20 is below a second angle threshold or a predetermined angle. The second angle threshold can be the same as the first angle threshold or a different value. In this embodiment, as described above, the first detection unit 20 is configured with a second angle of 90°.

[0062] In this embodiment, the first detection unit 20 is an active sensor that detects the reflected wave of electromagnetic waves irradiated from the irradiation unit 12 onto the object ob. The first detection unit 20 may include, for example, a single element such as an APD (Avalanche Photo Diode), a PD (Photo Diode), or a ranging image sensor. Alternatively, the first detection unit 20 may include an array of elements such as an APD array, a PD array, a ranging imaging array, and a ranging image sensor.

[0063] In this embodiment, the first detection unit 20 sends detection information indicating that a reflected wave from the subject has been detected as a signal to the control unit 14. More specifically, the first detection unit 20 detects electromagnetic waves in the infrared frequency band.

[0064] Furthermore, in this embodiment, the first detection unit 20 is used as a detection element for determining the distance to the object ob. In other words, the first detection unit 20 is a component constituting a ranging sensor, and it only needs to be able to detect electromagnetic waves; it does not need to image on the detection surface. Therefore, the first detection unit 20 does not need to be located at the imaging position, i.e., the secondary imaging position, of the first post-stage optical system 19. That is, in this structure, the first detection unit 20 only needs to be positioned where electromagnetic waves from all viewing angles can be incident on the detection surface, and it can be positioned at any position on the path of the electromagnetic waves that travel through the switching unit 18 in the third direction d3 and then through the first post-stage optical system 19.

[0065] By having the structure described above, the electromagnetic wave detection device 10 aligns a predetermined position on the image with the optical axis of the reflected wave used to determine the distance to that position.

[0066] (Control Department)

[0067] The image information acquisition unit 141 acquires image information of the space where the object ob is located from the second detection unit 17. More specifically, the image information acquisition unit 141 includes a row buffer that temporarily stores several rows of image information and outputs them to the contour detection unit 142. The row buffer is, for example, composed of a semiconductor memory or a magnetic memory.

[0068] The contour detection unit 142 detects the contour of object ob contained in the image information acquired from the image information acquisition unit 141. The contour detection unit 142 can detect the contour of object ob in one-dimensional or two-dimensional directions. The contour detection unit 142 can use, for example, the Laplacian method, the Sobel method, or the Canny method. The method of contour detection is not limited.

[0069] The illumination control unit 143 controls the illumination system 111. For example, the illumination control unit 143 causes the illumination unit 12 to switch between illuminating and stopping the electromagnetic waves. The illumination control unit 143 also causes the deflection unit 13 to change the direction of the reflected electromagnetic waves. As will be explained in detail later, the illumination control unit 143 controls the illumination system 111 to irradiate electromagnetic waves in space in a direction corresponding to the contour of the object ob, based on the contour information of the object ob detected by the contour detection unit 142.

[0070] The optical receiving control unit 144 controls the optical receiving system 110. For example, the optical receiving control unit 144 causes the switching unit 18 to switch the first state and the second state for each switching element se.

[0071] The calculation unit 145 calculates the distance between itself and the object ob based on the detection information from the first detection unit 20. The calculation unit 145 can calculate the distance based on the acquired detection information, for example, using a ToF (Time-of-Flight) method.

[0072] like Figure 3 As shown, the control unit 14 inputs an electromagnetic wave emission signal to the irradiation unit 12, causing the irradiation unit 12 to irradiate pulsed electromagnetic waves (see the "Electromagnetic Wave Emission Signal" column). The irradiation unit 12 irradiates electromagnetic waves based on the input electromagnetic wave emission signal (see the "Irradiation Unit Emission Amount" column). The electromagnetic waves irradiated by the irradiation unit 12 and reflected by the deflection unit 13, irradiating the space where the object ob is located, i.e., the irradiation area, are reflected in the irradiation area. The control unit 14 switches at least a portion of the switching elements se in the imaging area of ​​the switching unit 18 based on the incident unit 15 to a first state, and switches the other switching elements se to a second state. Furthermore, when the first detection unit 20 detects the reflected electromagnetic waves in the irradiation area (see the "Electromagnetic Wave Detection Amount" column), it notifies the control unit 14 of the detection information.

[0073] The arithmetic unit 145 acquires information about the signal, including the detection information. The arithmetic unit 145 includes, for example, a time-measuring LSI (Large Scale Integrated Circuit) that measures the time ΔT from the period T1 when the irradiation unit 12 is irradiated with electromagnetic waves to the period T2 when the detection information is acquired (see the "Detection Information Acquisition" column). The arithmetic unit 145 calculates the distance to the irradiation position by multiplying the time ΔT by the speed of light and dividing by 2.

[0074] Here, the control unit 14 may include one or more processors. The processors can load programs from accessible memory and operate as the image information acquisition unit 141, contour detection unit 142, illumination control unit 143, light receiving control unit 144, and arithmetic unit 145. The processors may include at least one of a general-purpose processor that performs a specific function by reading a specific program and a dedicated processor for a specific process. The dedicated processor may include an application-specific integrated circuit (ASIC). The processor may include a programmable logic device (PLD). The PLD may include a field-programmable gate array (FPGA). The control unit 14 may include at least one of a system-on-a-chip (SoC) and a system-in-a-package (SiP) in cooperation with one or more processors.

[0075] (The moment of image acquisition and electromagnetic wave detection)

[0076] The electromagnetic wave detection device 10 with the above-described structure can acquire an image including the object ob (which is the subject) via the second detection unit 17, and calculate the distance to the object ob by detecting the reflected wave via the first detection unit 20. Typically, the time required to detect electromagnetic waves containing reflected waves is longer than the time required to acquire one frame of an image using an imaging element. For example, an imaging element can acquire 30 frames of 1920×1080 pixel images per second. On the other hand, the time required to determine the distance at a single point in the event of receiving the reflected wave of the irradiated electromagnetic wave can sometimes be as long as 20 μs. Therefore, in the prior art of beam-based scanning of the entire space, the resolution of electromagnetic wave detection is very low when detecting electromagnetic waves based on the frame rate of the image acquisition by the imaging element. For example, when performing electromagnetic wave detection at 30 frames per second, only 54×31 points can be measured, making it impossible to properly determine the distance to the object ob. Furthermore, in the prior art, when the resolution of electromagnetic wave detection is increased, the frame rate of distance measurement decreases, resulting in a large deviation between the time of image acquisition and the time of electromagnetic wave detection. In the previous example, when electromagnetic wave detection was performed at 240×135 points, the frame rate for distance measurement was approximately 1.5, potentially corresponding to images more than 20 frames prior to the detection. In other words, during the time it takes for the distance measurement of one frame performed by the electromagnetic wave detection device to complete, the imaging element can acquire more than 20 frames of images, thus the acquisition time of the latest image differs significantly from the time of distance measurement. The electromagnetic wave detection device 10 of this embodiment, as described below, solves this problem by reducing the number of times electromagnetic waves are irradiated per frame through the illumination system 111 irradiating electromagnetic waves based on the contour information of the object ob.

[0077] Figure 4 This is an example of a diagram representing the space in which the object ob resides. In the following text, it will sometimes be referred to as... Figure 4 The image is referred to as captured image 50. Captured image 50 is a single-frame image, for example, with a size of 1920×1080 pixels. In captured image 50, the objects ob are the sky, mountains, road, white lines, side walls, and central divider. Captured image 50 is an image of the front of a moving vehicle equipped with electromagnetic wave detection device 10, with the center C located in front of the vehicle's direction of travel.

[0078] The image information acquisition unit 141 acquires the image signal of the captured image 50 from the second detection unit 17 on a line-by-line basis. That is, the image information acquisition unit 141 acquires the image information of the space where the object ob is located, not on a frame-by-frame basis, but on a partial information basis. In this embodiment, the image information acquisition unit 141 includes an 8-line buffer and acquires 8 lines of partial information as a unit. For example, the image information acquisition unit 141 acquires a portion of the image information, namely first partial information 51, and a portion of the image information, namely second partial information 52, which is different from the first partial information 51. The first partial information 51 and the second partial information 52 are each 8 lines of partial information. After acquiring the first partial information 51, the line buffer of the image information acquisition unit 141 overwrites and acquires the second partial information 52. Furthermore, after outputting the first partial information 51 to the contour detection unit 142, the image information acquisition unit 141 outputs the second partial information 52.

[0079] Figure 5 It means Figure 4 The outline of the captured image 50. In the following text, it will sometimes be referred to as... Figure 5 The image is called contour image 60. Contour image 60 corresponds to the image obtained by imagerizing the contour of the captured image 50 extracted by contour detection unit 142. In contour image 60, the sky, mountains, roads, white lines, side walls, and boundary portions of the central median strip, which are objects ob, are extracted and shown. The center C of contour image 60 is located at the same position as the captured image 50.

[0080] The contour detection unit 142 acquires image signals of the captured image 50 from the image information acquisition unit 141 on a row-by-row basis. The contour detection unit 142 is capable of acquiring image signals within the capacity range of the row buffer of the image information acquisition unit 141. As in this embodiment, the contour detection unit 142 can acquire partial information row by row, performing contour detection based on the partial information of each acquired row. As another example, the contour detection unit 142 can acquire partial information of 8 rows that can be stored in the row buffer of the image information acquisition unit 141 at once, performing contour detection based on the acquired 8 rows of partial information. In this embodiment, the contour detection unit 142 outputs contour information for the 8 rows of partial information to the illumination control unit 143 as a unit. For example, after detecting the contour of object ob (i.e., the first contour) based on the first partial information 51 of the 8 rows, the contour detection unit 142 detects the contour of object ob (i.e., the second contour) based on the second partial information 52 of the 8 rows. Furthermore, after outputting the first contour information 61 to the illumination control unit 143, the contour detection unit 142 outputs the second contour information 62 to the illumination control unit 143.

[0081] Figure 6This is a diagram showing an example of the location where the irradiation system 111 irradiates electromagnetic waves. Figure 6 The first contour information 61 shown is to Figure 5 The first contour information 61 is magnified. The 12 lines contained in the first contour information 61 represent the contour of the object ob contained in the partial information. When the first contour information 61 is acquired, the illumination control unit 143 controls the illumination system 111 to illuminate electromagnetic waves in the direction corresponding to the contour of the object ob in space. That is, the illumination control unit 143 controls the illumination system 111 to illuminate pulsed electromagnetic waves when the illumination direction corresponds to the contour of the object ob, and not to illuminate electromagnetic waves and move to the next contour position when the illumination direction does not correspond to the contour of the object ob. In the electromagnetic wave detection device 10, since the light used to acquire an image of a predetermined position in space is aligned with the optical axis of the reflected wave used to measure the distance to that position, electromagnetic waves can be accurately illuminated in the direction corresponding to the contour of the object ob.

[0082] Here, the outline of object ob sometimes has length in the horizontal or vertical direction. For example, the illumination control unit 143 can determine the position of the midpoint of a continuous outline as the outline position P1 to P2. 12 In actual space, the contour positions P1 to P1 are... 12 Electromagnetic waves are irradiated in the corresponding direction. Upon acquiring the second contour information 62, the irradiation control unit 143 also controls the irradiation system 111 to irradiate electromagnetic waves in the space in the direction corresponding to the contour of the object ob. That is, after irradiating electromagnetic waves in the space in the direction corresponding to the first contour, the irradiation control unit 143 irradiates electromagnetic waves in the space in the direction corresponding to the second contour.

[0083] Direction and contour position P1~P 12 Electromagnetic waves are irradiated in the corresponding direction, enabling reliable irradiation of electromagnetic waves towards the location of the object ob whose distance is to be measured. For example, if electromagnetic waves are irradiated at equal intervals, sometimes small objects ob located between irradiation points will not be irradiated. The electromagnetic wave detection device 10 of this embodiment irradiates electromagnetic waves based on the contour information of the object ob through the irradiation system 111, thereby enabling reliable irradiation of electromagnetic waves towards the direction in which the object ob exists.

[0084] The upper limit value can be determined based on the time taken for the first detection unit 20 to detect the reflected wave. This upper limit value is the number of points that the illumination system 111 can illuminate per frame of electromagnetic waves. For example, when the second detection unit 17 detects 30 frames of a 1920×1080 pixel image per second, it detects 8 rows of partial information that the row buffer can store within 240 μs. However, considering the blanking period, the detection time for one row of the image is 30 μs. In contrast, the time required for the first detection unit 20 to detect the reflected wave of the illuminated electromagnetic wave is 20 μs per point. The illumination of the electromagnetic waves by the illumination system 111 is performed based on the contour detection of 8 rows of partial information. Therefore, in this example, the upper limit value is 12, obtained by dividing 240 μs by 20 μs. The illumination control unit 143 controls the illumination system 111 to move the illumination direction, for example, along the horizontal direction, and illuminate the electromagnetic waves with points below the upper limit value. The illumination system 111 can, for example, perform electromagnetic wave illumination of the upper limit value during a period when the illumination direction is biased in one direction, or it can perform electromagnetic wave illumination of the upper limit value during a period when the illumination direction is moved back and forth. As a specific example of the latter, the irradiation system 111 can deflect in a round trip manner in the horizontal direction for 240 μs, performing electromagnetic wave irradiation at 6 points during the deflection in one direction and performing electromagnetic wave irradiation at another 6 points during the deflection in the other direction.

[0085] Here, the third part information 53 in the captured image 50 is partial information of an 8-line empty image. The third contour information 63 in the contour image 60 corresponds to the third part information 53, but does not contain any contours. For example, when the illumination control unit 143 acquires the third contour information 63, the illumination control unit 143 can prevent the illumination system 111 from irradiating electromagnetic waves. The illumination control unit 143 can irradiate electromagnetic waves to points less than the upper limit value if the number of contours contained in the acquired contour information is less than the upper limit value. That is, the illumination control unit 143 can irradiate electromagnetic waves only in the direction corresponding to the contour in space.

[0086] When the number of object ob outlines exceeds an upper limit, the illumination control unit 143 can illuminate electromagnetic waves in a direction corresponding to the outline of the object ob with a higher priority than the upper limit. The priority order can be determined, for example, by making the outline of the object ob closer to the center C of the captured image 50 in space appear higher. That is, compared to object obs located in the periphery of the captured image 50, object obs closer to the center C can be determined to have a higher priority. As another example, the priority order can be determined by making the clearest outline in the outline appear higher. That is, the portion that is definitely the boundary of the object ob can be detected first. As another example, the priority order can be determined by making the outline of the object ob located in the closer portion appear higher. That is, the detection of nearby object obs that require immediate action such as hazard avoidance can be prioritized. Furthermore, as another example, the priority order can be determined based on the moving speed of the vehicle or the like equipped with the electromagnetic wave detection device 10. That is, when the moving speed is high, object obs located far away can be prioritized, and when the moving speed is slow, object obs located nearby can be prioritized. Additionally, as another example, the priority order can be determined based on the type of object (ob) identified from the captured image, the size of the object (ob), and the speed at which the object (ob) moves to the left or right.

[0087] Figure 7This diagram illustrates the timing of image information output, contour detection, and illumination control. The image information acquisition unit 141 outputs partial information (e.g., first partial information 51) from eight rows temporarily stored in the row buffer to the contour detection unit 142. In this example, the image information acquisition unit 141 outputs partial information from one row to the contour detection unit 142 in eight parts. The contour detection unit 142 performs contour detection each time it acquires partial information from one row. The contour detection unit 142 performs contour detection on the first partial information 51 and outputs first contour information 61 to the illumination control unit 143. The illumination control unit 143 controls the illumination system 111 to irradiate electromagnetic waves in a direction corresponding to the contour of the object ob in space based on the first contour information 61. At this time, the image information acquisition unit 141 outputs partial information (e.g., second partial information 52) from eight rows temporarily stored in the row buffer to the contour detection unit 142. The contour detection unit 142 performs contour detection on the second partial information 52 and outputs second contour information 62 to the illumination control unit 143. The illumination control unit 143 controls the illumination system 111 to irradiate electromagnetic waves in a direction corresponding to the contour of the object ob in space, based on the second contour information 62. In this way, the image information acquisition unit 141, the contour detection unit 142, and the illumination control unit 143 perform processing in parallel, treating a portion of the image information, i.e., partial information, as a unit. At this time, the delay from image acquisition to the illumination system 11 irradiating electromagnetic waves includes the delay when storing the partial information in the line buffer and the delay caused by the contour detection processing.

[0088] (Electromagnetic wave detection method)

[0089] The control unit 14 of the electromagnetic wave detection device 10 involved in this embodiment is, for example, according to Figure 8 The flowchart executes electromagnetic wave detection for one frame.

[0090] The image information acquisition unit 141 of the control unit 14 acquires image information of the space where the object ob is located from the first detection unit 20 (step S1). The image information acquisition unit 141 of the control unit 14 acquires image information in units of partial information of a portion rather than the entire frame.

[0091] The contour detection unit 142 of the control unit 14 detects the contour of the object ob contained in the partial information output from the image information acquisition unit 141 (step S2).

[0092] The illumination control unit 143 of the control unit 14 causes the illumination system 111 to irradiate electromagnetic waves in a direction corresponding to the outline of the object ob in space. The control unit 14 sets the next part information in parallel (step S3). The setting of the next part information is, for example, changing the parameters in the control unit 14 (as a specific example, specifying the row position) so that the control of irradiating electromagnetic waves is performed based on the first part information 51.

[0093] If the acquisition of one frame of image information is completed (step S4 is "Yes"), the control unit 14 ends the series of processes. If no frame of image information is acquired (step S4 is "No"), the control unit 14 returns to the processing of step S1.

[0094] As described above, the electromagnetic wave detection device 10 of this embodiment, according to the above structure, irradiates electromagnetic waves in a direction corresponding to the outline of the object ob in space. Therefore, the electromagnetic wave detection device 10 can reliably irradiate electromagnetic waves to the location of the object ob even without improving the resolution of electromagnetic wave detection. Therefore, the deviation between the time of image acquisition and the time of electromagnetic wave detection can be reduced.

[0095] Furthermore, preferably, the electromagnetic wave detection device 10 acquires a first portion of information 51, irradiates electromagnetic waves in the direction of the contour position based on the first contour information 61 acquired from the first portion of information 51, and acquires a second portion of information 52. Through this parallel processing, the electromagnetic wave detection device 10 can further reduce the deviation between the time of image acquisition and the time of electromagnetic wave detection. That is, it can further reduce the deviation between the time of acquiring an image of the object ob and the time of ranging measurement of the object ob.

[0096] In this embodiment, the electromagnetic wave is irradiated in a direction corresponding to the outline of the object ob. When the outline of the object ob cannot be detected based on partial information, or when the detectable outline portion is less than an upper limit value, the electromagnetic wave can be irradiated in a direction corresponding to the portion of the outline that cannot be detected within the upper limit value range. In this case, the electromagnetic wave detection device 10 can, for example, irradiate electromagnetic waves at equal intervals.

[0097] (Modified Example)

[0098] Although this disclosure has been described with reference to the accompanying drawings and embodiments, it should be noted that various modifications and variations can be readily made based on this disclosure by those skilled in the art. Therefore, it should be understood that such modifications and variations are included within the scope of this disclosure.

[0099] In the above-described embodiment, as described above, the electromagnetic wave detection device 10 generates distance information using DirectToF, which directly measures the time from the irradiation of the laser to the return of the laser. However, the electromagnetic wave detection device 10 is not limited to this structure. For example, the electromagnetic wave detection device 10 can generate distance information using FlashToF, which irradiates electromagnetic waves at a certain period and indirectly measures the time until the return based on the phase difference between the irradiated electromagnetic wave and the returned electromagnetic wave. Alternatively, the electromagnetic wave detection device 10 can generate distance information using other ToF methods, such as PhasedToF.

[0100] In the above-described embodiment, the switching unit 18 can switch the direction of travel of the electromagnetic wave incident on the action surface as to two directions, but it can also switch to three or more directions instead of switching to either of the two directions.

[0101] In the switching unit 18 of the above-described embodiment, the first state and the second state are the first reflection state in which the electromagnetic wave incident on the action surface as is reflected in the third direction d3 and the second reflection state in the fourth direction d4, respectively, but other methods may also be used.

[0102] For example, such as Figure 9 As shown, the first state can be a transmission state in which electromagnetic waves incident on the action surface as pass through and travel in a third direction d3. More specifically, the switching unit 181 may include a shutter for each switching element having a reflective surface that reflects electromagnetic waves in a fourth direction d4. In the switching unit 181 with this structure, by opening and closing the shutter of each switching element, the transmission state as the first state and the reflection state as the second state can be switched for each switching element.

[0103] Examples of switching units 181 with this structure include MEMS shutters that arrange multiple shutters that can be opened and closed in an array. Alternatively, switching units 181 may include liquid crystal shutters that can switch between a reflection state and a transmission state of electromagnetic waves based on the liquid crystal orientation. In this type of switching unit 181, by switching the liquid crystal orientation of each switching element, it is possible to switch between a transmission state (a first state) and a reflection state (a second state) for each switching element.

[0104] Furthermore, in the electromagnetic wave detection device 10, the optical receiving system 110 may further include a second post-stage optical system and a third detection unit. The second post-stage optical system is positioned in a fourth direction d4 starting from the switching unit 18, and images of the object ob are formed. The third detection unit is positioned on the path of the electromagnetic wave traveling from the switching unit 18 in the fourth direction d4 and then through the second post-stage optical system, and detects the electromagnetic wave traveling in the fourth direction d4.

[0105] Furthermore, in the above-described embodiment, the electromagnetic wave detection device 10 has a structure in which the second detection unit 17 is a passive sensor and the first detection unit 20 is an active sensor. However, the electromagnetic wave detection device 10 is not limited to this structure. For example, in the electromagnetic wave detection device 10, whether both the second detection unit 17 and the first detection unit 20 are active sensors or passive sensors, similar effects to the above-described embodiment can be obtained.

[0106] In this embodiment, the control unit 14 includes an arithmetic unit 145. The electromagnetic wave detection device 10 of this embodiment, with its arithmetic unit 145 measuring the distance to the object ob, functions as a distance measuring device. Here, the electromagnetic wave detection device 10 is not limited to a distance measuring device. For example, the electromagnetic wave detection device 10 could be a device that detects the presence of an obstacle on the road, i.e., the object ob, and issues a warning. In this case, the control unit 14 may have a structure that does not include the arithmetic unit 145. Alternatively, the arithmetic unit 145 may not be included in the control unit 14 and may be provided separately from it.

[0107] While representative examples have been described in the above embodiments, it will be apparent to those skilled in the art that many modifications and substitutions can be made within the spirit and scope of this disclosure. Therefore, this disclosure should not be construed as limited to the above embodiments, and various variations and modifications can be made without departing from the scope of the claims. For example, multiple structural blocks shown in the structural diagrams of the embodiments can be combined into one, or a single structural block can be divided.

[0108] Furthermore, although the solutions disclosed herein have been described as apparatuses, this disclosure may also be implemented in ways that include the above, and may also be implemented as methods, programs, or storage media storing programs that are substantially equivalent to the above. It should be understood that the scope of this invention also includes the above.

[0109] Explanation of reference numerals in the attached figures

[0110] 10 Electromagnetic wave detection device

[0111] 12Irradiation Department

[0112] 13 Deflection section

[0113] 14 Control Department

[0114] 15 entrance section

[0115] 16 Separation Section

[0116] 17 Second Inspection Department

[0117] Switching Units 18 and 181

[0118] 19 First Post-Stage Optical System

[0119] 20 First Testing Department

[0120] 21 Computing Department

[0121] 50 images

[0122] 51. First Part of Information

[0123] 52. Part Two Information

[0124] 53. Part Three Information

[0125] 60 contour images

[0126] 61 First contour information

[0127] 62 Second contour information

[0128] 63 Third contour information

[0129] 110 optical receiving system

[0130] 111 Irradiation System

[0131] 141 Image Information Acquisition Department

[0132] 142 Contour Detection Department

[0133] 143 Irradiation Control Department

[0134] 144 Optical Receiver Control Unit

[0135] 145 Computing Department

[0136] as the working surface

[0137] d1, d2, d3, d4 (first direction, second direction, third direction, fourth direction)

[0138] ob object

Claims

1. An electromagnetic wave detection device, wherein, have: The second detection unit includes a camera element; The image information acquisition unit acquires two-dimensional image information from the second detection unit, including pixels in the space where the object is located. A contour detection unit detects the contour of the object contained in the image information based on the image information; An illumination system that irradiates electromagnetic waves into the space; An illumination control unit controls the illumination system to illuminate the electromagnetic waves in the space in a direction corresponding to the outline of the object; and The first detection unit detects the reflected wave after the electromagnetic wave is reflected by the object.

2. The electromagnetic wave detection device according to claim 1, wherein, The irradiation control unit does not irradiate the electromagnetic waves in a direction that corresponds to a portion that is not part of the outline of the object.

3. The electromagnetic wave detection device according to claim 1 or 2, wherein, When the number of object contours exceeds an upper limit determined based on the time from when the irradiation system irradiates the electromagnetic wave to when the first detection unit detects the reflected wave, the irradiation control unit irradiates the electromagnetic wave in a direction corresponding to the object contour with a higher priority than the upper limit.

4. The electromagnetic wave detection device according to claim 3, wherein, The priority order is determined by the way that the outline of the object becomes higher as it approaches the center of the image of the space.

5. The electromagnetic wave detection device according to claim 1 or 2, wherein, The image information acquisition unit acquires a first part of information and a second part of information. The first part of information is image information of a portion of the space, and the second part of information is image information of a portion of the space. The second part of information is different from the first part of information. After detecting a first contour that serves as the contour of the object based on the first part of the information, the contour detection unit detects a second contour that serves as the contour of the object based on the second part of the information. After irradiating the electromagnetic wave in the direction corresponding to the first contour in the space, the irradiation control unit irradiates the electromagnetic wave in the direction corresponding to the second contour in the space.

6. The electromagnetic wave detection device according to claim 5, wherein, During the period when the irradiation system irradiates the electromagnetic wave in a direction corresponding to the first contour, the image information acquisition unit acquires the second part of the information.

7. The electromagnetic wave detection device according to claim 1 or 2, wherein, have: The separation section separates the electromagnetic wave containing the reflected wave in a manner that allows it to travel in a first direction and a second direction. The second detection unit detects electromagnetic waves traveling along the first direction. The first detection unit detects the reflected wave contained in the electromagnetic wave traveling along the second direction. The image information acquisition unit acquires image information of the space based on the detection information of the second detection unit.

8. The electromagnetic wave detection device according to claim 7, wherein, have: The switching unit has a plurality of switching elements, which are capable of switching an electromagnetic wave traveling in the second direction to a first state that causes the electromagnetic wave to travel in a third direction or a second state that causes the electromagnetic wave to travel in a fourth direction. as well as The light receiving control unit switches the plurality of switching elements to the first state or the second state respectively, based on the irradiation direction or irradiation position of the electromagnetic waves output from the irradiation system. The first detection unit is positioned at a location capable of detecting electromagnetic waves traveling in the direction of the third party.

9. The electromagnetic wave detection device according to claim 1 or 2, wherein, The electromagnetic wave detection device is configured such that the optical axis of the light used to acquire image information about a predetermined position of the object is aligned with the optical axis of the reflected wave detected by the first detection unit from the predetermined position.

10. A ranging device, wherein, have: The second detection unit includes a camera element; The image information acquisition unit acquires two-dimensional image information from the second detection unit, including pixels in the space where the object is located. A contour detection unit detects the contour of the object contained in the image information based on the image information; An illumination system that irradiates electromagnetic waves into the space; An illumination control unit controls the illumination system to irradiate the electromagnetic waves in the space in a direction corresponding to the outline of the object. The first detection unit detects the reflected wave after the electromagnetic wave is reflected by the object; as well as The calculation unit calculates the distance between itself and the object based on the detection information from the first detection unit.

Images