Vehicle front viewing device

Abstract

To provide a vehicle front viewing device that allows another device to determine the usefulness of a sensing result early.SOLUTION: A vehicle front viewing device 10A to be mounted on a vehicle Va includes: a light probe device that delivers a laser light ray group forming a dot pattern in a measurement target region set ahead of the vehicle; a detection camera device that captures return light delivered from the light probe device and reflected on an object existing in the measurement target region and generates a distance image in which a pixel value is a distance value; a first-stage processing unit that performs first-stage processing including a predetermined recognition process on the distance image; a processing code generation unit that performs a processing code generation process of generating a processing code representing a result of the predetermined recognition process; and a communication unit that transmits the distance image and the processing code to another device.SELECTED DRAWING: Figure 19

Description

[Technical field]

[0001] The present disclosure relates to a vehicle forward visibility device. [Background technology]

[0002] 2. Description of the Related Art There is known a vehicle forward visibility device that uses a LiDAR to sense objects such as surrounding vehicles that exist in an area in front of the vehicle (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2023-505 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the vehicle forward visibility device described in Patent Document 1 has a problem in that when sensing results (e.g., distance images) of objects such as surrounding vehicles are transmitted to another device, the mechanism for the other device to quickly determine the usefulness of the sensing results is insufficient.

[0005] The present disclosure has been made to solve such problems, and aims to provide a vehicle forward visibility device that allows another device to quickly determine the usefulness of sensing results. [Means for solving the problem]

[0006] The vehicle forward visibility device according to the present disclosure is a vehicle forward visibility device mounted on a vehicle, The system comprises an optical probe device that irradiates a group of laser beams that form a dot pattern on a measurement target area set in front of the vehicle, a detection camera device that images returned light irradiated from the optical probe device and reflected by an object present in the measurement target area and generates a distance image whose pixel values ​​are distance values, a first-stage processing unit that performs first-stage processing including a predetermined recognition processing on the distance image, a processing code generation unit that performs a processing code generation process that generates a processing code representing a result of the predetermined recognition processing, and a communication unit that transmits the distance image and the processing code to another device.

[0007] With this configuration, it is possible to provide a vehicle forward visibility device that allows another device to quickly determine the usefulness of the sensing results.

[0008] This is achieved by transmitting a processing code indicating the usefulness of the sensing result to another device in addition to the synthetic range image, which is the sensing result.

[0009] In addition, in the above vehicle forward visual recognition device, the optical probe device includes a right optical probe device provided on the right side of the front end of the vehicle, which irradiates a group of laser beams forming a dot pattern in a right measurement target area set at the predetermined reference distance forward of the vehicle, and a left optical probe device provided on the left side of the front end of the vehicle, which irradiates a group of laser beams forming a dot pattern in a left measurement target area set at the predetermined reference distance forward of the vehicle, and the detection camera device captures the return light irradiated from the right optical probe device and reflected by an object present in the right measurement target area, and generates a right distance image as a primary distance image whose pixel value is a distance value, and on the other hand, captures the return light irradiated from the left optical probe device and reflected by an object present in the left measurement target area, and generates a left distance image as a primary distance image whose pixel value is a distance value, The right measurement target area and the left measurement target area are arranged symmetrically and at least partially overlap each other in the horizontal direction in front of the vehicle, and the device further includes a secondary distance image generation unit that generates a secondary distance image to be transmitted to the other device based on the right distance image and the left distance image, and the initial processing unit performs the specified recognition processing on the primary distance image and the secondary distance image, and the communication unit transmits the processing code and the secondary distance image to the other device.

[0010] In addition, the above-mentioned vehicle forward visibility device may further include a memory unit that stores predetermined conditions and processing codes that are generated when the conditions are satisfied, in correspondence with each other, and the processing code generation unit may determine whether the predetermined conditions are satisfied, and if the conditions are satisfied, may generate a processing code corresponding to the condition by referring to the memory unit.

[0011] In the above vehicle forward visibility device, the predetermined condition may be at least one of an out-of-range condition, a transition state condition, and a valid condition. Effect of the Invention

[0012] The present disclosure makes it possible to provide a vehicle forward visibility device that allows another device to quickly determine the usefulness of sensing results. [Brief description of the drawings]

[0013] [Figure 1] 1 is a schematic diagram of a vehicle forward visual confirmation device 10. FIG. [Diagram 2] FIG. 1 is a front view of a vehicle Va to which a vehicle forward visual confirmation device 10 is attached. [Diagram 3] 13 is an example of a right measurement target area A20R and a left measurement target area A20L. [Figure 4] 13 is an example of a right measurement target area A20R and a left measurement target area A20L. [Diagram 5] 1 is an example of a reference distance. [Figure 6] 1 is an example of a reference distance. [Figure 7] FIG. 7A is a schematic diagram of the right optical probe device 20R, and FIG. 7B is a cross-sectional view taken along line BB in FIG. [Figure 8] 3 is a cross-sectional view taken along line AA in FIG. 2. [Figure 9] 9(a) is a schematic diagram of a detection camera device 30, and FIG. 9(b) is a cross-sectional view taken along line CC of FIG. 9(a). [Figure 10] 4 is a flowchart of an example of the operation of the device 10 for visually confirming a forward direction of a vehicle. [Figure 11] 13 is an example of a flowchart of an imaging process. [Figure 12] 13 is an example flowchart of a secondary distance image generation process. [Figure 13] 13 is an example of a flowchart of an object distance calculation process. [Figure 14] FIG. 2 is a diagram showing each element in an equation for calculating a distance Lc to an object. [Figure 15] 13 is an example of a flowchart of a dot density detection process. [Figure 16] 13(a) to 13(c) are diagrams showing how dot density changes periodically. [Figure 17] 13 shows modified examples of the right measurement target area A20R and the left measurement target area A20L. [Figure 18] (a) is a modified example of the light emitting element 21 of the optical probe device 20, and (b) is a modified example of the right measurement target area A20R and the left measurement target area A20L. [Figure 19] FIG. 11 is a schematic configuration diagram of a device 10A for visually confirming a forward direction of a vehicle according to a second embodiment. [Figure 20] 13 is an example of a process code generation process. [Figure 21] 13 is an example of an out-of-range condition and a processing code that is generated when the out-of-range condition is satisfied. [Figure 22] 1 is an example of a transition state condition and a processing code that is generated when the transition state condition is satisfied. [Diagram 23] 13 is an example of a valid condition and a processing code that is generated when the valid condition is satisfied. [Figure 24] 1 is an example of transmission data (packet) including a processing code and a synthetic distance image. [Diagram 25] 4 is a flowchart of an operation example of the device 10A for visually confirming a forward direction of a vehicle. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, a vehicle forward visual confirmation device 10 according to an embodiment of the present invention will be described with reference to the accompanying drawings. In each drawing, corresponding components are given the same reference numerals, and duplicated explanations will be omitted.

[0015] <Embodiment 1> FIG. 1 is a schematic configuration diagram of a device 10 for visually confirming a forward direction of a vehicle according to the first embodiment.

[0016] The vehicle forward visual confirmation device 10 is a forward information acquisition device that provides the driving support device with information about the area in front of the vehicle when the vehicle is traveling mainly on suburban roads at medium to high speeds (for example, 40 to 120 km / h).

[0017] As shown in FIG. 1, the vehicle forward visibility device 10 includes an optical probe device 20 (a right optical probe device 20R and a left optical probe device 20L), a detection camera device 30, a control device 40, and a driving support device 60. The optical probe device 20 and the detection camera device 30 constitute a LiDAR device. The vehicle forward visibility device 10 is mounted on a vehicle such as an automobile. Hereinafter, the vehicle on which the vehicle forward visibility device 10 is mounted is referred to as the host vehicle Va.

[0018] FIG. 2 is a front view of a vehicle Va to which the device 10 for visually confirming a forward direction of a vehicle is attached.

[0019] As shown in Fig. 2, the optical probe device 20 is provided on both the left and right sides of the front end of the vehicle Va. Hereinafter, the optical probe device 20 provided on the right side of the front end of the vehicle Va (the right side when facing the front of the vehicle) will be referred to as the right optical probe device 20R. Also, the optical probe device 20 provided on the left side of the front end of the vehicle Va (the left side when facing the front of the vehicle) will be referred to as the left optical probe device 20L.

[0020] 3 and 4 show the right measurement target area A at a reference distance, which will be described later. 20Rand left measurement area A 20L This is an example.

[0021] The right optical probe device 20R is a right measurement target area A set in front of the host vehicle Va. 20R The left optical probe device 20L is configured to irradiate a left measurement target area A (see FIG. 3) with a pulse oscillation frequency (MHz) according to the measurement distance and with a short pulse width (ns to ps) having a distance resolution of about 1 / 100 or less of the measurement distance. 20L The right optical probe device 20R (see FIG. 3) is configured to irradiate a group of laser beams (group of infrared laser beams) that form a dot pattern on the right optical probe device 20R (right measurement target area A) with a pulse oscillation frequency (MHz) according to the measurement distance and with a short pulse width (ns to ps) that has a distance resolution of about 1 / 100 or less of the measurement distance. For example, if the pulse oscillation frequency is 1 MHz (period: 1 μs), distance measurement of a measurement distance of up to 150 m (round trip 300 m) is possible. Furthermore, if the pulse width is 10 ns, the distance resolution can be 1.5 m (round trip 3 m). Furthermore, if the pulse width is 1 ns, the distance resolution can be improved to 0.15 m. 20R ) and the left optical probe device 20L (left measurement target area A 20L ) are symmetrical, so in the following, the right optical probe device 20R (right measurement target area A 20R ) will be explained.

[0022] Right measurement area A 20R The right measurement area A has a trapezoidal shape whose vertical width increases from the center of the vehicle width direction of the host vehicle Va to the right side of the vehicle width direction. Specifically, as shown in FIG. 20R is a trapezoid bounded by an upper side L1R, a lower side L2R, an inner perpendicular line L3R, and an outer perpendicular line L4R.

[0023] In FIG. 4, the symbol AX 20R The symbol AX represents the optical axis of the right optical probe device 20R. 20L The symbol Wp denotes the optical axis of the left optical probe device 20L. 20R and optical axis AX20L The symbol Cp1 represents the distance between the optical axes AX 20R and optical axis AX 20L and Cp2. The vertical line V1-V1 represents a vertical line passing through the center point Cp1. The horizontal line H1-H1 represents a horizontal line passing through the center points Cp1 and Cp2. The vertical line V2-V2 represents a vertical line passing through the center point Cp2. The symbol P1t represents an upper end point set above the optical probe device height Hp with respect to the center point Cp1. The symbol P1b represents a lower end point set below the optical probe device height Hp with respect to the center point Cp1. The symbol P2t represents an upper end point set above the optical probe device height Hp x 2 with respect to the center point Cp2. The symbol P2b represents a lower end point set below the optical probe device height Hp x 2 with respect to the center point Cp2.

[0024] The upper side L1R is a straight line passing through the upper end point P1t and the upper end point P2t. The lower side L2R is a straight line passing through the lower end point P1b and the lower end point P2b. The inner perpendicular line L3R is a straight line passing through the optical axis AX of the left optical probe device 20L. 20L The outer perpendicular line L4R is a perpendicular line passing through the roadside of the right lane adjacent to the own lane.

[0025] Right measurement area A 20R The dot density of the dot pattern formed in the right measurement area A is set so that the dot density is high at the center in the vehicle width direction and decreases toward the right side in the vehicle width direction. 20R The dot intervals of the dot pattern formed are about 10 cm apart in the vertical and horizontal directions with respect to the center point Cp1, and about 20 cm apart in the vertical and horizontal directions with respect to the center point Cp2.

[0026] On the other hand, left measurement area A 20L is the right measurement area A 20R That is, the left measurement area A 20L The left measurement target area A has a trapezoidal shape whose vertical width increases from the center of the vehicle width direction of the host vehicle Va toward the left side of the vehicle width direction. Specifically, as shown in FIG. 20Lis a trapezoid bounded by an upper side L1L, a lower side L2L, an inner perpendicular line L3L, and an outer perpendicular line L4L.

[0027] Left measurement area A 20L The dot density of the dot pattern formed in the left measurement area A is set so that the dot density is high at the center in the vehicle width direction and decreases toward the left side in the vehicle width direction. 20L The dot intervals of the dot pattern formed are about 10 cm apart in the vertical and horizontal directions from a center point similar to the center point Cp1, and about 20 cm apart in the vertical and horizontal directions from a center point similar to the center point Cp2.

[0028] As shown in Figure 4, the right measurement area A 20R and left measurement area A 20L are arranged symmetrically on the left and right and overlap at least partially with each other in the horizontal direction in front of the host vehicle Va. 20R and left measurement area A 20L covers the vehicle's own lane and the lanes adjacent to the vehicle's own lane on both the left and right sides. 20R and left measurement area A 20L The overlapping area of ​​overlapping area A 20R+20L In this way, the overlapping area A 20R+20L By providing the above, it becomes possible to dimerize the distance information as described later.

[0029] Right measurement area A 20R Dots in the dot pattern formed on the left and measurement target area A 20L The dots of the dot pattern formed in the overlapping area A 20R+20L are superimposed on each other in

[0030] In this way, the overlapping area A 20L+20R Since the dot density is set high, it is easy to form a distance image of a preceding vehicle in the own lane. It is also easy to measure the distance to a preceding vehicle that is farther away than the reference distance.

[0031] 5 and 6 are examples of the reference distance.

[0032] As shown in Figure 5, the right measurement area A 20R and left measurement area A 20L are set as the first reference distance, the second reference distance, and the third reference distance, respectively. In Fig. 5, the first reference distance is 40 m, which corresponds to the distance between vehicles at a vehicle speed of 55 km / h, the second reference distance is 80 m, which corresponds to the distance between vehicles at a vehicle speed of 85 km / h, and the third reference distance is 120 m, which corresponds to the distance between vehicles at a vehicle speed of 110 km / h.

[0033] As described above, the right measurement target area A is set at the first reference distance, the second reference distance, and the third reference distance. 20R and left measurement area A 20L By setting the above, the vehicle ahead of the host vehicle Va can be easily detected.

[0034] Next, a description will be given of a configuration example of the optical probe device 20. Below, a configuration example of the right optical probe device 20R will be described as a representative example.

[0035] FIG. 7(a) is a schematic diagram of the right optical probe device 20R, and FIG. 7(b) is a cross-sectional view taken along line BB of FIG. 7(a).

[0036] 7(a) and 7(b), the right optical probe device 20R includes a light emitting element 21, a correction lens 22, a mounting board 23 on which the light emitting element 21 is attached, and an oscillator circuit 24. These components are housed in a housing 25, and a light exit window 26 is provided in a part of the housing 25 through which the light emitted from the light emitting element 21 and transmitted through the correction lens 22 passes. The right optical probe device 20R is also connected to a control device 40 by a signal line 27.

[0037] The light emitting element 21 includes a semiconductor laser that oscillates at a wavelength (e.g., 905 nm, 940 nm, 1450 nm) that causes a solar blind, and a diffractive optical element (DOE) that converts the emitted light of the semiconductor laser into a predetermined dot light pattern. The semiconductor laser is, for example, a VCSEL (Vertical Cavity Surface Emitting Laser) that includes an emitter that emits a plurality of laser beams (24×64). The diffractive optical element is, for example, a DOE that is configured with a multi-bend type cell array and multiply branches the laser beam emitted from the VCSEL emitter. The semiconductor laser is not limited to a VCSEL, and may be a stripe LD (Laser Diode) or a PCSEL (Photonic Crystal Surface Emitting Laser). In addition to the diffractive optical element, a rotating mirror (MEMS mirror, galvanometer mirror, polygon mirror) may be used to form the dot light pattern.

[0038] The correction lens 22 diffuses (concave lens) and collects (convex lens) the dot light pattern emitted from the light emitting element 21 so that it matches the shape of the irradiation area set for each reference distance.

[0039] The oscillator circuit 24 receives a control signal from the control device 40 and turns on or off the light emitting element 21 in a predetermined pattern (pulse oscillation).

[0040] The light exit window 26 has a filter function of transmitting the light emitted from the light emitting element 21 and preventing the intrusion of external light.

[0041] The right optical probe device 20R includes a plurality of light emitting elements 21 corresponding to a plurality of measurement distances (first reference distance to third reference distance). Specifically, the right optical probe device 20R includes a right measurement target area A 20R A combination of a light emitting element 21 that irradiates a group of laser beams that form a dot pattern on a right measurement target area A set at a second reference distance and a correction lens 22. 20Ra combination of a light emitting element 21 and a correction lens 22 that irradiates a group of laser beams that form a dot pattern on the right measurement target area A set at a third reference distance; 20R The optical element 21 is a combination of a light emitting element 21 that emits a group of laser beams that form a dot pattern on the optical axis, and a correction lens 22.

[0042] The right optical probe device 20R switches the light emitting element 21 among the plurality of light emitting elements 21 to a light emitting element 21 according to the vehicle speed of the host vehicle Va, and detects the right measurement target area A by the switched light emitting element 21. 20R The vehicle speed of the host vehicle Va may be provided by the driving support device 60 or obtained from a vehicle speed sensor (not shown) attached to the host vehicle Va.

[0043] For example, when the vehicle speed of the host vehicle Va is 55 km / h, the right optical probe device 20R switches to the light emitting element 21 corresponding to the first reference distance, and the right measurement target area A set to the first reference distance is detected by the switched light emitting element 21. 20R On the other hand, when the vehicle speed of the host vehicle Va is 85 km / h, the right optical probe device 20R switches to the light emitting element 21 corresponding to the second reference distance, and the right measurement target area A set at the second reference distance is illuminated by the switched light emitting element 21. 20R When the vehicle speed of the vehicle Va is 110 km / h, the right optical probe device 20R switches to the light emitting element 21 corresponding to the third reference distance, and the right measurement target area A set at the third reference distance is illuminated by the switched light emitting element 21. 20R The laser beams that form a dot pattern are irradiated onto the substrate.

[0044] Similarly, the left optical probe device 20L includes a plurality of light emitting elements 21 corresponding to a plurality of measurement distances (first reference distance to third reference distance). Specifically, the left optical probe device 20L includes a left measurement target area A set at the first reference distance. 20LA combination of a light emitting element 21 that irradiates a laser beam group that forms a dot pattern on a left measurement target area A set at a second reference distance and a correction lens 22. 20L a combination of a light emitting element 21 and a correction lens 22 that irradiates a laser beam group that forms a dot pattern on the left measurement target area A set at a third reference distance; 20L The optical element 21 is a combination of a light emitting element 21 that emits a group of laser beams that form a dot pattern on the optical axis, and a correction lens 22.

[0045] The left optical probe device 20L switches to a light-emitting element 21 corresponding to the vehicle speed of the host vehicle Va among the plurality of light-emitting elements 21, and detects the left measurement target area A by the switched light-emitting element 21. 20L The laser beams that form a dot pattern are irradiated onto the substrate.

[0046] For example, when the vehicle speed of the host vehicle Va is 55 km / h, the left optical probe device 20L switches to the light emitting element 21 corresponding to the first reference distance, and the left measurement target area A set to the first reference distance is detected by the switched light emitting element 21. 20L On the other hand, when the vehicle speed of the host vehicle Va is 85 km / h, the left optical probe device 20L switches to the light emitting element 21 corresponding to the second reference distance, and the left measurement target area A set at the second reference distance is illuminated by the switched light emitting element 21. 20L When the vehicle speed of the vehicle Va is 110 km / h, the left optical probe device 20L switches to the light emitting element 21 corresponding to the third reference distance, and the left measurement target area A set at the third reference distance is illuminated by the switched light emitting element 21. 20L The laser beams that form a dot pattern are irradiated onto the substrate.

[0047] The right optical probe device 20R having the above configuration is disposed, for example, in the lamp chamber S1 of the vehicle lamp 50R mounted on the right side of the front end of the vehicle (the right side when facing the front of the vehicle). Similarly, the left optical probe device 20L having the above configuration is disposed, for example, in the lamp chamber S1 of the vehicle lamp 50L mounted on the left side of the front end of the vehicle (the left side when facing the front of the vehicle).

[0048] FIG. 8 is a cross-sectional view taken along line AA in FIG.

[0049] As shown in Fig. 8, a vehicle lamp 50R includes a bracket 52 to which a lamp unit 51 (semiconductor light emitting element 51a, reflecting surface 51b, projection lens 51c, movable shade 51d) is attached, and an optical axis adjustment mechanism 53 (e.g., an aiming screw) that adjusts the optical axis of the lamp unit 51 by tilting the bracket 52. The lamp unit 51 and the optical axis adjustment mechanism 53 are disposed in a lamp chamber S1 formed between an outer lens 54 and a housing 55 attached thereto. In Fig. 8, reference numeral 56 denotes an extension.

[0050] The right optical probe device 20R is disposed in the lamp chamber S1 of the vehicle lamp 50R in a state where it is attached to the bracket 52 to which the lamp unit 51 is attached (see FIG. 8). Similarly, the left optical probe device 20L is disposed in the lamp chamber S1 of the vehicle lamp 50L in a state where it is attached to the bracket 52 to which the lamp unit 51 is attached. In this way, by attaching the right optical probe device 20R (and the left optical probe device 20L) to the bracket 52 tilted by the optical axis adjustment mechanism 53, it is possible to adjust the deviation of the laser beam group irradiated from the right optical probe device 20R (and the left optical probe device 20L) together with the optical axis adjustment of the lamp unit 51 by the optical axis adjustment mechanism 53. Note that the right optical probe device 20R (and the left optical probe device 20L) does not necessarily need to be disposed in the lamp chamber S1 of the vehicle lamp 50, and may be disposed in an appropriate location such as the bumper, grill, side mirror, or front window of the vehicle Va. In this case, by electrically connecting it to the optical axis adjustment mechanism 53 of the lamp unit 51, the deviation of the laser beam group can be adjusted in the same way as when it is attached to the bracket 52.

[0051] Next, the detection camera device 30 will be described.

[0052] The detection camera device 30 is configured to receive return light irradiated from the optical probe device 20 and reflected by an object present in the measurement target area, and generate a primary distance image in which pixel values ​​are distance values. The detection camera device 30 is provided in the center of the vehicle width direction inside the vehicle cabin of the host vehicle Va, as shown in FIG. 2. The detection camera device 30 captures an image of the area ahead of the host vehicle Va through the front windshield (wiper sweep area). This captures an overlapping area A ahead of the vehicle center line. 20R+20L Since the sensor can be positioned in the center of the field of view, the accuracy of measuring the distance to the vehicle ahead can be improved.

[0053] FIG. 9(a) is a schematic diagram of the detection camera device 30, and FIG. 9(b) is a cross-sectional view taken along line CC of FIG. 9(a).

[0054] 9(a) and 9(b), the detection camera device 30 includes an image sensor 31, a light receiving lens 32, a mounting board 33 on which the image sensor 31 is mounted, and a receiving circuit 34. These components are housed in a housing 35, and a light receiving window 36 is provided in part of the housing 35, through which light emitted from the optical probe device 20 and reflected from the object to be detected passes. The detection camera device 30 is also connected to a control device 40 by a signal line 37.

[0055] The imaging element 31 is a single photo counting type imaging element (SPAD: Single Photon Avalanche Diode) capable of measuring the time of flight (direct ToF) of light emitted from the optical probe device 20. The imaging element 31 has a number of pixels (10 (597×168) megapixels to 3 megapixels) capable of resolving a distant detection target. A charge distribution type imaging element that detects a phase shift of the optical probe device 20 (indirect ToF) to obtain the distance to the detection target may be used.

[0056] The light receiving lens 32 is a right measurement target area A 20R and left measurement area A 20L The lens adjusts the angle of view so that the object to be detected is of a size that can be resolved on the light receiving surface of the image sensor 31. 20R and left measurement area A20L is set to 0.3 to 0.7 times on the imaging element 31.

[0057] The receiving circuit 34 receives a control signal from the control device 40, executes imaging, and outputs the captured image (distance image).

[0058] The light receiving window 36 functions as a bandpass filter that selectively transmits only the light emitted from the optical probe device 20 .

[0059] The detection camera device 30 includes a plurality of image pickup elements 31 corresponding to a plurality of measurement distances (first reference distance to third reference distance). Specifically, the detection camera device 30 includes a combination of a light receiving lens 32 and an image pickup element 31 corresponding to the first reference distance, a combination of a light receiving lens 32 and an image pickup element 31 corresponding to the second reference distance, and a combination of a light receiving lens 32 and an image pickup element 31 corresponding to the third reference distance.

[0060] The detection camera device 30 switches the imaging element 31 among the multiple imaging elements 31 according to the vehicle speed of the host vehicle Va, and captures the right measurement target area A set at the first reference distance by the switched imaging element 31. 20R Or left measurement area A 20L The camera captures the returning light reflected from an object present in the vicinity of the target object, and generates a right distance image or a left distance image as a primary distance image in which pixel values ​​are distance values.

[0061] For example, when the vehicle speed of the host vehicle Va is 55 km / h, the detection camera device 30 switches to the imaging element 31 corresponding to the first reference distance, and the imaging element 31 captures the right measurement target area A set to the first reference distance. 20R Or left measurement area A 20L The detection camera device 30 captures the reflected light from the object present in the right measurement target area A, and generates a right distance image or a left distance image as a primary distance image in which pixel values ​​are distance values. On the other hand, when the vehicle speed of the host vehicle Va is 85 km / h, the detection camera device 30 switches to the image sensor 31 corresponding to the second reference distance, and the switched image sensor 31 captures the image of the right measurement target area A, which is set to the second reference distance. 20R Or left measurement area A20L The detection camera device 30 captures the reflected light from the object present in the right measurement target area A, and generates a right distance image or a left distance image as a primary distance image in which pixel values ​​are distance values. When the vehicle speed of the host vehicle Va is 110 km / h, the detection camera device 30 switches to the image sensor 31 corresponding to the third reference distance, and the switched image sensor 31 captures the reflected light from the object present in the right measurement target area A, which is set to the third reference distance. 20R Or left measurement area A 20L The camera captures the returning light reflected from an object present in the vicinity of the target object, and generates a right distance image or a left distance image as a primary distance image in which pixel values ​​are distance values.

[0062] As described above, by preparing an optical probe device 20 (image sensor 31) and a detection camera device 30 for each reference distance, it is possible to prevent a decrease in distant resolution when the nearby distance is used as the reference distance, and also to prevent a narrowing of the nearby field of view when the distant distance is used as the reference distance, thereby enabling accurate road information to be obtained at all times.

[0063] Next, the control device 40 (control ECU) will be described.

[0064] The control device 40 (control ECU: Electronic Control Unit) includes, for example, a processor (not shown), a storage unit 41, and a memory 42.

[0065] The processor is, for example, a CPU (Central Processing Unit). There may be one processor or multiple processors. The processor functions as an imaging unit 43, an object distance calculation unit 44, a dot density detection unit 45, a secondary distance image generation unit 46, and a communication unit 47 by executing a predetermined program (not shown) loaded from a storage unit 41 (e.g., a ROM) to a memory 42 (e.g., a RAM). Some or all of these may be realized by hardware.

[0066] The imaging unit 43 controls the right optical probe device 20R, the left optical probe device 20L, and the detection camera device 30 to capture a right distance image and a left distance image. For example, the imaging unit 43, in response to a command from the upper driving support device 60, turns on the right optical probe device 20R and the left optical probe device 20L at a predetermined light emission timing and causes the detection camera device 30 to capture an image at a predetermined timing.

[0067] The object distance calculation unit 44 calculates the right measurement object area A 20R and left measurement area A 20L Overlap area A 20R+20L Calculate the distance to an object located in.

[0068] The dot density detection unit 45 detects the dot density in the right measurement target area A. 20R and left measurement area A 20L Overlap area A 20R+20L Detect the dot density within the pixel.

[0069] The secondary distance image generation unit 46 generates a secondary distance image to be transmitted to the higher-level driving assistance device 60 based on the right distance image and left distance image, which are the primary distance images transmitted from the detection camera device 30. The secondary distance image generated by the secondary distance image generation unit 46 is, for example, a composite distance image obtained by combining the right distance image and the left distance image.

[0070] The communication unit 47 transmits the secondary distance image generated by the secondary distance image generation unit 46 to the upper-level driving assistance device 60 as response information to the command.

[0071] Next, an example of the operation of the device 10 for visually confirming a forward direction of a vehicle will be described.

[0072] FIG. 10 is a flowchart of an example of the operation of the device 10 for visually confirming a forward direction of a vehicle.

[0073] First, an imaging process is executed (step S10).

[0074] 11 is a flowchart showing an example of the imaging process. The following process is mainly executed by the imaging unit 43.

[0075] First, the right optical probe device 20R switches the light emitting element 21 among the plurality of light emitting elements 21 to a light emitting element 21 corresponding to the vehicle speed of the host vehicle Va (step S101). 20R For example, when the vehicle speed of the vehicle Va is 55 km / h, the right optical probe device 20R switches to the light-emitting element 21 corresponding to the first reference distance, and emits a laser beam group that forms a dot pattern in the right measurement target area A set at the first reference distance by the switched light-emitting element 21. 20R On the other hand, when the vehicle speed of the host vehicle Va is 85 km / h, the right optical probe device 20R switches to the light emitting element 21 corresponding to the second reference distance, and the right measurement target area A set at the second reference distance is illuminated by the switched light emitting element 21. 20R When the vehicle speed of the vehicle Va is 110 km / h, the right optical probe device 20R switches to the light emitting element 21 corresponding to the third reference distance, and the right measurement target area A set at the third reference distance is illuminated by the switched light emitting element 21. 20R The laser beams that form a dot pattern are irradiated onto the substrate.

[0076] Next, the detection camera device 30 switches to an image pickup element 31 corresponding to the vehicle speed of the host vehicle Va among the multiple image pickup elements 31 (step S103), and captures the right measurement target area A 20R The controller 40 then captures the reflected light from an object in the vicinity of the center of the right-hand side (step S104) and generates a right distance image as a primary distance image in which pixel values ​​are distance values ​​(step S105). The captured (generated) right distance image is transmitted to the controller 40.

[0077] Next, the left optical probe device 20L switches to a light-emitting element 21 corresponding to the vehicle speed of the host vehicle Va among the plurality of light-emitting elements 21 (step S106), and detects the left measurement target area A 20LFor example, when the vehicle speed of the host vehicle Va is 55 km / h, the left optical probe device 20L switches to the light-emitting element 21 corresponding to the first reference distance, and the left measurement target area A set at the first reference distance is irradiated with the light-emitting element 21. 20L On the other hand, when the vehicle speed of the host vehicle Va is 85 km / h, the left optical probe device 20L switches to the light emitting element 21 corresponding to the second reference distance, and the left measurement target area A set at the second reference distance is illuminated by the switched light emitting element 21. 20L When the vehicle speed of the vehicle Va is 110 km / h, the left optical probe device 20L switches to the light emitting element 21 corresponding to the third reference distance, and the left measurement target area A set at the third reference distance is illuminated by the switched light emitting element 21. 20L The laser beams that form a dot pattern are irradiated onto the substrate.

[0078] Next, the detection camera device 30 switches to an image pickup element 31 corresponding to the vehicle speed of the host vehicle Va among the multiple image pickup elements 31 (step S108), and captures the left measurement target area A 20L The control device 40 then captures the reflected light from an object present in the right direction (step S109) and generates a right distance image as a primary distance image in which pixel values ​​are distance values ​​(step S110). The captured (generated) right distance image is then transmitted to the control device 40.

[0079] Here, the overlapping area A 20L+20R If an object (e.g., a preceding vehicle) is present in the right direction, two distances to the object are obtained: a distance based on the right distance image and a distance based on the left distance image. In other words, the distance information can be dimerized.

[0080] Next, returning to FIG. 10, a secondary distance image generating process is executed (step S20).

[0081] 12 is an example of a flowchart of a secondary distance image generation process. The following process is mainly executed by secondary distance image generation unit 46.

[0082] First, the right distance image transmitted in step S105 and the left distance image transmitted in step S110 are acquired (step S201).

[0083] Next, a composite distance image (an example of a secondary distance image of the present disclosure) is generated by combining the right distance image and the left distance image acquired in step S201 (step S202).

[0084] Next, the synthetic distance image generated in step S202 is transmitted to the host driving support device 60 (step S203).

[0085] Next, returning to FIG. 10, the object distance calculation process is executed (step S30).

[0086] 13 is a flowchart showing an example of the object distance calculation process. The following process is mainly executed by the object distance calculation unit 44.

[0087] First, right measurement area A 20R The inner perpendicular line L3R (see Figure 4) and the left measurement area A 20L The inner perpendicular lines L3L (see FIG. 4) of the right measurement target area A are detected (step S301). 20R The inner perpendicular line L3R of the left measurement target area A can be detected based on the right distance image transmitted in step S105 (or the composite distance image generated in step S202), for example. 20L The inner perpendicular line L3L can be detected, for example, based on the left distance image transmitted in step S110 (or the synthetic distance image generated in step S202).

[0088] Next, the interval Wf between the inner perpendicular lines detected in step S301 (hereinafter also referred to as the inner boundary interval Wf) is calculated (step S302).

[0089] Next, the distance to the object Lc=(Wf / 2) / tan(θi) is calculated using the interval Wf between the inner perpendicular lines calculated in step S302 and the known angle θi (step S22). Each element in this equation can be illustrated as shown in FIG. 14. By calculating the inner boundary interval Wf in this manner, it becomes possible to tri-quantify the distance information. FIG. 14 is a diagram showing each element in the equation for calculating the distance Lc to the object.

[0090] In addition, when an object (e.g., a preceding vehicle) is present farther than the reference distance, the inner boundary distance Wf is set to the optical axis AX 20R and optical axis AX 20L On the other hand, when an object (e.g., a preceding vehicle) is present at the reference distance, the inner boundary distance Wf is greater than the optical axis distance Wp between the optical axes AX 20R and optical axis AX 20L When an object (for example, a preceding vehicle) is present closer than the reference distance, the inner boundary distance Wf is equal to the optical axis distance Wp between the optical axes AX 20R and optical axis AX 20L The distance Wp between the optical axes is smaller than the distance Wp between the optical axes.

[0091] Next, returning to FIG. 10, the dot density detection process is executed (step S40).

[0092] Fig. 15 is an example of a flowchart of the dot density detection process. Figs. 16(a) to 16(c) are diagrams showing how the dot density changes periodically. The following process is mainly executed by the dot density detection unit 45.

[0093] First, overlapping area A 20R+20L (See FIG. 4) to detect the dot density (Step S401). 20R+20L The dot density in can be detected, for example, based on the synthetic distance image generated in step S202.

[0094] If an object (e.g., a preceding vehicle) is present at the reference distance, the right measurement target area A 20R Dots in and left measurement area A 20LSince the dots in overlap each other, overlapping area A 20R+20L The dot density in the area decreases (see FIG. 16(a)).

[0095] On the other hand, when the object (e.g., a preceding vehicle) approaches closer than the reference distance, the dot density increases temporarily (see FIG. 16(b)), and when it approaches further, the dot density decreases (see FIG. 16(c)). In this way, the dot density oscillates. This phenomenon also occurs when the object (e.g., a preceding vehicle) moves away from the reference distance.

[0096] Next, the change period of the dot density is calculated (detected) (step S402) to estimate the approach speed (or separation speed) to the object (step S403). This can be calculated, for example, based on the results of executing the process of step S401 multiple times. 20R and left measurement area A 20L Overlap area A 20R+20L The dot density vibrates periodically as the preceding vehicle approaches, making it possible to quaternize distance information.

[0097] As described above, according to this embodiment, a large amount of distance information can be obtained, and distance information with high reliability (preciseness) can be provided to a higher-level device (driving support device 60). In the above description, for the sake of simplicity, the first to third reference distances corresponding to the vehicle speed of the host vehicle Va are used as the basis, but when the vehicle speed of the host vehicle Va is other than the above-mentioned vehicle speeds, the same function can be achieved by selecting a reference distance corresponding to a close vehicle speed.

[0098] As described above, according to the first embodiment, it is possible to provide the device 10 for visually confirming a forward direction of a vehicle, which is capable of preventing erroneous detection of an object such as a nearby vehicle.

[0099] This is the right measurement area A 20R and left measurement area A 20Lare arranged symmetrically on the left and right and overlap at least partially in front of the vehicle Va in the horizontal direction, so that if an object (e.g., a preceding vehicle) is present in this overlapping area, two distances to the object are obtained: a distance based on the right distance image and a distance based on the left distance image.

[0100] Next, a modified example will be described.

[0101] In the above embodiment 1, the right measurement target area A 20R A trapezoidal area whose vertical width increases from the center of the vehicle width to the right side of the vehicle width is used as the left measurement target area A 20L In the above embodiment, a trapezoidal region whose width in the vertical direction increases from the center in the vehicle width direction of the host vehicle Va toward the left side in the vehicle width direction is used as the trapezoidal region. However, the present invention is not limited to this.

[0102] For example, as shown in FIG. 17, the right measurement target area A 20R A rectangular area extending from the center side of the vehicle width direction of the host vehicle Va to the right side in the vehicle width direction is used as the left measurement target area A 20L Alternatively, a rectangular area extending from the center side in the vehicle width direction of the host vehicle Va to the left side in the vehicle width direction may be used as the right measurement target area A. 20R and left measurement area A 20L This is a modified example of the above.

[0103] As shown in FIG. 18(a), the light emitting element 21 (VCSEL / LD) of the optical probe device 20 is divided into a plurality of parts, and the right measurement target area A 20R (or left measurement area A 20L When using light-emitting units 1A to 1E corresponding to the respective regions 1A to 1E (see FIG. 18(b)), some of the light-emitting units 1A to 1E may be turned on and off and their outputs may be increased or decreased. FIG. 18(a) shows a modified example of the light-emitting element 21 of the optical probe device 20, and FIG. 18(b) shows a modified example of the light-emitting element 21 of the right measurement target region A. 20R and left measurement area A 20L This is a modified example of the above.

[0104] For example, when it is desired to obtain information on a preceding vehicle in the own lane with high accuracy, the light output of the light emitting unit 1A can be increased to increase the reflection intensity of the probe light from the preceding vehicle. Also, when information on adjacent vehicles (adjacent lanes) is not required, the light emitting units (e.g., 1C to 1E) corresponding to the adjacent vehicles (adjacent lanes) may be turned off.

[0105] <Embodiment 2> Next, a second embodiment will be described.

[0106] FIG. 19 is a schematic configuration diagram of an apparatus 10A for visually confirming a forward direction of a vehicle according to the second embodiment.

[0107] 19, the device 10A for visually confirming a forward view of a vehicle is different from the device 10 for visually confirming a forward view of a vehicle in the first embodiment in that it further includes a first-stage processing unit 48, a processing code generating unit 49, and a transmission data generating unit 49A. Other than that, the device 10A has the same configuration as the device 10 for visually confirming a forward view of a vehicle in the first embodiment. The following description will focus on the differences from the device 10 for visually confirming a forward view of a vehicle in the first embodiment, and the same components as those in the device 10 for visually confirming a forward view of a vehicle in the first embodiment will be denoted by the same reference numerals and will not be described as appropriate.

[0108] The initial stage processing unit 48, the processing code generation unit 49, and the transmission data generation unit 49A are realized by a processor (not shown) executing a predetermined program (not shown) that is loaded from the storage unit 41 (e.g., ROM) to the memory 42 (e.g., RAM).

[0109] The first-stage processing unit 48 performs first-stage processing including a predetermined recognition process on the distance image. The distance images on which the first-stage processing unit 48 performs first-stage processing are, for example, a right distance image and a left distance image (an example of a primary distance image in the present disclosure). However, the distance image on which the first-stage processing unit 48 performs first-stage processing may be a composite distance image generated by the secondary distance image generation unit 46 (an example of a secondary distance image in the present disclosure).

[0110] The first-stage processing executed by the first-stage processing unit 48 includes, for example, noise light removal, pattern recognition, and processing code generation.20R The inner perpendicular line L3R (see Figure 4) and the left measurement area A 20L 13, and may include a process of detecting each of the inner perpendicular lines L3L (see FIG. 4) (step S301 in FIG. 13) and a dot density detection process (step S40 in FIG. 15).

[0111] Noise light removal is a process for removing noise light from the right and left distance images. Noise light exists during the day (strong external light) or at night (flash of light). Therefore, noise removal is performed from the measurement distance image (primary distance image) generated by the detection camera device. For example, in ToF measurement, natural light noise is random over time, and the reflected light of the object is constant over time. Therefore, by superimposing the distance measurement data, the noise component can be smoothed (background) and the target component can be amplified. This makes it possible to separate and extract the background and the target component. This noise light removal is achieved by performing a specified image processing (noise light removal processing) on ​​the right and left distance images. The specified image processing (noise light removal processing) may be a known noise light removal processing.

[0112] Pattern recognition is an example of a predetermined recognition process of the present disclosure, which is a process for recognizing specific patterns (e.g., lane markings, lane drawings, preceding vehicles, inner perpendicular lines, dot density) in the right distance image and the left distance image. Recognition of lane markings, lane drawings, and preceding vehicles is achieved by applying predetermined image processing (image recognition processing) to the right distance image and the left distance image. The predetermined image processing (image recognition processing) may be a known image recognition process. Meanwhile, recognition of inner perpendicular lines (detection of the interval Wf between inner perpendicular lines) is achieved by steps S301 and S302 in FIG. 13.

[0113] The lane markings extend forward from the left and right sides of the host vehicle in parallel to the vehicle's travel direction line. Therefore, recognition of lane markings is evaluated by narrowing down to the range where lane markings should be. Recognition of the preceding vehicle is evaluated by narrowing down to the area ahead of the host vehicle's lane. Furthermore, when the host vehicle Va changes lanes, evaluation is evaluated by narrowing down to the area ahead of the virtual host vehicle's lane, which is sandwiched between the virtual lane markings. Note that the virtual lane markings are lane markings that can be drawn on the left and right sides of the host vehicle Va, with the host vehicle Va as the reference point.

[0114] The processing code generation is realized, for example, by a processing code generation unit 49. The processing code generation unit 49 executes a processing code generation process for generating a processing code representing a result of the pattern recognition. The processing code has a smaller data amount than a synthetic distance image described later.

[0115] FIG. 20 shows an example of the process code generation process.

[0116] As shown in FIG. 20, the processing code generation unit 49 determines whether or not predetermined conditions (out-of-range condition, transition state condition, valid condition) are satisfied (steps S51, S52, S53), and if the conditions are satisfied (step S51: YES, step S52: YES, step S53: YES), generates a processing code corresponding to the condition (step S53).

[0117] First, the out-of-range conditions will be described.

[0118] FIG. 21 shows an example of an out-of-range condition and a processing code generated when the out-of-range condition is satisfied. The correspondence between the out-of-range condition and the processing code is stored in the storage unit 41. In FIG. 21, the out-of-range conditions are indicated by "x", "▲", and "-" in the "(1) lane marking" column and the "(2) preceding vehicle" column, and "ND" in the "(3) boundary line interval (Wf)" column. When the out-of-range condition is satisfied, a processing code (e.g., "O1", "O2") is generated, which indicates that lane markings, etc. cannot be recognized. This processing code is a status code generated when the vehicle forward visual confirmation device 10A cannot recognize lane markings.

[0119] Here, "virtual vehicle lane," "within reference distance," and "closest vehicle" in the "(2) Preceding vehicle" column indicate the range in which the presence or absence of a preceding vehicle is judged. Here, "virtual vehicle lane" refers to the virtual vehicle lane sandwiched between the virtual dividing lines mentioned above. "Within reference distance" refers to within the "reference distance" of the optical probe device (light-emitting element 21) selected according to the vehicle speed of the vehicle Va. "Closest vehicle" refers to the closest vehicle within the virtual vehicle lane. In other words, if a preceding vehicle is present within the reference distance within the virtual vehicle lane, the presence or absence of a preceding vehicle is judged based on the closest vehicle. Note that if a lane marking can be recognized, the virtual lane marking is replaced.

[0120] The reasons why "within the reference distance" is used as the target for judging the presence or absence of a preceding vehicle are as follows. The first reason is that the reference distance of the probe device is set to the safe inter-vehicle distance + α corresponding to the vehicle speed. In other words, a preceding vehicle within the safe inter-vehicle distance is a preceding vehicle that requires caution. The second reason is that the measurement distance of dToF is finite. For example, when the frequency of the probe light emitted from the probe device is 1 MHz, the maximum measurement distance is 150 m (round trip distance 300 m). In addition, when a preceding vehicle is traveling vertically with a shift to the left or right within the own lane, different preceding vehicles may be detected in the irradiation range of the right probe device (for example, a distant vehicle traveling on the right side) and the irradiation range of the left probe device (for example, a nearby vehicle traveling on the left side). In that case, the closest vehicle is a preceding vehicle that requires caution.

[0121] The out-of-range conditions will be explained in more detail below.

[0122] An "x" in the "(1) lane markings" column in FIG. 21 indicates that the recognition result of the pattern recognition by the initial processing unit 48 is "not recognized," i.e., as a result of pattern recognition by the initial processing unit 48, a lane marking or a line drawing within the lane could not be recognized. On the other hand, an "▲" in the "(1) lane markings" column indicates that the recognition result of the pattern recognition by the initial processing unit 48 is "line shadow," i.e., as a result of pattern recognition by the initial processing unit 48, a lane marking or a line shadow drawing within the lane was present and was recognized (distance measured). Also, a "-" in the "(1) lane markings" column in FIG. 21 indicates that the recognition result of the pattern recognition by the initial processing unit 48 is "not evaluated."

[0123] In addition, the "x", "▲", and "-" written to the right of "Right side lane marking:", "Drawing in lane:", and "Left side lane marking:" in the "traffic lane marking" column in FIG. 21 represent one state in a vertical row. That is, three states are shown in the "traffic lane marking" column. For example, the first state is "Right side lane marking: x", "Drawing in lane: -", and "Left side lane marking: x". Similarly, the second state is "Right side lane marking: ▲", "Drawing in lane: -", and "Left side lane marking: x", and the third state is "Right side lane marking: x", "Drawing in lane: -", and "Left side lane marking: ▲". That is, if the result of pattern recognition by the initial processing unit 48 corresponds to any of the three states, this condition is satisfied.

[0124] Also, the "X" in the "(2) Preceding vehicle" column in FIG. 21 indicates that the recognition result of the pattern recognition by the initial processing unit 48 is "not recognized", that is, the preceding vehicle cannot be recognized as a result of the pattern recognition by the initial processing unit 48. Specifically, this is the case when the preceding vehicle is traveling farther away than the measurement reference distance (farther away than the safe inter-vehicle distance), or when there is no preceding vehicle far away in the own lane. On the other hand, the "△ (single quantification)" in the "(2) Preceding vehicle" column indicates that the recognition result of the pattern recognition by the initial processing unit 48 is "one-sided distance measurement", that is, the preceding vehicle was recognized (distance measurement) by one of the right distance image and the left distance image, and the preceding vehicle could not be recognized (distance measurement) by the other (single quantification).

[0125] Moreover, "ND" written in the "(3) Border line interval (Wf)" column in FIG. 21 indicates that as a result of pattern recognition by the pre-processing unit 48, the border line interval (Wf) cannot be recognized (detected).

[0126] In addition, in Figs. 21 to 23, "traffic lane markings" refer to white lines that divide the lane in which the host vehicle Va is traveling (the host vehicle lane). "Right-side lane markings" refer to the traffic lane markings on the right side of the host vehicle Va. "Left-side lane markings" refer to traffic lane markings on the left side of the host vehicle Va. "Intra-lane markings" refer to marks on the road surface between the right-side lane markings and the left-side lane markings (for example, road markings at merging or branching roads). "Preceding vehicle" refers to a vehicle ahead of the host vehicle Va that is in the host vehicle lane and within a reference distance. "Boundary line interval (Wf)" refers to the interval Wf between the inner perpendicular lines (see Fig. 4).

[0127] The processing codes ("O1", "O2") shown in the "Processing Code" column in FIG. 21 are examples of processing codes that are generated when the out-of-range condition is met (step S51: YES in FIG. 20). The cause of generating this processing code is as shown in the "Cause / Status" column in FIG. 21. The above describes the processing codes that are generated when the out-of-range condition and the out-of-range condition are met, but the processing codes that are generated when the out-of-range condition and the out-of-range condition are met are not limited to those shown in FIG. 21.

[0128] Next, the conditions for the transition state will be described.

[0129] FIG. 22 shows an example of a condition for a transition state and a processing code generated when the condition for the transition state is satisfied. The correspondence between the condition for the transition state and the processing code is stored in the storage unit 41. In FIG. 22, the "(1) lane dividing line" column, "x", "▲", and "-" in the "(2) preceding vehicle" column, and "ND" in the "(3) boundary line interval (Wf)" column represent the condition for the transition state. When the condition for the transition state is satisfied, a processing code (for example, "T1", "T2", "T3", and "T4") indicating that the host vehicle Va is in a transition state is generated. This processing code is a status code generated when the vehicle forward visual confirmation device 10A (1) recognizes a road marking or the like in the lane dividing line, (2) when a vehicle in another lane enters the host vehicle lane area by merging or changing lanes, (3) when the host vehicle Va crosses the lane dividing line while changing lanes, and (4) when a vehicle appears ahead of the host vehicle Va in the virtual lane dividing line (virtual host vehicle lane area) while the host vehicle Va is changing lanes.

[0130] The conditions for the transition state will be explained in more detail below.

[0131] The "▲" in the "(1) Traffic lane markings" column in Fig. 22 indicates that the recognition result of the pattern recognition by the initial processing unit 48 is a "line shadow", that is, that the pattern recognition by the initial processing unit 48 resulted in the presence of a traffic lane marking or a line shadow drawn within the lane, and recognition (distance measurement) was possible. On the other hand, the "-" in the "(1) Traffic lane markings" column in Fig. 21 indicates that the recognition result of the pattern recognition by the initial processing unit 48 is "not evaluated".

[0132] Also, the "X" in the "(2) Preceding vehicle" column in FIG. 22 indicates that the recognition result of the pattern recognition by the initial processing unit 48 is "not recognized," i.e., the preceding vehicle cannot be recognized as a result of the pattern recognition by the initial processing unit 48. Specifically, this is the case when the preceding vehicle is traveling farther away than the measurement reference distance (farther away than the safe inter-vehicle distance), or when there is no preceding vehicle far away in the own lane. On the other hand, the "△ (single quantification)" in the "(2) Preceding vehicle" column indicates that the recognition result of the pattern recognition by the initial processing unit 48 is "one-sided distance measurement," i.e., the preceding vehicle was recognized (distance measurement) by one of the right distance image and the left distance image, but was not recognized (distance measurement) by the other.

[0133] Moreover, "ND" written in the "(3) Border line interval (Wf)" column in FIG. 22 indicates that as a result of pattern recognition by the pre-processing unit 48, the border line interval (Wf) cannot be recognized (detected).

[0134] The processing codes ("T1", "T2", "T3", "T4") shown in the "Processing Code" column in FIG. 22 are examples of processing codes that are generated when the above transition state conditions are met (step S52: YES in FIG. 20). The cause of generating this processing code is as shown in the "Cause / Status" column in FIG. 22. Although the transition state conditions and the processing codes that are generated when the transition state conditions are met have been described above, the transition state conditions and the processing codes that are generated when the transition state conditions are met are not limited to those shown in FIG. 22.

[0135] Next, the valid conditions will be described.

[0136] FIG. 23 shows an example of a valid condition and a processing code generated when the valid condition is satisfied. Here, valid means that the vehicle Va can recognize the actual lane markings drawn on the road surface and is traveling in the vehicle's lane. In other words, it means that the vehicle Va is in a state in which it can recognize the presence or absence of a preceding vehicle traveling far away from the vehicle's lane. The correspondence between the valid condition and the processing code is stored in the storage unit 41. In FIG. 23, "x", "▲", and "-" in the "(1) lane markings" column and "(2) preceding vehicle" column, and "ND", "invalid", and "valid value (tri-quantified)" in the "(3) boundary line interval (Wf)" column indicate a practically reliable state. When the valid condition is satisfied, a processing code (for example, "V1", "V2", "V3", and "V4") is generated, which indicates that the distance measurement value of the preceding vehicle is valid.

[0137] In FIG. 23, "zero quantification" means that the vehicle forward visibility device 10A cannot recognize the preceding vehicle. "Single quantification" means that the vehicle forward visibility device 10A recognizes the shadow of the preceding vehicle and can measure the distance with a probe light on one side, that is, the vehicle forward visibility device 10A recognizes (measures the distance to) the preceding vehicle based on the right distance image or the left distance image. "Dual quantification" means that the vehicle forward visibility device 10A recognizes the shadow of the preceding vehicle and can measure the distance with probe light on both sides. That is, the vehicle forward visibility device 10A recognizes (measures the distance to) the preceding vehicle based on the right distance image and can recognize (measures the distance to) the preceding vehicle based on the left distance image. "Tri-quantification" means that the vehicle forward visibility device 10A recognizes the shadow of the preceding vehicle and can measure the distance with probe light on both sides, and the boundary line interval (Wf) is a reasonable approximation to the measured distance value (measured distance value by single quantification or double quantification).

[0138] The effective conditions will be explained in more detail below.

[0139] 23, an "x" in the "(1) Traffic lane markings" column indicates that the recognition result of the pattern recognition by the initial stage processing unit 48 is "not recognized," i.e., as a result of pattern recognition by the initial stage processing unit 48, a traffic lane marking or a line drawing in the lane could not be recognized. On the other hand, an "▲" in the "(1) Traffic lane markings" column indicates that the recognition result of the pattern recognition by the initial stage processing unit 48 is "line shadow," i.e., as a result of pattern recognition by the initial stage processing unit 48, a traffic lane marking or a line shadow drawing in the lane was present and was recognized (distance measurement).

[0140] 23 indicates that the recognition result of the pattern recognition by the initial processing unit 48 is "not recognized", that is, the initial processing unit 48 is unable to recognize the preceding vehicle as a result of the pattern recognition. Meanwhile, "△ (single quantification)" in the "(2) Preceding vehicle" column indicates that the recognition result of the pattern recognition by the initial processing unit 48 is "one-sided distance measurement", that is, the preceding vehicle was recognized (distance measurement) by one of the right distance image and the left distance image, and was not recognized (distance measurement) by the other (single quantification). Meanwhile, "○ (two quantifications)" in the "(1) Preceding vehicle" column indicates that the recognition result of the pattern recognition by the initial processing unit 48 is "double distance measurement", that is, the distance measurement value by the right distance image (distance measurement value to the preceding vehicle) and the distance measurement value by the left distance image (distance measurement value to the preceding vehicle) match (two quantifications).

[0141] 23 indicates that the boundary line spacing (Wf) cannot be recognized (detected) as a result of pattern recognition by the initial processing unit 48. On the other hand, "invalid" in the "(3) boundary line spacing (Wf)" column indicates that the boundary line spacing (Wf) is not a valid approximation to the measured distance value. Furthermore, "valid value" in the "(3) boundary line spacing (Wf)" column indicates that the boundary line spacing (Wf) is a valid approximation to the measured distance value.

[0142] The processing codes ("V1", "V2", "V3", "V4") shown in the "Processing Code" column in FIG. 23 are examples of processing codes that are generated when the valid conditions are met (step S53: YES in FIG. 20). The factors that cause this processing code to be generated are as shown in the "Cause / Status" column in FIG. 23. Although the valid conditions and processing codes that are generated when the valid conditions are met have been described above, the valid conditions and processing codes that are generated when the valid conditions are met are not limited to those shown in FIG. 23.

[0143] As described above, the processing code generating unit 49 executes a processing code generating process for generating a processing code representing the result of pattern recognition. Specifically, the processing code generating unit 49 judges whether or not a predetermined out-of-range condition (see FIG. 21) is satisfied (see step S51 in FIG. 20), and if the condition is satisfied (step S51: YES), generates a processing code corresponding to the condition. Specifically, the processing code corresponding to the condition is read from the storage unit 41.

[0144] Similarly, the processing code generation unit 49 determines whether or not a transition state condition (see FIG. 22) is satisfied as a predetermined condition (see step S52 in FIG. 20), and if the condition is satisfied (step S52: YES), generates a processing code corresponding to the condition.

[0145] Similarly, the processing code generation unit 49 determines whether or not a valid condition (see FIG. 23) is satisfied as a predetermined condition (see step S53 in FIG. 20), and if the condition is satisfied (step S53: YES), generates a processing code corresponding to the condition.

[0146] The transmission data generation unit 49A generates transmission data (e.g., a packet) including the processing code generated by the processing code generation unit 49 and the synthetic distance image generated by the secondary distance image generation unit 46. Figure 24 is an example of transmission data (packet) including the processing code and the synthetic distance image.

[0147] As shown in FIG. 24, the transmission data includes a header DT1, a processing code DT2, basic information DT3, and a synthetic distance image DT4.

[0148] The header DT1 includes the address of the driving support device 60 as the destination (an address on the in-vehicle network) and the like.

[0149] The processing code DT2 is a processing code generated by the processing code generating unit 49. The driving support device 60 receives the processing code DT2 before the response information body (basic information DT3 and synthetic distance image DT4). This allows the driving support device 60 to determine how to handle the response information body before receiving it. That is, the driving support device 60 can select a processing operation early. In other words, the distance image "DT4" has a large data volume, which increases the communication time, but when immediacy of situation judgment is important for the upper device (driving support device 60), the upper device (driving support device 60) can receive "DT4" and select the next process by referring to "DT2", which has a smaller data volume than "DT4".

[0150] Furthermore, by referring to the processing code DT2, the driving assistance device 60 can determine the usefulness of the composite distance image DT4 without performing any processing on the composite distance image DT4. In other words, the driving assistance device 60 can quickly determine the usefulness of the composite distance image DT4, which is the sensing result. Therefore, the driving assistance device 60 can quickly execute various subsequent processes.

[0151] The basic information DT3 is basic information data for handling a processed distance image (synthetic distance image). The basic information DT3 is, for example, a measurement time (for example, a time when the optical probe devices 20R, 20L irradiate the probe light (laser beam group)), a received vehicle speed (the vehicle speed of the host vehicle Va transmitted by the driving assistance device 60 to the vehicle forward visual confirmation device 10A), and a reference distance (a reference distance set by the vehicle forward visual confirmation device 10A). The basic information DT3 may include other information. For example, when narrowing the readout angle of view of the image sensor 31, the basic information DT3 may include information on the left, right, top, and bottom angles of view, a start pixel address, etc.

[0152] The synthetic distance image DT4 is data of a synthetic distance image (secondary distance image) generated by the secondary distance image generating unit 46.

[0153] The communication unit 47 transmits the transmission data generated by the transmission data generation unit 49A to a higher-level driving assistance device 60 (an example of another device of the present disclosure). For example, the data is transmitted to the higher-level driving assistance device 60 as response information to a command. Note that the processing code generated by the processing code generation unit 49 and the synthetic distance image generated by the secondary distance image generation unit 46 may be transmitted to the higher-level driving assistance device 60 in a mutually associated state, not necessarily in the form of a packet.

[0154] Next, an example of the operation of the device 10A for visually confirming a forward direction of a vehicle will be described.

[0155] FIG. 25 is a flowchart of an operation example of the device 10A for visually confirming a forward direction of a vehicle.

[0156] First, the imaging process is executed (step S10). The imaging process has already been described (see FIG. 11), so a description thereof will be omitted.

[0157] Next, the initial stage processing is executed (step S60). This initial stage processing is realized by the initial stage processing unit 48. The initial stage processing unit 48 executes initial stage processing including predetermined recognition processing (noise light removal, pattern recognition, processing code generation) on the right distance image and the left distance image captured in step S10. The initial stage processing unit 48 also executes the initial stage processing including predetermined recognition processing (noise light removal, pattern recognition, processing code generation) on the right measurement target area A. 20R The inner perpendicular line L3R (see Figure 4) and the left measurement area A 20L 13. In this case, the process code generation unit 49 generates the process code in step S60.

[0158] As shown in FIG. 20, the processing code generation unit 49 determines whether or not predetermined conditions (out-of-range condition, transition state condition, valid condition) are satisfied (steps S51, S52, S53), and if the conditions are satisfied (step S51: YES, step S52: YES, step S53: YES), it generates a processing code corresponding to the condition (for example, processing code "O1" in FIG. 21) (step S53).

[0159] Next, a composite distance image (an example of the secondary distance image of the present disclosure) is generated by combining the right distance image and the left distance image subjected to the initial processing in step S60 (step S61). This is realized by, for example, the secondary distance image generator 46.

[0160] Next, transmission data is created (step S62). This is realized by transmission data generation unit 49A. Transmission data generation unit 49A generates, as the transmission data, a packet (see FIG. 24) including the processing code generated in step S60 and the synthetic distance image generated in step S62.

[0161] Next, the transmission data generated in step S62 is transmitted to the upper driving support device 60 (step S20A5). This is realized by the communication unit 47.

[0162] As described above, according to the second embodiment, it is possible to provide the device 10A for visualizing a forward direction of a vehicle that enables another device (for example, the driving assistance device 60) to quickly determine the usefulness of a synthetic distance image that is a sensing result.

[0163] This is because, in addition to the synthetic range image that is the sensing result, a processing code that indicates the usefulness of the sensing result is transmitted to another device (for example, the driving assistance device 60).

[0164] The numerical values ​​shown in the above embodiments are all examples, and it goes without saying that suitable numerical values ​​different from these can be used.

[0165] The above-mentioned embodiments are merely examples in every respect. The present invention is not to be interpreted as being limited by the description of the above-mentioned embodiments. For example, by combining the light emitting element 21 of the optical probe device 20 shown in FIG. 18(a) with the imaging element 31 of high pixel count in the detection camera device 30, the light emitting element 21 and the imaging element 31 provided at each of the first to third reference distances can be integrated into one set. The detection camera device 30 can also be provided at the edge of the front window (wiper sweep area). In this case, the function can be achieved in the same manner as described above by adding a perspective correction to the captured image. Thus, the present invention can be embodied in various other forms without departing from its spirit or main features. [Explanation of symbols]

[0166] 10, 10A…Vehicle forward visibility device 20...Optical probe device 20L…Left optical probe device 20R…Right optical probe device 21...Light emitting element 22...Corrective lenses 23...Mounting board 24...Oscillation circuit 25…Housing 26…Idemitsu window 30...Detection camera device 31...Image sensor 32…Receiving lens 33...Mounting board 34…Receiver circuit 35…Housing 36…Light receiving window 40...Control device 41...Storage section 42…Memory 43...imaging unit 44...Object distance calculation unit 45…Dot density detector 46...Secondary distance image generator 47…Communications Department 48…First stage processing section 49... Processing code generator 49A…Transmission data generation unit 50, 50R…vehicle lighting fixtures 51...Lamp unit 51a...semiconductor light emitting element 51b…Reflective surface 51c…Projection lens 51d…movable shade 52…Bracket 53…Optical axis adjustment mechanism 54…Outer lens 55…Housing 60...Driving assistance device A 20L … Left measurement area A 20R …Right measurement area AX 20L , AX 20R …optical axis Cp1, Cp2…center point H1-H1…Horizontal line Hp: Optical probe device height L1L, L1R...Top L2L, L2R...Bottom edge L3L, L3R…Inner perpendicular lines L4L, L4R…outside perpendicular line Lc…distance P1b, P2b…lower end point P1t, P2t...Top end point S1…Light room V1-V1…Plumb line V2-V2…Plumb line Va…Car Wf: Inner border interval Wp…distance between optical axes θi…angle

Claims

1. A vehicle forward visibility device mounted on a vehicle, an optical probe device that irradiates a group of laser beams that form a dot pattern on a measurement target area set in front of the vehicle; a detection camera device that captures an image of return light irradiated from the optical probe device and reflected by an object present in the measurement target area, and generates a distance image whose pixel values ​​are distance values; a first-stage processing unit that performs a first-stage process including a predetermined recognition process on the distance image; a processing code generating unit that executes a processing code generating process for generating a processing code representing a result of the predetermined recognition process; A communication unit that transmits the distance image and the processing code to another device.

2. The optical probe device includes: a right optical probe device provided on the right side of a front end portion of the vehicle, the right optical probe device irradiating a group of laser beams that form a dot pattern in a right measurement target area set at the predetermined reference distance forward of the vehicle; and a left optical probe device provided on the left side of a front end portion of the vehicle, the left optical probe device irradiating a group of laser beams that form a dot pattern in a left measurement target area set at the predetermined reference distance forward of the vehicle; the detection camera device captures the return light irradiated from the right optical probe device and reflected by an object present in the right measurement target area, and generates a right distance image as a primary distance image in which pixel values ​​are distance values; on the other hand, the detection camera device captures the return light irradiated from the left optical probe device and reflected by an object present in the left measurement target area, and generates a left distance image as a primary distance image in which pixel values ​​are distance values; the right measurement target area and the left measurement target area are arranged symmetrically and at least partially overlap each other in a horizontal direction in front of the vehicle; a secondary distance image generating unit that generates a secondary distance image to be transmitted to the other device based on the right distance image and the left distance image, the first-stage processing unit executes the predetermined recognition processing on the first distance image and the second distance image; The device for visually confirming a forward view of a vehicle according to claim 1 , wherein the communication unit transmits the processing code and the secondary distance image to the separate device.

3. a storage unit that stores a predetermined condition and a processing code that is generated when the predetermined condition is satisfied, in association with each other; 2 . The device for visually confirming a forward view of a vehicle according to claim 1 , wherein the processing code generating unit determines whether the predetermined condition is satisfied, and if the predetermined condition is satisfied, generates a processing code corresponding to the condition by referring to the storage unit.

4. The device for visually confirming a forward movement of a vehicle according to claim 3 , wherein the predetermined condition is at least one of an out-of-range condition, a transition state condition, and a valid condition.

Images