Apparatus, method, and program for detecting deceleration of a preceding vehicle

Abstract

To make it possible to detect deceleration of a preceding vehicle without depending on detection of lighting of a brake lamp of the preceding vehicle in a vehicle on which a monocular camera is mounted.SOLUTION: A device detects a preceding vehicle 4 and white broken lines 6L, 6R provided at equal intervals along a road 2 from an image captured by a monocular camera. The device acquires speed information of an own vehicle. The device counts the number of the white broken lines 6L, 6R included in a section from a longitudinal position y1 of the preceding vehicle 4 to a fixed reference position y2 below the longitudinal position y1 in a camera image, and detects deceleration of the preceding vehicle 4 based on time-series transition of the number of the white broken lines 6L, 6R and the speed information of the own vehicle.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a technique for detecting deceleration of a leading vehicle, and in particular to a technique suitable for use in a vehicle equipped with a monocular camera. [Background technology]

[0002] Japanese Patent Laid-Open Publication No. 9-267686 discloses a method for detecting the illumination of the brake lights of a leading vehicle by performing recognition processing on a forward image captured by an on-board camera, and predicting deceleration of the leading vehicle from the illumination of the brake lights. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 9-267686 Summary of the Invention [Problem to be solved by the invention]

[0004] However, it is not always possible to detect the brake lights of a preceding vehicle from an image captured by an on-board camera. For example, when the sun is behind the vehicle, it is difficult to detect the brake lights due to the reflection of sunlight. Also, the brake lights do not necessarily come on when the preceding vehicle decelerates. For example, engine braking or going uphill will slow down the vehicle even if the brake lights are not on. Conversely, because the brake pedal has play, the preceding vehicle may not decelerate even if the brake lights are on.

[0005] In-vehicle cameras include stereo cameras and monocular cameras. Stereo cameras can detect changes in the distance of a preceding vehicle relative to the vehicle itself. Therefore, if the in-vehicle camera is a stereo camera, it can easily detect the deceleration of the preceding vehicle without relying on detection of the illumination of the brake lights. However, since stereo cameras are more expensive than monocular cameras, the number of vehicles that can be equipped with them is limited. In order to improve safety in more vehicles, it is desirable to be able to detect the deceleration of a preceding vehicle without relying on detection of the illumination of the brake lights even in vehicles that do not have expensive sensors such as stereo cameras.

[0006] The present disclosure has been made in consideration of the above problems. One objective of the present disclosure is to enable a vehicle equipped with a monocular camera to detect deceleration of a preceding vehicle without relying on detection of the illumination of the brake lights of the preceding vehicle. [Means for solving the problem]

[0007] The present disclosure provides an apparatus for achieving the above object. The apparatus of the present disclosure includes a monocular camera that captures an image ahead of the vehicle, a processor, and a memory coupled to the processor and storing a plurality of instructions. The plurality of instructions are configured to cause the processor to execute the following first to fifth processes. The first process is to detect a preceding vehicle from a camera image captured by the monocular camera. The second process is to detect equally-spaced stationary objects provided at equal intervals along a road from the camera image. The third process is to obtain speed information of the vehicle. The fourth process is to count the number of equally-spaced stationary objects included in a section from the longitudinal position of the preceding vehicle in the camera image to a fixed reference position below the longitudinal position. And the fifth process is to detect deceleration of the preceding vehicle based on the time series change in the number of equally-spaced stationary objects and the speed information of the vehicle.

[0008] The present disclosure also provides a method for achieving the above object. The method of the present disclosure is a method executed by an on-board computer and includes the following first to fifth steps. The first step is to detect a leading vehicle from a camera image of the front of the host vehicle captured by a monocular camera mounted on the host vehicle. The second step is to detect equally-spaced stationary objects provided at equal intervals along the road from the camera image. The third step is to obtain speed information of the host vehicle. The fourth step is to count the number of equally-spaced stationary objects included in the section from the longitudinal position of the leading vehicle in the camera image to a fixed reference position below the longitudinal position. And the fifth step is to detect deceleration of the leading vehicle based on the time series transition of the number of equally-spaced stationary objects and the speed information of the host vehicle.

[0009] Furthermore, the present disclosure provides a program for achieving the above object. The program of the present disclosure is configured to cause an in-vehicle computer to execute the method including the above-mentioned steps 1 to 5. The program of the present disclosure is recorded in a computer-readable recording medium. Effect of the Invention

[0010] When a camera image captured by a monocular camera captures stationary objects at equal intervals together with the preceding vehicle, the distance between the vehicle and the preceding vehicle can be measured by counting the number of stationary objects included in the section from the longitudinal position of the preceding vehicle to a fixed reference position in the camera image. By taking into account the speed information of the vehicle in question in the time series transition of the number of stationary objects at equal intervals, the speed information of the preceding vehicle can be obtained from the information. The speed information of the preceding vehicle that can be obtained includes the deceleration of the preceding vehicle. Therefore, according to the technology disclosed herein, in a vehicle equipped with a monocular camera, the deceleration of the preceding vehicle can be detected without relying on the detection of the illumination of the brake lights of the preceding vehicle. [Brief description of the drawings]

[0011] [Figure 1] 1 is a diagram illustrating an overview of a method for detecting deceleration of a leading vehicle according to an embodiment of the present disclosure. [Diagram 2] FIG. 2A is a diagram for explaining a method for counting white dashed lines in a distant area, FIG. 2B is a diagram for explaining a method for counting faint white dashed lines, and FIG. 2C is a diagram for explaining a method for counting white dashed lines hidden under a leading vehicle. [Diagram 3] Figure 3A shows the time series change in the number of white dashed lines when the preceding vehicle starts to decelerate while the vehicle is traveling at a constant speed. Figure 3B shows the time series change in the number of white dashed lines when the preceding vehicle starts to decelerate while the vehicle is accelerating. Figure 3C shows the time series change in the number of white dashed lines when the preceding vehicle starts to decelerate while the vehicle is decelerating. [Figure 4] 4 is a flowchart illustrating a method for detecting deceleration of a leading vehicle according to an embodiment of the present disclosure. [Diagram 5] 1 is a diagram illustrating a configuration of a device for detecting deceleration of a leading vehicle according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] 1. Method for detecting the deceleration of the preceding vehicle An overview of a method for detecting deceleration of a leading vehicle according to an embodiment of the present disclosure (hereinafter, referred to as the method according to the present embodiment) will be described with reference to Fig. 1. Fig. 1 shows an image of the forward field of view of the host vehicle captured by an on-board monocular camera. The image shows two white dashed lines 6L, 6R drawn on a road 2. The white dashed lines 6L, 6R are lane boundary lines that define the lane in which the host vehicle is traveling. The image shows a leading vehicle 4 traveling ahead of the host vehicle in the lane defined by the white dashed lines 6L, 6R.

[0013] Two dotted lines are drawn in the horizontal direction (X direction) on the image. The upper dotted line is a line indicating the bottom end of the preceding vehicle 4 in the vertical direction (Y direction) of the image. In this embodiment, the position y1 of the bottom end of the preceding vehicle 4 in the vertical direction is defined as the vertical position of the preceding vehicle 4. The lower dotted line is a line indicating the bottom end of the vertical direction of the image. In this embodiment, the position y2 of the bottom end of the image in the vertical direction is defined as a fixed reference position.

[0014] The white dashed lines 6L, 6R are stationary objects that are equally spaced apart along the road 2. The distance from the vehicle to the preceding vehicle 4 can be measured by counting the number of white dashed lines 6L, 6R included in the area from the fixed reference position y2 to the longitudinal position y1 of the preceding vehicle 4. Hereinafter, the area from the fixed reference position y2 to the longitudinal position y1 is referred to as a white dashed line count area. In the method according to this embodiment, the number of white dashed lines 6L, 6R included in the white dashed line count area is counted.

[0015] In the example shown in FIG. 1, the number of white dashed lines 6L, 6R included in the white dashed line count area is six. However, the number of white dashed lines 6L, 6R is not always clear-cut. For example, as shown in FIG. 2A, when a leading vehicle 4 is traveling at a distance, it is difficult to distinguish and recognize each of the white dashed lines 6L, 6R in the distant area. In this case, the number of white dashed lines 6L, 6R in the distant area is estimated based on the recognition results of the white dashed lines 6L, 6R in the close range. Also, as shown in FIG. 2B, some of the white dashed lines 6R i However, the size of the white dashed lines 6L and 6R is determined by the standard, so the white dashed line 6R is not visible. i The white dashed lines before and after it are 6R i-1 , 6R i+1 As shown in FIG. 2C, the dashed white line 6R hidden under the leading vehicle 4 is j For the opposite white dashed line 6L j can be estimated from

[0016] In the method disclosed herein, the vertical position defining the white dashed line count area does not necessarily have to be the bottom end of the preceding vehicle 4, so long as it is a position set based on the preceding vehicle 4. For example, the vertical position may be the center position of the preceding vehicle 4. Also, the fixed reference position does not necessarily have to be the bottom end of the image, so long as it is always located below the vertical position of the preceding vehicle 4 and is a fixed position within the image. For example, if the bonnet of the host vehicle is shown in the image, the tip of the bonnet may be set as the fixed reference position.

[0017] 3A, 3B, and 3C show examples of time series transition of the number of white dashed lines included in the white dashed line count area. Hereinafter, the number of white dashed lines means the number of white dashed lines included in the white dashed line count area. FIG. 3A shows the time series transition of the number of white dashed lines when the preceding vehicle starts to decelerate while the host vehicle is traveling at a constant speed. FIG. 3B shows the time series transition of the number of white dashed lines when the preceding vehicle starts to decelerate while the host vehicle is accelerating. And FIG. 3C shows the time series transition of the number of white dashed lines when the preceding vehicle starts to decelerate while the host vehicle is decelerating. The number of white dashed lines changes in time according to the speed state of the host vehicle and the speed state of the preceding vehicle.

[0018] The deceleration start point shown in Figures 3A, 3B, and 3C is the time when the preceding vehicle starts decelerating. The time series transition of the number of white dashed lines shown by solid lines is the time series transition when the preceding vehicle starts decelerating at the deceleration start point. The time series transition of the number of white dashed lines shown by dotted lines is the time series transition when the preceding vehicle continues to travel at a constant speed. In addition, in each of Figures 3A, 3B, and 3C, Case A shows the time series transition of the number of white dashed lines when the speeds of the host vehicle and the preceding vehicle are equal at time zero. Case B shows the time series transition of the number of white dashed lines when the speed of the preceding vehicle is higher than the speed of the host vehicle at time zero. Case C shows the time series transition of the number of white dashed lines when the speed of the preceding vehicle is lower than the speed of the host vehicle at time zero.

[0019] As shown in each case of FIG. 3A, 3B, and 3C, the time series transition of the number of white dashed lines changes uniquely at the deceleration start point of the preceding vehicle. Detecting the deceleration of the preceding vehicle is synonymous with detecting a unique deceleration start point from the time series transition of the number of white dashed lines. However, as is clear from the comparison of FIG. 3A, 3B, and 3C, the time series transition of the number of white dashed lines shows different aspects depending on the speed state of the host vehicle. In consideration of such characteristics, in the method according to the present embodiment, the deceleration of the preceding vehicle is detected based on the time series transition of the number of white dashed lines and the speed information of the host vehicle. Note that the speed information of the host vehicle includes at least one of information related to the speed of the host vehicle and information related to the acceleration.

[0020] There are at least two methods for detecting deceleration of a preceding vehicle from the time series change in the number of white dashed lines and the speed information of the vehicle itself. The first deceleration detection method is a method in which the time series change in the number of white dashed lines is standardized using the speed information of the vehicle itself, and the deceleration start point is detected from the time series change in the standardized number of white dashed lines. The second deceleration detection method is a method in which the time series change in the number of white dashed lines when the preceding vehicle is assumed to be moving at a constant speed is estimated based on the speed information of the vehicle itself, and the deceleration start point is detected from the difference between the time series change in the estimated number of white dashed lines and the time series change in the actual number of white dashed lines.

[0021] The first deceleration detection method will be specifically described. The difference in the speed of the preceding vehicle relative to the speed of the own vehicle represents the time rate of change of the distance from the own vehicle to the preceding vehicle. Since the time rate of change of the distance from the own vehicle to the preceding vehicle corresponds to the time rate of change of the number of white dashed lines, the time rate of change of the number of white dashed lines corresponds to the difference in the speed of the preceding vehicle relative to the speed of the own vehicle. And the time rate of change of the time rate of change of the number of white dashed lines corresponds to the difference in the acceleration of the preceding vehicle relative to the acceleration of the own vehicle. Therefore, by correcting the time rate of change of the time rate of change of the number of white dashed lines with the acceleration of the own vehicle, information on the acceleration or deceleration of the preceding vehicle can be obtained. Correcting the time rate of change of the time rate of change of the number of white dashed lines with the acceleration of the own vehicle means standardizing the time series transition of the number of white dashed lines with the speed information of the own vehicle.

[0022] Next, the second deceleration detection method will be specifically described. The time series transition of the number of white broken lines when the preceding vehicle continues to run at a constant speed can be estimated from the speed and acceleration of the vehicle. The time series transition of the number of white broken lines shown by dotted lines in Figs. 3A, 3B, and 3C is the time series transition when the preceding vehicle is assumed to be running at a constant speed, estimated from the speed information of the vehicle. On the other hand, the time series transition of the number of white broken lines shown by solid lines in Figs. 3A, 3B, and 3C is the time series transition of the actual number of white broken lines counted in the camera image. While the preceding vehicle is running at a constant speed, the time series transition of the actual number of white broken lines and the time series transition when the preceding vehicle is assumed to be running at a constant speed are approximately the same. However, when the preceding vehicle starts to decelerate, a discrepancy begins to occur between the time series transition of the actual number of white broken lines and the time series transition when the preceding vehicle is assumed to be running at a constant speed. Therefore, the time point when the difference between the actual number of white broken lines and the assumed number becomes equal to or greater than a predetermined value can be detected as the deceleration start point of the preceding vehicle.

[0023] The above-mentioned first deceleration detection method and second deceleration detection method can be used in combination. For example, when the deceleration start point of the preceding vehicle is detected by the first deceleration detection method and the deceleration start point of the preceding vehicle is detected by the second deceleration detection method, it may be determined that the preceding vehicle has decelerated. Such a combination can improve the detection accuracy of the deceleration of the preceding vehicle. Also, when the deceleration start point of the preceding vehicle is detected by the first deceleration detection method or the deceleration start point of the preceding vehicle is detected by the second deceleration detection method, it may be determined that the preceding vehicle has decelerated. By using the two deceleration detection methods in combination, it is possible to prevent missed detection or delayed detection of the deceleration of the preceding vehicle.

[0024] There is also a third deceleration detection method that can be applied only when the vehicle is decelerating. As shown in FIG. 3C, when the vehicle is decelerating, the number of white dashed lines increases over time, but when the preceding vehicle decelerates, the increase becomes gentle. Finally, the time series transition of the number of white dashed lines becomes constant or decreases. Therefore, when the vehicle is determined to be decelerating based on the speed information of the vehicle, the preceding vehicle may be determined to have decelerated when the time series transition of the number of white dashed lines becomes constant or when the time series transition shows a decreasing trend. The third deceleration detection method can be combined with the first deceleration detection method or the second deceleration detection method. By using the third deceleration detection method in combination, it is possible to prevent missed detection or delayed detection of the deceleration of the preceding vehicle when the vehicle is decelerating.

[0025] 4 is a flowchart showing a method according to this embodiment. The steps of the method according to this embodiment will be described with reference to this flowchart together with FIG.

[0026] Step S01 is to recognize the leading vehicle 4 with an on-board monocular camera and obtain the bottom edge of the image of the leading vehicle 4 as the longitudinal position y1 of the leading vehicle 4. Step S02 is to recognize white dashed lines 6L, 6R on the road surface with the on-board monocular camera. The white dashed lines 6L, 6R are a pair of white dashed lines that define the lane in which the host vehicle is traveling. The image used to recognize the white dashed lines 6L, 6R in step S02 is the image used to recognize the leading vehicle 4 in step S01. Steps S01 and S02 may be executed in the reverse order, or may be executed simultaneously.

[0027] Step S03 is to count the number of white dashed lines 6L, 6R between the longitudinal position y1 of the preceding vehicle 4 and a fixed reference position y2 at the bottom edge of the image. Then, step S04 is to detect deceleration of the preceding vehicle 4 based on the time series transition of the number of white dashed lines 6L, 6R counted in step S03 and the speed information of the host vehicle. For example, the first to third deceleration detection methods described above are used to detect the deceleration of the preceding vehicle 4.

[0028] By executing the above steps S01 to S04, the method according to this embodiment makes it possible to detect the deceleration of the preceding vehicle 4 without relying on the detection of the illumination of the brake lights of the preceding vehicle 4.

[0029] 2. Device that detects the deceleration of the preceding vehicle Next, an apparatus for implementing the method according to the present embodiment will be described. Fig. 5 is a diagram showing a configuration of an apparatus for detecting deceleration of a leading vehicle according to the embodiment of the present disclosure (hereinafter, referred to as the apparatus according to the present embodiment).

[0030] The vehicle 10 to which the device according to the present embodiment is applied may be a manually driven vehicle driven by a driver aboard the vehicle 10, or may be an autonomous vehicle driven by an autonomous driving system. For example, when the device according to the present embodiment is applied to an autonomous vehicle equipped with LiDAR and a monocular camera, the deceleration of the preceding vehicle 4 can be detected when the LiDAR is not functioning properly or in an environment where the accuracy of the LiDAR is reduced, such as in rainy weather. The vehicle 10 to which the device according to the present embodiment is applied may be a remotely driven vehicle remotely driven using wireless communication from a remote cockpit. In remote driving, the operator has a weak sense of speed relative to the preceding vehicle 4, so that the application of the device according to the present embodiment can be expected to have a high effect. Hereinafter, the vehicle 10 will be referred to as the host vehicle in order to distinguish it from the preceding vehicle 4.

[0031] The device according to this embodiment is composed of an on-board computer 12 and a monocular camera 14 mounted on the host vehicle 10. The monocular camera 14 is attached facing the front of the host vehicle 10 so as to capture an image of the area ahead of the host vehicle 10. An image of a forward field of view 16 captured by the monocular camera 14 is input to the on-board computer 12. The on-board computer 12 includes a processor 20 and a program memory 22 coupled to the processor 20. The program memory 22 is a computer-readable recording medium. The program memory 22 stores a plurality of instructions 24 executable by the processor 20.

[0032] The instructions 24 stored in the program memory 22 include instructions for detecting deceleration of the preceding vehicle 4. By executing these instructions in the processor 20, the processor 20 functions as an image acquisition unit 202, a speed information acquisition unit 204, an equally-spaced stationary object detection unit 206, a preceding vehicle detection unit 208, a number counting unit 210, and a deceleration detection unit 212.

[0033] The image acquisition unit 202 is configured to capture an image of the forward field of view 16 of the host vehicle 10 captured by the monocular camera 14. The speed information acquisition unit 204 is configured to acquire speed information of the host vehicle 10 from a speed sensor (not shown), for example, a wheel speed sensor, mounted on the host vehicle 10. The speed information includes information related to speed and information related to acceleration.

[0034] The equally-spaced stationary object detection unit 206 is configured to detect equally-spaced stationary objects provided at equal intervals along the road 2 from the image acquired by the image acquisition unit 202. Examples of equally-spaced stationary objects to be detected include white dashed lines, as well as pole cones separating lanes, dotted lines drawn inside lane boundary lines, and guardrail legs. When the detection target is a white dashed line, the detection method for the white dashed line may be an edge-based white line recognition technique or a recognizer represented by deep learning. In the case of a road having multiple lanes, multiple white dashed lines are recognized from the image. The equally-spaced stationary object detection unit 206 is configured to group the white dashed lines 6R and 6L that define the lane in which the host vehicle 10 is traveling.

[0035] The preceding vehicle detection unit 208 is configured to detect the preceding vehicle 4 from the image acquired by the image acquisition unit 202. The preceding vehicle 4 is a vehicle traveling in a lane defined by the white dashed lines 6R and 6L among the vehicles traveling in front of the host vehicle 10. When a plurality of vehicles traveling in the lane defined by the white dashed lines 6R and 6L are recognized, the vehicle closest to the host vehicle 10 is the preceding vehicle. For example, a recognition device represented by deep learning can be used as a method for detecting the preceding vehicle 4. The preceding vehicle detection unit 208 is configured to acquire the lower end of the preceding vehicle 4 in the vertical direction of the image as the vertical position of the preceding vehicle 4, and set the section from the vertical position of the preceding vehicle 4 in the image to a fixed reference position as a white line count area.

[0036] The number counting unit 210 is configured to count the number of white dashed lines 6R, 6L included in a white line count area set by the preceding vehicle detection unit 208, among the white dashed lines 6R, 6L grouped by the equally-spaced stationary object detection unit 206. Depending on the state of the white dashed lines 6R, 6L shown in the image, the number counting unit 210 counts the number of white dashed lines 6R, 6L using the method described with reference to Figures 2A, 2B, and 2C.

[0037] The deceleration detection unit 212 is configured to detect the deceleration of the preceding vehicle 4 based on the time series transition of the number of the white dashed lines 6R, 6L counted by the number counting unit 210 and the speed information of the host vehicle 10 acquired by the speed information acquisition unit 204. Between the time series transition of the number of the white dashed lines 6R, 6L and the speed information of the host vehicle 10, there are combinations represented by the cases of Figs. 3A, 3B, and 3C. The deceleration detection unit 212 is configured to be able to detect the deceleration start point of the preceding vehicle 4 in all of these combinations. The deceleration detection unit 212 detects the deceleration start point of the preceding vehicle 4 using, for example, any of the first to third deceleration detection methods described above.

[0038] 3.Other The method according to the present embodiment can also be applied to detecting the acceleration of a preceding vehicle. In other words, it is possible to detect the acceleration of a preceding vehicle from the number of white dashed lines included in the white line count area and the speed information of the vehicle itself. This is because, as in the case of deceleration of the preceding vehicle, the time series transition of the number of white dashed lines changes significantly at the point where the preceding vehicle starts accelerating. [Explanation of symbols]

[0039] 2 road 4 Leading vehicle 6L, 6R white dashed lines 10 Vehicle 12 Vehicle-mounted computer 14 Monocular Camera 20 processors 22 Program Memory 24 Instructions

Claims

1. A monocular camera that captures an image in front of the vehicle; A processor; a memory coupled to the processor and storing a plurality of instructions; The instructions may include instructions for causing the processor to: Detecting a leading vehicle from an image captured by the monocular camera; detecting stationary objects at equal intervals along a road from the image; acquiring speed information of the host vehicle; Counting the number of stationary objects included in a section from a longitudinal position of the preceding vehicle in the image to a fixed reference position below the longitudinal position; and detecting deceleration of the preceding vehicle based on the time-series transition of the number and the speed information. An apparatus comprising:

2. 2. The apparatus of claim 1, Detecting deceleration of the preceding vehicle includes: standardizing a time series transition of the number using the speed information; and detecting a specific deceleration start point of the number from a time series transition of the standardized number. An apparatus comprising:

3. 2. The apparatus of claim 1, Detecting deceleration of the preceding vehicle includes: estimating a time series transition of the number of preceding vehicles based on the speed information when it is assumed that the preceding vehicles are moving at a constant speed; and detecting a specific deceleration start point of the number of the objects from a difference between a time series transition of the estimated number of the objects and a time series transition of the actual number of the objects. An apparatus comprising:

4. A method implemented by an on-board computer, comprising: Detecting a leading vehicle from an image of a front area of ​​the host vehicle captured by a monocular camera mounted on the host vehicle; detecting stationary objects at equal intervals along a road from the image; acquiring speed information of the host vehicle; Counting the number of stationary objects included in a section from a longitudinal position of the preceding vehicle in the image to a fixed reference position below the longitudinal position; and detecting deceleration of the preceding vehicle based on the time series transition of the number and the speed information. A method comprising:

5. Detecting a leading vehicle from an image of a front area of ​​the host vehicle captured by a monocular camera mounted on the host vehicle; detecting stationary objects at equal intervals along a road from the image; acquiring speed information of the host vehicle; Counting the number of stationary objects included in a section from a longitudinal position of the preceding vehicle in the image to a fixed reference position below the longitudinal position; and detecting deceleration of the preceding vehicle based on the time-series transition of the number and the speed information. A program characterized by:

Images