Object detection device
The object detection device uses image analysis to determine vehicle lane position by comparing vehicle positions across multiple images, addressing the challenge of undetectable lane markings at night, ensuring safe driving decisions.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- TOYOTA JIDOSHA KK
- Filing Date
- 2023-04-20
- Publication Date
- 2026-06-23
AI Technical Summary
In low ambient light conditions, such as at night, vehicles at a distance from the host vehicle can be detected by their taillights, but lane markings may not be discernible, making it difficult to determine if the vehicles are in the same lane.
An object detection device that uses a camera and an ECU to analyze multiple images over time, comparing the relative position of detected vehicles to a reference line representing the vehicle's direction of travel, determining if the vehicles are in the same lane based on changes in position relative to this line.
Enables accurate determination of whether vehicles are in the same lane even when lane markings are undetectable, thereby facilitating safe driving decisions.
Smart Images
Figure 0007878141000001 
Figure 0007878141000002 
Figure 0007878141000003
Abstract
Description
Technical Field
[0006] , ,
[0005] , , ,
[0001] The present disclosure relates to an object detection device that detects an object from an image.
Background Art
[0002] There is known a driving control device that automatically controls the driving of a vehicle based on a peripheral image generated by a camera mounted on the vehicle. The driving control device controls the driving of the vehicle based on whether the lane in which another vehicle exists in front of the host vehicle is the host vehicle's own driving lane or an adjacent lane. For example, when another vehicle is stopped in front of the own lane and the host vehicle and the other vehicle are approaching, the driving control device performs driving control such as deceleration or lane change so that the host vehicle does not collide with the other vehicle.
[0003] Patent Document 1 describes an obstacle detection device for a vehicle that detects a preceding vehicle region and lane dividing lines from an image obtained by a camera attached to the vehicle and recognizes the behavior of the preceding vehicle based on the temporal variation in the positions of the preceding vehicle region and the lane dividing lines.
Prior Art Documents
Patent Documents
[0004]
Patent Document 1
Summary of the Invention
Problems to be Solved by the Invention
[0005] At night when the ambient light amount is insufficient, for another vehicle at a large distance from the host vehicle, even if the other vehicle can be detected from the image of the in-vehicle camera by a taillight or the like, the lane dividing lines around it may not be detected. In such a situation, it is not possible to appropriately determine whether the other vehicle exists in the own lane based on the positions of the other vehicle and the lane dividing lines.
[0006] The purpose of this disclosure is to provide an object detection device that can appropriately determine whether or not another vehicle is present in its own lane, even in situations where lane markings cannot be detected. [Means for solving the problem]
[0007] The gist of this disclosure is as follows:
[0008] (1) A detection unit that detects other vehicle regions representing other vehicles traveling in front of the vehicle from each of a plurality of images representing the direction of travel of the vehicle, generated at multiple times, A determination unit determines whether the other vehicle is in the same lane as the vehicle at the second time, by comparing the relative position of the other vehicle's region with respect to a reference line representing the direction of travel of the vehicle in the first image generated at the first time from among the plurality of images, and the relative position of the other vehicle's region with respect to the reference line in the second image generated at the second time from among the plurality of images, which is after the first time; An object detection device equipped with the following features.
[0009] According to the object detection device described herein, it is possible to appropriately determine whether or not another vehicle is present in the lane, even in situations where lane markings cannot be detected. [Brief explanation of the drawing]
[0010] [Figure 1] This is a schematic diagram of a vehicle equipped with an object detection device. [Figure 2] This is a schematic diagram of the ECU hardware. [Figure 3] This is a functional block diagram of the processor in the ECU. [Figure 4] (a) is a schematic diagram illustrating the first image generated at a first time for the first other vehicle, and (b) is a schematic diagram illustrating the second image generated at a second time for the first other vehicle. [Figure 5](a) is a schematic diagram illustrating the first image generated at a first time for a second vehicle, and (b) is a schematic diagram illustrating the second image generated at a second time for a second vehicle. [Figure 6] This is a flowchart of the object detection process. [Modes for carrying out the invention]
[0011] The following describes in detail an object detection device that can appropriately determine whether or not another vehicle is present in its own lane, even when lane markings cannot be detected, with reference to the drawings.
[0012] The object detection device detects areas representing other vehicles traveling in front of the vehicle from each of multiple images representing the vehicle's direction of travel, which are generated at multiple times.
[0013] The object detection device compares the relative position of the other vehicle's area with respect to a reference line in the first image generated at a first time from among multiple images, and the relative position of the other vehicle's area with respect to the reference line in the second image generated at a second time from among multiple images. The reference line is a line set corresponding to the direction of travel of the vehicle. The second time is a time later than the first time. Based on this comparison of relative positions, the object detection device determines whether the other vehicle is in the same lane as the vehicle traveling at the second time.
[0014] Figure 1 is a schematic diagram of a vehicle equipped with an object detection device.
[0015] Vehicle 1 has a camera 2 and an ECU 3 (Electronic Control Unit). The ECU 3 is an example of an object detection device. Camera 2 and the ECU 3 are connected to communicate via an in-vehicle network compliant with standards such as a controller area network.
[0016] The camera 2 is an example of an imaging unit that captures an image representing the situation in the traveling direction of the vehicle. The camera 2 includes a two-dimensional detector composed of an array of photoelectric conversion elements having sensitivity to infrared light, such as a CCD or a C-MOS, and an imaging optical system that forms an image of the area to be imaged on the two-dimensional detector. The camera 2 is disposed, for example, at the upper front inside the vehicle cabin, facing forward, and captures the situation in front of the vehicle 1 through the windshield at a predetermined imaging cycle (e.g., 1 / 30 second to 1 / 10 second), and outputs an image corresponding to the surrounding situation.
[0017] The ECU 3 has a communication interface, a memory, and a processor. The ECU 3 detects other vehicles in front of the vehicle 1 based on the image received from the camera 2 via the communication interface, and determines whether the other vehicle exists in the own lane in which the vehicle 1 is traveling.
[0018] FIG. 2 is a hardware schematic diagram of the ECU 3. The ECU 3 includes a communication interface 31, a memory 32, and a processor 33.
[0019] The communication interface 31 is an example of a communication unit and has a communication interface circuit for connecting the ECU 3 to the in-vehicle network. The communication interface 31 supplies the received data to the processor 33. Further, the communication interface 31 outputs the data supplied from the processor 33 to the outside.
[0020] The memory 32 is an example of a storage unit and has a volatile semiconductor memory and a non-volatile semiconductor memory. The memory 32 stores various data used for processing by the processor 33, such as parameters of an identifier used for detecting an other vehicle area from an image, etc. Further, the memory 32 stores various application programs, such as a computer program for object detection that executes object detection processing.
[0021] The processor 33 is an example of a control unit and has one or more processors and their peripheral circuits. The processor 33 may further have other arithmetic circuits such as a logical arithmetic unit, a numerical arithmetic unit, or a graphic processing unit.
[0022] FIG. 3 is a functional block diagram of the processor 33 included in the ECU 3.
[0023] The processor 33 of the ECU 3 has, as functional blocks, a detection unit 331 and a determination unit 332. Each of these units included in the processor 33 is a functional module implemented by a program executed on the processor 33. Alternatively, each of these units included in the processor 33 may be implemented in the ECU 3 as an independent integrated circuit, microprocessor, or firmware.
[0024] The detection unit 331 receives, from the camera 2 via the communication interface, a plurality of images representing the situation of the traveling direction of the vehicle 1 generated at a plurality of times. The detection unit 331 detects, from each of the received plurality of images, an other-vehicle region in which another vehicle traveling ahead of the vehicle 1 is represented.
[0025] The detection unit 331 specifies an other-vehicle region in which the rear surface of the preceding other vehicle is represented by inputting the image received from the camera 2 into an identifier that has been pre-learned to detect an other-vehicle region.
[0026] The identifier can be, for example, a convolutional neural network (CNN) having a plurality of convolutional layers connected in series from an input side to an output side. By inputting an image including an other-vehicle region as teacher data into the CNN in advance and performing learning, the CNN operates as an identifier that detects a lane dividing line and an other-vehicle region.
[0027] Figure 4(a) is a schematic diagram illustrating the first image generated at a first time for the first other vehicle, and Figure 4(b) is a schematic diagram illustrating the second image generated at a second time for the first other vehicle. Figure 5(a) is a schematic diagram illustrating the first image generated at a first time for the second other vehicle, and Figure 5(b) is a schematic diagram illustrating the second image generated at a second time for the second other vehicle. The first time is earlier than the second time.
[0028] Figures 4(a) and 4(b) show a first situation in which a first other vehicle 10 is traveling ahead of vehicle 1 in the same lane L1 in which vehicle 1 is traveling. The same lane L1 is demarcated by lane markings LL11-LL12. The detection unit 331 may or may not detect the lane markings LL11-LL12.
[0029] In Figures 4(a) and 4(b), the reference line RL1 represents the direction of travel of vehicle 1. The reference line RL1 is preset based on the mounting position and mounting direction of camera 2 on vehicle 1. The reference line RL1 may be a center line that divides the width of the image equally, or a vertical line of the image that passes through the point where the optical flow of feature points emerges in multiple images.
[0030] Referring to Figure 4(a), the other vehicle region 101 representing the first other vehicle 10 detected from the first image has a width W11 as its horizontal length in the image. The distance from the horizontal center C11 of the other vehicle region 101 to the reference line RL1 is D11.
[0031] Referring to Figure 4(b), the other vehicle region 102 representing the first other vehicle 10, detected from the second image, has a width W12 as its horizontal length in the image. The distance from the horizontal center C12 of the other vehicle region 102 to the reference line RL1 is D12.
[0032] Figures 5(a) and 5(b) show a second situation in which a second vehicle 20 is traveling ahead of vehicle 1 in the same lane L1 in which vehicle 1 is traveling.
[0033] In Figures 5(a) and 5(b), the reference line RL2 represents the direction of travel of vehicle 1. The reference line RL2 is set in the same way as the reference line RL1.
[0034] Referring to Figure 5(a), the other vehicle region 201 representing the second other vehicle 20 detected from the first image has a width W21 as its horizontal length in the image. The distance from the horizontal center C21 of the other vehicle region 201 to the reference line RL2 is D21.
[0035] Referring to Figure 5(b), the other vehicle region 202 representing the second other vehicle 20 detected from the second image has a width W22 as its horizontal length in the image. The distance from the lateral center C22 of the other vehicle region 202 to the reference line RL2 is D22.
[0036] The determination unit 332 compares the relative position of the other vehicle's area with respect to the reference line in the first image generated at a first time from among the multiple images, and the relative position of the other vehicle's area with respect to the reference line in the second image generated at a second time from among the multiple images. Based on this comparison of relative positions, the determination unit 332 determines whether the other vehicle is in the same lane as the vehicle traveling at the second time.
[0037] The distance from the lateral center of the other vehicle area to the reference line is an example of the relative position of the other vehicle area with respect to the reference line.
[0038] In the first situation shown in Figures 4(a) and 4(b), the distance D12 from the center C12 of the other vehicle region 102 where the first other vehicle 10 is represented in the second image generated at the second time, to the reference line RL1, is shorter than the distance D11 from the center C11 of the other vehicle region 101 where the first other vehicle 10 is represented in the first image generated at the first time, to the reference line RL1. In this case, the determination unit 332 determines that the first other vehicle 10 is in its own lane L1.
[0039] On the other hand, in the second situation shown in Figures 5(a) and 5(b), the distance D22 from the center C22 of the other vehicle region 202 where the second other vehicle 20 is represented in the second image generated at the second time, to the reference line RL2, is longer than the distance D21 from the center C21 of the other vehicle region 201 where the second other vehicle 20 is represented in the first image generated at the first time, to the reference line RL2. In this case, the determination unit 332 determines that the second other vehicle 20 is not present in the vehicle's own lane L1.
[0040] The determination unit 332 may determine that the first other vehicle 10 is present in the own lane L1 if the distance from the center of the other vehicle area in the second image to the reference line is smaller than a predetermined distance threshold, even if that distance is longer than the distance from the center of the other vehicle area to the reference line in the first image. By making this determination, the determination unit 332 can suppress misdeterminations caused by deviations in the trajectory of the other vehicle or the own vehicle, or by measurement errors.
[0041] Figure 6 is a flowchart of the object detection process. The ECU 3 repeatedly performs the object detection process at predetermined time intervals (for example, every 1 / 10 second) while the vehicle 1 is in motion.
[0042] First, the detection unit 331 of the ECU3 processor 33 detects an area representing another vehicle traveling in front of vehicle 1 from among multiple images received from camera 2, specifically from the first image generated at a first time and the second image generated at a second time (step S1). The second time is a time later than the first time.
[0043] Next, the determination unit 332 of the processor 33 determines whether the area of other vehicles in the second image is closer to the reference line than the area of other vehicles in the first image (step S2).
[0044] If the region of the other vehicle in the second image is determined to be closer to the reference line than the region of the other vehicle in the first image (step S2:Y), the determination unit 332 determines that the other vehicle is in the same lane that vehicle 1 is traveling in (step S3), and terminates the object detection process.
[0045] If it is determined that the area of the other vehicle in the second image is not closer to the reference line than the area of the other vehicle in the first image (step S2:N), the determination unit 332 determines that the other vehicle is not in the lane in which vehicle 1 is traveling (step S3), and terminates the object detection process.
[0046] By performing object detection processing in this manner, the ECU3 can appropriately determine whether or not another vehicle is present in its own lane, even when lane markings cannot be detected.
[0047] The ECU3 may further execute driving control processing, for example, by using the determination result from object detection processing to decelerate vehicle 1 and / or change lanes if the distance to another vehicle determined to be in the same lane falls below a predetermined distance threshold.
[0048] Those skilled in the art should understand that various changes, substitutions, and modifications can be made to this disclosure without departing from its spirit and scope. [Explanation of symbols]
[0049] 1 vehicle 3 ECU 331 Detection unit 332 Judgment section
Claims
[Claim 1] A detection unit detects, from each of multiple images representing the vehicle's direction of travel, generated at multiple times, a region representing another vehicle traveling in front of the vehicle. A determination unit determines whether the other vehicle is in the same lane as the vehicle at the second time by comparing the relative position of the other vehicle's region with respect to a reference line representing the direction of travel of the vehicle in the first image generated at a first time from among the plurality of images, with the relative position of the other vehicle's region with respect to the reference line in the second image generated at a second time after the first time from among the plurality of images, An object detection device equipped with the following features.