Image processing device for lean vehicles

Abstract

According to the present invention, in a case in which five automobiles (61 to 65) are traveling side by side in one row in a right-left direction ahead of a leaning vehicle (1) on a five-lane (51 to 55) road, and a distance in a front-rear direction between the five automobiles (61 to 65) and the leaning vehicle (1) is a distance necessary for capturing an image so as to complete image processing at a time point of TTC of three seconds when assuming that the time from image-capturing to completion of image processing is 0.5 seconds and an object is approaching the leaning vehicle 1 at a relative speed of 15 km / h, an image-capturing device (11) includes a monocular camera the lateral-direction angle of view of which is configured such that, when the leaning vehicle (1) is traveling in a right side area (50R) of a center lane (53), a left end (61BL) of a rear face of the automobile (61) is not included in the image, but a rear face (63B) and a right side face (63R) of the automobile (63) are included in the image, and an arrival time estimation device (12) estimates arrival time until the automobile (63) reaches the leaning vehicle (1) on the basis of the image captured by the image-capturing device (11).

Description

[Technical Field]

[0001] The present invention relates to an image processing device for a lean vehicle that is mounted on a lean vehicle. [Background technology]

[0002] Conventionally, there has been known a vehicle periphery monitoring device that is mounted on a vehicle and includes an arrival time estimation device that estimates the time it will take for the vehicle to reach an object, such as another vehicle or a pedestrian, located ahead of the vehicle. An example of a vehicle periphery monitoring device is disclosed in Patent Document 1. The vehicle periphery monitoring device of Patent Document 1 includes an arrival time estimation device that calculates a time change in the width of the object in the image, i.e., a change rate, from multiple images captured by an imaging device at predetermined time intervals, and estimates the arrival time using the calculated change rate. Patent Document 2 proposes an arrival time estimation device that extracts feature points from images of an object captured in time series by an imaging device mounted on a vehicle and estimates the time it will take for the vehicle to reach the object based on the extracted feature points, and a driving assistance control device that uses the estimated arrival time to perform control to assist the driver in operating the vehicle. Patent Document 2 also proposes a lean vehicle that is equipped with an imaging device, an arrival time estimation device, and a driving assistance control device. The lean vehicle is, for example, a motorcycle or other vehicle. An imaging device generally includes an optical device such as a lens and an imaging element such as a CMOS (Complementary Metal Oxide Semiconductor) sensor. An arrival time estimation device generally includes a processor for processing images, such as an FPGA (Field-Programmable Gate Array), a GPU (Graphics Processing Unit), a CPU (Central Processing Unit), or an SoC (System on Chip), and a memory such as a RAM (Random Access Memory). Note that an SoC has a configuration in which a processor and a memory are integrated. Patent Document 2 describes that the imaging device may be a monocular camera or a stereo camera.

[0003] Furthermore, the time it takes for the vehicle to reach an object described in Patent Documents 1 and 2 corresponds to the so-called TTC (Time to Collision). Non-Patent Document 1 describes that a TTC threshold of 3 seconds is appropriate for evaluating a potential collision.

[0004] The width of lanes and the width of vehicles are regulated by national or regional administrative bodies. In Japan, lane widths are regulated to 2.75 to 4 m depending on the road classification in accordance with Article 5, Paragraphs 4 and 5 of the Road Structure Ordinance (see Non-Patent Document 2). Generally, lane widths are 3.5 to 4 m. Furthermore, in Japan, Article 59 of the Road Transport Vehicle Safety Standards stipulates that the width of motorcycles must be 1.3 m or less, and Article 2 of the Road Transport Vehicle Safety Standards stipulates that the width of automobiles must be 2.5 m or less (see Non-Patent Document 3). Generally, the width of lean-to vehicles is less than 1 m, and the width of automobiles is 2 to 2.5 m. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-213561 [Patent Document 2] Patent Publication No. 2021-111246 [Non-patent literature]

[0006] [Non-Patent Document 1] Oumaima Barhoumi, Mohamed H. Zaki, SOFIENE TAHAR, “A Formal Approach to Road Safety Assessment Using Traffic Conflict Techniques”, “VI. SENSITIVITY STUDY”, [online], last updated April 26, 2024, IEEE Open Journal of Vehicular Technology, [searched on June 4, 2020], Internet<URL:https: / / ieeexplore.ieee.org / stamp / stamp.jsp?arnumber=10496854> [Non-patent document 2] Road Structure Ordinance, [Retrieved June 4, 2024], Internet<URL:https: / / elaws.e-gov.go.jp / document?lawid=345CO0000000320> [Non-patent document 3] Road Transport Vehicle Safety Standards, [Retrieved June 4, 2024], Internet<URL:https: / / elaws.e-gov.go.jp / document?lawid=326M50000800067> Summary of the Invention [Problem to be solved by the invention]

[0007] The calculation of the rate of change in Patent Document 1 requires the determination of the change over time in the width of an object in a captured image, which requires an imaging device that captures high-resolution images. Imaging devices that capture high-resolution images tend to be large in size because they use larger image sensors. In addition, calculating the rate of change requires determining the change over time in the width of the object in the captured image, which necessitates an arrival time estimation device that processes high-resolution images to estimate the arrival time. Arrival time estimation devices that process high-resolution images to estimate the arrival time, for example, require a heat dissipation structure for the processor because the processor processing the images places a high load on the device, which can easily result in an increase in size.

[0008] However, when an image processing device mounted on an automobile as in Patent Document 2 is mounted on a lean vehicle, the following problems arise.

[0009] Since lean vehicles have slimmer (narrower) bodies than automobiles, an image processing device for lean vehicles that can be easily installed in lean vehicles is desired.

[0010] Furthermore, the width of a lean vehicle is much smaller than that of an automobile. In other words, the body of a lean vehicle is slimmer (narrower) than that of an automobile. Furthermore, the width of a lean-to vehicle is very small compared to the width of a lane. Therefore, while traveling in a single lane, a lean-to vehicle may travel in either the right or left side area of ​​the lane. Therefore, compared to automobiles, a lean-to vehicle has a much higher degree of freedom in its lateral traveling position within a single lane. Therefore, there is a demand for an image processing device for a lean-to vehicle that includes an imaging device and an arrival time estimation device that can accommodate usage scenarios specific to lean-to vehicles, such as traveling in the right or left side area of ​​a lane while traveling in a single lane. In this specification, when a single lane is divided into thirds in the left-right direction, the area including the right edge of the lane is defined as the right side area, and when a single lane is divided into thirds in the left-right direction, the area including the left edge of the lane is defined as the left side area.

[0011] The object of the present invention is to provide an image processing device for a lean vehicle that can be adapted to situations in which a lean vehicle, which has a slimmer body than an automobile, travels in the right side area and left side area of ​​a lane while traveling in the same lane as an automobile, and that is highly mountable on a lean vehicle, which has a slimmer body than an automobile. [Means for solving the problem]

[0012] (1) An image processing device for a lean vehicle according to one embodiment of the present invention has the following configuration. An image processing device for a lean vehicle includes an imaging device that captures an image, and an arrival time estimation device that processes the image captured by the imaging device to estimate an arrival time for an object to reach the lean vehicle, and is mounted on the lean vehicle. When a single lane is divided into thirds in the left-right direction, an area including the right end of the single lane is defined as a right side area, an area including the left end of the single lane is defined as a left side area, and an area between the right side area and the left side area is defined as a center area. If the lanes are defined as the leftmost lane, center left lane, center lane, center right lane, and rightmost lane, respectively, from the left to the right, the lean vehicle is traveling in the right side area or left side area of ​​the center lane of the five-lane road, and five automobiles are traveling in a line in the left-right direction ahead of the lean vehicle, and the distance between the five automobiles and the lean vehicle in the longitudinal direction is TTC (Time to Capture Time), assuming that the time from image capture to completion of image processing is 0.5 seconds, and an object approaches the lean vehicle at a relative speed of 15 km / h. When the distance (To Collision) is the distance required to capture an image so that the image processing is completed at 3 seconds, the imaging device and the arrival time estimation device are configured as follows, so that the imaging device and the arrival time estimation device are configured to estimate the arrival time of the vehicle traveling in the center lane to reach the lean vehicle based on the image that includes the left side or right side of the vehicle traveling in the center lane, both when the lean vehicle is traveling in the right side area of ​​the center lane of the five-lane road and when the lean vehicle is traveling in the left side area of ​​the center lane of the five-lane road. The imaging device includes a monocular camera with a left-right angle of view configured so that, while the leaned vehicle is traveling in the right side area of ​​the center lane of the five-lane road, the left edge of the rear of the vehicle traveling in the leftmost lane is not included in the image, but at least the rear of the vehicle traveling in the center lane and the right side of the vehicle traveling in the center lane are included in the image, and, while the leaned vehicle is traveling in the left side area of ​​the center lane of the five-lane road, the right edge of the rear of the vehicle traveling in the rightmost lane is not included in the image, but at least the rear of the vehicle traveling in the center lane and the left side of the vehicle traveling in the center lane are included in the image. The arrival time estimation device is configured to estimate the arrival time of the vehicle traveling in the center lane to reach the lean vehicle based on the image that includes at least the rear of the vehicle traveling in the center lane and the right side of the vehicle traveling in the center lane while the lean vehicle is traveling in the right side area of ​​the center lane of the five-lane road, and to estimate the arrival time of the vehicle traveling in the center lane to reach the lean vehicle based on the image that includes at least the rear of the vehicle traveling in the center lane and the left side of the vehicle traveling in the center lane while the lean vehicle is traveling in the left side area of ​​the center lane of the five-lane road.

[0013] The width of a lean vehicle is very small compared to the width of an automobile. In other words, the body of a lean vehicle is slimmer than that of an automobile. The width of a lean vehicle is very small compared to the width of a lane. Therefore, when a lean vehicle and an automobile are traveling in the same lane, the lean vehicle may travel in the right side area or the left side area of ​​the lane. When a lean-to vehicle is traveling in the right side area of ​​the center lane of a five-lane road, the rear and right side of a car traveling in the center lane may be located in front of the lean-to vehicle.When a lean-to vehicle is traveling in the left side area of ​​the center lane of a five-lane road, the rear and left side of a car traveling in the center lane may be located in front of the lean-to vehicle. The imaging device includes a monocular camera. The monocular camera of the imaging device has a horizontal angle of view configured so that, when a leaned vehicle is traveling in the right side area of ​​the center lane of a five-lane road, at least the rear and right sides of the vehicle traveling in the center lane are included in the image. The monocular camera of the imaging device has a horizontal angle of view configured so that, when a leaned vehicle is traveling in the left side area of ​​the center lane of a five-lane road, at least the rear and left sides of the vehicle traveling in the center lane are included in the image. The monocular camera of the imaging device is configured with a horizontal angle of view so that when a leaned vehicle is traveling in the right side area of ​​the center lane of a five-lane road, the left edge of the rear of a vehicle traveling in the leftmost lane is not included in the image.The monocular camera of the imaging device is configured with a horizontal angle of view so that when a leaned vehicle is traveling in the left side area of ​​the center lane of a five-lane road, the right edge of the rear of a vehicle traveling in the rightmost lane is not included in the image. When a lean-to vehicle is traveling in the right-hand side area of ​​the center lane of a five-lane road, a car traveling in the left-hand lane is located more than two car lengths to the left of the lean-to vehicle.When a lean-to vehicle is traveling in the left-hand side area of ​​the center lane of a five-lane road, a car traveling in the right-hand lane is located more than two car lengths to the right of the lean-to vehicle. The width of a lean vehicle is very small compared to the width of an automobile. In other words, a lean vehicle has a slimmer body compared to an automobile. Therefore, a vehicle located more than two automobile lengths away from the lean vehicle in the left-right direction has less impact on the driving of the lean vehicle, which has a slimmer body compared to an automobile, compared to the two vehicles located in front of the lean vehicle. With the above configuration, whether the lean vehicle is traveling in the right side area or the left side area, it is possible to limit the imaging of areas located more than two car lengths away from the lean vehicle in the left-right direction. As a result, it is possible to increase the resolution of the images of the two cars located in front of the lean vehicle while reducing the number of pixels in the entire image. In other words, it is possible to increase the resolution of the images of the two cars located in front of the lean vehicle without increasing the size of the imaging device. Therefore, it is possible to downsize the imaging element and thereby reduce the size of the imaging device while ensuring high resolution of the images of the two cars located in front of the lean vehicle. The monocular camera of the imaging device has a horizontal angle of view configured so that, when a leaned vehicle is traveling in the right side area of ​​the center lane of a five-lane road, at least the rear of the vehicle traveling in the center lane and the right side of the vehicle traveling in the center lane are included in the image.The monocular camera of the imaging device has a horizontal angle of view configured so that, when a leaned vehicle is traveling in the left side area of ​​the center lane of a five-lane road, at least the rear of the vehicle traveling in the center lane and the left side of the vehicle traveling in the center lane are included in the image. The arrival time estimation device estimates the arrival time for a vehicle traveling in the center lane to reach the lean vehicle based on an image that includes at least the rear of the vehicle traveling in the center lane and the right side of the vehicle traveling in the center lane while the lean vehicle is traveling in the right side area of ​​the center lane.The arrival time estimation device estimates the arrival time for a vehicle traveling in the center lane to reach the lean vehicle based on an image that includes at least the rear of the vehicle traveling in the center lane and the left side of the vehicle traveling in the center lane while the lean vehicle is traveling in the left side area of ​​the center lane. Since the arrival time is estimated based on an image that includes at least the rear of the vehicle traveling in the center lane as well as the left or right side of the vehicle traveling in the center lane, it is possible to estimate the arrival time in usage scenarios specific to lean vehicles, such as when the vehicle is traveling in the right side area of ​​the lane and when the vehicle is traveling in the left side area. In this case, since the image is based on an image that includes at least the rear of the vehicle traveling in the center lane as well as the left or right side of the vehicle traveling in the center lane, the amount of information about the vehicle traveling in the center lane is large. This increases the degree of freedom in designing the algorithm for estimating the arrival time, thereby enabling the hardware of the arrival time estimation device to be miniaturized. This also improves the mountability of the image processing device for lean vehicles, which have slimmer bodies than automobiles. For example, in an algorithm that estimates arrival time from the size of the rear of a car in an image (left-right width, up-down width, diagonal width), the size of the rear of the car can be easily recognized by using an image that includes both the rear and side of the car. For example, in an algorithm that estimates arrival time from the distance between multiple feature points of a car in an image, using an image that includes the rear and side of the car increases the freedom of feature point selection. For example, in an algorithm that estimates arrival time using a deep learning algorithm to estimate the rate of change of the size of a car in an image from image information, the amount of information can be increased by using images that include the rear and sides of the car, thereby improving the estimation accuracy. In this way, the arrival time is estimated based on an image that includes at least the rear of the vehicle traveling in the center lane, as well as the left or right side of the vehicle traveling in the center lane, which increases the design freedom of the arrival time estimation algorithm. As a result of the above, it is possible to provide an image processing device for lean vehicles that can adapt to situations where a lean vehicle, which has a slimmer body than an automobile, is traveling in the right side area and left side area of ​​the lane while traveling in the same lane as an automobile, and that is highly mountable on lean vehicles, which have a slimmer body than an automobile.

[0014] (2) The image processing device for a lean vehicle according to one embodiment of the present invention may have the following configuration in addition to the configuration described in (1) above. The image processing device for lean vehicles further includes an alarm device that outputs an alarm based on the arrival time of an object to reach the lean vehicle estimated by the arrival time estimation device, and the alarm device is configured not to output the alarm when two automobiles are traveling straight ahead of the lean vehicle side by side with a gap between them in the left and right direction, and the size of the gap is too small for the width of the automobiles but large enough for the width of the lean vehicle, even if the lean vehicle travels straight towards the center between the two automobiles and approaches them; and is configured to output the alarm when two automobiles are traveling straight ahead of the lean vehicle side by side with a gap between them in the left and right direction, and the size of the gap is too small for the width of the lean vehicle, even if the lean vehicle travels straight towards the center between the two automobiles and approaches them, and outputs the alarm according to the width of the lean vehicle.

[0015] According to this configuration, the image processing device for a lean vehicle further includes an alarm device that outputs an alarm based on the arrival time of the object to reach the lean vehicle, estimated by the arrival time estimation device. The warning device is configured not to output a warning when two automobiles are traveling straight ahead of a lean-to vehicle, side by side with a gap between them in the left-right direction, and the gap is large enough to fit the width of the automobiles but large enough to fit the width of the lean-to vehicle, even if the lean-to vehicle moves straight towards the center between the two automobiles and approaches the two automobiles.The warning device is configured to output a warning when two automobiles are traveling straight ahead of a lean-to vehicle, side by side with a gap between them in the left-right direction, and the gap is large enough to fit the width of the lean-to vehicle, even if the lean-to vehicle moves straight towards the center between the two automobiles and approaches the two automobiles.In this way, the warning device outputs a warning according to the width of the lean-to vehicle. For example, when a leaning vehicle is traveling in the right side area of ​​the center lane of a five-lane road, two vehicles traveling in the center lane and the center-right lane will be located in front of the leaning vehicle. For example, when a leaning vehicle is traveling in the right side area of ​​the center lane, a portion of the vehicle traveling in the center lane is located on the left side area of ​​the center lane and a portion of the vehicle traveling in the center-right lane is located on the right side area of ​​the center-right lane, and a portion of the vehicle traveling in the center lane is located on the right side area of ​​the center-right lane. For example, when a leaning vehicle is traveling in the left side area of ​​the center lane of a five-lane road, two vehicles traveling in the center lane and the center-left lane will be located in front of the leaning vehicle. For example, when a leaning vehicle is traveling in the left side area of ​​the center lane, a portion of the vehicle traveling in the center lane is located on the right side area of ​​the center lane and a portion of the vehicle traveling in the center-left lane is located on the left side area of ​​the center-left lane, and a portion of the vehicle traveling in the center lane is located on the left side area of ​​the center-left lane, and the presence or absence of an alarm can be set to differ depending on the situation. With the above configuration, the image processing device for lean-in vehicles can capture and recognize two automobiles located in front of the lean-in vehicle, whether the lean-in vehicle is traveling in the right side area or the left side area, and can output a warning according to the width of the lean-in vehicle. Therefore, when a lean-in vehicle, which has a slimmer body than an automobile, is traveling in the same lane as an automobile, the image capture, recognition, and warning can be adapted to scenes where the lean-in vehicle is traveling in the right side area or the left side area of ​​the lane. As a result of the above, it is possible to provide an image processing device for lean vehicles that is more adaptable to situations where a lean vehicle, which has a slimmer body than an automobile, is traveling in the right side area and left side area of ​​the lane while traveling in the same lane as an automobile, and that is highly mountable on lean vehicles, which have a slimmer body than an automobile. The image processing device for lean vehicles may be an image processing device dedicated to lean vehicles in which the warning device is configured in advance according to the vehicle width of the lean vehicle.The image processing device for lean vehicles may be an image processing device for both lean vehicles and automobiles in which the warning device is configured to adapt to the vehicle width of the lean vehicle or the vehicle width of an automobile by switching.The switching may be performed by a user operation or automatically based on a captured image.

[0016] (3) The image processing device for a lean vehicle according to one embodiment of the present invention may have the following configuration in addition to the configuration (1) or (2) above. The imaging device includes the monocular camera that captures a plurality of images including the left side or right side and rear of the automobile at regular time intervals when the automobile is traveling ahead of the lean vehicle, and the arrival time estimation device acquires the plurality of consecutive images including the left side or right side and rear of the automobile captured by the imaging device at the regular time intervals, and estimates the arrival time until the automobile reaches the lean vehicle based on the plurality of consecutive images including the left side or right side and rear of the automobile and the regular time intervals. do.

[0017] For example, when a leaning vehicle is traveling in the right side area of ​​the center lane of a five-lane road, the imaging device captures an image including the rear and right side of the vehicle traveling in the center lane.For example, when a leaning vehicle is traveling in the left side area of ​​the center lane of a five-lane road, the imaging device captures an image including the rear and left side of the vehicle traveling in the center lane. According to the above configuration, in such a scene, the imaging device captures images including the left or right side and rear of the vehicle traveling in the center lane at regular time intervals, so the rate of change in the size of the vehicle in multiple images including the left or right side and rear of the vehicle is the rate of change in the size of the vehicle in images taken at regular time intervals. Therefore, compared to when the imaging device captures images including the left or right side and rear of the vehicle at irregular time intervals, the rate of change in the size of the vehicle in multiple images including the left or right side and rear of the vehicle can be more easily used to estimate the arrival time. This reduces the processing load of the arrival time estimation device, and for example, the heat dissipation structure of the arrival time estimation device can be made simple and compact, thereby making it possible to make the arrival time estimation device compact. Therefore, it is possible to improve the mountability of the lean vehicle image processing device in a lean vehicle, which has a slimmer body than an automobile. Furthermore, since the arrival time is estimated based on multiple images including the left or right side and rear of the vehicle, and a fixed time interval at which the multiple images are taken, it can be matched to estimating the arrival time in situations where the vehicle is traveling on the right side area of ​​the lane and in usage situations specific to lean vehicles traveling on the left side area.

[0018] (4) The image processing device for a lean vehicle according to one embodiment of the present invention may have the following configuration in addition to any one of the configurations (1) to (3) above. The imaging device is The monocular camera captures a plurality of color images including a left side or right side and a rear of the automobile when the automobile is traveling ahead of the lean vehicle, and the arrival time estimation device acquires the plurality of color images including the left side or right side and the rear of the automobile captured by the imaging device, and estimates the arrival time until the automobile reaches the lean vehicle based on the plurality of color images including the left side or right side and the rear of the automobile that have been acquired. do.

[0019] With this configuration, for example, when a leaning vehicle is traveling in the right side area of ​​the center lane of a five-lane road, the imaging device captures a color image including the rear and right side of the vehicle traveling in the center lane. For example, when a leaning vehicle is traveling in the left side area of ​​the center lane of a five-lane road, the imaging device captures a color image including the rear and left side of the vehicle traveling in the center lane. The arrival time estimation device uses a color image that includes at least the rear of the vehicle traveling in the center lane, as well as the left or right side of the vehicle traveling in the center lane, to estimate the arrival time, thereby increasing the amount of information about the vehicle traveling in the center lane. This increases the degree of freedom in designing the arrival time estimation algorithm. For example, in an algorithm that estimates arrival time from the size of the rear of a car in an image (left-right width, up-down width, diagonal width), the size of the rear of the car can be easily recognized by using a color image that includes the rear and sides of the car. For example, in an algorithm that estimates arrival time from the distance between multiple feature points of a car in an image, using a color image that includes the rear and sides of the car increases the freedom of feature point selection. For example, in an algorithm that estimates arrival time using a deep learning algorithm to estimate the rate of change of the size of a car in an image from image information, the amount of information can be increased by using color images that include the rear and sides of the car, thereby improving the estimation accuracy. In this way, in usage scenarios specific to lean-to vehicles traveling in the right side area of ​​a lane and the left side area of ​​a lane, the arrival time is estimated based on a color image that includes at least the rear of the vehicle traveling in the center lane as well as the left or right side of the vehicle traveling in the center lane, thereby increasing the design flexibility of the arrival time estimation algorithm. This allows the hardware of the arrival time estimation device to be miniaturized. Therefore, the installability of the image processing device for lean-to vehicles can be further improved in lean-to vehicles, which have slimmer bodies than automobiles.

[0020] (5) The image processing device for a lean vehicle according to one embodiment of the present invention may have the following configuration in addition to any one of the configurations (1) to (4) above. The arrival time estimation device acquires a plurality of consecutive images including the left side or right side and rear of the automobile captured by the imaging device when the automobile is traveling in front of the lean vehicle, and (I) uses an algorithm that acquires the size of the rear of the automobile in the images based on the plurality of consecutive images including the left side or right side and rear of the automobile and estimates the arrival time until the automobile reaches the lean vehicle, (II) uses an algorithm that acquires the distance between a plurality of feature points on the left side or right side and rear of the automobile in the images based on the plurality of consecutive images including the left side or right side and rear of the automobile and estimates the arrival time until the automobile reaches the lean vehicle, or (III) uses a deep-learned algorithm that estimates the rate of change in size of the automobile having the left side or right side and rear in the images based on the plurality of consecutive images including the left side or right side and rear of the automobile to estimate the arrival time until the automobile reaches the lean vehicle. do.

[0021] For example, when a leaning vehicle is traveling in the right side area of ​​the center lane of a five-lane road, the imaging device captures an image including the rear and right side of the vehicle traveling in the center lane.For example, when a leaning vehicle is traveling in the left side area of ​​the center lane of a five-lane road, the imaging device captures an image including the rear and left side of the vehicle traveling in the center lane. According to the above configuration, in such a scene, the algorithm (I) for estimating the arrival time from the size (left-right width, up-down width, diagonal width) of the rear of the vehicle in the image can easily recognize the size of the rear of the vehicle by using an image that includes the rear and side of the vehicle traveling in the center lane. Furthermore, in the algorithm (II) above, which estimates the arrival time from the distance between multiple feature points of a vehicle in an image, the degree of freedom in selecting feature points can be increased by using an image that includes the rear and side of a vehicle traveling in the center lane. Furthermore, in the algorithm (III) described above, which estimates arrival time using a deep learning algorithm to estimate the rate of change in the size of a vehicle in an image from image information, the amount of information can be increased by using images that include the rear and sides of a vehicle traveling in the center lane, thereby improving the estimation accuracy. In this manner, in the present configuration, in the scenarios specific to lean-to vehicles traveling in the right side area of ​​a lane and the left side area of ​​a lane, the arrival time is estimated based on an image that includes at least the rear of the vehicle traveling in the center lane as well as the left or right side of the vehicle traveling in the center lane, thereby increasing the design flexibility of the arrival time estimation algorithm. Note that, since the image includes at least the rear of the vehicle traveling in the center lane as well as the left or right side of the vehicle traveling in the center lane, the amount of information about the vehicle traveling in the center lane is large. This further increases the design flexibility of the arrival time estimation algorithm, thereby enabling the hardware of the arrival time estimation device to be miniaturized. This further improves the mountability of lean-to vehicles, whose bodies are slimmer than automobiles.

[0022] In the present invention and its embodiments, a lean vehicle may be a vehicle having a body frame that leans left when turning left and leans right when turning right. In the present invention and its embodiments, a lean vehicle may be, for example, a motorcycle or a three-wheeled motor vehicle. Motorcycles include scooters, mopeds, and motorized bicycles. The width of a lean vehicle is, for example, less than 1 meter. On the other hand, the width of a car is, for example, 2 to 2.5 meters.

[0023] In the present invention and embodiments, the monocular camera has a single imaging element. An imaging device including a monocular camera has an optical device such as a lens and a single imaging element. An imaging device having a monocular camera may also be an imaging device having a single lens. The imaging element may be an image sensor such as a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor. In the present invention and embodiments, the imaging device may be disposed in the center of the vehicle in the left-right direction. However, the location of the imaging device is not limited to this.

[0024] In the present invention and embodiments, the arrival time estimation device includes a processor and a memory. The memory includes, for example, a random access memory (RAM). The memory may include a read-only memory (ROM). The processor includes any circuit capable of processing images, such as a field-programmable gate array (FPGA), a graphics processing unit (GPU), a central processing unit (CPU), or a system on chip (SoC). An SoC has a configuration in which the processor and memory are integrated.

[0025] In the present invention and its embodiments, a lane means a driving lane. A lane may or may not have marks (e.g., white lines) indicating the edges of the lane. The width of a lane is, for example, 3.5 to 4 m. In the present invention and the embodiments, the right side area, the left side area, and the center area correspond to three areas obtained by dividing a lane into three equal parts in the left and right direction. In the present invention and embodiments, the five-lane road is a straight road.

[0026] In the present invention and embodiments, a lean vehicle traveling in the right side area of ​​a lane means that the lean vehicle travels so that the left-right center of the lean vehicle is located within the right side area. A lean vehicle traveling in the left side area of ​​a lane means that the lean vehicle travels so that the left-right center of the lean vehicle is located within the left side area. In the present invention, a lean vehicle traveling in the right side area (left side area) of a lane includes traveling so that the entire lean vehicle is located within the right side area (left side area) in a plan view. In the present invention, a lean vehicle traveling in the right side area (left side area) of a lane may also include traveling so that part of the lean vehicle is located outside the right side area (left side area).

[0027] In the present invention and embodiments, the configurations of the imaging device and the arrival time estimation device are defined using a situation in which the longitudinal distance between the five automobiles and the lean vehicle is the distance required to capture an image so that the TTC is 3 seconds, assuming that the time from image capture to completion of image processing is 0.5 seconds and the object approaches the lean vehicle at a relative speed of 15 km / h. "The distance required to capture an image so that the TTC is 3 seconds, assuming that the time from image capture to completion of image processing is 0.5 seconds and the object approaches the lean vehicle at a relative speed of 15 km / h" can be rephrased as follows: That is, this distance is the distance required to capture an image so that the TTC is 3 seconds, assuming that the lean vehicle image processing device is configured to perform image processing so that the time from image capture to completion of image processing is 0.5 seconds, and that the object approaches the lean vehicle at a relative speed of 15 km / h. The TTC is the time it takes for the object to reach the lean vehicle. Specifically, this distance is 14.58335 m (= 15 km / h × (3.0 s + 0.5 s) ÷ 3600 × 1000). "An object approaches a lean vehicle at a relative speed of 15 km / h" means that the object and lean vehicle are both moving and the lean vehicle is moving at a speed 15 km / h faster than the object, or the object is stationary and the lean vehicle is moving and the lean vehicle is moving at a speed of 15 km / h. The "time from capturing an image to completing image processing" is an assumed time and does not have to be the same as the time required for the image processing that is actually performed by the lean vehicle image processing device. The "time from capturing an image to completing image processing" may be, for example, the time from capturing an image with an imaging device to processing the image to estimating the time of arrival. Generally, it takes about 0.5 seconds from capturing an image to estimating the TTC. The "time from capturing an image to completing image processing" may be, for example, the time from when an image capturing device captures an image to when a command to output an alarm to an alarm device (described later) is output.

[0028] In the present invention and embodiments, "the rear of the vehicle is included in the image" in the configuration in which "when a leaning vehicle is traveling in the right side area of ​​the center lane of a five-lane road, the horizontal angle of view is configured so that an image of at least the rear of the vehicle traveling in the center lane and the right side of the vehicle traveling in the center lane is included" means that the entire rear of the vehicle is included in the image. Also, "the right side of the vehicle is included in the image" means that, if the right side of the vehicle is flat, the entire right side of the vehicle is included in the image. In other words, "the right side of the vehicle is included in the image" means that at least a portion of the right side of the vehicle is included in the image. In the present invention and embodiments, included in an image means appearing in the image. Using a similar interpretation to these definitions, the meaning of "the left side of the vehicle is included in the image" in the configuration "when a leaned vehicle is traveling in the left side area of ​​the center lane of a five-lane road, the horizontal angle of view is configured so that at least the rear of the vehicle traveling in the center lane and the left side of the vehicle traveling in the center lane are included in the image" can also be interpreted.

[0029] In the present invention and embodiments, a vehicle traveling ahead of a lean vehicle refers to a vehicle located ahead of the lean vehicle when the forward direction of the lean vehicle is defined as the forward direction. In the present invention and embodiments, a vehicle traveling ahead of a lean vehicle refers to a vehicle traveling directly or almost directly in front of the lean vehicle. A vehicle traveling ahead of a lean vehicle is not limited to a vehicle traveling ahead of the lean vehicle.

[0030] The image processing device for a lean vehicle of the present invention and embodiments may or may not have an alarm device that outputs an alarm based on the arrival time of an object to reach the lean vehicle, estimated by the arrival time estimation device. In the present invention and embodiments, if the lean vehicle image processing device has a warning device, the use of the arrival time estimated by the arrival time estimation device is not limited to the warning device. Regardless of whether the lean vehicle image processing device has a warning device, the arrival time estimated by the arrival time estimation device may be used, for example, to control the vehicle speed of the lean vehicle. More specifically, for example, the brake device of the lean vehicle may be activated based on the arrival time estimated by the arrival time estimation device without the driver's brake operation.

[0031] In the present invention and embodiments, the alarm device that outputs an alarm based on the arrival time estimated by the arrival time estimation device for an object to reach a lean vehicle may output an alarm urging the driver to be careful of automobiles, or may output an alarm urging the driver to be careful of objects other than automobiles. In the present invention and its embodiments, the alarm output by the alarm device is an alarm to alert the driver of the lean vehicle. The alarm is an alarm that alerts at least one of the driver's hearing, sight, or touch. An auditory alarm is an alarm that uses sound. A visual alarm is, for example, an alarm that uses light. A tactile alarm is, for example, an alarm that uses vibration. The vibration may be, for example, a vibration of the grip or a vibration of the seat. The means for generating the vibration may be a dedicated means of the alarm device, or may be a brake or a drive source.

[0032] In the present invention and embodiments, "two automobiles are traveling straight ahead of a lean vehicle, side by side with a gap in the left-right direction, and the size of the gap is such that there is no room for the width of the lean vehicle" means that the size of the gap is smaller than or equal to the width of the lean vehicle. In other words, it means that the size of the gap is such that the lean vehicle cannot pass through the gap under normal traffic conditions. By interpreting it in a similar way to this definition, it is also possible to mean "two automobiles are traveling straight ahead of a lean vehicle, side by side with a gap in the left-right direction, and the size of the gap is such that there is no room for the width of the automobiles, but there is enough room for the width of the lean vehicle."

[0033] In the present invention or embodiments, a monocular camera that captures multiple images at regular time intervals may be able to change the regular time interval at which images are captured. That is, a monocular camera that captures multiple images at regular time intervals may be configured to be able to switch between capturing multiple images at a first regular time interval and capturing multiple images at a second regular time interval that is different from the first time interval. For example, the monocular camera may be configured to be able to switch between capturing multiple images at 30 frames per second (FPS) and capturing multiple images at 60 FPS.

[0034] In the present invention or embodiments, a color image is an image in which each pixel of the image contains information of multiple colors (e.g., red, green, and blue). A color image does not include a monochromatic image that contains, for example, only red information.

[0035] In the present invention or embodiments, (I) the algorithm for acquiring the size of the rear surface of the vehicle based on a plurality of consecutive images including the left or right side and rear surface of the vehicle and estimating the arrival time until the vehicle reaches the lean vehicle may be an algorithm such as the following: That is, the algorithm may acquire the size of the rear surface of the vehicle in a plurality of consecutive images, derive a rate of change in the size of the rear surface of the vehicle, and estimate the arrival time based on the derived rate of change. In the present invention or embodiments, (II) an algorithm for estimating a time it takes for a vehicle to reach a lean-to vehicle by acquiring distances between multiple feature points on the left or right side and rear of a vehicle based on multiple consecutive images including the left or right side and rear of the vehicle may be an algorithm such as the following: For each of the consecutive images, the algorithm acquires the positions of multiple feature points on the left or right side and rear of the vehicle, derives distances between the multiple feature points in each image based on the acquired positions of the feature points, and estimates the time it takes for a vehicle traveling in the center lane to reach a lean-to vehicle based on the derived distances. The multiple feature points on the left or right side and rear of the vehicle may include at least one feature point on the left or right side of the vehicle. The multiple feature points on the left or right side and rear of the vehicle may include at least one feature point on the rear of the vehicle. The multiple feature points on the left or right side and rear of the vehicle may include at least one feature point on a boundary line between the left or right side and rear of the vehicle. When the number of feature points on the left or right side and rear of the vehicle is two, the distance between the feature points on the left or right side and rear of the vehicle is the distance between these two feature points. When the number of feature points on the left or right side and rear of the vehicle is three or more, the distance between the feature points on the left or right side and rear of the vehicle may be the distance between any two of these three or more feature points. Furthermore, when the number of feature points on the left or right side and rear of the vehicle is three or more, the distance between the two feature points may be obtained for each of two or more pairs of two feature points that are different combinations among these three or more feature points. In the present invention or embodiments, (III) an algorithm for estimating the arrival time until an automobile reaches a lean vehicle using a deep-learned algorithm to estimate the rate of change in size of an automobile having the left side or right side and rear, based on a plurality of consecutive images including the left side or right side and rear of the automobile, may be an algorithm such as the following: That is, the deep-learned algorithm may be an algorithm that is deep-learned based on data of a large number of images captured by an imaging device, includes a trained model that takes a plurality of consecutive images captured by the imaging device as input and outputs the rate of change in size of the automobile in the images, and estimates the arrival time based on the rate of change in size of the automobile output from the trained model.

[0036] The situation in which the lean-in vehicle image processing device of the present invention is used is not limited to a situation in which a lean-in vehicle is traveling in the right or left side area of ​​the center lane of a five-lane road. The lean-in vehicle image processing device may also be used in the following driving situations, for example. The driving situation may be, for example, a situation in which a lean-in vehicle is traveling on a road other than a five-lane road. A road other than a five-lane road may be a road with six or more lanes, a two-lane road, or a road with one lane in each direction. The driving situation may also be, for example, a situation in which a lean-in vehicle is traveling in the center area of ​​a certain lane. The driving situation may also be, for example, a situation in which a lean-in vehicle is traveling in the right or left side area of ​​a lane other than the center lane of a five-lane road.

[0037] In the present invention and embodiments, "at least one (one) of a plurality of options" includes all possible combinations of the plurality of options. "At least one (one) of a plurality of options" may be any one of the plurality of options, or may be all of the plurality of options. For example, "at least one of A, B, and C" may be only A, only B, only C, A and B, A and C, B and C, or A, B, and C.

[0038] In this specification, for example, "1 to 10" means 1 or more and 10 or less. The same definition applies to numbers other than 1 and 10.

[0039] In the present invention and embodiments, the words including, comprising, having, and their derivatives are used herein to encompass the listed items and equivalents thereof as well as additional items.

[0040] Unless otherwise defined, all terms (including technical and scientific terms) used in the present specification and claims have the same meaning as commonly understood by those skilled in the art to which this invention belongs. Terms, such as those defined in commonly used dictionaries, should be interpreted to have a meaning consistent with the meaning in the context of the relevant technology and this disclosure, and should not be interpreted in an idealized or overly formal sense.

[0041] In this specification, the term "may" is non-exclusive. "may" means "may, but is not limited to." In this specification, "may" implicitly includes the possibility that "may not." In this specification, a configuration described as "may" at least achieves the above-mentioned effect obtained by the configuration of claim 1.

[0042] Before describing embodiments of the present invention in detail, it is to be understood that the invention is not limited to the details of construction and the arrangement of components set forth in the following description or illustrated in the drawings. The present invention is capable of embodiments other than those described below. The present invention is also capable of embodiments incorporating various variations of the embodiments described below. [Effects of the Invention]

[0043] According to the present invention, an image processing device for a lean vehicle can be provided that can adapt to situations in which a lean vehicle, which has a slimmer body than an automobile, is traveling in the right side area and left side area of ​​a lane while traveling in the same lane, and that is highly mountable on a lean vehicle, which has a slimmer body than an automobile. [Brief explanation of the drawings]

[0044] [Figure 1] 1 is a diagram for explaining the configuration of an image processing device for a lean vehicle according to a first embodiment; [Figure 2] FIG. 10 is a diagram for explaining the configuration of an image processing device for a lean vehicle according to a second embodiment. [Figure 3] FIG. 10 is a diagram for explaining the configuration of an image processing device for a lean vehicle according to a third embodiment. [Figure 4] FIG. 10 is a diagram for explaining the configuration of an image processing device for a lean vehicle according to a fourth embodiment. [Figure 5] FIG. 10 is a diagram for explaining the configuration of an image processing device for a lean vehicle according to a fifth embodiment. [Figure 6] FIG. 10 is a diagram for explaining the configuration of an image processing device for a lean vehicle according to a sixth embodiment. [Figure 7] FIG. 13 is a diagram for explaining the configuration of an image processing device for a lean vehicle according to a seventh embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0045] Arrows F, Re, L, and R shown in the drawings represent the forward, backward, leftward, and rightward directions, respectively.

[0046] [First embodiment] An image processing device 10 for a lean vehicle according to a first embodiment of the present invention will be described below with reference to Fig. 1. The image processing device 10 for a lean vehicle is mounted on a lean vehicle 1. In Fig. 1, the lean vehicle 1 is a motorcycle, but the lean vehicle 1 is not limited to motorcycles. The image processing device 10 for a lean vehicle includes an imaging device 11 that captures images, and an arrival time estimation device 12 that processes the images captured by the imaging device 11 and estimates the arrival time of an object to reach the lean vehicle 1. The imaging device 11 includes a monocular camera.

[0047] FIG. 1 includes plan views showing two examples (CASE 1 and CASE 2) of a situation in which a lean-to vehicle 1 and five automobiles 61 to 65 are traveling on a five-lane road. The five-lane road includes a left-most lane 51, a center-left lane 52, a center lane 53, a center-right lane 54, and a right-most lane 55. In CASE 1 and CASE 2, the lean-to vehicle 1 travels in the center lane 53, and the five automobiles 61 to 65 travel in a line ahead of the lean-to vehicle 1 in the left-right direction. In CASE 1, the lean-to vehicle 1 travels in a right-side area 50R of the center lane 53, and in CASE 2, the lean-to vehicle 1 travels in a left-side area 50L of the center lane 53. The right side area 50R is an area including the right edge of a single lane 50 (e.g., center lane 53) when the single lane 50 is divided into thirds in the left-right direction, and the left side area 50L is an area including the left edge of the single lane 50. The area between the right side area 50R and the left side area 50L is a center area 50C. The width of the lean-to vehicle 1 is very small compared to the width of an automobile 60 (e.g., automobiles 61-65). The width of the lean-to vehicle 1 is very small compared to the width of the lane 50. Therefore, the lean-to vehicle 1 has a high degree of freedom in its traveling position in the left-right direction within the center lane 53, and can move left and right within the center lane 53, and can travel in either the right side area 50R or the left side area 50L within the center lane 53.

[0048] In Case 1 and Case 2, the distance D in the longitudinal direction between the five automobiles 61-65 and the lean-to vehicle 1 is the distance required to capture an image so that image processing is completed at a time to capture (TTC) of 3 seconds, assuming that the time from image capture to completion of image processing (Ti) is 0.5 seconds and the object approaches the lean-to vehicle 1 at a relative speed (Vr) of 15 km / h. This distance is approximately 15 m. Note that, hereinafter, this distance may be referred to as the "specific distance."

[0049] When a lean vehicle 1 is traveling in the right side area 50R or left side area 50L of the center lane 53 of a five-lane road, and five automobiles 61 to 65 are traveling in a line in the left-right direction ahead of the lean vehicle 1, and the longitudinal distance D between the five automobiles 61 to 65 and the lean vehicle 1 is the specific distance described above, the imaging device 11 and the arrival time estimation device 12 have the following configuration.

[0050] The monocular camera of the imaging device 11 is configured such that, while the leaning vehicle 1 is traveling in the right side area 50R of the center lane 53, the left edge 61BL of the rear of the automobile 61 traveling in the leftmost lane 51 is not included in the captured image PR, but the angle of view in the left-right direction is configured so that at least the rear 63B of the automobile 63 traveling in the center lane 53 and the right side 63R of the automobile 63 traveling in the center lane 53 are included in the image PR. The monocular camera of the imaging device 11 is configured such that, while the leaning vehicle 1 is traveling in the left side area 50L of the center lane 53, the right edge 65BR of the rear of the automobile 65 traveling in the rightmost lane 55 is not included in the captured image PL, but the angle of view in the left-right direction is configured so that the rear 63B of the automobile 63 traveling in the center lane 53 and the left side 63L of the automobile 63 traveling in the center lane 53 are included in the image PL.

[0051] While the lean vehicle 1 is traveling in the right side area 50R of the center lane 53, the arrival time estimation device 12 estimates the arrival time Ta required for the vehicle 63 traveling in the center lane 53 to reach the lean vehicle 1 based on an image PR that includes at least the rear 63B of the vehicle 63 traveling in the center lane 53 and the right side 63R of the vehicle 63 traveling in the center lane 53. At this time, the arrival time estimation device 12 estimates the arrival time Ta based at least on the image PR captured by the imaging device 11 when the distance D is the above-mentioned specific distance. While the lean vehicle 1 is traveling in the left side area 50L of the center lane 53, the arrival time estimation device 12 estimates the arrival time Ta required for the automobile 63 traveling in the center lane 53 to reach the lean vehicle 1 based on an image PL that includes at least the rear 63B of the automobile 63 traveling in the center lane 53 and the left side 63L of the automobile 63 traveling in the center lane 53. At this time, the arrival time estimation device 12 estimates the arrival time Ta based at least on the image PL captured by the imaging device 11 when the distance D is the above-mentioned specific distance.

[0052] 1 is a schematic representation of the minimum horizontal angle of view of the monocular camera of the imaging device 11 for conceptual understanding. Angle θmax in Fig. 1 is a schematic representation of the maximum horizontal angle of view of the monocular camera of the imaging device 11 for conceptual understanding.

[0053] In FIG. 1, the symbols "◯", "△", and "×" attached to the automobiles 61 to 65 have the following meanings: The symbol "◯" indicates that the entire rear of the automobile is captured. The symbol "×" indicates that the rear of the automobile is not captured at all, or only a portion of the rear of the automobile is captured. The symbol "△" indicates that the rear of the automobile is not captured at all, only a portion of the rear of automobile 60 is captured, or the entire rear of the automobile is captured.

[0054] The lean vehicle 1 or the lean vehicle image processing device 10 may or may not have a display device that displays an image captured by the imaging device 11. When an image is displayed, for example, an object (e.g., a car) recognized in the image may be surrounded by a frame, as shown in the two lower diagrams of FIG.

[0055] According to the first embodiment, an image processing device 10 for a lean vehicle can be realized that can adapt to scenes in which a lean vehicle 1, which has a slimmer body than an automobile 60, is traveling in the right side area 50R and left side area 50L of the lane 50 while traveling in the same lane 50 as the automobile 60, and that is highly mountable on a lean vehicle 1, which has a slimmer body than an automobile 60.

[0056] [Second embodiment] Next, an image processing device 10 for a lean vehicle according to a second embodiment of the present invention will be described with reference to Fig. 2. The second embodiment has all of the configurations of the first embodiment.

[0057] 2, the lean vehicle image processing device 10 further includes a warning device 13 that outputs a warning to urge the driver to pay attention to the vehicle 60, based on the arrival time Ta required for the vehicle 60 to reach the lean vehicle 1, estimated by the arrival time estimation device 12. The warning device 13 outputs a warning according to the vehicle width of the lean vehicle 1, as will be described below.

[0058] As shown in CASE 3 of Figure 2, when two automobiles 60 are traveling straight ahead of the lean vehicle 1, side by side with a gap G in the left-right direction, and the size of the gap G is such that there is no room left-right for the width of the automobiles 60, but there is room left-right for the width of the lean vehicle 1, the warning device 13 is configured not to output an alarm even if the lean vehicle 1 travels straight towards the center between the two automobiles 60 and approaches them. As shown in CASE 4 of Fig. 2, when two automobiles 60 are traveling straight ahead of the lean vehicle 1 side by side with a gap G in the left-right direction, and the size of the gap G is too large for the width of the lean vehicle 1 in the left-right direction, the warning device 13 is configured to output a warning when the lean vehicle 1 travels straight towards and approaches the center between the two automobiles 60. Note that in CASE 4 of Fig. 2, the gap G is slightly larger than the width of the lean vehicle 1, but it may also be smaller than the width of the lean vehicle 1.

[0059] As shown in the comparative example of Figure 2, two automobiles 60 are traveling straight ahead of an automobile 66 (60) equipped with an alarm device, side by side with a gap G in the left-right direction, and the size of the gap G is too small for the width of the automobiles 66 in the left-right direction, but is large enough for the width of the lean vehicle 1 in the left-right direction.When the automobile 66 travels straight toward the center between the two automobiles 60 and approaches them, the alarm device of the automobile 66 will output an alarm. Although not shown in the figure, two automobiles 60 are traveling straight ahead of an automobile 66 equipped with an alarm device, side by side with a gap G between them in the left and right direction, and if the size of the gap G is such that there is no room left and right for both the width of the automobile 66 and the width of the lean vehicle 1, when the automobile 66 travels straight towards the center between the two automobiles 60 and approaches, the alarm device of the automobile 66 will output an alarm.

[0060] EXAMPLE 1 and EXAMPLE 2 in Figure 2 are specific examples of CASE 2. EXAMPLE 3 and EXAMPLE 4 in Figure 2 are specific examples of CASE 4.

[0061] In EXAMPLE 1 and EXAMPLE 3, a lean vehicle 1 is traveling in the right side area 50R of the center lane 53 of a five-lane road, and two cars 63 and 64 traveling in the center lane 53 and center right lane 54 are located in front of the lean vehicle 1. In EXAMPLE 1, a portion of vehicle 63 is located on the left side area 50L of center lane 53, and a portion of vehicle 64 is located on the right side area 50R of center right lane 54. In EXAMPLE 3, a portion of vehicle 63 is located on the right side area 50R of center lane 53, and a portion of vehicle 64 is located on the left side area 50L of center right lane 54.

[0062] In EXAMPLE 2 and EXAMPLE 4, the lean vehicle 1 is traveling in the left side area 50L of the center lane 53 of a five-lane road, and two automobiles 63, 62 traveling in the center lane 53 and center-left lane 52 are located in front of the lean vehicle 1. In EXAMPLE 2, a portion of the automobile 63 is located in the right side area 50R of the center lane 53, and a portion of the automobile 62 is located on the left side area 50L of the center-left lane 52. In EXAMPLE 4, a portion of the automobile 63 is located on the left side area 50L of the center lane 53, and a portion of the automobile 62 is located on the right side area 50R of the center-left lane 52.

[0063] In the second embodiment, if the lean vehicle 1 has a display device that displays an image captured by the imaging device 11, and a frame surrounding the vehicle in the image captured by the imaging device 11 is displayed together with the captured image, the color of the frame surrounding the vehicle that is the subject of the alarm may be different when the alarm device 13 outputs an alarm and when the alarm device 13 does not output an alarm.

[0064] [Third embodiment] Next, a lean vehicle image processing device 10 according to a third embodiment will be described with reference to Fig. 3. The second embodiment has all of the configurations of the first embodiment. The third embodiment may also have the configurations of the second embodiment.

[0065] In the third embodiment of the image processing device 10 for a lean vehicle, when the lean vehicle 1 is traveling in the right side area 50R of the lane 50, the imaging device 11 captures multiple images PR including the right side 60R and rear 60B of the automobile 60 traveling in front of the lean vehicle 1 in the lane 50 at a fixed time interval ΔT. 3 shows three consecutive images PR1, PR2, and PR3 among a plurality of images PR captured by the imaging device 11 at regular time intervals ΔT while the leaning vehicle 1 is traveling in the right side area 50R of the lane 50. Specifically, images PR1, PR2, and PR3 represent images PR captured at times TR1, TR2, and TR3, respectively. Time TR2 is later than time TR1, and time TR3 is later than time TR2. The time interval between time TR1 and time TR2 and the time interval between time TR2 and time TR3 are both ΔT. In the image processing device 10 for a lean vehicle of the third embodiment, when the lean vehicle 1 is traveling in the right side area 50R of the lane 50, the arrival time estimation device 12 estimates the arrival time Ta for the automobile 60 to reach the lean vehicle 1 based on multiple images PR including the right side 60R and rear 60B of the automobile 60 traveling in front of the lean vehicle 1 in the lane 50, captured at a fixed time interval ΔT, and the fixed time interval ΔT.

[0066] In addition, in the image processing device 10 for a lean vehicle of the third embodiment, when the lean vehicle 1 is traveling in the left side area 50L of the lane 50, the imaging device 11 captures multiple images PL including the left side 60L and rear side 60B of the automobile 60 traveling in front of the lean vehicle 1 on the lane 50 at a constant time interval ΔT. 3 shows three consecutive images PL1, PL2, and PL3 among a plurality of images PL captured by the imaging device 11 at regular time intervals ΔT while the lean-to vehicle 1 is traveling in the left side area 50L of the lane 50. Specifically, images PL1, PL2, and PL3 represent images PL captured at times TL1, TL2, and TL3, respectively. Time TL2 is later than time TL1, and time TL3 is later than time TL2. The time interval between time TL1 and time TL2 and the time interval between time TL2 and time TL3 are both ΔT. In the image processing device 10 for a lean vehicle of the third embodiment, when the lean vehicle 1 is traveling in the left side area 50L of the lane 50, the arrival time estimation device 12 estimates the arrival time Ta for the automobile 60 to reach the lean vehicle 1 based on multiple images PL including the left side 60L and rear 60B of the automobile 60 traveling in front of the lean vehicle 1 in the lane 50, captured at a fixed time interval ΔT, and the fixed time interval ΔT.

[0067] In this way, the arrival time estimation device 12 of the third embodiment acquires a plurality of consecutive images (a plurality of images PR or a plurality of images PL) including the left side 60L or the right side 60R and the rear 60B of the automobile 60 captured by the imaging device 11 at a fixed time interval ΔT, and estimates the arrival time Ta until the automobile 60 reaches the lean-to vehicle 1 based on the plurality of consecutive images (a plurality of images PR or a plurality of images PL) including the left side 60L or the right side 60R and the rear 60B of the automobile 60 and the fixed time interval ΔT. do.

[0068] In the image processing device 10 for a lean vehicle of the third embodiment, the time interval ΔT at which images are captured by the imaging device 11 may be changeable. That is, the imaging device 11 may be able to switch at least between capturing images at a first constant time interval and capturing images at a second constant time interval. Alternatively, in the image processing device 10 for a lean vehicle of the third embodiment, the time interval ΔT at which images are captured by the imaging device 11 may not be changeable.

[0069] [Fourth embodiment] Next, a lean vehicle image processing device 10 according to a fourth embodiment will be described with reference to Fig. 4. The fourth embodiment has all of the configurations of the first embodiment. The fourth embodiment may also have at least one of the configurations of the second embodiment and the third embodiment.

[0070] In the lean vehicle image processing device 10 of the fourth embodiment, the imaging device 11 captures color images as images PR and PL, as shown in Fig. 4. In Fig. 4, the color density of the rear surface 63B, right side surface 60R, and left side surface 60L of the automobile 60 is lighter than in Fig. 3, etc., to indicate that the captured images PR (PR1 to PR3) and PL (PL1 to PL3) are color images.

[0071] In the fourth embodiment of the image processing device 10 for a lean vehicle, when the lean vehicle 1 is traveling in the right side area 50R of the lane 50, the imaging device 11 continuously captures multiple images PR including the right side 60R and rear 60B of the automobile 60 traveling in front of the lean vehicle 1 in the lane 50. 4, three consecutively captured images PR1, PR2, and PR3 are shown among the multiple images PR, as in the third embodiment. However, in the fourth embodiment, as described above, the images PR1, PR2, and PR3 are color images. Also, in the fourth embodiment, the time interval between time TR1 and time TR2 and the time interval between time TR2 and time TR3 may be the same or different. In the image processing device 10 for a lean vehicle of the fourth embodiment, when the lean vehicle 1 is traveling in the right side area 50R of the lane 50, the arrival time estimation device 12 estimates the arrival time Ta for the automobile 60 to reach the lean vehicle 1 based on multiple images PR (images PR1 to PR3, etc.) including the right side 60R and rear 60B of the automobile 60 traveling in front of the lean vehicle 1 in the lane 50 and the time interval between capturing the multiple images PR.

[0072] In addition, in the image processing device 10 for a lean vehicle of the fourth embodiment, when the lean vehicle 1 is traveling in the left side area 50L of the lane 50, the imaging device 11 continuously captures multiple images PL including the left side 60L and rear side 60B of the automobile 60 traveling in front of the lean vehicle 1 on the lane 50. 4, three consecutively captured images PL1, PL2, and PL3 are shown among the multiple images PL, as in the third embodiment. However, in the fourth embodiment, as described above, the images PL1, PL2, and PL3 are color images. Also, in the fourth embodiment, the time interval between time TL1 and time TL2 and the time interval between time TL2 and time TL3 may be the same or different. In the image processing device 10 for a lean vehicle of the fourth embodiment, when the lean vehicle 1 is traveling in the left side area 50L of the lane 50, the arrival time estimation device 12 estimates the arrival time Ta for the automobile 60 to reach the lean vehicle 1 based on multiple images PL (images PL1 to PL3, etc.) captured consecutively, including the left side 60L and rear 60B of the automobile 60 traveling in front of the lean vehicle 1 on the lane 50, and the time interval between capturing the multiple images PL.

[0073] In this way, the arrival time estimation device 12 of the fourth embodiment acquires a plurality of color images (a plurality of images PR or a plurality of images PL) including the left side 60L or the right side 60R and the rear 60B of the automobile 60 captured by the imaging device 11, and estimates the arrival time Ta until the automobile 60 reaches the lean vehicle 1 based on the acquired plurality of color images (a plurality of images PR or a plurality of images PL) including the left side 60L or the right side 60R and the rear 60B of the automobile 60. do.

[0074] [Fifth embodiment] Next, an image processing device 10 for a lean vehicle according to a fifth embodiment will be described with reference to Fig. 5. The fifth embodiment has all of the configurations of the first embodiment. The fifth embodiment may also have at least one of the configurations of the second to fourth embodiments.

[0075] In the fifth embodiment of the image processing device 10 for a lean vehicle, when the lean vehicle 1 is traveling in the right side area 50R of the lane 50, the imaging device 11 continuously captures multiple images PR including the right side 60R and rear 60B of the automobile 60 traveling in front of the lean vehicle 1 in the lane 50. 5, three consecutively captured images PR1, PR2, and PR3 are shown among a plurality of images PR, as in the third embodiment. However, in the fifth embodiment, the time interval between time TR1 and time TR2 and the time interval between time TR2 and time TR3 may be the same or different. In the image processing device 10 for a lean vehicle of the fifth embodiment, when the lean vehicle 1 is traveling in the right side area 50R of the lane 50, the arrival time estimation device 12 acquires the width (WR1, WR2, WR3, etc.) of the automobile 60 as the size of the automobile 60 in the image PR based on each of multiple images PR including the right side 60R and rear 60B of the automobile 60 traveling in front of the lean vehicle 1 in the lane 50, and estimates the arrival time Ta for the automobile 60 to reach the lean vehicle 1 based on the width of the automobile in the multiple images PR and the time interval between capturing the multiple images PR.

[0076] In the fifth embodiment of the image processing device 10 for a lean vehicle, when the lean vehicle 1 is traveling in the left side area 50L of the lane 50, the imaging device 11 continuously captures multiple images PL including the left side 60L and rear 60B of the automobile 60 traveling in front of the lean vehicle 1 on the lane 50. 5, three consecutively captured images PL1, PL2, and PL3 are shown among the multiple images PL, as in the third embodiment. However, in the fifth embodiment, the time interval between time TL1 and time TL2 and the time interval between time TL2 and time TL3 may be the same or different. In the image processing device 10 for a lean vehicle of the fifth embodiment, when the lean vehicle 1 is traveling in the left side area 50L of the lane 50, the arrival time estimation device 12 acquires the width (WL1, WL2, WL3, etc.) of the automobile 60 as the size of the automobile 60 in the image PL based on each of multiple images PL including the left side 60L and rear 60B of the automobile 60 traveling in the lane 50, and estimates and outputs the arrival time Ta for the automobile 60 to reach the lean vehicle 1 based on the width of the automobile in the multiple images PL and the time interval between capturing the multiple images PL.

[0077] In this way, the arrival time estimation device 12 of the fifth embodiment acquires a plurality of consecutive images (a plurality of images PR or a plurality of images PL) including the left side 60L or the right side 60R and the rear 60B of the automobile 60 captured by the imaging device 11 when the automobile 60 is traveling ahead of the lean vehicle 1, and acquires the size of the rear of the automobile 60 in the image PR or the image PL based on the plurality of consecutive images (a plurality of images PR or a plurality of images PL) including the left side 60L or the right side 60R and the rear 60B of the automobile 60, and estimates the arrival time Ta until the automobile 60 reaches the lean vehicle 1. do.

[0078] In the fifth embodiment, the arrival time estimation device 12 acquired the width of the automobile 60 as the size of the automobile 60 in the image, but it may also acquire, for example, the height (vertical length) of the rear surface 60B of the automobile 60, the area of ​​the rear surface 60B of the automobile 60 in the image, etc. as the size of the rear surface 60B of the automobile 60 in the image.

[0079] [Sixth embodiment] Next, a lean vehicle image processing device 10 according to a sixth embodiment will be described with reference to Fig. 6. The sixth embodiment has all of the configurations of the first embodiment. The sixth embodiment may also have at least one of the configurations of the second to fourth embodiments.

[0080] In the sixth embodiment of the image processing device 10 for a lean vehicle, when the lean vehicle 1 is traveling in the right side area 50R of the lane 50, the imaging device 11 continuously captures multiple images PR including the right side 60R and rear 60B of the automobile 60 traveling in front of the lean vehicle 1 in the lane 50. 6, three consecutively captured images PR1, PR2, and PR3 are shown among a plurality of images PR, as in the third embodiment. However, in the sixth embodiment, the time interval between time TR1 and time TR2 and the time interval between time TR2 and time TR3 may be the same or different. In the lean-vehicle image processing device 10 of the sixth embodiment, when the lean-vehicle 1 is traveling in the right side area 50R of the lane 50, the arrival time estimation device 12 acquires distances between multiple feature points of the automobile 60 for each of multiple images PR including the right side 60R and rear 60B of the automobile 60 traveling in the lane 50. At this time, the arrival time estimation device 12 may acquire positions of the multiple feature points of the automobile 60 for each of the multiple images PR and derive the distances between the multiple feature points based on the acquired positions of the multiple feature points. FIG. 6 illustrates a case where the feature points of the automobile 60 are the upper end F1 and lower end F2 of the boundary line between the right side 60R and rear 60B of the automobile 60, and the distances between the multiple feature points of the automobile 60 are the distances between the upper end F1 and the lower end F2 (HR1, HR2, HR3, etc.). Then, the arrival time estimation device 12 estimates and outputs the arrival time Ta for the automobile 60 to arrive at the lean vehicle 1 based on the distance between the multiple feature points in each of the multiple images PR and the time interval between capturing the multiple images PR.

[0081] In the sixth embodiment of the image processing device 10 for a lean vehicle, when the lean vehicle 1 is traveling in the left side area 50L of the lane 50, the imaging device 11 continuously captures multiple images PL including the left side 60L and rear 60B of the automobile 60 traveling in front of the lean vehicle 1 on the lane 50. 6, three consecutively captured images PL1, PL2, and PL3 are shown among the multiple images PL, as in the third embodiment. However, in the sixth embodiment, the time interval between time TL1 and time TL2 and the time interval between time TL2 and time TL3 may be the same or different. In the lean-vehicle image processing device 10 of the sixth embodiment, when the lean-vehicle 1 is traveling in the left side area 50L of the lane 50, the arrival time estimation device 12 acquires distances between multiple feature points of the automobile 60 for each of multiple images PL including the left side 60L and rear 60B of the automobile 60 traveling ahead of the lean-vehicle 1 on the lane 50. At this time, the arrival time estimation device 12 may acquire positions of the multiple feature points of the automobile 60 for each of the multiple images PL and derive distances between the multiple feature points based on the acquired positions of the multiple feature points. FIG. 6 illustrates a case where the feature points of the automobile 60 are the upper end F3 and lower end F4 of the boundary line between the rear 60B and left side 60L of the automobile 60, and the distances between the multiple feature points of the automobile 60 are the distances (HL1, HL2, HL3, etc.) between the upper end F3 and the lower end F4. Then, the arrival time estimation device 12 estimates and outputs the arrival time Ta for the automobile 60 to arrive at the lean vehicle 1 based on the distance between the multiple feature points in each of the multiple images PL and the time interval between capturing the multiple images PL.

[0082] In this way, the arrival time estimation device 12 of the sixth embodiment acquires a plurality of consecutive images (a plurality of images PR or a plurality of images PL) including the left side 60L or the right side 60R and the rear 60B of the automobile 60 captured by the imaging device 11 when the automobile 60 is traveling ahead of the lean vehicle 1, and acquires distances between a plurality of feature points on the left side 60L or the right side 60R and the rear 60B of the automobile 60 in the image PR or the image PL based on the plurality of consecutive images (a plurality of images PR or a plurality of images PL) including the left side 60L or the right side 60R and the rear 60B of the automobile 60, thereby estimating the arrival time Ta until the automobile 60 reaches the lean vehicle 1. do.

[0083] In the sixth embodiment, the multiple feature points of the automobile 60 in the images PR and PL are the upper and lower ends of the boundary line between the right side 60R and the rear side 60B or the left side 60L of the automobile 60, but this is not limited to this. At least one of the multiple feature points of the automobile 60 in the images PR and PL may be another point on the rear side 60B, the right side 60R, or the left side 60L of the automobile 60, or on the above-mentioned boundary line. The number of feature points may also be three or more. When the number of feature points is three or more, the arrival time estimation device 12 may obtain the distances between two or more pairs of two feature points that are different combinations from among the three or more feature points, and estimate the arrival time for the automobile 60 to arrive at the lean-to vehicle 1 based on these distances.

[0084] [Seventh embodiment] Next, an image processing device 10 for a lean vehicle according to a seventh embodiment will be described with reference to Fig. 7. The seventh embodiment has all of the configurations of the first embodiment. The seventh embodiment may also have at least one of the configurations of the second to fourth embodiments.

[0085] As shown in FIG. 7, the arrival time estimation device 12 of the image processing device 10 for a lean vehicle according to the seventh embodiment includes a trained model 21 and an arrival time derivation unit 22.

[0086] The trained model 21 may be a model that has undergone deep learning based on data of a large number of images captured by the imaging device 11. The trained model 21 is a model that receives as input a plurality of consecutive images P (a plurality of images PR or a plurality of images PL) captured by the imaging device 11, and outputs the rate of change E in the size of the automobile 60 between the plurality of images P. The rate of change E in the size of the automobile 60 in the image P may be the rate of change in the width of the rear surface 60B of the automobile 60. The rate of change E in the size of the automobile 60 in the image P may be the rate of change in the height (vertical length) of the rear surface 60B of the automobile 60. The rate of change E in the size of the automobile 60 in the image P may be the rate of change in the area of ​​the rear surface 60B of the automobile 60 in the image P. The arrival time derivation unit 22 derives the arrival time Ta required for the automobile 60 to arrive at the lean vehicle 1 based on the change rate E output from the trained model 21.

[0087] In the seventh embodiment, when a lean vehicle 1 is traveling in the right side area 50R of a lane 50, an imaging device 11 continuously captures multiple images PR including the right side 60R and rear 60B of a car 60 traveling in front of the lean vehicle 1 in the lane 50, and inputs the images into a trained model 21. 7, three consecutively captured images PR1, PR2, and PR3 are shown among a plurality of images PR, as in the third embodiment. However, in the seventh embodiment, the time interval between time TR1 and time TR2 and the time interval between time TR2 and time TR3 may be the same or different. The trained model 21 outputs the rate of change E of the size of the automobile 60 between the plurality of images PR based on the plurality of input images PR. The output rate of change E is, for example, the rate of change ER1 of the size of the automobile 60 in image PR2 relative to the size of the automobile 60 in image PR1, the rate of change ER2 of the size of the automobile 60 in image PR3 relative to the size of the automobile 60 in image PR2, etc. Then, the arrival time derivation unit 22 derives the arrival time Ta for the automobile 60 to arrive at the lean vehicle 1 based on the change rate E output from the trained model 21 and the time interval at which the multiple images PR were captured.

[0088] In the seventh embodiment, when a lean vehicle 1 is traveling in the left side area 50L of a lane 50, an imaging device 11 continuously captures multiple images PL including the left side 60L and rear 60B of a car 60 traveling in front of the lean vehicle 1 in the lane 50, and inputs the images into a trained model 21. 7, three consecutively captured images PL1, PL2, and PL3 are shown among the multiple images PL, as in the third embodiment. However, in the seventh embodiment, the time interval between time TL1 and time TL2 and the time interval between time TL2 and time TL3 may be the same or different. The trained model 21 outputs the rate of change E of the size of the automobile 60 between the plurality of images PL based on the plurality of input images PL. The rate of change E that is output is, for example, the rate of change EL1 of the size of the automobile 60 in image PL2 relative to the size of the automobile 60 in image PL1, the rate of change EL2 of the size of the automobile 60 in image PL3 relative to the size of the automobile 60 in image PL2, etc. Then, the arrival time derivation unit 22 derives the arrival time Ta for the lean vehicle 1 to reach the automobile 60 traveling in the lane 50 based on the change rate E output from the learned model 21 and the time interval at which the multiple images PL were captured.

[0089] In this way, the arrival time estimation device 12 of the seventh embodiment acquires a plurality of consecutive images (a plurality of images PR or a plurality of images PL) including the left side 60L or the right side 60R and the rear 60B of the automobile 60 captured by the imaging device 11 when the automobile 60 is traveling ahead of the lean vehicle 1, and estimates the arrival time Ta until the automobile 60 reaches the lean vehicle 1 using a deep learning algorithm to estimate the rate of change E of the size of the automobile 60 having the left side 60L or the right side 60R and the rear 60B in the image PR or the image PL based on the plurality of consecutive images (a plurality of images PR or a plurality of images PL) including the left side 60L or the right side 60R and the rear 60B of the automobile 60. do.

[0090] The arrival time estimation device 12 may perform, for example, the following processing to improve the accuracy of the change rate E output from the trained model 21. For example, when the leaning vehicle 1 is traveling in the right side area 50R of the lane 50, the arrival time estimation device 12 inputs two images PR and enlarges the image PR captured earlier of the two images PR based on the change rate E output from the trained model 21. The trained model 21 then learns the error obtained by comparing the automobile 60 in the enlarged image PR with the automobile 60 in the image PR captured later of the two images PR. In other words, the trained model 21 is adjusted to reduce this error. This error may be obtained, for example, from the difference in pixel values ​​(e.g., RGB values) of each pixel of the automobile 60 when the enlarged image PR and the image PR captured later are superimposed. More specifically, for example, the error may be the root mean square or absolute mean value of the difference in pixel values ​​of each pixel. The arrival time estimation device 12 also causes the trained model 21 to learn an error when the lean vehicle 1 is traveling in the left side area 50L of the lane 50. Such error learning may be performed both when the trained model 21 is used and when the trained model 21 is generated. Furthermore, such error learning may not be performed when the trained model 21 is used, but only when the trained model 21 is generated.

[0091] Here, unlike the seventh embodiment, consider a case where the trained model 21 takes an image as input and outputs the distance between the lean vehicle 1 and the automobile 60, and the arrival time derivation unit 22 derives the arrival time Ta based on the distance between the lean vehicle 1 and the automobile 60 output from the trained model 21. In this case, when generating a learned model by deep learning, the correct value of the distance between the lean vehicle 1 and the automobile 60 is required as training data. Therefore, it is necessary to prepare the correct value of the distance between the lean vehicle 1 and the automobile 60 in a large number of situations where the surrounding environment and the inclination angle of the lean vehicle 1 are different. In contrast, in the seventh embodiment, as described above, the trained model 21 takes an image as input and outputs the rate of change E, and the arrival time derivation unit 22 derives the arrival time Ta based on the rate of change E output from the trained model 21. In this case, the rate of change E is information obtained from the image, so there is no need for a correct value for the rate of change E. Therefore, compared to when the trained model 21 takes an image as input and outputs the distance between the lean vehicle 1 and the automobile 60, the cost and labor required to generate the trained model can be reduced.

[0092] The right side area in this specification is included in the right driving area in the specification of Japanese Patent Application No. 2023-099900, which is the basic application of the present application. The left side area in this specification is included in the left driving area in the specification of the same application. The central area in this specification is included in the central driving area in the specification of the same application. The automobile in this specification corresponds to a four-wheeled vehicle in the specification of the same application. [Explanation of symbols]

[0093] 1: lean vehicle, 10: image processing device for lean vehicle, 11: imaging device, 12: arrival time estimation device, 16: warning device, 21: trained model, 22: arrival time derivation unit, 51: leftmost lane, 52: center left lane, 53: center lane, 54: center right lane, 55: rightmost lane, 50L: left side area, 50C: center area, 50R: right side area, 61 to 65: automobile, 63R: right side, 63L: left side, 63B: rear, E: rate of change, P, PR, PL: image, Ta: arrival time

Claims

1. An image processing device for a lean vehicle, the image processing device including: an imaging device that captures an image; and an arrival time estimation device that processes the image captured by the imaging device to estimate an arrival time of an object to reach the lean vehicle, the image processing device being mounted on the lean vehicle, When a single lane is divided into thirds in the left-right direction, the area including the right edge of the single lane is defined as the right side area, the area including the left edge of the single lane is defined as the left side area, and the area between the right side area and the left side area is defined as the center area, If we define the lanes of a five-lane road as the leftmost lane, center left lane, center lane, center right lane, and rightmost lane, from left to right, When the lean vehicle is traveling in the right or left side area of ​​the center lane of the five-lane road, and five automobiles are traveling in a line left and right in front of the lean vehicle, and the longitudinal distance between the five automobiles and the lean vehicle is the distance required to capture an image so that the image processing is completed when the TTC (Time To Collision) is 3 seconds, assuming that the time from capturing an image to completing image processing is 0.5 seconds and an object approaches the lean vehicle at a relative speed of 15 km / h, The imaging device and the arrival time estimation device are configured as follows, and are configured to estimate the arrival time for a vehicle traveling in the center lane to reach the lean vehicle based on an image that includes the left or right side of the vehicle traveling in the center lane, both when the lean vehicle is traveling in the right side area of ​​the center lane of the five-lane road and when the lean vehicle is traveling in the left side area of ​​the center lane of the five-lane road. The imaging device is When the lean vehicle is traveling in the right side area of ​​the center lane of the five-lane road, the left edge of the rear surface of the vehicle traveling in the leftmost lane is not included in the image, but at least the rear surface of the vehicle traveling in the center lane and the right side of the vehicle traveling in the center lane are included in the image, and, The vehicle includes a monocular camera having a horizontal angle of view configured so that, while the lean vehicle is traveling in the left side area of ​​the center lane of the five-lane road, the right edge of the rear of the vehicle traveling in the rightmost lane is not included in the image, but at least the rear of the vehicle traveling in the center lane and the left side of the vehicle traveling in the center lane are included in the image. The arrival time estimation device includes: While the lean vehicle is traveling in the right side area of ​​the center lane of the five-lane road, an arrival time of the vehicle traveling in the center lane to reach the lean vehicle is estimated based on the image including at least the rear of the vehicle traveling in the center lane and the right side of the vehicle traveling in the center lane; and, While the lean vehicle is traveling in the left side area of ​​the center lane of the five-lane road, the system is configured to estimate the arrival time of the vehicle traveling in the center lane to reach the lean vehicle based on the image that includes at least the rear of the vehicle traveling in the center lane and the left side of the vehicle traveling in the center lane.

2. the lean vehicle image processing device further includes an alarm device that outputs an alarm based on the arrival time of an object to reach the lean vehicle, the arrival time estimated by the arrival time estimation device; The alarm device When two automobiles are traveling straight ahead of the lean vehicle, side by side with a gap in the left-right direction, and the size of the gap is such that there is no room in the left-right direction in terms of the width of the automobiles, but there is room in the left-right direction in terms of the width of the lean vehicle, the alarm is not output even if the lean vehicle travels straight towards the center between the two automobiles and approaches them, and, When two automobiles are traveling straight ahead of the lean vehicle, side by side with a gap in the left-right direction, and the size of the gap is such that there is no room in the left-right direction for the width of the lean vehicle, the warning is output when the lean vehicle travels straight towards and approaches the center between the two automobiles, outputting the warning according to the vehicle width of the lean vehicle; 2. The image processing device for a lean vehicle according to claim 1.

3. The imaging device is The monocular camera captures a plurality of images including a left side or a right side and a rear side of the vehicle at regular time intervals when the vehicle is traveling in front of the lean vehicle, The arrival time estimation device includes: acquiring the plurality of consecutive images including the left side or the right side and the rear of the automobile captured by the imaging device at the fixed time intervals, and estimating the arrival time of the automobile to reach the lean vehicle based on the plurality of consecutive images including the left side or the right side and the rear of the automobile and the fixed time intervals; 2. The image processing device for a lean vehicle according to claim 1.

4. The imaging device is The monocular camera captures a plurality of images including a left side or a right side and a rear side of the vehicle at regular time intervals when the vehicle is traveling in front of the lean vehicle, The arrival time estimation device includes: acquiring the plurality of consecutive images including the left side or the right side and the rear of the automobile captured by the imaging device at the fixed time intervals, and estimating the arrival time of the automobile to reach the lean vehicle based on the plurality of consecutive images including the left side or the right side and the rear of the automobile and the fixed time intervals; 3. The image processing device for a lean vehicle according to claim 2.

5. The imaging device is The monocular camera captures a plurality of color images including a left side or a right side and a rear side of the vehicle when the vehicle is traveling in front of the lean vehicle, The arrival time estimation device includes: acquiring the plurality of color images including the left side or the right side and the rear of the automobile captured by the imaging device, and estimating the arrival time until the automobile reaches the lean vehicle based on the plurality of color images including the left side or the right side and the rear of the automobile that have been acquired; 5. The image processing device for a lean vehicle according to claim 1.

6. The arrival time estimation device includes: When an automobile is traveling in front of the lean vehicle, a plurality of consecutive images including a left side or a right side and a rear side of the automobile are acquired by the imaging device, (I) an algorithm for obtaining a size of the rear surface of the vehicle in the images based on the plurality of consecutive images including the left side or right side and the rear surface of the vehicle, and estimating the arrival time of the vehicle to reach the lean vehicle; (II) An algorithm for estimating the arrival time of the vehicle to the lean vehicle by acquiring distances between a plurality of feature points on the left side or the right side and the rear of the vehicle in the images based on the plurality of consecutive images including the left side or the right side and the rear of the vehicle, or (III) estimating the arrival time of the vehicle to the lean vehicle using a deep learning algorithm to estimate a rate of change of the size of the vehicle having the left side or right side and the rear in the images based on the plurality of consecutive images including the left side or right side and the rear of the vehicle; 5. The image processing device for a lean vehicle according to claim 1.

7. The arrival time estimation device includes: When an automobile is traveling in front of the lean vehicle, a plurality of consecutive images including a left side or a right side and a rear side of the automobile are acquired by the imaging device, (I) an algorithm for obtaining a size of the rear surface of the vehicle in the images based on the plurality of consecutive images including the left side or right side and the rear surface of the vehicle, and estimating the arrival time of the vehicle to reach the lean vehicle; (II) An algorithm for estimating the arrival time of the vehicle to the lean vehicle by acquiring distances between a plurality of feature points on the left side or the right side and the rear of the vehicle in the images based on the plurality of consecutive images including the left side or the right side and the rear of the vehicle, or (III) estimating the arrival time of the vehicle to the lean vehicle using a deep learning algorithm to estimate a rate of change of the size of the vehicle having the left side or right side and the rear in the images based on the plurality of consecutive images including the left side or right side and the rear of the vehicle; 6. The image processing device for a lean vehicle according to claim 5.

Images