Image processing device for leaning vehicle

Abstract

In the present invention, in a case where a leaning vehicle (1) travels in a right side area (50R) of a center lane (53) of a five-lane road, and the inter-vehicle distance (D) between the leaning vehicle and five automobiles (61-65) traveling in parallel thereto is a distance which is required to complete image processing at a time point of 3 seconds to Time to Collision (TTC) on the assumption that the image processing time is 0.5 seconds and the relative speed is 15km / h: an image capture device (11) of an image processing device (10) for a leaning vehicle includes a monocular camera in which the angle of view in the left-right direction is configured such that the left end (61bl) of the rear face of an automobile (61) in the left end lane (51) is not imaged, but at least the left end (63bl) of the rear face of an automobile (3) in the center lane and the right end (64br) of the rear face of an automobile (64) in the center right lane (54) are imaged; and a recognition device (12) of the image processing device for a leaning vehicle at least recognizes the automobile in the center lane and the automobile in the center right lane.

Description

Image processing device for lean vehicles

[0001] The present invention relates to an image processing device for a lean vehicle that is mounted on a lean vehicle.

[0002] Patent Document 1 proposes an arrival time estimation device that extracts feature points from images of an object captured in time series by an imaging device mounted on a vehicle and estimates the time it takes for the vehicle to reach the object based on the extracted feature points, and a driving assistance control device that performs control to assist the driver in operating the vehicle using the estimated arrival time. Patent Document 1 also proposes a lean vehicle that includes an imaging device, an arrival time estimation device, and a driving assistance control device. The lean vehicle is, for example, a motorcycle.

[0003] The device proposed in Patent Document 1, which combines an imaging device, an arrival time estimation device, and a driving assistance control device, can be rephrased as an image processing device including an imaging device that captures images and a recognition device that processes the captured images to recognize objects. The imaging device typically includes an optical device, such as a lens, and an imaging element, such as a complementary metal oxide semiconductor (CMOS) sensor. The recognition device typically includes a processor, such as a field-programmable gate array (FPGA), graphics processing unit (GPU), central processing unit (CPU), or system on chip (SoC), that processes images, and memory, such as random access memory (RAM). Note that SoCs have a configuration in which the processor and memory are integrated. Patent Document 1 also describes that the imaging device may be a monocular camera or a stereo camera.

[0004] The arrival time described in Patent Document 1 corresponds to the so-called Time to Collision (TTC). Non-Patent Document 1 describes that a TTC threshold value of 3 seconds is appropriate for evaluating a potential collision.

[0005] The width of lanes and the width of vehicles are regulated by national or regional administrative bodies. In Japan, lane widths are regulated to 2.75 to 4 meters depending on the road classification under Article 5, Paragraphs 4 and 5 of the Road Structure Ordinance (see Non-Patent Document 2). Generally, lane widths are 3.5 to 4 meters. Furthermore, in Japan, Article 59 of the Road Transport Vehicle Safety Standards stipulates that the width of motorcycles must be 1.3 meters or less, and Article 2 of the Road Transport Vehicle Safety Standards stipulates that the width of automobiles must be 2.5 meters or less (see Non-Patent Document 3). Generally, the width of lean-to vehicles is less than 1 meter, and the width of automobiles is 2 to 2.5 meters.

[0006] Japanese Patent Application Laid-Open No. 2021-111246

[0007] Oumaima Barhoumi, Mohamed H. Zaki, SOFIENE TAHAR, "A Formal Approach to Road Safety Assessment Using Traffic Conflict Techniques," "VI. SENSITIVITY STUDY," [online], last updated April 26, 2024, IEEE Open Journal of Vehicular Technology, [Retrieved June 4, 2024], Internet <URL: https: / / ieeexplore.ieee.org / stamp / stamp.jsp?arnumber=10496854> Road Structure Ordinance, [Retrieved June 4, 2024], Internet <URL: https: / / elaws.e-gov.go.jp / document?lawid=345CO0000000320> Road Transport Vehicle Safety Standards, [Retrieved June 4, 2024], Internet <URL: https: / / elaws.e-gov.go.jp / document?lawid=326M50000800067>

[0008] The width of a lean-to vehicle is much smaller than that of an automobile. In other words, the body of a lean-to vehicle is slimmer (narrower) than that of an automobile. The width of a lean-to vehicle is also much smaller than the width of a lane. Therefore, when a lean-to vehicle is traveling in a single lane, it may travel in either the right or left side area of ​​the lane. Therefore, a lean-to vehicle has a much greater degree of freedom in its lateral traveling position within a single lane than an automobile. Therefore, there is a demand for an image processing device for a lean-to vehicle, including an imaging device and a recognition device, that can accommodate usage scenarios unique to lean-to vehicles, such as when a lean-to vehicle is traveling in the right or left side area of ​​a lane while traveling in the same lane as an automobile. In this specification, when a single lane is divided into thirds horizontally, the area including the right edge is defined as the right side area, and the area including the left edge is defined as the left side area.

[0009] Furthermore, since lean vehicles have slimmer (narrower) bodies than automobiles, there is a demand for an image processing device for lean vehicles that can be easily installed in lean vehicles.

[0010] The object of the present invention is to provide an image processing device for a lean vehicle that can be adapted to situations in which a lean vehicle, which has a slimmer body than an automobile, travels in the right side area and left side area of ​​a lane while traveling in the same lane as an automobile, and that is highly mountable on a lean vehicle, which has a slimmer body than an automobile.

[0011] (1) An image processing device for a lean vehicle according to one embodiment of the present invention has the following configuration. An image processing device for a lean vehicle that is mounted on a lean vehicle includes an imaging device that captures images and a recognition device that recognizes an object from the image acquired by the imaging device, and the image processing device is configured to calculate a TTC (Time to Action) based on the assumption that, when a single lane is divided into thirds in the left-right direction, an area including the right edge of the single lane is defined as a right side area, an area including the left edge of the single lane is defined as a left side area, and an area between the right side area and the left side area is defined as a center area, and the lanes of a five-lane road are defined from left to right as the leftmost lane, center left lane, center lane, center right lane, and rightmost lane, the lean vehicle is traveling in the right side area or the left side area of ​​the center lane of the five-lane road, five automobiles are traveling in a line in the left-right direction ahead of the lean vehicle, and the distance between the five automobiles and the lean vehicle in the longitudinal direction is 0.5 seconds from image capture to completion of image processing, and an object approaches the lean vehicle at a relative speed of 15 km / h. When the time (times until collision) is a distance necessary to capture an image so that the image processing is completed at 3 seconds, the imaging device and the recognition device are configured as follows: The imaging device includes a monocular camera having a horizontal angle of view configured such that, while the leaned vehicle is traveling in the right side area of ​​the center lane of the five-lane road, the left edge of the rear of the vehicle traveling in the leftmost lane is not captured, but at least the left edge of the rear of the vehicle traveling in the center lane and the right edge of the rear of the vehicle traveling in the center right lane are captured, and, while the leaned vehicle is traveling in the left side area of ​​the center lane of the five-lane road, the right edge of the rear of the vehicle traveling in the rightmost lane is not captured, but at least the right edge of the rear of the vehicle traveling in the center lane and the left edge of the rear of the vehicle traveling in the center left lane are captured.The recognition device is configured to at least recognize both the vehicle traveling in the center lane and the vehicle traveling in the center right lane from the image captured while the lean vehicle is traveling in the right side area of ​​the center lane of the five-lane road, and to at least recognize both the vehicle traveling in the center lane and the vehicle traveling in the center left lane from the image captured while the lean vehicle is traveling in the left side area of ​​the center lane of the five-lane road.

[0012] The width of a lean-to vehicle is very small compared to the width of an automobile. In other words, the body of a lean-to vehicle is slimmer than that of an automobile. The width of a lean-to vehicle is very small compared to the width of a lane. In other words, when a lean-to vehicle and an automobile are traveling in the same lane, the lean-to vehicle may travel in the right side area or the left side area of ​​the lane. When a lean-to vehicle is traveling in the right side area of ​​the center lane of a five-lane road, two automobiles traveling in the center lane and the center-right lane will be located in front of the lean-to vehicle. When a lean-to vehicle is traveling in the left side area of ​​the center lane of a five-lane road, two automobiles traveling in the center lane and the center-left lane will be located in front of the lean-to vehicle. The imaging device includes a monocular camera. The monocular camera of the imaging device has a horizontal angle of view configured so that when the lean-to vehicle is traveling in the right side area of ​​the center lane of a five-lane road, at least the left edge of the rear of the automobile traveling in the center lane and the right edge of the rear of the automobile traveling in the center-right lane are captured. The monocular camera of the imaging device has a horizontal angle of view configured so that, when a lean-in vehicle is traveling in the left side area of ​​the center lane of a five-lane road, at least the right edge of the rear of a vehicle traveling in the center lane and the left edge of the rear of a vehicle traveling in the center left lane are captured. The recognition device recognizes at least both the vehicle traveling in the center lane and the vehicle traveling in the center right lane from images captured while the lean-in vehicle is traveling in the right side area of ​​the center lane of a five-lane road. The recognition device recognizes at least both the vehicle traveling in the center lane and the vehicle traveling in the center left lane from images captured while the lean-in vehicle is traveling in the left side area of ​​the center lane of a five-lane road. With this configuration, the image processing device for a lean-in vehicle can capture and recognize two vehicles located in front of the lean-in vehicle whether the lean-in vehicle is traveling in the right side area or the left side area. In other words, it is possible to ensure the minimum lateral field of view required in usage scenarios unique to lean vehicles, which have a much higher degree of freedom in lateral driving position within a single lane than automobiles.Therefore, the image processing device for lean-back vehicles can be applied to scenes in which a lean-back vehicle, which has a slimmer body than an automobile, travels in the right side area and left side area of ​​a lane while traveling in the same lane as an automobile. Furthermore, the monocular camera of the imaging device is configured with a left-right angle of view so that the left edge of the rear of an automobile traveling in the leftmost lane is not captured when the lean-back vehicle is traveling in the right side area of ​​the center lane of a five-lane road. The monocular camera of the imaging device is configured with a left-right angle of view so that the right edge of the rear of an automobile traveling in the rightmost lane is not captured when the lean-back vehicle is traveling in the left side area of ​​the center lane of a five-lane road. When a lean-back vehicle is traveling in the right side area of ​​the center lane of a five-lane road, the automobile traveling in the leftmost lane is located more than two car lengths to the left of the lean-back vehicle. When a lean-back vehicle is traveling in the left side area of ​​the center lane of a five-lane road, the automobile traveling in the rightmost lane is located more than two car lengths to the right of the lean-back vehicle. The width of a lean vehicle is very small compared to the width of an automobile. In other words, a lean vehicle has a slimmer body compared to an automobile. Therefore, a vehicle located more than two car lengths away from the lean vehicle in the left-right direction has less impact on the running of the lean vehicle, which has a slimmer body compared to an automobile, compared to two vehicles located in front of the lean vehicle. With the above configuration, even when the lean vehicle is traveling in the right side area or the left side area, it is possible to limit the imaging of areas located more than two car lengths away from the lean vehicle in the left-right direction. As a result, the capacity (data volume) of the image acquired by the imaging device can be reduced. Furthermore, by limiting the imaging area, for example, a compact optical device and imaging element can be selected, thereby making the imaging device more compact. In addition, because the capacity of the image acquired by the imaging device is small, the capacity of the image of the target to be recognized by the recognition device can be reduced. As a result, the processing load of the recognition device can be reduced, and for example, the heat dissipation structure of the recognition device can be made simple and compact, making the recognition device more compact. Therefore, it is possible to improve the mountability of the image processing device for a lean vehicle in a lean vehicle whose body is slimmer than that of an automobile.As a result of the above, it is possible to provide an image processing device for lean vehicles that can adapt to situations where a lean vehicle, which has a slimmer body than an automobile, is traveling in the right side area and left side area of ​​the lane while traveling in the same lane as an automobile, and that is highly mountable on lean vehicles, which have a slimmer body than an automobile.

[0013] (2) An image processing device for a lean vehicle according to one embodiment of the present invention may have the following configuration in addition to the configuration of (1) above: When the lean vehicle is traveling in the right side area or the left side area of ​​the center lane of the five-lane road, and the five automobiles are traveling in a line in the left-right direction ahead of the lean vehicle, and the longitudinal distance between the five automobiles and the lean vehicle is a distance necessary to capture an image so that the image processing is completed when the TTC is 3 seconds, assuming that the time from capturing an image to completing image processing is 0.5 seconds and that an object approaches the lean vehicle at a relative speed of 15 km / h, the imaging device and the recognition device are further configured as follows: The imaging device captures images with the monocular camera while the lean vehicle is traveling in the right side area of ​​the center lane of the five-lane road so that at least the rear of the vehicle traveling in the center lane, the right side of the vehicle traveling in the center lane, the rear of the vehicle traveling in the center right lane, and the left side of the vehicle traveling in the center right lane are included in the image, and while the lean vehicle is traveling in the left side area of ​​the center lane of the five-lane road, the imaging device captures images with the monocular camera while the lean vehicle is traveling in the left side area of ​​the center lane of the five-lane road so that at least the rear of the vehicle traveling in the center lane, the left side of the vehicle traveling in the center lane, the rear of the vehicle traveling in the center left lane, and the right side of the vehicle traveling in the center left lane are included in the image. The recognition device recognizes at least both the vehicle traveling in the center lane and the vehicle traveling in the center right lane from the image including the rear of the vehicle traveling in the center lane, the right side of the vehicle traveling in the center right lane, the rear of the vehicle traveling in the center right lane, and the left side of the vehicle traveling in the center right lane, and recognizes at least both the vehicle traveling in the center lane and the vehicle traveling in the center left lane from the image including the rear of the vehicle traveling in the center lane, the left side of the vehicle traveling in the center lane, the rear of the vehicle traveling in the center left lane, and the right side of the vehicle traveling in the center left lane.

[0014] The width of a lean vehicle is very small compared to the width of an automobile. In other words, the body of a lean vehicle is slimmer than that of an automobile. The width of a lean vehicle is very small compared to the width of a lane. In other words, when a lean vehicle and an automobile are traveling in the same lane, the lean vehicle may travel in the right side area or the left side area of ​​the lane. When a lean vehicle is traveling in the right side area of ​​the center lane of a five-lane road, in front of the lean vehicle, the right side of the automobile traveling in the center lane and the left side of the automobile traveling in the center right lane are side by side, between the rear of the automobile traveling in the center lane and the rear of the automobile traveling in the center right lane. When a lean vehicle is traveling in the right side area of ​​the center lane of a five-lane road, in front of the lean vehicle, the right side of the automobile traveling in the center lane and the left side of the automobile traveling in the center right lane are side by side, between the rear of the automobile traveling in the center lane and the rear of the automobile traveling in the center right lane. The imaging device captures an image with a monocular camera while the leaned vehicle is traveling in the right side area of ​​the center lane of a five-lane road, so that the rear of the vehicle traveling in the center lane, the right side of the vehicle traveling in the center-right lane, and the left side of the vehicle traveling in the center-right lane are included in the image. The imaging device captures an image with a monocular camera while the leaned vehicle is traveling in the left side area of ​​the center lane of the five-lane road, so that the rear of the vehicle traveling in the center lane, the left side of the vehicle traveling in the center lane, the rear of the vehicle traveling in the center-left lane, and the right side of the vehicle traveling in the center-left lane are included in the image. The recognition device recognizes at least both the vehicle traveling in the center lane and the vehicle traveling in the center-right lane from the image that includes the rear of the vehicle traveling in the center lane, the right side of the vehicle traveling in the center lane, the rear of the vehicle traveling in the center-right lane, and the left side of the vehicle traveling in the center-right lane. The recognition device recognizes at least both the vehicle traveling in the center lane and the vehicle traveling in the center left lane from an image including the rear of the vehicle traveling in the center lane, the left side of the vehicle traveling in the center lane, the rear of the vehicle traveling in the center left lane, and the right side of the vehicle traveling in the center left lane.With this configuration, the image processing device for a lean-mounted vehicle can be adapted to a usage scenario in which, while a lean-mounted vehicle is traveling in the right side area of ​​the center lane of a five-lane road, the right side of the vehicle traveling in the center lane and the left side of the vehicle traveling in the center-right lane are side by side in front of the lean-mounted vehicle, between the rear of the vehicle traveling in the center lane and the rear of the vehicle traveling in the center-right lane, and also, while a lean-mounted vehicle is traveling in the right side area of ​​the center lane of a five-lane road, the right side of the vehicle traveling in the center lane and the left side of the vehicle traveling in the center-right lane are side by side in front of the lean-mounted vehicle.As described above, it is possible to provide an image processing device for a lean-mounted vehicle that is more adaptable to situations in which a lean-mounted vehicle, which has a slimmer body than an automobile, travels in the right and left side areas of a lane while traveling in the same lane as an automobile, and that is highly mountable on a lean-mounted vehicle, which has a slimmer body than an automobile.

[0015] (3) An image processing device for a lean vehicle according to one embodiment of the present invention may have the following configuration in addition to the configuration of (1) or (2) above: The image processing device for a lean vehicle further includes an alarm device that outputs an alarm urging the driver to pay attention to a vehicle recognized by the recognition device from the image captured by the imaging device, and the alarm device is configured not to output the alarm even if the lean vehicle travels straight toward and approaches the center between two vehicles when two vehicles are traveling straight ahead of the lean vehicle side by side with a gap between them in the lateral direction and the size of the gap is too small for the width of the vehicles but is large enough for the width of the lean vehicle, and is configured to output the alarm when the lean vehicle travels straight toward and approaches the center between two vehicles when two vehicles are traveling straight ahead of the lean vehicle side by side with a gap between them in the lateral direction and the size of the gap is too small for the width of the lean vehicle, and the alarm output corresponds to the width of the lean vehicle.

[0016] According to this configuration, the lean-vehicle image processing device further includes an alarm device that outputs an alarm to prompt the driver to pay attention to the vehicle recognized by the recognition device from the image captured by the imaging device. The alarm device is configured not to output an alarm even if the lean-vehicle moves straight toward and approaches the center between two vehicles when two vehicles are traveling straight ahead of the lean-vehicle with a gap between them in the left and right directions and the gap is large enough to fit the width of the vehicles but large enough to fit the width of the lean-vehicle. The alarm device is configured to output an alarm when the lean-vehicle moves straight toward and approaches the center between two vehicles when two vehicles are traveling straight ahead of the lean-vehicle with a gap between them in the left and right directions and the gap is large enough to fit the width of the lean-vehicle. In this way, the alarm device outputs an alarm according to the width of the lean-vehicle. For example, when a lean-to vehicle is traveling in the right side area of ​​the center lane of a five-lane road, two vehicles traveling in the center lane and the center-right lane will be located in front of the lean-to vehicle. For example, when a lean-to vehicle is traveling in the right side area of ​​the center lane, a portion of the vehicle traveling in the center lane is located on the left side area of ​​the center lane and a portion of the vehicle traveling in the center-right lane is located on the right side area of ​​the center-right lane, and a portion of the vehicle traveling in the center lane is located on the right side area of ​​the center-right lane. The presence or absence of an alarm can be different depending on the situation. For example, when a lean-to vehicle is traveling in the left side area of ​​the center lane of a five-lane road, two vehicles traveling in the center lane and the center-left lane will be located in front of the lean-to vehicle.For example, when a lean-to vehicle is traveling in the left side area of ​​the center lane, a portion of the vehicle traveling in the center lane is located in the right side area and a portion of the vehicle traveling in the center-left lane is located in the left side area of ​​the center-left lane, and when a portion of the vehicle traveling in the center lane is located in the left side area of ​​the center-left lane, and when a portion of the vehicle traveling in the center lane is located in the left side area of ​​the center-left lane, and a portion of the vehicle traveling in the center-left lane is located in the right side area of ​​the center-left lane, the presence or absence of an alarm can be different. With the above configuration, the image processing device for a lean-to vehicle can capture and recognize two vehicles located in front of the lean-to vehicle, whether the lean-to vehicle is traveling in the right side area or the left side area, and output an alarm according to the vehicle width of the lean-to vehicle. Therefore, when a lean-to vehicle, which has a slimmer body than a car, is traveling in the same lane as a car, the image capture, recognition, and alarm can be adapted to situations where the lean-to vehicle is traveling in the right side area or the left side area of ​​the lane. As a result, it is possible to provide an image processing device for a lean vehicle that is more adaptable to situations in which a lean vehicle, which has a slimmer body than an automobile, travels in the right side area and left side area of ​​a lane while traveling in the same lane as an automobile, and that is highly mountable in a lean vehicle, which has a slimmer body than an automobile. Note that the image processing device for a lean vehicle may be an image processing device dedicated to lean vehicles in which the warning device is configured in advance according to the width of the lean vehicle. The image processing device for a lean vehicle may also be an image processing device for both lean vehicles and automobiles in which the warning device is configured to adapt to the width of the lean vehicle or the width of an automobile by switching. The switching may be performed by a user operation or automatically based on a captured image.

[0017] (4) An image processing device for a lean vehicle according to one embodiment of the present invention may have the following configuration in addition to at least one of the configurations (1) to (3) above: When the lean vehicle is traveling in the right side area or the left side area of ​​the center lane of the five-lane road, and the five automobiles are traveling in a line in the left-right direction ahead of the lean vehicle, and the longitudinal distance between the five automobiles and the lean vehicle is a distance necessary to capture an image so that the image processing is completed when the TTC is 3 seconds, assuming that the time from image capture to completion of image processing is 0.5 seconds and that an object approaches the lean vehicle at a relative speed of 15 km / h, the recognition device is further configured as follows: The recognition device is configured to process, when the image captured while the lean vehicle is traveling in the right side area of ​​the center lane of the five-lane road includes an image of at least one of the vehicle traveling in the center left lane and the vehicle traveling in the rightmost lane, the image of at least one of the vehicle traveling in the center left lane and the vehicle traveling in the rightmost lane at a resolution lower than the resolution used to recognize the vehicle traveling in the center lane and the vehicle traveling in the center right lane, and when the image captured while the lean vehicle is traveling in the left side area of ​​the center lane of the five-lane road includes an image of at least one of the vehicle traveling in the center right lane and the vehicle traveling in the leftmost lane, the image of at least one of the vehicle traveling in the center right lane and the vehicle traveling in the leftmost lane is configured to process, when the image captured while the lean vehicle is traveling in the left side area of ​​the center lane of the five-lane road includes an image of at least one of the vehicle traveling in the center right lane and the vehicle traveling in the leftmost lane, the image of at least one of the vehicle traveling in the center lane and the vehicle traveling in the center left lane at a resolution lower than the resolution used to recognize the vehicle traveling in the center lane and the vehicle traveling in the center left lane.

[0018] The width of a lean vehicle is very small compared to the width of an automobile. In other words, the body of a lean vehicle is slimmer than that of an automobile. The width of a lean vehicle is very small compared to the width of a lane. In other words, when a lean vehicle and an automobile are traveling in the same lane, the lean vehicle may travel in the right or left side area of ​​the lane. When a lean vehicle is traveling in the right side area of ​​the center lane of a five-lane road, the automobile traveling in the center left lane is located one car length to the left of the lean vehicle, and the automobile traveling in the rightmost lane is located one car length to the right of the lean vehicle. When a lean vehicle is traveling in the left side area of ​​the center lane of a five-lane road, the automobile traveling in the center right lane is located one car length to the right of the lean vehicle, and the automobile traveling in the leftmost lane is located one car length to the left of the lean vehicle. The recognition device is configured to process at least one of the images of a vehicle traveling in the center left lane and a vehicle traveling in the rightmost lane at a resolution lower than the resolution used to recognize the vehicle traveling in the center lane and the center right lane, when an image captured while the leaned vehicle is traveling in the right side area of ​​the center lane of a five-lane road includes at least one of an image of a vehicle traveling in the center left lane and a vehicle traveling in the rightmost lane.The recognition device is configured to process at least one of the images of a vehicle traveling in the center right lane and a vehicle traveling in the leftmost lane at a resolution lower than the resolution used to recognize the vehicle traveling in the center lane and the center left lane, when an image captured while the leaned vehicle is traveling in the left side area of ​​the center lane of a five-lane road includes at least one of an image of a vehicle traveling in the center right lane and a vehicle traveling in the leftmost lane. With this configuration, even if the lean vehicle is traveling in the right side area or the left side area, it is possible to recognize vehicles that are located in an area located more than one vehicle's width away from the lean vehicle in the left-right direction by processing them at a lower resolution, thereby reducing the size of the image of the object to be recognized by the recognition device.As a result, the processing load of the recognition device can be reduced, and for example, the heat dissipation structure of the recognition device can be made simple and compact, allowing the recognition device to be made compact.As a result, it is possible to provide an image processing device for a lean vehicle that is adaptable to scenes in which a lean vehicle, which has a slimmer body than an automobile, travels in the right side area and the left side area of ​​the lane while traveling in the same lane as the automobile, and that is more easily installable in a lean vehicle, which has a slimmer body than an automobile.

[0019] In the present invention and embodiments, a lean vehicle may be a vehicle having a body frame that leans leftward when turning left and leans rightward when turning right. In the present invention and embodiments, a lean vehicle may be, for example, a motorcycle or a three-wheeled motor vehicle. Motorcycles include scooters, mopeds, and motorized bicycles. The width of a lean vehicle is, for example, less than 1 meter. On the other hand, the width of a car is, for example, 2 to 2.5 meters.

[0020] In the present invention and embodiments, the monocular camera has a single imaging element. An imaging device including a monocular camera has an optical device such as a lens and a single imaging element. An imaging device including a monocular camera may also have a single lens. The imaging element may be an image sensor such as a charge coupled device (CCD) sensor or a complementary metal oxide semiconductor (CMOS) sensor. In the present invention and embodiments, the imaging device may be disposed in the center of the vehicle in the left-right direction. However, the location of the imaging device is not limited to this.

[0021] In the present invention and embodiments, the recognition device includes a processor and a memory. The memory includes, for example, a random access memory (RAM). The memory may also include a read-only memory (ROM). The processor includes any circuit capable of processing images, such as a field-programmable gate array (FPGA), a graphics processing unit (GPU), a central processing unit (CPU), or a system on chip (SoC). An SoC has a configuration in which the processor and memory are integrated.

[0022] In the present invention and embodiments, when the recognition device recognizes an object (e.g., an automobile), it means that the recognition device detects the object (e.g., an automobile). In the present invention and embodiments, when the recognition device processes an image of an automobile, it includes, for example, the recognition device recognizing the automobile from the image.

[0023] In the present invention and its embodiments, a lane refers to a driving lane. A lane may or may not have marks (e.g., white lines) indicating the edges of the lane. The width of a lane is, for example, 3.5 to 4 m. In the present invention and its embodiments, the right side area, left side area, and center area correspond to the three areas obtained by dividing a lane into thirds in the left-right direction. In the present invention and its embodiments, a five-lane road is a straight road.

[0024] In the present invention and embodiments, a lean vehicle traveling in the right side area of ​​a lane means that the lean vehicle travels so that the left-right center of the lean vehicle is located within the right side area. A lean vehicle traveling in the left side area of ​​a lane means that the lean vehicle travels so that the left-right center of the lean vehicle is located within the left side area. In the present invention, a lean vehicle traveling in the right side area (left side area) of a lane includes traveling so that the entire lean vehicle is located within the right side area (left side area) in a plan view. In the present invention, a lean vehicle traveling in the right side area (left side area) of a lane may also include traveling so that part of the lean vehicle is located outside the right side area (left side area).

[0025] In this invention, the configuration of the imaging device and the recognition device is defined using a situation in which the longitudinal distance between the five automobiles and the lean vehicle is the distance required to capture an image so that image processing is completed at a TTC of 3 seconds, assuming that the time from image capture to completion of image processing is 0.5 seconds and that the object approaches the lean vehicle at a relative speed of 15 km / h. "The distance required to capture an image so that image processing is completed at a TTC of 3 seconds, assuming that the time from image capture to completion of image processing is 0.5 seconds and that the object approaches the lean vehicle at a relative speed of 15 km / h" can be rephrased as follows: That is, this distance is the distance required to capture an image so that image processing is completed at a TTC of 3 seconds, assuming that the lean vehicle image processing device is configured to perform image processing so that the time from image capture to completion of image processing is 0.5 seconds and that the object approaches the lean vehicle at a relative speed of 15 km / h. The TTC is the time it takes for the object to reach the lean vehicle. Specifically, this distance is 14.58335 m (= 15 km / h × (3.0 s + 0.5 s) ÷ 3600 × 1000). "An object approaches the lean vehicle at a relative speed of 15 km / h" means that both the object and the lean vehicle are traveling, with the lean vehicle traveling at a speed 15 km / h faster than the object, or that the object is stationary and the lean vehicle is traveling at a speed of 15 km / h. The "time from capturing an image to completing image processing" is a hypothetical time and does not need to be the same as the time actually required for image processing by the lean vehicle image processing device. The "time from capturing an image to completing image processing" may be, for example, the time from when the imaging device captures an image to when the recognition device estimates the TTC for the vehicle recognized from this image. Generally, it takes about 0.5 seconds from capturing an image to estimating the TTC. The "time from capturing an image to completing image processing" may be, for example, the time from when an image capturing device captures an image to when a command is output to an alarm device to issue an alarm urging the driver to be careful of the vehicle recognized by the recognition device from the image.

[0026] In the present invention and embodiments, "the rear of the automobile is included in the image" means that the entire rear of the automobile is included in the image. In the present invention and embodiments, "the right side of the automobile is included in the image" means that, if the right side of the automobile is flat, the entire right side of the automobile is included in the image. In other words, "the right side of the automobile is included in the image" means that at least a portion of the right side of the automobile is included in the image. The meaning of "the left side of the automobile is included in the image" can also be interpreted in a similar way to this definition. In the present invention and embodiments, "included in the image" means appearing in the image.

[0027] In the present invention and embodiments, a vehicle traveling ahead of a lean vehicle refers to a vehicle located ahead of the lean vehicle when the forward direction of the lean vehicle is defined as the forward direction. In the present invention and embodiments, a vehicle traveling ahead of a lean vehicle refers to a vehicle traveling directly or almost directly in front of the lean vehicle. A vehicle traveling ahead of a lean vehicle is not limited to a vehicle traveling ahead of the lean vehicle.

[0028] In the present invention and its embodiments, when the size of the gap between two vehicles is too small in the lateral direction for the vehicle width but large enough in the lateral direction for the lean vehicle width, it means that the size of the gap is such that, under normal traffic conditions, the vehicle cannot pass through the gap, but the lean vehicle can. In other words, it means that the size of the gap is larger than the width of the lean vehicle and smaller than or equal to the width of the vehicle. Therefore, cases in which the size of the gap between two vehicles is too small in the lateral direction for the vehicle width but large enough in the lateral direction for the lean vehicle width include cases in which the size of the gap between two vehicles is smaller than the width of the vehicle but larger than the width of the lean vehicle. In the present invention and its embodiments, when the size of the gap between two vehicles is too small in the lateral direction for the lean vehicle width, it means that the size of the gap is such that, under normal traffic conditions, the lean vehicle cannot pass through the gap. In other words, it means that the size of the gap is smaller than or equal to the width of the lean vehicle. Therefore, cases where the distance between two vehicles is too large for the width of the vehicles to allow for sufficient clearance in the left-right direction include cases where the distance between the two vehicles is smaller than the width of a lean vehicle.

[0029] The lean-vehicle image processing device of the present invention and embodiments may or may not include an alarm device that outputs an alarm urging the driver to pay attention to a vehicle recognized by a recognition device from an image captured by an imaging device. In the present invention and embodiments, if the lean-vehicle image processing device includes an alarm device, the use of the recognition result by the recognition device is not limited to the alarm device. Regardless of whether the lean-vehicle image processing device includes an alarm device, the recognition result by the recognition device may be used, for example, to control the vehicle speed of the lean-vehicle. More specifically, for example, the brake device of the lean-vehicle may be activated based on the recognition result by the recognition device without the driver's brake operation. In the present invention and embodiments, the alarm device that outputs an alarm urging the driver to pay attention to a vehicle recognized by the recognition device may also output an alarm urging the driver to pay attention to objects other than a vehicle recognized by the recognition device.

[0030] In the present invention and embodiments, the alarm output by the alarm device is an alarm to alert the driver of the lean vehicle. The alarm is an alarm that alerts at least one of the driver's hearing, sight, or touch. An auditory alarm is an alarm that uses sound. A visual alarm is, for example, an alarm that uses light. A tactile alarm is, for example, an alarm that uses vibration. The vibration may be, for example, a grip vibration or a seat vibration. The means for generating the vibration may be a dedicated means of the alarm device, or may be a brake or a drive source.

[0031] In the present invention and embodiments, low resolution means low pixel density. In the present invention and embodiments, processing an image of a vehicle at a predetermined resolution includes recognizing an image of a vehicle at the predetermined resolution.

[0032] The situation in which the lean-vehicle image processing device of the present invention is used is not limited to a situation in which a lean-vehicle is traveling in the right or left side area of ​​a center lane of a five-lane road. The lean-vehicle image processing device may also be used in the following driving situations, for example. The driving situation may be, for example, a situation in which a lean-vehicle is traveling on a road other than a five-lane road. A road other than a five-lane road may be a road with six or more lanes, a two-lane road, or a road with one lane in each direction. The driving situation may also be, for example, a situation in which a lean-vehicle is traveling in the center area of ​​a certain lane. The driving situation may also be, for example, a situation in which a lean-vehicle is traveling in the right or left side area of ​​a lane other than the center lane of a five-lane road.

[0033] In the present invention and embodiments, "at least one (one) of a plurality of options" includes all possible combinations of the plurality of options. "At least one (one) of a plurality of options" may be any one of the plurality of options or all of the plurality of options. For example, "at least one of A, B, and C" may be only A, only B, only C, A and B, A and C, B and C, or A, B, and C.

[0034] In this specification, for example, "1 to 10" means 1 or more and 10 or less. The same definition applies to numbers other than 1 and 10.

[0035] In the present invention and embodiments, the words including, comprising, having, and their derivatives are used herein to encompass additional items in addition to the listed items and equivalents thereof.

[0036] Unless otherwise defined, all terms (including technical and scientific terms) used in the present specification and claims have the same meaning as commonly understood by those skilled in the art to which this invention belongs. Terms, such as those defined in commonly used dictionaries, should be interpreted to have a meaning consistent with the meaning in the context of the relevant technology and this disclosure, and should not be interpreted in an idealized or overly formal sense.

[0037] In this specification, the term "may" is non-exclusive. "may" means "may, but is not limited to." In this specification, "may" implicitly includes the possibility that "may not." In this specification, a configuration described as "may" at least achieves the above-mentioned effect obtained by the configuration of claim 1.

[0038] Before describing embodiments of the present invention in detail, it is to be understood that the invention is not limited to the details of construction and the arrangement of components set forth in the following description or illustrated in the drawings. The present invention is capable of embodiments other than those described below. The present invention is also capable of embodiments incorporating various variations of the embodiments described below.

[0039] According to the present invention, an image processing device for a lean vehicle can be provided that can adapt to situations in which a lean vehicle, which has a slimmer body than an automobile, travels in the right side area and left side area of ​​a lane while traveling in the same lane as an automobile, and that is highly mountable on a lean vehicle, which has a slimmer body than an automobile.

[0040] Fig. 1 is a diagram for explaining the configuration of an image processing device for a lean vehicle according to a first embodiment of the present invention; Fig. 2 is a diagram for explaining the configuration of an image processing device for a lean vehicle according to a second embodiment of the present invention; Fig. 3 is a diagram for explaining the configuration of an image processing device for a lean vehicle according to a third embodiment of the present invention; Fig. 4 is a diagram for explaining the configuration of an image processing device for a lean vehicle according to a fourth embodiment of the present invention; Fig. 5 is a diagram for explaining three examples of recognition by the image processing device for a lean vehicle according to the fourth embodiment of the present invention;

[0041] Arrows F, Re, L, and R shown in the drawings represent the forward direction, the backward direction, the left direction, and the right direction, respectively.

[0042] First Embodiment An image processing device 10 for a lean vehicle according to an embodiment of the present invention will now be described with reference to Fig. 1. The image processing device 10 for a lean vehicle is mounted on a lean vehicle 1. In Fig. 1, the lean vehicle 1 is a motorcycle, but the lean vehicle 1 is not limited to motorcycles. The image processing device 10 for a lean vehicle includes an imaging device 11 that captures images and a recognition device 12 that recognizes objects from the images acquired by the imaging device 11. The imaging device 11 includes a monocular camera.

[0043] FIG. 1 includes plan views showing two examples (CASE 1 and CASE 2) of a situation in which a lean vehicle 1 and five automobiles 61 to 65 are traveling on a five-lane road. The five-lane road includes a left-most lane 51, a center-left lane 52, a center lane 53, a center-right lane 54, and a right-most lane 55. In CASE 1 and CASE 2, the lean vehicle 1 travels in the center lane 53, and the five automobiles 61 to 65 travel in a line in the left-right direction ahead of the lean vehicle 1. In CASE 1, the lean vehicle 1 travels in a right-side area 50R of the center lane 53, and in CASE 2, the lean vehicle 1 travels in a left-side area 50L of the center lane 53. The right side area 50R is an area including the right end of a single lane 50 (e.g., center lane 53) when the single lane 50 is divided into thirds in the left-right direction, and the left side area 50L is an area including the left end of the single lane 50. The area between the right side area 50R and the left side area 50L is a center area 50C. The width of the lean vehicle 1 is very small compared to the width of an automobile 60 (e.g., automobiles 61-65). The width of the lean vehicle 1 is very small compared to the width of the lane 50. Therefore, the lean vehicle 1 has a high degree of freedom in its left-right traveling position within the center lane 53, and can move left and right within the center lane 53, and can travel in either the right side area 50R or the left side area 50L within the center lane 53.

[0044] In Cases 1 and 2, the distance D in the longitudinal direction between the five automobiles 61-65 and the lean vehicle 1 is the distance required to capture an image so that image processing is completed at a time to collision (TTC) of 3 seconds, assuming that the time Ti from capturing an image to completing image processing is 0.5 seconds and that the object approaches the lean vehicle 1 at a relative speed Vr of 15 km / h. This distance is approximately 15 m. Note that, hereinafter, this distance may be referred to as the "specific distance."

[0045] When a lean vehicle 1 is traveling in the right side area 50R or left side area 50L of the center lane 53 of a five-lane road, and five automobiles 61 to 65 are traveling in a line in the left-right direction ahead of the lean vehicle 1, and the distance D in the fore-and-aft direction between the five automobiles 61 to 65 and the lean vehicle 1 is the specific distance described above, the imaging device 11 and the recognition device 12 have the following configuration.

[0046] The monocular camera of the imaging device 11 is configured with a left-right angle of view so that, while the lean-to vehicle 1 is traveling in the right side area 50R of the center lane 53, the left edge 61bl of the rear of the automobile 61 traveling in the leftmost lane 51 is not captured, but at least the left edge 63bl of the rear of the automobile 63 traveling in the center lane 53 and the right edge 64br of the rear of the automobile 64 traveling in the center right lane 54 are captured. The monocular camera of the imaging device 11 is configured with a left-right angle of view so that, while the lean-to vehicle 1 is traveling in the left side area 50L of the center lane 53, the right edge 65br of the rear of the automobile 65 traveling in the rightmost lane 55 is not captured, but at least the right edge 63br of the rear of the automobile 63 traveling in the center lane 53 and the left edge 63bl of the rear of the automobile 62 traveling in the center left lane 52 are captured.

[0047] The recognition device 12 is configured to recognize at least both the vehicle 63 traveling in the center lane 53 and the vehicle 64 traveling in the center right lane 54 from an image captured while the lean vehicle 1 is traveling in the right side area 50R of the center lane 53. At this time, the recognition device 12 recognizes at least the vehicle 63 and the vehicle 64 from an image captured by the imaging device 11 when the distance D is the specific distance described above. In other words, the recognition device 12 recognizes at least the vehicle 63 and the vehicle 64 from an image that does not include the left edge 61bl of the rear of the vehicle 61 traveling in the leftmost lane 51, but does include the left edge 63bl of the rear of the vehicle 63 traveling in the center lane 53 and the right edge 64br of the rear of the vehicle 64 traveling in the center right lane 54. The recognition device 12 is configured to recognize at least both the vehicle 63 traveling in the center lane 53 and the vehicle 62 traveling in the center-left lane 52 from an image captured while the lean vehicle 1 is traveling in the left side area 50L of the center lane 53. At this time, the recognition device 12 recognizes at least the vehicle 63 and the vehicle 62 from an image captured by the imaging device 11 when the distance D is the specific distance described above. In other words, the recognition device 12 recognizes at least the vehicle 63 and the vehicle 62 from an image that does not include the right edge 65br of the rear of the vehicle 65 traveling in the rightmost lane 55, but includes the right edge 63br of the rear of the vehicle 63 traveling in the center lane 53 and the left edge 63bl of the rear of the vehicle 62 traveling in the center-left lane 52.

[0048] 1, the angle θmin is a schematic representation of the minimum horizontal angle of view of the monocular camera of the imaging device 11, and the angle θmax is a schematic representation of the maximum horizontal angle of view of the monocular camera of the imaging device 11, for conceptual understanding.

[0049] In FIG. 1, the symbols "o", "△", and "x" attached to the automobiles 61 to 65 have the following meanings: The symbol "o" indicates that the entire rear of the automobile is captured. The symbol "x" indicates that the rear of the automobile is not captured at all, or only a portion of the rear of the automobile is captured. The symbol "△" indicates that the rear of the automobile is not captured at all, only a portion of the rear of the automobile is captured, or the entire rear of the automobile is captured.

[0050] According to this embodiment, an image processing device 10 for a lean vehicle can be realized that can be adapted to scenes in which a lean vehicle 1, which has a slimmer body than an automobile 60, is traveling in the right side area 50R and left side area 50L of the lane 50 while traveling in the same lane 50 as the automobile 60, and that is highly mountable on a lean vehicle 1, which has a slimmer body than an automobile 60.

[0051] The lean vehicle 1 or the lean vehicle image processing device 10 may or may not have a display device that displays the image captured by the imaging device 11. When the image is displayed, the object (e.g., a car) recognized by the recognition device 12 may be surrounded by a frame (see FIG. 4 described later).

[0052] Second Embodiment Next, an image processing device 10 for a lean vehicle according to a second embodiment of the present invention will be described with reference to FIG. 2. The second embodiment has all of the configurations of the first embodiment. FIG. 2 is a diagram schematically illustrating an example of a situation in which a lean vehicle 1 is traveling in a right side area 50R of a center lane 53 of a five-lane road (CASE 3) and an example of a situation in which the lean vehicle 1 is traveling in a left side area 50L of the center lane 53 (CASE 4). CASE 3 is an example of CASE 1 of the first embodiment. CASE 4 is an example of CASE 2 of the first embodiment.

[0053] When a lean vehicle 1 is traveling in the right side area 50R or left side area 50L of the center lane 53 of a five-lane road, and five automobiles 61 to 65 are traveling in a line in the left-right direction ahead of the lean vehicle 1, and the longitudinal distance D between the five automobiles 61 to 65 and the lean vehicle 1 is the specific distance described in the first embodiment, the imaging device 11 and the recognition device 12 may have the following configuration.

[0054] 2, while the lean-to vehicle 1 is traveling in the right side area 50R of the center lane 53, the monocular camera of the imaging device 11 captures images so that at least a rear surface 63b of the automobile 63 traveling in the center lane 53, a right side surface 63r of the automobile 63 traveling in the center lane 53, a rear surface 64b of the automobile 64 traveling in the center-right lane 54, and a left side surface 64l of the automobile 64 traveling in the center-right lane 54 are included in the image. As shown in CASE 4 of FIG. 2, while the lean-to vehicle 1 is traveling in the left side area 50L of the center lane 53, the monocular camera of the imaging device 11 captures images so that at least a rear surface 63b of the automobile 63 traveling in the center lane 53, a left side surface 63l of the automobile 63 traveling in the center lane 53, a rear surface 62b of the automobile 62 traveling in the center-left lane 52, and a right side surface 62r of the automobile 62 traveling in the center-left lane 52 are included in the image.

[0055] The recognition device 12 recognizes at least both the vehicle 63 traveling in the center lane 53 and the vehicle 64 traveling in the center right lane 54 from an image including the rear surface 63b of the vehicle 63 traveling in the center lane 53, the right side surface 63r of the vehicle 63 traveling in the center lane 53, the rear surface 64b of the vehicle 64 traveling in the center right lane 54, and the left side surface 64l of the vehicle 64 traveling in the center right lane 54. The recognition device 12 recognizes at least both the vehicle 63 traveling in the center lane 53 and the vehicle 62 traveling in the center left lane 52 from an image including the rear surface 63b of the vehicle 63 traveling in the center lane 53, the left side surface 63l of the vehicle 63 traveling in the center lane 53, the rear surface 62b of the vehicle 62 traveling in the center left lane 52, and the right side surface 62r of the vehicle 62 traveling in the center left lane 52.

[0056] Whether or not the right side 63r of the automobile 63 traveling in the center lane 53 is included in the image while the lean vehicle 1 is traveling in the right side area 50R of the center lane 53 is affected by the left-right position of the lean vehicle 1 in the right side area 50R of the center lane 53, the left-right position of the automobile 63 traveling in the center lane 53 in the center lane 53, the width of the center lane 53, the width of the automobile 63, the width of the lean vehicle 1, and the left-right position of the imaging device 11 on the lean vehicle 1. Therefore, the imaging device 11 of the second embodiment does not necessarily have the above-described configuration when the lean vehicle 1 is traveling in the right side area 50R of the center lane 53 and the distance D is the specific distance described above, and may not have the above-described configuration. The same applies when the lean vehicle 1 is traveling in the left side area 50L of the center lane 53.

[0057] Third Embodiment Next, a lean vehicle image processing device 10 according to a third embodiment of the present invention will be described with reference to Fig. 3. The third embodiment has all of the configurations of the first embodiment. The third embodiment may also have the configuration of the second embodiment.

[0058] 3, the lean vehicle image processing device 10 further includes a warning device 13 that outputs a warning to urge the driver to pay attention to the vehicle 60 recognized by the recognition device 12 from the image captured by the imaging device 11. The warning device 13 outputs a warning according to the vehicle width of the lean vehicle 1, as will be described below.

[0059] As shown in CASE 5 of FIG. 3 , when two automobiles 60 are traveling straight ahead of the lean vehicle 1, side by side with a gap G in the left-right direction, and the size of the gap G is too small for the width of the automobiles 60 but large enough for the width of the lean vehicle 1, the warning device 13 is configured not to output an alarm even if the lean vehicle 1 travels straight towards and approaches the center between the two automobiles 60. As shown in CASE 6 of FIG. 3 , when two automobiles 60 are traveling straight ahead of the lean vehicle 1, side by side with a gap G in the left-right direction, and the size of the gap G is too small for the width of the lean vehicle 1, the warning device 13 is configured to output an alarm when the lean vehicle 1 travels straight towards and approaches the center between the two automobiles 60. Note that in CASE 6 of FIG. 3 , the gap G is slightly larger than the width of the lean vehicle 1, but may be smaller than the width of the lean vehicle 1.

[0060] 3, when two automobiles 60 are traveling straight ahead of an automobile 66 (60) having an alarm device, side by side with a gap G in the left-right direction, and the size of the gap G is too small for the width of the automobiles 66 but is large enough for the width of the lean-to vehicle 1, the alarm device of the automobile 66 will output an alarm when the automobile 66 travels straight towards and approaches the center between the two automobiles 60. Although not shown, when two automobiles 60 are traveling straight ahead of an automobile 66 having an alarm device, side by side with a gap G in the left-right direction, and the size of the gap G is too small for the width of both the automobile 66 and the lean-to vehicle 1, the alarm device of the automobile 66 will output an alarm when the automobile 66 travels straight towards and approaches the center between the two automobiles 60.

[0061] EXAMPLE 1 and EXAMPLE 2 in Fig. 3 are specific examples of CASE 5. EXAMPLE 3 and EXAMPLE 4 in Fig. 3 are specific examples of CASE 6.

[0062] In EXAMPLE 1 and EXAMPLE 3, a lean vehicle 1 is traveling in a right side area 50R of a center lane 53 of a five-lane road, and two automobiles 63, 64 traveling in the center lane 53 and center right lane 54 are located in front of the lean vehicle 1. In EXAMPLE 1, a portion of the automobile 63 is located on the left side area 50L of the center lane 53, and a portion of the automobile 64 is located on the right side area 50R of the center right lane 54. In EXAMPLE 3, a portion of the automobile 63 is located on the right side area 50R of the center lane 53, and a portion of the automobile 64 is located on the left side area 50L of the center right lane 54.

[0063] In EXAMPLE 2 and EXAMPLE 4, the lean vehicle 1 is traveling in the left side area 50L of the center lane 53 of a five-lane road, and two automobiles 63, 62 traveling in the center lane 53 and center-left lane 52 are located in front of the lean vehicle 1. In EXAMPLE 2, a portion of the automobile 63 is located on the right side area 50R of the center lane 53, and a portion of the automobile 62 is located on the left side area 50L of the center-left lane 52. In EXAMPLE 4, a portion of the automobile 63 is located on the left side area 50L of the center lane 53, and a portion of the automobile 62 is located on the right side area 50R of the center-left lane 52.

[0064] In the third embodiment, if the lean vehicle 1 has a display device that displays an image captured by the imaging device 11, and a frame surrounding the vehicle in the image captured by the imaging device 11 is displayed together with the captured image, the color of the frame surrounding the vehicle that is the subject of the alarm may be different when the alarm device 13 outputs an alarm and when the alarm device 13 does not output an alarm.

[0065] Fourth Embodiment Next, a lean vehicle image processing device 10 according to a fourth embodiment of the present invention will be described with reference to FIGS. 4 and 5. The fourth embodiment has all of the configurations of the first embodiment. The fourth embodiment may also have the configuration of the second embodiment. The fourth embodiment may also have the configuration of the third embodiment.

[0066] 4 includes plan views showing two examples (CASE 7 and CASE 8) of a situation in which the lean vehicle 1 and five automobiles 61 to 65 are traveling on a five-lane road. CASE 7 is an example of CASE 1 in the first embodiment. CASE 8 is an example of CASE 2 in the first embodiment. FIG. 4 includes an example of an image P1 captured by the imaging device 11 in CASE 7, and an example of an image P2 captured by the imaging device 11 in CASE 8.

[0067] When a lean vehicle 1 is traveling in the right side area 50R or left side area 50L of the center lane 53 of a five-lane road, and five automobiles 61 to 65 are traveling in a line in the left-right direction ahead of the lean vehicle 1, and the longitudinal distance D between the five automobiles 61 to 65 and the lean vehicle 1 is the specific distance described in the first embodiment, the recognition device 12 has the following configuration.

[0068] When an image P1 captured while the lean vehicle 1 is traveling in the right side area 50R of the center lane 53 includes an image of at least one of a vehicle 62 traveling in the center left lane 52 and a vehicle 65 traveling in the rightmost lane 55, the recognition device 12 is configured to process the image of at least one of the vehicle 62 traveling in the center left lane 52 and the vehicle 65 traveling in the rightmost lane 55 at a resolution lower than the resolution used to recognize the vehicle 63 traveling in the center lane 53 and the vehicle 64 traveling in the center right lane 54. When an image P2 captured while the lean vehicle 1 is traveling in the left side area 50L of the center lane 53 includes an image of at least one of a vehicle 64 traveling in the center right lane 54 and a vehicle 61 traveling in the leftmost lane 51, the recognition device 12 is configured to process the image of at least one of the vehicle 64 traveling in the center right lane 54 and the vehicle 61 traveling in the leftmost lane 51 at a resolution lower than the resolution used to recognize the vehicle 63 traveling in the center lane 53 and the vehicle 62 traveling in the center left lane 52.

[0069] In Fig. 4, image P1 includes images of both automobiles 62 and 65, but may include images of only one of automobiles 62 and 65. In Fig. 4, image P2 includes images of both automobiles 64 and 61, but may include images of only one of automobiles 64 and 61. In Fig. 4, images P1 and P2 are images captured by imaging device 11 having a horizontal angle of view slightly smaller than the maximum horizontal angle of view θmax.

[0070] In Figure 4, the symbols "Hi" and "Lo" attached to the cars 61 to 65 have the following meanings: A car with the symbol "Hi" attached is a car that is recognized with high resolution. A car with the symbol "Lo" attached is a car that is recognized with low resolution.

[0071] Next, details of the processing by the recognition device 12 when an image P1 captured while the lean vehicle 1 is traveling in the right side area 50R of the center lane 53 includes an image of at least one of a vehicle 62 traveling in the center left lane 52 and a vehicle 65 traveling in the rightmost lane 55 will be described with reference to Figures 5(a) to 5(c). Note that when an image P2 captured while the lean vehicle 1 is traveling in the left side area 50L of the center lane 53 includes an image of at least one of a vehicle 64 traveling in the center right lane 54 and a vehicle 61 traveling in the leftmost lane 51, the processing by the recognition device 12 is reversed left and right.

[0072] As shown in Figures 5(a) to 5(c), the recognition device 12 cuts out a predetermined area from the image P1 that includes an image of a vehicle 63 traveling in the center lane 53 and an image of a vehicle 64 traveling in the center-right lane 54. The cut-out image is referred to as a center area image P1C. The center area image P1C is in the center or approximately the center of the left-right direction of the image P1. The center area image P1C is in the center or approximately the center of the up-down direction of the image P1. The area to be cut out may be set in advance or may be changeable.

[0073] Next, the recognition device 12 recognizes the automobiles 63 and 64 from the central region image P1C at a predetermined first resolution. The first resolution may be the same as or lower than the resolution of the original image P1 captured by the imaging device 11.

[0074] The recognition device 12 also sets a peripheral region image P1S, which is different from the central region image P1C, from the image P1. The peripheral region image P1S may be the entire image P1, as shown in FIG. 5(a). The peripheral region image P1S may be an image cut out from the image P1 so as not to include the central region image P1C, as shown in FIGS. 5(b) and 5(c), for example. The peripheral region image P1S shown in FIG. 5(b) is an image obtained by removing only the central region image P1C from the image P1. The peripheral region image P1S shown in FIG. 5(c) is an image obtained by removing from the image P1 an image in the same left-right direction as the central region image P1C.

[0075] Next, the recognition device 12 processes the surrounding area image P1S at a second resolution lower than the first resolution. Specifically, the recognition device 12 first reduces the resolution of the surrounding area image P1S to the second resolution lower than the first resolution. Then, the recognition device 12 recognizes at least one of the vehicle 62 traveling in the center left lane 52 and the vehicle 65 traveling in the rightmost lane 55 from the surrounding area image P1S. In the example of Figures 5(a) to (c), the recognition device 12 recognizes both the vehicle 62 and the vehicle 65 from the surrounding area image P1S.

[0076] 5( a), the recognition device 12 recognizes, from the surrounding area image P1S, not only the automobile 62 and the automobile 65, but also the automobile 63 traveling in the center lane 53 and the automobile 64 traveling in the center right lane 54. That is, the automobile 63 traveling in the center lane 53 and the automobile 64 traveling in the center right lane 54 are recognized using both the center area image P1C and the surrounding area image P1S. In this case, the recognition device 12 may use the results of recognition from the center area image P1C as final data for recognizing the automobiles 63 and 64.

[0077] 5(b) and 5(c), the surrounding area image P1S includes a portion of the automobile 64 traveling in the center right lane 54. In this case, the recognition device 12 may or may not recognize the automobile 64 traveling in the center right lane 54 from the surrounding area image P1S.

[0078] The right side area in this specification is included in the right driving area in the specification of Japanese Patent Application No. 2023-100003, which is the basic application of the present application. The left side area in this specification is included in the left driving area in the specification of the same application. The central area in this specification is included in the central driving area in the specification of the same application. The automobile in this specification corresponds to a four-wheeled vehicle in the specification of the same application. The central area image in this specification corresponds to the central recognition area in the specification of the same application. The peripheral area image in this specification corresponds to the peripheral recognition area in the specification of the same application.

[0079] 1: lean vehicle, 10: lean vehicle image processing device, 11: imaging device, 12: recognition device, 13: warning device, 50: lane, 50C: center area, 50L: left side area, 50R: right side area, 51: leftmost lane, 52: center left lane, 53: center lane, 54: center right lane, 55: rightmost lane, 60, 61 to 65: automobile

Claims

1. An image processing apparatus for a lean vehicle, comprising an imaging device that captures an image and a recognition device that recognizes an object from the image acquired by the imaging device, and being mounted on a lean vehicle, when a single lane is trisected in the left-right direction, an area including the right end of the single lane is defined as a right side area, an area including the left end of the single lane is defined as a left side area, and an area between the right side area and the left side area is defined as a central area, when each lane of a five-lane road is defined as the leftmost lane, the central left lane, the central lane, the central right lane, and the rightmost lane in order from left to right, when the lean vehicle travels in the right side area or the left side area of the central lane of the five-lane road, and five automobiles travel in a line in the left-right direction in front of the lean vehicle, and the distance in the front-rear direction between the five automobiles and the lean vehicle is such that when assuming that the time from imaging to completion of image processing is 0.5 seconds and an object approaches the lean vehicle at a relative speed of 15 km / h, the image processing is completed at the point in time when TTC (Time to Collision) is 3 seconds, the imaging device and the recognition device are configured as follows, and this is a characteristic of the image processing apparatus for a lean vehicle. The imaging device is when the lean vehicle travels in the right side area of the central lane of the five-lane road, the left end of the rear surface of the automobile traveling in the leftmost lane is not imaged, but at least the left end of the rear surface of the automobile traveling in the central lane and the right end of the rear surface of the automobile traveling in the central right lane are imaged, and when the lean vehicle travels in the left side area of the central lane of the five-lane road, the right end of the rear surface of the automobile traveling in the rightmost lane is not imaged, but at least the right end of the rear surface of the automobile traveling in the central lane and the left end of the rear surface of the automobile traveling in the central left lane are imaged, and includes a monocular camera configured with an angle of view in the left-right direction. The recognition device is when the lean vehicle travels in the right side area of the central lane of the five-lane road, at least both the automobile traveling in the central lane and the automobile traveling in the central right lane are recognized from the image captured, and The image captured while the lean vehicle is traveling in the left side area of the center lane of the five-lane road is configured to recognize at least both the automobile traveling in the center lane and the automobile traveling in the center left lane.

2. The lean vehicle travels in the right side area or the left side area of the center lane of the five-lane road, and the five automobiles travel in a row in the left-right direction in front of the lean vehicle, and the distance in the front-rear direction between the five automobiles and the lean vehicle is such that when assuming a time of 0.5 seconds from imaging to completion of image processing and a relative speed of an object approaching the lean vehicle at 15 km / h, the image is captured so that the image processing is completed at the time when the TTC is 3 seconds. In this case, the imaging device and the recognition device are further configured as follows. The image processing device for a lean vehicle according to claim 1. The imaging device is When the lean vehicle is traveling in the right side area of the center lane of the five-lane road, the monocular camera captures an image such that at least the rear surface of the automobile traveling in the center lane, the right side surface of the automobile traveling in the center lane, the rear surface of the automobile traveling in the center right lane, and the left side surface of the automobile traveling in the center right lane are included in the image. And When the lean vehicle is traveling in the left side area of the center lane of the five-lane road, the monocular camera captures an image such that at least the rear surface of the automobile traveling in the center lane, the left side surface of the automobile traveling in the center lane, the rear surface of the automobile traveling in the center left lane, and the right side surface of the automobile traveling in the center left lane are included in the image. The recognition device is From the image including the rear surface of the automobile traveling in the center lane, the right side surface of the automobile traveling in the center lane, the rear surface of the automobile traveling in the center right lane, and the left side surface of the automobile traveling in the center right lane, at least both the automobile traveling in the center lane and the automobile traveling in the center right lane are recognized. And At least recognize both the vehicle traveling in the center lane and the vehicle traveling in the center left lane from the image including the rear surface of the vehicle traveling in the center lane, the left side surface of the vehicle traveling in the center lane, the rear surface of the vehicle traveling in the center left lane, and the right side surface of the vehicle traveling in the center left lane.

3. The image processing apparatus for a lean vehicle further includes an alarm device that outputs an alarm to prompt the driver to pay attention to the vehicle recognized by the recognition device from the image captured by the imaging device. The alarm device is When two vehicles are traveling straight ahead side by side with a gap in the left-right direction in front of the lean vehicle, and the size of the gap has no margin in the left-right direction for the vehicle width of the vehicle, but has a margin in the left-right direction for the vehicle width of the lean vehicle, the lean vehicle is configured not to output the alarm even when traveling straight ahead and approaching the center between the two vehicles. And When two vehicles are traveling straight ahead side by side with a gap in the left-right direction in front of the lean vehicle, and the size of the gap has no margin in the left-right direction for the vehicle width of the lean vehicle, the lean vehicle is configured to output the alarm when traveling straight ahead and approaching the center between the two vehicles. Output the alarm according to the vehicle width of the lean vehicle. The image processing apparatus for a lean vehicle according to claim 1.

4. The image processing apparatus for a lean vehicle further includes an alarm device that outputs an alarm to prompt the driver to pay attention to the vehicle recognized by the recognition device from the image captured by the imaging device. The alarm device is When two vehicles are traveling straight ahead side by side with a gap in the left-right direction in front of the lean vehicle, and the size of the gap has no margin in the left-right direction for the vehicle width of the vehicle, but has a margin in the left-right direction for the vehicle width of the lean vehicle, the lean vehicle is configured not to output the alarm even when traveling straight ahead and approaching the center between the two vehicles. And In front of the lean vehicle, two automobiles are moving straight ahead side by side with a space therebetween in the left-right direction. When the size of the space is such that there is no margin in the left-right direction with respect to the vehicle width of the lean vehicle, the lean vehicle is configured to move straight ahead toward the center between the two automobiles and, when approaching, output the alarm. Output the alarm according to the vehicle width of the lean vehicle. The image processing apparatus for a lean vehicle according to claim 2.

5. When the lean vehicle travels in the right side area or the left side area of the center lane of the five-lane road, and the five automobiles travel in a line side by side in the left-right direction in front of the lean vehicle, and the distance in the front-rear direction between the five automobiles and the lean vehicle is such that, assuming a relative speed of the object approaching the lean vehicle at 15 km / h, the image is captured so that the image processing is completed at the time when the TTC is 3 seconds at the time when the time from imaging to completion of the image processing is 0.5 second, the recognition device is further configured as follows. The recognition device is When the image captured while the lean vehicle travels in the right side area of the center lane of the five-lane road includes at least one image of the automobile traveling in the center left lane and the automobile traveling in the rightmost lane, it is configured to process at least one of the images of the automobile traveling in the center left lane and the automobile traveling in the rightmost lane at a resolution lower than the resolution when recognizing the automobile traveling in the center lane and the automobile traveling in the center right lane. And When the image captured while the lean vehicle travels in the left side area of the center lane of the five-lane road includes at least one image of the automobile traveling in the center right lane and the automobile traveling in the leftmost lane, it is configured to process at least one of the images of the automobile traveling in the center right lane and the automobile traveling in the leftmost lane at a resolution lower than the resolution when recognizing the automobile traveling in the center lane and the automobile traveling in the center left lane. The image processing apparatus for a lean vehicle according to any one of claims 2 to 4.