Image processing device for lean vehicles

Abstract

In the present invention, in a case where a leaning vehicle (1) travels in a right side area (50R) of a center lane (53) of a five-lane road, and the inter-vehicle distance (D) between the leaning vehicle and five automobiles (61-65) traveling in parallel thereto is a distance which is required to complete image processing at a time point of 3 seconds to Time to Collision (TTC) on the assumption that the image processing time is 0.5 seconds and the relative speed is 15km / h: an image capture device (11) of an image processing device (10) for a leaning vehicle includes a monocular camera in which the angle of view in the left-right direction is configured such that the left end (61bl) of the rear face of an automobile (61) in the left end lane (51) is not imaged, but at least the left end (63bl) of the rear face of an automobile (3) in the center lane and the right end (64br) of the rear face of an automobile (64) in the center right lane (54) are imaged; and a recognition device (12) of the image processing device for a leaning vehicle at least recognizes the automobile in the center lane and the automobile in the center right lane.

Description

[Technical Field]

[0001] The present invention relates to an image processing device for a lean vehicle that is mounted on a lean vehicle. [Background technology]

[0002] Patent Document 1 proposes an arrival time estimation device that extracts feature points from images of an object captured in time series by an imaging device mounted on a vehicle and estimates the time it will take for the vehicle to reach the object based on the extracted feature points, and a driving assistance control device that uses the estimated arrival time to perform control to assist the driver in operating the vehicle. Patent Document 1 also proposes a lean vehicle that is equipped with an imaging device, an arrival time estimation device, and a driving assistance control device. The lean vehicle is, for example, a motorcycle or other vehicle.

[0003] The device proposed in Patent Document 1, which combines an imaging device, an arrival time estimation device, and a driving assistance control device, can be rephrased as an image processing device including an imaging device that captures images and a recognition device that processes the captured images to recognize objects. The imaging device generally includes an optical device, such as a lens, and an imaging element, such as a CMOS (Complementary Metal Oxide Semiconductor) sensor. The recognition device generally includes a processor, such as an FPGA (Field-Programmable Gate Array), a GPU (Graphics Processing Unit), a CPU (Central Processing Unit), or an SoC (System on Chip) that processes images, and a memory, such as a RAM (Random Access Memory). Note that an SoC has a configuration in which the processor and memory are integrated. Patent Document 1 also describes that the imaging device may be a monocular camera or a stereo camera.

[0004] The arrival time described in Patent Document 1 corresponds to the so-called TTC (Time to Collision). Non-Patent Document 1 describes that a TTC threshold of 3 seconds is appropriate for evaluating a potential collision.

[0005] The width of lanes and the width of vehicles are regulated by national or regional administrative bodies. In Japan, lane widths are regulated to 2.75 to 4 m depending on the road classification in accordance with Article 5, Paragraphs 4 and 5 of the Road Structure Ordinance (see Non-Patent Document 2). Generally, lane widths are 3.5 to 4 m. Furthermore, in Japan, Article 59 of the Road Transport Vehicle Safety Standards stipulates that the width of motorcycles must be 1.3 m or less, and Article 2 of the Road Transport Vehicle Safety Standards stipulates that the width of automobiles must be 2.5 m or less (see Non-Patent Document 3). Generally, the width of lean-to vehicles is less than 1 m, and the width of automobiles is 2 to 2.5 m. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent Publication No. 2021-111246 [Non-patent literature]

[0007] [Non-Patent Document 1] Oumaima Barhoumi, Mohamed H. Zaki, SOFIENE TAHAR, “A Formal Approach to Road Safety Assessment Using Traffic Conflict Techniques”, “VI. SENSITIVITY STUDY”, [online], last updated April 26, 2024, IEEE Open Journal of Vehicular Technology, [searched on June 4, 2020], Internet<URL:https: / / ieeexplore.ieee.org / stamp / stamp.jsp?arnumber=10496854> [Non-patent document 2] Road Structure Ordinance, [Retrieved June 4, 2024], Internet<URL:https: / / elaws.e-gov.go.jp / document?lawid=345CO0000000320> [Non-patent document 3] Road Transport Vehicle Safety Standards, [Retrieved June 4, 2024], Internet<URL:https: / / elaws.e-gov.go.jp / document?lawid=326M50000800067> Summary of the Invention [Problem to be solved by the invention]

[0008] The width of a lean vehicle is very small compared to the width of an automobile. In other words, the lean vehicle has a slimmer (narrower) body compared to an automobile. In addition, the width of a lean-to vehicle is very small compared to the width of a lane. Therefore, when a lean-to vehicle is traveling in a single lane, it may travel in the right or left side area of ​​the lane. Therefore, a lean-to vehicle has a much higher degree of freedom in its left-right traveling position within a single lane than a car. Therefore, there is a demand for an image processing device for lean vehicles that includes an imaging device and a recognition device that can accommodate usage scenarios unique to lean vehicles, such as when a lean vehicle is traveling in the right side area and left side area of ​​a lane while traveling in the same lane as an automobile. In this specification, when a single lane is divided into thirds in the left-right direction, the area including the right edge is defined as the right side area, and the area including the left edge is defined as the left side area.

[0009] Furthermore, since lean vehicles have slimmer (narrower) bodies than automobiles, there is a demand for image processing devices for lean vehicles that are easily mountable on lean vehicles.

[0010] The object of the present invention is to provide an image processing device for a lean vehicle that can be adapted to situations in which a lean vehicle, which has a slimmer body than an automobile, travels in the right side area and left side area of ​​a lane while traveling in the same lane as an automobile, and that is highly mountable on a lean vehicle, which has a slimmer body than an automobile. [Means for solving the problem]

[0011] (1) An image processing device for a lean vehicle according to one embodiment of the present invention has the following configuration. An image processing device for a lean vehicle that is mounted on a lean vehicle includes an imaging device that captures images and a recognition device that recognizes an object from the image acquired by the imaging device, and the image processing device is configured to: when a single lane is divided into thirds in the left-right direction, an area including the right edge of the single lane is defined as a right side area, an area including the left edge of the single lane is defined as a left side area, and an area between the right side area and the left side area is defined as a center area; and the lanes of a five-lane road are defined from left to right as the leftmost lane, center left lane, center lane, center right lane, and rightmost lane, the lean vehicle is traveling in the right side area or left side area of ​​the center lane of the five-lane road, five automobiles are traveling in a line in the left-right direction ahead of the lean vehicle, and the distance between the five automobiles and the lean vehicle in the longitudinal direction is TTC (Time to Interchange Time), assuming that the time from image capture to completion of image processing is 0.5 seconds, and an object approaches the lean vehicle at a relative speed of 15 km / h. When the collision is the distance required to capture an image so that the image processing is completed at 3 seconds, the imaging device and the recognition device are configured as follows. The imaging device includes a monocular camera having a left-right angle of view configured so that, while the leaned vehicle is traveling in the right side area of ​​the center lane of the five-lane road, the left edge of the rear of the vehicle traveling in the leftmost lane is not captured, but at least the left edge of the rear of the vehicle traveling in the center lane and the right edge of the rear of the vehicle traveling in the center right lane are captured, and, while the leaned vehicle is traveling in the left side area of ​​the center lane of the five-lane road, the right edge of the rear of the vehicle traveling in the rightmost lane is not captured, but at least the right edge of the rear of the vehicle traveling in the center lane and the left edge of the rear of the vehicle traveling in the center left lane are captured. The recognition device is configured to at least recognize both the vehicle traveling in the center lane and the vehicle traveling in the center right lane from the image captured while the lean vehicle is traveling in the right side area of ​​the center lane of the five-lane road, and to at least recognize both the vehicle traveling in the center lane and the vehicle traveling in the center left lane from the image captured while the lean vehicle is traveling in the left side area of ​​the center lane of the five-lane road.

[0012] The width of a lean vehicle is very small compared to the width of an automobile. In other words, the body of a lean vehicle is slimmer than that of an automobile. The width of a lean vehicle is very small compared to the width of a lane. In other words, when a lean vehicle and an automobile are traveling in the same lane, the lean vehicle may travel in the right side area of ​​the lane or in the left side area. When a lean-to vehicle is traveling in the right side area of ​​the center lane of a five-lane road, two vehicles traveling in the center lane and center-right lane will be located in front of the lean-to vehicle.When a lean-to vehicle is traveling in the left side area of ​​the center lane of a five-lane road, two vehicles traveling in the center lane and center-left lane will be located in front of the lean-to vehicle. The imaging device includes a monocular camera. The monocular camera of the imaging device has a horizontal angle of view configured so that, when a leaned vehicle is traveling in the right side area of ​​the center lane of a five-lane road, at least the left edge of the rear of the vehicle traveling in the center lane and the right edge of the rear of the vehicle traveling in the center right lane are captured. The monocular camera of the imaging device has a horizontal angle of view configured so that, when a leaned vehicle is traveling in the left side area of ​​the center lane of a five-lane road, at least the right edge of the rear of the vehicle traveling in the center lane and the left edge of the rear of the vehicle traveling in the center left lane are captured. The recognition device recognizes at least both a vehicle traveling in the center lane and a vehicle traveling in the center right lane from an image captured while the leaned vehicle is traveling in the right side area of ​​the center lane of a five-lane road.The recognition device recognizes at least both a vehicle traveling in the center lane and a vehicle traveling in the center left lane from an image captured while the leaned vehicle is traveling in the left side area of ​​the center lane of a five-lane road. With this configuration, the image processing device for lean-mounted vehicles can capture and recognize the two vehicles positioned ahead of the lean-mounted vehicle, whether the lean-mounted vehicle is traveling in the right side area or the left side area. In other words, it is possible to ensure the minimum lateral angle of view required for usage scenarios unique to lean-mounted vehicles, which have a much higher degree of freedom in lateral travel positioning within a single lane than automobiles. Therefore, the image processing device for lean-mounted vehicles can be adapted to situations in which a lean-mounted vehicle, which has a slimmer body than an automobile, travels in the right side area and the left side area of ​​a lane while traveling in the same lane as an automobile. Furthermore, the monocular camera of the imaging device is configured with a horizontal angle of view so that when a leaned vehicle is traveling in the right side area of ​​the center lane of a five-lane road, the left edge of the rear of a vehicle traveling in the leftmost lane is not captured.The monocular camera of the imaging device is configured with a horizontal angle of view so that when a leaned vehicle is traveling in the left side area of ​​the center lane of a five-lane road, the right edge of the rear of a vehicle traveling in the rightmost lane is not captured. When a lean-to vehicle is traveling in the right-hand side area of ​​the center lane of a five-lane road, a car traveling in the left-hand lane is located more than two car lengths to the left of the lean-to vehicle.When a lean-to vehicle is traveling in the left-hand side area of ​​the center lane of a five-lane road, a car traveling in the right-hand lane is located more than two car lengths to the right of the lean-to vehicle. The width of a lean vehicle is very small compared to the width of an automobile. In other words, the body of a lean vehicle is slimmer than that of an automobile. Therefore, a vehicle located more than two automobile lengths away from the lean vehicle in the left-right direction has less impact on the driving of the lean vehicle, which has a slimmer body compared to an automobile, compared to the two vehicles located in front of the lean vehicle. With the above configuration, whether the lean vehicle is traveling in the right side area or the left side area, it is possible to limit the imaging of areas located more than two car lengths away from the lean vehicle in the left-right direction. As a result, the capacity (data volume) of the image acquired by the imaging device can be reduced. Furthermore, since the imaging area is limited, it is possible to select, for example, a compact optical device and imaging element, thereby making the imaging device more compact. In addition, since the capacity of the image acquired by the imaging device is small, it is possible to reduce the capacity of the image of the object to be recognized by the recognition device. As a result, the processing load of the recognition device can be reduced, and for example, the heat dissipation structure of the recognition device can be made simple and compact, making the recognition device more compact. Therefore, it is possible to improve the mountability of the lean vehicle image processing device in a lean vehicle, which has a slimmer body than an automobile. As a result of the above, it is possible to provide an image processing device for lean vehicles that can adapt to situations where a lean vehicle, which has a slimmer body than an automobile, is traveling in the right side area and left side area of ​​the lane while traveling in the same lane as an automobile, and that is highly mountable on lean vehicles, which have a slimmer body than an automobile.

[0013] (2) The image processing device for a lean vehicle according to one embodiment of the present invention may have the following configuration in addition to the configuration described in (1) above. When the lean vehicle is traveling in the right side area or the left side area of ​​the center lane of the five-lane road, and the five automobiles are traveling in a line left and right in front of the lean vehicle, and the longitudinal distance between the five automobiles and the lean vehicle is the distance required to capture an image so that the image processing is completed when the TTC is 3 seconds, assuming that the time from capturing image to completing image processing is 0.5 seconds and an object approaches the lean vehicle at a relative speed of 15 km / h, the imaging device and the recognition device are further configured as follows: The imaging device captures images with the monocular camera while the leaned vehicle is traveling in the right side area of ​​the center lane of the five-lane road so that at least the rear of the vehicle traveling in the center lane, the right side of the vehicle traveling in the center lane, the rear of the vehicle traveling in the center right lane, and the left side of the vehicle traveling in the center right lane are included in the image, and while the leaned vehicle is traveling in the left side area of ​​the center lane of the five-lane road, the imaging device captures images with the monocular camera while the leaned vehicle is traveling in the left side area of ​​the center lane of the five-lane road so that at least the rear of the vehicle traveling in the center lane, the left side of the vehicle traveling in the center lane, the rear of the vehicle traveling in the center left lane, and the right side of the vehicle traveling in the center left lane are included in the image. The recognition device recognizes at least both the vehicle traveling in the center lane and the vehicle traveling in the center right lane from the image including the rear of the vehicle traveling in the center lane, the right side of the vehicle traveling in the center right lane, the rear of the vehicle traveling in the center right lane, and the left side of the vehicle traveling in the center right lane, and recognizes at least both the vehicle traveling in the center lane and the vehicle traveling in the center left lane from the image including the rear of the vehicle traveling in the center lane, the left side of the vehicle traveling in the center lane, the rear of the vehicle traveling in the center left lane, and the right side of the vehicle traveling in the center left lane.

[0014] The width of a lean vehicle is very small compared to the width of an automobile. In other words, the body of a lean vehicle is slimmer than that of an automobile. The width of a lean vehicle is very small compared to the width of a lane. In other words, when a lean vehicle and an automobile are traveling in the same lane, the lean vehicle may travel in the right side area of ​​the lane or in the left side area. As a lean-to vehicle is traveling in the right side area of ​​the center lane of a five-lane road, in front of the lean-to vehicle, between the rear of the vehicle traveling in the center lane and the rear of the vehicle traveling in the center-right lane, the right side of the vehicle traveling in the center lane and the left side of the vehicle traveling in the center-right lane are side by side.As a lean-to vehicle is traveling in the right side area of ​​the center lane of a five-lane road, in front of the lean-to vehicle, between the rear of the vehicle traveling in the center lane and the rear of the vehicle traveling in the center-right lane, the right side of the vehicle traveling in the center lane and the left side of the vehicle traveling in the center-right lane are side by side. The imaging device captures an image with a monocular camera while the leaned vehicle is traveling in the right side area of ​​the center lane of a five-lane road so that the rear of the vehicle traveling in the center lane, the right side of the vehicle traveling in the center-right lane, and the left side of the vehicle traveling in the center-right lane are included in the image.The imaging device captures an image with a monocular camera while the leaned vehicle is traveling in the left side area of ​​the center lane of a five-lane road so that the rear of the vehicle traveling in the center lane, the left side of the vehicle traveling in the center lane, the rear of the vehicle traveling in the center-left lane, and the right side of the vehicle traveling in the center-left lane are included in the image. The recognition device recognizes at least both a vehicle traveling in the center lane and a vehicle traveling in the center right lane from an image including the rear of a vehicle traveling in the center lane, the right side of a vehicle traveling in the center right lane, the rear of a vehicle traveling in the center right lane, and the left side of a vehicle traveling in the center right lane.The recognition device recognizes at least both a vehicle traveling in the center lane and a vehicle traveling in the center left lane from an image including the rear of a vehicle traveling in the center lane, the left side of a vehicle traveling in the center lane, the rear of a vehicle traveling in the center left lane, and the right side of a vehicle traveling in the center left lane. With this configuration, the image processing device for lean-mounted vehicles can be adapted to usage scenarios in which, while a lean-mounted vehicle is traveling in the right side area of ​​the center lane of a five-lane road, the right side of a vehicle traveling in the center lane and the left side of a vehicle traveling in the center-right lane are side by side in front of the lean-mounted vehicle, between the rear of the vehicle traveling in the center lane and the rear of the vehicle traveling in the center-right lane; and, while a lean-mounted vehicle is traveling in the right side area of ​​the center lane of a five-lane road, the right side of a vehicle traveling in the center lane and the left side of a vehicle traveling in the center-right lane are side by side in front of the lean-mounted vehicle, between the rear of the vehicle traveling in the center lane and the rear of the vehicle traveling in the center-right lane. As a result of the above, it is possible to provide an image processing device for lean vehicles that is more adaptable to situations where a lean vehicle, which has a slimmer body than an automobile, is traveling in the right side area and left side area of ​​the lane while traveling in the same lane as an automobile, and that is highly mountable on lean vehicles, which have a slimmer body than an automobile.

[0015] (3) The image processing device for a lean vehicle according to one embodiment of the present invention may have the following configuration in addition to the configuration of (1) or (2) above. The lean vehicle image processing device further includes an alarm device that outputs an alarm to urge a driver to pay attention to the vehicle recognized by the recognition device from the image captured by the imaging device, The warning device is configured not to output the warning when two automobiles are traveling straight ahead, side by side, with a gap between them in the left and right direction in front of the lean vehicle, and the size of the gap is too small for the width of the automobiles but large enough for the width of the lean vehicle to have left and right clearance, even if the lean vehicle travels straight towards the center between the two automobiles and approaches them; and is configured to output the warning when two automobiles are traveling straight ahead, side by side, with a gap between them in the left and right direction in front of the lean vehicle, and the size of the gap is too small for the width of the lean vehicle to have left and right clearance, even if the lean vehicle travels straight towards the center between the two automobiles and approaches them, and outputs the warning according to the width of the lean vehicle.

[0016] According to this configuration, the lean vehicle image processing device further includes an alarm device that outputs an alarm to urge the driver to pay attention to the vehicle recognized by the recognition device from the image captured by the imaging device. The warning device is configured not to output a warning when two automobiles are traveling straight ahead of a lean-to vehicle, side by side with a gap between them in the left-right direction, and the gap is large enough to fit the width of the automobiles but large enough to fit the width of the lean-to vehicle, even if the lean-to vehicle moves straight towards the center between the two automobiles and approaches the two automobiles.The warning device is configured to output a warning when two automobiles are traveling straight ahead of a lean-to vehicle, side by side with a gap between them in the left-right direction, and the gap is large enough to fit the width of the lean-to vehicle, even if the lean-to vehicle moves straight towards the center between the two automobiles and approaches the two automobiles.In this way, the warning device outputs a warning according to the width of the lean-to vehicle. For example, when a leaning vehicle is traveling in the right side area of ​​the center lane of a five-lane road, two vehicles traveling in the center lane and the center-right lane will be located in front of the leaning vehicle. For example, when a leaning vehicle is traveling in the right side area of ​​the center lane, a portion of the vehicle traveling in the center lane is located on the left side area of ​​the center lane and a portion of the vehicle traveling in the center-right lane is located on the right side area of ​​the center-right lane, and a portion of the vehicle traveling in the center lane is located on the right side area of ​​the center-right lane. The presence or absence of an alarm can be set to differ depending on the situation. For example, when a lean-to vehicle is traveling in the left side area of ​​the center lane of a five-lane road, two vehicles traveling in the center lane and the center-left lane will be located in front of the lean-to vehicle. For example, when a lean-to vehicle is traveling in the left side area of ​​the center lane, a portion of the vehicle traveling in the center lane is located on the right side area and a portion of the vehicle traveling in the center-left lane is located on the left side area of ​​the center-left lane, and a portion of the vehicle traveling in the center lane is located on the left side area of ​​the center lane and a portion of the vehicle traveling in the center-left lane is located on the right side area of ​​the center-left lane, respectively. With the above configuration, the image processing device for lean-in vehicles can capture and recognize two automobiles located in front of the lean-in vehicle, whether the lean-in vehicle is traveling in the right side area or the left side area, and can output a warning according to the width of the lean-in vehicle. Therefore, when a lean-in vehicle, which has a slimmer body than an automobile, is traveling in the same lane as an automobile, the image capture, recognition, and warning can be adapted to scenes where the lean-in vehicle is traveling in the right side area or the left side area of ​​the lane. As a result of the above, it is possible to provide an image processing device for lean vehicles that is more adaptable to situations where a lean vehicle, which has a slimmer body than an automobile, is traveling in the right side area and left side area of ​​the lane while traveling in the same lane as an automobile, and that is highly mountable on lean vehicles, which have a slimmer body than an automobile. The image processing device for lean vehicles may be an image processing device dedicated to lean vehicles in which the warning device is configured in advance according to the vehicle width of the lean vehicle.The image processing device for lean vehicles may be an image processing device for both lean vehicles and automobiles in which the warning device is configured to adapt to the vehicle width of the lean vehicle or the vehicle width of an automobile by switching.The switching may be performed by a user operation or automatically based on a captured image.

[0017] (4) The image processing device for a lean vehicle according to one embodiment of the present invention may have the following configuration in addition to at least one of the configurations (1) to (3) above. When the lean vehicle is traveling in the right side area or the left side area of ​​the center lane of the five-lane road, and the five automobiles are traveling in a line left and right in front of the lean vehicle, and the longitudinal distance between the five automobiles and the lean vehicle is the distance required to capture an image so that the image processing is completed when the TTC is 3 seconds, assuming that the time from capturing image to completing image processing is 0.5 seconds and an object approaches the lean vehicle at a relative speed of 15 km / h, the recognition device is further configured as follows: The recognition device is configured to process, when the image captured while the lean vehicle is traveling in the right side area of ​​the center lane of the five-lane road includes an image of at least one of the vehicle traveling in the center left lane and the vehicle traveling in the rightmost lane, the image of at least one of the vehicle traveling in the center left lane and the vehicle traveling in the rightmost lane, at a resolution lower than the resolution used to recognize the vehicle traveling in the center lane and the vehicle traveling in the center right lane, and to process, when the image captured while the lean vehicle is traveling in the left side area of ​​the center lane of the five-lane road includes an image of at least one of the vehicle traveling in the center right lane and the vehicle traveling in the leftmost lane, the image of at least one of the vehicle traveling in the center right lane and the vehicle traveling in the leftmost lane, at a resolution lower than the resolution used to recognize the vehicle traveling in the center lane and the vehicle traveling in the center left lane.

[0018] The width of a lean vehicle is very small compared to the width of an automobile. In other words, the body of a lean vehicle is slimmer than that of an automobile. The width of a lean vehicle is very small compared to the width of a lane. In other words, when a lean vehicle and an automobile are traveling in the same lane, the lean vehicle may travel in the right side area of ​​the lane or in the left side area. When a lean-to vehicle is traveling in the right side area of ​​the center lane of a five-lane road, a car traveling in the center left lane is located one car length to the left of the lean-to vehicle, and a car traveling in the rightmost lane is located one car length to the right of the lean-to vehicle.When a lean-to vehicle is traveling in the left side area of ​​the center lane of a five-lane road, a car traveling in the center right lane is located one car length to the right of the lean-to vehicle, and a car traveling in the leftmost lane is located one car length to the left of the lean-to vehicle. The recognition device is configured to process at least one of the images of a vehicle traveling in the center left lane and a vehicle traveling in the rightmost lane at a resolution lower than the resolution used to recognize the vehicle traveling in the center lane and the center right lane, when an image captured while the leaned vehicle is traveling in the right side area of ​​the center lane of a five-lane road includes at least one of an image of a vehicle traveling in the center left lane and a vehicle traveling in the rightmost lane.The recognition device is configured to process at least one of the images of a vehicle traveling in the center right lane and a vehicle traveling in the leftmost lane at a resolution lower than the resolution used to recognize the vehicle traveling in the center lane and the center left lane, when an image captured while the leaned vehicle is traveling in the left side area of ​​the center lane of a five-lane road includes at least one of an image of a vehicle traveling in the center right lane and a vehicle traveling in the leftmost lane. With this configuration, even if the lean-to vehicle is traveling in the right-side area or the left-side area, it is possible to recognize vehicles in areas located more than one vehicle's width away from the lean-to vehicle in the left-right direction by processing them at a lower resolution. This allows the size of the image of the object to be recognized by the recognition device to be reduced. As a result, the processing load of the recognition device can be reduced, and for example, the heat dissipation structure of the recognition device can be made simple and compact, allowing the recognition device to be made more compact. As a result of the above, it is possible to provide an image processing device for lean vehicles that can adapt to situations where a lean vehicle, which has a slimmer body than an automobile, is traveling in the right side area and left side area of ​​the lane while traveling in the same lane as an automobile, and that is more easily installable in lean vehicles, which have a slimmer body than an automobile.

[0019] In the present invention and embodiments, a lean vehicle may be a vehicle having a body frame that leans left when turning left and leans right when turning right. In the present invention and embodiments, a lean vehicle may be, for example, a motorcycle or a three-wheeled motor vehicle. Motorcycles include scooters, mopeds, and motorized bicycles. The width of a lean vehicle is, for example, less than 1 meter. On the other hand, the width of a car is, for example, 2 to 2.5 meters.

[0020] In the present invention and embodiments, the monocular camera has a single imaging element. An imaging device including a monocular camera has an optical device such as a lens and a single imaging element. An imaging device having a monocular camera may also be an imaging device having a single lens. The imaging element may be an image sensor such as a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor. In the present invention and embodiments, the imaging device may be disposed in the center of the vehicle in the left-right direction. However, the location of the imaging device is not limited to this.

[0021] In the present invention and embodiments, the recognition device includes a processor and a memory. The memory includes, for example, a random access memory (RAM). The memory may include a read-only memory (ROM). The processor includes any circuit capable of processing images, such as a field-programmable gate array (FPGA), a graphics processing unit (GPU), a central processing unit (CPU), or a system on chip (SoC). An SoC has a configuration in which the processor and memory are integrated.

[0022] In the present invention and embodiments, when a recognition device recognizes an object (for example, an automobile), it means that the recognition device detects the object (for example, an automobile). In the present invention and embodiments, the recognition device processing an image of a vehicle includes, for example, the recognition device recognizing a vehicle from this image.

[0023] In the present invention and its embodiments, a lane means a driving lane. A lane may or may not have marks (e.g., white lines) indicating the edges of the lane. The width of a lane is, for example, 3.5 to 4 m. In the present invention and the embodiments, the right side area, the left side area, and the center area correspond to three areas obtained by dividing a lane into three equal parts in the left and right direction. In the present invention and embodiments, the five-lane road is a straight road.

[0024] In the present invention and embodiments, a lean vehicle traveling in the right side area of ​​a lane means that the lean vehicle travels so that the left-right center of the lean vehicle is located within the right side area. A lean vehicle traveling in the left side area of ​​a lane means that the lean vehicle travels so that the left-right center of the lean vehicle is located within the left side area. In the present invention, a lean vehicle traveling in the right side area (left side area) of a lane includes traveling so that the entire lean vehicle is located within the right side area (left side area) in a plan view. In the present invention, a lean vehicle traveling in the right side area (left side area) of a lane may also include traveling so that part of the lean vehicle is located outside the right side area (left side area).

[0025] In this invention, the configuration of the imaging device and the recognition device is defined based on the assumption that the longitudinal distance between the five automobiles and the lean vehicle is the distance required to capture an image so that image processing is completed at a time to collision (TTC) of 3 seconds, assuming that the time from image capture to completion of image processing is 0.5 seconds and that the object approaches the lean vehicle at a relative speed of 15 km / h. The phrase "the distance required to capture an image so that image processing is completed at a time to collision (TTC) of 3 seconds, assuming that the time from image capture to completion of image processing is 0.5 seconds and that the object approaches the lean vehicle at a relative speed of 15 km / h" can be rephrased as follows: That is, this distance is the distance required to capture an image so that image processing is completed at a time to collision (TTC) of 3 seconds, assuming that the lean vehicle image processing device is configured to perform image processing so that the time from image capture to completion of image processing is 0.5 seconds and that the object approaches the lean vehicle at a relative speed of 15 km / h. The TTC is the time it takes for the object to reach the lean vehicle. Specifically, this distance is 14.58335 m (= 15 km / h × (3.0 s + 0.5 s) ÷ 3600 × 1000). "An object approaches a lean vehicle at a relative speed of 15 km / h" means that the object and lean vehicle are both moving and the lean vehicle is moving at a speed 15 km / h faster than the object, or the object is stationary and the lean vehicle is moving and the lean vehicle is moving at a speed of 15 km / h. The "time from capturing an image to completing image processing" is an assumed time and does not have to be the same as the time required for the image processing that is actually performed by the lean vehicle image processing device. The "time from capturing an image to completing image processing" may be, for example, the time from when an image capture device captures an image to when a recognition device estimates the TTC for a vehicle recognized from the image. Generally, it takes about 0.5 seconds from capturing an image to estimating the TTC. The "time from capturing an image to completing image processing" may be, for example, the time from when an image capturing device captures an image to when a command is output to an alarm device to issue an alarm urging the driver to be careful of the vehicle recognized by the recognition device from the image.

[0026] In the present invention and embodiments, "the rear surface of the automobile is included in the image" means that the entire rear surface of the automobile is included in the image. In the present invention and embodiments, "the right side of the automobile is included in the image" means that, if the right side of the automobile is flat, the entire right side of the automobile is included in the image. In other words, "the right side of the automobile is included in the image" means that at least a portion of the right side of the automobile is included in the image. The meaning of "the left side of the automobile is included in the image" can also be interpreted in a similar way to this definition. In the present invention and embodiments, included in an image means appearing in the image.

[0027] In the present invention and embodiments, a vehicle traveling ahead of a lean vehicle refers to a vehicle located ahead of the lean vehicle when the forward direction of the lean vehicle is defined as the forward direction. In the present invention and embodiments, a vehicle traveling ahead of a lean vehicle refers to a vehicle traveling directly or almost directly in front of the lean vehicle. A vehicle traveling ahead of a lean vehicle is not limited to a vehicle traveling ahead of the lean vehicle.

[0028] In the present invention and its embodiments, the term "the size of the gap between two vehicles is too small in the lateral direction for the vehicle width but large enough in the lateral direction for the lean vehicle width" means that the size of the gap is such that, under normal traffic conditions, a vehicle cannot pass through the gap, but a lean vehicle can. In other words, the size of the gap is larger than the width of the lean vehicle, but smaller than or equal to the width of the vehicle. Therefore, cases in which the size of the gap between two vehicles is too small in the lateral direction for the vehicle width but large enough in the lateral direction for the lean vehicle width include cases in which the size of the gap between the two vehicles is smaller than the width of the vehicle, but larger than the width of the lean vehicle. In the present invention and its embodiments, when the distance between two vehicles is too narrow in the lateral direction for the width of a lean vehicle, this means that the distance is so narrow that a lean vehicle cannot pass through it under normal traffic conditions. In other words, this means that the distance is smaller than or equal to the width of the lean vehicle. Therefore, cases where the distance between two vehicles is too narrow in the lateral direction for the width of the vehicles include cases where the distance between the two vehicles is smaller than the width of the lean vehicle.

[0029] The image processing device for a lean vehicle of the present invention and the embodiments may or may not have an alarm device that outputs an alarm to urge the driver to pay attention to the vehicle recognized by the recognition device from the image captured by the imaging device. In the present invention and embodiments, if the lean vehicle image processing device has an alarm device, the use of the recognition result by the recognition device is not limited to the alarm device. Regardless of whether the lean vehicle image processing device has an alarm device, the recognition result by the recognition device may be used, for example, to control the vehicle speed of the lean vehicle. More specifically, for example, the brake device of the lean vehicle may be activated based on the recognition result by the recognition device without the driver's brake operation. In the present invention and embodiments, an alarm device that outputs an alarm urging the driver to be careful of a vehicle recognized by a recognition device may also output an alarm urging the driver to be careful of objects other than a vehicle recognized by the recognition device.

[0030] In the present invention and its embodiments, the alarm output by the alarm device is an alarm to alert the driver of the lean vehicle. The alarm is an alarm that alerts at least one of the driver's hearing, sight, or touch. An auditory alarm is an alarm that uses sound. A visual alarm is, for example, an alarm that uses light. A tactile alarm is, for example, an alarm that uses vibration. The vibration may be, for example, a vibration of the grip or a vibration of the seat. The means for generating the vibration may be a dedicated means of the alarm device, or may be a brake or a drive source.

[0031] In the present invention and embodiments, low resolution means low pixel density. In the present invention and embodiments, processing an image of a vehicle at a predetermined resolution includes recognizing an image of a vehicle at a predetermined resolution.

[0032] The situation in which the lean-in vehicle image processing device of the present invention is used is not limited to a situation in which a lean-in vehicle is traveling in the right or left side area of ​​the center lane of a five-lane road. The lean-in vehicle image processing device may also be used in the following driving situations, for example. The driving situation may be, for example, a situation in which a lean-in vehicle is traveling on a road other than a five-lane road. A road other than a five-lane road may be a road with six or more lanes, a two-lane road, or a road with one lane in each direction. The driving situation may also be, for example, a situation in which a lean-in vehicle is traveling in the center area of ​​a certain lane. The driving situation may also be, for example, a situation in which a lean-in vehicle is traveling in the right or left side area of ​​a lane other than the center lane of a five-lane road.

[0033] In the present invention and embodiments, "at least one (one) of a plurality of options" includes all possible combinations of the plurality of options. "At least one (one) of a plurality of options" may be any one of the plurality of options, or may be all of the plurality of options. For example, "at least one of A, B, and C" may be only A, only B, only C, A and B, A and C, B and C, or A, B, and C.

[0034] In this specification, for example, "1 to 10" means 1 or more and 10 or less. The same definition applies to numbers other than 1 and 10.

[0035] In the present invention and embodiments, the words including, comprising, having, and their derivatives are used herein to encompass additional items in addition to the listed items and equivalents thereof.

[0036] Unless otherwise defined, all terms (including technical and scientific terms) used in the present specification and claims have the same meaning as commonly understood by those skilled in the art to which this invention belongs. Terms, such as those defined in commonly used dictionaries, should be interpreted to have a meaning consistent with the meaning in the context of the relevant technology and this disclosure, and should not be interpreted in an idealized or overly formal sense.

[0037] In this specification, the term "may" is non-exclusive. "may" means "may, but is not limited to." In this specification, "may" implicitly includes the possibility that "may not." In this specification, a configuration described as "may" at least achieves the above-mentioned effect obtained by the configuration of claim 1.

[0038] Before describing embodiments of the present invention in detail, it is to be understood that the invention is not limited to the details of construction and the arrangement of components set forth in the following description or illustrated in the drawings. The present invention is capable of embodiments other than those described below. The present invention is also capable of embodiments incorporating various variations of the embodiments described below. [Effects of the Invention]

[0039] According to the present invention, an image processing device for a lean vehicle can be provided that can adapt to situations in which a lean vehicle, which has a slimmer body than an automobile, travels in the right side area and left side area of ​​a lane while traveling in the same lane as an automobile, and that is highly mountable on a lean vehicle, which has a slimmer body than an automobile. [Brief explanation of the drawings]

[0040] [Figure 1] 1 is a diagram illustrating a configuration of an image processing device for a lean vehicle according to a first embodiment of the present invention; [Figure 2] FIG. 6 is a diagram illustrating the configuration of an image processing device for a lean vehicle according to a second embodiment of the present invention. [Figure 3] FIG. 10 is a diagram illustrating the configuration of an image processing device for a lean vehicle according to a third embodiment of the present invention. [Figure 4] FIG. 10 is a diagram illustrating the configuration of an image processing device for a lean vehicle according to a fourth embodiment of the present invention. [Figure 5] 10A to 10C are diagrams illustrating three examples of recognition by an image processing device for a lean vehicle according to a fourth embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0041] Arrows F, Re, L, and R shown in the drawings represent the forward, backward, leftward, and rightward directions, respectively.

[0042] First Embodiment An image processing device 10 for a lean vehicle according to an embodiment of the present invention will be described below with reference to Fig. 1. The image processing device 10 for a lean vehicle is mounted on a lean vehicle 1. In Fig. 1, the lean vehicle 1 is a motorcycle, but the lean vehicle 1 is not limited to motorcycles. The image processing device 10 for a lean vehicle includes an imaging device 11 that captures images, and a recognition device 12 that recognizes objects from the images acquired by the imaging device 11. The imaging device 11 includes a monocular camera.

[0043] FIG. 1 includes plan views showing two examples (CASE 1 and CASE 2) of a situation in which a lean-to vehicle 1 and five automobiles 61 to 65 are traveling on a five-lane road. The five-lane road includes a left-most lane 51, a center-left lane 52, a center lane 53, a center-right lane 54, and a right-most lane 55. In CASE 1 and CASE 2, the lean-to vehicle 1 travels in the center lane 53, and the five automobiles 61 to 65 travel in a line ahead of the lean-to vehicle 1 in the left-right direction. In CASE 1, the lean-to vehicle 1 travels in a right-side area 50R of the center lane 53, and in CASE 2, the lean-to vehicle 1 travels in a left-side area 50L of the center lane 53. The right side area 50R is an area including the right edge of a single lane 50 (e.g., center lane 53) when the single lane 50 is divided into thirds in the left-right direction, and the left side area 50L is an area including the left edge of the single lane 50. The area between the right side area 50R and the left side area 50L is a center area 50C. The width of the lean-to vehicle 1 is very small compared to the width of an automobile 60 (e.g., automobiles 61-65). The width of the lean-to vehicle 1 is very small compared to the width of the lane 50. Therefore, the lean-to vehicle 1 has a high degree of freedom in its traveling position in the left-right direction within the center lane 53, and can move left and right within the center lane 53, and can travel in either the right side area 50R or the left side area 50L within the center lane 53.

[0044] In Case 1 and Case 2, the distance D in the longitudinal direction between the five automobiles 61-65 and the lean-to vehicle 1 is the distance required to capture an image so that image processing is completed at a time to capture (TTC) of 3 seconds, assuming that the time from image capture to completion of image processing (Ti) is 0.5 seconds and the object approaches the lean-to vehicle 1 at a relative speed (Vr) of 15 km / h. This distance is approximately 15 m. Note that, hereinafter, this distance may be referred to as the "specific distance."

[0045] When the lean vehicle 1 is traveling in the right side area 50R or left side area 50L of the center lane 53 of a five-lane road, and five automobiles 61 to 65 are traveling in a line in the left-right direction ahead of the lean vehicle 1, and the longitudinal distance D between the five automobiles 61 to 65 and the lean vehicle 1 is the specific distance described above, the imaging device 11 and the recognition device 12 have the following configuration.

[0046] The monocular camera of the imaging device 11 is configured such that, while the leaning vehicle 1 is traveling in the right side area 50R of the center lane 53, the left edge 61bl of the rear of the automobile 61 traveling in the leftmost lane 51 is not captured, but the angle of view in the left-right direction is configured so that at least the left edge 63bl of the rear of the automobile 63 traveling in the center lane 53 and the right edge 64br of the rear of the automobile 64 traveling in the center right lane 54 are captured. The monocular camera of the imaging device 11 is configured such that, while the leaning vehicle 1 is traveling in the left side area 50L of the center lane 53, the right edge 65br of the rear of the automobile 65 traveling in the rightmost lane 55 is not captured, but the angle of view in the left-right direction is configured so that at least the right edge 63br of the rear of the automobile 63 traveling in the center lane 53 and the left edge 63bl of the rear of the automobile 62 traveling in the center left lane 52 are captured.

[0047] The recognition device 12 is configured to recognize at least both the vehicle 63 traveling in the center lane 53 and the vehicle 64 traveling in the center right lane 54 from an image captured while the leaning vehicle 1 is traveling in the right side area 50R of the center lane 53. At this time, the recognition device 12 recognizes at least the vehicle 63 and the vehicle 64 from an image captured by the imaging device 11 when the distance D is the specific distance described above. In other words, the recognition device 12 recognizes at least the vehicle 63 and the vehicle 64 from an image that does not include the left edge 61bl of the rear of the vehicle 61 traveling in the leftmost lane 51, but does include the left edge 63bl of the rear of the vehicle 63 traveling in the center lane 53 and the right edge 64br of the rear of the vehicle 64 traveling in the center right lane 54. The recognition device 12 is configured to recognize at least both the vehicle 63 traveling in the center lane 53 and the vehicle 62 traveling in the center-left lane 52 from an image captured while the lean-to vehicle 1 is traveling in the left side area 50L of the center lane 53. In this case, the recognition device 12 recognizes at least the vehicle 63 and the vehicle 62 from an image captured by the imaging device 11 when the distance D is the specific distance described above. In other words, the recognition device 12 recognizes at least the vehicle 63 and the vehicle 62 from an image that does not include the right edge 65br of the rear of the vehicle 65 traveling in the rightmost lane 55, but includes the right edge 63br of the rear of the vehicle 63 traveling in the center lane 53 and the left edge 63bl of the rear of the vehicle 62 traveling in the center-left lane 52.

[0048] 1 is a schematic representation of the minimum horizontal angle of view of the monocular camera of the imaging device 11 for conceptual understanding. Angle θmax in Fig. 1 is a schematic representation of the maximum horizontal angle of view of the monocular camera of the imaging device 11 for conceptual understanding.

[0049] In FIG. 1, the symbols "◯", "△", and "×" attached to the automobiles 61 to 65 have the following meanings: The symbol "◯" indicates that the entire rear of the automobile is captured. The symbol "×" indicates that the rear of the automobile is not captured at all or only a portion of the rear of the automobile is captured. The symbol "△" indicates that the rear of the automobile is not captured at all, only a portion of the rear of the automobile is captured, or the entire rear of the automobile is captured.

[0050] According to this embodiment, an image processing device 10 for a lean vehicle can be realized that can adapt to scenes in which a lean vehicle 1, which has a slimmer body than an automobile 60, is traveling in the right side area 50R and left side area 50L of the lane 50 while traveling in the same lane 50 as the automobile 60, and that is highly mountable on a lean vehicle 1, which has a slimmer body than an automobile 60.

[0051] The lean vehicle 1 or the lean vehicle image processing device 10 may or may not have a display device that displays the image captured by the imaging device 11. When the image is displayed, the object (e.g., a car) recognized by the recognition device 12 may be surrounded by a frame (see FIG. 4 described later).

[0052] Second Embodiment Next, an image processing device 10 for a lean vehicle according to a second embodiment of the present invention will be described with reference to FIG. 2. The second embodiment has all of the configurations of the first embodiment. FIG. 2 is a diagram schematically illustrating an example of a situation in which a lean vehicle 1 is traveling in the right side area 50R of the center lane 53 of a five-lane road (CASE 3) and an example of a situation in which the lean vehicle 1 is traveling in the left side area 50L of the center lane 53 (CASE 4). CASE 3 is an example of CASE 1 in the first embodiment. CASE 4 is an example of CASE 2 in the first embodiment.

[0053] When the lean vehicle 1 is traveling in the right side area 50R or left side area 50L of the center lane 53 of a five-lane road, and five automobiles 61 to 65 are traveling in a line in the left-right direction ahead of the lean vehicle 1, and the longitudinal distance D between the five automobiles 61 to 65 and the lean vehicle 1 is the specific distance described in the first embodiment, the imaging device 11 and the recognition device 12 may have the following configuration.

[0054] As shown in CASE 3 of Figure 2, while the leaning vehicle 1 is traveling in the right side area 50R of the center lane 53, the monocular camera of the imaging device 11 captures an image so that at least the rear surface 63b of the automobile 63 traveling in the center lane 53, the right side surface 63r of the automobile 63 traveling in the center lane 53, the rear surface 64b of the automobile 64 traveling in the center right lane 54, and the left side surface 64l of the automobile 64 traveling in the center right lane 54 are included in the image. As shown in CASE 4 of Figure 2, while the leaning vehicle 1 is traveling in the left side area 50L of the center lane 53, the monocular camera of the imaging device 11 captures an image so that at least the rear surface 63b of the automobile 63 traveling in the center lane 53, the left side surface 63l of the automobile 63 traveling in the center lane 53, the rear surface 62b of the automobile 62 traveling in the center left lane 52, and the right side surface 62r of the automobile 62 traveling in the center left lane 52 are included in the image.

[0055] The recognition device 12 recognizes at least both the automobile 63 traveling in the center lane 53 and the automobile 64 traveling in the center right lane 54 from an image that includes the rear 63b of the automobile 63 traveling in the center lane 53, the right side 63r of the automobile 63 traveling in the center lane 53, the rear 64b of the automobile 64 traveling in the center right lane 54, and the left side 64l of the automobile 64 traveling in the center right lane 54. The recognition device 12 recognizes at least both the automobile 63 traveling in the center lane 53 and the automobile 62 traveling in the center left lane 52 from an image that includes the rear 63b of the automobile 63 traveling in the center lane 53, the left side 63l of the automobile 63 traveling in the center lane 53, the rear 62b of the automobile 62 traveling in the center left lane 52, and the right side 62r of the automobile 62 traveling in the center left lane 52.

[0056] Whether or not the right side 63r of the automobile 63 traveling in the center lane 53 is included in the image while the lean-in vehicle 1 is traveling in the right side area 50R of the center lane 53 is affected by the left-right position of the lean-in vehicle 1 in the right side area 50R of the center lane 53, the left-right position of the automobile 63 traveling in the center lane 53 in the center lane 53, the width of the center lane 53, the width of the automobile 63, the width of the lean-in vehicle 1, and the left-right position of the imaging device 11 in the lean-in vehicle 1. Therefore, when the lean-in vehicle 1 is traveling in the right side area 50R of the center lane 53 and the distance D is the specific distance described above, the imaging device 11 of the second embodiment does not necessarily have the above-described configuration, and may not have the above-described configuration. The same applies when the lean-in vehicle 1 is traveling in the left side area 50L of the center lane 53.

[0057] Third Embodiment Next, an image processing device 10 for a lean vehicle according to a third embodiment of the present invention will be described with reference to Fig. 3. The third embodiment has all of the configurations of the first embodiment. The third embodiment may also have the configuration of the second embodiment.

[0058] 3, the lean vehicle image processing device 10 further includes a warning device 13 that outputs a warning to urge the driver to pay attention to the automobile 60 recognized by the recognition device 12 from the image captured by the imaging device 11. The warning device 13 outputs a warning according to the vehicle width of the lean vehicle 1, as will be described below.

[0059] As shown in CASE 5 of Figure 3, when two automobiles 60 are traveling straight ahead of the lean vehicle 1, side by side with a gap G in the left-right direction, and the size of the gap G is such that there is no room left-right for the width of the automobiles 60, but there is room left-right for the width of the lean vehicle 1, the warning device 13 is configured not to output an alarm even if the lean vehicle 1 travels straight towards the center between the two automobiles 60 and approaches them. As shown in CASE 6 of Fig. 3, when two automobiles 60 are traveling straight ahead of the lean vehicle 1 side by side with a gap G in the left-right direction, and the size of the gap G is too large for the width of the lean vehicle 1 in the left-right direction, the warning device 13 is configured to output a warning when the lean vehicle 1 travels straight towards and approaches the center between the two automobiles 60. Note that in CASE 6 of Fig. 3, the gap G is slightly larger than the width of the lean vehicle 1, but it may also be smaller than the width of the lean vehicle 1.

[0060] As shown in the comparative example of Figure 3, two automobiles 60 are traveling straight ahead of an automobile 66 (60) equipped with an alarm device, side by side with a gap G in the left-right direction, and the size of the gap G is too small for the width of the automobiles 66, but is large enough for the width of the lean vehicle 1 to have some margin in the left-right direction. When the automobile 66 travels straight toward the center between the two automobiles 60 and approaches them, the alarm device of the automobile 66 will output an alarm. Although not shown in the figure, two automobiles 60 are traveling straight ahead of an automobile 66 equipped with an alarm device, side by side with a gap G between them in the left and right direction, and if the size of the gap G is such that there is no room left and right for both the width of the automobile 66 and the width of the lean vehicle 1, when the automobile 66 travels straight towards the center between the two automobiles 60 and approaches, the alarm device of the automobile 66 will output an alarm.

[0061] EXAMPLE 1 and EXAMPLE 2 in Figure 3 are specific examples of CASE 5. EXAMPLE 3 and EXAMPLE 4 in Figure 3 are specific examples of CASE 6.

[0062] In EXAMPLE 1 and EXAMPLE 3, a lean-to vehicle 1 is traveling in the right side area 50R of a center lane 53 of a five-lane road, and two automobiles 63, 64 traveling in the center lane 53 and center-right lane 54 are located in front of the lean-to vehicle 1. In EXAMPLE 1, a portion of the automobile 63 is located on the left side area 50L of the center lane 53, and a portion of the automobile 64 is located on the right side area 50R of the center-right lane 54. In EXAMPLE 3, a portion of the automobile 63 is located on the right side area 50R of the center lane 53, and a portion of the automobile 64 is located on the left side area 50L of the center-right lane 54.

[0063] In EXAMPLE 2 and EXAMPLE 4, the lean-to vehicle 1 is traveling in the left side area 50L of the center lane 53 of a five-lane road, and two automobiles 63, 62 traveling in the center lane 53 and center-left lane 52 are located in front of the lean-to vehicle 1. In EXAMPLE 2, a portion of the automobile 63 is located on the right side area 50R of the center lane 53, and a portion of the automobile 62 is located on the left side area 50L of the center-left lane 52. In EXAMPLE 4, a portion of the automobile 63 is located on the left side area 50L of the center lane 53, and a portion of the automobile 62 is located on the right side area 50R of the center-left lane 52.

[0064] In the third embodiment, if the lean vehicle 1 has a display device that displays an image captured by the imaging device 11, and a frame surrounding the vehicle in the image captured by the imaging device 11 is displayed together with the captured image, the color of the frame surrounding the vehicle that is the subject of the alarm may be different when the alarm device 13 outputs an alarm and when the alarm device 13 does not output an alarm.

[0065] <Fourth embodiment> Next, a lean vehicle image processing device 10 according to a fourth embodiment of the present invention will be described with reference to Figs. 4 and 5. The fourth embodiment has all of the configurations of the first embodiment. The fourth embodiment may also have the configurations of the second embodiment. The fourth embodiment may also have the configurations of the third embodiment.

[0066] FIG. 4 includes plan views showing two examples (CASE 7 and CASE 8) of a situation in which a lean vehicle 1 and five automobiles 61 to 65 are traveling on a five-lane road. CASE 7 is an example of CASE 1 in the first embodiment. CASE 8 is an example of CASE 2 in the first embodiment. FIG. 4 includes an example of an image P1 captured by the imaging device 11 in CASE 7, and an example of an image P2 captured by the imaging device 11 in CASE 8.

[0067] When the lean vehicle 1 is traveling in the right side area 50R or left side area 50L of the center lane 53 of a five-lane road, and five automobiles 61 to 65 are traveling in a line in the left-right direction ahead of the lean vehicle 1, and the longitudinal distance D between the five automobiles 61 to 65 and the lean vehicle 1 is the specific distance described in the first embodiment, the recognition device 12 has the following configuration.

[0068] When an image P1 captured while the lean vehicle 1 is traveling in the right side area 50R of the center lane 53 includes an image of at least one of a vehicle 62 traveling in the center left lane 52 and a vehicle 65 traveling in the rightmost lane 55, the recognition device 12 is configured to process the image of at least one of the vehicle 62 traveling in the center left lane 52 and the vehicle 65 traveling in the rightmost lane 55 at a resolution lower than the resolution used to recognize the vehicle 63 traveling in the center lane 53 and the vehicle 64 traveling in the center right lane 54. When an image P2 captured while the lean vehicle 1 is traveling in the left side area 50L of the center lane 53 includes an image of at least one of a vehicle 64 traveling in the center right lane 54 and a vehicle 61 traveling in the leftmost lane 51, the recognition device 12 is configured to process the image of at least one of the vehicle 64 traveling in the center right lane 54 and the vehicle 61 traveling in the leftmost lane 51 at a resolution lower than the resolution used to recognize the vehicle 63 traveling in the center lane 53 and the vehicle 62 traveling in the center left lane 52.

[0069] In Fig. 4, image P1 includes images of both automobile 62 and automobile 65, but may include images of only one of automobile 62 and automobile 65. In Fig. 4, image P2 includes images of both automobile 64 and automobile 61, but may include images of only one of automobile 64 and automobile 61. In Fig. 4, images P1 and P2 are images captured by imaging device 11 having a horizontal angle of view slightly smaller than the maximum horizontal angle of view θmax.

[0070] In FIG. 4, the symbols "Hi" and "Lo" attached to the automobiles 61 to 65 have the following meanings. An automobile with the symbol "Hi" attached to it is recognized with high resolution. An automobile with the symbol "Lo" attached to it is recognized with low resolution.

[0071] Next, details of the processing by the recognition device 12 when an image P1 captured while the lean-to vehicle 1 is traveling in the right-side area 50R of the center lane 53 includes an image of at least one of a vehicle 62 traveling in the center-left lane 52 and a vehicle 65 traveling in the right-most lane 55 will be described with reference to Figures 5(a) to 5(c). Note that when an image P2 captured while the lean-to vehicle 1 is traveling in the left-side area 50L of the center lane 53 includes an image of at least one of a vehicle 64 traveling in the center-right lane 54 and a vehicle 61 traveling in the left-most lane 51, the processing by the recognition device 12 is reversed left and right.

[0072] As shown in FIGS. 5(a) to 5(c), the recognition device 12 cuts out a predetermined area from the image P1 that includes an image of a vehicle 63 traveling in the center lane 53 and an image of a vehicle 64 traveling in the center-right lane 54. The cut-out image is referred to as a center area image P1C. The center area image P1C is in the center or approximately the center in the left-right direction of the image P1. The center area image P1C is in the center or approximately the center in the up-down direction of the image P1. The area to be cut out may be set in advance or may be changeable.

[0073] Next, the recognition device 12 recognizes the cars 63 and 64 from the central region image P1C at a predetermined first resolution. The first resolution may be the same as or lower than the resolution of the original image P1 captured by the imaging device 11.

[0074] The recognition device 12 also sets a peripheral region image P1S, which is different from the central region image P1C, from the image P1. The peripheral region image P1S may be the entire image P1, as shown in FIG. 5(a). The peripheral region image P1S may be an image cut out from the image P1 so as not to include the central region image P1C, as shown in FIGS. 5(b) and 5(c), for example. The peripheral region image P1S shown in FIG. 5(b) is an image obtained by removing only the central region image P1C from the image P1. The peripheral region image P1S shown in FIG. 5(c) is an image obtained by removing from the image P1 an image of the same left-right region as the central region image P1C.

[0075] Next, the recognition device 12 processes the surrounding area image P1S at a second resolution lower than the first resolution. Specifically, the recognition device 12 first reduces the resolution of the surrounding area image P1S to the second resolution lower than the first resolution. Then, the recognition device 12 recognizes at least one of the vehicle 62 traveling in the center left lane 52 and the vehicle 65 traveling in the rightmost lane 55 from the surrounding area image P1S. In the example of FIGS. 5(a) to (c), the recognition device 12 recognizes both the vehicle 62 and the vehicle 65 from the surrounding area image P1S.

[0076] 5(a), the recognition device 12 recognizes, from the surrounding area image P1S, not only the automobile 62 and the automobile 65, but also the automobile 63 traveling in the center lane 53 and the automobile 64 traveling in the center right lane 54. In other words, the automobile 63 traveling in the center lane 53 and the automobile 64 traveling in the center right lane 54 are recognized using both the center area image P1C and the surrounding area image P1S. In this case, the recognition device 12 may use the recognition results from the center area image P1C as final data for recognizing the automobiles 63 and 64.

[0077] 5(b) and 5(c), the surrounding area image P1S includes a part of the automobile 64 traveling in the center right lane 54. In this case, the recognition device 12 may or may not recognize the automobile 64 traveling in the center right lane 54 from the surrounding area image P1S.

[0078] The right side area in this specification is included in the right driving area in the specification of Japanese Patent Application No. 2023-100003, which is the basic application of the present application. The left side area in this specification is included in the left driving area in the specification of the same application. The central area in this specification is included in the central driving area in the specification of the same application. The automobile in this specification corresponds to a four-wheeled vehicle in the specification of the same application. The central area image in this specification corresponds to the central recognition area in the specification of the same application. The peripheral area image in this specification corresponds to the peripheral recognition area in the specification of the same application. [Explanation of symbols]

[0079] 1: lean vehicle, 10: lean vehicle image processing device, 11: imaging device, 12: recognition device, 13: warning device, 50: lane, 50C: center area, 50L: left side area, 50R: right side area, 51: leftmost lane, 52: center left lane, 53: center lane, 54: center right lane, 55: rightmost lane, 60, 61 to 65: automobile

Claims

1. An image processing device for a lean vehicle, the image processing device including: an imaging device that captures an image; and a recognition device that recognizes an object from the image acquired by the imaging device, the image processing device being mounted on a lean vehicle, When a single lane is divided into thirds in the left-right direction, an area including the right edge of the single lane is defined as a right side area, an area including the left edge of the single lane is defined as a left side area, and an area between the right side area and the left side area is defined as a center area, If we define the lanes of a five-lane road as the leftmost lane, center left lane, center lane, center right lane, and rightmost lane, from left to right, An image processing device for a lean vehicle, characterized in that the imaging device and the recognition device are configured as follows: the lean vehicle is traveling in the right or left side area of ​​the center lane of the five-lane road, five automobiles are traveling in a line left and right in front of the lean vehicle, and the longitudinal distance between the five automobiles and the lean vehicle is the distance required to capture an image so that the image processing is completed when the TTC (Time to Collision) is 3 seconds, assuming that the time from capturing image to completing image processing is 0.5 seconds and an object approaches the lean vehicle at a relative speed of 15 km / h. The imaging device is While the lean vehicle is traveling in the right side area of ​​the center lane of the five-lane road, the left edge of the rear of the vehicle traveling in the leftmost lane is not captured, but at least the left edge of the rear of the vehicle traveling in the center lane and the right edge of the rear of the vehicle traveling in the center right lane are captured, and, The vehicle includes a monocular camera having a horizontal angle of view configured so that, while the lean vehicle is traveling in the left side area of ​​the center lane of the five-lane road, the right edge of the rear of the vehicle traveling in the rightmost lane is not captured, but at least the right edge of the rear of the vehicle traveling in the center lane and the left edge of the rear of the vehicle traveling in the center left lane are captured. The recognition device Recognizing at least both the vehicle traveling in the center lane and the vehicle traveling in the center right lane from the image captured while the lean vehicle is traveling in the right side area of ​​the center lane of the five-lane road; and, The system is configured to recognize at least both the vehicle traveling in the center lane and the vehicle traveling in the center-left lane from the image captured while the lean vehicle is traveling in the left side area of ​​the center lane of the five-lane road.

2. When the lean vehicle is traveling in the right side area or the left side area of ​​the center lane of the five-lane road, and the five automobiles are traveling in a line in the left-right direction ahead of the lean vehicle, and the longitudinal distance between the five automobiles and the lean vehicle is the distance necessary to capture an image so that the image processing is completed when the TTC is 3 seconds, assuming that the time from capturing an image to completing image processing is 0.5 seconds and an object approaches the lean vehicle at a relative speed of 15 km / h, the imaging device and the recognition device are further configured as follows: The image processing device for a lean vehicle according to claim 1 . The imaging device is While the lean vehicle is traveling in the right side area of ​​the center lane of the five-lane road, an image is captured by the monocular camera so that at least a rear surface of the vehicle traveling in the center lane, a right side surface of the vehicle traveling in the center lane, a rear surface of the vehicle traveling in the center right lane, and a left side surface of the vehicle traveling in the center right lane are included in the image; and, While the lean vehicle is traveling in the left side area of ​​the center lane of the five-lane road, an image is captured by the monocular camera so that at least the rear of the vehicle traveling in the center lane, the left side of the vehicle traveling in the center lane, the rear of the vehicle traveling in the center left lane, and the right side of the vehicle traveling in the center left lane are included in the image. The recognition device recognize at least both the vehicle traveling in the center lane and the vehicle traveling in the center right lane from the image including the rear of the vehicle traveling in the center lane, the right side of the vehicle traveling in the center lane, the rear of the vehicle traveling in the center right lane, and the left side of the vehicle traveling in the center right lane; and, At least both the vehicle traveling in the center lane and the vehicle traveling in the center left lane are recognized from the image that includes the rear of the vehicle traveling in the center lane, the left side of the vehicle traveling in the center lane, the rear of the vehicle traveling in the center left lane, and the right side of the vehicle traveling in the center left lane.

3. The lean vehicle image processing device further includes an alarm device that outputs an alarm to urge a driver to pay attention to the vehicle recognized by the recognition device from the image captured by the imaging device, The alarm device When two automobiles are traveling straight ahead of the lean vehicle, side by side with a gap in the left-right direction, and the size of the gap is such that there is no room in the left-right direction for the width of the automobiles, but there is room in the left-right direction for the width of the lean vehicle, the alarm is not output even if the lean vehicle travels straight towards the center between the two automobiles and approaches them, and, When two automobiles are traveling straight ahead of the lean vehicle, side by side with a gap in the left-right direction, and the size of the gap is such that there is no room in the left-right direction for the width of the lean vehicle, the warning is output when the lean vehicle travels straight towards and approaches the center between the two automobiles, outputting the warning according to the vehicle width of the lean vehicle; The image processing device for a lean vehicle according to claim 1 .

4. The lean vehicle image processing device further includes an alarm device that outputs an alarm to urge a driver to pay attention to a vehicle recognized by the recognition device from the image captured by the imaging device, The alarm device When two automobiles are traveling straight ahead of the lean vehicle, side by side with a gap in the left-right direction, and the size of the gap is such that there is no room in the left-right direction for the width of the automobiles, but there is room in the left-right direction for the width of the lean vehicle, the alarm is not output even if the lean vehicle travels straight towards the center between the two automobiles and approaches them, and, When two automobiles are traveling straight ahead of the lean vehicle, side by side with a gap in the left-right direction, and the size of the gap is such that there is no room in the left-right direction for the width of the lean vehicle, the warning is output when the lean vehicle travels straight towards and approaches the center between the two automobiles, outputting the warning according to the vehicle width of the lean vehicle; The image processing device for a lean vehicle according to claim 2 .

5. When the lean vehicle is traveling in the right side area or the left side area of ​​the center lane of the five-lane road, and the five automobiles are traveling in a line left and right in front of the lean vehicle, and the longitudinal distance between the five automobiles and the lean vehicle is the distance required to capture an image so that the image processing is completed when the TTC is 3 seconds, assuming that the time from capturing image to completing image processing is 0.5 seconds and an object approaches the lean vehicle at a relative speed of 15 km / h, the recognition device is further configured as follows. The recognition device When the image captured while the lean vehicle is traveling in the right side area of ​​the center lane of the five-lane road includes an image of at least one of the vehicle traveling in the center left lane and the vehicle traveling in the rightmost lane, the image of at least one of the vehicle traveling in the center left lane and the vehicle traveling in the rightmost lane is processed at a resolution lower than a resolution for recognizing the vehicle traveling in the center lane and the vehicle traveling in the center right lane, and, When the image captured while the lean vehicle is traveling in the left side area of ​​the center lane of the five-lane road includes an image of at least one of the vehicle traveling in the center right lane and the vehicle traveling in the leftmost lane, the image of at least one of the vehicle traveling in the center right lane and the vehicle traveling in the leftmost lane is processed at a resolution lower than a resolution for recognizing the vehicle traveling in the center lane and the vehicle traveling in the center left lane. The image processing device for a lean vehicle according to any one of claims 2 to 4.

Images