Driving support device and driving support method

Abstract

It supports the safe crossing of vehicles that are permitted to travel on sidewalks. [Solution] The driving assistance device 10 of the present invention includes a time acquisition unit 23 that acquires the estimated time for a moving object to complete its crossing of a road and the remaining time until the traffic light changes to a stop signal when the moving object crosses a road where a traffic light is installed; a crossing determination unit 24 that determines whether or not the moving object is crossing the road diagonally; a risk determination unit 26 that determines the degree of risk to crossing the road based on the estimated time for completion of crossing, the remaining time and whether or not the moving object is crossing diagonally; and a warning processing unit 27 that issues a warning based on the degree of risk.

Description

Technical Field

[0001] The present invention relates to a driving support device and a driving support method.

Background Art

[0002] In recent years, electric vehicles capable of traveling on sidewalks have been widely used. Electric vehicles capable of traveling on sidewalks include, for example, electric vehicles that can be used by elderly people or those who have difficulty walking due to physical disabilities, and are generally referred to as senior cars or wheelchair-type electric wheelchairs.

[0003] In such electric vehicles capable of traveling on sidewalks, it is known that the risk of accidents increases when crossing the road. In order to avoid the danger when crossing the road, for example, when passing through a crosswalk with a signal, the possibility of starting to cross in the automatic driving mode is efficiently determined (see Patent Document 1), or the presence or absence of a safety zone in the crosswalk is detected, and a crossing method is determined based on the limited time given for crossing the crosswalk (see Patent Document 2).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, support considering the case of trying to cross the road diagonally is not provided.

[0006] Therefore, an object of the present invention is to provide a driving support device and a driving support method capable of providing support corresponding to the case where a vehicle capable of traveling on a sidewalk tries to cross the road diagonally.

Means for Solving the Problems

[0007] To solve the above problems, the driving assistance device according to the present invention includes a time acquisition unit that acquires the estimated time for a moving object to complete its crossing of a road where traffic lights are installed, and the remaining time until the traffic lights change to a stop signal, A crossing determination unit that determines whether or not a moving object is crossing the road diagonally. A risk assessment unit that determines the degree of risk to crossing the road based on the estimated time to complete the crossing, the remaining time, and whether or not the moving object is crossing diagonally, It has a warning processing unit that issues warnings based on the degree of risk. [Effects of the Invention]

[0008] According to the present invention, it is possible to support safe road crossings for vehicles that can travel on sidewalks. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 is a block diagram showing an example of the functional configuration of the mobile body 1 in this embodiment. [Figure 2] Figure 2 shows an example of an image obtained after processing by the image processing unit 21. [Figure 3] Figure 3 is a flowchart illustrating the driver assistance process. [Figure 4] Figure 4 shows an example of an image obtained after processing by the image processing unit 21. [Modes for carrying out the invention]

[0010] <One Embodiment> (Configuration of Mobile Unit 1) The following describes the driver assistance device 10 in this embodiment. Figure 1 is a block diagram showing an example of the functional configuration of the parts related to driver assistance in the mobile body 1 on which the driver assistance device 10 is installed. The embodiments described below are all preferred specific examples of the present invention. The numerical values, components, arrangement positions and connection configurations of the components, and processing order in the flowcharts shown in the following embodiments are examples and are not intended to limit the present invention. In addition, the figures are not necessarily strictly accurate.

[0011] The mobile device 1 is a mobile device that can travel on roads, including sidewalks, and may be a mobility scooter, wheelchair, bicycle, or kick scooter. The mobile device 1 is equipped with a driving assistance device 10, which comprises an imaging unit 11, a driving assistance processing unit 12, an information acquisition unit 13, and an output unit 14. Furthermore, each of these components may not only be mounted on the mobile device 1, but may also be configured as other devices owned by the user of the mobile device 1, and information may be exchanged using communication means (not shown). The driving assistance device 10 may also consist of the imaging unit 11, the information acquisition unit 13, and the output unit 14, or any or all of them, configured as separate units in the driving assistance processing unit 12.

[0012] The imaging unit 11 captures an image of the area including the front of the moving body 1. Preferably, the imaging unit 11 is configured to include one or more imaging devices capable of capturing images with a field of view of 180 degrees or more horizontally with respect to the direction of travel of the moving body 1, so that the conditions to the left, right, and rear of the moving body 1 can be detected. The image data acquired by the imaging unit 11 is supplied to the driving support processing unit 12.

[0013] The information acquisition unit 13 can acquire map information stored internally or from an external source via a communication unit (not shown). Furthermore, it can acquire the current location information of the mobile object 1 using, for example, a GNSS (Global Navigation Satellite System), and supply this information to the driving support processing unit 12.

[0014] When the moving body 1 crosses a road where a traffic signal is arranged, the driving support processing unit 12 determines the degree of danger and issues a warning according to the degree of danger.

[0015] The driving support processing unit 12 includes an image processing unit 21, an object detection unit 22, a time acquisition unit 23, a crossing determination unit 24, a congestion determination unit 25, a danger degree determination unit 26, and a warning processing unit 27.

[0016] The image processing unit 21 processes the image data supplied from the imaging unit 11, generates image data that can be processed by the object detection unit 22, and supplies it to the object detection unit 22.

[0017] For example, when the imaging unit 11 performs imaging processing using a wide-angle lens having a horizontal angle of view of 180 degrees or more, that is, a lens generally called a fish-eye lens, the obtained image mainly has two types of distortions, "distortion" and "image plane distortion", and has the characteristic that the distortion becomes stronger as it goes from the center to the outside, and the shape of the photographed object is deformed by this distortion. The image processing unit 21 performs processing to correct the strain and the like resulting from these imaging processes.

[0018] Also, for example, when the object detection unit 22 is configured to perform object detection processing using image data in a total of three directions for front monitoring and left and right monitoring of the moving body 1, the image processing unit 21 generates image data such that at least a part of the boundary portions of the image data in the three directions overlap so that no omission occurs in object detection, and supplies it to the object detection unit 22.

[0019] The object detection unit 22 executes detection processing of objects necessary for performing the following-described processing, such as lanes, outside lane lines, white lines indicating the roadside strip and green belts, curbstones, guardrails, crosswalks, traffic signals, vehicles, pedestrians, and other moving bodies, from the image data processed by the image processing unit 21.

[0020] As a method for the object detection unit 22 to detect an object and the surrounding environment, for example, pattern matching or the like can be used, but other known methods may also be used. The object detection unit 22 supplies the detection result of the object and the supplied image data to the time acquisition unit 23 and the crossing determination unit 24.

[0021] The time acquisition unit 23 acquires the detection result by the object detection unit 22 and calculates the width of the lane that the moving body 1 is crossing. Then, when the moving body 1 is crossing the road where the traffic signal is installed, the time acquisition unit 23 calculates the crossing prediction time from the traveling speed of the moving body 1 and the width of the lane, and calculates the predicted crossing completion time of the moving body 1. Further, the time acquisition unit 23 estimates the remaining time until the traffic signal turns red. The time acquisition unit 23 supplies the predicted crossing completion time of the moving body 1 and the estimation result of the remaining time until the traffic signal turns red to the risk determination unit 26.

[0022] The time acquisition unit 23 may acquire information on the width of the lane that the moving body 1 is crossing by referring to the map information acquired by the information acquisition unit 13 and the current position information of the moving body 1. The time acquisition unit 23 may calculate the traveling speed of the moving body 1 from the change amount of the current position information of the moving body 1.

[0023] Based on the detection result by the object detection unit 22, the crossing determination unit 24 determines whether the moving body 1 is crossing the road where the traffic signal is installed, whether it is a diagonal crossing, or whether it is on a crosswalk, and supplies the determination result to the risk determination unit 26. Note that the diagonal crossing is assumed to be a case where the moving body heads towards the crosswalk from a position several meters away from the crosswalk or vice versa, excluding cases where diagonal crossing is allowed at scramble intersections or the like.

[0024] Specifically, the crossing determination unit 24 detects the direction of travel of the moving body 1 relative to the road based on the detection results by the object detection unit 22. For example, it determines whether the angle between the detection results of edge portions such as lines, road edge lines, white lines or green belts indicating road shoulders, curbs, guardrails, and surrounding buildings and the direction estimated to be the direction of travel of the moving body 1 is approximately right-angled, or whether the angle between the detection results of the edge portion of the pedestrian crossing and the direction estimated to be the direction of travel of the moving body 1 is approximately parallel. The amount of deviation from parallel or right-angled for a crossing to be determined to be diagonal is a value that can be set as appropriate. The crossing determination unit 24 may also determine the direction of travel of the moving body 1 relative to the road by detecting the map information acquired by the information acquisition unit 13 and the information on the change in the current position information of the moving body 1, and determine that it is a diagonal crossing.

[0025] The congestion determination unit 25 determines the degree of congestion on the road being crossed by the moving object based on the detection results from the object detection unit 22, and supplies the determination result to the risk determination unit 26.

[0026] The congestion determination unit 25 can determine the degree of congestion based on the number of pedestrians, bicycles, strollers, or other vehicles that can travel on other sidewalks, similar to the mobile unit 1. Alternatively, the congestion determination unit 25 may determine the degree of congestion as "crowded" or "not crowded" rather than strictly counting the number of people. Furthermore, the congestion determination unit 25 may be able to determine the degree of congestion on sidewalks and crosswalks by receiving information from external cameras installed at intersections or from external devices via a communication unit (not shown).

[0027] The risk determination unit 26 determines the degree of risk to the mobile body 1 crossing the road based on the estimated crossing completion time supplied from the time acquisition unit 23, the estimated remaining time until the traffic light turns red, and the determination results from the crossing determination unit 24 and the congestion determination unit 25, and supplies the determination result to the warning processing unit 27.

[0028] For example, the risk assessment unit 26 determines that the risk is high when the remaining time until the traffic light turns red is shorter than the estimated time to complete the crossing, or when there is insufficient time. Furthermore, the risk assessment unit 26 determines that the risk is high if the crossing is diagonal and the moving object 1 is not present on the crosswalk. In addition, the risk assessment unit 26 determines that the risk is high if the level of congestion is high.

[0029] Furthermore, the risk assessment unit 26 determines that the estimated crossing completion time will be even longer if the crossing is diagonal or if the level of congestion is high. If a diagonal crossing is detected, the risk assessment unit 26 may recalculate the estimated crossing completion time from the angle, or it may multiply the estimated crossing completion time for a normal crossing by, for example, 1.2 times. Also, if a high level of congestion is detected, the risk assessment unit 26 may multiply the estimated crossing completion time for a normal crossing by a predetermined coefficient according to the level of congestion.

[0030] The warning processing unit 27 acquires the detection result from the risk determination unit 26, determines the processing that matches the warning output method of the mobile body 1, and controls the issuance of an alarm by the output unit 14.

[0031] In other words, if the output unit 14, described later, is an LED (light-emitting diode) mounted in a position visible to the user of the mobile device 1, the warning processing unit 27 decides whether to light up or flash the LED according to the degree of danger. Also, if the output unit 14, described later, is a liquid crystal screen mounted in a position visible to the user of the mobile device 1, the warning processing unit 27 decides what to warn the user about according to the degree of danger. Also, if the output unit 14, described later, is a buzzer or speaker capable of outputting sound, the warning processing unit 27 decides what to output sound according to the degree of danger.

[0032] The output unit 14 outputs warnings, etc., to the user of the mobile unit 1 or others near the mobile unit 1, based on the control of the warning processing unit 27 of the driving assistance processing unit 12. For example, this could be hazard lights or turn signals already installed in conventional vehicles, or it could be an LED or LCD screen mounted in a position visible to the user of the mobile unit 1 or others near the mobile unit 1, or a buzzer or speaker capable of outputting sound. Furthermore, the output unit 14 may also be equipped with a communication unit, and for example, it may transmit control signals that command the user of the mobile unit 1 to turn on a lamp or emit a warning sound on a wearable device such as a smartwatch.

[0033] (Specific example) Next, with reference to Figure 2, the specific processing of the driver assistance processing unit 12 will be explained.

[0034] For example, if the forward-facing monitoring image generated by the image processing unit 21 is the image shown in Figure 2, the object detection unit 22 detects the crosswalk 51, traffic lights 52, roadway outer line 53, pedestrians 54-1 to 54-2, etc.

[0035] The time acquisition unit 23 determines that the mobile body 1 is crossing a road with traffic lights because the pedestrian crossing 51 and traffic light 52 are in front of the mobile body 1, and the traffic light 52 is getting closer in the image as time progresses. Therefore, it calculates the estimated crossing time from the mobile body 1's speed and the width of the roadway, and calculates the estimated time for the mobile body 1 to complete its crossing. Furthermore, the time acquisition unit 23 estimates the remaining time until the traffic light turns red.

[0036] When the time acquisition unit 23 calculates the estimated crossing completion time, it estimates the width of the road to be crossed and calculates the crossing time of the mobile body 1 from the speed of the mobile body 1 (up to 6 km / h if the mobile body 1 is a so-called senior mobility scooter). To estimate the width of the road to be crossed, map information acquired from the information acquisition unit 13 may be used, or the distance from the current position of the mobile body 1 to the opposing traffic light may be calculated using the image detection results of the opposing traffic light, which has a fixed size, or, simply, it may be determined from the image detection results whether "there is still time until the crossing is finished" or "the crossing will be finished soon".

[0037] Furthermore, the time acquisition unit 23 may, for example, acquire information from a communication unit (not shown) such as a traffic signal information utilization driving support system or other sources to estimate the remaining time until the traffic signal turns red, or it may determine that the remaining time is "short" when it detects from the image that the pedestrian traffic signal is flashing, rather than using an exact number of seconds.

[0038] The crossing determination unit 24 then determines that the moving body 1 is crossing a road where traffic signals are installed, and calculates the angle between the detection result of at least one edge of the pedestrian crossing 51 and the outer edge line of the roadway 53 and the direction of travel of the moving body 1, thereby detecting the angle of travel of the moving body 1 relative to the road. The crossing determination unit 24 determines that the moving body 1 is not making a diagonal crossing because the angle of travel of the moving body 1 relative to the road is less than a predetermined amount of deviation. The crossing determination unit 24 also detects the positional relationship between the moving body 1 and the pedestrian crossing 51 and determines that the moving body 1 is traveling on the pedestrian crossing.

[0039] Furthermore, the congestion determination unit 25 determines that the road is not congested because pedestrians 54-1 to 54-2 have not yet begun crossing.

[0040] The risk assessment unit 26 then determines the degree of risk for the mobile vehicle 1 to cross the road based on the above assessment results. In this case, the mobile vehicle 1 is not driving diagonally, is traveling on a pedestrian crossing, and the degree of congestion is determined to be low. Therefore, the risk assessment unit 26 determines that the risk is high when the remaining time until the traffic light turns red is shorter than the estimated time to complete the crossing, or when there is no margin for error, and determines that the risk is low when there is ample time to complete the crossing.

[0041] The risk assessment unit 26 may determine the level of risk in multiple stages or in two stages, either safe or unsafe. For example, the risk assessment unit 26 can determine that a situation is safe only if there is ample time to complete the crossing, it is not a diagonal crossing, and the level of congestion is low, and that it is dangerous if any of these conditions are not met. Alternatively, the risk assessment unit 26 can determine that the risk is very high if the remaining time until the traffic light turns red is shorter than the predicted time to complete the crossing, and that the risk is moderately high if there is no time to spare. The risk assessment unit 26 may also evaluate the level of risk in multiple stages by indexing the amount of time to complete the crossing, whether or not it is a diagonal crossing, and the level of congestion, and summing these values. Furthermore, the relationship between each condition and the level of risk may be appropriately set, for example, depending on the age of the user of the controlled vehicle, the presence or absence of a disability, and the degree of the disability.

[0042] The warning processing unit 27 controls the issuance of an alarm by the output unit based on the determination result of the risk determination unit 26.

[0043] Furthermore, if the hazard determination unit determines a multi-level hazard, the warning processing unit 27 can change the content of the warning issued depending on the level of hazard determination. For example, if the hazard level is higher or the situation is more urgent, the unit can increase the intensity of the LED light, make it flash, or increase the volume of the buzzer or speaker to strongly warn the user of the mobile device 1.

[0044] For example, if the forward monitoring image generated by the image processing unit 21 is the image shown in Figure 2, and the angle between the detection result of at least one edge of the pedestrian crossing 51 or the outer edge line of the roadway 53 and the direction of travel of the moving body 1 changes and exceeds a predetermined amount of deviation, the crossing determination unit 24 determines that the moving body 1 is crossing diagonally. Therefore, the degree of danger determined by the danger determination unit 26 increases, and the warning processing unit 27 controls the output unit to increase the light intensity of the LED, make it blink, or increase the volume of the buzzer or speaker.

[0045] Furthermore, for example, if it is determined that the person is crossing diagonally and not on a pedestrian crossing, the danger level determined by the danger level determination unit 26 will increase further, and the warning processing unit 27 will control the output unit to further increase the light intensity of the LED, increase the flashing speed, or further increase the volume of the buzzer or speaker.

[0046] (Driving support processing) Next, we will explain the driver assistance process by referring to the flowchart in Figure 3.

[0047] In step S1, the imaging unit 11 captures an image with a field of view of 180 degrees or more horizontally with respect to the direction of travel of the moving body 1, so as to be able to detect vehicles on the left and right when crossing the roadway, and supplies it to the image processing unit 21 of the driving support processing unit 12.

[0048] In step S2, the image processing unit 21 processes the image data supplied from the imaging unit 11, generates image data that can be processed by the object detection unit 22, and supplies it to the object detection unit 22.

[0049] In step S3, as explained with reference to Figure 2, the object detection unit 22 performs the object detection processing necessary for subsequent processing from the supplied image and supplies the object detection results, along with information about the surrounding environment, to the time acquisition unit 23 and the cross-section determination unit 24.

[0050] In step S4, the time acquisition unit 23 and the crossing determination unit 24 determine whether the moving body 1 is crossing a road with traffic lights. If it is determined in step S4 that the moving body 1 is not crossing a road with traffic lights, the process returns to step S1, and the subsequent processes are repeated.

[0051] If it is determined in step S4 that the moving object 1 is crossing a road with traffic lights, in step S5 the time acquisition unit 23 acquires the estimated time for the moving object 1 to complete its crossing and the remaining time until the traffic lights turn red, and supplies these to the risk determination unit 26.

[0052] In step S6, the crossing determination unit 24 determines whether the crossing by the moving body 1 is diagonal and whether the moving body 1 is on a pedestrian crossing, and supplies the determination results to the risk determination unit 26.

[0053] In step S7, the congestion determination unit 25 determines the degree of congestion on the road being crossed by the moving object based on the detection results from the object detection unit 22, and supplies the determination result to the risk determination unit 26.

[0054] In step S8, the risk determination unit 26 determines the degree of risk to the mobile body 1 crossing the road based on the supplied information and supplies the determination result to the warning processing unit 27.

[0055] In step S9, the warning processing unit 27 obtains the detection result from the risk determination unit 26, determines the processing that matches the warning output method of the mobile body 1, controls the issuance of the alarm by the output unit 14, and the process returns to step S1, and the subsequent processing is repeated.

[0056] Through this process, the captured image is processed to detect objects near the moving object 1. When crossing a road with traffic lights, the estimated time to complete the crossing and the lingering presence until the light turns red are obtained. It is also determined whether the crossing is diagonal or not, and whether the vehicle is on a pedestrian crossing. Based on this information, the degree of danger is determined, and a warning is issued based on the determination result, thereby supporting the safe crossing of vehicles that can travel on sidewalks.

[0057] In the embodiment described above, the risk determination unit 26 determined the degree of risk to the moving body 1 crossing the road based on the estimated crossing completion time supplied from the time acquisition unit 23, the estimated remaining time until the traffic light turns red, and the determination results of the crossing determination unit 24 and the congestion determination unit 25. However, the degree of risk can also be determined based on the estimated crossing completion time, the estimated remaining time, and the determination result of the crossing determination unit 24.

[0058] (Other configuration examples) (Detection of driving environment and driving conditions) The driving support processing unit 12 may perform a more detailed detection of the driving environment and driving state of the moving body 1 based on the detection results from the object detection unit 22.

[0059] For example, the driving support processing unit 12 may detect the driving state, including whether the moving object 1 is traveling in a direction substantially parallel to the road, crossing the road, or stopped, and if stopped, which direction it is most likely to travel next, based on the detection results from the object detection unit 22 and the temporal progression of the detection results.

[0060] For example, the driving assistance processing unit 12 detects, based on the detection results from the object detection unit 22 or information acquired by the information acquisition unit 13, whether there is a sidewalk or similar on the road where the moving object 1 is located, the location of the moving object 1 on the road, for example, whether it is on a sidewalk or on the road, whether it is traveling on the right side or the left side of the road, whether it is on the road or on a sidewalk, and whether there are pedestrian crossings, intersections, T-junctions, etc. nearby. Furthermore, the detection results for pedestrians who should be on the sidewalk or shoulder, and vehicles that should be on the roadway, can improve the accuracy of detecting whether the location of the moving object 1 is on the roadway, sidewalk or shoulder.

[0061] The driving support processing unit 12 then detects the relationship between the position and direction of travel of the moving body 1 when there are crosswalks, traffic lights, vehicles such as automobiles, pedestrians, or other objects near the moving body 1, based on the information supplied from the object detection unit 22 and the information supplied from the information acquisition unit 13.

[0062] Furthermore, the object detection unit 22 may detect objects such as the sun, clouds, raindrops, snow, or icy roads based on the supplied image. Additionally, the information acquisition unit 13 may acquire information about the environment surrounding the mobile body 1, such as the weather, brightness, and road conditions, via a communication unit (not shown) or based on information from sensors (not shown) mounted on the mobile body 1. The driving support processing unit 12 may, based on the detection results from the object detection unit 22 and the information supplied from the information acquisition unit 13, detect the relationship between the position and direction of travel of the mobile body 1 when there are crosswalks, traffic lights, vehicles such as automobiles, pedestrians, or other objects near the mobile body 1.

[0063] (Assessment of risk level) The risk determination unit 26 detects the risk level of the mobile body 1's travel based on the detection results of the travel environment and travel state of the mobile body 1, making a comprehensive judgment based on the travel environment and travel state for each case in which the mobile body 1 is crossing the road, traveling, and stopped. Preferably, the risk determination unit 26 can acquire various information that can be used to detect the risk level of the mobile body 1's travel, such as whether or not there is a sidewalk or shoulder on the road where the mobile body 1 is located, and the speed limit for vehicles on the corresponding road or nearby roads, based on the information supplied from the information acquisition unit 13.

[0064] The risk determination unit 26 determines the degree of risk when the moving object 1 is crossing the road, using factors such as whether it is crossing diagonally, the presence or absence of a pedestrian crossing or traffic light, the color of the traffic light, and the detection results of nearby vehicles. Specifically, the risk determination unit 26 determines that crossing at a pedestrian crossing without traffic lights is more dangerous than crossing at a pedestrian crossing with traffic lights, and that crossing at a location without a pedestrian crossing is more dangerous than crossing at a location with a pedestrian crossing. In any case, it determines that diagonal crossing is more dangerous. Furthermore, it is preferable that the risk determination unit 26 can determine the degree of risk by taking into account not only the predicted crossing time of the moving object 1 but also the frequency of nearby vehicles.

[0065] Furthermore, when the mobile body 1 is in normal motion, the risk determination unit 26 detects the mobile body 1's position, the surrounding environment such as the presence or absence of vehicles and pedestrians, and the possibility of diagonal crossing inferred from the mobile body 1's direction of travel, and determines the risk level of the mobile body 1.

[0066] Furthermore, even when the moving object 1 is stopped, the risk determination unit 26 detects the stopping position and duration of the moving object 1, the surrounding environment such as the presence or absence of vehicles and pedestrians, and the possibility of diagonal crossing inferred from the direction the front of the moving object 1 is facing, and detects the risk level of the moving object 1.

[0067] Furthermore, the risk determination unit 26 can determine the degree of risk based on the temporal changes in the driving environment and driving state of the moving object 1. For example, even if the possibility of diagonal crossing starting is detected and a warning is issued by the warning processing unit 27, if it is detected that crossing has started, the risk determination unit 26 may determine that the degree of risk has increased further.

[0068] Furthermore, if a pedestrian crossing exists in the direction of travel, it is preferable that the driving support processing unit 12 can perform preparatory processing for the risk assessment unit 26 to determine the risk of crossing, such as the timing of the signal change and the volume of traffic, in preparation for the moving object 1 crossing the corresponding signal.

[0069] (Control of mobile unit 1 according to the level of danger) Furthermore, if the warning processing unit 27 is capable of controlling the operation of any of the following: the motor that drives the wheels of the mobile body 1, the brake-related mechanism for stopping the vehicle's movement, or the steering wheel or other mechanism for controlling the direction of the vehicle's movement, the warning processing unit 27 will determine the operation control content that is appropriate for safety, according to the controllable content. For example, if the possibility of diagonal crossing is detected and a warning is issued by the output unit 14, but diagonal crossing is detected and safety is further deteriorated, the warning processing unit 27 can execute a process to forcibly stop the mobility scooter.

[0070] Furthermore, the warning processing unit 27 not only determines the content of the warning to be issued to the user of the mobile unit 1, but can also determine, if necessary, the output of a warning to alert pedestrians and vehicles near the mobile unit 1. The output unit 14 can, for example, turn on the lights to alert surrounding pedestrians and vehicles in poor visibility conditions or while crossing a road.

[0071] (The first specific example in other configuration examples) For example, let's consider the case where the forward monitoring image generated by the image processing unit 21 is the image shown in Figure 4, and the left monitoring image is the image shown in Figure 2.

[0072] In this case, the object detection unit 22 detects lanes 71, road edge lines 72, white lines 73 indicating the roadside, sidewalk edge lines 74, vehicles 75, pedestrians 76-1 to 76-2, etc. from the forward monitoring image shown in Figure 4. The object detection unit 22 also detects pedestrian crossings 51, traffic lights 52, road edge lines 53, pedestrians 54-1 to 54-2, etc. from the left monitoring image shown in Figure 2.

[0073] The driver assistance processing unit 12 can use these detection results to calculate the angles between the direction of travel of the moving object 1 and the lane 71, road edge line 72, white line 73 indicating the road shoulder, and sidewalk edge line 74 detected from the forward monitoring image shown in Figure 4, as well as the angles between the direction of travel of the moving object 1 and the crosswalk 51, road edge line 53 detected from the left monitoring image shown in Figure 2. The driver assistance processing unit 12 can then detect the positional relationship between these and the moving object 1. As a result, the driver assistance processing unit 12 can detect that the moving object 1 is on the sidewalk in front of a crosswalk with a signal and is facing in a direction approximately parallel to the road.

[0074] The driving support processing unit 12 then detects whether the moving object 1 is moving or stopped based on the temporal progression of the information detected by the object detection unit 22. The driving support processing unit 12 can also subsequently detect, based on the temporal progression of the detection results from the object detection unit 22, whether the moving object 1 is moving in a nearly straight line along the sidewalk, approaching the roadway, or changing direction and beginning to cross a pedestrian crossing.

[0075] The danger determination unit 26 then informs the warning processing unit 27 that the mobile body 1 is in a safe state if it is located on a sidewalk in front of a signalized pedestrian crossing, at a certain distance from the roadway, facing approximately parallel to the road, driving normally, or stopped for a certain period of time, and there are no problems with the weather, brightness, or road surface conditions. In such cases, there is no need to issue a warning to the user of the mobile body 1 or to surrounding vehicles or pedestrians. If the output unit 14 has a display output function that indicates safety, such as lighting a green LED or displaying text such as "Driving safely," the warning processing unit 27 controls these safety-indicating display outputs.

[0076] Furthermore, the danger determination unit 26 considers the case where the mobile body 1 has not yet begun to cross but has simply not changed direction from its previous direction of travel, and if it remains stopped near the pedestrian crossing for a certain period of time or longer, it notifies the warning processing unit 27 that there is a possibility that the mobile body is preparing to cross the road. The warning processing unit 27 decides to notify the user of the mobile body 1 of information in text or voice, such as, "Are you going to cross the road? If you are going to cross the road, change the direction of the vehicle toward the pedestrian crossing, check left and right, and then cross," and the warning processing unit 27 controls the output of these notification contents from the output unit 14.

[0077] Furthermore, the risk assessment unit 26 determines that safety is reduced if there are problems with the weather, brightness, or road surface conditions, specifically in cases of rain, during the hours from dusk to night, or when the road surface is frozen, even if there are no other problems.

[0078] Furthermore, if there are problems with the weather, brightness, or road surface conditions, it is preferable to notify not only the user of the mobile body 1 but also nearby vehicles and pedestrians of the presence of the mobile body 1. Therefore, the warning processing unit 27 decides to turn on lights to draw attention to the surrounding area, and the warning processing unit 27 controls the turning on of the lights.

[0079] (A second specific example in other configuration examples) Next, we will explain the case where the forward-facing monitoring image changes from the one shown in Figure 4 to the one shown in Figure 2.

[0080] In this case, the object detection results from the forward-monitoring image of the object detection unit 22 change from the lane 71, road edge line 72, white line 73 indicating the roadside, sidewalk edge line 74, vehicle 75, and pedestrians 76-1 to 76-2 shown in Figure 4, to the crosswalk 51, traffic light 52, road edge line 53, and pedestrians 54-1 to 54-2 shown in the left-monitoring image of Figure 2.

[0081] In this case, it is detected that the moving object 1 is on the sidewalk immediately before a crosswalk with a signal, and has started crossing the crosswalk, or is about to start crossing. At this time, the time acquisition unit 23 estimates the width of the road to be crossed and calculates the crossing time of the moving object 1.

[0082] The risk assessment unit 26 makes a comprehensive judgment based on this information to detect the safety of the moving object 1. For example, if there is a possibility that the object may not be able to cross the pedestrian crossing before the traffic light turns red, or if the object is crossing diagonally, the risk is determined to be very high.

[0083] The warning processing unit 27 acquires the detection results from the risk determination unit 26 and determines the method of outputting warnings or the controllable content of the mobile body 1 that matches the content. For example, if, at the start of crossing a pedestrian crossing, the possibility of not being able to cross before the signal turns red or the possibility of diagonal crossing is detected, and a warning is issued by the processing of the warning processing unit 27, but the start of crossing is detected and it is detected that the safety has further deteriorated, the risk determination unit 26 may notify the warning processing unit 27 to forcibly stop the mobile body 1.

[0084] (A third specific example in other configuration examples) Next, we will explain what happens when road crossing is detected at a pedestrian crossing without traffic signals.

[0085] The risk assessment unit 26 determines that safety is high if no vehicles are detected in the left and right monitoring images, or if detected vehicles are stopped, or if it predicts the arrival time of a vehicle based on the distance and speed from the detected vehicle and estimates that the crossing will be completed before the vehicle arrives. In such cases, there is no need to issue a warning to the user of the mobile device 1 or to surrounding vehicles and pedestrians. If the output unit 14 has a display output function that indicates safety, such as lighting a green LED or displaying text such as "Safe driving in progress", the risk assessment unit 26 notifies the warning processing unit 27 to control these safety display outputs.

[0086] In response, if a vehicle that is not stopped is detected in any of the monitoring images, the risk determination unit 26 predicts the vehicle's arrival time from the distance and speed from the detected vehicle. If it is estimated that the crossing will not be completed by the time the vehicle arrives, or if the moving object 1 is crossing diagonally, the risk determination unit 26 determines that the risk level is very high. The risk determination unit 26 notifies the warning processing unit 27 to execute warnings and controls appropriate to the risk level.

[0087] (A fourth specific example in other configuration examples) Next, we will explain the case where the object detection unit 22 does not detect a signal or pedestrian crossing from the forward monitoring image (not shown), which is the image for monitoring the right side as shown in Figure 4.

[0088] In this case, the moving object 1 may be about to begin crossing an unmarked pedestrian crossing or a road without a pedestrian crossing. Also, vehicle 75 may be passing through the crossing route soon. Based on this information, the danger determination unit 26 determines that the moving object 1 is in a very dangerous situation, and the warning processing unit 27 decides to make an emergency stop to the moving object 1 and issue the strongest possible warning, such as illuminating or flashing a red LED, or emitting a loud warning sound.

[0089] Furthermore, if it is detected that the mobile object 1 has begun or is about to begin crossing an unsignaled pedestrian crossing or a road without a pedestrian crossing, it may be determined to be a dangerous crossing and a warning issued, even if no vehicles are detected nearby. The location where the mobile object 1 is about to cross may be detected by referring to the map information acquired by the information acquisition unit 13 and the current location information of the mobile object 1.

[0090] (Other possible actions) Furthermore, if the mobile body 1 is attempting to cross the road in an area without a crosswalk, the warning processing unit 27 decides to notify the user of the mobile body 1 of text or voice information to guide it to a crosswalk or traffic light if one is nearby, and the warning processing unit 27 controls the output of these notification contents from the output unit 14.

[0091] Furthermore, regardless of the conditions under which a road crossing occurs, if diagonal crossing is detected, it will be judged as a dangerous situation, regardless of the presence of vehicles or pedestrian crossings. In addition, diagonal crossing at intersections or T-junctions, or diagonal crossing near intersections or T-junctions, may be detected as having an even greater risk.

[0092] Furthermore, if the possibility of diagonal crossing is detected, not only a warning may be issued, but the system may also control mechanisms such as handles to control the direction of travel of the mobile body 1, orienting the mobile body 1 perpendicular to the roadway, and issuing a warning to encourage checking left and right.

[0093] Furthermore, the risk determination unit 26 preferably stores the detection results from the object detection unit over time to measure the frequency of traffic of vehicles on the road while the moving body 1 is traveling, and uses this to detect safety when crossing under conditions without traffic lights. In situations where vehicles frequently pass, it is preferable that a warning for crossing is not issued only when the safety level is even higher.

[0094] In the embodiment described above, the driving assistance device 10 is mounted on the mobile body 1. In contrast, the driving assistance device 10 may be mounted on a portable device carried by a pedestrian while walking, and may assist the pedestrian's walking. The driving assistance device 10 may be mounted on a communication device such as a smartphone, incorporated into clothing, a cane, a hat, etc. worn by a pedestrian, or be a device that can be attached to a person, an animal, a stroller, etc. The driving assistance device 10 may be configured as a combination of these types of devices, or a part of the configuration of the driving assistance device 10 may be mounted on a general-purpose device such as a server to which it is connected via a communication unit (not shown).

[0095] The series of processes described above can be executed by hardware or by software. When the series of processes are executed by software, the programs that make up the software are installed from a program storage medium on a computer that is built into dedicated hardware, or on a general-purpose computer that can perform various functions by installing various programs.

[0096] The programs executed by the computer may be programs that are processed chronologically in the order described herein, or they may be programs that are processed in parallel or at necessary times, such as when a call is made.

[0097] [Note] The contents described in some of the embodiments above can be understood, for example, as follows:

[0098] (1) Warning to users of mobile vehicle 1 The above-mentioned driving assistance device 10 is, A time acquisition unit 23 (step S5) acquires the estimated time for the moving object to complete its crossing of a road where traffic lights are installed, and the remaining time until the traffic lights change to a stop signal, A crossing determination unit 24 (step S6) that determines whether or not the moving object is crossing the road diagonally, A risk determination unit 26 (step S8) determines the degree of risk to crossing the road based on the estimated time to complete the crossing, the remaining time, and whether or not the moving object is crossing diagonally, Based on the degree of danger, a warning processing unit 27 (step S9) issues a warning. It holds.

[0099] With this configuration, the degree of danger can be determined based on the estimated time to complete the crossing, the remaining time, and whether or not the moving object is crossing diagonally, thus allowing for the warning of moving objects crossing diagonally.

[0100] (2) Detection of congestion level Furthermore, the aforementioned driving assistance device 10 is The system further includes a congestion determination unit 25 (step S7) that acquires the degree of congestion in the surrounding area where the road is crossed. The risk assessment unit 26 determines the risk level based on the estimated crossing completion time, remaining time, whether the moving object is crossing diagonally, and the degree of congestion. It is possible.

[0101] With this configuration, it is possible to alert people to the degree of danger based on the level of congestion on the road that the moving object is about to cross.

[0102] (3) Control of warning intensity Furthermore, in the above-mentioned driving support device 10, Warnings include being presented as the illumination or flashing of a light unit mounted on a mobile vehicle. The warning processing unit 27 controls the mode of illumination or flashing based on the degree of danger. It is possible.

[0103] This configuration allows for the issuance of more easily understandable warnings to users of the mobile device or to external vehicles.

[0104] (4) Determining the safety of diagonal crossing in areas without pedestrian crossings. Furthermore, in the above-mentioned driving support device 10, The crossing determination unit 24 further determines whether the moving object is diagonally crossing a place other than a pedestrian crossing, The risk assessment unit 26 determines that the risk level is higher when a moving object is diagonally crossing a location other than a pedestrian crossing compared to when the moving object is crossing a pedestrian crossing. It is possible.

[0105] With this configuration, when a moving object is diagonally crossing a location other than a pedestrian crossing, it is possible to draw attention to a higher level of danger compared to when the moving object is crossing a pedestrian crossing. [Explanation of Symbols]

[0106] 1...Moving object, 10...Driving support device, 11...Imaging unit, 12...Driving support processing unit, 13...Information acquisition unit, 14...Output unit, 21...Image processing unit, 22...Object detection unit, 23...Time acquisition unit, 24...Crossing determination unit, 25...Congestion determination unit, 26...Danger level determination unit, 27...Warning processing unit

Claims

1. A time acquisition unit that acquires the estimated time for a moving object to complete its crossing of a road where a traffic light is installed, and the remaining time until the traffic light changes to a stop signal, A crossing determination unit that determines whether the moving body is crossing the road diagonally. A risk determination unit that determines the degree of risk to crossing the road based on the estimated time to complete the crossing, the remaining time, and whether or not the moving object is crossing diagonally, Based on the aforementioned degree of risk, a warning processing unit issues a warning. A driving assistance device characterized by having the following features.

2. The system further includes a congestion determination unit that acquires the degree of congestion in the surrounding area crossed by the aforementioned road. The risk determination unit determines the risk level based on the estimated crossing completion time, the remaining time, whether the moving object is crossing diagonally or not, and the degree of congestion. The driving support device according to claim 1, characterized in that

3. The warning includes being presented as the illumination or flashing of a light unit mounted on the mobile body, The warning processing unit controls the mode of illumination or flashing based on the degree of danger. The driving support device according to claim 1, characterized in that

4. The crossing determination unit further determines whether the moving object is diagonally crossing a place other than a pedestrian crossing, The risk determination unit determines that the risk level is higher when the moving object is crossing diagonally at a location other than a pedestrian crossing compared to when the moving object is crossing a pedestrian crossing. The driving support device according to claim 1, characterized in that

5. In a driver assistance method for a driver assistance system that assists the driving of vehicles that can travel on sidewalks, A time acquisition step in which a moving object crosses a road where a traffic light is installed, and obtains the estimated time for the moving object to complete its crossing of the road and the remaining time until the traffic light changes to a stop signal, A crossing determination step to determine whether the moving object is crossing the road diagonally. A risk determination step in which the degree of risk to crossing the road is determined based on the estimated time for completion of the crossing, the remaining time, and whether or not the moving object is crossing diagonally, A warning processing step is performed to issue a warning based on the aforementioned level of risk. A driving assistance method characterized by including the following.

Images