Driving assistance control device, driving assistance method, and computer program

Abstract

To perform deceleration stop control when a section line of a vehicle traveling lane and a preceding vehicle cannot be recognized.SOLUTION: The driving support control device 6 of the vehicle 100 performs the driving support to stop the vehicle 100 when it is determined that the driver of the vehicle 100 is in the abnormal state in which it is difficult for the driver to continue driving the vehicle 100. When the recognition level of the lane markings of the own vehicle traveling lane in which the vehicle 100 is traveling is lower than the predetermined level and the recognition level of the pair of left and right roadway division objects or roadway division lines that divide the roadway on the road in the same direction as the traveling direction of the vehicle 100 is equal to or higher than the predetermined level during the driving assistance, the driving assistance ECU 10 is configured to stop the vehicle 100 between the pair of left and right roadway division objects or roadway division lines.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a driving assistance control device, a driving assistance method, and a computer program. [Background technology]

[0002] Patent Document 1 discloses a conventional vehicle cruise control device that is configured to perform deceleration / stop control to decelerate the vehicle and hold the vehicle in a stopped state when it is determined that the driver is in an abnormal state in which he or she has lost the ability to drive the vehicle. Specifically, this conventional vehicle cruise control device is configured to stop the vehicle along the dividing line of the lane in which the vehicle is traveling (the vehicle's own lane) if the dividing line of the lane in which the vehicle is traveling (the vehicle's own lane) can be recognized, and to stop the vehicle along the traveling trajectory of the vehicle ahead if the dividing line of the lane in which the vehicle is traveling cannot be recognized. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-109559 Summary of the Invention [Problem to be solved by the invention]

[0004] However, this conventional vehicle cruise control device halts deceleration and stop control when it is unable to recognize both the lane markings of the vehicle's own lane and the vehicle ahead.

[0005] The present invention was made with an eye on such problems, and aims to enable deceleration and stopping control to be performed even when both the lane markings of the vehicle's own driving lane and the vehicle ahead cannot be recognized. [Means for solving the problem]

[0006] In order to solve the above problems, a vehicle driving assistance control device according to one aspect of the present invention is configured to provide driving assistance to stop the vehicle when it is determined that the driver of the vehicle is in an abnormal state making it difficult for him to continue driving the vehicle, and to stop the vehicle between the pair of left and right roadway division objects or roadway division lines when, during driving assistance, the recognition level of the dividing line of the vehicle's own driving lane in which the vehicle is traveling is below a predetermined level and the recognition level of a pair of left and right roadway division objects or roadway division lines that divide the road in the same direction as the vehicle's traveling direction is above a predetermined level.

[0007] In addition, a vehicle driving assistance method according to one aspect of the present invention is implemented by a control device, and provides driving assistance to stop the vehicle when it is determined that the driver of the vehicle is in an abnormal state where it is difficult to continue driving the vehicle.During driving assistance, if the recognition level of the dividing line of the vehicle's own driving lane in which the vehicle is traveling is below a predetermined level and the recognition level of a pair of left and right roadway dividing objects or roadway dividing lines that divide the road in the same direction as the vehicle's traveling direction is above the predetermined level, the vehicle is stopped between the pair of left and right roadway dividing objects or roadway dividing lines.

[0008] Furthermore, a computer program according to one aspect of the present invention performs driving assistance to stop the vehicle when it is determined that the driver of the vehicle is in an abnormal state that makes it difficult for him to continue driving the vehicle, and when, during driving assistance, the recognition level of the lane markings in the vehicle's own lane in which the vehicle is traveling is below a predetermined level and the recognition level of a pair of left and right roadway markings or roadway markings that mark a roadway on the road whose traveling direction is the same as that of the vehicle is above the predetermined level, the computer is caused to execute processing to stop the vehicle between the pair of left and right roadway markings or roadway markings. [Effects of the Invention]

[0009] According to these aspects of the present invention, when the recognition level is low, including when the lane markings of the vehicle's own lane cannot be recognized, deceleration and stop control can be performed even if the vehicle ahead cannot be recognized, as long as the roadway markings or lane markings can be recognized. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a schematic configuration diagram of a vehicle according to an embodiment of the present invention; [Figure 2] FIG. 2 is a diagram showing an example of lane markings and roadway markings of a vehicle's driving lane; [Figure 3] 3 is a flowchart illustrating deceleration and stop control according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0011] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description, like components are designated by like reference numerals.

[0012] FIG. 1 is a schematic diagram of a vehicle 100 according to an embodiment of the present invention.

[0013] The vehicle 100 includes a surrounding sensor 1, a vehicle sensor 2, a driver sensor 3, an HMI (Human Machine Interface) 4, an actuator 5, and a control device 6. The surrounding sensor 1, the vehicle sensor 2, the driver sensor 3, the HMI 4, the actuator 5, and the control device 6 are communicatively connected to each other via an in-vehicle network 9 that complies with a standard such as a controller area network.

[0014] The surrounding sensor 1 is a sensor for generating surrounding data that represents the situation around the vehicle 100. The vehicle 100 according to this embodiment is equipped with one or more external cameras 11 as the surrounding sensor 1 for capturing images of the surroundings of the vehicle 100. The external cameras 11 capture images of the surroundings of the vehicle 100 at a predetermined frame rate (for example, 10 [Hz] to 40 [Hz]) and generate surrounding images that show the surroundings of the vehicle 100. Every time the external cameras 11 generate a surrounding image, they transmit the generated surrounding image to the control device 6 as surrounding data.

[0015] In addition to the external camera 11, a distance measurement sensor that measures the distance to targets and features present around the vehicle 100 may be provided as the surrounding sensor 1. Examples of distance measurement sensors include a LiDAR (Light Detection And Ranging) that emits radar light and measures distance based on the reflected light, and a millimeter wave radar sensor that emits radio waves and measures distance based on the reflected waves.

[0016] The vehicle sensors 2 are sensors for generating vehicle data that represent the state of the vehicle 100. The vehicle 100 according to this embodiment includes, as the vehicle sensors 2, a speed sensor 21 that generates speed data that indicates the traveling speed of the vehicle 100, and a positioning sensor 22 that generates current position data that indicates the current position of the vehicle 100, such as latitude and longitude. However, the vehicle sensors 2 are not limited to these sensors. The data acquired by the sensors 21 and 22 is transmitted to the control device 6 as vehicle data.

[0017] The driver sensor 3 is a sensor for generating driver data that represents the state of the driver. The vehicle 100 according to this embodiment is equipped with a driver monitor camera 31 as the driver sensor 3 for capturing an image of the driver's appearance, including the driver's face. The driver monitor camera 31 captures an image of the driver's appearance at a predetermined frame rate (for example, 10 [Hz] to 40 [Hz]) and generates an appearance image showing the driver's appearance. Every time the driver monitor camera 31 generates an appearance image of the driver, it transmits the generated appearance image to the control device 6 as driver data.

[0018] The HMI 4 is a user interface for exchanging information between the vehicle 100 and its occupants. The HMI 4 includes an output device 41 for notifying the vehicle occupants through their bodily senses (for example, vision, hearing, and touch), and an input device 42 for the vehicle occupants to perform input operations and response operations. The output device 41 is, for example, a display (for example, a meter display, a center display, a head-up display, etc.) or a speaker. The input device 42 is, for example, a touch panel or a microphone.

[0019] The HMI 4 notifies the vehicle occupant of information corresponding to the output signal received from the control device 6 via the output device 41, and transmits data input by the vehicle occupant to the control device 6 via the input device 42.

[0020] The HMI 4 may be pre-installed in the vehicle 100, or may be a terminal such as a smartphone owned by a vehicle occupant. In the latter case, for example, information may be exchanged between the vehicle 100 and the vehicle occupant's terminal by short-range wireless communication, or information may be exchanged indirectly via communication between the vehicle occupant's terminal and an external server (not shown).

[0021] The actuator 5 is a device used for driving control of the vehicle 100. The vehicle 100 according to this embodiment includes, as the actuators 5, an acceleration actuator 51 (for example, at least one of an engine and a motor) that controls acceleration of the vehicle 100, a brake actuator 52 (for example, a hydraulic actuator) that controls braking of the vehicle 100, and a steering actuator 53 (for example, a steering motor) that controls steering of the vehicle 100.

[0022] The control device 6 is an ECU (Electronic Control Unit) including a communication unit 61, a storage unit 62, and a processing unit 63.

[0023] The communication unit 61 includes an interface circuit for connecting the control device 6 to the in-vehicle network 9. The communication unit 61 supplies various data received from various sensors, the HMI 4, etc. to the processing unit 63. The communication unit 61 also outputs various signals output from the processing unit 63 to the HMI 4, the actuator 5, etc.

[0024] The storage unit 62 has a storage medium such as a hard disk drive (HDD), a solid disk drive (SSD), or a semiconductor memory, and stores various computer programs and data used for processing by the processing unit 63.

[0025] The processing unit 63 has one or more CPUs (Central Processing Units) and their peripheral circuits, and executes various computer programs stored in the storage unit 62. The processing unit 63 is, for example, a processor. The processing unit 63 may further include other arithmetic circuits such as a logic operation unit, a numerical operation unit, or a graphics processing unit. The processing unit 63 executes processing in accordance with the computer programs, thereby functioning as a recognition unit 71, a recognition level setting unit 72, and a driving assistance unit 73, and operates as functional units (modules) that realize predetermined functions. In the following description, when processing is described using each of the functional units 71 to 73 as the subject, it indicates that the processing unit 63 is executing a program that realizes each of the functional units 71 to 73.

[0026] The following describes the specific processing carried out by the control device 6. That is, the details of the functional units 71 to 73 realized by the processing unit 63 executing processing in accordance with a computer program will be described.

[0027] The recognition unit 71 recognizes targets and features around the vehicle 100. For example, the recognition unit 71 sequentially inputs surrounding images received from the external camera 11 into a classifier, thereby recognizing targets in the surrounding images, such as other vehicles, motorcycles, and pedestrians, as well as features, such as curbs, fences, and similar structures (hereinafter referred to as "demarcation features") and road markings (e.g., lane markings). The classifier may be, for example, a convolutional neural network (CNN) having multiple convolution layers connected in series from the input side to the output side. Furthermore, the recognition unit 71 calculates the distances from the vehicle to the targets and features and the positions of the targets and features using, for example, standard sizes of the targets and features stored in the memory unit 62 for each type of target and feature and the sizes of the targets and features recognized in the surrounding images. Note that the method for recognizing targets and features is not limited to the above method, and various known methods may be used.

[0028] The recognition level setting unit 72 sets a reliability level (hereinafter referred to as the "recognition level") for the recognition results of the division objects and lane markings on the road. In this embodiment, the recognition level setting unit 73 sets a recognition level for a pair of left and right section lines (hereinafter referred to as "dividing lines of the own vehicle driving lane") that define the driving lane in which the vehicle 100 is driving (hereinafter referred to as the "own vehicle driving lane"), and a recognition level for a pair of left and right division objects or dividing lines (hereinafter referred to as "roadway dividing objects or roadway markings") that divide the roadway whose traveling direction is the same as that of the vehicle 100. For example, when the vehicle 100 is driving on a road with three lanes in each direction as shown in FIG. 2, dividing lines L1 and R1 are dividing lines of the own vehicle driving lane, and dividing lines L2 and R2 are roadway markings.

[0029] In this embodiment, if the recognition level setting unit 72 is unable to recognize both the left and right lane markings of the host vehicle's lane, it sets the recognition level of the host vehicle's lane markings to the lowest level, level 0, if it is able to recognize either the left or right lane markings of the host vehicle, it sets the recognition level of the host vehicle's lane markings to level 1, and if it is able to recognize both the left and right lane markings of the host vehicle, it sets the recognition level of the host vehicle's lane markings to level 2. Furthermore, if the recognition level setting unit 72 is able to recognize both the left and right roadway markings or roadway markings, it sets the recognition level of the roadway markings or roadway markings to level 1, and otherwise sets it to level 0.

[0030] The reasons why a section marking cannot be recognized include, for example, blurred or broken section marks due to deterioration of the section marking, the influence of sunlight reflected from the road surface, etc. The method for setting the recognition level is not limited to the above method, and the recognition level can be set more finely.

[0031] For example, even if both the left and right lane markings of the vehicle 100 can be recognized, if the difference between the distance in the vehicle width direction from the center of gravity of the vehicle 100 to the recognized right lane marking and the distance in the vehicle width direction from the center of gravity of the vehicle 100 to the left lane marking is equal to or greater than a predetermined value, there is a possibility that one of the lane markings has been erroneously recognized. Therefore, in such cases, the recognition level of the lane markings of the vehicle's driving lane may be lowered, for example, by setting it to 1.5 in the above example. Also, even if one of the left and right lane markings of the vehicle can be recognized, if the distance in the vehicle width direction from the center of gravity of the vehicle 100 to the recognized lane marking is equal to or greater than a predetermined distance, there is a possibility that the lane marking has been erroneously recognized. Therefore, in such cases, the recognition level of the lane markings of the vehicle's driving lane may be lowered, for example, by setting it to 0.5 in the above example.

[0032] The driving assistance unit 73 controls the actuators 5 based on the targets and features recognized by the recognition unit 71, and performs driving assistance involving driving control of the vehicle 100. In this embodiment, the driving assistance unit 73 can perform driving assistance involving driving control of the vehicle 100 at a driving control level of Level 3 defined by the Society of Automotive Engineers (SAE), that is, a driving control level that does not require the driver to operate the actuators 51 to 53 or monitor the surroundings. Furthermore, the driving assistance unit 73 can perform driving assistance involving driving control of the vehicle 100 at a driving control level in which the driver is involved in driving the vehicle 100, for example, a driving control level of Level 1 or Level 2 defined by the SAE.

[0033] As one of the driving assistance functions involving driving control of the vehicle 100, the driving assistance unit 73 performs deceleration / stop control to decelerate the vehicle 100 and keep the vehicle 100 in a stopped state when the driver falls into an abnormal state where it is difficult for him to continue driving the vehicle 100, for example, due to a sudden change in his physical condition.

[0034] Here, when performing deceleration and stop control, in order to reduce the risks associated with performing deceleration and stop control (for example, the risk of contact with road users outside the vehicle), it is desirable to decelerate vehicle 100 and keep vehicle 100 in a stopped state so that vehicle 100 does not deviate from the roadway (a portion of the road separated by curbs, fences, or other similar structures or road markings for use by vehicles).

[0035] Therefore, in the past, when the lane markings of the vehicle's own lane could be recognized, the vehicle 100 was decelerated along the lane markings of the vehicle's own lane to keep the vehicle 100 stopped, and when the lane markings of the vehicle's own lane could not be recognized, the vehicle 100 was decelerated along the trajectory of the vehicle ahead to keep the vehicle 100 stopped. However, in this conventional system, when the lane markings of the vehicle's own lane could not be recognized and there was no vehicle ahead, the deceleration and stop control was stopped.

[0036] Therefore, in this embodiment, when the deceleration / stop control is being performed, if the recognition level of the lane markings of the vehicle's own lane is below a certain level and the recognition level of the roadway division objects or roadway division lines is equal to or higher than a certain level, the vehicle 100 is decelerated between the left and right roadway division objects or roadway division lines to keep the vehicle 100 in a stopped state. As a result, when the recognition level is low, including when the lane markings of the vehicle's own lane cannot be recognized, even if there is no vehicle ahead, as long as the roadway division objects or roadway division lines can be recognized, the vehicle 100 can be decelerated between the left and right roadway division objects or roadway division lines without terminating the deceleration / stop control to keep the vehicle 100 in a stopped state.

[0037] FIG. 3 is a flowchart illustrating the deceleration and stop control according to this embodiment.

[0038] In step S1, the control device 6 determines whether the driver has fallen into an abnormal state (hereinafter simply referred to as "abnormal state") that makes it difficult for the driver to continue driving. In this embodiment, the control device 6 determines that the driver has fallen into an abnormal state if a predetermined estimated abnormal state continues for a predetermined time T1. If the driver has fallen into an abnormal state, the control device 6 proceeds to processing in step S2. On the other hand, if the driver has not fallen into an abnormal state, the control device 6 ends this processing.

[0039] The estimated abnormal state is a state in which the driver is deemed to be in an abnormal state. For example, an example of the estimated abnormal state is a state in which the driver is not operating the steering wheel when driving assistance involving driving control of the vehicle 100 is being performed at a driving control level of level 2 or lower. Whether or not the driver is operating the steering wheel can be determined, for example, based on the magnitude of the steering torque acting on the steering shaft that rotates integrally with the steering wheel. Furthermore, regardless of the driving control level, it can also be determined whether or not the driver is in an estimated abnormal state based on the driver's appearance, for example, based on an image from a driver monitor camera.

[0040] In this embodiment, when the estimated abnormal state continues for a predetermined time T0 (T0 is a time shorter than T1), a warning is issued to the driver via the HMI. This allows deceleration and stop control to be performed only when the driver is in an abnormal state where he or she cannot respond to the warning. Alternatively, whether the driver is in an abnormal state may be determined by a separate control device, and the determination result may be acquired.

[0041] In step S2, the control device 6 determines whether the recognition level of the lane markings in the vehicle's lane is equal to or higher than a predetermined level (e.g., level 1). If the recognition level of the lane markings in the vehicle's lane is equal to or higher than the predetermined level, the control device 6 proceeds to step S3. On the other hand, if the recognition level of the lane markings in the vehicle's lane is lower than the predetermined level, the control device 6 proceeds to step S4.

[0042] In step S3, the control device 6 decelerates the vehicle 100 along the host vehicle driving lane and stops the vehicle 100 within the host vehicle driving lane.

[0043] In step S4, the control device 6 determines whether the recognition level of the roadway division objects or roadway dividing lines is equal to or higher than a predetermined level (e.g., level 1). If the recognition level of the roadway division objects or roadway dividing lines is equal to or higher than the predetermined level, the control device 6 proceeds to processing in step S5. On the other hand, if the recognition level of the roadway division objects or roadway dividing lines is lower than the predetermined level, the control device 6 proceeds to processing in step S6.

[0044] In step S5, the control device 6 decelerates the vehicle 100 between the left and right roadway dividing lines and stops the vehicle 100 within the roadway. When decelerating the vehicle 100 between the left and right roadway dividing lines, for example, the vehicle 100 may be decelerated while traveling straight as long as the vehicle 100 does not extend beyond the left and right roadway dividing lines, or the vehicle 100 may be decelerated along the left or right roadway dividing line. In this case, for example, by decelerating and stopping the vehicle 100 along the left roadway dividing line that is closer to the sidewalk, rescue of the driver after stopping the vehicle 100 can be easily and safely performed from the sidewalk. Conversely, by decelerating and stopping the vehicle 100 along the right roadway dividing line that is farther from the sidewalk, accidental contact with road users on the sidewalk or motorcycles passing on the left side of the roadway can be prevented.

[0045] In step S6, the control device 6 determines that the recognition level of the vehicle's driving lane and the recognition level of roadway divisions or roadway dividing lines are both below a predetermined level, and therefore that there is a relatively high possibility that the division lines, roadway divisions, and roadway dividing lines are not recognized or are being misrecognized, and therefore simply causes the vehicle 100 to travel straight while slowing down and stopping the vehicle 100.

[0046] The control device 6 (driving assistance control device) of the vehicle 100 according to the present embodiment described above is configured to perform driving assistance to stop the vehicle 100 when it is determined that the driver of the vehicle 100 is in an abnormal state making it difficult for him to continue driving the vehicle 100, and to stop the vehicle 100 between the pair of left and right roadway dividing objects or roadway dividing lines that divide the roadway in the same direction as the vehicle 100 during the driving assistance if the recognition level of the dividing lines of the vehicle's own driving lane in which the vehicle 100 is traveling is below a predetermined level and the recognition level of the pair of left and right roadway dividing objects or roadway dividing lines that divide the roadway in the same direction as the vehicle 100 is above a predetermined level.

[0047] As a result, when the recognition level is low, including when the section lines of the vehicle's own driving lane cannot be recognized, even if the vehicle ahead cannot be recognized, as long as the roadway dividing object or roadway dividing line can be recognized, the vehicle 100 can be decelerated between the left and right roadway dividing objects or roadway dividing lines without terminating the deceleration / stop control, and the vehicle 100 can be kept in a stopped state.

[0048] Furthermore, when stopping the vehicle 100 between a pair of left and right roadway division objects or roadway division lines, by configuring the control device 6 to stop the vehicle 100 along the roadway division object or roadway division line closer to the sidewalk, the driver can be easily and safely rescued from the sidewalk side after the vehicle 100 has stopped.

[0049] Conversely, when stopping the vehicle 100 between a pair of left and right roadway divisions or roadway division lines, the control device 6 can be configured to stop the vehicle 100 along the roadway division or roadway division line farther from the sidewalk, thereby preventing accidental contact with road users on the sidewalk or motorcycles passing on the left side of the roadway.

[0050] In addition, the control device 6 according to this embodiment is configured to recognize the demarcation objects and demarcation lines on the road based on the images captured by the external camera 11 (imaging device), and to set the recognition level of the demarcation lines of the vehicle's driving lane and the recognition level of the roadway demarcation objects or the roadway demarcation lines based on the recognition results of the demarcation objects and demarcation lines.

[0051] Although the embodiments of the present invention have been described above, the above embodiments merely illustrate some of the application examples of the present invention, and it is not intended that the technical scope of the present invention be limited to the specific configurations of the above embodiments.

[0052] For example, in the above embodiment, the computer program executed by the control device 6 may be provided in a form recorded on a computer-readable portable recording medium such as a semiconductor memory, a magnetic recording medium, or an optical recording medium. Also, in the above embodiment, the control device 6 determines whether the driver is in an abnormal state, but the determination may be made by another control device, and the control device 6 may acquire the determination result. [Explanation of symbols]

[0053] 6. Control device 11 External camera (imaging device) 100 vehicles

Claims

1. A driving assistance control device for a vehicle, When it is determined that the driver of the vehicle is in an abnormal state in which it is difficult for him or her to continue driving the vehicle, driving assistance is performed to stop the vehicle, During the driving assistance, when the recognition level of the lane markings of the vehicle's own lane in which the vehicle is traveling is below a predetermined level, and the recognition level of a pair of left and right roadway markings or roadway markings that mark the roadway in the same traveling direction as the vehicle on the road is equal to or higher than the predetermined level, the system is configured to stop the vehicle between the pair of left and right roadway markings or roadway markings. Driver assistance control device.

2. When the vehicle is stopped between a pair of left and right roadway partitions or roadway dividing lines, the vehicle is stopped along the roadway partition or roadway dividing line closer to the sidewalk. The driving assistance control device according to claim 1 .

3. When the vehicle is stopped between a pair of left and right roadway partitions or roadway lines, the vehicle is stopped along the roadway partition or roadway line farther from the sidewalk. The driving assistance control device according to claim 1 .

4. Recognizing road markings and demarcation lines on the basis of the image captured by the imaging device; and setting a recognition level of the lane markings of the vehicle and a recognition level of the roadway markings or the roadway markings based on the recognition results of the lane markings and the lane markings. The driving assistance control device according to any one of claims 1 to 3.

5. A vehicle driving assistance method implemented by a control device, comprising: When it is determined that the driver of the vehicle is in an abnormal state in which it is difficult for him or her to continue driving the vehicle, driving assistance is performed to stop the vehicle, During the driving assistance, when the recognition level of the lane markings of the vehicle's own lane in which the vehicle is traveling is below a predetermined level, and the recognition level of a pair of left and right roadway markings or roadway markings that mark the roadway in the same traveling direction as the vehicle on the road is equal to or higher than the predetermined level, the vehicle is stopped between the pair of left and right roadway markings or roadway markings. Driving assistance methods.

6. When it is determined that the driver of the vehicle is in an abnormal state where it is difficult for the driver to continue driving the vehicle, driving assistance is performed to stop the vehicle; During the driving assistance, when the recognition level of the lane markings of the vehicle's own lane in which the vehicle is traveling is below a predetermined level, and the recognition level of a pair of left and right roadway markings or roadway markings that mark the roadway in the same traveling direction as the vehicle on the road is equal to or higher than the predetermined level, the vehicle is stopped between the pair of left and right roadway markings or roadway markings. A computer program that causes a computer to perform a process.

Images