Mobile device control device, mobile device control method, and program

Abstract

To implement more appropriate deviation suppression control. [Solution] The mobile body control device of the embodiment includes a recognition unit that recognizes the surrounding conditions including the position of the mobile body with respect to the travel path; a deviation determination unit that determines whether the mobile body deviates from the travel path based on the position of the mobile body and a first condition; a control unit that performs deviation suppression control including at least one of notification control and behavior control in a direction that suppresses deviation when it is determined that the mobile body deviates from the travel path; and a driving state detection unit that detects the state of the driver of the mobile body. If it is determined that the driver's state is not suitable for driving the mobile body, a deviation determination is made based on a second condition which is more likely to determine that the mobile body deviates from the travel path than the first condition. If deviation suppression control is performed based on the result of the deviation determination by the second condition, the next deviation determination is made based on the first condition, or a third condition which is more likely to determine that the mobile body deviates than the first condition and less likely to determine that the mobile body deviates than the second condition.

Description

Technical Field

[0001] The present invention relates to a movement control device, a movement control method, and a program.

Background Art

[0002] In recent years, efforts have been actively made to provide access to a sustainable transportation system that takes into account people in vulnerable positions among traffic participants. Toward this realization, research and development have focused on further improving traffic safety and convenience through research and development related to preventive safety technology. In this context, conventionally, in a lane departure warning device that energizes an in-vehicle alarm when the operation of the in-vehicle blinker has not been performed when the distance between the lane and the vehicle becomes less than or equal to a threshold value, the time during which the alarm is energized is held until the energized state is released, and when the current energization time is longer than the previous energization time when comparing the current energization time with the previous energization time, the output level to the alarm is increased, and in the opposite case, the output level is decreased. This technology is known (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, in preventive safety technology, after departure suppression control including an alarm has been executed once, there are cases where the output level of the alarm increases according to the energization time of the alarm even though the driver can understand the situation. Therefore, there is a problem that the driver may feel bothered by departure suppression control such as an alarm.

[0005] One of the objectives of this application is to provide a mobile vehicle control device, a mobile vehicle control method, and a program that can perform more appropriate deviation suppression control in order to solve the above-mentioned problems. Ultimately, this will contribute to the development of sustainable transportation systems. [Means for solving the problem]

[0006] The mobile device control device, mobile device control method, and program according to this invention employ the following configuration. (1): A mobile body control device according to one aspect of the present invention comprises: a recognition unit that recognizes the surrounding conditions including the position of a mobile body with respect to a travel path; a deviation determination unit that determines whether or not the mobile body deviates from the travel path based on the position of the mobile body and a first condition; a control unit that performs deviation suppression control including at least one of notification control and behavior control in a direction that suppresses deviation when the deviation determination unit determines that the mobile body deviates from the travel path; and a driving state detection unit that detects the state of the driver of the mobile body, wherein the deviation determination unit makes the deviation determination based on a second condition which makes it easier to determine that the mobile body deviates from the travel path than the first condition when it determines that the state of the driver is not suitable for driving the mobile body, and when the deviation determination unit performs the deviation determination based on the result of the deviation determination based on the second condition, it makes the next deviation determination based on the first condition, or based on a third condition which makes it easier to determine that the mobile body deviates than the first condition and less likely to determine that the mobile body deviates than the second condition.

[0007] (2) In the embodiment of (1) above, the deviation determination unit performs the deviation determination using the first condition or the third condition until a predetermined time has elapsed since the deviation suppression control was performed based on the result of the deviation determination by the second condition, or until the moving body has moved a predetermined distance or more.

[0008] (3) In the embodiment of (1) above, the deviation determination unit performs the deviation determination using the first condition or the third condition even if the driver remains in an unsuitable state for driving after the deviation suppression control has been performed based on the result of the deviation determination by the second condition.

[0009] (4) In the embodiment of (2) above, the deviation determination unit makes the deviation determination using the second condition or a fourth condition which is more likely to be determined to be a deviation than the second condition, if the driver remains in an unsuitable state for driving after the predetermined time has elapsed or the moving body has moved a predetermined distance or more.

[0010] (5) In the embodiment of (2) above, the deviation determination unit makes the deviation determination using the first condition after the predetermined time has elapsed or after the moving body has moved a predetermined distance or more, if the driver's condition is suitable for driving.

[0011] (6) In the embodiment of (4) above, of the first, second, third, and fourth conditions, at least the first condition can be set by the occupant of the mobile body.

[0012] (7): A mobile body control method according to another aspect of the present invention is a mobile body control method in which a computer recognizes the surrounding conditions including the position of the mobile body with respect to the travel path, makes a deviation determination of whether the mobile body deviates from the travel path based on the position of the mobile body and a first condition, performs deviation suppression control including at least one of notification control and behavior control in a direction that suppresses deviation if it is determined that the mobile body deviates from the travel path, detects the state of the driver of the mobile body, and if it is determined that the state of the driver is not suitable for driving the mobile body, makes the deviation determination based on a second condition which is more likely to determine that the mobile body deviates from the travel path than the first condition, and when the deviation suppression control is performed based on the result of the deviation determination based on the second condition, makes the next deviation determination based on the first condition or based on a third condition which is more likely to determine that the mobile body deviates than the first condition and less likely to determine that the mobile body deviates than the second condition.

[0013] (8) A program according to another aspect of the present invention causes a computer to recognize the surrounding conditions, including the position of a moving body with respect to a travel path; to make a deviation determination based on the position of the moving body and a first condition, to determine whether the moving body deviates from the travel path; if it is determined that the moving body deviates from the travel path, to perform deviation suppression control, which includes at least one of notification control and behavioral control for the moving body in a direction that suppresses deviation; to detect the state of the driver of the moving body; if it is determined that the state of the driver is not suitable for driving the moving body, to make the deviation determination based on a second condition which makes it more likely than the first condition that the moving body will deviate from the travel path; and if the deviation suppression control is performed based on the result of the deviation determination based on the second condition, to make the next deviation determination based on the first condition, or based on a third condition which makes it more likely than the first condition that the body will deviate and less likely than the second condition that the body will deviate. [Effects of the Invention]

[0014] According to the embodiments described in (1) to (8) above, more appropriate deviation suppression control can be performed. [Brief explanation of the drawing]

[0015] [Figure 1] This is a configuration diagram of a vehicle system 1 including a mobile control device according to an embodiment. [Figure 2] This diagram explains the determination and degree of distractedness. [Figure 3] This is a diagram illustrating the determination of deviations from the embodiments. [Figure 4] This flowchart shows an example of a process performed by the driver assistance device 100 in the embodiment. [Figure 5] This is a flowchart showing the first example of the process for setting deviation detection conditions. [Figure 6] This flowchart shows a second example of the process for setting deviation detection conditions. [Modes for carrying out the invention]

[0016] The embodiments of the mobile device control device, mobile device control method, and program of the present invention will be described below with reference to the drawings. In the following, a vehicle will be used as an example of a mobile device. In addition to vehicles, mobile devices may include, for example, ships that can move on the ground (roads) such as hovercraft, flying vehicles that can travel on roads, and stand-up vehicles with power units.

[0017] [Overall structure] Figure 1 is a diagram showing the configuration of a vehicle system 1 including a mobile control device according to an embodiment. The vehicle on which the vehicle system 1 is mounted (hereinafter referred to as vehicle M) is, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle or a micromobility, and its drive source is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. The electric motor operates using power generated by a generator connected to the internal combustion engine, or power discharged from a battery (storage battery) such as a secondary battery or a fuel cell.

[0018] The vehicle system 1 includes, for example, a camera 10, a radar device 12, a LIDAR (Light Detection and Ranging) 14, a communication device 20, a HMI (Human Machine Interface) 30, a vehicle sensor 40, a navigation device 50, an in-vehicle camera 60, a driving operator 80, a driving support device 100, a traveling driving force output device 200, a brake device 210, and a steering device 220. These devices and equipment are connected to each other by a multiplex communication line such as a CAN (Controller Area Network) communication line, a serial communication line, a wireless communication network, etc. Note that the configuration shown in FIG. 1 is merely an example, and a part of the configuration may be omitted, or another configuration may be added. A combination of the camera 10, the radar device 12, and the LIDAR 14 is an example of the "detection device DD". The HMI 30 is an example of the "notification unit". The driving support device 100 is an example of the "movement control device".

[0019] The camera 10 is, for example, a digital camera using a solid-state imaging device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 10 is attached to an arbitrary location of the vehicle M on which the vehicle system 1 is mounted. When imaging the front, the camera 10 is attached to the upper part of the front windshield, the back surface of the rearview mirror, the front head of the vehicle body, etc. When imaging the rear, the camera 10 is attached to the upper part of the rear windshield, the back door, etc. When imaging the side, the camera 10 is attached to the door mirror, etc. The camera 10, for example, periodically repeats imaging the periphery of the vehicle M. The camera 10 may be a stereo camera.

[0020] The radar device 12 emits radio waves (radar) such as millimeter waves around the vehicle M, and detects radio waves (reflected waves) reflected by surrounding objects to detect at least the position (distance and azimuth) of the objects. The radar device 12 is attached to an arbitrary location of the vehicle M. The radar device 12 may detect the position and speed of an object by an FM-CW (Frequency Modulated Continuous Wave) method.

[0021] The LIDAR 14 irradiates light around the vehicle M and measures scattered light. The LIDAR 14 detects the distance to the target based on the time from light emission to light reception. The irradiated light is, for example, pulsed laser light. The LIDAR 14 is attached to an arbitrary location of the vehicle M.

[0022] The communication device 20 communicates with, for example, other vehicles existing around the vehicle M, the terminal device of the user using the vehicle M, or various server devices by using networks such as a cellular network, a Wi-Fi network, Bluetooth (registered trademark), DSRC (Dedicated Short Range Communication), a LAN (Local Area Network), a WAN (Wide Area Network), and the Internet.

[0023] The HMI 30 outputs various information to the passengers (including the driver) of the vehicle M and receives input operations by the passengers. The HMI 30 includes, for example, a display unit 32, a speaker 34, and a vibration unit 36. The display unit 32 is, for example, an LCD (Liquid Crystal Display), an organic EL (Electro Luminescence) display device, or the like. The display unit 32 displays various images (including videos) in the embodiment. The display unit 32 may be integrally configured with the input unit as a touch panel. The speaker 34 outputs a predetermined sound (for example, an alarm sound).

[0024] The vibration unit 36 ​​provides vibrational stimulation to the driver, for example, based on instructions from the driver assistance device 100. There are multiple vibration units 36, and they are installed, for example, on the seat, the driver controls 80 (e.g., the steering wheel 82), the seat belt in use, etc. The vibration unit 36 ​​may use, for example, an LRA (Linear Resonant Actuator), an eccentric motor, a linear motor, etc. In addition to (or instead of) the display unit 32, speaker 34, and vibration unit 36, the HMI 30 may also include a microphone, buzzer, touch panel, switch, key, etc.

[0025] The vehicle sensor 40 includes a vehicle speed sensor for detecting the speed of the vehicle M, an acceleration sensor for detecting acceleration, and a yaw rate sensor for detecting yaw rate (for example, the rotational angular velocity around the vertical axis passing through the center of gravity of the vehicle M). The vehicle sensor 40 may also include a lateral acceleration sensor (lateral G sensor) for detecting the lateral velocity (lateral G) of the vehicle M, a steering angle sensor for detecting the steering angle of the vehicle M (which may be the angle of the steering wheels or the operating angle of the steering wheel 82) and steering torque, a steering angular velocity sensor for detecting the steering angular velocity, and a compass sensor for detecting the orientation of the vehicle M.

[0026] The vehicle sensor 40 may also include a position sensor that detects the position of the vehicle M. The position sensor is, for example, a sensor that acquires position information (longitude and latitude information) from a GPS (Global Positioning System) device. Alternatively, the position sensor may be a sensor that acquires position information using, for example, a GNSS (Global Navigation Satellite System) receiver of a navigation device 50. The vehicle sensor 40 may derive the speed of the vehicle M from the difference (i.e., distance) of position information at a predetermined time from the position sensor. The results detected by the vehicle sensor 40 are output to the driver assistance device 100.

[0027] The navigation device 50 includes, for example, a GNSS receiver, a navigation HMI, and a route determination unit. The navigation device 50 may store map information in a storage device such as an HDD (Hard Disk Drive) or flash memory, or it may acquire map information 162 stored in a storage unit 160, which will be described later. The GNSS receiver determines the position of the vehicle M based on signals received from GNSS satellites. The position of the vehicle M may be determined or supplemented by an INS (Inertial Navigation System) that utilizes the output of the vehicle sensor 40. The navigation HMI includes a display device, speaker, touch panel, keys, etc. The GNSS receiver may be provided on the vehicle sensor 40. The navigation HMI may be partially or completely shared with the HMI 30 described above. The route determination unit determines, for example, a route (hereinafter referred to as a route on a map) from the position of the vehicle M determined by the GNSS receiver (or any input position) to a destination input by the occupant using the navigation HMI, by referring to, for example, map information 162, etc. Furthermore, the navigation device 50 provides route guidance using the navigation HMI based on the determined route on the map. The navigation device 50 may also transmit its current location and destination to the navigation server via the communication device 20 and obtain a route equivalent to the route on the map from the navigation server.

[0028] Here, map information 162 is information in which the road shape is represented by links indicating roads (an example of a travel route) and nodes connected by those links. Map information 162 may also include POI (Point of Interest) information, etc. Map information 162 also includes, for example, the number of lanes (number of travel routes), the type and shape of road markings, information on the center of the lanes, or information on road boundaries. Map information 162 may also include information on whether the road boundary is a boundary (physical boundary) that includes structures that vehicles cannot pass through (including crossing and contact). Physical boundaries include, for example, guardrails, curbs, median strips, fences, etc. Map information 162 may also include road shape information, traffic regulation information, address information (address and postal code), facility information, parking information, telephone number information, etc. Road shape information includes, for example, information on the curvature of the road (which may be rephrased as radius of curvature; the same applies below), width, road surface gradient, branching and merging points, intersections, T-junctions, etc. Map information 162 may be updated as needed by the communication device 20 communicating with an external device.

[0029] The in-vehicle camera 60 is a digital camera that uses a solid-state image sensor such as a CCD or CMOS. The in-vehicle camera 60 is mounted at any location in the vehicle M in a position and orientation that allows it to capture the head of the occupant (driver) I seated in the driver's seat of the vehicle M from the front (in a direction that captures the face). For example, the in-vehicle camera 60 is mounted above the display device located in the center of the instrument panel of the vehicle M. The in-vehicle camera 60 also captures images of the interior of the vehicle, including the area where the occupant (passenger) seated in the front passenger seat of the vehicle M is located.

[0030] The driver controls 80 include, for example, a steering wheel 82, an accelerator pedal, a brake pedal, a shift lever, and other controls. The driver controls 80 are equipped with sensors (detection units) that detect the amount of operation or whether or not an operation is being performed.

[0031] The steering wheel 82 is an example of an "operator that accepts steering input from the driver." The steering wheel 82 does not necessarily have to be ring-shaped and may take the form of an irregularly shaped steering wheel, a joystick, buttons, etc. A grip sensor (not shown) is attached to the steering wheel 82. The grip sensor is implemented by a capacitive sensor or the like and detects whether the occupant (driver) is gripping the steering wheel 82 (meaning making contact with it in a state where force can be applied) based on capacitance. The steering wheel 82 may also be provided with a vibration unit 36. The steering wheel 82 may also be provided with a reaction force device that adjusts the amount of steering input from the driver's manual operation. The reaction force device, for example, applies a reaction force to the steering wheel 82 in response to steering input from the steering wheel 82 when it is determined that the vehicle M may deviate from its lane, so that the vehicle M travels in the center of the lane.

[0032] Furthermore, sensors on the accelerator and brake pedals detect the amount of pedal depression, and a detection unit on the steering wheel detects the steering angle and steering torque of the steering wheel 82. Each sensor (including the grip sensor) on the driver control unit 80 outputs a detection signal indicating the detection result to the driver assistance device 100, or to one or both of the driving force output device 200, the brake device 210, and the steering device 220.

[0033] The driver assistance device 100 performs various controls to assist the driver of the vehicle M. The driver assistance device 100 includes, for example, a recognition unit 110, a driving state detection unit 120, a deviation determination unit 130, a control unit 140, and a storage unit 160. The recognition unit 110, the driving state detection unit 120, the deviation determination unit 130, and the control unit 140 are each realized by a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Furthermore, some or all of these components may be realized by hardware (including circuitry) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit), and SOC (System On Chip), or by the cooperation of software and hardware. The above-mentioned program may be stored in advance in a storage device (a storage device equipped with a non-transient storage medium) such as the HDD or flash memory of the driver assistance device 100, or it may be stored in a removable storage medium such as a DVD, CD-ROM, or memory card, and installed in the storage device of the driver assistance device 100 when the storage medium (non-transient storage medium) is inserted into a drive device or card slot. The HMI control unit 142 is an example of a "notification control unit".

[0034] The storage unit 160 may be implemented using the various storage devices described above, or an EEPROM (Electrically Erasable Programmable Read Only Memory), ROM (Read Only Memory), or RAM (Random Access Memory), etc. The storage unit 160 stores, for example, map information 162, judgment condition data 164, various information in the embodiment, programs, etc. The storage unit 160 may also store the judgment condition data used in the processing in this embodiment.

[0035] The determination condition data 164 includes, for example, determination conditions for determining whether or not to perform deviation suppression control by the deviation determination unit 130. The determination condition data 164 may also include values ​​related to determination conditions for determining whether or not the driver is in a state appropriate for driving the vehicle M by the driving state detection unit 120.

[0036] The recognition unit 110 recognizes the surrounding conditions of the vehicle M based on, for example, the detection results of the detection device DD (information input from the camera 10, radar device 12, and LIDAR 14). For example, the recognition unit 110 performs sensor fusion processing on some or all of the detection results from the camera 10, radar device 12, and LIDAR 14 to recognize the position (relative position), size, speed (relative speed), acceleration, and other states of objects present around the vehicle M (within a predetermined distance). Objects recognized by the recognition unit 110 may include, for example, physical boundaries that demarcate roads (travel paths), as well as other vehicles, pedestrians, bicycles, and other traffic participants (examples of obstacles). The position of an object is recognized as a position on an absolute coordinate system with a representative point of the vehicle M (such as the center of gravity or the center of the drive axis) as the origin, and is used for control. The position of an object may be represented by a representative point such as the center of gravity or a corner of the object, or by a represented region. The "state" of an object may include, for example, the acceleration or jerk of another moving object, such as another vehicle, or the "action state" (for example, whether the other vehicle is changing lanes or is about to change lanes).

[0037] Furthermore, the recognition unit 110 recognizes, for example, the lane in which the vehicle M is traveling (driving lane) and other surrounding lanes (for example, oncoming lanes). For example, the recognition unit 110 recognizes road markings from images captured by the camera 10 and recognizes the driving lane and other lanes based on the positional relationship of the road markings as seen from the recognized vehicle M. Alternatively, the recognition unit 110 may refer to map information 162 based on the position information of the vehicle M obtained from the vehicle sensor 40, etc., to recognize the lane in which the vehicle M is traveling and other lanes. The recognition unit 110 may also recognize, for example, stop lines, red lights, toll booths, other road events, road signs, and markings drawn on the road (for example, speed limits).

[0038] Furthermore, the recognition unit 110 recognizes the behavior of vehicle M based on the detection results of the vehicle sensor 40. For example, the recognition unit 110 recognizes the lateral position of vehicle M relative to the driving lane (position in the lane width direction) and the attitude (orientation) of vehicle M relative to the direction of extension of the driving lane, based on the positional relationship of vehicle M with respect to the driving lane. For example, the recognition unit 110 may recognize the deviation of the vehicle M's reference point from the center of the lane, and the angle it makes with a line connecting the centers of the lanes in the direction of vehicle M's travel, as the relative position and attitude of vehicle M with respect to the driving lane. Alternatively, the recognition unit 110 may recognize the position of the vehicle M's reference point relative to any side edge of the driving lane (road marking or road boundary), etc., as the relative position (lateral position) of vehicle M with respect to the driving lane. Furthermore, the recognition unit 110 may recognize the lateral behavior of vehicle M (for example, whether or not it has moved laterally by a predetermined distance or more) from the amount of change in the lateral position and orientation of vehicle M as described above. Furthermore, the recognition unit 110 may recognize the behavior of the vehicle M based on at least one of the following values ​​obtained from the vehicle sensor 40, such as the steering angle, steering torque, steering angular velocity (derivative value of steering angle), and yaw rate, or the steering torque and steering torque change rate obtained from the driver control unit 80.

[0039] The driving state detection unit 120 detects the state of the driver operating the vehicle M. The driver's state refers to, for example, whether the driver's state is suitable for operating (driving) the vehicle M. Whether the driver is suitable for operating the vehicle M may be determined based on, for example, the behavior of the vehicle M due to the driver's driving (degree of stability of behavior, etc.), the state of the driver's distraction (degree of distraction, etc.), the driver's level of alertness (or level of drowsiness), or a combination of several of these. Details of the functions of the driving state detection unit 120 will be described later.

[0040] The deviation determination unit 130 determines whether or not vehicle M deviates from the driving lane (path of travel). "Whether or not to deviate" means whether or not vehicle M has the potential to deviate in the future, but it may also include whether or not it has already deviated from the driving lane. For example, if the behavior of vehicle M traveling on the driving lane satisfies the deviation determination conditions, the deviation determination unit 130 determines that vehicle M has deviated from the driving lane, and if the deviation determination conditions are not met, it determines that vehicle M has not deviated from the driving lane. The deviation determination conditions are information stored in the determination condition data 164 stored in the storage unit 160, for example, but these conditions may be adjusted by the detection results of the driving state detection unit 120, the behavior of vehicle M recognized by the recognition unit 110, or settings made by the driver, etc. Details of the functions of the deviation determination unit 130 will be described later.

[0041] The control unit 140 controls all the components included in the driver assistance device 100 and the vehicle system 1. For example, when the departure determination unit 130 determines that the vehicle M is deviating from its lane, the control unit 140 performs departure suppression control. Departure suppression control includes, for example, at least one of the following: notification control for the driver or other occupants, and behavioral control of the vehicle M in a direction that suppresses departure. Notification control may be, for example, an image display to draw the attention of the occupants, an output of a warning (sound), or a vibration output. Behavioral control may be, for example, reaction force control on the steering wheel 82, steering control of the vehicle M, or both. Behavioral control may also include speed control of the vehicle M.

[0042] For example, lane departure prevention control includes, for instance, LDW (Lane Departure Warning) and RDM (Road Departure Mitigation). LDW is a control that notifies the occupants that the vehicle M may deviate from its lane. The notification may be information to draw attention to the vehicle, or it may be driving instructions to prevent the deviation (e.g., steering direction). The notification may also be a warning image, a warning sound, a warning vibration, etc. RDM, for example, may perform control to notify the occupants when the vehicle M deviates from its lane, or it may perform reaction force control that applies a reaction force to the driver's steering wheel 82 operation in the direction of lane departure, or it may perform steering control to move the vehicle M towards the center of the lane.

[0043] The control unit 140 includes, for example, an HMI control unit 142 and a driving control unit 144. The HMI control unit 142 is an example of a "notification control unit". The HMI control unit 142 notifies the occupants of the vehicle M (including the driver) of predetermined information via the HMI 30, and receives information input by the HMI 30. The predetermined information includes, for example, information related to the driving of the vehicle M, such as information about the status of the vehicle M and information about driving control. Information about the status of the vehicle M includes, for example, the speed of the vehicle M, engine speed, and shift position. Information about driving control includes, for example, whether or not the driving control unit 144 is executing driving control and information about the execution status of driving control. The predetermined information may also include information about the surrounding conditions recognized by the detection device DD. The predetermined information may also include information unrelated to the driving of the vehicle M, such as content stored on a storage medium such as a television program or DVD (for example, a movie). The predetermined information may also include, for example, the current location and destination of the vehicle M, and information about the remaining fuel level of the vehicle M.

[0044] Furthermore, the HMI control unit 142 may output to the HMI 30 information such as inquiry information for the occupants, recognition results from the recognition unit 110, and determination results from the deviation determination unit 130. The HMI control unit 142 may also output information received by the HMI 30 to the communication device 20, the recognition unit 110, the deviation determination unit 130, the navigation device 50, etc. The HMI control unit 142 may also transmit various information to be output to the HMI 30 to terminal devices used by the occupants of the vehicle M via the communication device 20.

[0045] For example, if the departure detection unit 130 determines that the vehicle M is deviating from its lane, the HMI control unit 142 outputs a notification message to the HMI 30 via the notification control. Specifically, the HMI control unit 142 displays a warning image from the display unit 32, outputs an alarm from the speaker 34, and vibrates the steering wheel 82, the driver's seat, seat belt, etc., using the vibration unit 36. The HMI control unit 142 performs at least one of the following as notification control: image display, sound output, and vibration output.

[0046] The driving control unit 144 performs driving control that controls at least one of the steering or speed of the vehicle M based on the recognition results from the recognition unit 110 and the determination results from the deviation determination unit 130. For example, if the deviation determination unit 130 determines that the vehicle M is deviating, the driving control unit 144 controls the steering wheel 82 to apply a reaction force in the direction that prevents deviation, or performs steering control to move the vehicle M to the center of the driving lane.

[0047] Furthermore, the driving control unit 144 may, for example, control the vehicle M to stop in a safe location such as the shoulder of the road if the driver's condition detected by the driving state detection unit 120 is unsuitable for operating the vehicle M for a predetermined period of time or longer. The driving control unit 144 may also perform the above-described driving control to avoid contact between the vehicle M and an obstacle recognized by the recognition unit 110. In addition, the driving control unit 144 may control at least one of the steering or speed of the vehicle M in response to occupant instructions input from the HMI 30 to perform driving control such as ACC (Adaptive Cruise Control System) or ALC (Auto Lane Change).

[0048] The driving force output device 200 outputs driving force (torque) to the drive wheels for the vehicle to move. The driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, and a transmission, and an ECU (Electronic Control Unit) that controls them. The ECU controls the above configuration according to information input from the driving control unit 144 or information input from the accelerator pedal of the driver control unit 80.

[0049] The brake system 210 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to information input from the driving control unit 144 or from the brake pedal of the driver control unit 80, so that brake torque corresponding to the braking operation is output to each wheel. The brake system 210 may be equipped with a backup mechanism that transmits hydraulic pressure generated by the operation of the brake pedal to the cylinder via a master cylinder. The brake system 210 is not limited to the configuration described above, and may also be an electronically controlled hydraulic brake system that controls an actuator according to information input from the driving control unit 144 to transmit hydraulic pressure from the master cylinder to the cylinder.

[0050] The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor, for example, applies force to a rack and pinion mechanism to change the direction of the steering wheels. The steering ECU drives the electric motor to change the direction of the steering wheels according to information input from the driving control unit 144 or from the steering wheel of the driver control unit 80.

[0051] [Operating status detection unit] Next, the functions of the driving state detection unit 120 will be described in detail. For example, when detecting whether the driver's state is suitable for operating the vehicle M, the driving state detection unit 120 detects the degree of stability of the vehicle M's behavior based on the amount of change in the deviation between the position of the vehicle M traveling in the driving lane and the center of the lane, as recognized by the recognition unit 110, over a predetermined time. In this case, for example, the smaller the amount of change, the greater the degree of stability of the behavior. Alternatively, the driving state detection unit 120 may detect the degree of stability from the amount of change in the steering angle, etc., over a predetermined time, obtained from the vehicle sensor 40. In this case, the smaller the amount of change in the steering angle, the greater the degree of stability of the behavior. Note that when driving on a curved road or turning right or left at an intersection, the steering angle changes, so the driving state detection unit 120 may refer to the map information 162 based on the position information of the vehicle M and may not detect the degree of stability of the vehicle M's behavior when the vehicle M is traveling on a curved road or at an intersection. The driving state detection unit 120 determines that the driver's state is suitable for operating vehicle M if the stability level is equal to or greater than a first threshold, and determines that the driver's state is unsuitable for operating vehicle M if it is less than the first threshold. The first threshold is stored, for example, in the determination condition data 164.

[0052] Furthermore, the driving state detection unit 120 may detect a state of driver distraction based on images obtained from the in-vehicle camera 60. For example, the driving state detection unit 120 performs known image analysis processing on images captured by the in-vehicle camera 60 and recognizes the driver's gaze (the direction the driver is looking) or the orientation of the driver's face based on the analysis results. For example, the driving state detection unit 120 recognizes the driver's gaze and face orientation using methods such as template matching. The driving state detection unit 120 also recognizes the orientation of the driver's face based on positional information of the eyes, nose, mouth, etc. within the face region obtained from the image analysis results (relative positional information of each part, etc.). In the recognition of each piece of information using the images described above, for example, a pre-trained model learned in advance by machine learning may be used.

[0053] Furthermore, the driving state detection unit 120 compares the driver's gaze or face direction with the distraction detection area to determine whether the driver is distracted and to detect the degree of distraction. Figure 2 is a diagram illustrating distraction detection and the degree of distraction. In the example in Figure 2, the relationship between the driver D's gaze and the distraction detection conditions is shown in a simplified manner. For example, as shown in Figure 2, a distraction detection area AR1 (an example of a distraction detection condition) is set that has predetermined angles to the left and right from the position of the driver D's head, with respect to the direction of travel V of the vehicle M. The distraction detection area AR1 may be adjusted according to the speed of the vehicle M, the shape of the road while driving, etc.

[0054] In this situation, the driving state detection unit 120 determines that driver D is not distracted if driver D's line of sight A1 is within the distraction determination area AR1 (angle θ1), and determines that driver D is distracted if the line of sight is not within the distraction determination area AR1. The driving state detection unit 120 may also detect the degree of distraction based on the degree of deviation between the distraction determination area AR1 and line of sight A1 (for example, deviation angle △θ) or the elapsed time when line of sight A1 is not within the distraction determination area AR1. In this case, the greater the deviation angle △θ or the elapsed time, the greater the degree of distraction is set. The driving state detection unit 120 may also compare the direction of driver D's head with the distraction determination area AR1 instead of driver D's line of sight. For example, if the driving state detection unit 120 could not recognize driver D's line of sight but could recognize the direction of the face, it would use the direction of the face to perform the distraction determination described above. The driving state detection unit 120 determines that the driver's condition is suitable for operating vehicle M if the degree of distraction is equal to or greater than the second threshold, and determines that the driver's condition is unsuitable for operating vehicle M if it is less than the second threshold. The distraction determination conditions and the second threshold are stored, for example, in the determination condition data 164.

[0055] Furthermore, the driving state detection unit 120 may detect the degree of alertness from the driver's facial expression obtained by image analysis of images captured by the in-vehicle camera 60, for example. The degree of alertness indicates the degree of alertness in stages, and includes normal state, slightly decreased, considerably decreased, etc.

[0056] Furthermore, when detecting the degree of alertness, the driving state detection unit 120, based on the analysis results of images captured by the in-vehicle camera 60, sets the degree of alertness to a lower value the longer the duration of a state in which the driver of vehicle M is not performing a predetermined action (for example, a state in which the driver is not monitoring the surroundings of vehicle M, or a state in which the driver's gaze is not directed in the direction of travel of vehicle M or its surroundings). The driving state detection unit 120 may also detect the degree of alertness by analyzing blink speed, degree of eye opening, blink interval (duration of time with eyes open), blink duration (duration of time with eyes closed), etc., using artificial intelligence or the like, based on the analysis results of images captured by the in-vehicle camera 60. The driving state detection unit 120 may also detect the degree of alertness according to the length of time the eyes are closed. In this case, the longer the time the eyes are continuously closed, the lower the degree of alertness is set (or higher in the case of drowsiness). The driving state detection unit 120 may also detect the degree of alertness based on the capacitance detected by a grip sensor provided on the steering wheel 82. In this case, if the capacitance is above a predetermined amount, the driver is gripping the steering wheel 82, so the level of alertness is increased, and if it is below the predetermined amount, the level of alertness is decreased. The driving state detection unit 120 then determines that the driver's state is suitable for operating the vehicle M if the level of alertness is above a third threshold, and that it is not suitable for operating the vehicle M if it is below the third threshold. The third threshold is stored, for example, in the determination condition data 164.

[0057] The driving state detection unit 120 determines whether the driver's condition is suitable for operating the vehicle M using at least one of the methods described above. In addition to (or instead of) the methods described above, the driving state detection unit 120 may also detect the driving condition, such as whether the driver's condition is suitable for operating the vehicle M, by considering the circumstances of conversations between the driver and other passengers, or whether the driver is eating or drinking, or operating a communication device such as a smartphone or tablet.

[0058] [Deviation detection unit] Next, the functions of the lane departure detection unit 130 will be described in detail. Note that the processing in the lane departure detection unit 130 is performed when driving in a lane, and does not need to be performed, for example, when the turn signal (direction indicator) is activated or when a sudden steering operation exceeding a predetermined steering change amount is performed.

[0059] When determining whether vehicle M deviates from its lane, the deviation determination unit 130 compares information such as the positional relationship between vehicle M and the left and right lane markings that demarcate the vehicle M's lane, the direction of travel (orientation), and speed of vehicle M, as recognized by the recognition unit 110, with the deviation determination conditions. If the deviation determination conditions are met, the unit determines that vehicle M has deviated from its lane.

[0060] Figure 3 is a diagram illustrating the deviation determination of an embodiment. In the example in Figure 3, it is assumed that vehicle M is traveling at a speed VM on lane L1 demarcated by road markings LL and RL. For example, the deviation determination unit 130 determines that vehicle M has deviated from lane L1 if the distance Dv between the reference position of vehicle M (e.g., the end) and the closer of the left and right road markings LL and RL that demarcate lane L1 recognized by the recognition unit 110 is less than a predetermined distance Dth, and determines that vehicle M has not deviated from lane L1 if the distance is greater than or equal to the predetermined distance Dth.

[0061] Furthermore, the departure determination unit 130 may determine that the vehicle M deviates from lane L1 if the angle θv between the extension direction of lane L1 (X-axis direction in the figure) recognized by the recognition unit 110 or the vehicle sensor 40 and the direction of the vehicle M (direction of travel) is greater than or equal to a predetermined angle θth, and determine that the vehicle M does not deviate from lane L1 if the angle is less than the predetermined angle θth.

[0062] Furthermore, the deviation determination unit 130 derives a predicted future path of vehicle M based on the vehicle speed VM and yaw rate detected by the vehicle sensor 40, and calculates the time to line crossing (TTLC) (=d / VM) until vehicle M reaches the lane line based on the distance between the derived predicted path and the lane line (deviation path length d) and the speed VM. The deviation determination unit 130 may then determine that vehicle M has deviated from lane L1 if the time to line crossing TTLC is less than a predetermined time Tth, and determine that vehicle M has not deviated from lane L1 if it is equal to or greater than the predetermined time Tth. The above-mentioned conditions, "distance Dv is less than a predetermined distance Dth", "angle θv is greater than or equal to a predetermined angle θth", and "time to line crossing TTLC is less than a predetermined time Tth", are examples of deviation determination conditions.

[0063] If the vehicle M is determined to have deviated from lane L1 based on a deviation determination using at least one of the above-mentioned deviation determination conditions, the control unit 140 will execute deviation suppression control. The deviation determination is performed repeatedly at predetermined intervals, and if it is determined that the vehicle M has not deviated from lane L1, the control unit 140 will terminate the deviation suppression control.

[0064] Here, the deviation determination unit 130 modifies (adjusts) the above-mentioned deviation determination conditions based on the content detected by the driving state detection unit 120 and past deviation determination processes and deviation suppression controls. In the following description, the deviation determination conditions will be explained as being changeable to the first, second, third, and fourth conditions, but the content and types of conditions are not limited to these.

[0065] For example, the deviation detection unit 130 makes a deviation determination based on a second condition that is more likely to be determined as a deviation than the first condition if the driver's condition is not suitable for operating the vehicle M at the time of deviation detection. The first condition is the basic (normal) deviation detection condition (initial condition). Here, the predetermined distance Dth in the first condition is defined as "first predetermined distance Dth1", the predetermined angle θth as "first predetermined angle θth1", and the predetermined time Tth as "first predetermined time Tth1". In this case, the second condition is defined as follows: the second predetermined distance Dth2 used in the second condition is made larger than the first predetermined distance Dth1, the second predetermined angle θth2 is made smaller than the first predetermined angle θth1, and the second predetermined time Tth2 is made larger than the first predetermined time Tth1. As a result, the second condition is a condition that is more likely to be determined as a deviation than the first condition.

[0066] Therefore, if the driver's condition is not suitable for operating the vehicle M, the deviation determination unit 130 determines that the vehicle M has deviated from lane L1 using the second condition if at least one of the following conditions is met: the distance Dv is less than the second predetermined distance Dth2, the angle θv is greater than or equal to the second predetermined angle θth2, and the margin time TTLC is less than the second predetermined time Tth2.

[0067] Furthermore, if deviation suppression control (e.g., notification control, reaction force control, steering control, etc.) is performed based on the deviation determination by the second condition, the deviation determination unit 130 either returns the deviation determination condition used for the next deviation determination to the first condition, or changes it to a third condition that is more likely to be determined to be a deviation than the first condition and less likely to be determined to be a deviation than the second condition. Specifically, the third predetermined distance Dth3 used in the third condition is made larger than the first predetermined distance Dth1 and smaller than the second predetermined distance Dth2. Also, the third predetermined angle θth3 used in the third condition is made smaller than the first predetermined angle θth1 and larger than the second predetermined angle θth2. Also, the third predetermined time Tth3 used in the third condition is made larger than the first predetermined time Tth1 and smaller than the second predetermined time Tth2.

[0068] The choice between the first and third conditions is made based on the driving state detected by the driving state detection unit 120, for example, according to the content of the driving state (e.g., stability, degree of distraction, level of alertness). For example, the deviation determination unit 130 changes to the first condition if the stability and level of alertness are above a predetermined level, and the degree of distraction is below a predetermined level (or if the driver is not distracted), and changes to the third condition otherwise. The deviation determination unit 130 may also change to the first condition based on the road conditions around the vehicle M if the driving lane is a lane that is difficult to deviate from (for example, if the lane is a straight line for a predetermined distance or longer (the curvature of the lane is less than a threshold)), and change to the third condition if the lane is not a lane that is difficult to deviate from.

[0069] As a result, according to this embodiment, once lane departure prevention control is performed, it is assumed that the driver understands the situation, and the intensity (execution conditions) of the lane departure prevention control is returned to its original level. This reduces the likelihood of the driver feeling annoyed by excessive lane departure prevention control. Therefore, more appropriate lane departure prevention control can be performed.

[0070] Furthermore, the deviation determination unit 130 may perform a deviation determination using the first or third condition until a predetermined time has elapsed since the deviation suppression control was performed based on the result of the deviation determination by the second condition, or until the vehicle M has traveled a predetermined distance or more. According to this embodiment, when the deviation suppression control is performed, the driver operates (drives) the vehicle so as not to deviate from the driving lane. Therefore, after the deviation suppression control is performed, it is expected that the driver will drive carefully for a while to prevent the vehicle M from deviating from the driving lane. Accordingly, by changing to the first or third condition, which is less likely to be determined to deviate than the second condition regardless of other conditions, until a predetermined time has elapsed since the deviation suppression control was performed, or until the vehicle has traveled a predetermined distance or more, it is possible to reduce the driver's discomfort caused by excessive deviation suppression control.

[0071] Furthermore, the deviation determination unit 130 may perform a deviation determination using the first or third condition even if the driver's condition remains unsuitable for operating the vehicle M after deviation suppression control has been performed based on the deviation determination using the second condition. Since there may be a time error (time lag) in the detection of the driving state by the driving state detection unit 120, even if the driver's condition remains unsuitable for operating the vehicle M for a certain period of time, the system may change to the first or third condition, which is less likely to be determined to be a deviation than the second condition. This suppresses misdetermination due to the effect of time lag and enables more appropriate deviation suppression control. Even if the second condition is changed to the first or third condition, the deviation determination continues, thus ensuring safety.

[0072] Furthermore, if the driver remains unsuitable for driving even after a predetermined time has elapsed since the execution of the deviation suppression control, or after the vehicle M has traveled a predetermined distance or more, the deviation determination unit 130 may perform a deviation determination using the second condition or a fourth condition which is more likely to be determined to be a deviation than the second condition. Specifically, the fourth predetermined distance Dth4 used in the fourth condition is made greater than the second predetermined distance Dth2. Also, the fourth predetermined angle θth4 used in the fourth condition is made smaller than the second predetermined angle θth2. Also, the fourth predetermined time Tth4 used in the fourth condition is made greater than the second predetermined time Tth2. As a result, the fourth condition is a condition which is more likely to be determined to be a deviation than the second condition.

[0073] The choice between the second and fourth conditions may be made according to the driving state detected by the driving state detection unit 120, as described above. In this case, the deviation determination unit 130 will change to the fourth condition if, for example, the stability level and alertness level are below a predetermined level, and the distraction level is above a predetermined level, and to the second condition otherwise. The deviation determination unit 130 may also change to the second condition if the driving lane is a lane where deviation is unlikely, based on the road conditions around the vehicle M, and to the fourth condition if it is not a lane where deviation is unlikely. The deviation determination unit 130 may also be set according to the number of times deviation suppression control is performed within a predetermined time. In this case, for example, the second condition is set if the number of times is less than a predetermined number, and the fourth condition is set if the number of times is predetermined or greater. This makes it easier to activate deviation suppression control when no improvement is seen in the driving state, thus enabling deviation suppression control to be performed at a more appropriate timing.

[0074] Furthermore, the deviation determination unit 130 may perform a deviation determination using the first condition after a predetermined time has elapsed since the execution of deviation suppression control, or after the vehicle M has traveled a predetermined distance or more, if the driver of the vehicle M is in a condition suitable for driving. This allows the deviation determination to be performed using the normal first condition when an improvement in the driving condition is observed.

[0075] The first to fourth conditions described above may each be registered in the judgment condition data 164. Furthermore, at least the first condition among the first to fourth conditions may be set by the driver or other occupants. For example, if the first condition is set by the driver, the second to fourth conditions are adjusted by the system (e.g., the deviation determination unit 130) based on the content of the first condition. This allows the driver to customize the timing of the deviation suppression control execution according to their intentions by setting the first condition (initial condition) used in normal deviation determination. Therefore, it is possible to prevent the driver from feeling any discomfort with the timing of the deviation suppression control execution. The first condition and other settings are configured, for example, by an occupant operating the HMI 30, and the configured information is stored in the judgment condition data 164.

[0076] Furthermore, at least one of the first to fourth conditions may be set by the system based on the driver's past driving information (operation history, driving conditions when lane departure occurs, etc.). In this case, the past driving information is stored in the memory unit 160, etc. This makes it possible to set more appropriate conditions by reflecting the driving tendencies of each driver.

[0077] In this way, deviation detection conditions can be set according to the driver's condition and the vehicle M's condition, allowing for more accurate deviation detection and, based on the results, more appropriate driving control (such as deviation suppression control) can be achieved.

[0078] [Processing flow] The following describes the processes performed by the driver assistance device 100 of this embodiment. Specifically, the following describes the vehicle M departure detection process and the departure suppression control process among the processes performed by the driver assistance device 100. Figure 4 is a flowchart showing an example of the processes performed by the driver assistance device 100 in this embodiment. Note that the processes shown in Figure 4 may be repeatedly executed at predetermined timings or predetermined cycles.

[0079] In the example shown in Figure 4, the recognition unit 110 recognizes the surrounding conditions of the vehicle M based on the output from the detection device DD (step S100). Next, the driving state detection unit 120 detects the driving state of the vehicle M by the driver (step S200). Next, the deviation determination unit 130 sets deviation determination conditions based on the driving state and the previous deviation determination conditions, etc. (step S300).

[0080] Next, the deviation detection unit 130 determines whether the state of vehicle M satisfies the set deviation detection conditions (step S400). If it is determined that the deviation detection conditions are met, the HMI control unit 142 notifies the driver (e.g., a deviation warning) (step S500). Next, the driving control unit 144 performs behavioral control of vehicle M (including reaction force control) (step S600). This completes the processing of this flowchart. Also, if it is determined in step S400 that the deviation detection conditions are not met, this flowchart ends. Note that the processing in steps S500 and S600 is an example of "deviation suppression control". Furthermore, deviation suppression control may be performed by executing either step S500 or S600.

[0081] [Specific example of the process in step S300] Next, the process of setting (changing) the deviation judgment conditions in the S300 process described above will be explained using a diagram.

[0082] Figure 5 is a flowchart illustrating a first example of the process for setting deviation judgment conditions. In the example in Figure 5, the deviation judgment unit 130 determines whether the deviation judgment conditions from the previous deviation judgment were applied to the second condition (step S310). If it is determined that the conditions were not applied to the second condition (including cases where a deviation judgment has not been performed for a predetermined time or longer), the deviation judgment unit 130 determines whether the driver's condition is suitable for driving vehicle M (step S320). If it is determined that the driver's condition is suitable for driving vehicle M, the deviation judgment unit 130 sets the deviation judgment condition used for deviation judgment in the process of step S400 to the first condition described above (step S330). Also, in the process of step S320, if it is determined that the driver's condition is not suitable for driving the vehicle, the deviation judgment condition used for deviation judgment is set (changed) to the second condition, which is more likely to be determined to be a deviation than the first condition (step S340).

[0083] Furthermore, in the process of step S310, if it is determined that the deviation determination condition used in the previous deviation determination was the second condition, the deviation determination unit 130 sets the deviation determination condition to the first condition, or to the third condition, which is more likely to be determined to be a deviation than the first condition, but less likely to be determined to be a deviation than the second condition (step S350). This completes the process of this flowchart.

[0084] Figure 6 is a flowchart showing a second example of the process for setting deviation judgment conditions. The process in Figure 6 differs from the process in steps S310 to S350 shown in Figure 5 in that it includes the process in step S312 instead of step S310, and also includes the processes in steps S360 to S380. Therefore, the following will mainly explain the differences, and a detailed explanation of the other parts mentioned above will be omitted.

[0085] In the example in Figure 6, the deviation determination unit 130 determines whether the deviation determination conditions from the previous deviation determination were met under the second or fourth condition (step S312). If it is determined that the conditions were not met under the second or fourth condition (including cases where a deviation determination has not been made for a predetermined time or longer), the deviation determination unit 130 determines whether the driver's condition is suitable for driving vehicle M (step S320). If it is determined that the driver's condition is suitable for driving vehicle M, the deviation determination unit 130 sets the deviation determination condition to be used for the deviation determination to the first condition (step S330).

[0086] Furthermore, in the process of step S320, if it is determined that the driver's condition is not suitable for driving the vehicle, the deviation determination unit 130 determines whether a predetermined time has elapsed or whether a predetermined distance has been traveled since the execution of the deviation suppression control corresponding to the processes of steps S500 and S600 (step S360). If it is determined that a predetermined time has elapsed or a predetermined distance has been traveled, the deviation determination unit 130 sets the deviation determination condition to the second condition (step S340). Furthermore, in the process of step S360, if it is determined that a predetermined time has not elapsed since the execution of the deviation suppression control and that the driver has not traveled a predetermined distance, the deviation determination unit 130 sets the deviation determination condition to the second condition, or to a fourth condition which is more likely to be determined to be a deviation than the second condition (step S370).

[0087] Furthermore, in the process of step S312, if it is determined that the previous deviation determination was made based on the second or fourth condition, the deviation determination unit 130 determines whether a predetermined time has elapsed or whether the vehicle M has traveled a predetermined distance or more since the deviation suppression control was executed (step S380). If it is determined that a predetermined time has elapsed or the vehicle M has traveled a predetermined distance or more, the process of step S320 is performed. If it is determined that a predetermined time has not elapsed and the vehicle has not traveled a predetermined distance or more, the process of step S350 is performed. This completes the process of this flowchart.

[0088] [Differentiation] In this embodiment, when the HMI control unit 142 performs notification control, it may change the output mode of the deviation alarm based on the determination conditions used by the deviation determination unit 130. In this case, for example, if the deviation determination is made under the first condition, the HMI control unit 142 will notify (deviate alarm) only by sound; if the deviation determination is made under the third condition, it will notify by sound and vibration; and if the deviation determination is made under the second or fourth condition, it will notify by sound, image, and vibration. This allows for more appropriate notification control depending on the situation.

[0089] According to the embodiments described above, the mobile body control device includes a recognition unit 110 that recognizes the surrounding conditions, including the position of a vehicle M (an example of a mobile body) relative to a driving lane (an example of a travel path); a deviation determination unit 130 that determines whether or not the vehicle M deviates from the driving lane based on the position of the vehicle M and a first condition; a control unit 140 that performs deviation suppression control, including at least one of notification control and behavioral control in a direction that suppresses deviation, when the deviation determination unit 130 determines that the vehicle M deviates from the driving lane; and a driving state detection unit 120 that detects the state of the driver of the vehicle M. The deviation determination unit 130 makes a deviation determination based on a second condition that makes it easier to determine that the vehicle M deviates from the driving lane than the first condition when it is determined that the driver's state is not suitable for driving the vehicle M. When the deviation determination unit 130 determines that the driver's state is not suitable for driving the vehicle M, it makes a deviation determination based on a second condition that makes it easier to determine that the vehicle M deviates from the driving lane than the first condition. When deviation suppression control is performed based on the result of the deviation determination based on the second condition, the device makes the next deviation determination based on the first condition, or based on a third condition that makes it easier to determine that the vehicle deviates than the first condition and less likely to determine that the vehicle deviates than the second condition, thereby enabling more appropriate deviation suppression control.

[0090] For example, according to the embodiment, once deviation prevention control is performed, it is assumed that the driver understands the situation, and the intensity of the deviation prevention control (deviation judgment condition) is returned to its original value, thereby reducing the annoyance caused by excessive deviation prevention control. Also, according to the embodiment, for example, once prevention is performed, for a predetermined period, regardless of other conditions, it is difficult to determine that a deviation has occurred, thereby reducing the annoyance caused by excessive control. If there is no improvement after the predetermined period, the second condition can be used to provide notification (warning) to the occupants at an earlier timing. Furthermore, if improvement is observed in the driver after deviation prevention control, the execution timing of the deviation prevention control can be returned to the original by changing to the initial judgment condition (first condition). This makes it possible to suppress a mismatch between the driver's state (e.g., attention to driving) and the timing of the deviation prevention control (e.g., deviation warning).

[0091] The embodiments described above can be expressed as follows. A storage medium that stores computer-readable instructions, A processor connected to the storage medium, The processor executes the computer-readable instructions to: Recognize the surrounding conditions, including the position of the moving object relative to the path of movement, Based on the position of the moving body and the first condition, a deviation determination is made to determine whether the moving body deviates from the movement path. If it is determined that the moving body deviates from the movement path, deviation suppression control is performed, which includes at least one of notification control and behavior control for the moving body in a direction that suppresses deviation. The driver's status of the moving object is detected, If it is determined that the driver's condition is unsuitable for driving the moving body, the deviation determination is made using a second condition that makes it more likely than the first condition that the moving body will deviate from the travel path. When the deviation suppression control is performed based on the result of the deviation determination under the second condition, the next deviation determination is made based on the first condition, or based on a third condition that is more likely to result in a deviation than the first condition and less likely to result in a deviation than the second condition. Mobile device control system.

[0092] Although embodiments for carrying out the present invention have been described above using examples, the present invention is not limited in any way to these embodiments, and various modifications and substitutions can be made without departing from the spirit of the present invention. [Explanation of Symbols]

[0093] 1...Vehicle system, 10...Camera, 12...Radar device, 14...LIDAR, 20...Communication device, 30...HMI, 32...Display unit, 34...Speaker, 36...Vibration unit, 40...Vehicle sensor, 50...Navigation device, 60...In-cabin camera, 80...Driver's control unit, 82...Steering wheel, 100...Driver's assistance device, 110...Recognition unit, 120...Driving state detection unit, 130...Deviation detection unit, 140...Control unit, 142...HMI control unit, 144...Driving control unit, 160...Storage unit, 200...Driving force output device, 210...Brake device, 220...Steering device, M...Vehicle

Claims

1. A recognition unit that recognizes the surrounding conditions, including the position of the moving object relative to the path of movement, A deviation determination unit that determines whether the moving body deviates from the movement path based on the position of the moving body and a first condition, When the deviation determination unit determines that the moving body deviates from the movement path, the control unit performs deviation suppression control which includes at least one of notification control and behavior control for the moving body in a direction that suppresses deviation. The system includes a driving state detection unit that detects the state of the driver of the moving body, If the deviation determination unit determines that the driver's condition is unsuitable for driving the moving body, it performs the deviation determination using a second condition that makes it more likely than the first condition that the moving body will deviate from the travel path. When the deviation suppression control is performed based on the result of the deviation determination under the second condition, the next deviation determination is made based on the first condition, or based on a third condition that is more likely to result in a deviation than the first condition and less likely to result in a deviation than the second condition. Mobile device control system.

2. The deviation determination unit performs the deviation determination using the first condition or the third condition until a predetermined time has elapsed since the deviation suppression control was performed based on the result of the deviation determination under the second condition, or until the moving body has moved a predetermined distance or more. The mobile device control device according to claim 1.

3. The deviation determination unit performs the deviation determination using the first condition or the third condition even if the driver remains unsuitable for driving after the deviation suppression control has been performed based on the result of the deviation determination based on the second condition. The mobile device control device according to claim 1.

4. The deviation determination unit performs the deviation determination using the second condition or a fourth condition which is more likely to be determined to be a deviation than the second condition, if the driver remains unsuitable for driving after the predetermined time has elapsed or after the moving body has moved a predetermined distance or more. The mobile device control device according to claim 2.

5. The deviation determination unit performs the deviation determination using the first condition after the predetermined time has elapsed or after the moving body has moved a predetermined distance or more, if the driver's condition is suitable for driving. The mobile device control device according to claim 2.

6. Of the first, second, third, and fourth conditions, at least the first condition can be set by the occupants of the mobile body. The mobile device control device according to claim 4.

7. Computers Recognize the surrounding conditions, including the position of the moving object relative to the path of movement, Based on the position of the moving body and the first condition, a deviation determination is made to determine whether the moving body deviates from the movement path. If it is determined that the moving body deviates from the movement path, deviation suppression control is performed, which includes at least one of notification control and behavior control for the moving body in a direction that suppresses deviation. The driver's status of the moving object is detected, If it is determined that the driver's condition is unsuitable for driving the moving body, the deviation determination is made using a second condition that makes it more likely than the first condition that the moving body will deviate from the path. When the deviation suppression control is performed based on the result of the deviation determination under the second condition, the next deviation determination is made based on the first condition, or based on a third condition that is more likely to result in a deviation than the first condition and less likely to result in a deviation than the second condition. A method for controlling a mobile object.

8. On the computer, To recognize the surrounding situation, including the position of the moving object relative to the path of movement, Based on the position of the moving body and the first condition, a deviation determination is made to determine whether the moving body deviates from the movement path. If it is determined that the moving body deviates from the movement path, the system will perform deviation suppression control, which includes at least one of notification control and behavioral control of the moving body in a direction that suppresses deviation. The state of the driver of the aforementioned mobile body is detected, If it is determined that the driver's condition is unsuitable for driving the moving body, the deviation determination is made based on a second condition that makes it more likely than the first condition that the moving body will deviate from the travel path. When the deviation suppression control is performed based on the result of the deviation determination under the second condition, the next deviation determination is made based on the first condition, or based on a third condition that is more likely to result in a deviation than the first condition and less likely to result in a deviation than the second condition. program.

Images