Mobile device control device, mobile device control method, and program

Abstract

To provide driving assistance that does not cause discomfort to the driver. [Solution] A mobile body control device comprising: a first recognition unit that recognizes objects around a mobile body based on the surrounding conditions of the mobile body detected by a first detection device; a second recognition unit that recognizes the driver's viewing direction based on information about the driver of the mobile body detected by a second detection device; and a control unit that, based on the recognition result of the first recognition unit, determines that the mobile body has deviated from the road or is in a road departure state where it may deviate, and activates road departure suppression control, which is at least one of the controls of a warning and steering assistance, wherein, with respect to the width direction of the road of the mobile body, an object to be avoided that is on the opposite side of the road departure direction in which the mobile body deviates from the road is recognized by the first recognition unit, and the second recognition unit determines that the driver is viewing the object to be avoided.

Description

Technical Field

[0001] The present invention relates to a movement control device, a movement control method, and a program.

Background Art

[0002] In recent years, research and practical application of automatic driving and driving support that automatically control the driving of vehicles have been advanced. For example, there is known a driving support system that identifies a road based on the current position and surrounding conditions of a vehicle and activates an alarm or steering support so that the vehicle does not deviate from the road. In relation to such technology, when an operation other than a driving operation such as a driver's steering operation, accelerator operation, or brake operation is detected, a technology has been proposed for determining the possibility of the vehicle deviating from the road in consideration of the operation other than the driving operation (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the conventional driving support system, driving support that gives a driver a sense of discomfort may be performed. For example, in the conventional driving support system, since an alarm or steering support is activated regardless of whether the driver recognizes an avoidance target, when the driver intends to intentionally deviate from the road, the driver may be given a sense of discomfort by the unintended driving support.

[0005] One of the objectives of this invention is to provide a mobile vehicle control device, a mobile vehicle control method, and a program that can provide driving assistance without causing discomfort to the driver, in order to solve the above-mentioned problems. More specifically, one of the objectives is to avoid causing discomfort to the driver by suppressing the operation of lane departure prevention control, which is a warning or steering assistance, when the driver is about to intentionally deviate from the road. [Means for solving the problem]

[0006] The mobile device control device, mobile device control method, and program according to this invention employ the following configuration. (1) A mobile body control device according to one aspect of the present invention includes: a first recognition unit that recognizes objects around a mobile body based on the surrounding conditions of the mobile body detected by a first detection device; a second recognition unit that recognizes the direction of the driver's view based on information about the driver of the mobile body detected by a second detection device; and a control unit that, based on the recognition result of the first recognition unit, determines that the mobile body has deviated from the road or is in a road departure state where it may deviate, and activates road departure suppression control, which is at least one of a warning and steering assistance, wherein the first recognition unit recognizes an object to be avoided that is on the opposite side of the road departure direction in which the mobile body deviates from the road with respect to the width direction of the road of the mobile body, and the second recognition unit determines that the driver is looking at the object to be avoided.

[0007] (2) In the embodiment of (1) above, the road departure state is when the moving body deviates from the road on which it is traveling, or is likely to deviate from the road, or when the moving body deviates from the lane on which it is traveling, or is likely to deviate from the lane.

[0008] (3) In the embodiment of (2) above, the control unit suppresses the operation of the lane departure prevention control by not activating at least one of the alarm and the steering assist.

[0009] (4) In the embodiment of (2) above, the control unit suppresses the operation of the road departure prevention control by controlling at least one of the warning and the steering assist so that the moving body moves along the target road, which is the road or lane.

[0010] (5) In the embodiment of (3) or (4) above, the first recognition unit further includes a determination unit that recognizes the conditions of an out-of-lane area which is an area outside the road or lane around the moving body, and determines whether or not the moving body can travel in the out-of-lane area based on the conditions of the out-of-lane area, wherein if the determination unit determines that the moving body can travel in the out-of-lane area, the control unit performs a first suppression control that suppresses the operation of the road departure prevention control, thereby allowing the moving body to travel in the out-of-lane area temporarily, and if the determination unit determines that the moving body cannot travel in the out-of-lane area, the control unit performs a second suppression control which suppresses the operation of the road departure prevention control to a lesser degree than the first suppression control, or does not suppress the operation of the road departure prevention control and does not allow the moving body to travel in the out-of-lane area temporarily.

[0011] (6): In the embodiment of (5) above, the determination unit determines that if a dynamic obstacle exists in the area outside the lane, the moving body cannot travel in the area outside the lane.

[0012] (7) In the embodiment of (5) above, the determination unit determines that if a static obstacle exists in the area outside the lane, the moving body cannot travel in the area outside the lane.

[0013] (8) In the embodiment of (1) above, if the first recognition unit recognizes the object to be avoided and the second recognition unit recognizes that the driver has not recognized the object to be avoided, the control unit performs the lane departure suppression control.

[0014] (9): In the embodiment of (8) above, the control unit suppresses the road departure prevention control when the moving body is traveling on a narrow road or when the moving body enters a road width reduction area which is an area in which the width decreases.

[0015] (10): In the embodiment of (9) above, the control unit controls the steering assistance such that the moving body is positioned at a location where the distance between the moving body and the object to be avoided is greater than or equal to a first threshold in the width direction of the moving body, or the time it takes for the moving body to reach the object to be avoided is greater than or equal to a second threshold in the width direction of the moving body.

[0016] (11): In the embodiment of (9) above, the control unit suppresses the operation of the lane departure prevention control by activating the warning without activating the steering assistance for the driver.

[0017] (12): A mobile body control method according to another aspect of the present invention, wherein the computer recognizes objects around the mobile body based on the surrounding conditions of the mobile body detected by a first detection device, recognizes the direction of the driver's view based on information about the driver of the mobile body detected by a second detection device, and, based on the surrounding conditions of the mobile body, determines that the mobile body has deviated from the road or is in a road departure state where it may deviate, activates road departure suppression control, which is at least one of the controls of a warning and steering assistance, and, with respect to the width direction of the mobile body, recognizes an object to be avoided that is on the opposite side of the road departure direction from which the mobile body deviates, and determines that the driver is looking at the object to be avoided based on information about the driver of the mobile body, the operation of the road departure suppression control is suppressed.

[0018] (13): A program according to another aspect of the present invention causes a computer to recognize objects around a moving body based on the surrounding conditions of the moving body detected by a first detection device, to recognize the direction in which the driver is looking based on information about the driver of the moving body detected by a second detection device, and if it is determined that the moving body has deviated from the road or is in a road departure state based on the surrounding conditions of the moving body, it activates a road departure suppression control which is at least one of a warning and steering assistance, and if, with respect to the width direction of the moving body, an object to be avoided that is located on the opposite side of the road departure direction from which the moving body deviates from the road is recognized, and it is determined that the driver is looking at the object to be avoided based on information about the driver of the moving body, it suppresses the operation of the road departure suppression control. [Effects of the Invention]

[0019] According to the embodiments described in (1) to (13) above, lane departure prevention control can be performed without causing discomfort to the driver. For example, when the driver intentionally attempts to deviate from the lane, the operation of the warning or steering assistance (e.g., lane departure prevention control) can be suppressed, thereby enabling lane departure prevention control that takes the driver's intentions into consideration. As a result, driving assistance can be provided without causing discomfort to the driver.

[0020] According to the embodiment described in (2) above, appropriate driving assistance can be provided when the moving object is unlikely to deviate from the road or road markings (including imaginary lines) on which it is traveling.

[0021] According to the embodiment described in (3) above, when the driver intentionally attempts to deviate from the road, the operation of the road departure prevention control that is not intended by the driver is suppressed, thereby realizing control that reflects the driver's intentions.

[0022] According to the embodiment described in (4) above, when the driver intentionally attempts to deviate from the road, the activation of the road departure prevention control that is not intended by the driver is reduced, thereby enabling control that reflects the driver's intentions.

[0023] According to the aspect (5) above, by recognizing the situation of the off-lane area, it is determined whether the moving body can deviate from the road and travel, and by allowing travel in the off-lane area according to the result of the determination, appropriate control according to the situation around the moving body is performed.

[0024] According to the aspect (6) above, when there is a dynamic obstacle in the off-lane area, it can be determined that the moving body cannot deviate from the road and travel. In this way, the movement control device can accurately determine the situation of the off-lane area and perform appropriate control according to the result of the determination.

[0025] According to the aspect (7) above, when there is a static obstacle in the off-lane area, it can be determined that the moving body cannot deviate from the road and travel. In this way, the movement control device can accurately determine the situation of the off-lane area and perform appropriate control according to the result of the determination.

[0026] According to the aspect (8) above, when the driver unintentionally tries to make the moving body deviate from the road, the lane departure suppression control is activated, and the moving body is appropriately controlled without deviating from the road.

[0027] According to the aspect (9) above, when the moving body travels in a narrow road or a width reduction area, the activation of the lane departure suppression control can be suppressed, and appropriate control according to the situation around the moving body is performed.

[0028] According to the aspect (10) above, when the moving body travels in a narrow road or a width reduction area, steering assistance can be provided to avoid the avoidance target object, and appropriate control according to the situation around the moving body is performed.

[0029] According to the aspect (11) above, when the moving body travels in a narrow road or a width reduction area, the steering assistance can be suppressed, and appropriate control according to the situation around the moving body is performed.

Brief Description of the Drawings

[0030] [Figure 1]This is a diagram showing the configuration of a vehicle system 1 utilizing a mobile control system according to an embodiment. [Figure 2] This figure shows an example of a scenario in which the processing of this embodiment is applied. [Figure 3] This figure shows an example of a scenario in which the processing of this embodiment is applied. [Figure 4] This figure shows an example of a scenario in which the processing of this embodiment is applied. [Figure 5] This flowchart shows an example of the processing flow performed by the driver assistance device 100. [Figure 6] This figure shows an example of a scenario in which the processing of this embodiment is applied. [Figure 7] This figure shows an example of a scenario in which the processing of this embodiment is applied. [Figure 8] This flowchart shows an example of the processing flow performed by the driver assistance device 100. [Figure 9] This figure shows an example of a scenario in which the processing of this embodiment is applied. [Figure 10] This figure shows an example of a scenario in which the processing of this embodiment is applied. [Figure 11] This flowchart shows an example of the processing flow performed by the driver assistance device 100. [Modes for carrying out the invention]

[0031] [Overall structure] Figure 1 is a diagram showing the configuration of a vehicle system 1 utilizing a vehicle control system according to an embodiment. The vehicle on which the vehicle system 1 is mounted is, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle, and its drive source is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. The electric motor operates using power generated by a generator connected to the internal combustion engine, or discharge power from a secondary battery or fuel cell. This embodiment is described as being applicable to a vehicle, but it may be applied to other mobile bodies instead of a vehicle.

[0032] Vehicle system 1 includes, for example, a camera 10, a radar device 12, a LiDAR (Light Detection and Ranging) 14, an object recognition device 16, an in-cabin camera 18, a communication device 20, an HMI (Human Machine Interface) 30, a vehicle sensor 40, a navigation device 50, an MPU 60, an operator 80, a driving assistance device 100, a driving force output device 200, a braking device 210, and a steering device 220. These devices and equipment are connected to each other by multiplex communication lines such as CAN (Controller Area Network) communication lines, serial communication lines, wireless communication networks, etc. The configuration shown in Figure 1 is merely an example, and some of the configuration may be omitted, or other configurations may be added. The driving assistance device 100 is an example of a "mobile vehicle control device".

[0033] Camera 10 is a digital camera that utilizes a solid-state image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor). Camera 10 is mounted at any location on the vehicle (hereinafter referred to as vehicle M) on which the vehicle system 1 is installed. When imaging the area in front, camera 10 is mounted on the top of the front windshield, behind the rearview mirror, etc. Camera 10 periodically and repeatedly images the area around vehicle M. Camera 10 may also be a stereo camera.

[0034] The radar device 12 emits radio waves such as millimeter waves around the vehicle M and detects radio waves reflected by objects (reflected waves) to determine at least the position (distance and bearing) of the object. The radar device 12 can be mounted at any location on the vehicle M. The radar device 12 may also detect the position and velocity of the object using the FM-CW (Frequency Modulated Continuous Wave) method.

[0035] LIDAR14 irradiates light (or electromagnetic waves with a wavelength close to light) around vehicle M and measures the scattered light. Based on the time from emission to reception, LIDAR14 detects the distance to the target. The irradiated light is, for example, pulsed laser light. LIDAR14 can be attached to any location on vehicle M.

[0036] The object recognition device 16 performs sensor fusion processing on the detection results from some or all of the camera 10, radar device 12, and LIDAR 14 to recognize the position, type, speed, etc., of an object. The object recognition device 16 outputs the recognition results to the driver assistance device 100. The object recognition device 16 may output the detection results from the camera 10, radar device 12, and LIDAR 14 directly to the driver assistance device 100. The object recognition device 16 may be omitted from the vehicle system 1.

[0037] The in-vehicle camera 18 is a digital camera that uses a solid-state image sensor such as a CCD or CMOS. The in-vehicle camera 18 is mounted in a position and orientation that allows it to capture the head of the occupant (driver) seated in the driver's seat of vehicle M from the front (in a direction that captures the face). The in-vehicle camera 18 may be mounted anywhere in vehicle M as long as the above conditions are met. For example, the in-vehicle camera 18 is mounted in front of the ceiling in the interior of vehicle M. The in-vehicle camera 18 generates image information that captures the scenery inside the vehicle. When a driver is in the vehicle, the in-vehicle camera 18 captures the driver's face. The captured image information includes the driver's face. The image information may also include the state of occupants (passengers) seated in seats other than the driver's seat of vehicle M. The in-vehicle camera 18 transmits the generated image information to the driver assistance device 100.

[0038] The communication device 20 communicates with other vehicles in the vicinity of vehicle M, or with various server devices via a wireless base station, for example, by using a cellular network, Wi-Fi network, Bluetooth®, DSRC (Dedicated Short Range Communication), etc.

[0039] The HMI30 presents various information to the occupants of vehicle M and accepts input operations from the occupants. The HMI30 includes various display devices, speakers, buzzers, touch panels, switches, keys, etc. The HMI30 is equipped with a display device. The display device is a display device, also known as a multi-information display, that displays various information in vehicle M, such as a speedometer showing the vehicle's speed or a tachometer showing the rotational speed of the internal combustion engine in vehicle M, and is located in the center of the instrument panel of vehicle M.

[0040] The vehicle sensor 40 includes a vehicle speed sensor for detecting the speed of the vehicle M, an acceleration sensor for detecting acceleration, a yaw rate sensor for detecting angular velocity around the vertical axis, and an orientation sensor for detecting the orientation of the vehicle M.

[0041] The navigation device 50 includes, for example, a GNSS (Global Navigation Satellite System) receiver 51, a navigation HMI 52, and a route determination unit 53. The navigation device 50 stores first map information 54 in a storage device such as an HDD (Hard Disk Drive) or flash memory. The GNSS receiver 51 determines the position of the vehicle M based on signals received from GNSS satellites. The position of the vehicle M may be determined or supplemented by an INS (Inertial Navigation System) that utilizes the output of the vehicle sensors 40. The navigation HMI 52 includes a display device, speakers, a touch panel, keys, etc. The navigation HMI 52 may be partially or completely shared with the HMI 30 described above. The route determination unit 53 determines, for example, a route (hereinafter referred to as the route on the map) from the position of the vehicle M determined by the GNSS receiver 51 (or any input position) to the destination input by the occupant using the navigation HMI 52, by referring to the first map information 54. The first map information 54 is, for example, information in which the road shape is represented by links indicating roads and nodes connected by those links. The first map information 54 may also include information such as road curvature and POI (Point of Interest) information. The route on the map is output to the MPU 60. The navigation device 50 may provide route guidance using the navigation HMI 52 based on the route on the map. The navigation device 50 may be implemented, for example, by the functions of a terminal device such as a smartphone or tablet held by an occupant. The navigation device 50 may transmit the current location and destination to the navigation server via the communication device 20 and obtain a route equivalent to the route on the map from the navigation server.

[0042] The MPU 60 includes, for example, a recommended lane determination unit 61 and stores second map information 62 in a storage device such as an HDD or flash memory. The recommended lane determination unit 61 divides the map route provided by the navigation device 50 into multiple blocks (for example, every 100m with respect to the vehicle's direction of travel) and determines a recommended lane for each block by referring to the second map information 62. The recommended lane determination unit 61 makes decisions such as which lane from the left the vehicle should travel in. If there is a branching point on the map route, the recommended lane determination unit 61 determines a recommended lane so that the vehicle M can travel along a reasonable route to proceed to the branching point. For example, if the vehicle M reaches a predetermined distance before a branching road it is traveling on, the recommended lane determination unit 61 determines the lane connecting to the branching road as the recommended lane. The recommended lane determination unit 61 and the second map information 62 may be functional units or information included in other devices such as the driver assistance device 100.

[0043] The second map information 62 is map information with higher accuracy than the first map information 54. The second map information 62 includes, for example, information on the center of lanes or information on lane boundaries. The second map information 62 may also include road information, traffic regulation information, address information (address and postal code), facility information, telephone number information, etc. The second map information 62 may be updated as needed by the communication device 20 communicating with other devices.

[0044] The control element 80 includes, for example, a steering wheel, accelerator pedal, brake pedal, shift lever, and other control elements. The control element 80 is equipped with a sensor that detects the amount of operation or whether or not an operation is performed, and the detection result is output to the driver assistance device 100, or to some or all of the driving force output device 200, brake device 210, and steering device 220. The steering wheel does not necessarily have to be annular in shape and may take the form of an irregularly shaped steering wheel, joystick, buttons, etc.

[0045] The driver assistance device 100 includes, for example, a first recognition unit 110, a second recognition unit 120, a departure detection unit 130, a control unit 140, and an out-of-lane area detection unit 150. The first recognition unit 110, the second recognition unit 120, the departure detection unit 130, the control unit 140, and the out-of-lane area detection unit 150 are realized, for example, by a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit), and SOC (System On Chip), or by the cooperation of software and hardware. The program may be stored in advance in a storage device (a storage device equipped with a non-transient storage medium) such as the HDD or flash memory of the driver assistance device 100, or it may be stored in a removable storage medium such as a DVD or CD-ROM and installed in the HDD or flash memory of the driver assistance device 100 when the storage medium (non-transient storage medium) is mounted on the drive device. The control unit 140, or the combined functional configuration of the departure detection unit 130 and the control unit 140, is an example of a "control unit". The lane out-of-lane area detection unit 150 is an example of a "determination unit".

[0046] The first recognition unit 110 recognizes the position and state, such as speed and acceleration, of objects around the vehicle M based on information input from the camera 10, radar device 12, and LIDAR 14 via the object recognition device 16. The position of an object is recognized as a position on an absolute coordinate system with a representative point of the vehicle M (such as the center of gravity or the center of the drive axis) as the origin, and is used for control. The position of an object may be represented by a representative point such as the center of gravity or a corner of the object, or it may be represented by a region. The "state" of an object may include the object's acceleration, jerk, or "action state" (for example, whether or not it is changing lanes or attempting to change lanes).

[0047] The first recognition unit 110 recognizes, for example, the roadway of vehicle M and other roadways in its vicinity. Roadways include the lane in which vehicle M is traveling (driving lane) and opposing lanes relative to the driving lane. For example, the first recognition unit 110 recognizes roadways by comparing the pattern of road markings (e.g., an arrangement of solid and dashed lines) obtained from the second map information 62 with the pattern of road markings around vehicle M recognized from the image captured by the camera 10. The first recognition unit 110 may recognize roadways not only by road markings but also by recognizing road boundaries (road boundaries) that include road markings, shoulders, curbs, median strips, guardrails, etc. In this recognition, the position of vehicle M obtained from the navigation device 50 and the processing results by INS may also be taken into consideration. The first recognition unit 110 recognizes stop lines, obstacles, red lights, toll booths, and other road events present around vehicle M.

[0048] The first recognition unit 110 recognizes the behavior of the vehicle M based on the detection results of the vehicle sensor 40. For example, when the first recognition unit 110 recognizes a road, it recognizes the position and attitude of the vehicle M relative to the road. The first recognition unit 110 may recognize, for example, the deviation of the vehicle M's reference point from the center of the lane, and the angle it makes with a line connecting the centers of the lanes in the direction of travel of the vehicle M, as the relative position and attitude of the vehicle M relative to the road. Alternatively, the first recognition unit 110 may recognize the position of the vehicle M's reference point relative to one of the side edges of the road (road markings or road boundary) as the relative position of the vehicle M relative to the road.

[0049] The second recognition unit 120 recognizes the driver's viewing direction based on image information obtained from the in-vehicle camera 18 (an example of "information about the driver"). The second recognition unit 120 performs known image processing techniques on the image captured by the in-vehicle camera 18 and recognizes the driver's viewing direction from the driver's gaze and the direction of the driver's face based on the analysis results. For example, the second recognition unit 120 recognizes the driver's gaze and the direction of the driver's face using methods such as template matching. The second recognition unit 120 also recognizes the direction of the driver's face based on positional information of the eyes, nose, mouth, etc. within the face region obtained from the image analysis results (relative positional information of each part, etc.). In the recognition of each piece of information using the image described above, for example, a pre-trained model learned in advance by machine learning may be used. The driver's gaze direction may be recognized by other sensors instead of (or in addition to) the in-vehicle camera 18. For example, the driver's gaze direction may be detected by other known optical methods.

[0050] The deviation determination unit 130 determines whether or not vehicle M is in a lane departure state based on information such as the positional relationship between vehicle M and the left and right road markings that demarcate the driving lane, the direction of travel of vehicle M, and the speed of vehicle M, as recognized by the first recognition unit 110. A lane departure state is a state in which vehicle M has deviated outside the driving lane, or a state in which vehicle M may deviate outside the driving lane. A state in which vehicle M has deviated outside the driving lane is, for example, a state in which the reference position of vehicle M (e.g., edge, center of gravity, center) has deviated outside either the left or right road marking. A state in which vehicle M may deviate outside the driving lane is, for example, a state in which the reference position of vehicle M is located on either the left or right road marking, or a state in which vehicle M is moving toward a road marking and the time until vehicle M reaches the road marking is predicted to be below a threshold.

[0051] The driving lane may be defined not only by road markings but also by road boundaries (curbs, guardrails, road edges) recognized by the first recognition unit 110. For example, on a road without road markings, the driving lane (or road) may be defined based on information such as the direction of the curbs on either side of the vehicle M recognized by the first recognition unit 110, and the width between the curbs. The driving lane may also be defined based on a virtual line virtually generated by the driver assistance device 100. For example, the driver assistance device 100 generates a virtual line based on recognized road markings or road boundaries when road markings are interrupted or faded.

[0052] Furthermore, the processing in the lane departure detection unit 130 is performed when driving that requires maintaining the road, and does not need to be performed, for example, when the turn signal is activated. Also, for example, when the turn signal is activated, it does not need to be determined that the vehicle is in a lane departure state.

[0053] The control unit 140 controls all the components included in the driver assistance device 100 and the vehicle system 1. For example, the control unit 140 controls the steering of the vehicle M and controls the HMI 30 to provide information to the driver.

[0054] The lane-out-of-lane area determination unit 150 determines whether or not vehicle M can travel in the lane-out-of-lane area based on the information of the lane-out-of-lane area recognized by the first recognition unit 110. The lane-out-of-lane area is the area near the lane in which vehicle M is scheduled to travel, as recognized by the first recognition unit 110. Details of the processing of the control unit 140 and the lane-out-of-lane area determination unit 150 will be described later.

[0055] The driving force output device 200 outputs driving force (torque) to the drive wheels for the vehicle M to move. The driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, and a transmission, and an ECU that controls them. The ECU controls the above configuration according to information input from the driver assistance device 100 or from the operator 80.

[0056] The braking system 210 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to information input from the driver assistance device 100 or from the control element 80, so that brake torque corresponding to the braking operation is output to each wheel.

[0057] The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor, for example, applies force to a rack and pinion mechanism to change the direction of the steering wheels. The steering ECU drives the electric motor to change the direction of the steering wheels according to information input from the driver assistance device 100 or from the control element 80.

[0058] [overview] In this embodiment, the control unit 140 suppresses the operation of the lane departure prevention control depending on whether the driver has recognized an object to be avoided. If the control unit 140 determines, based on the recognition result of the first recognition unit 110, that the vehicle M has deviated from the lane or is in a lane departure state where it may deviate, it activates the lane departure prevention control, which is at least one of the following controls: a warning and steering assistance.

[0059] A warning is a warning that alerts the driver that vehicle M has deviated from the roadway or is about to deviate, or a warning that alerts the driver to move vehicle M closer to the center of the roadway, such as the driving lane. The warning may be, for example, an image display, an audible output, or a vibration of the steering wheel. The warning is not limited to these, and any method that draws the driver's attention is acceptable.

[0060] Steering assistance refers to controlling the steering to prevent the vehicle M from deviating from the road, or to controlling the steering to move the vehicle M closer to the center of the road. Steering assistance may also include control to move the vehicle M closer to the center of the driving lane and to move the vehicle M parallel to the road markings. The details of the process are described below.

[0061] In this specification, the term "center" does not necessarily mean "the exact center." In other words, the term "center" in this specification may include cases that can be considered substantially "center."

[0062] [Processing details] (Scene 1) The control unit 140 suppresses the operation of the lane departure suppression control when, with respect to the width direction of the vehicle M's lane, the first recognition unit 110 recognizes an object to be avoided that is on the opposite side of the direction of lane departure from which the vehicle M is deviating from the lane, and the second recognition unit 120 determines that the driver has visually confirmed the object to be avoided. For example, in the following process, the warning and steering assistance operate as lane departure suppression control.

[0063] Figure 2 shows an exemplary scenario in which the processing of this embodiment is applied. In Figure 2, vehicle M is traveling in the driving lane L1, and vehicle M1 is traveling in the oncoming lane L2. The driving lane L1 is a lane formed by road markings RL1 and RL2. The oncoming lane L2 is a lane formed by road markings RL2 and RL3. Road marking RL2 is the road marking in the center between the driving lane L1 and the oncoming lane L2.

[0064] At time T, vehicle M is traveling in the center of lane L1 due to the driver's operation. At time T+1, vehicle M approaches the road marking RL1, and the distance between vehicle M's reference position and road marking RL1 becomes a predetermined distance. At this time, the first recognition unit 110 recognizes vehicle M1, which is traveling in the oncoming lane L2, as an object to be avoided. Furthermore, the driver is visually aware of vehicle M1. In this case, the control unit 140 suppresses the operation of the lane departure prevention control. At time T+1, the control unit 140 does not move vehicle M to the center of lane L1, and at time T+2, the control unit 140 allows vehicle M to travel in a state of deviating from lane L1, and allows vehicle M to travel. Note that in Figure 2 and the figures described later, the position of vehicle M1 represents the position at any time between time T and time T+3.

[0065] As described above, the control unit 140 can suppress the operation of the vehicle departure prevention control when the object to be avoided is in front of the vehicle M and the driver of the vehicle M has the object in sight, thereby preventing the driver from feeling any discomfort. For example, when the driver intentionally tries to deviate from the road so that the vehicle M is traveling at a position away from the object to be avoided, the control unit 140 can achieve control of the vehicle M that matches the driver's intention.

[0066] Furthermore, suppression may mean activating the lane departure prevention control to prevent vehicle M from approaching the center of the driving lane L1, compared to when the lane departure prevention control is activated without suppression. Suppression may also mean, for example, vehicle M driving on the road marking RL1 or vehicle M driving near the road marking RL1.

[0067] The object to be avoided may be a vehicle, a pedestrian, or any other obstacle, as described above. The object to be avoided may be a stationary object or a moving object.

[0068] Figure 3 shows an example scenario in which the processing of this embodiment is applied. Compared to Figure 2, in Figure 3, a construction site AR is located on the opposing lane L2. In the case of Figure 3, the same processing as in Figure 2 is performed. The object to be avoided may be a construction site as shown in Figure 3.

[0069] The control unit 140 may suppress the operation of the lane departure prevention control by not activating at least one of the following: the warning and / or the steering assist. In the above example, the suppression of steering assist was described as suppressing the operation of the lane departure prevention control, but the warning may or may not be suppressed. Suppression of the lane departure prevention control may also involve suppressing the warning instead of (or in addition to) suppressing the steering assist. Suppressing the warning means not outputting the warning or reducing the level of the warning output compared to when it is not suppressed. Suppressing the warning may also mean delaying the timing of the warning output compared to when it is not suppressed.

[0070] The control unit 140 may suppress the operation of the lane departure prevention control by controlling at least one of a warning and steering assistance so that the vehicle M moves along the target road, which is a road or lane. The control unit 140 may suppress the operation of the lane departure prevention control by controlling at least one of a warning and steering assistance so that the vehicle M moves along the road boundary or road markings at a position other than the center of the target road on which the vehicle M is traveling. For example, in the example of Figure 2, the control unit 140 may cause the vehicle M to travel on or near the road markings RL1, or along the road markings RL1, without bringing the vehicle M close to the center of the driving lane L1 (without returning the vehicle M to the center).

[0071] (Scene 2) The control unit 140 does not suppress the operation of the lane departure suppression control if, with respect to the width direction of the vehicle M's lane, the first recognition unit 110 recognizes an object to be avoided that is on the opposite side of the direction of lane departure from which the vehicle M deviates from the lane, and the second recognition unit 120 determines, based on the recognition result, that the driver has not seen the object to be avoided.

[0072] Figure 4 shows an exemplary scenario in which the processing of this embodiment is applied. The differences from Figure 2 will be explained in detail. At time T, vehicle M is driving in the center of lane L1 due to the driver's operation. At time T+1, vehicle M approaches road marking RL1, and the distance between vehicle M and road marking RL1 becomes a predetermined distance. At this time, the first recognition unit 110 recognizes vehicle M1, which is driving in the oncoming lane L2, as an object to be avoided, and the driver does not see vehicle M1. In this case, the control unit 140 does not suppress the operation of lane departure suppression control. At time T+2, the control unit 140 does not allow vehicle M to drive in a state of deviating from lane L1, and moves vehicle M closer to the center of lane L1. As a result, at time T+3, the control unit 140 makes vehicle M drive in the center of lane L1 along lane L1.

[0073] As described above, the control unit 140 does not suppress the operation of the vehicle departure prevention control when the object to be avoided is in front of the vehicle M and the driver is not visually seeing the object to be avoided, thereby preventing the vehicle M from deviating from the road unintentionally. For example, when the driver is looking at something other than the object to be avoided and the vehicle M is about to deviate from the road unintentionally, control such as a warning or steering control is implemented to prevent the vehicle from deviating from the road.

[0074] (Flowchart 1) Figure 5 is a flowchart showing an example of the processing flow performed by the driver assistance device 100. The flowchart shown in Figure 5 is, for example, executed when the vehicle M is in motion.

[0075] First, the first recognition unit 110 recognizes objects around the vehicle M (step S100). These surrounding objects include, for example, road markings, curbs, and other objects used to recognize lanes, as well as objects to be avoided, which will be described later. Next, the deviation determination unit 130 determines whether the vehicle M is in a state of lane departure (step S102).

[0076] If it is determined that vehicle M is not in a lane departure state, the process returns to step S100. If it is determined that vehicle M is in a lane departure state, the control unit 140 determines whether the first recognition unit 110 has recognized an object to be avoided (step S104).

[0077] If the first recognition unit 110 recognizes the object to be avoided, the second recognition unit 120 recognizes the direction in which the driver is looking (step S106). Next, the control unit 140 determines whether or not the driver is looking at the object to be avoided (step S108).

[0078] If the control unit 140 determines that the driver has visually identified the object to be avoided, it suppresses the operation of the lane departure prevention control (step S110). If, in the process of step S104, the first recognition unit 110 does not recognize the object to be avoided, or if, in the process of step S108, it determines that the driver has not visually identified the object to be avoided, the control unit 140 activates the lane departure prevention control without suppressing its operation (step S112). This completes the process of this flowchart.

[0079] Note that some of the above processes may be omitted, and the order of the above processes may be changed. For example, the process in step S102 may be executed after the process in step S104, or the process in step S102 may be executed after the process in step S108. The same applies to Figures 8 and 11, which will be described later.

[0080] Through the above process, the control unit 140 can suppress causing discomfort to the driver by controlling the operation of the vehicle departure prevention control based on the presence or absence of an object to be avoided and whether or not the driver of the vehicle M has visually confirmed the object to be avoided.

[0081] (Scene 3) In Scene 3, control is performed taking into account the situation outside the lane. In the situation of Scene 3, the out-of-lane area determination unit 150 recognizes the situation outside the lane based on the recognition result of the first recognition unit 110 and determines whether or not vehicle M can travel in the out-of-lane area. For example, if there are no obstacles in the out-of-lane area that would obstruct vehicle M's movement (such as people or other traffic participants, curbs, guardrails, or other objects), the out-of-lane area determination unit 150 determines that vehicle M can travel in the out-of-lane area. For example, if there are obstacles in the out-of-lane area that would obstruct vehicle M's movement, the out-of-lane area determination unit 150 determines that vehicle M cannot travel in the out-of-lane area. If there are no road markings, the unit determines whether or not vehicle M can travel outside the road. The out-of-lane area is the area near the lane in which vehicle M is scheduled to travel, and is, for example, an area within a predetermined distance (for example, several meters) in the width direction from the lane.

[0082] If the lane departure prevention unit 150 determines that vehicle M can travel in the lane departure prevention area, the control unit 140 performs first suppression control, allowing vehicle M to temporarily travel in the lane departure prevention area. In the situation of scenario 3, if the lane departure prevention unit 150 determines that vehicle M cannot travel in the lane departure prevention area, the control unit 140 performs second suppression control, or does not suppress the operation of the lane departure prevention control, and does not allow vehicle M to temporarily travel in the lane departure prevention area.

[0083] The first suppression control is a control that suppresses the operation of the lane departure prevention control so as to allow driving in the lane-out area. The second suppression control is a control that suppresses the operation of the lane departure prevention control so as to not allow driving in the lane-out area. The second suppression control may be a control that suppresses the operation of the lane departure prevention control to a smaller degree than the first suppression control. The second suppression control may be, for example, as described above, a control unit 140 controlling at least one of a warning and steering assistance so that the vehicle M proceeds along the target road (without approaching the center).

[0084] The first suppression control or the second suppression control may, for example, involve the control unit 140 performing control to prevent the activation of at least one of the alarm and steering assistance, as described above. For example, in the first suppression control, both the alarm and steering assistance may be activated, while in the second suppression control, only one of the alarm or steering assistance may be activated.

[0085] Vehicle M is permitted to travel in the area outside the lane if the first recognition unit 110 recognizes that there are no obstacles in the area outside the lane. Obstacles may be dynamic obstacles such as pedestrians or bicycles, or static obstacles such as guardrails or gutters.

[0086] A situation in which vehicle M cannot travel in the area outside the lane is when the first recognition unit 110 recognizes the presence of an obstacle in the area outside the lane.

[0087] Figure 6 shows an exemplary scenario to which the processing of this embodiment is applied. The differences from Figure 2 will be explained in detail. The lane-out region AR1 is the region outside the road markings RL1 in front of the vehicle M in the direction of travel, as recognized by the first recognition unit 110.

[0088] At time T, vehicle M is traveling in the center of lane L1 due to the driver's operation. At time T+1, vehicle M approaches road marking RL1, and the distance between vehicle M and road marking RL1 becomes a predetermined distance. At this time, the first recognition unit 110 recognizes vehicle M1 traveling in the oncoming lane L2 as an object to be avoided, and the driver is visually aware of vehicle M1. Furthermore, the lane out-of-lane area determination unit 150 determines that vehicle M can travel in the lane out-of-lane area AR1. In this case, the control unit 140 performs first suppression control. At time T+1, the control unit 140 does not move vehicle M to the center of lane L1, and at time T+2, the control unit 140 allows vehicle M to travel in a state deviating from lane L1, and allows vehicle M to travel.

[0089] Figure 7 shows an exemplary scenario to which the processing of this embodiment is applied. The differences from Figure 6 will be explained in detail. In the example in Figure 7, a pedestrian P is located in the area AR1 outside the lane. Pedestrian P is an example of a dynamic obstacle.

[0090] At time T, vehicle M is driving in the center of lane L1 due to the driver's operation. At time T+1, vehicle M approaches road marking RL1, and the distance between vehicle M and road marking RL1 becomes a predetermined distance. At this time, the first recognition unit 110 recognizes vehicle M1, which is driving in the oncoming lane L2, as an object to be avoided, and the driver is visually aware of vehicle M1. Furthermore, the first recognition unit 110 recognizes that pedestrian P is in the out-of-lane area AR1, and the out-of-lane area determination unit 150 determines that vehicle M cannot drive in the out-of-lane area AR1. In this case, the control unit 140 performs second suppression control. At time T+2, the control unit 140 does not allow vehicle M to drive in a state of deviating from lane L1, and moves vehicle M closer to the center of lane L1. As a result, at time T+3, the control unit 140 causes vehicle M to travel in a way that prevents it from traveling in the lane-out area AR1. For example, the control unit 140 causes vehicle M to travel along the driving lane L1 at a position close to (or in) the center of the driving lane L1.

[0091] As described above, the control unit 140 performs first suppression control when the object to be avoided is in front of the vehicle M, the driver has visually confirmed the object to be avoided, and there are no obstacles in the out-of-lane area AR1. Furthermore, the control unit 140 performs second suppression control when the object to be avoided is in front of the vehicle M, the driver has visually confirmed the object to be avoided, and there are obstacles in the out-of-lane area AR1. Through these measures, the vehicle M can drive in accordance with the surrounding conditions. For example, even if the driver intentionally tries to deviate from the road so that the vehicle M is driving away from the object to be avoided, if there are obstacles in the out-of-lane area, control is implemented to prevent the vehicle M from deviating from the road.

[0092] (Flowchart 2) Figure 8 is a flowchart showing an example of the processing flow performed by the driver assistance device 100. The explanation will focus on the processing that differs from the flowchart in Figure 5.

[0093] The processing in steps S100 to S108 and step S112 is the same as the explanation in flowchart 1 of Figure 5 above.

[0094] In step S108, if it is recognized that the driver has sighted the object to be avoided, the lane-out-of-lane area determination unit 150 determines whether or not it is possible to drive in the lane-out-of-lane area (step S114).

[0095] If it is determined that the vehicle M can travel in the out-of-lane area, the control unit 140 allows the vehicle M to travel in the out-of-lane area (step S116). Next, the control unit 140 performs first suppression control (step S118). If it is determined that the vehicle M cannot travel in the out-of-lane area, the control unit 140 does not allow the vehicle M to travel in the out-of-lane area (step S120). Next, the control unit 140 performs second suppression control (step 122). This completes the processing of this flowchart. Note that in step S122, lane departure prevention control may be performed instead of second suppression control.

[0096] Through the above processing, the control unit 140 controls the operation of the lane departure prevention control by considering the presence or absence of an object to be avoided, whether the driver of vehicle M has visually confirmed the object to be avoided, and the conditions of the area outside the lane. This allows for control that is more appropriate to the surrounding conditions and reduces the likelihood of causing discomfort to the driver.

[0097] (Scene 4) In scenario 4, control is performed taking into account the width of the road or lane. If the first recognition unit 110 recognizes that vehicle M is traveling on a narrow road (see Figure 9 described later), or if the first recognition unit 110 recognizes that vehicle M has entered a narrowing area AR2, which is an area where the width decreases (see Figure 10 described later), the control unit 140 suppresses the operation of the lane departure prevention control.

[0098] Figure 9 shows an exemplary scenario in which the processing of this embodiment is applied. In Figure 9, vehicle M is traveling in a narrow road lane L1, and vehicle M1 is also traveling in the same lane L1 in the opposite direction to vehicle M. Lane L1 is a lane formed by road markings RL1 and RL2. Lane L1 is a narrow road with a width less than or equal to a predetermined value. A narrow road is, for example, a road whose width is narrower than the sum of the widths of vehicle M and vehicle M1, or a road whose width is narrower than the sum of the above plus a margin. Furthermore, a narrow road is a road in which, when vehicles M1 and M1 pass each other, one of the vehicles travels on the road markings or crosses over the road markings.

[0099] At time T, vehicle M is traveling in the center of lane L1 due to the driver's operation. At time T+1, vehicle M approaches road marking RL1, and the distance between vehicle M and road marking RL1 becomes a predetermined distance. At this time, the first recognition unit 110 recognizes vehicle M1 traveling in the oncoming lane L2 as an object to be avoided. Furthermore, the first recognition unit 110 recognizes that lane L1 is a narrow road with a width less than a predetermined value. In this case, the control unit 140 suppresses the operation of lane departure suppression control. At time T+1, the control unit 140 does not move vehicle M to the center of lane L1, and at time T+2, the control unit 140 allows vehicle M to travel in a state of deviating from lane L1, and allows vehicle M to travel.

[0100] As described above, the control unit 140 suppresses the operation of the vehicle departure prevention control when the object to be avoided is in front of the vehicle M, the driver has visually confirmed the object to be avoided, and the vehicle M is traveling on a narrow road, thereby enabling the vehicle M to travel in accordance with the surrounding conditions. For example, if the driver of the vehicle M has visually confirmed a narrow road in addition to the object to be avoided, and the driver intends to deviate from the road, the control unit 140 enables the vehicle M to be controlled in a manner that matches the driver's intention.

[0101] Figure 10 shows an exemplary scenario in which the processing of this embodiment is applied. In Figure 10, vehicle M is traveling in lane L1, and vehicle M1 is also traveling in lane L1 in the opposite direction to vehicle M. Lane L1 is a lane formed by road markings RL1 and RL2. Lane L1 is a road that includes a width reduction area AR2, which is a region in front of vehicle M in the direction of travel where the width of the road narrows (the width decreases). The width reduction area AR2 is, for example, a region in which the width of the road decreases to a road that is narrower than the sum of the widths of vehicle M and vehicle M1, or a region in which the width of the road decreases to a road that is narrower than the sum of the above plus a margin. The width reduction area is also a region in which the width of the road decreases to a road that, when vehicles M1 and M1 pass each other, one of the vehicles travels on the road markings or crosses over the road markings.

[0102] At time T, vehicle M is traveling in the center of lane L1 due to the driver's operation. At time T+1, vehicle M approaches road marking RL1, and the distance between vehicle M and road marking RL1 becomes a predetermined distance. At this time, the first recognition unit 110 recognizes vehicle M1 traveling in the oncoming lane L2 as an object to be avoided. Furthermore, the first recognition unit 110 recognizes that vehicle M is entering a narrowing area AR2. In this case, the control unit 140 suppresses the operation of the lane departure suppression control. At time T+1, the control unit 140 does not move vehicle M to the center of lane L1, and at time T+2, the control unit 140 allows vehicle M to travel in a state of deviating from lane L1, and allows vehicle M to travel.

[0103] As described above, when the object to be avoided is in front of the vehicle M, the driver sees the object to be avoided, and the vehicle M enters the road narrowing area AR2, the control unit 140 suppresses the operation of the vehicle departure prevention control, thereby enabling the vehicle M to drive in accordance with the surrounding conditions. For example, when the driver of the vehicle M sees the road narrowing area in addition to the object to be avoided, and the driver intends to deviate from the road, the control unit 140 enables the vehicle M to be controlled in a manner that matches the driver's intention.

[0104] In the above, the operation of the lane departure prevention control was suppressed to allow the vehicle M to travel while deviating from the driving lane L1. However, for example, the operation of the lane departure prevention control may be suppressed by the control unit 140 controlling steering assistance so that the vehicle M is positioned at a location where the distance between the vehicle M and the object to be avoided in the width direction of the driving lane is greater than or equal to a first threshold. The distance between the vehicle M and the object to be avoided in the width direction of the driving lane may be determined, for example, based on the position of the vehicle M recognized by the first recognition unit 110 and the position of the object to be avoided.

[0105] Furthermore, for example, the operation of the lane departure prevention control may be suppressed by controlling the steering assist so that the vehicle M is positioned such that the time it takes for the vehicle M to reach the object to be avoided is equal to or greater than a second threshold, relative to the width direction of the driving lane. The time it takes for the vehicle M to reach the object to be avoided may be determined based on the distance between the vehicle M and the object to be avoided, as recognized by the first recognition unit 110, and the speed of the vehicle M.

[0106] The first threshold and the second threshold may be determined by the driver assistance device 100, or they may be input and determined by the occupant of the vehicle M via the HMI 30. However, the first threshold and the second threshold are not limited to these, and may be any numerical values ​​determined based on the position and speed of the vehicle M and the object to be avoided. For example, the driver may be able to adjust the degree of suppression of the lane departure prevention control. For example, when the degree of suppression is adjusted to "strong", the control unit 140 controls the operation of the lane departure prevention control so that the vehicle M tends to travel away from the center of the driving lane due to the driver's operation, compared to when the degree of suppression is adjusted to "weak".

[0107] The flowchart corresponding to the above scenario may be the flowchart in Figure 5 or the flowchart in Figure 8. In the flowchart in Figure 5 or Figure 8, when vehicle M approaches or is traveling through a narrow road or a road with a reduced width, the process in step S102 determines that the vehicle is in a road departure state, and the processes from step S104 onwards are executed.

[0108] Furthermore, for example, the control unit 140 may suppress the operation of the lane departure prevention control by not activating at least one of the alarm and steering assistance. If the operation of the lane departure prevention control is an alarm and steering assistance, suppressing the operation of the lane departure prevention control may be done by suppressing the operation of either the alarm or the steering assistance. For example, in the examples of Figures 9 and 10, the control unit 140 may suppress the operation of the lane departure prevention control by activating the alarm without activating the steering assistance (for example, by activating only the alarm).

[0109] The above example describes a case where the driver has visual contact with the object to be avoided. However, even if the driver does not have visual contact with the object to be avoided, if the object is located in front of the vehicle M and the first recognition unit 110 recognizes that the vehicle M is traveling on a narrow road, or if the first recognition unit 110 recognizes that the vehicle M is entering a narrowing area AR2, which is an area where the width decreases, the control unit 140 may suppress the operation of the road departure prevention control. An example of the processing flow performed by the driver assistance device 100 in this case is shown in Figure 11.

[0110] (Flowchart 3) Figure 11 is a flowchart showing an example of the processing flow performed by the driver assistance device 100. The explanation will focus on the processing that differs from the flowcharts in Figures 5 and 8.

[0111] The processes in steps S100 to S108 and S112 are the same as those described in flowchart 1 in Figure 5 above. Furthermore, the processes in steps S114 to S122 are the same as those described in flowchart 2 in Figure 8 above.

[0112] In step S108, if it is determined that the driver has not visually identified the object to be avoided, the control unit 140 determines whether the vehicle M is traveling on a narrow road or whether the vehicle M has entered a road narrowing area.

[0113] If it is determined that vehicle M is traveling on a narrow road, or that vehicle M is entering a road narrowing area, the control unit 140 suppresses the operation of the road departure prevention control (step S126). If vehicle M is not traveling on a narrow road, or if vehicle M is not entering a road narrowing area, the control unit 140 does not suppress the operation of the road departure prevention control (step S112). This completes the processing of this flowchart.

[0114] Note that in the flowchart of Figure 11, steps S114 to S122 may be omitted. In this case, if the control unit 140 determines in step S108 that the driver has not seen the object to be avoided, it may suppress the operation of the lane departure prevention control as in step S110 of Figure 5.

[0115] As described above, even when the object to be avoided is in front of the vehicle M and the driver does not see the object to be avoided, the control unit 140 suppresses the operation of the vehicle departure prevention control when the vehicle M is traveling on a narrow road or when the vehicle M enters a road width reduction area AR2, thereby enabling the vehicle M to travel in accordance with the surrounding conditions. For example, when the driver of the vehicle M sees the road boundary of a narrow road or road width reduction area and the driver intends to deviate from the road, the control unit 140 enables the vehicle M to be controlled in accordance with the driver's intention.

[0116] According to the embodiments described above, even if the vehicle is about to deviate from its lane, the lane departure prevention control is suppressed if the driver has visual contact with an object to avoid. This enables vehicle control that matches the driver's intentions.

[0117] The embodiments described above can be expressed as follows. A memory device that stores the program, Equipped with a hardware processor, The hardware processor executes the program stored in the memory device, Based on the output of the first detection device that detects the surrounding conditions of the moving object, the surrounding conditions of the moving object are recognized. Based on the output of the second detection device that detects information about the driver of the moving object, the driver's viewing direction is recognized. If, based on the recognized surrounding conditions, it is determined that the moving object has deviated from the road or is in a state of road departure where it may deviate, then a road departure suppression control, which is at least one of the following controls, will be activated: warning and steering assistance. With respect to the width direction of the track of the moving body, if the moving body recognizes an object to be avoided that is located on the opposite side of the direction of track departure from the track, and if it is determined, based on the driver's information, that the driver has visually identified the object to be avoided, then the operation of the track departure suppression control is suppressed. A mobile control device configured in such a way.

[0118] Although embodiments for carrying out the present invention have been described above using examples, the present invention is not limited in any way to these embodiments, and various modifications and substitutions can be made without departing from the spirit of the present invention. [Explanation of symbols]

[0119] 1. Vehicle System 10 Cameras 12 Radar equipment 14 LIDAR 18. In-car camera 110 1st recognition part 120 Second recognition part 130 Deviation detection unit 140 Control Unit 150 Lane Out-of-Boundary Area Determination Unit

Claims

1. A first recognition unit recognizes objects around a moving object based on the surrounding conditions of the moving object detected by the first detection device, A second recognition unit recognizes the driver's viewing direction based on information about the driver of the moving object detected by the second detection device, Based on the recognition results of the first recognition unit, if it is determined that the moving body has deviated from the road or is in a state of road departure where it may deviate, the control unit activates road departure suppression control, which is at least one of the controls of a warning and steering assistance, With respect to the width direction of the track of the moving body, if the first recognition unit recognizes an object to be avoided that is on the opposite side of the direction of track departure from which the moving body deviates from the track, and the second recognition unit determines that the driver has visually confirmed the object to be avoided, the control unit suppresses the operation of the track departure suppression control. A mobile control device equipped with the following features.

2. The aforementioned lane departure condition is when the moving body deviates from the lane on which it is traveling, or is likely to deviate from the lane on which it is traveling, or when the moving body deviates from the lane on which it is traveling, or is likely to deviate from the lane on which it is traveling. The mobile device control device according to claim 1.

3. The control unit suppresses the operation of the lane departure prevention control by not activating at least one of the alarm and the steering assist. The mobile device control device according to claim 2.

4. The control unit suppresses the operation of the road departure prevention control by controlling at least one of the warning and the steering assist so that the moving body moves along the target road, which is the road or lane. The mobile device control device according to claim 2.

5. The first recognition unit recognizes the conditions of the out-of-lane area, which is the area outside the road or lane surrounding the moving body. The system further includes a determination unit that determines whether or not the moving body can travel in the area outside the lane based on the conditions of the area outside the lane. If the determination unit determines that the moving body can travel in the area outside the lane, the control unit performs a first suppression control to suppress the operation of the lane departure prevention control, thereby allowing the moving body to temporarily travel in the area outside the lane. If the determination unit determines that the moving body cannot travel in the area outside the lane, the control unit performs a second suppression control that suppresses the operation of the lane departure prevention control to a lesser degree than the first suppression control, or does not suppress the operation of the lane departure prevention control and does not allow the moving body to travel in the area outside the lane temporarily. The mobile device control device according to claim 3 or 4.

6. The determination unit determines that if a dynamic obstacle exists in the area outside the lane, the moving body cannot travel in the area outside the lane. The mobile device control device according to claim 5.

7. The determination unit determines that if a static obstacle exists in the area outside the lane, the moving body cannot travel in the area outside the lane. The mobile device control device according to claim 5.

8. When the first recognition unit recognizes the object to be avoided, and the second recognition unit recognizes that the driver has not recognized the object to be avoided, the control unit performs the lane departure prevention control. The mobile device control device according to claim 1.

9. The control unit suppresses the road departure prevention control when the moving body is traveling on a narrow road or when the moving body enters a road width reduction area, which is an area where the road width decreases. The mobile device control device according to claim 8.

10. The control unit controls the steering assistance so that the moving body is positioned such that the distance between the moving body and the object to be avoided is greater than or equal to a first threshold in the width direction of the moving body, or so that the time it takes for the moving body to reach the object to be avoided is greater than or equal to a second threshold in the width direction of the moving body. The mobile device control device according to claim 9.

11. The control unit suppresses the operation of the lane departure prevention control by activating the warning without activating the steering assistance to the driver. The mobile device control device according to claim 9.

12. Computers Based on the surrounding conditions of the moving object detected by the first detection device, objects in the vicinity of the moving object are recognized. Based on the information about the driver of the moving object detected by the second detection device, the direction in which the driver is looking is recognized. If, based on the surrounding conditions of the moving body, it is determined that the moving body has deviated from the road or is in a state of road departure where it may deviate, then a road departure suppression control, which is at least one of the following controls: a warning and steering assistance, is activated. With respect to the width direction of the moving body, if an object to be avoided is recognized on the opposite side of the direction of the moving body's departure from the road, and if it is determined that the driver has visually identified the object to be avoided based on the information regarding the driver of the moving body, the operation of the road departure prevention control is suppressed. A method for controlling a mobile object.

13. On the computer, Based on the surrounding conditions of the moving object detected by the first detection device, the device recognizes objects around the moving object. Based on the information about the driver of the moving object detected by the second detection device, the direction in which the driver is looking is recognized. If, based on the surrounding conditions of the moving body, it is determined that the moving body has deviated from the road or is in a state of road departure where it may deviate, then a road departure suppression control, which is at least one of the following controls: a warning and steering assistance, is activated. With respect to the width direction of the moving body, if an object to be avoided is recognized on the opposite side of the direction of the moving body's departure from the road, and if it is determined that the driver is visually viewing the object to be avoided based on the information about the driver of the moving body, the operation of the road departure prevention control is suppressed. program.

Images