Vehicle control device, vehicle control method, and program

Abstract

To implement more appropriate driving control according to the driving conditions. [Solution] The vehicle control device of the embodiment includes: an acquisition unit that acquires the surrounding conditions of the vehicle based on the output of at least a detection device mounted on the vehicle; a lane marking recognition unit that recognizes lane markings present on the road surface of the vehicle's travel path based on the acquisition results; a structure recognition unit that recognizes structures present on the side of the vehicle's travel path based on the acquisition results; an extraction unit that derives the degree of curvature of the travel path on which the vehicle travels based on the recognition results of the lane marking recognition unit and the structure recognition unit; and a driving control unit that controls the vehicle based on the degree of curvature of the travel path derived by the extraction unit. The extraction unit derives a first degree of curvature based on lane markings for the travel path within a range less than a predetermined distance from the vehicle, and a second degree of curvature based on structures for the range greater than or equal to a predetermined distance from the vehicle. The driving control unit controls the vehicle based on the first degree of curvature and the second degree of curvature.

Description

Technical Field

[0001] The present invention relates to a vehicle control device, a vehicle control method, and a program.

Background Art

[0002] In recent years, efforts to provide access to sustainable transportation systems that also consider vulnerable people among traffic participants have become active. Toward achieving this goal, efforts are focused on research and development to further improve traffic safety and convenience through research and development related to autonomous driving technology. In relation to this, in recent years, there has been known a technology that includes: a linear target recognition unit that recognizes, in an image captured by an imaging unit, a linear target that exists at a height different from that of a road and extends along the road when viewed from above; a driving control unit that controls traveling of the vehicle based on the position of the linear target in the image; and a curve estimation unit that estimates a curve existing ahead of the vehicle and its curvature based on the extension mode of the linear target recognized by the linear target recognition unit, wherein the technology performs feedback control to keep the inclination of the linear target in the image constant with respect to the steering angle of the vehicle, and when the vehicle reaches the estimated start point of the curve and a surrounding situation recognition unit does not recognize a road marking at that time, controls the steering angle of the vehicle based on the curvature of the curve estimated by the curve estimation unit (see, for example, Patent Document 1).

Prior Art Literature

Patent Literature

[0003]

Patent Document 1

Summary of the Invention

Problem to be Solved by the Invention

[0004] Incidentally, with conventional autonomous driving technology, there was a possibility that the degree of curvature of the road could not be properly recognized depending on the driving conditions, such as the shape of the road and the accuracy of camera recognition. Therefore, there was a problem that appropriate driving control could not be performed depending on the driving conditions.

[0005] One of the objectives of this application is to provide a vehicle control device, a vehicle control method, and a program that can perform more appropriate driving control according to driving conditions, in order to solve the above-mentioned problems. Ultimately, this will contribute to the development of a sustainable transportation system. [Means for solving the problem]

[0006] The vehicle control device, vehicle control method, and program according to this invention employ the following configuration. (1) A vehicle control device according to one aspect of the present invention comprises: an acquisition unit that acquires the surrounding conditions of the vehicle based on the output of at least a detection device mounted on the vehicle; a lane marking recognition unit that recognizes lane markings present on the road surface of the vehicle's travel path based on the acquisition results of the acquisition unit; a structure recognition unit that recognizes structures present on the side of the vehicle's travel path based on the acquisition results of the acquisition unit; an extraction unit that derives the degree of curvature of the travel path on which the vehicle travels based on the recognition results of the lane marking recognition unit and the structure recognition unit; and a driving control unit that controls the vehicle based on the degree of curvature of the travel path derived by the extraction unit, wherein the extraction unit derives a first degree of curvature based on the lane markings for the travel path less than a predetermined distance from the vehicle, and a second degree of curvature based on the structures for the travel path greater than or equal to the predetermined distance from the vehicle, and the driving control unit controls the vehicle based on the first degree of curvature and the second degree of curvature.

[0007] (2): In the embodiment of (1) above, the detection device includes a camera or LIDAR (Light Detection and Ranging), and the predetermined distance is the distance from the vehicle to a point where the lane markings can no longer be recognized from the image captured by the camera or the detection result of the LIDAR due to the influence of the shape of the road in the direction of travel of the vehicle.

[0008] (3) In the embodiment of (1) above, the derivation unit derives the first degree of curvature based on the road markings recognized by the road marking recognition unit within a range less than the predetermined distance, and the second degree of curvature based on the structures recognized by the structure recognition unit within a range greater than or equal to the predetermined distance.

[0009] (4): In the embodiment of (3) above, the derivation unit derives the second degree of curvature based on the structure located on the outside of the curve of the road when the road is a curved road.

[0010] (5) In the embodiment of (1) above, the driving control unit controls at least one of the vehicle's speed and steering based on the first degree of curvature and the second degree of curvature.

[0011] (6) In the embodiment of (5) above, the driving control unit performs a first braking based on the second degree of curvature when a curved road exists in the direction of travel of the vehicle, and performs a second braking based on the first degree of curvature after the first braking.

[0012] (7): In the embodiment of (6) above, the first braking is suppression of the acceleration of the vehicle, and the second braking is deceleration of the vehicle.

[0013] (8) In the embodiment of (6) above, the first brake and the second brake are deceleration of the vehicle, and the first brake is less deceleration than the second brake.

[0014] (9) In the embodiment of (3) above, the driving control unit controls the steering of the vehicle based on the first degree of turning and controls the speed of the vehicle based on the second degree of turning.

[0015] (10): In the embodiment of (1) above, the derivation unit corrects the degree of curvature of the first degree of curvature and the second degree of curvature based on the other degree of curvature.

[0016] (11): Another aspect of the present invention is a vehicle control method in which a computer acquires the surrounding conditions of the vehicle based on the output of at least a detection device mounted on the vehicle, recognizes lane markings on the road surface of the vehicle's travel path based on the acquired results, recognizes structures on the side of the vehicle's travel path based on the acquired results, derives the degree of curvature of the travel path on which the vehicle travels based on the recognition results of the lane markings and the structures, controls the vehicle based on the derived degree of curvature of the travel path, derives a first degree of curvature based on the lane markings for the travel path less than a predetermined distance from the vehicle, derives a second degree of curvature based on the structures for the travel path greater than or equal to the predetermined distance from the vehicle, and controls the vehicle based on the first degree of curvature and the second degree of curvature.

[0017] (12): A program according to another aspect of the present invention causes a computer to: acquire the surrounding conditions of a vehicle based at least on an output of a detection device mounted on the vehicle; recognize a division line existing on a road surface of the travel path of the vehicle based on the acquired result; recognize a structure existing on a side of the travel path of the vehicle based on the acquired result; derive a curvature degree of the travel path on which the vehicle travels based on recognition results of the division line and the structure; cause the vehicle to be controlled based on the derived curvature degree of the travel path; derive a first curvature degree based on the division line for a range of the travel path that is less than a predetermined distance from the vehicle; derive a second curvature degree based on the structure for a range of the travel path that is equal to or greater than the predetermined distance from the vehicle; and cause the vehicle to be controlled based on the first curvature degree and the second curvature degree. Effects of the Invention

[0018] According to the aspects of (1) to (12) above, more appropriate driving control can be executed according to driving conditions. Brief Description of the Drawings

[0019] [Figure 1] It is a configuration diagram of a vehicle system 1 including a vehicle control device according to an embodiment. [Figure 2] It is a functional configuration diagram of a first control unit 120 and a second control unit 160. [Figure 3] It is a diagram showing an example of a traveling condition of a vehicle M in an embodiment. [Figure 4] It is a diagram for explaining an example of braking control of the vehicle M based on traveling conditions. [Figure 5] It is a diagram for explaining a relationship between a control target and conditions when deriving a curvature degree. [Figure 6] It is a diagram for explaining a state of change when switching between a division line curvature and a structure curvature. [Figure 7] It is a flowchart showing an example of a flow of driving control processing in an embodiment. [Figure 8] It is a flowchart showing another example of the flow of driving control processing in an embodiment. Mode for Carrying Out the Invention

[0020] Hereinafter, embodiments of a vehicle control device, a vehicle control method, and a program according to the present invention will be described with reference to the drawings. Hereinafter, an embodiment in which the vehicle control device is applied to an autonomous driving vehicle will be described. Autonomous driving refers to, for example, automatically controlling at least one of the speed and steering of a vehicle to execute driving control. The driving control described above may include various types of driving control such as ACC (Adaptive Cruise Control System), LKAS (Lane Keeping Assistance System), ALC (Automated Lane Change), TJP (Traffic Jam Pilot), and CMBS (Collision Mitigation Brake System), for example. Further, the autonomous driving vehicle may perform driving control by manual operation of a user of the vehicle (e.g., an occupant), that is, so-called manual driving. Note that the vehicle control device in the embodiment may be applied to moving bodies other than vehicles, for example, ships movable on the ground such as hovercraft, flying vehicles capable of traveling on roads, and standing riding vehicles having a power unit.

[0021] [Overall Configuration] FIG. 1 is a configuration diagram of a vehicle system 1 including a vehicle control device according to an embodiment. A vehicle on which the vehicle system 1 is mounted (hereinafter referred to as vehicle M) is, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle or a micromobility, and the drive source thereof is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. The electric motor operates using electric power generated by a generator connected to the internal combustion engine, or electric power discharged from a battery (storage battery) such as a secondary battery or a fuel cell.

[0022] Vehicle system 1 includes, for example, a camera 10, a radar device 12, a LiDAR (Light Detection and Ranging) 14, a communication device 20, an HMI (Human Machine Interface) 30, a vehicle sensor 40, a navigation device 50, an MPU (Map Positioning Unit) 60, a driver control device 80, an automatic driving control device 100, a driving force output device 200, a brake device 210, and a steering device 220. These devices and equipment are connected to each other by multiplex communication lines such as CAN (Controller Area Network) communication lines, serial communication lines, wireless communication networks, etc. Note that the configuration shown in Figure 1 is merely an example, and some of the configuration may be omitted, or other configurations may be added. Camera 10, radar device 12, and LiDAR 14 are examples of "detection devices DD". HMI 30 is an example of an "output device". Automatic driving control device 100 is an example of a "vehicle control device".

[0023] Camera 10 is a digital camera that uses a solid-state image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor). Camera 10 is mounted at any location on the vehicle M on which the vehicle system 1 is installed. When imaging the front, camera 10 is mounted on the top of the front windshield, behind the rearview mirror, or on the front of the vehicle body. When imaging the rear, camera 10 is mounted on the top of the rear windshield or on the tailgate. When imaging the side, camera 10 is mounted on the door mirror or the like. Camera 10 periodically and repeatedly images the area around vehicle M. Camera 10 may also be a stereo camera.

[0024] The radar device 12 emits radio waves such as millimeter waves around the vehicle M and detects radio waves reflected by surrounding objects (reflected waves) to determine at least the position (distance and direction) of an object. The radar device 12 can be mounted at any location on the vehicle M. The radar device 12 may also detect the position and velocity of an object using the FM-CW (Frequency Modulated Continuous Wave) method.

[0025] The LIDAR 14 illuminates the area around the vehicle M with light and measures the scattered light. The LIDAR 14 detects the distance to the target based on the time from emission to reception. The emitted light is, for example, pulsed laser light. The LIDAR 14 can be mounted at any location on the vehicle M.

[0026] The communication device 20 communicates with other vehicles in the vicinity of vehicle M, terminal devices of users using vehicle M, or various server devices, for example, by utilizing networks such as cellular networks, Wi-Fi networks, Bluetooth®, DSRC (Dedicated Short Range Communication), LAN (Local Area Network), WAN (Wide Area Network), and the Internet.

[0027] The HMI30 outputs various information to the occupants of vehicle M (including the driver) and accepts input operations from the occupants. The HMI30 includes, for example, a display unit and a speaker. The display unit is, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) display device. The display unit displays various images (including video) in the embodiment. The display unit may be configured integrally with the input unit as a touch panel. The speaker outputs predetermined sounds (e.g., alarm sounds or message sounds). In addition to the display unit and speaker, the HMI30 may also include (or replace) a microphone, buzzer, touch panel, switches, keys, etc. Switches include switches that execute or terminate predetermined driving controls (e.g., ACC or LKAS) that can be executed by the driving control unit described later, and switches that approve (permit) or reject driving control recommendations (suggestions) from the system (vehicle system 1). Switches may also include switches for operating the turn signals (turn signal switches).

[0028] The vehicle sensor 40 includes a vehicle speed sensor for detecting the speed of the vehicle M, an acceleration sensor for detecting acceleration, a yaw rate sensor for detecting yaw rate (for example, the angular velocity of rotation around the vertical axis passing through the center of gravity of the vehicle M), and an orientation sensor for detecting the orientation of the vehicle M. The vehicle sensor 40 may also be provided with a position sensor for detecting the position of the vehicle. The position sensor is, for example, a sensor that acquires position information (longitude and latitude information) from a GPS (Global Positioning System) device. Alternatively, the position sensor may be a sensor that acquires position information using a GNSS (Global Navigation Satellite System) receiver 51 of the navigation device 50. The vehicle sensor 40 may derive the speed of the vehicle M from the difference (i.e., distance) of position information at a predetermined time from the position sensor. The results detected by the vehicle sensor 40 are output to the automatic driving control device 100.

[0029] The navigation device 50 includes, for example, a GNSS receiver 51, a navigation HMI 52, and a route determination unit 53. The navigation device 50 stores first map information 54 in a storage device such as an HDD (Hard Disk Drive) or flash memory. The GNSS receiver 51 determines the position of the vehicle M based on signals received from GNSS satellites. The position of the vehicle M may be determined or supplemented by an INS (Inertial Navigation System) that utilizes the output of the vehicle sensor 40. The navigation HMI 52 includes a display device, speaker, touch panel, keys, etc. The GNSS receiver 51 may be provided on the vehicle sensor 40. The navigation HMI 52 may be partially or completely shared with the HMI 30 described above. The route determination unit 53 determines, for example, a route (hereinafter referred to as a route on the map) from the position of the vehicle M determined by the GNSS receiver 51 (or any input position) to the destination input by the occupant using the navigation HMI 52, by referring to the first map information 54. The first map information 54 is information in which the shape of a road is represented by links indicating roads (an example of a travel route) and nodes connected by those links. The first map information 54 may also include POI (Point of Interest) information, etc. The route on the map is output to the MPU 60. The navigation device 50 may perform route guidance using the navigation HMI 52 based on the route on the map. The navigation device 50 may transmit its current location and destination to the navigation server via the communication device 20 and obtain a route equivalent to the route on the map from the navigation server. The navigation device 50 outputs the determined route on the map to the MPU 60.

[0030] The MPU 60 includes, for example, a recommended lane determination unit 61 and stores second map information 62 in a storage device such as an HDD or flash memory. The recommended lane determination unit 61 divides the map route provided by the navigation device 50 into multiple blocks (for example, every 100m with respect to the vehicle's direction of travel) and determines a recommended lane for each block by referring to the second map information 62. The recommended lane determination unit 61 makes decisions such as which lane from the left the vehicle should travel in. If there is a branching point in the map route, the recommended lane determination unit 61 determines a recommended lane so that the vehicle M can travel along a reasonable route to proceed to the branching point.

[0031] The second map information 62 is map information with higher accuracy than the first map information 54. The second map information 62 includes, for example, the number of lanes, the type and shape of road markings (hereinafter referred to as markings), information on the center of the lanes, or information on road boundaries. Physical boundaries include, for example, walls (e.g., sound barriers, tunnel walls, etc.), guardrails, fences, curbs, median strips, etc. The second map information 62 may also include road shape information, traffic regulation information, address information (address and postal code), facility information, parking information, telephone number information, etc. Road shape information includes, for example, lane width (width), gradient, branching, merging, intersections, road curvature (may be rephrased as radius of curvature; the same applies hereinafter), etc. The second map information 62 may be updated as needed by the communication device 20 communicating with an external device. The first map information 54 and the second map information 62 may be provided together as map information. The map information may also be stored in the storage unit 190.

[0032] The driver control unit 80 includes, for example, a steering wheel, an accelerator pedal, and a brake pedal. The driver control unit 80 may also include a shift lever, a modified steering wheel, a joystick, or other controls. Each control of the driver control unit 80 is equipped with an operation detection unit that detects, for example, the amount of operation performed by the occupant on the control unit or whether or not an operation has been performed. The operation detection unit detects, for example, the steering angle and steering torque of the steering wheel, the amount of depression of the accelerator pedal and brake pedal, etc. The operation detection unit then outputs the detection result to the automatic driving control device 100, or to one or both of the driving force output device 200, the brake device 210, and the steering device 220.

[0033] The automatic driving control device 100 performs various driving controls belonging to automatic driving on the vehicle M. The automatic driving control device 100 includes, for example, a first control unit 120, a second control unit 160, an HMI control unit 180, and a storage unit 190. The first control unit 120, the second control unit 160, and the HMI control unit 180 are each realized by a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Furthermore, some or all of these components may be realized by hardware (including circuitry) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit), and SOC (System On Chip), or by the cooperation of software and hardware. The above-mentioned program may be stored in advance in a storage device (a storage device equipped with a non-transient storage medium) such as the HDD or flash memory of the automatic driving control device 100, or it may be stored in a removable storage medium such as a DVD, CD-ROM, or memory card, and installed in the storage device of the automatic driving control device 100 when the storage medium (non-transient storage medium) is inserted into a drive device or card slot.

[0034] The storage unit 190 may be implemented using the various storage devices described above, or an EEPROM (Electrically Erasable Programmable Read Only Memory), ROM (Read Only Memory), or RAM (Random Access Memory), etc. The storage unit 190 may store, for example, various information and programs as described in the embodiment. The storage unit 190 may also store map information (for example, first map information 54 and second map information 62).

[0035] Figure 2 is a functional configuration diagram of the first control unit 120 and the second control unit 160. The first control unit 120 includes, for example, a recognition unit 130 and an action plan generation unit 140. The action plan generation unit 140 and the second control unit 160 are examples of a "driving control unit". The first control unit 120 implements, for example, functions by AI (Artificial Intelligence) and functions by a pre-given model in parallel. For example, the function of "recognizing intersections" may be implemented by simultaneously performing intersection recognition by deep learning, etc., and recognition based on pre-given conditions (such as pattern-matchable signals and road markings), and then scoring both and comprehensively evaluating them. This ensures the reliability of autonomous driving. The first control unit 120 also executes control related to the autonomous driving of the vehicle M based on instructions from, for example, the MPU 60 and the HMI control unit 180.

[0036] The recognition unit 130 recognizes the surrounding conditions of the vehicle M based on the recognition results of the detection device DD (information input from the camera 10, radar device 12, and LIDAR 14), etc. The recognition unit 130 includes, for example, an acquisition unit 132, a lane marking recognition unit 134, a structure recognition unit 136, and an output unit 138. Details of these functions will be described later.

[0037] The action plan generation unit 140 generates an action plan for driving vehicle M by autonomous driving based on the recognition results of the recognition unit 130, etc. For example, the action plan generation unit 140, in principle, drives in the recommended lane determined by the recommended lane determination unit 61, and further generates a target trajectory (target driving path) that vehicle M will automatically (without driver operation) travel in the future, based on the recognition results of the recognition unit 130, etc., so as to respond to the surrounding conditions of vehicle M. The target trajectory includes, for example, a speed element. For example, the target trajectory is expressed as a sequence of points (trajectory points) that vehicle M should reach. The trajectory points are points that vehicle M should reach at predetermined driving distances (e.g., a few meters) along the road, and separately, target speed and target acceleration at predetermined sampling times (e.g., a few tenths of a second) are generated as part of the target trajectory. Alternatively, the trajectory points may be the positions that vehicle M should reach at the sampling time for each predetermined sampling time. In this case, the information on target speed and target acceleration is expressed by the intervals between trajectory points.

[0038] The action plan generation unit 140 may set automated driving events when generating a target trajectory. These events include, for example, a constant speed driving event that drives vehicle M in the same lane at a constant speed, a follow driving event corresponding to ACC that causes vehicle M to follow the nearest vehicle that is within a predetermined distance (for example, within 100 [m]) in front of vehicle M, a lane keeping driving event corresponding to LKAS that causes vehicle M to drive in the center of the driving lane, a lane change event corresponding to ALC that causes vehicle M to change lanes from its own lane to an adjacent lane, a branching event that branches vehicle M to the destination lane at a road branching point, a merging event that causes vehicle M to merge onto the main road at a merging point, and a takeover event to end automated driving and switch to manual driving. Furthermore, events may include, for example, an overtaking event in which vehicle M temporarily changes lanes to an adjacent lane, overtakes a preceding vehicle in the adjacent lane, and then changes lanes back to its original lane, and an avoidance event in which vehicle M is instructed to increase its speed (brake) and steer to avoid an obstacle in front of it.

[0039] Furthermore, the action plan generation unit 140 may, for example, change an event already determined for the current section to another event, or set a new event for the current section, depending on the surrounding conditions of vehicle M recognized while vehicle M is in motion. Also, the action plan generation unit 140 may, in response to the occupant's operation on the HMI 30, change an event already set for the current section to another event, or set a new event for the current section. The action plan generation unit 140 generates a target trajectory according to the set event. The action plan generation unit 140 includes, for example, a driving control unit 142. Details of the functions of the driving control unit 142 will be described later.

[0040] The second control unit 160 controls the driving force output device 200, the braking device 210, and the steering device 220 so that the vehicle M passes through the target trajectory generated by the action plan generation unit 140 at the scheduled time.

[0041] The second control unit 160 includes, for example, a target trajectory acquisition unit 162, a speed control unit 164, and a steering control unit 166. The target trajectory acquisition unit 162 acquires information on the target trajectory (trajectory points) generated by the action plan generation unit 140 and stores it in memory (not shown). The speed control unit 164 controls the driving force output device 200 or the brake device 210 based on the speed elements associated with the target trajectory stored in memory. The steering control unit 166 controls the steering device 220 according to the curvature of the target trajectory stored in memory. The processing of the speed control unit 164 and the steering control unit 166 is realized, for example, by a combination of feedforward control and feedback control. As an example, the steering control unit 166 performs a combination of feedforward control according to the curvature of the road in front of the vehicle M and feedback control based on the deviation from the target trajectory.

[0042] Returning to Figure 1, the HMI control unit 180 notifies the occupant of vehicle M of predetermined information via the HMI 30. The predetermined information includes, for example, information related to the driving of vehicle M, such as information regarding the status of vehicle M and information regarding driving control. Information regarding the status of vehicle M includes, for example, the speed of vehicle M, engine speed, and shift position. Information regarding driving control includes, for example, information inquiring whether or not to perform driving control by automatic driving, whether or not to start automatic driving, information regarding the status of driving control by automatic driving, information regarding the driving mode, and information prompting the occupant to take action when switching from automatic driving to manual driving. The predetermined information may also include information regarding the surrounding conditions recognized by the detection device DD. The predetermined information may also include information unrelated to the driving of vehicle M, such as content stored on a storage medium such as a television program or DVD (e.g., a movie). The predetermined information may also include, for example, the current location and destination in automatic driving, and information regarding the remaining fuel level of vehicle M. The HMI control unit 180 may output the information received by the HMI 30 to the communication device 20, the navigation device 50, the first control unit 120, etc.

[0043] Furthermore, the HMI control unit 180 may output to the HMI 30 information such as inquiry information for the occupants and processing results from the first control unit 120 and the second control unit 160. In addition, the HMI control unit 180 may transmit various information to be output to the HMI 30 to terminal devices used by the occupants of the vehicle M via the communication device 20.

[0044] The driving force output device 200 outputs driving force (torque) to the drive wheels for the vehicle to move. The driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, and a transmission, and an ECU (Electronic Control Unit) that controls them. The ECU controls the above configuration according to information input from the second control unit 160 or information input from the accelerator pedal of the driver control unit 80.

[0045] The brake system 210 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to information input from the second control unit 160 or from the brake pedal of the driver control unit 80, so that brake torque corresponding to the braking operation is output to each wheel. The brake system 210 may be equipped with a backup mechanism that transmits the hydraulic pressure generated by the operation of the brake pedal to the cylinder via a master cylinder. The brake system 210 is not limited to the configuration described above, and may also be an electronically controlled hydraulic brake system that controls an actuator according to information input from the second control unit 160 to transmit hydraulic pressure from the master cylinder to the cylinder.

[0046] The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor, for example, applies force to a rack and pinion mechanism to change the direction of the steering wheels. The steering ECU drives the electric motor to change the direction of the steering wheels according to information input from the second control unit 160 or information input from the steering wheel of the driver control unit 80.

[0047] [Recognition unit, operation control unit] Next, we will explain in detail the driving control of the vehicle M based on the functions of the recognition unit 130 (mainly the acquisition unit 132, the lane marking recognition unit 134, the structure recognition unit 136, and the output unit 138) and the driving control unit (mainly the driving control unit 142).

[0048] Figure 3 is a diagram illustrating an example of the driving conditions of vehicle M in the embodiment. In Figure 3, two lanes L1 and L2 that can travel in the same direction are shown as an example. Lane L1 is demarcated by lane markings LL and CL, and lane L2 is demarcated by lane markings CL and RL. Lanes L1 and L2 are examples of "driving paths (roads)" that vehicle M can travel on. In the example in Figure 3, vehicle M is assumed to be traveling on lane L1 at a speed VM. Also, in the example in Figure 3, vehicle M is traveling on a straight road just before a curved road with a curvature greater than a predetermined value that exists in the direction of travel. Also, in the example in Figure 3, structure OB1 is located to the side of lane L1 and further away (in the background) than lane marking LL, as seen from vehicle M, and structure OB2 is located to the side of lane L2 and further away (in the background) than lane marking RL. Structures OB1 and OB2 are landmarks such as walls (e.g., sound barriers, tunnel walls, etc.), guardrails, fences, curbs, median strips, etc., provided along lanes L1 and L2, but are not limited to these.

[0049] In a driving situation as shown in Figure 3, for example, the acquisition unit 132 performs sensor fusion processing on the output results from some or all of the camera 10, radar device 12, and LIDAR 14 included in the detection device DD mounted on the vehicle M, to acquire the position, speed, acceleration, shape, and other states of objects present around the vehicle M (within a predetermined distance). Objects include other vehicles, pedestrians, bicycles, and other traffic participants, as well as road markings and structures that demarcate the driving path (road, lane, etc.). Objects may also include stop lines, obstacles, red lights, toll booths, other road events, road markings (speed limits), and road signs indicating speed limits. The position of an object is recognized as a position on an absolute coordinate system with the vehicle M's representative point (such as the center of gravity or drive axis center) as the origin, and is used for control. The position of an object may be represented by a representative point such as the object's center of gravity or corner, or by a represented region. The "state" of an object may include, for example, the acceleration or jerk of a moving object such as another vehicle, or its "action state" (for example, whether or not the other vehicle is changing lanes or is about to change lanes).

[0050] For example, the acquisition unit 132 performs known image analysis processing (e.g., feature extraction of edges, shape, size, and color, pattern matching, etc.) on the image captured by the camera 10 (hereinafter referred to as the camera image) and acquires objects around the vehicle M based on the image analysis results. Alternatively, the acquisition unit 132 may, instead of (or in addition to) the output of the detection device DD, refer to map information based on the position information of the vehicle M obtained from the vehicle sensor 40, etc., to acquire information on structures around the vehicle M (e.g., physical boundaries, etc.), road shape, and the position and shape of lane markings (e.g., number of lanes, curvature, etc.). At least some of the functions of the acquisition unit 132 may be provided in the lane marking recognition unit 134 or the structure recognition unit 136.

[0051] The lane marking recognition unit 134 recognizes lane markings present on the road surface of the vehicle M's travel path (road) based on the output results of the acquisition unit 132. For example, the lane marking recognition unit 134 extracts edge points with large brightness differences from adjacent pixels by performing image analysis processing on the camera image, and recognizes lane markings present on the road surface of the travel path based on the extension direction and length in the image plane of the sequence of points formed by connecting the extracted edge points. Alternatively, the lane marking recognition unit 134 may recognize lane markings based on detection results from, for example, the LIDAR 14. Furthermore, the lane marking recognition unit 134 converts the position of the lane markings, relative to the position of a representative point of the vehicle M, into a vehicle coordinate system (for example, an XY plane coordinate system where the front direction of the vehicle M is the X axis and the vehicle width direction (lateral direction) is the Y axis, as shown in Figure 3).

[0052] Furthermore, the lane marking recognition unit 134 may recognize lane markings present on the vehicle M's roadway from map information based on the vehicle M's location information. For example, the lane marking recognition unit 134 may refer to map information (first map information 54, second map information 62) based on the vehicle M's location information acquired by the vehicle sensor 40 or GNSS receiver 51, and recognize lane markings and the number of lanes present around the vehicle M (or on the roadway) from the map information. Alternatively, the lane marking recognition unit 134 may recognize lane markings that demarcate lanes that can be traveled in the same direction as the vehicle M, based on lane markings obtained from camera images and lane markings obtained from a map.

[0053] In the example shown in Figure 3, the lane marking recognition unit 134 recognizes the lane markings LL and CL that demarcate lane L1 around vehicle M (in other words, the left and right lane markings closest to vehicle M). In addition to the lane markings LL and CL that demarcate lane L1, the lane marking recognition unit 134 may also recognize the lane markings CL and RL that demarcate lane L2.

[0054] The structure recognition unit 136 recognizes structures located to the side of the vehicle M's travel path based on the acquisition results from the acquisition unit 132. For example, the structure recognition unit 136 extracts edge points from the camera image using known image analysis processing, and recognizes structures included in the camera image by performing pattern matching processing between the shape, size, and other characteristic information of the extracted edge point cloud and a feature table in which characteristic information for each structure is stored in advance. Note that the characteristic information may also include other characteristic information such as color information. Furthermore, the structure recognition unit 136 recognizes structures that are located to the side of the vehicle M's travel path (within a predetermined distance from the travel path) among the recognized structures.

[0055] Furthermore, the structure recognition unit 136 may recognize structures present around the vehicle M based on detection results from the radar device 12, instead of (or in addition to) the camera image. For example, the structure recognition unit 136 extracts groups of objects within a predetermined distance from the detection results (radar object group) from the radar device 12 by clustering, and recognizes the position and shape of structures present to the side of the vehicle M's path based on the extraction results.

[0056] Furthermore, the structure recognition unit 136 may recognize the position and shape of structures in front of or to the side of the vehicle M based on the detection results from the LIDAR 14, instead of (or in addition to) the radar device 12. The structure recognition unit 136 may also correct the position and shape of structures recognized from the camera image using information (position, shape, etc.) of structures detected by the radar device 12 or LIDAR 14.

[0057] In the example shown in Figure 3, the structure recognition unit 136 recognizes structures OB1 and OB2 located to the sides of lanes L1 and L2.

[0058] The derivation unit 138 derives the degree of curvature of the vehicle M's travel path based on the recognition results of the lane marking recognition unit 134 and the structure recognition unit 136. The degree of curvature may be an index value based on the size of the curvature or radius of curvature, or it may be an index value indicating the size of the curve. For example, the degree of curvature is greater the larger the curvature (or the smaller the radius of curvature). Alternatively, a curve may be determined to be small if the degree of curvature is below a threshold, and a curve may be determined to be large if the degree of curvature is above a threshold.

[0059] For example, the derivation unit 138 derives the degree of curvature in the direction of extension of the lane markings based on the shape of the lane markings recognized by the lane marking recognition unit 134. The derivation unit 138 also derives the degree of curvature of a structure based on the shape of the structure recognized by the structure recognition unit 136. The degree of curvature of a structure is, for example, the degree of curvature in the longitudinal direction (extension direction) of the structure along the direction of extension of the lane markings. For example, if multiple structures such as guardrails are arranged along the road, the multiple arranged structures may be considered as a single structure and the overall degree of curvature may be derived.

[0060] Furthermore, the derivation unit 138 derives a first degree of curvature of the roadway (lane) based on the lane markings recognized by the lane marking recognition unit 134 in the vicinity of point P1 in Figure 3 (a range less than a predetermined distance D1 in the direction of travel from vehicle M) among lanes L1 and L2, where the vehicle M is located (point P0 in Figure 3). Furthermore, the derivation unit 138 derives a second degree of curvature of the roadway based on structures recognized by the structure recognition unit 136 in the vicinity of point P1 in Figure 3 (a range greater than a predetermined distance D1 in the direction of travel from vehicle M). Then, the derivation unit 138 derives the degree of curvature of the roadway in the direction of travel of vehicle M based on the first degree of curvature and the second degree of curvature.

[0061] For example, as shown in Figure 3, if there is a curved road (curve) with a degree of curvature exceeding a predetermined value on the vehicle M's travel path (direction of travel), the predetermined distance D1 is set to the distance from the vehicle M's current position to the point where the lane markings can no longer be recognized from the camera image or the LIDAR 14 detection results due to the influence of the curved road in the vehicle M's direction of travel. This allows the degree of curvature of the entire travel path to be derived using the degree of curvature of structures recognized by the radar device 12, etc., in the portion where the lane markings cannot be recognized from the camera image, thereby improving the accuracy of overall curvature recognition. Furthermore, because the degree of curvature of the entire travel path can be recognized more quickly, more appropriate driving control can be performed.

[0062] Even if there are no curves in the direction of travel and the road is straight, the predetermined distance D1 may be the distance from the vehicle M's current position to the point where the lane markings can no longer be recognized from the camera image or the detection results of the LIDAR 14. Furthermore, the predetermined distance D1 may be a pre-set fixed distance, or it may be adjusted depending on the vehicle M's speed VM, etc.

[0063] Furthermore, when deriving the first degree of curvature, the derivation unit 138 derives the first degree of curvature based on lane markings located within a predetermined distance D1 from the vehicle M in the direction of travel, and when deriving the second degree of curvature, it derives the second degree of curvature based on structures located within a predetermined distance D1 or more from the vehicle M in the direction of travel. By using lane markings or structures corresponding to each range, the degree of curvature of the road in that range can be derived with greater accuracy.

[0064] In the example shown in Figure 3, the derivation unit 138 derives a first degree of curvature from point P0 to point P1 based on lane markings LL, CL, and RL. Based on the lane markings means, for example, based on the portion of a lane marking that is within a range of less than a predetermined distance D1 in the direction of travel from the vehicle M. In this case, the derivation unit 138 derives a first degree of curvature from at least one curvature (or radius of curvature) of the lane markings LL, CL, and RL. For example, the derivation unit 138 may use the average of the curvatures of each of the lane markings LL, CL, and RL as the first degree of curvature, or the curvature of the lane marking closest to the vehicle M as the first degree of curvature, or the average of the curvatures of the lane markings LL and CL that define the lane L1 on which the vehicle M is traveling as the first degree of curvature.

[0065] Furthermore, the derivation unit 138 derives a second degree of curvature based on structures OB1 and OB2 for sections further away from point P1 as seen from the vehicle M. Based on the structures means, for example, based on a portion of the entire structure that is included in a range of a predetermined distance D1 or more. For example, if structures exist on both sides of the road, the derivation unit 138 may derive the second degree of curvature using only the structure on one predetermined side, or using only the structure on the other side, or using both. In this case, the derivation unit 138 may, for example, use the average of the curvatures of structures OB1 and OB2 (the curvature of the surfaces along the road) as the second degree of curvature, or use the curvature of the structure closest to the vehicle M as the second degree of curvature, or use the curvature of structure OB1 located to the side of the lane L1 on which the vehicle M is traveling as the second degree of curvature. Furthermore, the derivation section 138 may use the curvature of the structure with the longest length (length in the direction along the road) among the recognized structures as the second degree of curvature.

[0066] Furthermore, as shown in Figure 3, when the derivation unit 138 derives the degree of curvature of a road (lanes L1, L2) based on structures within a predetermined distance D1 or more from the vehicle M, and when a curved road (curve) exists within a predetermined distance D1 or more, it may also derive a second degree of curvature based on structures located on the outer (convex) side of the curved road. Since structures on the outer side of a curved road are easier to recognize as a whole from the vehicle's perspective than structures on the inner (concave) side of the curve, the degree of curvature can be derived with higher accuracy by utilizing structures on the outer side of the curve.

[0067] For example, when the vehicle M is far away, the accuracy of recognizing the lane markings decreases. Therefore, by deriving a second degree of curvature based on structures around the road, the accuracy of recognizing (deriving) the overall degree of curvature of the road, which is a combination of the first and second degrees of curvature, can be further improved.

[0068] The derivation unit 138 may derive the values ​​using a predetermined function (mathematical formula) that takes a first degree of curvature and a second degree of curvature as inputs and outputs the degree of curvature of the entire road. Alternatively, the derivation unit 138 may derive the degree of curvature of the entire road in relation to the first degree of curvature and the second degree of curvature using a pre-trained model that has been trained in advance by machine learning or the like. Furthermore, the derivation unit 138 may use a function or pre-trained model that takes the shape of the lane markings as input instead of the first degree of curvature, or use a function or pre-trained model that takes the shape of a structure as input instead of the second degree of curvature.

[0069] The driving control unit (driving control unit 142 and second control unit 160) controls the vehicle M (driving control) based on the recognition results of the recognition unit 130 described above. For example, the driving control unit 142 determines the driving control for the vehicle M based on the recognition results of the recognition unit 130 and generates a target trajectory based on the determined driving control. "Determining driving control" may include, for example, determining the content (type) of the driving control, determining the acceleration / deceleration amount (longitudinal control amount) and steering amount (lateral control amount), and deciding whether or not to execute (suppress) the driving control. Furthermore, "executing driving control" may include, for example, switching and executing the content of the driving control, as well as continuing the driving control that is already being executed. "Suppressing driving control" may include not only not executing driving control, but also lowering the mode (automation level) of the driving control.

[0070] Here, the driving control may include, in addition to (or instead of) controlling at least one of the vehicle M's speed (longitudinal control) and steering (lateral control) to perform driving control such as LKAS, a first driving mode and a second driving mode in which the degree of driving assistance is lower than that of the first driving mode, or in which the tasks of the vehicle M occupants are greater than those of the first driving mode. A lower degree of driving assistance means, for example, a low automation rate in the driving control. A low automation rate means, for example, a low degree to which the automatic driving control device 100 controls the speed or steering of the vehicle M (a high degree to which the driver needs to intervene in acceleration, deceleration, or steering operations). A greater tasks for the occupants includes, for example, a large number of tasks assigned to the occupants or tasks that are heavy. Tasks include, for example, monitoring the surroundings of the vehicle M or the occupants operating the driving controls 80. Operating the driving controls 80 includes, for example, the driver gripping the steering wheel (hereinafter referred to as the hands-on state). Furthermore, the driving control may include a third driving mode, which has a lower degree of driving assistance than the second driving mode, or a greater task for the occupants of vehicle M than the second driving mode. In addition, the driving mode with the lowest degree of driving assistance or the greatest task for the occupants of vehicle M may be a fully manual driving mode (a mode in which no driving control is performed).

[0071] For example, in the first driving mode, the occupant has no (or minimal) tasks and driving control (e.g., ACC, LKAS, ALC, TJP, CMBS, etc.) is permitted when the occupant of vehicle M is not holding the steering wheel (hereinafter referred to as the hands-off state). In the second driving mode, tasks assigned to the occupant may include, for example, monitoring the surroundings of vehicle M while also being in the hands-on state.

[0072] For example, if the driving control unit 142 receives instructions from the occupant via the HMI 30, or if the surrounding conditions recognized by the recognition unit 130 satisfy the conditions for starting driving control and it is determined that predetermined driving control such as LKAS or ACC should be started, it determines whether the condition of the vehicle M satisfies the hands-off condition. The hands-off condition is the condition for executing driving control in a hands-off state (first driving mode). For example, in the case of LKAS, this includes, but is not limited to, the recognition of the left and right lane markings and the contact margin time (TTC) with surrounding obstacles (other vehicles, etc.) being longer than a predetermined time. The contact margin time (TTC) is derived, for example, by dividing the relative distance between the vehicle M and the obstacle by the relative speed. If the hands-off condition is met, the driving control unit 142 performs control to execute the first driving mode (in other words, to switch from the second driving mode to the first driving mode).

[0073] Furthermore, the driving control unit 142 controls the vehicle M, for example, by controlling at least one of the vehicle M's speed and steering based on a first degree of curvature and a second degree of curvature. For example, when LKAS is running, the driving control unit 142 controls the vehicle M's speed (e.g., braking control) and steering based on the degree of curvature of the driving path (lane) derived by the derivation unit 138, so that the vehicle M maintains its movement within the driving lane (does not deviate from the lane). For example, the greater the degree of curvature, the greater the steering amount (lateral control amount) or braking amount (longitudinal control amount) the vehicle will move along that driving path.

[0074] Furthermore, the travel control unit 142 may include the fact that the degree of curvature of the travel path derived by the derivation unit 138 is less than a threshold as one of the conditions for switching from the second operating mode to the first operating mode. Also, the travel control unit 142 may perform control to switch from the first operating mode to the second operating mode if the first operating mode is in operation and the degree of curvature of the travel path is greater than or equal to a threshold. Furthermore, the travel control unit 142 may perform control to suppress the execution of the first operating mode (switch from the first operating mode to the second operating mode, or not switch from the second operating mode to the first operating mode) if the first and second degrees of curvature have not been derived.

[0075] The driving control unit 142 generates a target trajectory K1 for performing the various controls according to the driving conditions described above, and causes the second control unit 160 to execute control so that the vehicle M travels along the generated target trajectory K1.

[0076] In this way, the degree of curvature of the road can be derived with greater accuracy based on the first and second degrees of curvature corresponding to the driving conditions. For example, in this embodiment, it is possible to recognize the overall shape (degree of curvature) of the road, including not only the degree of curvature near the vehicle M (first degree of curvature) but also the degree of curvature far from the vehicle M (second degree of curvature). Therefore, it is possible to perform more appropriate driving control using information on the degree of curvature of the road, and to improve the continuity of control.

[0077] Here, a specific example of speed control (braking control) in the embodiment will be described. Figure 4 is a diagram illustrating an example of braking control of vehicle M based on driving conditions. In Figure 4, the position and speed of vehicle M at time T* are represented as M(T*) and VM(T*), respectively. In the example in Figure 4, time T0 is the earliest, and times T1 and T2 are the latest. Also, in the example in Figure 4, the driving control unit is assumed to be, for example, performing LKAS control (first driving mode).

[0078] Time T0 indicates the point in time when the vehicle M does not recognize that there is a curved road in the direction of its travel (in other words, the point in time when it recognizes that the road is a straight road with a curvature less than a predetermined value). In this case, the driving control unit 142 generates a target trajectory based on the curvature of the lane lines LL, CL, and RL derived by the derivation unit 138, so that the vehicle M travels in the center of the road (lane L1) without deviating from lane L1.

[0079] Time T1 indicates the point in time when the system recognizes the presence of a curved road in the direction of travel of the vehicle M. In this case, the driving control unit 142 generates a target trajectory so that it first performs a first braking based on the second degree of curvature, and then performs a second braking based on the first degree of curvature after the first braking has been performed. Here, the first braking may be performed until the vehicle M travels a predetermined distance △D from its position at time T1 (point Pa) to point Pb, or it may be performed at time T2, after a predetermined time △T has elapsed from time T1. In other words, in the example in Figure 4, from point Pa to point Pb (from time T1 to time T2), deceleration control is performed based on structures OB1 and OB2 before the vehicle M enters a curved road with the second degree of curvature, and when traveling from point P2 onwards (from time T2 onwards), a second braking is performed based on the degree of curvature of the lane markings LL, CL, and RL recognized at that time. The second braking may be performed until the vehicle M enters the curved road, and may also be performed after entering the curved road (for example, until the vehicle passes the curved road).

[0080] This allows for earlier and more accurate detection of the presence of a curved road based on the curvature of structures OB1 and OB2 (second curvature), enabling an earlier start to braking and more appropriate deceleration control before entering a curved road. Furthermore, after decelerating to a certain extent with the first braking, a second braking based on nearby lane markings allows for more appropriate driving control according to the surrounding conditions.

[0081] For example, the first braking described above is to suppress the acceleration of vehicle M, and the second braking is to decelerate vehicle M. By using this type of braking control, acceleration can be suppressed earlier in preparation for curves, thereby suppressing changes (disturbances) in behavior due to acceleration and deceleration, and enabling more appropriate driving control.

[0082] Alternatively, instead of the above control, both the first and second braking may be used to decelerate the vehicle M. In this case, the first braking will decelerate less than the second braking. Therefore, the speed VM(T1) of vehicle M(T1) due to the first braking will be less than the speed VM(T0) of vehicle M(T0) before the first braking, and the speed VM(T2) of vehicle M(T2) due to the second braking will be less than the speed VM(T1) of vehicle M(T1) due to the first braking.

[0083] Furthermore, the driving control unit 142 may gradually change the deceleration rate over a predetermined time or distance so that the deceleration rate does not fluctuate significantly when switching between the first and second braking. This enables smoother braking control.

[0084] Furthermore, the driving control unit 142 may perform steering control (lateral control) of the vehicle M based on the first degree of curvature and speed control (longitudinal control) of the vehicle M based on the second degree of curvature. In this case, the above-described control may be performed, for example, while driving on a curved road (until the curve is passed), after the execution of the second braking, during the execution of the first or second braking, or a combination thereof. In this embodiment, the accuracy of the degree of curvature of the road near and far from the vehicle M can be improved. Therefore, by performing steering control based on the degree of curvature of the lane markings near the vehicle and braking control based on the degree of curvature of structures far away, more appropriate driving control can be performed, and the continuity of driving control can be improved.

[0085] Furthermore, in the embodiment, when the derivation unit 138 derives a first degree of curvature and a second degree of curvature, it may use one degree of curvature to correct the other degree of curvature. For example, the derivation unit 138 corrects the first degree of curvature derived from the lane markings based on the second degree of curvature derived from the structure, thereby deriving the degree of curvature of the road that is farther than a predetermined distance from the vehicle M based on the corrected first degree of curvature. Also, when the section using the first degree of curvature and the section using the second degree of curvature are switched, the derivation unit 138 may use one degree of curvature to correct the other degree of curvature so that the driving control does not change significantly due to the difference in the degree of curvature (so that each section can be smoothly connected). Furthermore, if a portion of the lane markings within a predetermined distance D1 from the vehicle M cannot be recognized, but a structure in that portion can be recognized, the derivation unit 138 corrects (derives) the first curvature based on the lane markings using a second curvature based on that structure. In this way, by correcting the curvature of one curvature using information from the other curvature, a more accurate (wider range) curvature can be derived, and this curvature can be used to perform more appropriate driving control.

[0086] In addition, in this embodiment, the derivation unit 138 may select information to be used to derive the degree of curvature based on conditions such as the presence or absence of a map, map accuracy, and recognition accuracy by the camera 10. Furthermore, the derivation unit 138 may select information to be used to derive the degree of curvature depending on whether vehicle M speed control (longitudinal control) or steering control (lateral control) is being performed.

[0087] Figure 5 is a diagram illustrating the relationship between the controlled object and the conditions when deriving the degree of curvature. In the example in Figure 5, the diagram explains the information used to derive the degree of curvature for each controlled object (speed control, steering control) in relation to the map information and camera recognition results.

[0088] In the example shown in Figure 5, if the memory unit 190 or the like contains map information (high-precision map information), the derivation unit 138 uses that map information to derive the degree of curvature (first degree of curvature, second degree of curvature). The driving control unit 142 then performs speed control and steering control based on the derived degree of curvature.

[0089] Furthermore, if map information is unavailable (or the map reliability is below a predetermined level), and the camera 10 can see far into the distance depending on the driving conditions (i.e., the camera image can recognize far into the distance), the derivation unit 138 derives the degree of curvature using the lane markings obtained from the camera image. The driving control unit 142 then performs speed control and steering control based on the derived degree of curvature. Map reliability below a predetermined level means, for example, that the map is not high-precision or that a predetermined period has passed since the map was last updated.

[0090] Furthermore, if map information is unavailable and the camera 10 cannot see far into the distance due to driving conditions (but can see up to the immediate vicinity in the camera image), the derivation unit 138 derives the degree of curvature in the vicinity using lane markings recognized from the camera image, and derives the degree of curvature in the distance using structures recognized from the output of the radar device 12, etc. Then, the driving control unit 142 performs speed control based on the degree of curvature based on the structures and performs steering control based on the degree of curvature based on the lane markings.

[0091] Furthermore, if map information is unavailable and the camera 10 cannot see anything from nearby to far away due to driving conditions (i.e., lane markings cannot be recognized from the camera image), the derivation unit 138 derives the degree of curvature using structures recognized from the output of the radar device 12, etc. The driving control unit 142 then performs speed control and steering control based on the degree of curvature based on the structures. In addition, in this embodiment, even if map information is available, the degree of curvature may be derived based on lane markings and structures, and a process of using both the degree of curvature based on map information and the degree of curvature based on lane markings and structures may be performed, for example.

[0092] In this way, by deriving the degree of curvature for vehicle M's speed control and steering control according to various situations, the continuity of driving control can be further improved.

[0093] Furthermore, in the embodiment, if, for example, the curvature based on the lane markings (an example of the first degree of curvature) and the curvature based on the structure (an example of the second degree of curvature) are different, the rate of change of curvature at the time of switching (transition rate) may be adjusted according to, for example, the elapsed time, so that the degree of curvature does not change drastically when switching the curvature used and affect speed control or steering control.

[0094] Figure 6 is a diagram illustrating the changes that occur when switching between lane marking curvature and structural curvature. Lane marking curvature refers to, for example, the curvature of the lane markings recognized by the lane marking recognition unit 134, and structural curvature refers to, for example, the curvature of the structure recognized by the structure recognition unit 136. In the example of Figure 6, the horizontal axis represents the distance traveled by the vehicle M in the direction of travel, and the vertical axis represents the curvature. In the example of Figure 6, it is assumed that there exists a road shape (for example, a road shape with changing curvature) in the direction of travel of the vehicle M that switches from a range of less than a predetermined distance D1 where the vehicle travels using lane marking curvature to a range of a predetermined distance D1 or more where the vehicle travels using structural curvature. In addition to lane marking curvature and structural curvature, Figure 6 also shows the transition of curvature over time when switching from lane marking curves to structural curves (transitional curvature). Transitional curvature is the curvature used in driving control (the curvature from which the degree of curvature is derived).

[0095] The derivation unit 138 gradually adjusts the transition rate over time when switching from a range of less than a predetermined distance D1 where the vehicle travels using the curvature of the road markings to a range of a predetermined distance D1 or more where the vehicle travels using the curvature of the structure, and derives the degree of curvature using the curvature during the transition. This allows for a smooth switch of curvature while suppressing the amount of change. In the example in Figure 6, the transition rate changes to 0%, 50%, and 100% over time, but the transition rate is also gradually changed over time in between these periods. As described above, by gradually changing the transition rate when switching between the curvature of the road markings and the curvature of the structure, large changes in driving control due to the effect of the switch can be suppressed, and the behavior (control) of the vehicle M during driving can be controlled more appropriately.

[0096] [Processing flow] Next, the processes executed by the automatic driving control device 100 of this embodiment will be described. In the following, the driving control processes based mainly on the curvature of the road will be described among the processes executed by the automatic driving control device 100. The processes shown below may be executed repeatedly at predetermined timings or predetermined cycles.

[0097] Figure 7 is a flowchart showing an example of the flow of the driving control process in an embodiment. In the example in Figure 7, the acquisition unit 132 acquires information about the surrounding conditions of the vehicle M (step S100). Next, the lane marking recognition unit 134 recognizes lane markings present around the vehicle M based on the acquired information (step S110). Next, the structure recognition unit 136 recognizes structures present around the vehicle M based on the acquired information (step S120).

[0098] Next, the derivation unit 138 derives a first degree of curvature based on recognized lane markings within a range less than a predetermined distance from the vehicle M (step S130). Next, the derivation unit 138 derives a second degree of curvature based on recognized structures within a range greater than or equal to a predetermined distance from the vehicle M (step S140). Next, the driving control unit controls the vehicle M based on the information of the first degree of curvature and the second degree of curvature (step S150). This completes the processing of this flowchart.

[0099] [Other examples] Next, other embodiments of the operation control process in the embodiment will be described. Figure 8 is a flowchart showing another embodiment of the operation control process flow in the embodiment. The process in Figure 8 may also be executed repeatedly at a predetermined cycle or timing.

[0100] In the example shown in Figure 8, the acquisition unit 132 acquires information about the surrounding conditions of the vehicle M (step S200). Next, the lane marking recognition unit 134 recognizes the lane markings based on the output from the detection device DD (e.g., camera 10) acquired by the acquisition unit 132 (step S210). Next, the derivation unit 138 derives a first degree of curvature based on the lane marking information (step S220). Next, the derivation unit 138 determines whether the recognition of the lane markings is interrupted (whether the lane markings become unrecognizable) based on the distance from the vehicle M due to the influence of driving conditions (e.g., curved road, nighttime, etc.) (step S230). If it is determined that the recognition of the lane markings is interrupted, the derivation unit 138 derives a predetermined distance (e.g., the distance from the position of the vehicle M to the point where the recognition of the lane markings is interrupted) (first predetermined distance) which will serve as a reference for deriving a second degree of curvature (step S240). Next, the structure recognition unit 136 recognizes a structure within a range of a predetermined distance or greater, based on the output from the detection device DD (e.g., radar device 12) acquired by the acquisition unit 132 (step S250). Next, the derivation unit 138 detects a second degree of curvature based on the recognized structure (step S260).

[0101] Next, the derivation unit 138 determines whether or not a curved road exists in the direction of travel of the vehicle M (step S270). If it determines that no curved road exists, the derivation unit 138 corrects the first degree of curvature based on the second degree of curvature so that, for example, the section with the first degree of curvature and the section with the second degree of curvature are connected without a large difference (step S280). Next, the driving control unit controls the vehicle M based on the corrected degree of curvature (step S290), and the processing of this flowchart ends.

[0102] Furthermore, if the driver control unit determines that a curved road exists during the process in step S270, it performs a first braking (step S300) and drives the vehicle M to a predetermined distance (second predetermined distance) before the curved road (step S310). Next, the driver control unit performs a second braking (step S320). After the second braking (or during the first and second braking), the driver control unit controls the steering of the vehicle based on the first degree of curvature and controls the speed of the vehicle based on the second degree of curvature (step S330). In this embodiment, the process in step S330 may also involve correcting the first degree of curvature based on the second degree of curvature and controlling the vehicle M with the corrected degree of curvature. After that, the driver control unit determines whether or not the vehicle has passed the curved road (step S340). If it determines that the vehicle has not passed the curved road, it returns to the process in step S330; if it determines that the vehicle has passed the curved road, it terminates the process in this flowchart.

[0103] Furthermore, if the system determines in step S230 that the recognition of the lane markings is not interrupted, the driving control unit controls the vehicle M based on the first degree of curvature obtained from the lane markings (step S350), and terminates the processing of this flowchart.

[0104] In the embodiments described above, as shown in Figures 3 and 4, the processing for driving on a straight road with a curve in the direction of travel was explained, but similar processing may be applied to other road conditions such as straight roads and S-curves.

[0105] As described above, according to the embodiment described above, the automatic driving control device 100 (an example of a vehicle control device) includes: an acquisition unit 132 that acquires the surrounding conditions of the vehicle M based on the output of a detection device DD mounted on the vehicle M; a lane marking recognition unit 134 that recognizes lane markings present on the road surface of the vehicle M's travel path based on the acquisition results of the acquisition unit 132; a structure recognition unit 136 that recognizes structures present on the side of the vehicle M's travel path based on the acquisition results of the acquisition unit 132; and a lane marking recognition unit 134 that recognizes curves in the travel path on which the vehicle M is traveling based on the recognition results of the lane marking recognition unit 134 and the structure recognition unit 136. The system includes a derivation unit 138 that derives the degree of curvature, and an operation control unit (action plan generation unit 140, second control unit) that controls the vehicle M based on the degree of curvature of the travel path derived by the derivation unit. The derivation unit 138 calculates a first degree of curvature based on lane markings for the travel path within a predetermined distance from the vehicle M, and derives a second degree of curvature based on structures for the travel path beyond a predetermined distance from the vehicle M. The operation control unit controls the vehicle M based on the first and second degrees of curvature, thereby enabling more appropriate operation control according to the travel conditions. This can ultimately contribute to the development of a sustainable transportation system.

[0106] For example, in the embodiment, in scenes where the detection range of the camera 10 and LIDAR 14 decreases, such as curved roads with large curvature or S-shaped roads, the degree of curvature derived from the detected lane markings and the degree of curvature derived from surrounding structures (barriers, landmarks), such as walls and guardrails, can be recognized with higher accuracy even for the shape of the road that cannot be detected by the camera. In particular, near curved roads, structures such as guardrails and fences are often installed to prevent the vehicle M from deviating significantly from the road. Therefore, in the embodiment, the degree of curvature near curved roads where lane markings cannot be recognized in the camera image or LIDAR 14 detection results can be derived using the degree of curvature of structures near the curved road, thereby enabling higher accuracy (wider range) recognition of the curvature of the road. Consequently, based on the recognized shape of the road, more appropriate (natural) driving control can be performed. Furthermore, according to the embodiment, contact with surrounding objects (structures, etc.) near curved roads, for example, can be avoided more appropriately.

[0107] The embodiments described above can be expressed as follows. A storage medium that stores computer-readable instructions, A processor connected to the storage medium, The processor executes the computer-readable instructions to: Based on the output of at least one detection device mounted on the vehicle, the surrounding conditions of the vehicle are acquired. Based on the acquired results, the lane markings present on the road surface of the vehicle's travel path are recognized. Based on the results obtained, the structure located to the side of the vehicle's travel path is recognized. Based on the recognition results of the lane markings and the structure, the degree of curvature of the road on which the vehicle travels is derived. Based on the degree of curvature of the aforementioned travel path, the vehicle is controlled. For the section of the aforementioned road less than a predetermined distance from the vehicle, a first degree of curvature is derived based on the lane markings, and for the section greater than or equal to the predetermined distance from the vehicle, a second degree of curvature is derived based on the structure. Based on the first degree of curvature and the second degree of curvature, the vehicle is controlled. Vehicle control device.

[0108] Although embodiments for carrying out the present invention have been described above using examples, the present invention is not limited in any way to these embodiments, and various modifications and substitutions can be made without departing from the spirit of the present invention. [Explanation of symbols]

[0109] 1...Vehicle system, 10...Camera, 12...Radar device, 14...LIDAR, 20...Communication device, 30...HMI, 40...Vehicle sensor, 50...Navigation device, 60...MPU, 80...Driver's control unit, 100...Automatic driving control device, 120...First control unit, 130...Recognition unit, 132...Acquisition unit, 134...Lane marking recognition unit, 136...Structure recognition unit, 138...Derivation unit, 140...Action plan generation unit, 142...Driving control unit, 160...Second control unit, 162...Target trajectory acquisition unit, 164...Speed ​​control unit, 166...Steering control unit, 180...HMI control unit, 190...Storage unit, 200...Driving force output device, 210...Brake device, 220...Steering device, M...Vehicle

Claims

1. An acquisition unit that acquires the surrounding conditions of the vehicle based on the output of at least one detection device mounted on the vehicle, A lane marking recognition unit recognizes lane markings present on the road surface of the vehicle's travel path based on the acquisition results of the acquisition unit, Based on the acquisition results of the acquisition unit, a structure recognition unit recognizes structures located to the side of the vehicle's travel path, Based on the recognition results of the lane marking recognition unit and the structure recognition unit, a derivation unit derives the degree of curvature of the road on which the vehicle travels, The vehicle is controlled by an operation control unit that controls the vehicle based on the degree of curvature of the travel path derived by the derivation unit, The derivation unit derives a first degree of curvature based on the lane markings for the portion of the road less than a predetermined distance from the vehicle, and derives a second degree of curvature based on the structure for the portion of the road greater than or equal to the predetermined distance from the vehicle. The operation control unit controls the vehicle based on the first degree of curvature and the second degree of curvature. Vehicle control device.

2. The detection device includes a camera or LIDAR (Light Detection and Ranging), The predetermined distance is the distance from the vehicle to the point where the lane markings can no longer be recognized from the image captured by the camera or the detection result of the LIDAR due to the influence of the shape of the road in the direction of travel of the vehicle. The vehicle control device according to claim 1.

3. The aforementioned derivation section is, The lane line recognition unit derives the first degree of curvature based on the lane lines recognized within a range less than the predetermined distance, The second degree of curvature is derived based on the structures recognized by the structure recognition unit within a range of a predetermined distance or greater. The vehicle control device according to claim 1.

4. The derivation unit derives the second degree of curvature based on the structure located on the outside of the curve of the road when the road is a curved road. The vehicle control device according to claim 3.

5. The driving control unit controls at least one of the vehicle's speed and steering based on the first degree of curvature and the second degree of curvature. The vehicle control device according to claim 1.

6. The driving control unit, when a curved road exists in the direction of travel of the vehicle, performs a first braking based on the second degree of curvature, and after the first braking, performs a second braking based on the first degree of curvature. The vehicle control device according to claim 5.

7. The first braking action described above is to suppress the acceleration of the vehicle, The second braking is the deceleration of the vehicle. The vehicle control device according to claim 6.

8. The first braking and the second braking are deceleration of the vehicle. The first braking method has a smaller degree of deceleration compared to the second braking method. The vehicle control device according to claim 6.

9. The driving control unit controls the steering of the vehicle based on the first degree of curvature and controls the speed of the vehicle based on the second degree of curvature. The vehicle control device according to claim 3.

10. The derivation unit corrects the degree of curvature of the first degree of curvature and the second degree of curvature based on the other degree of curvature. The vehicle control device according to claim 1.

11. Computers Based on the output of at least one detection device mounted on the vehicle, the surrounding conditions of the vehicle are acquired. Based on the acquired results, the lane markings present on the road surface of the vehicle's travel path are recognized. Based on the results obtained, the structure located to the side of the vehicle's travel path is recognized. Based on the recognition results of the lane markings and the structure, the degree of curvature of the road on which the vehicle travels is derived. Based on the degree of curvature of the aforementioned travel path, the vehicle is controlled. For the section of the aforementioned road less than a predetermined distance from the vehicle, a first degree of curvature is derived based on the lane markings, and for the section greater than or equal to the predetermined distance from the vehicle, a second degree of curvature is derived based on the structure. Based on the first degree of curvature and the second degree of curvature, the vehicle is controlled. Vehicle control method.

12. On the computer, Based at least the output of a detection device mounted on the vehicle, the surrounding conditions of the vehicle are acquired. Based on the acquired results, the vehicle recognizes the lane markings present on the road surface of the vehicle's travel path. Based on the results obtained, the vehicle recognizes structures located to the side of its travel path. Based on the recognition results of the lane markings and the structure, the degree of curvature of the road on which the vehicle travels is derived. The vehicle is controlled based on the degree of curvature of the derived travel path. For the section of the aforementioned road less than a predetermined distance from the vehicle, a first degree of curvature is derived based on the lane markings, and for the section greater than or equal to the predetermined distance from the vehicle, a second degree of curvature is derived based on the structure. The vehicle is controlled based on the first degree of curvature and the second degree of curvature. program.

Images