Vehicle control device and vehicle control method

Abstract

To establish a more appropriate redundant configuration for external sensors mounted on vehicles. [Solution] The vehicle control device of the embodiment comprises an external sensor, a processing device that performs predetermined processing related to the vehicle, a control device that controls the movement of at least the vehicle, and a power supply unit that supplies power to at least the external sensor and the processing device. The external sensor includes a first sensor, a second sensor, and a third sensor. The processing device includes a first processing device and a second processing device. The outputs of the first sensor and the second sensor are output to the first processing device, and the output of the third sensor is output to the second processing device. The power supply unit includes a first power supply unit and a second power supply unit. The first power supply unit supplies power to at least the first sensor, the third sensor, and the first processing device, and the second power supply unit supplies power to at least the second sensor and the second processing device.

Description

Technical Field

[0001] The present invention relates to a vehicle control device and a vehicle control method.

Background Art

[0002] In recent years, efforts have been actively made to provide access to a sustainable transportation system that takes into account people in vulnerable positions among traffic participants. In order to achieve this, research and development efforts are focused on further improving traffic safety and convenience through research and development related to preventive safety technologies. In this context, conventionally, a first driving control means has a first monitoring unit that monitors the communication status of a first communication means and a second communication means, and a second driving control means has a second monitoring unit that monitors the communication status of a third communication means and a fourth communication means. When the first monitoring unit or the second monitoring unit detects a vehicle function degradation based on the communication status being monitored, at least one of the first driving control means and the second driving control means performs alternative control (see, for example, Patent Document 1). Also, conventionally, when it is determined that a failure has occurred in any of a plurality of controllers, the defective controller transmits failure information via a network communication line to a normal controller group other than the defective controller group to which the defective controller belongs. When the normal controller group receives the failure information via the network communication line, until the driver returns to operation, the controllers constituting the normal controller group back up the operation functions lost due to the failure (see, for example, Patent Document 2). Further, conventionally, a first control device that controls a vehicle based on the detection result of a first sensor unit that operates by power supply from a first power supply unit, and a second control device that controls a vehicle based on the detection result of a second sensor unit that operates by power supply from a second power supply unit are provided, and a technique for controlling the vehicle using the second sensor unit when the first sensor unit is abnormal is known (see, for example, Patent Document 3).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

[0004] Incidentally, in preventive safety technology, it is necessary to adjust the redundant configuration according to the type of external sensor that detects the surrounding conditions of the vehicle, but there was still room for consideration regarding the appropriate redundant configuration for the external sensors that are installed.

[0005] This invention was made in consideration of these circumstances, and one of its objectives is to provide a vehicle control device and a vehicle control method that can construct a more appropriate redundant configuration for external sensors mounted on a vehicle. Ultimately, this will contribute to the development of a sustainable transportation system. [Means for solving the problem]

[0006] The vehicle control device and vehicle control method according to this invention employ the following configuration. (1) A vehicle control device according to one aspect of the present invention comprises: an external sensor for detecting the surrounding conditions of a vehicle; a processing device for performing predetermined processing related to the vehicle based on the output of the external sensor; a control device for controlling the movement of at least the vehicle based on the processing results by the processing device; and a power supply unit for supplying power to at least the external sensor and the processing device, wherein the external sensor includes a first sensor, a second sensor, and a third sensor; the processing device includes a first processing device and a second processing device; the outputs of the first sensor and the second sensor are output to the first processing device; the output of the third sensor is output to the second processing device; the power supply unit includes a first power supply unit and a second power supply unit; the first power supply unit supplies power to at least the first sensor, the third sensor, and the first processing device; and the second power supply unit supplies power to at least the second sensor and the second processing device.

[0007] (2) In the embodiment of (1) above, the first sensor and the second sensor are each a plurality of cameras that image areas in multiple different directions from the vehicle, and the third sensor is a plurality of radar devices that detect objects located in multiple different directions from the vehicle.

[0008] (3) In the embodiment of (1) above, the first sensor and the second sensor are each a plurality of cameras having the same purpose or the same field of view, and the third sensor is a plurality of radar devices that detect objects from multiple different directions from the vehicle.

[0009] (4) In the embodiment of (1) above, the second sensor is connected to the first processing unit by a first communication line and to the second processing unit by a second communication line different from the first communication line.

[0010] (5) In the embodiment of (4) above, the first communication line is a communication network capable of higher capacity and faster communication than the second communication line.

[0011] (6) In the embodiment of (1) above, the first processing unit has a higher processing capacity than the second processing unit.

[0012] (7) In the embodiment of (1) above, the first processing unit generates a target trajectory of the vehicle based on the output of the external sensor, and the second processing unit outputs instructions to the actuator device included in the control device based on the target trajectory generated by the first processing unit.

[0013] (8) In the embodiment of (7) above, the actuator device includes a steering control device for controlling the steering of the vehicle and a speed control device for controlling the speed of the vehicle, wherein the steering control device and the speed control device are equipped with a communication line capable of receiving instructions from the second processing device and a communication line capable of receiving instructions from the first processing device.

[0014] (9) In the embodiment of (1) above, the first power supply unit supplies power to a first group consisting of at least the first sensor, the third sensor, and the first processing unit, the second power supply unit supplies power to a second group consisting of at least the second sensor and the second processing unit, and the control device, when it is determined that one of the first group and the second group is lost, uses the other group to determine the stopping position of the vehicle from the surrounding conditions of the vehicle and performs vehicle control to drive the vehicle to the determined stopping position.

[0015] (10): A vehicle control method according to an aspect of this invention is such that an external sensor detects the surrounding situation of the vehicle, a processing device performs predetermined processing related to the vehicle based on the output of the external sensor, at least the running of the vehicle is controlled based on the processing result by the processing device, a power supply unit supplies power to at least the external sensor and the processing device, the external sensor includes a first sensor, a second sensor, and a third sensor, the processing device includes a first processing device and a second processing device, the outputs of the first sensor and the second sensor are output to the first processing device, the output of the third sensor is output to the second processing device, the power supply unit includes a first power supply unit and a second power supply unit, the first power supply unit supplies power to at least the first sensor, the third sensor, and the first processing device, and the second power supply unit supplies power to at least the second sensor and the second processing device.

Advantages of the Invention

[0016] According to the aspects (1) to (10) above, a more appropriate redundant configuration can be constructed for the external sensors mounted on the vehicle.

Brief Description of the Drawings

[0017] [Figure 1] It is a configuration diagram of a vehicle system 1 including a vehicle control device according to an embodiment. [Figure 2] It is a functional configuration diagram of a first processing unit 124 according to an embodiment. [Figure 3] It is a diagram for explaining an example of the arrangement of an external sensor 10 in an embodiment. [Figure 4] It is a diagram for explaining an example of a redundant configuration including power supply in an embodiment. [Figure 5] It is a diagram showing a schematic view of the detection range of each sensor included in an external sensor 10 according to an embodiment.

Modes for Carrying Out the Invention

[0018] Hereinafter, embodiments of the vehicle control device and the vehicle control method of the present invention will be described with reference to the drawings.

[0019] [Overall Configuration] FIG. 1 is a configuration diagram of a vehicle system 1 including a vehicle control device according to an embodiment. The vehicle (hereinafter referred to as vehicle M) on which the vehicle system 1 is mounted is, for example, a two-wheeled, three-wheeled, four-wheeled or other vehicle, and its drive source is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. The electric motor operates using electric power generated by a generator connected to the internal combustion engine, or discharge power of a secondary battery or a fuel cell.

[0020] The vehicle system 1 includes, for example, an external sensor (external detection unit) 10 composed of a plurality of types of sensors, a processing device 100, a communication device 210, an HMI (Human Machine Interface) 220, a vehicle sensor 230, a navigation device 240, an MPU (Map Positioning Unit) 250, a driver monitor camera 260, a driving operator 270, a storage unit 280, a travel control device 300, and a power supply unit 400. These devices and equipment are connected to each other by, for example, a multiplex communication line such as a CAN (Controller Area Network) communication line or an Ethernet (registered trademark), a serial communication line, a wireless communication network, or the like. Ethernet is a communication network capable of high-capacity and high-speed communication compared to CAN. The configuration shown in FIG. 1 is merely an example, and a part of the configuration may be omitted, or another configuration may be added. Further, each functional configuration may be integrated or provided dispersedly. The travel control device 300 is an example of a "control device". The external sensor 10, the processing device 100, the travel control device 300, and the power supply unit 400 are examples of a "vehicle control device".

[0021] The external sensor 10 detects the conditions around the vehicle M (within a predetermined distance from the vehicle M). The external sensor 10 includes, for example, a surround camera 11, a LiDAR (Light Detection and Ranging) 12, an MVC (Multi View Camera) 13, a sonar 14, a camera 15, and a radar device 16. Of the external sensor 10, at least some of the surround camera 11, MVC 13, sonar 14, camera 15, and radar device 16 may be provided in multiples. The surround camera 11 is an example of a "first sensor," the MVC 13 is an example of a "second sensor," and the radar device 16 is an example of a "third sensor." In addition to the surround camera 11, the first sensor may also include a camera 15.

[0022] The surround camera 11 is a camera installed around the vehicle body (the main body of the vehicle M), including the left, right, and rear. The surround camera 11 may be a digital camera using a solid-state image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor), or it may be a stereo camera. The surround camera 11 senses at least a few meters to the left, right, and rear (including above the vehicle M) of the vehicle M and acquires images of the area including the sides and rear of the vehicle M. The surround camera 11 may also acquire images of the entire surroundings of the vehicle M, including images captured by camera 15.

[0023] The LIDAR 12 irradiates light (or electromagnetic waves with a wavelength close to light) around the vehicle M and measures the scattered light. Based on the time from emission to reception, the LIDAR 12 detects the distance to surrounding objects. The irradiated light is, for example, pulsed laser light. The LIDAR 12 is installed, for example, on the roof of the vehicle M and senses objects in the surrounding area, including in front of the vehicle M. The LIDAR 12 may be mounted at any other location.

[0024] The MVC13 is a camera installed on the front, rear, left, and right sides of the vehicle body. The MVC13 may be a digital camera using a solid-state image sensor such as a CCD or CMOS, or it may be a stereo camera. The MVC may also be a wide-angle camera such as a fisheye camera (in other words, a camera with a wider field of view than the surround camera 11 or camera 15). The MVC13 senses the area around the vehicle M (mainly near the ground) and acquires surrounding images. The MVC13 captures images of areas that are blind spots for the driver of the vehicle M, for example.

[0025] The sonar 14 emits ultrasonic waves around the vehicle M and detects the distance or position of an object by detecting reflection or scattering from an object within a predetermined distance from the vehicle M. Multiple sonar 14s are installed at arbitrary locations on the vehicle M.

[0026] Camera 15 is installed at any location on the vehicle M and captures images of an area including at least the front of the vehicle M. Camera 15 may be a digital camera using a solid-state image sensor such as a CCD or CMOS, or it may be a stereo camera. In addition, multiple cameras 15 may be provided.

[0027] The radar device 16 emits radio waves such as millimeter waves around the vehicle M and detects radio waves reflected by objects (reflected waves) to determine at least the position (distance and bearing) of an object. The radar device 16 can be mounted at any location on the vehicle M. The radar device 16 may also detect the position and velocity of an object using the FM-CW (Frequency Modulated Continuous Wave) method.

[0028] Details regarding the placement of the external sensor 10 described above will be described later. Note that the external sensor 10 may also be equipped with other configurations. Furthermore, depending on the grade, generation (version), and functions of the vehicle M, some types of sensors (e.g., LIDAR 12) may not be provided, or other configurations may be included. The external sensor 10 performs sensing at predetermined intervals and outputs the results (detection results) to the processing unit 100.

[0029] The processing unit 100 performs predetermined processing related to the vehicle M. The processing unit 100 comprises, for example, a first processing unit 120, a second processing unit 140, and a loss determination unit 160. The first processing unit 120, the second processing unit 140, and the loss determination unit 160 are each implemented by a hardware processor, such as a CPU (Central Processing Unit), executing a program (software). Some or all of these components may be implemented by hardware (including circuitry) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit), and SOC (System On Chip), or by the cooperation of software and hardware. The program may be stored in advance on a device such as the HDD (Hard Disk Drive) or flash memory of the processing unit 100 (or vehicle system 1) (a storage device equipped with a non-transient storage medium), or it may be stored on a removable storage medium such as a DVD or CD-ROM, and installed on the HDD or flash memory of the processing unit 100 (or vehicle system 1) when the storage medium (non-transient storage medium) is mounted on a drive device.

[0030] The first processing unit 120 is an ECU (Electronic Control Unit) that performs driving control, including autonomous driving of the vehicle M. The second processing unit 140 is an ECU that performs driving control, including driver assistance control such as ADAS (Advanced Driver Assistance System) that assists the driver of the vehicle M. Driving control refers to performing driving control by automatically controlling either the steering or the speed of the vehicle M, or both. Examples of driving control include LKAS (Lane Keeping Assistance System), ACC (Adaptive Cruise Control System), ALC (Auto Lane Changing), TJP (Traffic Jam Pilot), and CMBS (Collision Mitigation Brake System). Driving control may also include MRM (Minimum Risk Maneuver; degraded driving) control, which moves the vehicle M to a safe location (e.g., the shoulder of the road) and stops it.

[0031] In the driving control of the first processing unit 120, basically all driving control is performed by the system (vehicle system 1), so driving control that controls the steering and speed of the vehicle M is performed without the driver's driving operation. However, if the system requests intervention from the driver, the driver will perform driving operation (manual driving). On the other hand, in the driving control of the second processing unit 140, one of the steering and speed is performed by the system, and the other is operated by the driver, or only specific driving control such as LKAS and ACC is performed by the system. The driving control of the second processing unit 140 may include control that provides warnings and alerts to the driver, or provides information (images and sounds) related to driving assistance. The specific functional configurations of the first processing unit 120 and the second processing unit 140 will be described later.

[0032] The failure detection unit 160 determines whether there is a failure (abnormality) in the first processing unit 120 and the second processing unit 140 based on detection results obtained from the vehicle sensor 230, etc. A failure is a situation in which the operation control of at least one of the first processing unit 120 and the second processing unit 140 does not operate correctly or does not operate at all due to an abnormality in hardware or software, for example. Failures may include the effects of abnormalities in the power supply unit 400, etc. Based on the determination result, the failure detection unit 160 uses a normal configuration from the redundant configuration described later to execute predetermined processes related to the vehicle M, such as surrounding object recognition and operation control. For example, if the failure detection unit 160 determines that there is a failure in either the first processing unit 120 or the second processing unit 140, it uses the other processing unit to execute operation control such as degraded operation (MRM).

[0033] The communication device 210 communicates with other vehicles in the vicinity of vehicle M, or with various server devices via a wireless base station, for example, by using a cellular network, Wi-Fi network, Bluetooth®, DSRC (Dedicated Short Range Communication), etc.

[0034] The HMI220 presents various information to the occupants of vehicle M (including the driver) and accepts input operations from the occupants. The HMI220 includes, for example, a display unit and a speaker. The display unit is, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) display device. The display unit is provided, for example, on the instrument panel or meter display unit. The display unit displays various images (including video) in the embodiment. The display unit may be configured integrally with the input unit as a touch panel. The speaker outputs predetermined sounds (e.g., alarms) into the vehicle interior. In addition to the display unit and speaker, the HMI220 may also include a microphone, buzzer, keys, etc. Furthermore, the HMI220 may include a HUD (Head Up Display).

[0035] The vehicle sensor 230 includes various sensors used to control the vehicle M, such as a vehicle speed sensor for detecting the vehicle's speed, an acceleration sensor for detecting acceleration, a yaw rate sensor for detecting angular velocity around the vertical axis, and a compass sensor for detecting the vehicle's orientation. The vehicle sensor 230 may also include a position sensor for detecting the vehicle M's position. The position sensor is, for example, a sensor that acquires position information (longitude and latitude information) from a GPS (Global Positioning System) device. Alternatively, the position sensor may be a sensor that acquires position information using a GNSS (Global Navigation Satellite System) receiver of the navigation device 240. Furthermore, the vehicle sensor 230 may also include measurement sensors for measuring the status of the power supply unit 400 (e.g., temperature, battery level, current value, voltage value, etc.) and measurement sensors for measuring the status of electrical lines and communication lines (disconnections and other communication errors), which will be described later.

[0036] The navigation device 240 includes, for example, a GNSS receiver, a navigation HMI, and a route determination unit. The navigation device 240 determines the position of vehicle M based on signals received from GNSS satellites. The position of vehicle M may be determined or supplemented by an INS utilizing the output of vehicle sensors 230. The navigation HMI includes a display device, speakers, a touch panel, keys, etc. The navigation HMI may be partially or completely shared with the HMI 220 described above. The route determination unit determines, for example, a route (hereinafter referred to as a route on a map) from the position of vehicle M determined by the GNSS receiver (or any input location) to a destination input by the occupant using the navigation HMI, by referring to map information 282 stored in the storage unit 280.

[0037] Map information 282 is information in which the shape of a road is represented by, for example, links indicating roads and nodes connected by those links. Map information may also include POI (Point of Interest) information, etc. Map information 282 may also include, for example, information on the center of a lane or information on lane boundaries (road markings, etc.). Map information 282 may also include road information, traffic regulation information, address information (address and postal code), facility information, telephone number information, etc. Road information may include, for example, road type information such as expressways and general roads, road shape information such as merges, junctions, T-junctions, curvature (or radius of curvature), number of lanes, road gradient, junctions (JCT), service areas, toll booths, zebra zones (traffic guide zones), etc. Map information 282 may be updated as needed by the communication device 210 communicating with other devices. Map information 282 may also be stored in a storage device such as the HDD or flash memory of the navigation device 240.

[0038] The navigation device 240 may provide route guidance using the navigation HMI 82 based on the route on the map. The navigation device 240 may also be implemented using the functions of a terminal device such as a smartphone or tablet held by an occupant. The navigation device 240 may transmit its current location and destination to the navigation server via the communication device 210 and obtain a route equivalent to the route on the map from the navigation server.

[0039] The MPU 250, for example, divides the map route determined by the navigation device 240 into multiple blocks (for example, every 100m in the direction of vehicle travel) and determines a recommended lane for each block by referring to the map information 282. The MPU 250 makes decisions such as which lane from the left the vehicle should travel in. If there is a branching point in the map route, the MPU 250 determines a recommended lane so that the vehicle M can travel along a reasonable route to proceed to the branching point. The MPU 250 also recognizes the position of the vehicle M based on the detection results of a gyro sensor (not shown) and the position of the vehicle M identified by the GNSS receiver.

[0040] The driver monitoring camera 260 is a digital camera that uses a solid-state image sensor such as a CCD or CMOS. The driver monitoring camera 260 is mounted at any location in the vehicle M in a position and orientation that allows it to capture the head of the driver seated in the driver's seat of the vehicle M from the front (in a direction that captures the face). For example, the driver monitoring camera 260 is mounted on top of a display device located in the center of the instrument panel of the vehicle M.

[0041] The driver controls 270 include, for example, a steering wheel, accelerator pedal, brake pedal, shift lever, and other controls. The driver controls 270 are equipped with sensors that detect the amount of operation or whether or not an operation is being performed, and the detection results are output to the processing unit 100 or the driving control device 300. The steering wheel may be, for example, formed in an annular shape, or it may take the form of an irregularly shaped steering wheel, a joystick, buttons, etc. A steering grip sensor is attached to the steering wheel. The steering grip sensor is implemented by a capacitive sensor or the like and outputs a signal to the processing unit 100 that can detect whether or not the driver is gripping the steering wheel (meaning that it is in contact with the steering wheel in a state where force can be applied).

[0042] The storage unit 280 may be implemented using the various storage devices described above, or by EEPROM (Electrically Erasable Programmable Read Only Memory), ROM (Read Only Memory), or RAM (Random Access Memory), etc. For example, the storage unit 280 stores map information 282, various information in the embodiment, programs, etc.

[0043] The driving control device 300 drives the vehicle M at a predetermined speed and steering position based on control information from the processing device 100 and the operation content of the driver control device 270. The driving control device 300 includes, for example, a driving force output device 310, a brake device 320, and a steering device 330. The driving force output device 310 and the brake device 320 are examples of a "speed control device". The steering device 330 is an example of a "steering control device". The speed control device and the steering control device are examples of actuator devices including actuators (operating parts). Other actuator devices included in the vehicle M include an air conditioning system, a power window system (automatic window opening and closing device), a wiper system, etc.

[0044] The driving force output device 310 outputs driving force (torque) to the drive wheels for the vehicle M to move. The driving force output device 310 includes, for example, a combination of an internal combustion engine, an electric motor, and a transmission, and an ECU that controls them. The ECU controls the above configuration according to information input from the first processing unit 120 or the second processing unit 140, or information input from the accelerator pedal of the driver control unit 270.

[0045] The brake system 320 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to information input from the first processing unit 120 or the second processing unit 140, or from the brake pedal of the driver control unit 270, so that brake torque corresponding to the braking operation is output to each wheel. The brake system 320 may be equipped with a backup mechanism that transmits the hydraulic pressure generated by the operation of the brake pedal to the cylinder via a master cylinder. The brake system 320 is not limited to the configuration described above, and may also be an electronically controlled hydraulic brake system that controls an actuator according to information input from the first processing unit 120 or the second processing unit 140 to transmit hydraulic pressure from the master cylinder to the cylinder.

[0046] The steering device 330 includes, for example, a steering ECU and an electric motor. The electric motor, for example, applies force to a rack and pinion mechanism to change the direction of the steering wheels. The steering ECU drives the electric motor to change the direction of the steering wheels according to information input from the first processing unit 120 or the second processing unit 140, or information input from the steering wheel of the driver control unit 270.

[0047] The power supply unit 400 supplies power to each component of the vehicle system 1, including at least the external sensor 10 and the processing unit 100. The power supply unit 400 comprises, for example, a first power supply unit 410 and a second power supply unit 420. The first power supply unit 410 and the second power supply unit 420 are, for example, rechargeable batteries (secondary batteries). The power supply unit 400 supplies power to each component from at least one of the first power supply unit 410 and the second power supply unit 420. For example, the first power supply unit 410 supplies power to, for example, a surround camera (first sensor) 11, a radar device (third sensor) 16, and a first processing unit 120, while the second power supply unit 420 supplies power to, for example, an MVC (second sensor) 13 and a second processing unit 140. Furthermore, the power supply unit 400 may be redundant so that if a malfunction occurs in either the first power supply unit 410 or the second power supply unit 420, power will be supplied from the other. Details of the power supply by the power supply unit 400 will be described later.

[0048] [First processing unit 120] Next, the functional configuration of the first processing unit 120 will be described in detail. The first processing unit 120 includes, for example, a first recognition unit 122, a first processing unit 124, and a first vehicle control unit 126.

[0049] The first recognition unit 122 performs sensor fusion processing on the outputs (detection results) from at least some of the multiple sensors included in the external sensor 10 to recognize the surrounding conditions of the vehicle M. For example, the first recognition unit 122 performs sensor fusion processing using the detection results of the surround camera 11, LIDAR 12, and radar device 16. In addition to (or instead of) the types of external sensors 10 described above, the first recognition unit 122 may also perform sensor fusion processing using detection results from other configurations (e.g., MVC 13, sonar 14, camera 15). For example, the first recognition unit 122 recognizes the position, type, speed, etc., of objects present around the vehicle M (within a predetermined distance) from the results of sensor fusion processing. The position of an object is recognized as a position on an absolute coordinate system with a representative point of the vehicle M (such as the center of gravity or the center of the drive axis) as the origin, and is used for control. The position of an object may be represented by a representative point such as the center of gravity or a corner of the object, or it may be represented by a represented region. The "state" of an object may include its acceleration, jerk, or "action state" (for example, whether or not it is changing lanes or attempting to change lanes).

[0050] Furthermore, the first recognition unit 122 recognizes road markings around the vehicle M from images acquired by the external sensor 10, for example, and recognizes the lane in which the vehicle M is traveling (driving lane). In this case, the first recognition unit 122 may recognize the driving lane by comparing the road marking pattern (for example, an arrangement of solid and dashed lines) obtained from map information 282 based on the vehicle M's position information with the road marking pattern around the vehicle M acquired by the external sensor 10. In addition, the first recognition unit 122 may recognize the driving lane not only by road markings, but also by recognizing road boundaries (road boundaries) including shoulders, curbs, median strips, guardrails, etc., from images and map information 282. In this recognition, the position of the vehicle M acquired from the vehicle sensor 230 and the navigation device 240, as well as the processing results from the INS (Inertial Navigation System), may be taken into consideration. Furthermore, the first recognition unit 122 recognizes stop lines, obstacles, red lights, toll booths, and other road events.

[0051] Furthermore, when recognizing a driving lane, the first recognition unit 122 recognizes the position and orientation of the vehicle M relative to the driving lane. For example, the first recognition unit 122 may recognize the deviation of the vehicle M's reference point from the center of the lane, and the angle the vehicle M makes with respect to a line connecting the centers of the lanes in the direction of travel, as the relative position and orientation of the vehicle M relative to the driving lane. Alternatively, the first recognition unit 122 may recognize the position of the vehicle M's reference point relative to any side edge of the driving lane (road marking or road boundary), etc., as the relative position of the vehicle M relative to the driving lane.

[0052] The first recognition unit 122 implements, for example, functions using AI (Artificial Intelligence) and functions using a pre-defined model in parallel. For example, the function of "recognizing intersections" may be implemented by performing intersection recognition using deep learning, etc., and recognition based on pre-defined conditions (such as pattern-matchable signals and road markings) in parallel, and then scoring both and evaluating them comprehensively. This ensures the reliability of driving control such as autonomous driving.

[0053] The first processing unit 124 performs driving control, including automatic driving of the vehicle M, based on the recognition result from the first recognition unit 122. Figure 2 is a functional configuration diagram of the first processing unit 124 of an embodiment. The first processing unit 124 includes, for example, an action plan generation unit 124A and a mode determination unit 124B.

[0054] The action plan generation unit 124A, in principle, drives along the recommended lane determined by the MPU 250, and further generates a target trajectory for the vehicle M to travel in the future, automatically (without driver intervention) in accordance with the surrounding conditions of the vehicle M. The target trajectory includes, for example, a velocity element. For example, the target trajectory is represented as a sequence of points (trajectory points) that the vehicle M should reach. The trajectory points are points that the vehicle M should reach at predetermined travel distances (e.g., a few meters) along the road, and separately, target velocity and target acceleration at predetermined sampling times (e.g., a few tenths of a second) are generated as part of the target trajectory. Alternatively, the trajectory points may be the positions that the vehicle M should reach at each sampling time. In this case, the target velocity and target acceleration information is represented by the intervals between trajectory points.

[0055] The action plan generation unit 124A may set autonomous driving events when generating a target trajectory. Autonomous driving events include constant speed driving events, low-speed follow driving events, lane change events, branching events, merging events, takeover events, and degraded driving events. The action plan generation unit 124A generates a target trajectory according to the activated event.

[0056] The mode determination unit 124B determines the driving mode of the vehicle M to be one of several driving modes in which the tasks assigned to the driver differ. The mode determination unit 124B includes, for example, a driver state determination unit 124B1 and a mode change processing unit 124B2.

[0057] Here, the vehicle system 1 can execute multiple vehicle M driving modes. These multiple driving modes are, for example, modes with different control states, i.e., different degrees of automation in the driving control of the vehicle M. A high degree of automation means that the vehicle system 1 controls the vehicle M to a high degree, or in other words, the degree to which the driver intervenes in the control (driving operation) of the vehicle M is low. Depending on the degree of automation, the tasks imposed on the driver differ. For example, the higher the degree of automation, the lighter the tasks. Tasks include, for example, the driver monitoring the road ahead, gripping the steering wheel, and accelerating and decelerating. For example, in a driving mode with a high degree of automation, the driver is not required to monitor the road ahead, grip the steering wheel, or operate the acceleration and deceleration, and automated driving is performed, with the steering and speed control of the vehicle M being handled automatically. "Forward" means the space in the direction of travel of the vehicle M as seen through the front windshield. For example, on an expressway or other road exclusively for motor vehicles, if vehicle M is traveling at a predetermined speed (e.g., around 60 km / h) or less, and conditions such as the presence of a preceding vehicle to be followed are met, a driving mode is executed in which the above tasks are not imposed on the driver. This driving mode is sometimes referred to as TJP. If these conditions are no longer met, the mode determination unit 124B changes the driving mode to another driving mode.

[0058] The mode determination unit 124B, based on information obtained from the driver control unit 270, driver monitor camera 260, etc. (for example, the driver gripping the steering wheel and monitoring the road ahead), determines that the tasks related to the determined driving mode (hereinafter referred to as the current driving mode) are not being performed by the driver, and changes the driving mode of the vehicle M to a driving mode with more demanding tasks. For example, in a driving mode with a high degree of automation, if the driver is in a position where they cannot switch to manual driving in response to a request from the system (for example, if they continue to look outside the permissible area or if signs of difficulty in driving are detected), the mode determination unit 124B uses the HMI 220 and a predetermined output unit that prompts the driver to grip the steering wheel to encourage the driver to switch to manual driving. If the driver does not comply, the mode determination unit 124B performs degraded driving control, such as pulling the vehicle M to the side of the road and gradually stopping it, thereby stopping automatic driving. After stopping automatic driving, the vehicle M switches to a driving mode with a low degree of automation, and the vehicle M can be started by the driver's manual operation. The same applies to "stopping automatic driving" below.

[0059] The driver state determination unit 124B1 monitors the driver's state for the above-mentioned mode change and determines whether the driver's state is appropriate for the task. For example, the driver state determination unit 124B1 analyzes the image captured by the driver monitor camera 260 to perform posture estimation processing and determines whether the driver is in a position where they cannot switch to manual driving in response to a request from the system. The driver state determination unit 124B1 also analyzes the image captured by the driver monitor camera 260 to perform gaze estimation processing and determines whether the driver is monitoring the road ahead.

[0060] The mode change processing unit 124B2 performs various processes for mode change. For example, the mode change processing unit 124B2 instructs the action plan generation unit 124A to generate a target trajectory for degraded operation, issues operation instructions to the second processing unit 140, and controls the HMI 220 to prompt the driver to take action.

[0061] The first vehicle control unit 126 acquires, for example, information on the target trajectory (trajectory point) generated by the second processing unit 144 and stores it in memory (not shown). The first vehicle control unit 126 controls each of the target actuators (driving force output device 310, brake device 320, and steering device 330) so that the vehicle M passes through the target trajectory generated by the action plan generation unit 124A at the scheduled time. For example, the first vehicle control unit 126 controls the driving force output device 310 and the brake device 320 based on the velocity element associated with the target trajectory stored in memory, etc. Furthermore, the first vehicle control unit 126 controls the steering device 330 according to the curvature of the target trajectory stored in memory, etc. The processing of the first vehicle control unit 126 is realized, for example, by a combination of feedforward control and feedback control. As an example, the first vehicle control unit 126 combines feedforward control according to the curvature of the road in front of the vehicle M and feedback control based on the deviation from the target trajectory to perform driving control so that the vehicle M travels along the target trajectory. Alternatively, the first vehicle control unit 126 may provide information on the target trajectory to the second processing unit 140 and control each of the target actuators (driving force output device 310, brake device 320, and steering device 330) via the second processing unit 140. This allows for the unification of communication lines (instruction systems) to each actuator, eliminating the need for arbitration control, etc. This reduces the processing load, enabling more appropriate (less time-lag) operation control for each target actuator.

[0062] [Second processing unit 140] Next, the functional configuration of the second processing unit 140 will be described in detail. Returning to Figure 1, the second processing unit 140 includes, for example, a second recognition unit 142, a second processing unit 144, and a second vehicle control unit 146.

[0063] The second recognition unit 142 performs sensor fusion processing on the detection results from at least some of the multiple sensors included in the external sensor 10 to recognize the surrounding conditions of the vehicle M. For example, the second recognition unit 142 performs sensor fusion processing using the detection results of the sonar 14 and the camera 15. In addition, the second recognition unit 142 may perform sensor fusion processing using other configurations (for example, a radar device 16 or MVC 13) in addition to (or instead of) the types of external sensors 10 described above. For example, the second recognition unit 142 recognizes the position, type, speed, etc. of objects present around the vehicle M (within a predetermined distance) from the results of the sensor fusion processing. The second recognition unit 142 may have the same functions as the first recognition unit 122, for example. The second recognition unit 142 may be omitted, and the processing results of the first recognition unit 122 described above may be used.

[0064] The second processing unit 144 performs driving control to assist the driver based on the recognition results from the second recognition unit 142. The second processing unit 144 generates a target trajectory for the vehicle M to travel in the future based on the vehicle M's driving state (vehicle M's position and speed) and surrounding conditions (road conditions, the positions of surrounding objects, etc.). The second processing unit 144 may perform driving control of the vehicle M in the same manner as the first processing unit 124.

[0065] The second vehicle control unit 146 acquires, for example, information on the target trajectory (trajectory point) generated by the second processing unit 144 and stores it in memory (not shown). Based on the target trajectory stored in memory, the second vehicle control unit 146 controls each of the target actuators (driving force output device 310, brake device 320, and steering device 330). For example, the second vehicle control unit 146 controls the driving force output device 310 or the brake device 320 based on the target trajectory stored in memory, or controls the steering device 330 according to the curvature of the target trajectory stored in memory. The processing of the second vehicle control unit 146 may be realized by a combination of feedforward control and feedback control, similar to the second vehicle control unit 146. The second vehicle control unit 146 may also perform driving control of the vehicle M based on the target trajectory generated by the first processing unit 124. Furthermore, the second vehicle control unit 146 may output information from the HMI 220 prompting the driver to perform a predetermined driving operation (manual driving) so that the vehicle M travels along the target trajectory.

[0066] [Placement of external sensor 10] Next, the details of the arrangement configuration of the external sensor 10 will be described. Figure 3 is a diagram illustrating an example of the arrangement of the external sensor 10 in an embodiment. In the example in Figure 3, the vehicle M is equipped with five surround cameras 11a to 11e, one Lidar 12, four MVCs 13a to 13d, twelve sonars 14a to 14l, two cameras 15a and 15b, and five radar devices 16a to 16e. Hereafter, unless each of the surround cameras 11a to 11e is described individually, they will be referred to as "surround camera 11". The same applies to the MVCs 13a to 13d, sonars 14a to 14l, cameras 15a and 15b, and radar devices 16a to 16e.

[0067] In this embodiment, each of the surround camera (first sensor) 11 and the MVC (second sensor) 13 consists of a group of cameras that capture images of areas in multiple directions different from the vehicle M (with the vehicle M as the reference). The radar device (third sensor) 16 consists of a group of radar devices that detect objects located in multiple directions different from the vehicle M. The sonar 14 consists of a group of radars directed at objects located in multiple directions different from the vehicle M.

[0068] In the example shown in Figure 3, surround cameras 11a and 11b are installed on the left side of the vehicle body (the main body of vehicle M) and capture images of the area including the left side of vehicle M (in the Y-axis direction in the figure). Surround cameras 11c and 11d are installed on the right side of the vehicle body and capture images of the area including the right side of vehicle M (in the Y-axis direction in the figure). Surround camera 11e is installed on the upper part of the rear windshield of vehicle M (near the roof) and captures images of the area including the rear of vehicle M (in the X-axis direction in the figure). Surround cameras 11a to 11e capture images for the same purpose. Furthermore, the field of view (image range) of surround cameras 11a to 11e may be the same size. Also, the image ranges of surround cameras 11a to 11e may partially overlap.

[0069] LIDAR12 is installed on the top of the vehicle (on the roof) and detects objects in the area including the area in front of the vehicle M (in the X-axis direction in the diagram).

[0070] MVC13a is installed at the front of the vehicle and captures an area including the front of vehicle M. MVC13b is installed near the left side mirror of vehicle M and captures an area including the left side of vehicle M. MVC13c is installed near the right side mirror of vehicle M and captures an area including the right side of vehicle M. MVC13d is installed at the rear of the vehicle and captures an area including the rear of vehicle M. MVC13a to 13d are used for the same purpose. Also, the field of view of MVC13a to 13d may be the same. Furthermore, the imaging ranges of MVC13a to 13d may partially overlap.

[0071] Sonars 14a to 14l are installed, for example, on bumpers at the front and rear ends of the vehicle body. Sonars 14a and 14b are installed at the front end of the vehicle body, sonars 14c and 14d are installed at the front end of the vehicle body, and sonars 14e and 14f are installed on the left and right sides of the front of the vehicle body. In addition, sonars 14g and 14h are installed on the left and right sides of the rear of the vehicle body, and sonars 14i, 14j, 14k, and 14l are installed at the rear end of the vehicle body. Sonars 14a to 14l detect objects present around the vehicle M. Sonars 14a to 14l perform detection processing for the same purpose. Furthermore, the detection ranges of sonars 14a to 14l may be the same size. Also, the detection ranges of sonars 14a to 14l may partially overlap.

[0072] Cameras 15a and 15b are installed, for example, on the top of the front windshield or behind the rearview mirror, and capture images of the area including the front of the vehicle M. One of cameras 15a or 15b may be a telephoto camera capable of capturing images of distant areas. Alternatively, one of cameras 15a or 15b may be a main camera that operates under normal conditions, and the other may be a sub-camera that captures images when the main camera is unable to do so. Furthermore, the imaging ranges of cameras 15a and 15b may partially overlap.

[0073] Radar device 16a is installed near the front of the vehicle body and detects objects in front of the vehicle M. Radar device 16b is installed near the front left of the vehicle body and detects objects in front of the front left and to the left of the vehicle M. Radar device 16c is installed near the front right of the vehicle body and detects objects in front of the front right and to the right of the vehicle M. Radar device 16d is installed near the rear left of the vehicle body and detects objects in the rear left and to the left of the vehicle M. Radar device 16e is installed near the front right of the vehicle body and detects objects in front of the front right and to the right of the vehicle M. Radar devices 16a to 16e perform detection processing for the same purpose. In addition, the detection ranges of radar devices 16a to 16e may be the same size. Also, the detection ranges of radar devices 16a to 16e may partially overlap.

[0074] It should be noted that the number and location of the external sensors 10 are not limited to the example in Figure 3. For example, depending on the grade, generation (version), and functions of the vehicle M, at least some of the placement locations, the number of sensors, and some sensors may be added, removed, or of different types.

[0075] [Examples of applications for external sensors] Next, we will explain some use cases for external sensors used to detect surrounding conditions. Note that the following examples assume no abnormalities (failures) in the external sensor 10, processing unit 100, power supply unit 400, etc., and are not limited to these examples.

[0076] <When driving on a highway> For example, when vehicle M is driving on a highway, the outputs (detection results) of camera 15, radar device 16, and LIDAR 12 are used for determining the start and end of TJP control and for determining follow-me driving. In addition, the outputs of surround camera 11 and MVC 13 may be used in addition to the above-mentioned sensors to detect excessive approach of obstacles, which is a condition for terminating TJP control. Furthermore, the outputs of surround camera 11 and camera 15 are used for detecting the approach of emergency vehicles or construction zones ahead of vehicle M, which are conditions for terminating TJP. The outputs of surround camera 11 and camera 15 are also used for recognizing road markings used in driving control such as LKAS. In addition, the outputs of surround camera 11, camera 15, and radar device 16 are used for driving control such as ALC (merging assistance). Furthermore, the output of camera 15 is used for recognizing toll booths, etc.

[0077] <When driving on public roads> For example, when vehicle M is driving on a public road, the output of camera 15 and LIDAR 12 is used for driving control such as LKAS, hands-off driving (autonomous driving) during traffic jams, and recognition of lane markings in narrow roads and construction zones. In addition, the output of camera 15 and radar device 16 is used for detecting surrounding objects (other vehicles and pedestrians) and recognizing construction zones. Furthermore, the recognition results of surround camera 11 and camera 15 are used for recognizing turn signals of surrounding vehicles, traffic lights, intersections, stop lines, and road signs. In addition, the detection results of camera 15, radar device 16, LIDAR 12, MVC 13, etc. are used for recognizing the drivable area around vehicle M. Furthermore, when detecting objects around vehicle M when parking, exiting a parking space, or turning left or right on narrow roads, the output of sonar 14 and radar device 16 is used.

[0078] [Regarding power supply to the external sensor 10 and the processing unit 100] In this embodiment, a redundant configuration is established so that even if an abnormality such as a power failure occurs in the external sensor 10 or the processing unit 100, for example, detection (recognition) of the surrounding conditions and operation control can continue. Figure 4 is a diagram illustrating an example of a redundant configuration including power supply in this embodiment. In the example in Figure 4, the equipment supplied with power from the first power supply unit 410 (the external sensor 10 and the processing unit 100 in the example in Figure 4) is referred to as the first group Gr1, and the equipment supplied with power from the second power supply unit 420 is referred to as the second group Gr2. Also, in the example in Figure 4, the solid line represents the electrical line EL that supplies power, and the dotted line represents the communication line CL that shows the flow of information (for example, the detection results of each sensor).

[0079] In this embodiment, the first power supply unit 410 supplies power to at least the surround camera (first sensor) 11 and the radar device (third sensor) 16, as well as the first processing unit 120. In the example shown in Figure 4, in addition to the surround camera 11 and the radar device 16, the LIDAR 12 is also supplied with power from the first power supply unit 410. The surround camera 11, LIDAR 12, and radar device 16 output their respective outputs (detection results) to the first processing unit 120. In the example shown in Figure 4, a communication line CL is provided so that the detection results of the radar device 16 can be output to the second processing unit 140.

[0080] In this embodiment, the second power supply unit 420 supplies power to at least the MVC (second sensor) 13 and also to the second processing unit 140. In the example shown in Figure 4, in addition to the MVC 13, the camera 15 and sonar 14 are also supplied with power from the second power supply unit 420. The camera 15, MVC 13, and sonar 14 output their respective detection results to the second processing unit 140. In the example shown in Figure 4, a communication line CL is provided so that the detection results of the MVC 13 and sonar 14 can be output to the first processing unit 120. In this embodiment, for example, driving control related to automatic driving is performed by the equipment of the first group Gr1, but by also using the outputs of the MVC 13 and sonar 14, the surrounding conditions of the vehicle M can be recognized with higher accuracy. In this embodiment, driving control related to driving assistance is performed by the equipment of the second group Gr2, but by also using the output of the radar device 16, the surrounding conditions of the vehicle M can be recognized with higher accuracy.

[0081] As shown in Figure 4, the MVC 13, radar device 16, and sonar 14 are equipped with a communication line (first communication line) CL to the first processing unit 120 and a communication line (second communication line) CL to the second processing unit 140, enabling dual-system sensor output and thus creating a more robust redundant configuration.

[0082] In this embodiment, the first communication line and the second communication line may be communication lines with different communication bandwidths. In this case, the first communication line will be a communication network capable of higher capacity and faster communication than the second communication line. Specifically, the first communication line will be Ethernet and the second communication line will be CAN, but the first and second communication lines may be other communication networks. This will enable high-capacity communication without delay to the first processing unit 120, which requires a large amount of information for driving control such as automatic driving.

[0083] Furthermore, in this embodiment, the first processing unit 120 may have a higher processing capacity than the second processing unit 140. Processing capacity is a value that can be compared using, for example, TOPS (Tera Operations Per Second), but is not limited to this. This allows for faster processing of more sensor information, enabling the execution of autonomous driving control without delay.

[0084] Furthermore, in this embodiment, for example, the first processing unit 120 may recognize the surrounding conditions of the vehicle M based on the detection results of the external sensor 10, generate a future target trajectory for the vehicle M based on the recognition results and output it to the second processing unit 140, and the second processing unit 140 may output instructions to each target actuator device (driving force output device 310, brake device 320, steering device 330) based on the target trajectory generated by the first processing unit 120. In this way, by unifying the communication lines to each target actuator device, arbitration control and the like can be made unnecessary.

[0085] Furthermore, each actuator device (driving force output device 310, brake device 320, steering device 330) may be configured to have a communication line with the first processing device 120 in addition to the communication line with the second processing device 140. This allows, for example, the second group Gr2 (second processing device 140) to fail and perform degraded operation, to operate each actuator device by instruction from the first processing device 120 of the first group Gr1 through the redundant configuration. In addition, in this embodiment, by constructing the redundant configuration of the above-mentioned communication lines not for all actuator devices included in the vehicle M, but for actuator devices related to the driving of the vehicle M (driving force output device 310, brake device 320, steering device 330), it is possible to achieve more appropriate operation control during degraded operation while suppressing costs.

[0086] Figure 5 is a schematic diagram showing the detection range of each sensor included in the external sensor 10 of the embodiment. In the example of Figure 5, the detection range of the external sensor 10 included in the first group Gr1, which is powered by the first power supply unit 410, and the detection range of the external sensor 10 included in the second group Gr2, which is powered by the second power supply unit 420 are shown. In the example of Figure 5, the separation of power supply by the first power supply unit 410 and the second power supply unit 420 and the three-dimensional detection area AR of each external sensor are schematically shown.

[0087] Figure 5 shows (A) to (C) of the first group Gr1, indicating the detection range A1a of the surround camera 11, the detection range A1b of the LIDAR 12, and the detection range A1c of the radar device 16 relative to the vehicle M. Figure 5 also shows (A) to (C) of the second group Gr2, indicating the detection range A2a of the camera 15, the detection range A2b of the MVC 13, and the detection range A2c of the sonar 14 relative to the vehicle M. By combining the detection ranges of (A) to (C) of the first group Gr1, the surrounding conditions of the vehicle M can be detected using only the power from the first power supply unit 410, and by combining the detection ranges of (A) to (C) of the second group Gr2, the surrounding conditions of the vehicle M can be detected using only the power from the second power supply unit 420. Thus, in this embodiment, a redundant configuration is constructed in which the surrounding conditions of the vehicle M can be detected by supplying power from one of the two power systems. As a result, even if one of the power supply units fails, the surrounding conditions of vehicle M can be detected based on the detection results of external sensors that are operated by power supplied from other power supply units, and corresponding driving control can be continuously executed based on the detection results.

[0088] [Operation control in case of loss] Next, the operation control in the event of failure in the redundant configuration of this embodiment will be specifically described. For example, the failure determination unit 160 determines whether a failure has occurred in either the first group Gr1 or the second group Gr2. For example, the failure determination unit 160 acquires the status of the first power supply unit 410 and the second power supply unit 420 from the vehicle sensor 230, and determines that there is a failure (abnormality) in the target group if, for example, the temperature is above a first threshold, the battery level is below a second threshold, or the output current value or voltage value exceeds a predetermined range. Alternatively, the failure determination unit 160 may acquire the status of the electrical line EL and communication line CL from the vehicle sensor 230, and if it determines that there is an abnormality such as a disconnection or other communication error, it may determine that there is a failure in the target group.

[0089] For example, if the failure determination unit 160 determines that the first group Gr1 is lost, the processing unit 100 uses the external sensor 10 included in the second group Gr2 to detect the surrounding conditions of the vehicle M and determines the stopping position of the vehicle M, and the second processing unit 140 executes driving control (degenerate operation) to move the vehicle M to the determined stopping position and stop it. Also, if the failure determination unit 160 determines that the second group Gr2 is lost, the processing unit 100 uses the external sensor 10 included in the first group Gr1 to detect the surrounding conditions of the vehicle M and determines the stopping position of the vehicle M, and the first processing unit 120 executes driving control (degenerate operation) to move the vehicle M to the determined stopping position and stop it.

[0090] Thus, since the first group Gr1 and the second group Gr2 each have a redundant configuration capable of detecting the situation all around (360 degrees) of the vehicle M, even if either the first group Gr1 or the second group Gr2 fails, as described above, the surrounding situation can still be recognized using the other group. Furthermore, according to this embodiment, since the sonar 14 and MVC 13 can be used to detect the surrounding situation at a closer distance with fewer blind spots, when stopping the vehicle M in degraded operation, it is possible to prevent the vehicle from stopping in no-parking zones (for example, in front of police stations, fire stations, near fire hydrants, etc.).

[0091] [Differentiation] In the embodiment, at least one of the first processing unit 120 and the second processing unit 140 may be composed of a plurality of different ECUs. Furthermore, the vehicle system 1 may include other different ECUs.

[0092] In this embodiment, the failure detection unit 160 may perform abnormality (failure) detection for each piece of equipment within a group, instead of performing failure detection for the group as a whole. For example, if the failure detection unit 160 determines that there is an abnormality only in the first power supply unit 410, the processing unit 100 may control the supply of power from the second power supply unit 420 to the equipment included in the first group Gr1. Alternatively, for example, if the failure detection unit 160 determines that there is a failure in the radar device 16, the processing unit 100 may output the detection results of the MVC 13 and sonar 14 to the first processing unit 120 to recognize the surrounding conditions. This makes it possible to improve the continuity of operation control through partial changes.

[0093] Furthermore, in this embodiment, if the failure detection unit 160 determines that there are failures in both the first group Gr1 and the second group Gr2, the HMI 220 notifies the driver that there are failures and therefore operation control cannot be continued (and instructs the driver to perform manual operation), and terminates operation control.

[0094] According to the embodiment described above, the vehicle control device includes an external sensor 10 for detecting the surrounding conditions of the vehicle M, a processing device 100 for performing predetermined processing related to the vehicle M based on the output of the external sensor 10, a driving control device 300 (an example of a control device) for controlling the driving of at least the vehicle M based on the processing results of the processing device 100, and a power supply unit 400 for supplying power to at least the external sensor 10 and the processing device 100. The external sensor 10 includes a surround camera 11 (an example of a first sensor), an MVC 13 (an example of a second sensor), and a radar device 16 (a third sensor), and the processing device 100 is The system includes a first processing unit 120 and a second processing unit 140, with the outputs of the surround camera 11 and MVC 13 being output to the first processing unit 120, and the output of the radar device 16 being output to the second processing unit 140. The power supply unit 400 includes a first power supply unit 410 and a second power supply unit 420, with the first power supply unit 410 supplying power to at least the surround camera 11, the radar device 16, and the first processing unit 120, and the second power supply unit 420 supplying power to at least the MVC 13 and the second processing unit 140. This allows for the construction of a more appropriate redundant configuration for external sensors mounted on a vehicle.

[0095] Specifically, according to the embodiment, by constructing a redundant power supply configuration for the external sensor 10, the surrounding conditions can be recognized even if some failures occur, allowing for more appropriate operation control such as degraded operation. Furthermore, according to the embodiment, a more robust redundant configuration is possible by providing duplicate output systems from the sensors. In addition, in the surrounding conditions recognition processing in the first processing unit 120 and the second processing unit 140, the outputs of sensors included in other groups can be used, allowing for more accurate recognition of the surrounding conditions based on the outputs of more sensors. Furthermore, according to the embodiment, arbitration control can be eliminated by unifying the communication lines to the actuator devices.

[0096] Although embodiments for carrying out the present invention have been described above using examples, the present invention is not limited in any way to these embodiments, and various modifications and substitutions can be made without departing from the spirit of the present invention. [Explanation of Symbols]

[0097] 1...Vehicle system, 10...External sensor, 11...Surround camera, 12...LIDAR, 13...MVC, 14...Sonar, 15...Camera, 16...Radar device, 100...Processing device, 120...First processing device, 122...First recognition unit, 124...First processing unit, 126...First vehicle control unit, 140...Second processing device, 142...Second recognition unit, 144...Second processing unit, 146...Second vehicle control unit, 160 ...Loss detection unit, 210...Communication device, 220...HMI, 230...Vehicle sensor, 240...Navigation device, 250...MPU, 260...Driver monitor camera, 270...Driver control unit, 280...Storage unit, 300...Driving control device, 310...Driving force output device, 320...Brake device, 330...Steering device, 400...Power supply unit, 410...First power supply unit, 420...Second power supply unit

Claims

1. External sensors that detect the surrounding conditions of the vehicle, A processing device that performs predetermined processing related to the vehicle based on the output of the external sensor, A control device that controls the movement of the vehicle based on the processing results of the processing device, It comprises at least the external sensor and the processing unit, and a power supply unit that supplies power to the processing unit. The aforementioned external sensor includes a first sensor, a second sensor, and a third sensor. The aforementioned processing apparatus includes a first processing apparatus and a second processing apparatus, The outputs of the first sensor and the second sensor are output to the first processing unit. The output of the third sensor is output to the second processing unit. The aforementioned power supply unit includes a first power supply unit and a second power supply unit. The first power supply unit supplies power to at least the first sensor, the third sensor, and the first processing unit. The second power supply unit supplies power to at least the second sensor and the second processing unit. Vehicle control system.

2. Each of the first and second sensors is a plurality of cameras that capture images of areas in multiple different directions from the vehicle, The third sensor is a plurality of radar devices that detect objects located in multiple different directions from the vehicle. The vehicle control device according to claim 1.

3. Each of the first and second sensors is a plurality of cameras having the same purpose or the same field of view. The third sensor is a plurality of radar devices that detect objects in multiple different directions from the vehicle. The vehicle control device according to claim 1.

4. The second sensor is connected to the first processing unit by a first communication line, and is connected to the second processing unit by a second communication line different from the first communication line. The vehicle control device according to claim 1.

5. The first communication line is a communication network capable of higher capacity and faster communication than the second communication line. The vehicle control device according to claim 4.

6. The first processing unit has a higher processing capacity than the second processing unit. The vehicle control device according to claim 1.

7. The first processing unit generates the target trajectory of the vehicle based on the output of the external sensor, The second processing unit outputs instructions to the actuator device included in the control unit based on the target trajectory generated by the first processing unit. The vehicle control device according to claim 1.

8. The actuator device includes a steering control device that controls the steering of the vehicle and a speed control device that controls the speed of the vehicle. The steering control device and the speed control device each include a communication line capable of receiving instructions from the second processing device and a communication line capable of receiving instructions from the first processing device. The vehicle control device according to claim 7.

9. The first power supply unit supplies power to a first group consisting of at least the first sensor, the third sensor, and the first processing unit. The second power supply unit supplies power to a second group consisting of at least the second sensor and the second processing unit. If the control device determines that one of the first group and the second group is missing, it uses the other group to determine the stopping position of the vehicle based on the surrounding conditions of the vehicle, and performs vehicle control to drive the vehicle to the determined stopping position. The vehicle control device according to claim 1.

10. External sensors detect the surrounding conditions of the vehicle, The processing unit performs predetermined processing related to the vehicle based on the output of the external sensor. Based on the processing results from the aforementioned processing device, the vehicle's movement is controlled to at least The power supply unit supplies power to at least the external sensor and the processing device. The aforementioned external sensor includes a first sensor, a second sensor, and a third sensor. The aforementioned processing apparatus includes a first processing apparatus and a second processing apparatus, The outputs of the first sensor and the second sensor are output to the first processing unit. The output of the third sensor is output to the second processing unit. The aforementioned power supply unit includes a first power supply unit and a second power supply unit. The first power supply unit supplies power to at least the first sensor, the third sensor, and the first processing unit. The second power supply unit supplies power to at least the second sensor and the second processing unit. Vehicle control method.

Images