Vehicle control device

Abstract

A vehicle control device includes a vehicle control section that transmits target control amount information to a suspension system that performs vehicle control for mitigating an impact on a vehicle body of a vehicle from a road surface; and a learning section that creates a Q table by acquiring, for each of one or more road surface shape characteristic amounts of a plurality of patterns, a Q value that is an evaluation value that is larger in value the smaller the impact on the vehicle body of the vehicle from the road surface in a case where simulation is performed based on one or more control parameters of M patterns with respect to the vehicle control section, and creates, for each of the road surface shape characteristic amounts of the plurality of patterns, Q value relationship information that holds a set of the first N Q values that are larger in value and the control parameters corresponding to the Q values, using simulation processing and Q learning.

Description

Technical Field

[0001] This invention relates to vehicle control devices. Background Technology

[0002] In the past, vehicle control based on the suspension system was used to mitigate the impact of road surfaces on the vehicle body (such as passenger cars). The suspension system, for example, works with the ECU (Electronic Control Unit) to control damping force adjustment devices such as solenoid actuators to mitigate the impact of road surfaces on the vehicle body.

[0003] In this case, for example, the following method can be used. First, by using deep learning with a DNN (Deep Neural Network), a learning model is created in the simulation process to learn the relationship between road surface shape features of multiple patterns and control parameters related to the control of the suspension system. Then, when the vehicle is driving on an actual road surface (hereinafter referred to as the "real environment"), vehicle control based on the suspension system is performed using the control parameters obtained based on the learning model and the road surface shape features of the real environment.

[0004] Patent Document 1: Japanese Patent Application Publication No. 2021-109517

[0005] However, in the aforementioned prior art, when there are differences between the simulated and actual environments, it is sometimes impossible to obtain control parameters suitable for the actual environment. Summary of the Invention

[0006] Therefore, the present invention has been made in view of the above circumstances, and its object is to provide a vehicle control device that can obtain control parameters suitable for the actual environment even when there are differences between the simulated and actual environments.

[0007] This invention is a vehicle control device mounted on a vehicle, wherein, have: The vehicle control unit sends target control quantity information to the suspension system, which performs vehicle control to mitigate impacts on the vehicle body from the road surface; and The learning department is configured as follows: Using simulation processing, Q-learning as reinforcement learning is employed to acquire Q-values ​​for each of more than one road surface shape feature quantities in a plurality of modes, thereby creating a Q-table. The Q-values ​​are evaluation values ​​where, under simulations based on more than one control parameter for each of M modes of the vehicle control unit, the smaller the impact on the vehicle body from the road surface, the higher the value. Based on the Q-table, for each of the road surface shape feature quantities in the plurality of modes, Q-value relationship information is created, including the top N Q-values ​​that maintain the largest Q-values ​​and the set of control parameters corresponding to those Q-values. M is a specified integer of 3 or higher, and N is an integer satisfying 2 ≤ N < M. When the vehicle is driving on the actual road surface, the control parameter corresponding to any one of the Q values ​​in the Q value relationship information corresponding to the road surface shape feature quantity is sent to the vehicle control unit to control the vehicle, using the Q value in this case to update the Q value in the Q value relationship information, and the update process is repeated. When the control parameter is selected for the acquired road surface shape feature quantity, and all N Q values ​​for the road surface shape feature quantity in the Q value relationship information have been updated, the control parameter corresponding to the largest Q value among the retained Q values ​​is selected and sent to the vehicle control unit.

[0008] According to the vehicle control device of the present invention, control parameters related to the control of the vehicle's suspension system can be obtained that are suitable for the actual environment, even when there are differences between the simulated and actual environments. Attached Figure Description

[0009] Figure 1 It is a perspective view showing the state of a portion of the vehicle's passenger compartment in a perspective implementation.

[0010] Figure 2 It is a top view showing the state of the vehicle body in the perspective implementation method.

[0011] Figure 3 This is a block diagram illustrating the structure of the vehicle control system of the vehicle described in the embodiment.

[0012] Figure 4 This is an explanatory diagram of the simulation processing performed by the vehicle control device according to the embodiment.

[0013] Figure 5 This is an explanatory diagram illustrating the processing performed by the vehicle control device according to the implementation method in a real-world environment.

[0014] Figure 6 This is a diagram showing an example of a Q-table, etc., in an implementation method.

[0015] Figure 7 This is a diagram illustrating an example of Q-value relationship information in an implementation scheme.

[0016] Figure 8 This is a diagram illustrating an example of location relationship information in an implementation method, which includes a set of road surface shape features and location information for each location.

[0017] Figure 9 This is an explanatory diagram of the updating of positional relationship information in the implementation method.

[0018] Figure 10 This is a flowchart illustrating the process of creating Q-value relationship information performed by the vehicle control device of the implementation method.

[0019] Figure 11 This is a flowchart illustrating the processing performed by the vehicle control device of the embodiment in a real environment.

[0020] Explanation of reference numerals in the attached figures: 1: Vehicle; 14: ECU (Vehicle Control Unit); 15, 15a, 15c: Camera Unit; 30: Suspension System; 31: Position Information Sensor; 102, 103, 105, 106: Actuators; 141: Vehicle Motion Data Processing Unit; 142: Road Surface Shape Output Unit; 143: Report Calculation Unit; 144: Learning Unit; 145: Vehicle Control Unit Detailed Implementation

[0021] The following describes exemplary embodiments of the present invention. The structures of the embodiments shown below, as well as the functions, results, and effects resulting from these structures, are examples. The present invention can be implemented in ways other than those disclosed in the following embodiments, and at least one of various effects or derived effects based on the basic structure can be obtained. Furthermore, for the sake of simplicity, for example, "acceleration data" or "acceleration information" will sometimes be abbreviated to "acceleration," and the description of "data" or "information" will be omitted.

[0022] The vehicle 1 in this embodiment can be, for example, an internal combustion engine vehicle (ECV) powered by an internal combustion engine (not shown), an electric vehicle (EV) or a fuel cell vehicle (FCV) powered by an electric motor (not shown), a hybrid vehicle powered by both, or a vehicle with other drive sources. Furthermore, the vehicle 1 can be equipped with various transmission devices, as well as various devices, systems, or components required to drive the internal combustion engine or electric motor. Additionally, the method, number, and layout of devices related to the drive of the wheels 3 in the vehicle 1 can be configured in various ways.

[0023] Figure 1 This is a perspective view showing the state of a portion of the carriage of vehicle 1 in a perspective embodiment. Figure 2 It is a top view showing the state of the vehicle body in the perspective implementation method. Figure 3 This is a block diagram illustrating the structure of the vehicle control system of the vehicle described in the embodiment.

[0024] First, use Figures 1-3 An example of the structure of vehicle 1 in this embodiment will be described. For example... Figure 1 As illustrated, the vehicle body 2 constitutes a passenger compartment 2a (not shown) for passengers. Inside the passenger compartment 2a, in a position facing the driver's seat 2b, there are a steering unit 4, an acceleration control unit 5, a braking control unit 6, a transmission control unit 7, etc.

[0025] The steering unit 4 is, for example, a steering wheel protruding from the instrument panel 24. The accelerator control unit 5 is, for example, the accelerator pedal located under the driver's feet. The brake control unit 6 is, for example, the brake pedal located under the driver's feet. The gear shift control unit 7 is, for example, a gear shift lever protruding from the center console. Furthermore, the steering unit 4, accelerator control unit 5, brake control unit 6, and gear shift control unit 7 are not limited to these.

[0026] In addition, a display device 8 serving as a display output unit and a sound output device 9 serving as a sound output unit are provided inside the carriage 2a. The display device 8 is, for example, an LCD (Liquid Crystal Display) or an OELD (Organic Electroluminescent Display). The sound output device 9 is, for example, a speaker. Furthermore, the display device 8 is covered by a transparent operation input section 10, such as a touch panel. Passengers can visually view the image displayed on the screen of the display device 8 through the operation input section 10. In addition, passengers can perform operation input by touching, pressing, or moving the operation input section 10 at the position corresponding to the image displayed on the screen of the display device 8.

[0027] These components, such as the display device 8, sound output device 9, and operation input unit 10, are provided, for example, in the monitoring device 11 located in the center of the dashboard 24 in the vehicle width direction (left-right direction). The monitoring device 11 can have operation input units (not shown), such as switches, knobs, joysticks, and buttons. Additionally, a sound output device (not shown) can be installed in another location within the passenger compartment 2a, different from the monitoring device 11, and sound can be output from the sound output device 9 of the monitoring device 11 and other sound output devices. Furthermore, the monitoring device 11 can, for example, be used as a navigation system or an audio system. Furthermore, a display device 12, different from the display device 8, is provided within the passenger compartment 2a.

[0028] In addition, such as Figure 2As illustrated, vehicle 1 is, for example, a four-wheeled car, having two front wheels 3F and two rear wheels 3R. All four wheels 3 are configured to be steerable. Figure 3 As illustrated, vehicle 1 has a steering system 13 that steers at least two wheels 3.

[0029] like Figure 3 As illustrated, the steering system 13 includes actuators 101 and 104 and a torque sensor 13b. The steering system 13 is electrically controlled by an ECU 14 (Electronic Control Unit) to actuate the actuators 101 and 104. Here, as... Figure 2 As shown, actuator 101 is connected to the front wheel 3F and is a front steering actuator for steering the front wheel 3F. Additionally, actuator 104 is connected to the rear wheel 3R and is a rear steering actuator for steering the rear wheel 3R.

[0030] The steering system 13 is, for example, an electric power steering system or a SBW (Steer By Wire) system. The steering system 13 supplements the steering force by applying torque, or auxiliary torque, to the steering unit 4 via actuators 101 and 104, or by steering the wheels 3 via actuators 101 and 104. In this case, actuators 101 and 104 can steer one wheel 3 or multiple wheels 3. Additionally, the torque sensor 13b detects, for example, the torque applied to the steering unit 4 by the driver.

[0031] In addition, such as Figure 3 As illustrated, the vehicle body 2 is equipped with, for example, eight imaging units 15a to 15f, forming a plurality of imaging units 15. Each imaging unit 15 is, for example, a digital camera with a built-in imaging element such as a CCD (Charge Coupled Device) or a CIS (CMOS Image Sensor). Each imaging unit 15 can output video data at a specified frame rate. Each imaging unit 15 has a wide-angle lens or a fisheye lens, enabling it to capture a range of, for example, 140° to 190° in the horizontal direction. Furthermore, the optical axis of each imaging unit 15 is set to point diagonally downwards. Therefore, each imaging unit 15 sequentially captures images of the external environment surrounding the vehicle body 2, including the road surface where the vehicle 1 can move and the area where the vehicle 1 can park, and outputs these images (image data).

[0032] Camera unit 15a is located, for example, at the rear end 2e of the vehicle body 2, and is installed on the wall below the trunk door 2h. Camera unit 15ba is located, for example, at the right end of the vehicle body 2 in the vehicle width direction, and is installed in front of the door rearview mirror 2g, which is a right-side protrusion. Camera unit 15bb is located, for example, at the right end of the vehicle body 2 in the vehicle width direction, and is installed behind the door rearview mirror 2g, which is a right-side protrusion.

[0033] The camera unit 15c is located, for example, at the front end of the vehicle body 2 in the longitudinal direction, such as on the front bumper. The camera unit 15da is located, for example, at the left end of the vehicle body 2 in the width direction, and is located in front of the door rearview mirror 2g, which is a left-side protrusion. The camera unit 15db is located, for example, at the left end of the vehicle body 2 in the width direction, and is located behind the door rearview mirror 2g, which is a left-side protrusion.

[0034] The camera unit 15e is located, for example, on the right side of the vehicle body 2, at the right end in the vehicle width direction, near the right-side door. The camera unit 15f is located, for example, on the left side of the vehicle body 2, at the left end in the vehicle width direction, near the left-side door.

[0035] The ECU 14 can perform computational processing and image processing based on image data obtained from the plurality of imaging units 15, generating images with a wider viewing angle, or generating virtual overhead images of the vehicle 1 viewed from above. Furthermore, the overhead image can also be called a top-down view. In this embodiment, the area surrounding the vehicle 1 is captured by each of the imaging units 15.

[0036] In addition, such as Figure 1 As illustrated, four ranging units 16a-16d and eight ranging units 17a-17h are provided on the vehicle body 2, forming a plurality of ranging units 16 and 17. The ranging units 16 and 17 are, for example, sonar units that emit ultrasonic waves and capture their reflected waves. Sonar is also called a sonar sensor or ultrasonic detector. The ECU 14 can determine the presence or absence of objects such as obstacles around the vehicle 1 and the distance to those objects based on the detection results of the ranging units 16 and 17. That is, the ranging units 16 and 17 are examples of object detection units. Furthermore, the ranging unit 17 is used, for example, for detecting objects at relatively close distances, while the ranging unit 16 is used, for example, for detecting objects at relatively long distances. Additionally, the ranging unit 17 can be used, for example, for detecting objects in front of and behind the vehicle 1, while the ranging unit 16 can be used for detecting objects to the side of the vehicle 1.

[0037] In addition, such as Figure 3As illustrated, in the vehicle control system 100, in addition to the ECU 14, monitoring device 11, steering system 13, distance measuring units 16 and 17, the braking system 18, suspension system 30, steering angle sensor 19, throttle sensor 20, shift sensor 21, wheel speed sensor 22, acceleration sensor 25, vehicle height sensor 26, spring sensor 27, actuator 107, etc., are also electrically connected via the in-vehicle network 23, which serves as an electrical communication line. The in-vehicle network 23 is configured, for example, as a CAN (controller area network). The ECU 14 is an example of an estimation device and a vehicle control device.

[0038] The ECU 14 controls the steering system 13, braking system 18, suspension system 30, actuator 107, etc., by sending control signals via the in-vehicle network 23. In addition, the ECU 14 receives detection results from torque sensor 13b, brake sensor 18b, steering angle sensor 19, distance measuring unit 16, distance measuring unit 17, throttle sensor 20, shift sensor 21, wheel speed sensor 22, acceleration sensor 25, vehicle height sensor 26, etc., as well as operation signals from operation input unit 10, etc., via the in-vehicle network 23.

[0039] ECU14 may include, for example, a CPU (Central Processing Unit) 14a, a ROM (Read-Only Memory) 14b, a RAM (Random Access Memory) 14c, a display control unit 14d, a sound control unit 14e, an SSD (Solid State Drive) 14f, etc.

[0040] CPU 14a performs various operations and controls, such as image processing associated with images displayed by display devices 8 and 12, determination of the target position of vehicle 1, calculation of the movement path of vehicle 1, judgment of whether there is interference with objects, automatic control of vehicle 1, deactivation of automatic control, attenuation control, spring constant switching control, steering control, stabilizer control, and drive force control. CPU 14a reads programs installed and stored in non-volatile storage devices such as ROM 14b and performs operations according to the programs.

[0041] RAM 14c temporarily stores various data used in the operations of CPU 14a. Additionally, the display control unit 14d, in the ECU 14's processing, primarily performs image processing using image data obtained from the imaging unit 15 and compositing image data displayed by the display device 8. Furthermore, the sound control unit 14e, in the ECU 14's processing, primarily processes the sound data output by the sound output device 9. The SSD 14f is a rewritable, non-volatile storage unit that can store data even when the ECU 14's power is off. Moreover, CPU 14a, ROM 14b, RAM 14c, etc., can be integrated into the same package. Alternatively, ECU 14 can be configured to use other logic processors or logic circuits such as a DSP (Digital Signal Processor) instead of CPU 14a. Furthermore, an HDD (Hard Disk Drive) can be provided instead of SSD 14f, or SSD 14f or HDD can be installed separately from ECU 14.

[0042] The location information sensor 31 is a sensor that acquires the current location of vehicle 1, such as a GPS (Global Positioning System) receiver. The location information sensor 31 sends the acquired current location as location information to CPU 14a.

[0043] The braking system 18 includes, for example, ABS (Anti-lock Braking System) to prevent brake lock-up, ESC (Electronic Stability Control) to prevent skidding of the vehicle 1 during cornering, an electric braking system to enhance braking force (perform brake assist), and BBW (Brake By Wire). The braking system 18 provides braking force to the wheels 3 and the vehicle 1 via actuator 18a. Furthermore, the braking system 18 can detect brake lock-up, wheel spin, and signs of skidding based on the rotational difference between the left and right wheels 3, and perform various controls accordingly. The brake sensor 18b is, for example, a sensor that detects the position of the movable part of the brake operating unit 6. The brake sensor 18b can detect the position of the brake pedal, which is a movable part. The brake sensor 18b includes a displacement sensor.

[0044] In addition, actuator 107 is Figure 2 The rear drive force control actuator shown is electrically controlled by ECU14 and is used to control the drive force of the rear wheel 3R.

[0045] The steering angle sensor 19 is, for example, a sensor that detects the amount of steering input to the steering unit 4, such as the steering wheel. The steering angle sensor 19 is configured, for example, to use a Hall element. The ECU 14 obtains information from the steering angle sensor 19 based on the amount of steering input to the steering unit 4 by the driver, the amount of steering input to each wheel 3 during autopilot, etc., and performs various controls. Furthermore, the steering angle sensor 19 detects the rotation angle of the rotating parts included in the steering unit 4.

[0046] The throttle sensor 20 is, for example, a sensor that detects the position of the movable part of the acceleration operation unit 5. The throttle sensor 20 is capable of detecting the position of the accelerator pedal, which is the movable part. The throttle sensor 20 includes a displacement sensor.

[0047] The shift sensor 21 is, for example, a sensor that detects the position of the movable part of the transmission control unit 7. The shift sensor 21 can detect the position of levers, arms, buttons, etc., which are movable parts. The shift sensor 21 may include a displacement sensor or be configured as a switch.

[0048] Wheel speed sensor 22 is a sensor that detects the amount of rotation of wheel 3 or its rotational speed per unit time. Wheel speed sensor 22 outputs the number of wheel speed pulses representing the detected rotational speed as a sensor value. Wheel speed sensor 22 can be configured, for example, using a Hall element. ECU 14 calculates the amount of movement of vehicle 1, etc., based on the sensor value obtained from wheel speed sensor 22, and performs various controls. Furthermore, wheel speed sensor 22 is sometimes installed on braking system 18. In this case, ECU 14 obtains the detection result of wheel speed sensor 22 via braking system 18.

[0049] The vehicle height sensor 26 is a sensor that detects the travel displacement of each wheel.

[0050] The suspension system 30 is a vehicle control system configured between the vehicle body 2 and the wheels 3 of the vehicle 1 to mitigate impacts on the vehicle body 1 from the road surface. The suspension system 30 includes: a spring that absorbs vibrations of the vehicle 1 caused by impacts from the road surface; and a damping force variable damper that dampens the vibration of the spring and can change the damping force of the spring. In this embodiment, the suspension system 30 cooperates with the ECU 14 to control damping force adjustment devices such as solenoid actuators to change the damping force of the damping force variable damper. Thus, the suspension system 30 implements a damping force control system that dampens the vertical, lateral, and longitudinal vibrations of the vehicle body caused by impacts from the road surface.

[0051] Acceleration sensors 25 are respectively provided near the vehicle portion on the side of the suspension system 30 on the body 2 (hereinafter also referred to as "upper spring") and the vehicle portion on the side of the suspension system 30 on the wheel 3 (hereinafter also referred to as "lower spring"). The acceleration sensors 25 include: a vertical acceleration sensor (i.e., upper spring G sensor and lower spring G sensor) that detects and outputs the vertical acceleration of the body 2; a longitudinal acceleration sensor that detects and outputs the longitudinal acceleration of the body 2 (vehicle 1); and a lateral acceleration sensor that detects and outputs the lateral (width direction) acceleration of the body 2 (vehicle 1).

[0052] Here, the upper and lower acceleration sensors are the upper acceleration sensor (hereinafter also referred to as "upper spring G") that detects the upper acceleration of the spring (hereinafter also referred to as "upper spring G") and the lower acceleration sensor (hereinafter also referred to as "lower spring G") that detects the lower acceleration of the spring (hereinafter also referred to as "lower spring G").

[0053] The suspension system 30 is electrically controlled by ECU 14, and actuators 102, 103, 105, and 106 are activated for damping force control and stabilizer control.

[0054] Here, as Figure 2 As shown, actuator 102 is mounted on the suspension on the front wheel 3F side and is a front automatic stabilizer actuator used for stabilizer control on the front wheel 3F side. Figure 2 As shown, actuator 105 is mounted on the suspension on the rear wheel 3R side and is a rear automatic stabilizer actuator for stabilizer control on the rear wheel 3R side.

[0055] In addition, such as Figure 2 As shown, actuator 103 is mounted on the suspension on the front wheel 3F side and is a front damping force control actuator used for damping force control on the front wheel 3F side. Figure 2 As shown, actuator 106 is mounted on the suspension on the rear wheel 3R side and is a rear damping force control actuator for damping force control on the rear wheel 3R side.

[0056] Furthermore, the structure, configuration, and electrical connection of the various sensors or actuators mentioned above are just examples, and various settings (changes) can be made.

[0057] Next, the vehicle control device implemented by ECU 14 will be described. Hereinafter, ECU 14 will sometimes be referred to as vehicle control device 14. Furthermore, in this specification, road surface shape includes not only the shape of the road surface itself, but also the concept of all road surface elements related to the impact on the vehicle body, such as the surface condition of the road surface (dry, wet, snow-covered, icy, etc.) and the material of the road.

[0058] Figure 4This is an explanatory diagram of the simulation processing performed by the vehicle control device 14 in the embodiment. (As shown) Figure 4 As shown, the vehicle control device 14 mainly includes a vehicle movement data processing unit 141, a road shape output unit 142, a report calculation unit 143, a learning unit 144, and a vehicle control unit 145 as functional units.

[0059] These functional units are read and executed by CPU 14a from the storage unit 50. Figure 3 The program is implemented in the memory. Here, the storage unit 50 includes ROM 14b, RAM 14c, and SSD 14f. That is, as an example, the program includes... Figure 4 The modules corresponding to each functional unit of the vehicle control device 14 shown. Furthermore, each functional unit module can be implemented using independent hardware, including ASIC (Application Specific Integrated Circuit) circuits.

[0060] Here, the vehicle motion sensors are the spring-loaded G sensor in the analog acceleration sensor 25, the wheel speed sensor 22, the vehicle height sensor 26, etc. The vehicle motion data output by the vehicle motion sensors are data related to the motion of vehicle 1, such as spring-loaded acceleration (spring-loaded G), vehicle height, vehicle speed, spring-loaded velocity, vehicle height change rate, and spring-loaded velocity.

[0061] The vehicle motion data processing unit 141 acquires vehicle motion data output by the vehicle motion sensor and calculates vehicle motion (including G on the spring).

[0062] For example, the vehicle motion data processing unit 141 receives the detection signal from the spring-loaded G sensor in the acceleration sensor 25, i.e., the spring-loaded acceleration (spring-loaded G), and outputs the spring-loaded acceleration as vehicle motion data. Additionally, the vehicle motion data processing unit 141 receives the detection signal from the wheel speed sensor 22, i.e., the wheel speed, calculates the speed of vehicle 1 based on the wheel speed, and outputs the calculated speed as vehicle motion data. Furthermore, the vehicle motion data processing unit 141 receives the detection signal from the vehicle height sensor 26, i.e., the height of vehicle 1, and outputs the vehicle height as vehicle motion data.

[0063] Furthermore, the vehicle motion data processing unit 141 integrates the acceleration detected by the spring-mounted G-sensor to calculate the spring-mounted velocity, and outputs this spring-mounted velocity as vehicle motion data. Additionally, the vehicle motion data processing unit 141 differentiates the vehicle height sensor data detected by the vehicle height sensor to calculate the vehicle height change rate, and outputs this vehicle height change rate as vehicle motion data. Finally, the vehicle motion data processing unit 141 calculates the spring-down velocity based on the difference between the spring-mounted velocity and the vehicle height change rate, and outputs this spring-down velocity as vehicle motion data.

[0064] In other words, the vehicle motion data processing unit 141 outputs the spring acceleration (spring G), vehicle height, vehicle speed, spring speed, vehicle height change rate, and spring speed, which are considered as vehicle motion data, to the vehicle control unit 145. Additionally, the vehicle motion data processing unit 141 outputs the spring G to the report calculation unit 143.

[0065] The road surface shape output unit 142 includes a feature quantity calculation unit 1421 and an output unit 1422. Here, Figure 6 This is a diagram illustrating an example of a Q-table, etc., in an implementation scheme. For example... Figure 6 As shown in (a), the feature quantity calculation unit 1421 calculates the road surface shape feature quantities A, B, and C.

[0066] The road surface shape characteristic quantity A is the undulation (up and down movement) component in the vibration of the center of gravity of vehicle 1.

[0067] The road surface shape characteristic quantity B is the roll (lateral sway) component in the vibration of the center of gravity of vehicle 1.

[0068] The road surface shape characteristic quantity C is the pitch (longitudinal sway) component in the vibration of the center of gravity of vehicle 1.

[0069] The feature calculation unit 1421 uses information such as the spring acceleration of each of the four wheels 3 to calculate the road surface shape feature quantities A, B, and C respectively. Hereinafter, the road surface shape feature quantities A, B, and C will sometimes be referred to simply as "road surface shape feature quantities".

[0070] The output unit 1422 outputs the road surface shape feature quantity received from the feature quantity calculation unit 1421 to the state index acquisition unit 1441 of the learning unit 144.

[0071] The reward calculation unit 143 inputs the acceleration on the spring from the vehicle motion data processing unit 141, calculates the reward value (evaluation value) according to the prescribed reward function, and outputs the calculated reward value to the table update unit 1443 of the learning unit 144.

[0072] Learning Department 144 uses simulation processing, through Q-learning as reinforcement learning, for a complex number of patterns (in Figure 6 In example (a), there are 4096 patterns) and more than one of them (in Figure 6 In example (a), for each of the three road surface shape features, obtain the Q value to create a Q-table. Figure 6 In (c)), the Q value is implemented based on M modes relative to the vehicle control unit 145 (M is a specified integer of 3 or more) (in Figure 6 In example (b), there are 256 patterns) and more than one of them (in Figure 6 In example (b), for each of the four control parameters (control values) under simulated conditions, the smaller the impact the vehicle experiences from the road surface, the higher the evaluation value (reward value). Additionally, the learning department 144 uses a Q-table ( Figure 6 In (c) of the complex number of patterns (4096 patterns), for each of the road surface shape features, create the top N (N is an integer satisfying 2≤N<M) that keep the Q value large. Figure 7 In the example, the first three) Q-values ​​and the Q-value relationship information of the set of control parameters corresponding to those Q-values ​​( Figure 7 (b) and (c) in the text. These will be explained in detail below.

[0073] The learning unit 144 includes a status index acquisition unit 1441, a control index selection unit 1442, a table update unit 1443, a table 1444, and a control gain conversion unit 1445.

[0074] The state index acquisition unit 1441 receives the road surface shape feature quantity from the output unit 1422 and assigns the state index ( ) corresponding to the road surface shape feature quantity to the output unit 1422. Figure 6 (a) is sent to the control index selection unit 1442 and the table update unit 1443.

[0075] Here, Figure 6 (b) in the table represents the control values ​​for each control index. The control values ​​can be set to 1, 2, 3, and 4 for R (roll), P (pitch), H (heave), and B (baseline), respectively.

[0076] The control index selection unit 1442 outputs control indexes from 1 to 256 sequentially to the table update unit 1443 and the control gain conversion unit 1445 based on the number of times the status index is selected, for example, according to the status index input from the status index acquisition unit 1441.

[0077] The control gain conversion unit 1445 converts the control index into a control gain. The control gain is a parameter used to calculate the target control quantity required by the vehicle control unit 145 to provide the target control quantity to the suspension system 30. The vehicle control unit 145 sends the target control quantity to the suspension system 30. Specifically, the vehicle control unit 145 calculates the target control quantity based on the vehicle movement data input from the vehicle movement data processing unit 141 and the control gain, and outputs the calculated target control quantity to the suspension system 30. As a result, the suspension system 30 instructs control currents to actuators 102, 103, 105, and 106 to perform damping force control, etc., in order to achieve the target control quantity.

[0078] The table update unit 1443, based on the state index obtained from the state index acquisition unit 1441, acquires the evaluation value (reward value) (Q value) from the simulation (based on the simulation of the suspension system 30) performed when the simulation based on the control parameters (control values) corresponding to the control index obtained from the control index selection unit 1442 was carried out, and creates (updates) the Q table. Figure 6 (c) in the middle.

[0079] By repeatedly performing the above process, the Q table is completed. Figure 6 (c) in the middle. Then, as in Figure 7 The table update unit 1443 updates the table according to the Q table ( Figure 7 In (a) of the above, for each state index, Q-value relationship information is created, which includes the three largest Q-values ​​and the set of control indices corresponding to those Q-values. Figure 7 (b)(c) in the middle).

[0080] Next, the processing in a real-world environment will be explained. Figure 5 This is an explanatory diagram illustrating the processing performed in a real-world environment by the vehicle control device 14 according to the embodiment. When the vehicle 1 is traveling on an actual road surface, the learning unit 144 uses Q-value relationship information ( Figure 7 (b) and (c) in the above and the obtained road surface shape features are used to determine the Q-value relationship information. Figure 7 The control parameter corresponding to any one of the Q values ​​in (b) and (c) of the road surface shape feature quantity is sent to the vehicle control unit 145 for vehicle control, the Q value in this case is obtained, and the Q value relationship information is updated. Figure 7 The update processing of Q-values ​​in (b) and (c) of the above.

[0081] Furthermore, the learning unit 144 repeatedly performs the aforementioned update process, selecting control parameters based on the acquired road surface shape feature quantities, and considering the Q-value relationship information ( Figure 7In (b) and (c) of the above, when all N (3) Q values ​​of the road surface shape feature have been updated, the control parameter corresponding to the largest Q value among the retained Q values ​​is selected and sent to the vehicle control unit 145. These will be explained in detail below.

[0082] The road surface shape output unit 142 also includes a storage unit 1423. The storage unit 1423 stores the road location information and road surface shape feature quantities in a corresponding manner. If a road surface shape feature quantity corresponding to the location information exists in the storage unit 1423, the output unit 1422 outputs the road surface shape feature quantity; otherwise, it outputs the road surface shape feature quantity calculated by the feature quantity calculation unit 1421.

[0083] The control index selection unit 1442 obtains the status index from the status index acquisition unit 1441, and selects the status index from the control index table based on the number of times the status index has been selected. Figure 7 (c) Select the control index. For example, output the first control index during the first selection ( Figure 7 In (b) and (c) of the second selection, the control index of the second bit is output. Figure 7 In (b) and (c) of the above, the control index of the third bit is output during the third selection. Figure 7 In (b) and (c) of the above, the control index corresponding to the largest evaluation value (Q value) is output during the fourth and subsequent selections. Alternatively, a predetermined control index can be output during the first selection, and the first control index can be output during the second selection. Figure 7 In (b) and (c) of the above, the second control index is output during the third selection. Figure 7 In (b) and (c) of the above, the control index of the third bit is output during the fourth selection. Figure 7 In (b) and (c), the control index corresponding to the largest evaluation value is output in the fifth and subsequent selections.

[0084] Here, Figure 8 This diagram illustrates an example of positional relationship information in the implementation, which includes a set of road surface shape features and location information for each location. For instance, if vehicle 1 travels multiple times on a given road surface, during the first travel, the learning unit 144 stores the set of acquired road surface shape features (state indexes) and location information for each location in the positional relationship information of the storage unit 50. During subsequent travels, the learning unit 144 acquires road surface shape features (state indexes) based on vehicle 1's current location information and positional relationship information, and performs subsequent processing.

[0085] In addition, for example, when selecting control parameters for the acquired road surface shape features, when all N (3) Q values ​​for the road surface shape features in the Q value relationship information have been updated, the learning unit 144 selects the control parameter corresponding to the largest Q value among the retained Q values ​​and sends it to the vehicle control unit. When the new Q value obtained during vehicle control by the vehicle control unit 145 changes more than the specified change threshold, the Q value can also be used to update the Q value relationship information.

[0086] in addition, Figure 9 This is an explanatory diagram of the update of position relationship information in the implementation method. The learning unit 144 can also, for a given road surface and for each location, after a predetermined period has elapsed since the set of acquired road surface shape feature quantities and the location information of that location was stored in the position relationship information of the storage unit 50, acquire road surface shape feature quantities again for that location to update the position relationship information. In the next subsequent driving, the road surface shape feature quantities are acquired based on the vehicle's current location information and the updated position relationship information for subsequent processing.

[0087] Next, the process of creating Q-value relationship information by the vehicle control device 14 will be explained. Figure 10 This is a flowchart illustrating the process of creating Q-value relationship information performed by the vehicle control device 14 in the embodiment. See also... Figure 4 .

[0088] In step S11, the learning unit 144 uses simulation processing and Q-learning to obtain the Q-value for each of the road surface shape feature quantities in one or more modes, based on the simulation of one or more control parameters (control values) relative to the vehicle control unit 145, and creates a Q-table. Figure 6 (c) in the middle.

[0089] Next, in step S12, the learning unit 144 bases its learning on the Q-table ( Figure 6 In (c) of the multiple modes, for each of the road surface shape features, a reduced table (Q-value relationship information) is created, which contains the top N Q values ​​that maintain the largest Q values ​​and the set of control parameters corresponding to those Q values. Figure 7 (b)(c) in the middle).

[0090] Next, the processing in the actual environment performed by the vehicle control device 14 will be explained. Figure 11 This is a flowchart illustrating the processing performed by the vehicle control device 14 in a real-world environment according to the embodiment. See also... Figure 5 .

[0091] In step S21, the state index acquisition unit 1441 acquires the road surface shape feature quantity from the output unit 1422 of the road surface shape output unit 142.

[0092] Next, in step S22, the control index selection unit 1442 determines the state index corresponding to the road surface shape feature quantity obtained in step S21. Figure 7 (b) in the middle.

[0093] Next, in step S23, the control index selection unit 1442 determines whether the state index determined in step S22 is selected for the first time. If it is "yes", it proceeds to step S24; if it is "no", it proceeds to step S26.

[0094] In step S24, the control index selection unit 1442 determines the first control index ( Figure 7 (c) in the middle.

[0095] Next, using the control index, vehicle control based on the suspension system 30 is performed, and in step S25, the table update unit 1443 updates accordingly. Figure 7 Find the Q value of (b) in the table and return to step S21.

[0096] In step S26, the control index selection unit 1442 determines whether the state index determined in step S22 is selected for the second time. If it is "yes", it proceeds to step S27; if it is "no", it proceeds to step S29.

[0097] In step S27, the control index selection unit 1442 determines the control index of the second bit ( Figure 7 (c) in the middle.

[0098] Next, vehicle control based on the suspension system 30 is performed using the control index, and in step S28, the table update unit 1443 updates accordingly. Figure 7 Find the Q value of (b) in the table and return to step S21.

[0099] In step S29, the control index selection unit 1442 determines whether the state index determined in step S22 is selected for the third time. If it is "yes", it proceeds to step S30; if it is "no", it proceeds to step S32.

[0100] In step S30, the control index selection unit 1442 determines the control index of the third bit ( Figure 7 (c) in the middle.

[0101] Next, vehicle control based on the suspension system 30 is performed using the control index, and in step S31, the table update unit 1443 updates accordingly. Figure 7 Find the Q value of (b) in the table and return to step S21.

[0102] Through the processing up to step S31, Figure 7 The evaluation value table in (b) has become content (Q value) suitable for the actual environment.

[0103] In step S32, the control index selection unit 1442 determines the control index with the largest Q value ( Figure 7 (b)). Then, vehicle control based on suspension system 30 is performed using the control index.

[0104] Thus, according to the vehicle control device 14 of this embodiment, a Q-table is created using simulation processing ( Figure 6 After (c) in the middle, create the reduced table (Q-value relationship information) Figure 7 Then, the reduced table is updated when vehicle 1 is driving on an actual road surface. Thus, control parameters related to the control of suspension system 30 can be obtained that are suitable for the actual environment, even when there are differences between the simulation and the actual environment.

[0105] Furthermore, if vehicle 1 travels multiple times on the actual designated road surface, during the first trip, if... Figure 8 As shown, for each location, the acquired road surface shape features and the location information of that location are stored in the location relationship information of the storage unit 50. Therefore, when vehicle 1 travels on the designated road surface a second time or more, the location relationship information can be used to acquire appropriate road surface shape features for subsequent processing. Thus, for example, even when vehicle 1 travels on the same road every day, a comfortable riding experience for the user in vehicle 1 can be achieved.

[0106] In addition, such as Figure 9 As shown, if a predetermined period has elapsed since the location relationship information was stored in the storage unit 50, the road surface shape feature quantity is acquired again for that location, and the location relationship information is updated. Therefore, for example, even if the road surface shape changes due to road construction, the vehicle 1 can be controlled using the optimal control parameters corresponding to that change.

[0107] Additionally, Q-value relationship information can be updated during processing in real-world environments. Figure 7 Following (b) and (c) in the above, when a new Q-value change obtained during vehicle control based on vehicle control device 14 exceeds a predetermined change threshold, the Q-value relationship information is updated using this Q-value. Thus, in vehicle 1, in the event of any change affecting the Q-value, such as changes in tire pressure or component deterioration, vehicle 1 can be controlled using the optimal control parameters corresponding to that change.

[0108] Furthermore, deep learning methods based on DNNs using previous techniques generally require a large amount of memory due to the large number of parameters. Moreover, the repeated learning process is time-consuming, making it difficult to directly update the DNN in vehicle 1.

[0109] On the other hand, the method according to this embodiment does not have these disadvantages. Specifically, for example, a reduced table is created by reducing the Q table ( Figure 7 Therefore, as shown in (b) and (c), it is possible to significantly reduce memory usage.

[0110] Furthermore, the vehicle control program executed by the vehicle control device 14 of the above embodiment can be provided, for example, by pre-loading ROM 14b.

[0111] Alternatively, the vehicle control program can be configured to be provided as an installable or executable file stored on a computer-readable recording medium such as a CD-ROM, floppy disk (FD), CD-R, or DVD (Digital Versatile Disk).

[0112] Alternatively, the vehicle control program can be provided by storing it on a computer connected to a network such as the Internet and downloading it over the network. Alternatively, the vehicle control program can be provided or distributed via a network such as the Internet.

[0113] Although embodiments of the present invention have been described, these embodiments are provided as examples and are not intended to limit the scope of the invention. This new embodiment can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. Furthermore, these embodiments and their variations are included within the scope or spirit of the invention, as well as within the scope of the invention as described in the claims and its equivalents.

[0114] For example, in Figure 6 In example (a), the number (types) of road surface shape features is three, but it is not limited to this; it can also be six or other numbers. In addition, the number of state indices is not limited to 4096; it can also be other numbers.

[0115] In addition, Figure 6 In example (b), the number (types) of control values ​​is four, but it is not limited to this; it can also be three, five, six, or other numbers. Furthermore, the number of control indices is not limited to 256; it can also be other numbers. Additionally, the number of stages for control values ​​is not limited to four stages; it can also be other numbers.

[0116] In addition, Figure 7In examples (b) and (c), the first three data points of Q are kept, but it is not limited to the first three; the first four or other numbers of data points can also be kept.

[0117] [Summary of this implementation method]

[0118] This embodiment has at least the following structure.

[0119] This embodiment is a vehicle control device mounted in a vehicle, comprising: a vehicle control unit that sends target control quantity information to a suspension system for vehicle control that mitigates impacts on the vehicle body from the road surface; and a learning unit configured to: use simulation processing, through Q-learning as reinforcement learning, acquire Q values ​​for each of one or more road surface shape features in a plurality of modes to create a Q-table, wherein the Q value is an evaluation value that is larger as the impact on the vehicle body from the road surface is smaller, based on simulations of one or more control parameters for M modes of the vehicle control unit; and based on the Q-table, for each of the road surface shape features in the plurality of modes, create the top N Q values ​​that maintain the largest Q values ​​and the corresponding control parameters. The set of Q-value relationship information, where M is a specified integer greater than or equal to 3, and N is an integer satisfying 2 ≤ N < M; when the vehicle is driving on the actual road surface, using the Q-value relationship information and the acquired road surface shape feature quantity, the control parameter corresponding to any one of the Q values ​​in the Q-value relationship information corresponding to the road surface shape feature quantity is sent to the vehicle control unit for vehicle control, the Q value in this case is obtained, and the Q value in the Q-value relationship information is updated, and the update process is repeated; when the control parameter is selected for the acquired road surface shape feature quantity, when all N Q values ​​in the Q-value relationship information for the road surface shape feature quantity have been updated, the control parameter corresponding to the largest Q value among the retained Q values ​​is selected and sent to the vehicle control unit.

[0120] With this structure, control parameters related to the control of the suspension system can be obtained that are suitable for the actual environment, even when there are differences between the simulation and the actual environment.

[0121] Furthermore, the learning unit is configured such that, when the vehicle travels multiple times on a specified road surface, during the first travel, it stores the set of road surface shape features and location information of each location in the location relationship information of the storage unit. During subsequent travels, it acquires the road surface shape features based on the vehicle's current location information and the location relationship information, and performs subsequent processing.

[0122] With this structure, when a vehicle travels on the designated road surface a second time or more, it can use the positional relationship information to obtain appropriate road shape features for subsequent processing.

[0123] Furthermore, for the specified road surface, the learning unit, for each location, after a specified period has elapsed since the set of acquired road surface shape features and location information of that location was stored in the location relationship information of the storage unit, acquires the road surface shape features again for that location and updates the location relationship information. In the next subsequent driving, the learning unit acquires the road surface shape features based on the vehicle's current location information and the updated location relationship information for subsequent processing.

[0124] With such a structure, even if the road surface shape changes due to road construction, the vehicle can be controlled using the optimal control parameters corresponding to that change.

[0125] Furthermore, the learning unit is configured such that, when selecting the control parameter for the acquired road surface shape feature quantity, when all N Q values ​​for the road surface shape feature quantity in the Q value relationship information have been updated, the control parameter corresponding to the largest Q value among the maintained Q values ​​is selected and sent to the vehicle control unit. When a new Q value obtained during vehicle control by the vehicle control unit changes more than a predetermined change threshold, the Q value is used to update the Q value relationship information.

[0126] With this structure, in the vehicle, in the event of any change in the Q value, such as changes in tire pressure or component deterioration, the vehicle can be controlled using the optimal control parameters corresponding to that change.

[0127] Furthermore, the effects produced by the dependents or embodiments of the claims are additional effects independent of the effects produced by the independent claims.

Claims

1. A vehicle control device, mounted on a vehicle, wherein, have: The vehicle control unit sends target control quantity information to the suspension system, which performs vehicle control to mitigate impacts on the vehicle body from the road surface; and The learning department is configured as follows: Using simulation processing, Q-learning as reinforcement learning is employed to acquire Q-values ​​for each of more than one road surface shape feature quantities in a plurality of modes, thereby creating a Q-table. The Q-values ​​are evaluation values ​​where, under simulations based on more than one control parameter for each of M modes of the vehicle control unit, the smaller the impact on the vehicle body from the road surface, the higher the value. Based on the Q-table, for each of the road surface shape feature quantities in the plurality of modes, Q-value relationship information is created, including the top N Q-values ​​that maintain the largest Q-values ​​and the set of control parameters corresponding to those Q-values. M is a specified integer of 3 or higher, and N is an integer satisfying 2 ≤ N < M. When the vehicle is driving on the actual road surface, the control parameter corresponding to any one of the Q values ​​in the Q value relationship information corresponding to the road surface shape feature quantity is sent to the vehicle control unit to control the vehicle, using the Q value in this case to update the Q value in the Q value relationship information, and the update process is repeated. When the control parameter is selected for the acquired road surface shape feature quantity, and all N Q values ​​for the road surface shape feature quantity in the Q value relationship information have been updated, the control parameter corresponding to the largest Q value among the retained Q values ​​is selected and sent to the vehicle control unit.

2. The vehicle control device according to claim 1, wherein, The learning unit is configured as follows: When the vehicle travels multiple times on the actual, designated road surface, During the first trip, for each location, the acquired road surface shape features and the location information of that location are stored in the location relationship information in the storage unit. During subsequent journeys, the road surface shape features are obtained based on the vehicle's current location information and the location relationship information, and then processed accordingly.

3. The vehicle control device according to claim 2, wherein, The learning department refers to the specified road surface. For each location, if a predetermined period has elapsed since the set of acquired road surface shape features and location information for that location was stored in the location relationship information in the storage unit, the road surface shape features for that location are acquired again and the location relationship information is updated. In subsequent driving, the road surface shape features are obtained based on the vehicle's current location information and the updated location relationship information, and then processed accordingly.

4. The vehicle control device according to claim 1, wherein, The learning unit is configured such that, when selecting the control parameter for the acquired road surface shape feature quantity, when all N Q values ​​for the road surface shape feature quantity in the Q value relationship information have been updated, it selects the control parameter corresponding to the largest Q value among the maintained Q values ​​and sends it to the vehicle control unit. When a new Q value obtained during vehicle control by the vehicle control unit changes more than a predetermined change threshold, it updates the Q value relationship information with that Q value.

Images