Vehicle control device

Abstract

To improve both of stability of a vehicle posture and riding comfort.SOLUTION: A vehicle control device includes: a peripheral environment estimation section which acquires a captured image obtained by capturing the periphery of a vehicle from an imaging device, inputs the acquired captured image into a peripheral environment estimation model constituted of base models being large-scaled models trained using various and large-scaled data, and obtains an output result of a future peripheral environment output from the peripheral environment estimation model; a vehicle behavior estimation section which estimates a vehicle state on the basis of the output result; and a vehicle control section which executes suspension damping force control of the vehicle on the basis of the vehicle state.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to a vehicle control device.

Background Art

[0002] Conventionally, a technique for recognizing the surrounding environment of a vehicle based on an image captured by a camera mounted on the vehicle and controlling the vehicle is known. For example, Patent Document 1 discloses a technique for recognizing a changing area where the wind condition around the host vehicle changes as the surrounding environment of the vehicle, and controlling the steering of the host vehicle based on the degree of change in the wind condition when it is predicted that the host vehicle will reach the changing area.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in such a conventional technique, only steering control is performed, so a roll unintended by the driver of the vehicle may occur in the vehicle, and in such a case, the vehicle posture and riding comfort may deteriorate.

[0005] The present invention has been made in view of the above, and provides a vehicle control device capable of improving both the stability of the vehicle posture and the riding comfort.

Means for Solving the Problems

[0006] The vehicle control device according to the present invention acquires a captured image obtained by capturing the periphery of the vehicle from an imaging device, inputs the acquired captured image into a surrounding environment estimation model configured by a base model, which is a large-scale model trained using diverse and large-scale data, and obtains an output result regarding the future surrounding environment output from the surrounding environment estimation model. The vehicle control device further includes a vehicle behavior estimation unit that estimates the vehicle state based on the output result, and a vehicle control unit that executes suspension damping force control of the vehicle based on the vehicle state.

Effect of the Invention

[0007] According to the vehicle control device of the present invention, both the stability of the vehicle posture and the riding comfort can be improved.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Embodiments for Carrying Out the Invention

[0009] Hereinafter, exemplary embodiments of the present invention will be disclosed. The configurations of the embodiments shown below, as well as the actions, results, and effects brought about by the configurations, are examples. The present invention can be realized by configurations other than those disclosed in the following embodiments, and it is possible to obtain at least one of various effects and derivative effects based on the basic configuration.

[0010] The vehicle 1 of the present embodiment may be, for example, an automobile having an internal combustion engine (not shown) as a drive source, that is, an internal combustion engine vehicle, or an automobile having an electric motor (not shown) as a drive source, that is, an electric vehicle or a fuel cell vehicle, etc., or a hybrid vehicle having both of them as drive sources, or an automobile having other drive sources. Further, the vehicle 1 can be equipped with various transmission devices, and can be equipped with various devices necessary for driving the internal combustion engine and the electric motor, such as systems and components. Also, the method, number, layout, etc. of the devices related to the drive of the wheels 3 in the vehicle 1 can be set in various ways.

[0011] (Configuration of Vehicle, Vehicle Control System) FIG. 1 is an exemplary perspective view showing a state in which a part of the passenger compartment of a vehicle according to an embodiment is seen through. FIG. 2 is a plan view showing a state in which the vehicle body of a vehicle according to an embodiment is seen through. FIG. 3 is an exemplary block diagram of the configuration of a vehicle control system of a vehicle according to an embodiment.

[0012] First, an example of the configuration of the vehicle 1 according to the present embodiment will be described with reference to FIGS. 1 to 3. As illustrated in FIG. 1, the vehicle body 2 constitutes a passenger compartment 2a in which a passenger (not shown) rides. Inside the passenger compartment 2a, a steering unit 4, an acceleration operation unit 5, a braking operation unit 6, a shift operation unit 7, etc. are provided in a state facing the driver's seat 2b as a passenger.

[0013] The steering unit 4 is, for example, a steering wheel protruding from the dashboard 24. The acceleration operation unit 5 is, for example, an accelerator pedal located under the driver's feet. The braking operation unit 6 is, for example, a brake pedal located under the driver's feet. The shift operation unit 7 is, for example, a shift lever protruding from the center console. Note that the steering unit 4, the acceleration operation unit 5, the braking operation unit 6, the shift operation unit 7, etc. are not limited to these.

[0014] Also, inside the passenger compartment 2a, a display device 8 as a display output unit and an audio output device 9 as an audio output unit are provided. The display device 8 is, for example, an LCD (Liquid Crystal Display), an OELD (Organic Electroluminescent Display), or the like. The audio output device 9 is, for example, a speaker. Further, the display device 8 is covered with a transparent operation input unit 10 such as a touch panel. The passenger can visually recognize the image displayed on the display screen of the display device 8 through the operation input unit 10. Also, the passenger can execute an operation input by touching, pressing, or moving the operation input unit 10 with a finger or the like at a position corresponding to the image displayed on the display screen of the display device 8.

[0015] These display devices 8, audio output devices 9, operation input units 10, etc. are provided, for example, in a monitor device 11 located at the center in the vehicle width direction, that is, the left - right direction, of the dashboard 24. The monitor device 11 can have operation input units (not shown) such as switches, dials, joysticks, push buttons, etc. Also, an audio output device (not shown) can be provided at another position in the passenger compartment 2a different from the monitor device 11, and audio can be output from the audio output device 9 of the monitor device 11 and the other audio output device. Note that the monitor device 11 can be used in combination with, for example, a navigation system or an audio system. Also, a display device 12 different from the display device 8 is provided in the passenger compartment 2a.

[0016] Also, as illustrated in FIG. 2, the vehicle 1 is, for example, a four - wheel vehicle and has two left - right front wheels 3F and two left - right rear wheels 3R. All of these four wheels 3 can be configured to be steerable. As illustrated in FIG. 3, the vehicle 1 has a steering system 13 for steering at least two wheels 3.

[0017] As illustrated in FIG. 3, the steering system 13 has actuators 101, 104 and a torque sensor 13b. The steering system 13 is electrically controlled by an ECU14 (Electronic Control Unit) or the like to operate the actuators 101, 104. Here, as shown in FIG. 2, the actuator 101 is connected to the front wheels 3F and is a front - wheel steering actuator for steering the front wheels 3F. Also, the actuator 104 is connected to the rear wheels 3R and is a rear - wheel steering actuator for steering the rear wheels 3R.

[0018] The steering system 13 is, for example, an electric power steering system, an SBW (Steer By Wire) system, or the like. The steering system 13 adds torque, that is, assist torque, to the steering section 4 by the actuators 101 and 104 to supplement the steering force, or steers the wheels 3 by the actuators 101 and 104. In this case, the actuators 101 and 104 may steer one wheel 3 or a plurality of wheels 3. Further, the torque sensor 13b detects, for example, the torque applied by the driver to the steering section 4.

[0019] Also, as illustrated in FIG. 3, the vehicle body 2 is provided with a plurality of imaging units 15, for example, eight imaging units 15a to 15f. The imaging unit 15 is, for example, a digital camera incorporating an image sensor such as a CCD (Charge Coupled Device) or a CIS (CMOS Image Sensor). The imaging unit 15 can output moving image data at a predetermined frame rate. Each of the imaging units 15 has a wide-angle lens or a fish-eye lens and can photograph, for example, a range of 140° to 190° in the horizontal direction. Further, the optical axis of the imaging unit 15 is set to be directed obliquely downward. Therefore, the imaging unit 15 sequentially photographs the external environment around the vehicle body 2 including the road surface on which the vehicle 1 can move and the area where the vehicle 1 can park, and outputs it as a captured image (image data). The imaging unit 15 is an example of an imaging device.

[0020] The imaging unit 15a is located, for example, at the rear end 2e of the vehicle body 2 and is provided on the wall portion below the door 2h of the rear trunk. The imaging unit 15ba is located, for example, on the right side of the vehicle body 2, that is, at the right end in the vehicle width direction, and is provided on the front side (front side) of the door mirror 2g as a right protrusion. The imaging unit 15bb is located, for example, on the right side of the vehicle body 2, that is, at the right end in the vehicle width direction, and is provided on the rear side (rear side) of the door mirror 2g as a right protrusion.

[0021] The imaging unit 15c is located, for example, at the front side of the vehicle body 2, that is, at the front end in the vehicle longitudinal direction, and is provided on a front bumper or the like. The imaging unit 15da is located, for example, on the left side of the vehicle body 2, that is, at the left end in the vehicle width direction, and is provided on the front side (front) of the door mirror 2g as a left protruding portion. The imaging unit 15db is located, for example, on the left side of the vehicle body 2, that is, at the left end in the vehicle width direction, and is provided on the rear side (rear) of the door mirror 2g as a left protruding portion.

[0022] The imaging unit 15e is located, for example, on the right side of the vehicle body 2, that is, at the right end in the vehicle width direction, and is provided near the right door. The imaging unit 15f is located, for example, on the left side of the vehicle body 2, that is, at the left end in the vehicle width direction, and is provided near the left door.

[0023] The ECU 14 can execute arithmetic processing and image processing based on the image data obtained by the plurality of imaging units 15, and can generate an image with a wider viewing angle or a virtual bird's-eye view image of the vehicle 1 seen from above. Note that the bird's-eye view image can also be referred to as a planar image. In the present embodiment, each imaging unit 15 images an area around the vehicle 1.

[0024] Also, as illustrated in FIG. 1, the vehicle body 2 is provided with, for example, four ranging units 16a to 16d and eight ranging units 17a to 17h as a plurality of ranging units 16 and 17. The ranging units 16 and 17 are, for example, sonars that emit ultrasonic waves and capture the reflected waves. A sonar can also be referred to as a sonar sensor or an ultrasonic detector. The ECU 14 can measure the presence or absence of an object such as an obstacle located around the vehicle 1 and the distance to the object based on the detection results of the ranging units 16 and 17. That is, the ranging units 16 and 17 are an example of a detection unit that detects an object. Note that the ranging unit 17 is used, for example, for detecting an object at a relatively short distance, and the ranging unit 16 can be used, for example, for detecting an object at a relatively long distance farther than the ranging unit 17. Also, the ranging unit 17 is used, for example, for detecting an object in front of and behind the vehicle 1, and the ranging unit 16 can be used for detecting an object on the side of the vehicle 1.

[0025] Also, as illustrated in FIG. 3, in the vehicle control system 100, in addition to the ECU 14, the monitor device 11, the steering system 13, the distance measurement units 16 and 17, etc., a brake system 18, a suspension system 30, a steering angle sensor 19, an accelerator sensor 20, a shift sensor 21, a wheel speed sensor 22, an acceleration sensor 25, an actuator 107, etc. are electrically connected via an in-vehicle network 23 as an electric communication line. The in-vehicle network 23 is configured as, for example, a CAN (controller area network). The ECU 14 is an example of an estimation device and a vehicle control device.

[0026] The ECU 14 can control the steering system 13, the brake system 18, the suspension system 30, the actuator 107, etc. by sending a control signal through the in-vehicle network 23. Also, the ECU 14 can receive detection results of a torque sensor 13b, a brake sensor 18b, a steering angle sensor 19, the distance measurement unit 16, the distance measurement unit 17, an accelerator sensor 20, a shift sensor 21, a wheel speed sensor 22, etc., and operation signals of an operation input unit 10, etc. via the in-vehicle network 23.

[0027] The ECU 14 has, for example, a CPU 14a (Central Processing Unit), a ROM (Read Only Memory) 14b, a RAM (Random Access Memory) 14c, a display control unit 14d, an audio control unit 14e, an SSD (Solid State Drive, flash memory) 14f, etc.

[0028] The CPU 14a can execute various arithmetic processes and controls, such as image processing related to images displayed on the display devices 8 and 12, determination of the target position of the vehicle 1, calculation of the moving route of the vehicle 1, judgment of the presence or absence of interference with an object, automatic control of the vehicle 1, cancellation of automatic control, damping control, spring constant switching control, steering control, stabilizer control, driving force control, etc. The CPU 14a can read a program installed and stored in a non-volatile storage device such as the ROM 14b and execute arithmetic processing according to the program.

[0029] The RAM 14c temporarily stores various data used in the arithmetic operations by the CPU 14a. Also, the display control unit 14d mainly executes image processing using the image data obtained by the imaging unit 15 and synthesis of the image data displayed on the display device 8, etc., among the arithmetic processes in the ECU 14. Also, the audio control unit 14e mainly executes processing of the audio data output from the audio output device 9, among the arithmetic processes in the ECU 14. Also, the SSD 14f is a rewritable non-volatile storage unit and can store data even when the power of the ECU 14 is turned off. Note that the CPU 14a, the ROM 14b, the RAM 14c, etc. can be integrated in the same package. Also, the ECU 14 may be configured such that another logical operation processor such as a DSP (Digital Signal Processor) or a logic circuit is used instead of the CPU 14a. Also, an HDD (Hard Disk Drive) may be provided instead of the SSD 14f, or the SSD 14f and the HDD may be provided separately from the ECU 14.

[0030] The braking system 18 is, for example, an ABS (Anti-lock Brake System) that suppresses brake lock, an anti-skid device (ESC: Electronic Stability Control) that suppresses the skidding of the vehicle 1 during cornering, an electric braking system that enhances the braking force (executes brake assist), a BBW (Brake By Wire), etc. The braking system 18 applies a braking force to the wheels 3 and thus to the vehicle 1 via the actuator 18a. Further, the braking system 18 can detect signs of brake lock, wheel spin, skidding, etc. from the rotational difference between the left and right wheels 3 and execute various controls. The brake sensor 18b is, for example, a sensor that detects the position of the movable part of the braking operation unit 6. The brake sensor 18b can detect the position of the brake pedal as the movable part. The brake sensor 18b includes a displacement sensor.

[0031] Further, the actuator 107 is the rear driving force control actuator shown in FIG. 2, and is an actuator that is electrically controlled by the ECU 14 or the like to control the driving force of the rear wheel 3R.

[0032] The steering angle sensor 19 is, for example, a sensor that detects the amount of steering of the steering unit 4 such as the steering wheel. The steering angle sensor 19 is configured using, for example, a Hall element or the like. The ECU 14 acquires the amount of steering of the steering unit 4 by the driver and the amount of steering of each wheel 3 during automatic steering from the steering angle sensor 19 and executes various controls. Note that the steering angle sensor 19 detects the rotation angle of the rotating part included in the steering unit 4.

[0033] The accelerator sensor 20 is, for example, a sensor that detects the position of the movable part of the acceleration operation unit 5. The accelerator sensor 20 can detect the position of the accelerator pedal as the movable part. The accelerator sensor 20 includes a displacement sensor.

[0034] The shift sensor 21 is, for example, a sensor that detects the position of the movable part of the shift operation unit 7. The shift sensor 21 can detect the positions of a lever, an arm, a button, etc. as the movable part. The shift sensor 21 may include a displacement sensor or may be configured as a switch.

[0035] The wheel speed sensor 22 is a sensor that detects the amount of rotation or the number of rotations per unit time of the wheel 3. The wheel speed sensor 22 outputs the number of wheel speed pulses indicating the detected number of rotations as a sensor value. The wheel speed sensor 22 can be configured using, for example, a Hall element or the like. The ECU 14 calculates the moving amount of the vehicle 1 and the like based on the sensor value acquired from the wheel speed sensor 22 and executes various controls. Note that the wheel speed sensor 22 may be provided in the brake system 18. In that case, the ECU 14 acquires the detection result of the wheel speed sensor 22 via the brake system 18.

[0036] The suspension system 30 is disposed between the vehicle body 2 and the wheel 3 of the vehicle 1. The suspension system 30 includes a spring that absorbs the vibration of the vehicle 1 due to the impact on the vehicle 1 from the road surface, and a damping force variable damper that damps the vibration of the spring and can change the damping force of the vibration of the spring. In the present embodiment, the suspension system 30 cooperates with the ECU 14 to control a damping force adjustment device such as a solenoid actuator to change the damping force of the damping force variable damper. Thus, the suspension system 30 realizes a damping force control system that damps the vertical, lateral, and longitudinal vibrations of the vehicle body due to the impact on the vehicle 1 from the road surface.

[0037] In the vicinity of each of the vehicle part on the vehicle body 2 side (also referred to as above the spring) with respect to the suspension system 30 and the vehicle 1 part on the wheel 3 side (also referred to as below the spring) with respect to the suspension system 30, an acceleration sensor 25 is provided. The acceleration sensor 25 includes a vertical acceleration sensor that detects and outputs the vertical acceleration of the vehicle body 2, a longitudinal acceleration sensor that detects and outputs the longitudinal acceleration of the vehicle body 2 (vehicle 1), and a lateral acceleration sensor that detects and outputs the lateral acceleration which is the lateral (width direction) acceleration of the vehicle body 2 (vehicle 1).

[0038] The suspension system 30 is electrically controlled by the ECU 14 or the like to operate the actuators 102, 103, 105, 106 for damping force control and stabilizer control.

[0039] Here, as shown in FIG. 2, the actuator 102 is provided in the suspension on the front wheel 3F side and is a front active stabilizer actuator for stabilizer control on the front wheel 3F side. The actuator 105 is provided in the suspension on the rear wheel 3R side as shown in FIG. 2 and is a rear active stabilizer actuator for stabilizer control on the rear wheel 3R side.

[0040] Also, as shown in FIG. 2, the actuator 103 is provided in the suspension on the front wheel 3F side and is a front damping force control actuator for damping force control on the front wheel 3F side. The actuator 106 is provided in the suspension on the rear wheel 3R side as shown in FIG. 2 and is a rear damping force control actuator for damping force control on the rear wheel 3R side.

[0041] Note that the configurations, arrangements, electrical connection forms, etc. of the various sensors and actuators described above are examples and can be set (changed) in various ways.

[0042] (Configuration of Vehicle Control Device) Next, the vehicle control device in which the ECU 14 functions will be described. Hereinafter, the ECU 14 may be referred to as the vehicle control device 14. FIG. 4 is a functional block diagram of the vehicle control device 14 according to the embodiment. As shown in FIG. 4, the vehicle control device 14 includes, as an example, a peripheral environment estimation unit 301, a vehicle behavior estimation unit 302, a control device selection unit 303, a vehicle control unit 340, and a storage unit 310 as functional units. These functional units are realized by the CPU 14a reading and executing a program stored in the storage unit 50d. Here, the storage unit 50d includes a ROM 14b, a RAM 14c, and an SSD 14f. That is, the program may include, as an example, modules corresponding to each block excluding the storage unit 50d shown in FIG. 4.

[0043] The storage unit 310 stores a peripheral environment estimation model 311, a vehicle behavior estimation model 312, and a device control model 313. Details of the peripheral environment estimation model 311, the vehicle behavior estimation model 312, and the device control model 313 will be described later.

[0044] The peripheral environment estimation unit 301 acquires a captured image obtained by the imaging unit 15 capturing the periphery of the vehicle, and inputs the acquired captured image to the peripheral environment estimation model 311 stored in the storage unit 310, and obtains an output result regarding the future peripheral environment output from the peripheral environment estimation model, thereby estimating the peripheral environment of the vehicle 1.

[0045] The peripheral environment estimation model 311 is a foundation model that inputs an image and outputs an output result regarding the future peripheral environment. Here, the foundation model is a large-scale model trained using diverse and large-scale data and applicable to various tasks.

[0046] By using the peripheral environment estimation model 311 constructed as this foundation model, the peripheral environment estimation unit 301 can recognize a change in the future peripheral environment of the vehicle as a wind condition change from the output result.

[0047] Note that a large language model (LLM) is an example of a base model. A large language model is a natural language processing model trained using a large amount of text data. The surrounding environment estimation model 311 may be constructed as a large language model.

[0048] FIG. 5 is a diagram for explaining an example of the input and output of the surrounding environment estimation model 311 according to the embodiment. As shown in FIG. 5, when an imaging image captured by the imaging unit 15 is input to the surrounding environment estimation model 311, the surrounding environment estimation model 311 outputs an output result of the future surrounding environment in the input imaging image. The surrounding environment estimation model 311 outputs, for example, the state of the wind received by the vehicle due to the influence of the surrounding environment as an output result. Here, the output results from the surrounding environment estimation model 311 include text and feature amounts indicating the surrounding environment. The text and feature amounts as output results are directly passed to the vehicle behavior estimation unit 302 and input to the vehicle behavior estimation model 312 described later. Here, the feature amount is information that can be analyzed by AI, that is, a learned model.

[0049] Here, an example of the output result of the text will be described. FIG. 6 is a diagram showing an example of text among the output results of the surrounding environment estimation model 311 according to the embodiment. The scene in FIG. 6 means what kind of situation the surrounding environment of the vehicle 1 is in. As shown in the example of FIG. 6, the scenes include, but are not limited to, a truck overtaking scene, a tunnel exit scene, a mountain road scene, etc., and are determined by the surrounding environment estimation model 311 according to the content of the imaging image.

[0050] FIG. 7 is a diagram for explaining an example of a truck overtaking scene in the embodiment. As shown in FIG. 7, the truck overtaking scene is, for example, a scene in which a truck 600 is trying to overtake a host vehicle (i.e., vehicle 1) from behind on a two-lane highway or the like. In this case, when the truck 600 approaches and overtakes the vehicle 1, the vehicle 1 receives wind pressure from the truck 600, and due to such a change in the wind condition, the posture of the vehicle 1 becomes unstable and it is difficult to maintain straight-ahead stability, or the riding comfort of the passengers may be impaired.

[0051] Therefore, the surrounding environment estimation unit 301 acquires a captured image behind the vehicle 1 captured by the imaging unit 15a provided at the rear of the vehicle 1 and inputs it to the surrounding environment estimation model 311. Then, as shown in FIG. 6, the surrounding environment estimation model 311 outputs, as text, that the truck 600 is approaching from behind from the captured image behind.

[0052] In addition, as shown in FIG. 6, the surrounding environment estimation model 311 estimates the size and shape of the approaching vehicle (i.e., the truck 600), the speed difference between the vehicle 1 and the truck 600, and the distance between the vehicle 1 and the truck 600 when the truck 600 overtakes the vehicle 1 as shown in the right figure of FIG. 7, and outputs them as text.

[0053] FIG. 8 is a diagram for explaining an example of a tunnel exit scene in the embodiment. As shown in FIG. 8, the tunnel exit scene is a scene in which the vehicle 1 is traveling near the exit of the tunnel 810. In this case, for example, when strong wind is blowing outside the tunnel 810, as soon as the vehicle 1 exits the tunnel 810, it is affected by the strong wind, and due to such a change in the wind condition, the posture of the vehicle 1 becomes unstable and it is difficult to maintain straight-ahead stability, or the riding comfort of the passengers or the like may be impaired.

[0054] Therefore, the surrounding environment estimation unit 301 acquires the captured image in front of the vehicle 1 captured by the imaging unit 15c provided at the front of the vehicle 1, and inputs it to the surrounding environment estimation model 311. Here, in the present embodiment, as shown in FIG. 8, a blowing flow indicating the strength of the wind is provided at the place where the tunnel 810 is exited, and it is assumed that this blowing flow is also reflected in the captured image in front.

[0055] As shown in FIG. 6, the surrounding environment estimation model 311 outputs, as text, that the vehicle 1 is currently traveling in front of the exit of the tunnel 810 from the captured image in front. Further, as shown in FIG. 6, the surrounding environment estimation model 311 estimates the distance to the tunnel exit, the wind condition, etc., and outputs them as text.

[0056] Returning to FIG. 4, the vehicle behavior estimation unit 302 estimates the vehicle state, which is the behavior of the vehicle 1 due to the influence of the surrounding environment, based on the output result of the surrounding environment estimation unit 301. Here, as the vehicle state, for example, at least one state among the six degrees of freedom of roll, yaw, pitch, vertical acceleration, lateral acceleration, and longitudinal acceleration corresponds.

[0057] In the present embodiment, the vehicle behavior estimation unit 302 estimates the vehicle state using the vehicle behavior estimation model 312 stored in the storage unit 310 from the output result of the surrounding environment estimation unit 301. Here, the vehicle behavior estimation model 312 is a learned model that inputs the output result and outputs the vehicle state, and is pre-learned by machine learning such as deep learning.

[0058] The control device selection unit 303 selects the device to be controlled using the device control model 313 stored in the storage unit 310 from the vehicle state estimated by the vehicle behavior estimation unit 302, and determines the control instruction for the device to be controlled. Here, the device is various actuators.

[0059] The device control model 313 is a learned model that inputs the vehicle state and outputs the device to be controlled and the control instruction for the device, and is pre-learned by machine learning such as deep learning.

[0060] FIG. 9 is a diagram showing output examples of the vehicle behavior estimation model 312 and the device control model 313 according to the embodiment. In FIG. 9, as output examples of the vehicle behavior estimation model 312, the magnitude of yaw and the magnitude of lateral G as vehicle states are taken as examples, but it is not limited thereto. The magnitude of yaw and the magnitude of lateral G in FIG. 9 change according to the output (scene of the surrounding environment) of the surrounding environment estimation model 311.

[0061] Also, in the example of FIG. 9, the device control model 313 outputs the degree of each control of damping force control, stabilizer control, steering control, and driving force control as a control instruction for the device. These values also differ according to the values of the magnitude of yaw and the magnitude of lateral G.

[0062] Returning to FIG. 4, the vehicle control unit 340 executes control of the vehicle 1 based on the output result by the above-described surrounding environment estimation unit 301. That is, the vehicle control unit 340, according to the vehicle state estimated based on the above output result by the vehicle behavior estimation unit 302, the device to be controlled obtained based on the vehicle state by the control device selection unit 303, and the control instruction for the device to be controlled, executes at least one of suspension damping force control, steering control, stabilizer control, suspension spring constant control, and driving force control in the vehicle 1.

[0063] As shown in FIG. 4, the vehicle control unit 340 includes a damping force control unit 344, a spring constant switching control unit 345, a steering control unit 346, a stabilizer control unit 347, and a driving force control unit 348.

[0064] The damping force control unit 344 controls the actuators 103 and 106, which are the devices to be controlled determined by the control device selection unit 303, in accordance with the control instruction, and executes damping force control of the suspension system 30. Specifically, the damping force control unit 344 executes damping force control of the suspension system 30 by controlling the actuators 103 and 106 to adjust the spring constant of the upper spring of the suspension system 30 or the damping coefficient of the shock absorber of the suspension system 30.

[0065] The spring constant switching control unit 345 controls the actuators 103 and 106, which are the devices to be controlled determined by the control device selection unit 303, in accordance with the control instruction, and executes switching control of the spring constant of the upper spring in the suspension system 30.

[0066] The steering control unit 346 controls the actuators 101 and 104, which are the devices to be controlled determined by the control device selection unit 303, in accordance with the control instruction, and executes steering control of the steering system 13.

[0067] The stabilizer control unit 347 controls the actuators 102 and 105, which are the devices to be controlled determined by the control device selection unit 303, in accordance with the control instruction, and executes stabilizer control of the suspension system 30.

[0068] The driving force control unit 348 controls the actuator 107, which is the device to be controlled determined by the control device selection unit 303, in accordance with the control instruction, and executes driving force control of the vehicle 1.

[0069] (Vehicle control process) Next, the vehicle control process by the vehicle control device 14 according to the present embodiment will be described. FIG. 10 is a flowchart showing an example of the procedure of vehicle control processing according to the embodiment.

[0070] First, the surrounding environment estimation unit 301 acquires a captured image obtained by imaging the surrounding area of the vehicle 1 from the imaging unit 15 (S101). The surrounding environment estimation unit 301 estimates the future surrounding environment from the acquired captured image using the surrounding environment estimation model 311 (S102). Specifically, as described above, the output result from the surrounding environment estimation model 311 is acquired.

[0071] Next, the vehicle behavior estimation unit 302 estimates the vehicle behavior, that is, the vehicle state, from the output result obtained in S102 using the vehicle behavior estimation model 312 (S103).

[0072] Next, the control device selection unit 303 selects a control target device from the vehicle state estimated in S103 using the device control model 313 (S104). The control device selection unit 303 also determines a control instruction for the control target device from the vehicle state estimated in S103. Then, the vehicle control unit 340 controls the vehicle 1 by performing the control instruction determined by the control device selection unit 303 on the control target device selected by the control device selection unit 303. Specifically, each control is executed on the vehicle 1 by the damping force control unit 344, the spring constant switching control unit 345, the steering control unit 346, the stabilizer control unit 347, and the driving force control unit 348 of the vehicle control unit 340.

[0073] (Summary) As described above, in this embodiment, the vehicle control device 14 acquires a captured image obtained by imaging the periphery of the vehicle 1 from the imaging unit 15, and inputs the acquired captured image into the surrounding environment estimation model 311 configured by a base model, which is a large-scale model trained using diverse and large-scale data and applicable to various tasks, to obtain an output result regarding the future surrounding environment output from the surrounding environment estimation model 311, and includes a surrounding environment estimation unit 301 and a vehicle control unit 340 that executes damping force control of the vehicle 1 based on the output result.

[0074] Therefore, according to this embodiment, by using the surrounding environment estimation model 311 composed of the base model to recognize the environmental changes around the vehicle 1, the future surrounding environment of the vehicle 1 can be accurately estimated, so that both the stability of the vehicle posture and the riding comfort can be improved. That is, in this embodiment, the imaging unit 15 captures various environmental changes around the vehicle 1, such as the approach of other vehicles from the rear of the vehicle 1, along the coast, bridges, tunnel exits, etc., and uses the surrounding environment estimation model 311 constructed by the base model to recognize these as wind condition changes, thereby stabilizing the posture of the vehicle 1 and ensuring an optimal riding comfort.

[0075] In addition, in this embodiment, the vehicle control unit 340 of the vehicle control device 14 further executes at least one of steering control, stabilizer control, spring constant switching control, and driving force control of the vehicle 1 based on the output result of the surrounding environment estimation unit 301.

[0076] Therefore, according to this embodiment, both the stability of the vehicle posture and the riding comfort can be improved more accurately. For example, during high-speed driving, in order to prevent the deterioration of straight-line stability due to the instability of the vehicle posture caused by changes in wind conditions, etc., it is preferable to increase the damping force of the shock absorber, for example. However, on the contrary, the riding comfort deteriorates. Therefore, in this embodiment, by enabling the execution of steering control, the stability of the vehicle posture, that is, the straight-line stability, is ensured, so that it is not necessary to increase the damping force, and an optimal riding comfort can be ensured. Thus, according to this embodiment, both the stability of the vehicle posture and the riding comfort can be further improved.

[0077] In addition, in this embodiment, the vehicle control device 14 includes a vehicle behavior estimation unit 302 that estimates a vehicle state indicating at least one state of roll, yaw, pitch, vertical acceleration, lateral acceleration, and longitudinal acceleration, which are the behaviors of the vehicle 1 due to the influence of the surrounding environment, based on the output result of the surrounding environment estimation unit 301. Therefore, according to this embodiment, since vehicle control is further performed based on the vehicle state, both the stability of the vehicle posture and the riding comfort can be further improved.

[0078] Also, in the present embodiment, in the vehicle control device 14, the captured image is an image capturing the rear of the vehicle 1, and the surrounding environment estimation model 311 inputs the captured image capturing the rear of the vehicle 1. When there is a following vehicle approaching from the rear in the captured image, it outputs, as an output result, that the situation is such that an overtaking by the following vehicle is imminent, the size and shape of the following vehicle, the speed difference between the vehicle and the following vehicle, and the distance between the following vehicle and the vehicle when the following vehicle overtakes the vehicle. Therefore, according to the present embodiment, the imaging unit 15 captures the environmental change around the vehicle 1 such as the approach of another vehicle from the rear of the vehicle 1, and by recognizing the wind condition change using the surrounding environment estimation model 311 constructed by the base model, the posture of the vehicle 1 can be made more stable and an optimal riding comfort can be ensured.

[0079] Also, in the present embodiment, in the vehicle control device 14, the captured image is an image capturing the front of the vehicle 1, and the surrounding environment estimation model 311 inputs the captured image capturing the front of the vehicle 1. When there is a blowing flow near the tunnel exit and at the tunnel exit in the captured image, it outputs, as an output result, that the vehicle 1 is in front of the tunnel exit, the distance to the tunnel exit, and the wind condition. Therefore, according to the present embodiment, the imaging unit 15 captures the environmental change around the vehicle 1 such as the vehicle 1 traveling in front of the tunnel exit, and by recognizing the wind condition change using the surrounding environment estimation model 311 constructed by the base model, the posture of the vehicle 1 can be made more stable and an optimal riding comfort can be ensured.

[0080] (Modification example) Various modification examples can be considered for the above embodiment. The surrounding environment estimation model 311 of this embodiment is constructed with a base model. Furthermore, the surrounding environment estimation model 311 may be constructed using a learned model learned by reinforcement learning from human feedback (RLHF: Reinforcement Learning from Human Feedback) from the driver, passengers, or third parties of the vehicle 1. In this case, since the surrounding environment estimation model 311 reflects human feedback and outputs the output result, the recognition of the surrounding environment becomes closer to human perception. As a result, the posture of the vehicle 1 can be stabilized more and a more optimal riding comfort can be ensured.

[0081] The vehicle behavior estimation model 312 of this embodiment is a learned model. As an example of the learned model, a large language model (LLM) can be used. In this case, the output from the vehicle behavior estimation model 312 can be the text of a language, and the vehicle state can be estimated more accurately. As a result, the posture of the vehicle 1 can be stabilized more and a more optimal riding comfort can be ensured.

[0082] In this case, the vehicle behavior estimation model 312 may be constructed using a learned model learned by reinforcement learning (RLHF) from the driver, passengers, or third parties of the vehicle 1. In this case, since the vehicle behavior estimation model 312 reflects human feedback and outputs the output result, the estimation of the vehicle state becomes closer to human perception. As a result, the posture of the vehicle 1 can be stabilized more and a more optimal riding comfort can be ensured.

[0083] Also, in the above embodiment, the vehicle behavior estimation unit 302 estimates the vehicle state using the vehicle behavior estimation model 312, but it is not limited to this. For example, the vehicle behavior estimation unit 302 can be configured to estimate the vehicle state based on a predetermined rule from the output result of the surrounding environment estimation unit 301 without using a learned model such as the vehicle behavior estimation model 312. In this case, there is an advantage that the labor of pre-learning the learned model can be omitted.

[0084] Also, although the device control model 313 of this embodiment is a learned model, as an example of a learned model, a large language model (LLM) can be used. In this case, the output from the device control model 313 can be the text of a language, and the control target device and the control instructions to the control target device can be estimated more accurately. As a result, the posture of the vehicle 1 can be made more stable and a more optimal riding comfort can be ensured.

[0085] In this case, the device control model 313 may be constructed as a learned model learned by reinforcement learning from human feedback (RLHF) from the driver, passengers, or a third party of the vehicle 1. In this case, since the device control model 313 reflects human feedback and outputs an output result, the selection of the control target device and the determination of the control instructions to the control target device become closer to human perception. As a result, the posture of the vehicle 1 can be made more stable and a more optimal riding comfort can be ensured.

[0086] Also, in the above embodiment, the control device selection unit 303 uses the device control model 313 to select the control target device and determine the control instructions to the control target device, but it is not limited to this. For example, without using a learned model such as the device control model 313, based on a predetermined rule from the vehicle state estimated by the vehicle behavior estimation unit 302, the control device selection unit 303 can be configured to select the control target device and determine the control instructions to the control target device. In this case, there is an advantage that the effort of pre-learning a learned model can be omitted.

[0087] In the above-described embodiment, the vehicle behavior estimation model 312 and the device control model 313 are provided separately, but they may be configured as a single learning model. That is, a learned model that inputs the output result of the surrounding environment estimation unit 301 and outputs the vehicle state, the selection of the control target device, and the control instruction to the control target device is learned and constructed, and the vehicle behavior estimation unit 302 and the control device selection unit 303 are configured to use the learned model. In this case, since a single learned model can output the vehicle state, the selection of the control target device, and the control instruction to the control target device, it is possible to quickly perform vehicle control processing.

[0088] In the above-described embodiment, the CPU 14a reads and executes a program stored in a storage device such as the ROM 14b or the SSD 14f, thereby realizing each functional unit, that is, various functional modules such as the surrounding environment estimation unit 301, the vehicle behavior estimation unit 302, the control device selection unit 303, and the vehicle control unit 340 (the damping force control unit 344, the spring constant switching control unit 345, the steering control unit 346, the stabilizer control unit 347, the driving force control unit 348). However, the present invention is not limited to this. For example, various functional modules such as the surrounding environment estimation unit 301, the vehicle behavior estimation unit 302, the control device selection unit 303, and the vehicle control unit 340 (the damping force control unit 344, the spring constant switching control unit 345, the steering control unit 346, the stabilizer control unit 347, the driving force control unit 348) can also be realized by independent hardware such as a circuit including an ASIC (Application Specific Integrated Circuit).

[0089] Note that the vehicle control program executed by the vehicle control device 14 according to the above-described embodiment is provided by being pre-embedded in the ROM 14b or the like.

[0090] The vehicle control program executed by the vehicle control device 14 according to the above embodiment may be configured to be recorded and provided on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, a DVD (Digital Versatile Disk), etc. in an installable or executable file format.

[0091] Furthermore, the vehicle control program executed by the vehicle control device 14 according to the above embodiment may be configured to be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Also, the vehicle control program executed by the vehicle control device 14 according to the above embodiment may be configured to be provided or distributed via a network such as the Internet.

[0092] The vehicle control program executed by the vehicle control device 14 according to the above embodiment has a module configuration including the above-described respective functional units, that is, the surrounding environment estimation unit 301, the vehicle behavior estimation unit 302, the control device selection unit 303, the vehicle control unit 340 (the damping force control unit 344, the spring constant switching control unit 345, the steering control unit 346, the stabilizer control unit 347, the driving force control unit 348), etc. As actual hardware, the CPU reads the vehicle control program from the above ROM and executes it, whereby the above units are loaded onto the main storage device, and the surrounding environment estimation unit 301, the vehicle behavior estimation unit 302, the control device selection unit 303, the vehicle control unit 340 (the damping force control unit 344, the spring constant switching control unit 345, the steering control unit 346, the stabilizer control unit 347, the driving force control unit 348), etc. are generated on the main storage device.

[0093] Although some embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalent scope thereof.

[0094] (Appendix) In the above vehicle control device, the output result is text indicating the surrounding environment. In the above vehicle control device, the output result is a feature quantity analyzable by a learned model.

[0095] In the above vehicle control device, the vehicle behavior estimation unit estimates the vehicle state using a vehicle behavior estimation model, which is a learned model that inputs the output result and outputs the vehicle state from the output result.

[0096] The above vehicle control device further includes a device selection unit that selects a device to be controlled and determines a control instruction for the device to be controlled, using a device control model, which is a learned model that inputs the vehicle state and outputs a control target device and a control instruction for the device from the vehicle state estimated by the vehicle behavior estimation unit.

Explanation of Reference Numerals

[0097] 1... Vehicle, 14... ECU (Vehicle Control Device), 15, 15a, 15c... Imaging Unit (Imaging Device), 301... Estimation Unit, 302... Vehicle Behavior Estimation Unit, 303... Control Device Selection Unit, 340... Vehicle Control Unit, 344... Damping Force Control Unit, 345... Spring Constant Switching Control Unit, 346... Steering Control Unit, 347... Stabilizer Control Unit, 348... Driving Force Control Unit, 311... Surrounding Environment Estimation Model, 312... Vehicle Behavior Estimation Model, 313... Device Control Model.

Claims

1. A peripheral environment estimation unit that acquires a captured image capturing the periphery of a vehicle from an imaging device, inputs the acquired captured image into a peripheral environment estimation model composed of a base model, which is a large-scale model trained using diverse and large-scale data, and obtains an output result regarding the future peripheral environment output from the peripheral environment estimation model; A vehicle behavior estimation unit that estimates a vehicle state based on the output result; A vehicle control unit that executes suspension damping force control of the vehicle based on the vehicle state; A vehicle control device comprising the above.

2. The vehicle control unit further executes at least one of steering control, stabilizer control, suspension spring constant switching control, and driving force control in the vehicle based on the output result. The vehicle control device according to Claim 1.

3. The vehicle behavior estimation unit estimates a vehicle state indicating at least one state of roll, yaw, pitch, vertical acceleration, lateral acceleration, and longitudinal acceleration, which are the behaviors of the vehicle due to the influence of the peripheral environment. The vehicle control device according to Claim 1.

4. The peripheral environment estimation model outputs, as the output result, the wind situation that the vehicle is subjected to due to the influence of the peripheral environment. The vehicle control device according to Claim 1.

5. The captured image is an image capturing the rear of the vehicle. The peripheral environment estimation model inputs a captured image capturing the rear of the vehicle, and when there is a rear vehicle approaching from the rear in the captured image, outputs, as the output result, that a situation where an overtaking by the rear vehicle is imminent, the size and shape of the rear vehicle, the speed difference between the vehicle and the rear vehicle, and the distance between the rear vehicle and the vehicle when the rear vehicle overtakes the vehicle. The vehicle control device according to Claim 3.

6. The captured image is an image capturing the front of the vehicle. The peripheral environment estimation model inputs a captured image capturing the front of the vehicle, and when there is a tunnel exit and a blowing current near the exit in the captured image, outputs, as the output result, that the vehicle is in front of the tunnel exit, the distance to the tunnel exit, and the wind situation. The vehicle control device according to Claim 3.

Images