Vehicle control device

Abstract

To reliably achieve an optimum vehicle control state by adjusting a vehicle control state according to a human sensation without requiring a long time for learning.SOLUTION: A vehicle control device includes: a vehicle characteristic calculation unit that obtains vehicle characteristic information indicating characteristics of a vehicle on the basis of a picked-up image obtained by picking up an image of the surroundings of the vehicle and a detected value of a state of the vehicle; a learning unit that obtains optimum vehicle characteristic information through reinforcement learning; and a control unit that controls the vehicle on the basis of the optimum vehicle characteristic information. The learning unit includes: a feedback unit that performs feedback to an action based on a predetermined measure for an output value of a large scale language model which is a learned model that receives the picked-up image as an input and outputs the output value including a road surface situation on the basis of a sensation of a driver or an occupant of the vehicle or a third party; a reward model that reflects the feedback and calculates a reward for the action for learning; and a determination unit that determines, as the optimum vehicle characteristic information, vehicle characteristic information in which the reward becomes maximum.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a vehicle control device. [Background technology]

[0002] There has been known a vehicle control device that controls various devices such as ride comfort and driving stability to optimize vehicle characteristics based on detection values ​​of various on-board sensors such as a vertical acceleration sensor and a vehicle height sensor while the vehicle is traveling. In such a vehicle control device, a technology is known that utilizes reinforcement learning to optimally control the vehicle state.

[0003] For example, the technology of Patent Document 1 employs a reinforcement learning method in which a planner's action (measure) is input to the environment, the planner receives the state and reward obtained from the environment, and the next action (measure) is reviewed to optimize the reward in order to optimally control the vehicle state. Also, the technology of Patent Document 2 uses the biological signals of the occupants to perform reinforcement learning to optimally control the vehicle state. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 7048456 [Patent Document 2] Patent No. 7011553 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in such conventional technology, the vehicle state is optimized by policy-based reinforcement learning, so the vehicle control state may not match human senses. In addition, when multiple control devices are used and a large number of parameters are required, the amount of data becomes huge, and it may take a long time to learn using such a large amount of data. For this reason, in the conventional technology, the optimal solution for the vehicle state may not be converged to, and the goal may not be achieved.

[0006] The present invention has been made in consideration of the above, and one of its objects is to provide a vehicle control device that can reliably achieve an optimal vehicle control state by adapting the vehicle control state to human senses without requiring a large amount of time for learning. [Means for solving the problem]

[0007] The vehicle control device of the present invention includes a vehicle characteristic calculation unit that determines vehicle characteristic information indicating the characteristics of the vehicle based on an image captured of the surroundings of the vehicle and a detection value of the state of the vehicle, a learning unit that determines optimal vehicle characteristic information by reinforcement learning, and a control unit that controls the vehicle based on the optimal vehicle characteristic information, wherein the learning unit includes a feedback unit that provides feedback based on the senses of the driver, passengers, or a third party of the vehicle in response to an action based on a predetermined policy for the output value of a large-scale language model, which is a trained model that inputs the captured image and outputs an output value including road surface conditions, a reward model that reflects the feedback and calculates and learns a reward for the action, and a determination unit that determines the vehicle characteristic information that maximizes the reward as the optimal vehicle characteristic information. Effect of the Invention

[0008] According to the vehicle control device of the present invention, it is possible to reliably achieve an optimal vehicle control state by adapting the vehicle control state to human sensations without requiring a significant amount of time for learning. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a top view of a vehicle equipped with a vehicle control device according to an embodiment. [Diagram 2] FIG. 2 is a perspective view of the interior of a vehicle cabin in which a vehicle control device according to an embodiment is mounted, as viewed from above. [Diagram 3] FIG. 3 is a block diagram illustrating an example of an overall configuration of a vehicle control system according to an embodiment. [Figure 4] FIG. 4 is a block diagram illustrating an example of a functional configuration of the vehicle control device according to the embodiment. [Diagram 5] FIG. 5 is a diagram illustrating an example in which the vehicle control device according to the embodiment calculates vibrations applied to an occupant from vibrations applied to the vehicle and vibrations of the occupant relative to the vehicle. [Figure 6] FIG. 6 is a diagram showing some examples of the state of an occupant that can be estimated from an image captured by the inner camera according to the embodiment. [Figure 7] FIG. 7 is a block diagram illustrating an example of a functional configuration of a learning unit according to the embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of a reward that an agent can obtain according to the embodiment. [Figure 9] FIG. 9 is a diagram illustrating a specific example of human feedback according to the embodiment. [Figure 10] FIG. 10 is a diagram showing an example of a screen when human feedback is performed by inputting an operation button according to the embodiment. [Figure 11] FIG. 11 is a diagram showing another example of a screen when human feedback is performed by inputting an operation button according to the embodiment. [Figure 12] FIG. 12 is a diagram showing a specific example of biofeedback according to the embodiment. [Figure 13] FIG. 13 is a diagram illustrating an example of an electroencephalogram according to the embodiment. [Figure 14] FIG. 14 is a graph showing an example of the relationship between the electroencephalogram and biological information measurable during running according to the embodiment. [Figure 15]FIG. 15 is a flowchart illustrating an example of a procedure of the vehicle control process according to the embodiment. [Figure 16] FIG. 16 is a flowchart illustrating an example of a procedure of the learning process according to the embodiment. [Figure 17] FIG. 17 is a diagram illustrating a specific example of the flow of the learning process according to the embodiment. [Figure 18] FIG. 18 is a graph showing an example of the relationship between the index (parameter) of ride comfort and the control current in the embodiment. [Figure 19] FIG. 19 is a diagram illustrating an example of a compensation model and a policy according to an embodiment. [Figure 20] FIG. 20 illustrates an example of a modified rank and probability based reward model and behavior decision strategy in an embodiment. [Figure 21] FIG. 21 is a graph showing the change in sprung acceleration of the suspension 41 in the embodiment with respect to frequency. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Exemplary embodiments of the present invention are disclosed below. The configurations of the embodiments described below, as well as the actions, results, and effects brought about by the configurations, are merely examples. The present invention can be realized with configurations other than those disclosed in the following embodiments, and it is possible to obtain at least one of various effects based on the basic configuration and derivative effects.

[0011] (Embodiment) (Vehicle configuration example) The vehicle 10 of this embodiment may be, for example, an automobile using an internal combustion engine (not shown) as a drive source, i.e., an internal combustion engine automobile, an automobile using an electric motor (not shown) as a drive source, i.e., an electric automobile or a fuel cell automobile, a hybrid automobile using both of them as drive sources, or an automobile equipped with other drive sources. The vehicle 1 may be equipped with various transmissions and various devices, such as systems and parts, required for driving the internal combustion engine or the electric motor. The type, number, layout, etc. of the devices related to driving the wheels 3 of the vehicle 1 may be set in various ways.

[0012] Fig. 1 is a top view of a vehicle 10 equipped with a vehicle control device 20 according to an embodiment. Note that Fig. 1 omits illustration and description of some components described later in Fig. 2. Also, the front, rear, left and right of the vehicle 10 in Fig. 1 and Fig. 2 described later indicate the directions when viewed from the driver's seat of the vehicle 10. Note that the type, number, layout, etc. of devices related to driving the wheels 13 of the vehicle 10 can be set in various ways.

[0013] 1, a vehicle 10 includes a vehicle body 12, a plurality of wheels 13, and a plurality of outer cameras 16a to 16d. When it is not necessary to distinguish between the individual outer cameras 16a to 16d, they will be simply referred to as outer cameras 16.

[0014] The vehicle body 12 constitutes a vehicle cabin in which passengers ride. A plurality of wheels 13 and a plurality of outer cameras 16 are attached to the vehicle body 12. In the example of Fig. 1, the vehicle body 12 is equipped with four wheels 13 and four outer cameras 16. However, the number of outer cameras 16 attached to the vehicle body 12 is arbitrary.

[0015] The four wheels 13 are provided on the front, rear, left and right sides of the vehicle body 12. The two front wheels 13 function, for example, as steering wheels, and the two rear wheels 13 function, for example, as driving wheels.

[0016] The outer camera 16 is a digital camera incorporating an imaging element such as a CCD (Charge Coupled Device) or a CIS (CMOS Image Sensor), etc. The outer camera 16 generates a moving image including a plurality of frame images captured at a predetermined frame rate, or a still image.

[0017] The outer cameras 16 are provided on the outer periphery of the vehicle body 12, each having a wide-angle lens or a fisheye lens, and capable of capturing images in a horizontal range of, for example, 140° to 190°. The optical axis of the outer cameras 16 is set to face diagonally downward.

[0018] As a result, the outer camera 16 collects peripheral images capturing the surroundings of the vehicle 10, including the road surface, and outputs the images to the vehicle control device 20. Based on the peripheral images collected by the outer camera 16, the vehicle control device 20 can detect vibrations acting on the vehicle 10 while it is traveling on a road surface or the like.

[0019] The outer camera 16a is provided in the center in the left-right direction of the front end of the vehicle body 12, for example, on the front bumper. The outer camera 16a collects captured images of the area in front of the vehicle 10 as peripheral images. The outer camera 16b is provided in the center in the left-right direction of the rear end of the vehicle body 12, for example, on the rear bumper. The outer camera 16b collects captured images of the area behind the vehicle 10 as peripheral images.

[0020] The outer camera 16c is provided in the center in the longitudinal direction of the left end of the vehicle body 12, for example, on the left side mirror. The outer camera 16c collects captured images of the left side of the vehicle 10 as peripheral images. The outer camera 16d is provided in the center in the longitudinal direction of the right end of the vehicle body 12, for example, on the right side mirror. The outer camera 16d collects captured images of the surroundings to the right of the vehicle 10 as peripheral images.

[0021] 2 is a perspective view of the interior of a vehicle cabin of a vehicle 10 equipped with a vehicle control device 20 according to an embodiment, as viewed from above. Note that in FIG. 2, some of the configurations described above in FIG. 1 are omitted.

[0022] 2, a plurality of seats 2a to 2e and a plurality of inner cameras 14a, 14b are provided in the cabin of the vehicle 10. When it is not necessary to distinguish between the individual seats 2a to 2e, they will be simply referred to as seats 2, and when it is not necessary to distinguish between the individual inner cameras 14a, 14b, they will be simply referred to as inner cameras 14.

[0023] Of the multiple seats 2a to 2e, a driver's seat 2a and a passenger seat 2b are provided at the front side of the vehicle interior, and multiple rear seats 2c to 2e are provided at the rear side. In the example of Fig. 2, a total of five seats 2a to 2e are provided in the vehicle interior. However, the number of seats 2 provided in the vehicle interior is arbitrary.

[0024] Of the multiple rear seats 2c to 2e, the rear seat 2c is provided behind the driver's seat 2a, the rear seat 2d is provided behind the passenger seat 2b, and the rear seat 2e is provided between the rear seats 2c and 2d. In the example of Fig. 2, the multiple rear seats 2c to 2e are so-called bench seat type seats with continuous seat surfaces. However, the rear seats 2c to 2e may be provided independently of each other, for example, like the driver's seat 2a and the passenger seat 2b.

[0025] The inner camera 14 is a digital camera incorporating an imaging element such as a CCD or CIS, similar to the outer camera 16. The inner camera 14 also generates a moving image including a plurality of frame images captured at a predetermined frame rate, or a still image.

[0026] The inner camera 14 is provided in the vehicle cabin, collects occupant images of occupants in the vehicle cabin, and outputs the images to the vehicle control device 20. The vehicle control device 20 can detect vibrations of the occupant relative to the vehicle 10 based on the occupant images collected by the inner camera 14.

[0027] Of the inner cameras 14a and 14b, the inner camera 14a is installed at a position where it can capture images of occupants at the front of the vehicle cabin. The inner camera 14a collects captured images of at least an occupant seated in the driver's seat 2a and an occupant seated in the passenger seat 2b as occupant images.

[0028] Of the inner cameras 14a and 14b, the inner camera 14b is installed at a position where it can capture images of passengers at the rear of the vehicle cabin. The inner camera 14b collects captured images of at least the passengers seated in the rear seats 2c to 2e as passenger images.

[0029] In the example of Fig. 2, two inner cameras 14a, 14b are provided in the vehicle cabin as described above. However, the number of inner cameras 14 provided in the vehicle cabin is arbitrary.

[0030] Furthermore, regardless of the example of Figure 2, if the vehicle 10 is a bus-type vehicle or the like capable of transporting a large number of passengers, the inner camera 14 will be able to capture images of not only passengers sitting in any of the seats, but also passengers who are standing or seated in a wheelchair, etc.

[0031] The vehicle control device 20 of the embodiment calculates the net vibration acting on the occupants of the vehicle 10 based on the images captured by the outer camera 16 and the inner camera 14. Furthermore, the vehicle control device 20 controls each part of the vehicle 10 based on the calculated net vibration to suppress vibration that the occupants find uncomfortable.

[0032] (Example of vehicle control system configuration) FIG. 3 is a block diagram illustrating an example of an overall configuration of a vehicle control system according to an embodiment.

[0033] 3, the vehicle control system includes a vehicle control device 20, a monitor device 30, a vibration control system 40, a lateral vibration control system 50, a steering system 60, an inner camera 14, and an outer camera 16. These components are connected to each other via an in-vehicle network NT so that they can transmit and receive information to each other.

[0034] The in-vehicle network NT includes, for example, a Controller Area Network (CAN) and a Local Interconnect Network (LIN), etc. The in-vehicle network NT may be included as a part of the vehicle control system 200.

[0035] The vehicle control device 20 is configured as a microcomputer such as an ECU (Electronic Control Unit), and controls each part of the vehicle 10, including vibration control.

[0036] The vehicle control device 20 includes a CPU (Central Processing Unit) 21, a display control circuit 23, an SSD (Solid State Drive) 24, a ROM (Read Only Memory) 25, and a RAM (Random Access Memory) 26. The CPU 21, the ROM 25, and the RAM 26 may be integrated in the same package.

[0037] The CPU 21 is an example of a hardware processor, which reads out a program stored in a non-volatile storage device such as the ROM 25, and executes various types of arithmetic processing and control in accordance with the program.

[0038] The ROM 25 stores various programs and parameters necessary for executing the programs. The RAM 26 temporarily stores various data used in the calculations by the CPU 21. The SSD 24 is a rewritable non-volatile storage device, and maintains data even when the power to the vehicle control device 20 is turned off.

[0039] The display control circuit 23 generates various data to be displayed on a display unit 31 (to be described later) provided in the monitor device 30.

[0040] The vibration control system 40 has a suspension 41, a suspension control section 42, a spring sensor 43, and two acceleration sensors 44, and mainly suppresses vibration of the vertical component of the vehicle 10. The vibration control system 40 is an example of a vibration control device.

[0041] The suspension 41 is a device including, for example, suspension arms that support the axles, springs that absorb shocks applied to the wheels 13, etc., and shock absorbers that dampen the shaking of the body 12 caused by the expansion and contraction of the springs, and reduces vibrations of the vehicle 10. The suspension control unit 42 is, for example, a microcomputer having a hardware processor such as a CPU. The suspension control unit 42 controls the spring constant of the springs included in the suspension 41 and the damping force of the shock absorbers based on the detection results of the spring sensor 43, etc. The spring sensor 43 detects, for example, the expansion and contraction of the springs included in the suspension 41.

[0042] However, it is also possible to substitute the spring sensor 43 of the suspension 41 with the above-mentioned outer camera 16. Since the inclination of the vehicle body 12 can be detected from the image captured by the outer camera 16, the amount of expansion and contraction of the springs geometrically arranged in the suspension 41 can also be estimated from the image captured by the outer camera 16.

[0043] The two acceleration sensors 44 are provided respectively in the vicinity of a portion of the vehicle 10 on the vehicle body 12 side relative to the suspension 41 (also referred to as sprung part) and a portion of the vehicle 10 on the wheel 13 side relative to the suspension 41 (also referred to as unsprung part). From the acceleration sensors provided respectively in the vicinity of the sprung part relative to the suspension 41 and the unsprung part relative to the suspension 41, vehicle characteristics based on the sprung part acceleration or the unsprung part acceleration, as well as high frequency vibration, low frequency vibration, roll rate, etc., can be obtained. In addition, a vehicle height sensor or the like may be provided in the vibration control system 40.

[0044] The lateral oscillation control system 50 has a stabilizer 51, a stabilizer control unit 52, and a lateral oscillation sensor 53, and mainly suppresses vibration of the horizontal component of the vehicle 10. The lateral oscillation control system 50 is an example of a vibration control device.

[0045] The stabilizer 51 is, for example, a device including connecting parts added to the suspension 41, and corrects the inclination of the axle of the vehicle 10. The stabilizer control unit 52 is, for example, a microcomputer having a hardware processor such as a CPU. The stabilizer control unit 52 controls the stabilizer 51 based on the detection result of the lateral sway sensor 53. The lateral sway sensor 53 is, for example, a position sensor such as an acceleration sensor, and detects the lateral sway of the vehicle 10.

[0046] However, it is also possible to use the above-mentioned outer camera 16 instead of the lateral oscillation sensor 53 of the stabilizer 51. According to the image captured by the outer camera 16, it is also possible to detect the lateral oscillation of the vehicle body 12.

[0047] The steering system 60 has a steering unit 61, a steering control unit 62, and a steering sensor 63, and controls the traveling direction of the vehicle 10. The steering system 60 is an example of a vibration control device.

[0048] The steering unit 61 is a device including, for example, a steering wheel, and steers the steered wheels of the vehicle 10 to steer the traveling direction of the vehicle 10. The steering control unit 62 is a microcomputer including a hardware processor such as a CPU. The steering control unit 62 controls the traveling direction of the vehicle 10 based on the operation of the steering wheel or steering wheel by the driver. The steering sensor 63 is an angle sensor including, for example, a Hall element, and detects the steering angle, which is the rotation angle of the steering unit 61.

[0049] The monitor device 30 is provided on an instrument panel or the like inside the passenger compartment of the vehicle 10 , and has a display unit 31 and an input unit 32 .

[0050] The display unit 31 is a display device such as a liquid crystal display (LCD) or an organic electroluminescent display (OLED). The display unit 31 displays, for example, a screen based on image data acquired by the vehicle control device 20 from the outer camera 16, a screen for presenting information to the occupant, and a screen for receiving various operational instructions from the occupant.

[0051] The input unit 32 is, for example, a touch panel provided on the display screen of the display unit 31. The input unit 32 is configured to be transparent to the contents displayed on the display screen of the display unit 31. This allows the input unit 32 to allow the occupant to visually recognize the display contents of the display unit 31.

[0052] The input unit 32 receives instructions input by the driver or the like touching a position corresponding to the display content of the display unit 31, and transmits the instructions to the vehicle control device 20 via the in-vehicle network NT. Note that the input unit 32 is not limited to a touch panel, and may be a hard switch such as a push button.

[0053] Fig. 4 is a block diagram showing an example of a functional configuration of the vehicle control device 20 according to the embodiment. As shown in Fig. 4, the vehicle control device 20 mainly includes a display control unit 201, an acquisition unit 204, a vehicle characteristic calculation unit 210, a learning unit 208, and a control unit 207 as functional units.

[0054] The display control unit 201 generates contents to be displayed on the display unit 31 of the monitor device 30, and causes the display unit 31 to display them.

[0055] The acquisition unit 204 acquires captured images of the periphery of the vehicle 10 from the outer camera 16 as images of the periphery of the vehicle 10. The acquisition unit 204 also acquires captured images of the interior of the vehicle from the inner camera 14 as images of passengers in the vehicle. Furthermore, the acquisition unit 204 acquires detection signals from each of the spring sensor 43, the acceleration sensor 44, the lateral vibration sensor 53, and the steering sensor 63.

[0056] The vehicle characteristic calculation unit 210 calculates vehicle characteristic information indicating the characteristics of the vehicle 10 based on the captured image of the surroundings of the vehicle 10 captured by the outer camera 16 acquired by the acquisition unit 204 and the detection signals from each sensor acquired by the acquisition unit 204, i.e., the detection values ​​of the state of the vehicle 10.

[0057] As shown in FIG. 4, the vehicle characteristic calculation unit 210 mainly includes a vibration calculation unit 202, a signal generation unit 203, a vehicle state estimation unit 205, and an occupant state estimation unit 206 as functional units.

[0058] The vehicle state estimation unit 205 estimates vibrations acting on the vehicle 10 while it is traveling on a road surface or the like, based on the surrounding image acquired by the acquisition unit 204.

[0059] The vibrations applied to the vehicle 10 due to road surface conditions and the like include, for example, up and down movement (pitch) which is a vertical component vibration, lateral shaking (roll) which is a horizontal component vibration, and lifting (heave) of the vehicle body 12. The vehicle state estimation unit 205 identifies at least the vertical component vibration among these. However, it is preferable that the vehicle state estimation unit 205 is capable of identifying not only the vertical component but also the horizontal component and all other vibrations in each direction.

[0060] The vehicle state estimation unit 205 can estimate such vibrations of the vehicle 10 from the undulations and steps of the road surface captured in the peripheral image, as well as a comparison with the surrounding scenery. Here, the vibrations estimated by the vehicle state estimation unit 205 can also be referred to as vibration acceleration.

[0061] The vehicle state estimation unit 205 may estimate the vibration of the vehicle 10 at each of the installation positions of the outer cameras 16a to 16d based on the captured images acquired from each of the outer cameras 16a to 16d.

[0062] The occupant state estimation unit 206 estimates the occupant's riding position, the occupant's attributes, the occupant's state, and the occupant's vibration relative to the vehicle 10 based on the occupant image acquired by the acquisition unit 204.

[0063] Specifically, the occupant state estimation unit 206 can estimate the riding positions of the occupants in the vehicle cabin from the positions of the occupants captured in the images captured by the inner cameras 14a and 14b. If the vehicle 10 is a bus-type vehicle capable of transporting a large number of occupants, the occupant state estimation unit 206 may be capable of estimating whether the occupants are standing or sitting, or whether they are sitting in a wheelchair or the like.

[0064] In addition, the occupant state estimation unit 206 can estimate attributes of the occupant, such as the sex, age, and physical build, from the facial image and whole-body image of the occupant captured by the inner cameras 14a and 14b.

[0065] The occupant state estimation unit 206 can detect the heartbeat, breathing, and the like of the occupant from the movement of the occupant's chest captured by the inner cameras 14a and 14b. The occupant state estimation unit 206 can estimate the state of the occupant, such as the degree of fatigue and discomfort, from the detected heartbeat and breathing and the behavior of the occupant, such as blinking, captured by the inner cameras 14a and 14b.

[0066] Moreover, the inner camera 14 is attached to a part of the structure of the vehicle body 12 inside the vehicle cabin. For this reason, it can be said that the vibration of the vehicle 10 and the vibration of the inner camera 14 are substantially synchronized. Therefore, by analyzing the image captured by the inner camera 14, the vibration of the occupant relative to the vehicle 10, minus the vibration applied to the vehicle 10, can be estimated. Here, the vibration estimated by the occupant state estimation unit 206 can also be referred to as a vibration acceleration.

[0067] The occupant state estimation unit 206 may estimate vibrations of each occupant captured by the multiple inner cameras 14a, 14b relative to the vehicle 10. The occupant state estimation unit 206 may also estimate vibrations of multiple body parts including at least one of the head, chest, abdomen, and legs of each occupant relative to the vehicle 10.

[0068] As described above, the occupant state estimation unit 206 is able to estimate not only the vibration of the occupant relative to the vehicle 10 from the image of the inner camera 14, but also the occupant's three-dimensional position within the vehicle cabin and the occupant's state including vital signs, etc.

[0069] The vibration calculation unit 202 calculates the vibration applied to the occupant based on the vibration of the vehicle 10 estimated by the vehicle state estimation unit 205 and the vibration of the occupant estimated by the occupant state estimation unit 206. That is, the vibration applied to the occupant is the vibration applied to the vehicle 10 plus the vibration of the occupant relative to the vehicle 10, and is the net vibration substantially applied to the occupant. Here, the vibration calculated by the vibration calculation unit 202 can also be referred to as vibration acceleration.

[0070] The vibration calculation unit 202 may calculate the vibration applied to each of the occupants of the vehicle 10. In this case, the vibration calculation unit 202 can calculate the vibration applied to each of the occupants by associating the vibration of the vehicle 10 at each installation position of the outer cameras 16a to 16d with the riding positions of each of the occupants captured by the multiple inner cameras 14a, 14b.

[0071] Furthermore, the vibration calculation section 202 may individually calculate the vibration acting on a plurality of body parts including at least one of the head, chest, abdomen, legs, etc. of the occupant.

[0072] The signal generating unit 203 generates a signal that causes at least one of the vibration control system 40, the lateral motion control system 50, and the steering system 60 to perform control to suppress vibrations that the occupants find uncomfortable, based on the vibrations acting on the occupants.

[0073] That is, for example, the signal to the vibration control system 40 may be a control signal that causes the suspension control unit 42 of the vibration control system 40 to control at least one of the spring constant of the spring included in the suspension 41 and the damping force of the shock absorber.

[0074] Also, for example, the signal to the roll control system 50 may be a control signal that causes the stabilizer control unit 52 of the roll control system 50 to control the stabilizer 51 .

[0075] Also, for example, the signal to the steering system 60 may be a control signal that causes the steering control unit 62 of the steering system 60 to control the rear wheels, which are drive wheels among the multiple wheels 13.

[0076] When the vibrations acting on each of the multiple occupants are calculated, the signal generating unit 203 may generate a signal for performing control to reduce the average discomfort felt by all the occupants. Alternatively, the signal generating unit 203 may generate a signal for suppressing vibrations of the occupant who is most subjected to the uncomfortable vibrations among the multiple occupants.

[0077] In addition, when the vibrations acting on multiple body parts of an individual occupant are calculated, the signal generating unit 203 may generate a signal that performs control to reduce the average discomfort felt by each body part.

[0078] Alternatively, the signal generating unit 203 may generate a signal that suppresses vibrations of a body part that is more likely to cause discomfort, such as the head or abdomen, among a plurality of parts. In this case, for example, the vibration of the headrest position may be controlled for the head position of the occupant. For example, the vibration of the seat position may be controlled for the chest and abdomen positions of the occupant. For example, the vibration of the floor position may be controlled for the legs of the occupant.

[0079] As described above, the vehicle control device 20 may preset whether to target all occupants or priority occupants or body parts, etc., for control to suppress unpleasant vibrations.

[0080] Here, what frequency and strength of vibration is felt as uncomfortable may differ depending on the attributes and body part of the occupant. This is because the muscle strength supporting the body, the weight per muscle fiber, the resonance frequency of the whole body, etc. differ depending on whether the occupant is male or female, or whether the occupant is a child or an elderly person, or whether the occupant is well-built or thin, etc. Similarly, the muscle strength supporting each of the occupant's body parts such as the head, chest, abdomen, and legs, the mass of these parts per muscle fiber, and the resonance frequency of each body part differ, so the vibration frequency and vibration strength felt as uncomfortable also differ depending on the body part.

[0081] Therefore, the signal generating unit 203 may take into account the degree of discomfort caused by vibration for each occupant and for each individual body part of the occupant, based on the attributes of the occupant estimated by the above-mentioned occupant state estimating unit 206 and the vibrations applied to each of the multiple body parts, and generate a signal that performs control to reduce the discomfort.

[0082] Such a signal can be generated by storing data on vibration characteristics, such as vibration frequencies that are felt to be uncomfortable, for each occupant's attributes and body parts in advance. That is, the signal generating unit 203 can refer to this vibration characteristics data to generate a control signal so that the vibration frequency, etc. applied to the occupant approaches an appropriate state. The control signal generated by the signal generating unit 203 corresponds to the vehicle characteristic information.

[0083] The learning unit 208 obtains optimal vehicle characteristic information, i.e., a control signal, by reinforcement learning. The learning unit 208 will be described in detail later.

[0084] The control unit 207 controls the vehicle 10 by outputting the signal generated by the signal generation unit 203 to a target system among the vibration control system 40, the lateral motion control system 50, and the steering system 60.

[0085] The vibration control system 40, the lateral oscillation control system 50, and the steering system 60 control each part in the system according to a signal from the vehicle control device 20, i.e., vehicle characteristic information (signal) generated by the signal generating unit 203 and further optimized by the learning unit 208. This suppresses vibrations of the vehicle 10 that are uncomfortable for the occupants.

[0086] Next, a more detailed example of functions of the vehicle control device 20 according to the embodiment will be described with reference to FIGS.

[0087] FIG. 5 is a diagram showing an example in which the vehicle control device 20 according to the embodiment calculates the vibration applied to the occupant from the vibration applied to the vehicle 10 and the vibration of the occupant relative to the vehicle 10. In FIG.

[0088] 5, the vehicle state estimation unit 205 of the vehicle control device 20 estimates the vibration acting on the vehicle 10 based on the peripheral image acquired from the outer camera 16. The vibration acting on the vehicle 10 is obtained, for example, as a graph showing the change over time in the magnitude of the vibration.

[0089] 5 shows a graph plotting the amplitude of the vertical component of vibration among the vibrations applied to the vehicle 10. However, as described above, in addition to the vertical component of vibration, vibrations in various other directions may also be estimated.

[0090] Further, the occupant state estimation unit 206 of the vehicle control device 20 estimates the vibration of the occupant relative to the vehicle 10 based on the occupant image acquired from the inner camera 14. The vibration of the occupant relative to the vehicle 10 is obtained, for example, as a graph showing the change over time in the magnitude of the vibration, similar to the vibration applied to the vehicle 10.

[0091] 5 shows a graph plotting the amplitude of vertical component vibration applied to the head, chest, abdomen, and legs of the occupant, among the vibrations of the occupant relative to the vehicle 10. However, as described above, in addition to the vertical component vibration, vibrations in various other directions may also be estimated.

[0092] The vibration calculation unit 202 of the vehicle control device 20 adds together the vibrations acting on the vehicle 10 estimated by the vehicle state estimation unit 205 and the vibrations of each part of the occupant relative to the vehicle 10 estimated by the occupant state estimation unit 206.

[0093] That is, the vibration applied to the vehicle 10 and the vibration of the occupant's head relative to the vehicle 10 are added together to calculate the vibration substantially applied to the occupant's head. Also, the vibration applied to the vehicle 10 and the vibration of the occupant's chest relative to the vehicle 10 are added together to calculate the vibration substantially applied to the occupant's chest. Also, the vibration applied to the vehicle 10 and the vibration of the occupant's abdomen relative to the vehicle 10 are added together to calculate the vibration substantially applied to the occupant's abdomen. Also, the vibration applied to the vehicle 10 and the vibration of the occupant's legs relative to the vehicle 10 are added together to calculate the vibration substantially applied to the occupant's legs.

[0094] At this time, the vibration of the vehicle 10 that is added to the vibration of each part for each occupant is common to all.

[0095] The vehicle control device 20 can calculate the substantial vibration of each body part of the occupant shown in Fig. 5 for each occupant in the vehicle cabin. At this time, when the vehicle state estimation unit 205 estimates the vibration of the vehicle 10 applied to each riding position of the occupant, it is possible to take into account the vibration of the vehicle 10 corresponding to the riding position of each occupant for each occupant.

[0096] Incidentally, as described above, the occupant state estimation unit 206 of the embodiment estimates the attributes and state of the occupant in addition to the vibration of the occupant relative to the vehicle 10. Furthermore, the signal generation unit 203 generates, as vehicle characteristic information, a signal for suppressing vibrations that the occupant feels uncomfortable, based on, for example, the attributes of the occupant and data on vibration characteristics of each body part.

[0097] FIG. 6 is a diagram showing some examples of the state of an occupant that can be estimated from an image captured by the inner camera 14 according to the embodiment.

[0098] As shown in Figure 6, heartbeats have low frequency (LF) and high frequency (HF), and it is known that when a person is comfortable, the ratio of these, LH / HF, decreases. Note that LH / HF can also be obtained from the pulse rate. In addition, when a person is tired or uncomfortable, breathing and blinking generally increase.

[0099] Next, learning section 208 will be described in detail. FIG. 7 is a block diagram illustrating an example of a functional configuration of the learning unit 208 according to the embodiment. As shown in FIG. 7, the learning unit 208 includes a large-scale language model (LLM) 2081, a determination unit 2080, a reward model 2084, a feedback unit 2085, and a loss calculation unit (loss) 2086.

[0100] The learning unit 208 of this embodiment is configured using a model of reinforcement learning using feedback from humans, that is, Reinforcement Learning Human Feedback (RLHF). Here, the feedback from humans is feedback based on the senses of the driver or passengers of the vehicle 10 or a third party.

[0101] In the learning unit 208 of this embodiment, in addition to such an RLHF model, biometric information of the driver, passengers, or third parties of the vehicle 10 is acquired by the above-mentioned acquisition unit 204, and feedback based on the acquired biometric information is reflected in the reinforcement learning.

[0102] The large-scale language model 2081 uses an existing language model. For example, an existing language model can be used that outputs information about road surface conditions when a captured image showing the front, which is the traveling direction of the vehicle 10, is input. This large-scale language model 2081 is stored in a storage unit such as the SSD 24, and is read by the CPU as part of the functional configuration of the learning unit 208. However, this is not limited to this, and for example, the large-scale language model 2081 may be provided outside the vehicle control device 20, and the learning unit 208 may input data to the large-scale language model 2081 using an API (Application Programming Interface) to obtain an output.

[0103] The determination unit 2080 determines, as the optimal vehicle characteristic information, the vehicle characteristic information that maximizes the reward obtained from a reward model 2084 described below. The determination unit 2080 includes an agent 2082 and an environment 2083.

[0104] The agent 2082 outputs an action regarding the vehicle characteristic information based on a predetermined policy for the output value of the large-scale language model 2081 to the environment 2083, the reward model 2084, and the feedback unit 2085. In addition, the agent 2082 determines the vehicle characteristic information that maximizes the reward based on the state calculated in the environment 2083 and the reward calculated in the reward model 2084.

[0105] The environment 2083 inputs actions from the agent 2082, calculates vehicle characteristic information by the various components of the vehicle characteristic calculation unit 210 described above, and returns the results to the agent 2082 as a state.

[0106] In the learning unit 208 of this embodiment, an actor-critic type reinforcement learning algorithm is used as the agent 2082. In such actor-critic type reinforcement learning, an environment 2083 and a reward model 2084 are set in which the agent 2082, who is a decision maker, makes decisions and learns. For this environment 2083, an actor (not shown), which is also called a behavioral device in the agent 2082, selects and executes an action based on a policy. In addition, a critic, which is also called an evaluator in the agent 2082, evaluates the action taken by the actor based on the state obtained from the environment 2083 and the reward obtained from the reward model 2084, and notifies the actor. The actor updates the policy based on the evaluation. By repeating these cycles, an optimal policy is acquired.

[0107] The feedback unit 2085 provides feedback based on the sense of the driver, passenger, or third party of the vehicle 10 for an action based on a predetermined policy for the output value of the large-scale language model 2081, which is the output of the agent 2082. This feedback is called human feedback. Specifically, the feedback unit 2085 ranks the action output from the agent 2082 based on the sense of the driver, passenger, or third party as human feedback. Here, the human feedback based on the sense of the driver, passenger, or third party is feedback of information regarding the ride comfort of the vehicle 10.

[0108] In addition to the human feedback, the feedback unit 2085 acquires bioinformation on the biological state of the driver or passenger or third party of the vehicle 10 by the acquisition unit 204, and provides feedback on the behavior based on the bioinformation. This feedback is called biofeedback. Specifically, the feedback unit 2085 ranks the behavior output from the agent 2082 based on the bioinformation of the driver or passenger or third party as biofeedback. Here, the biofeedback based on the bioinformation of the driver or passenger or third party of the vehicle 10 is feedback based on information on at least one of brain waves, heart rate, pulse, breathing, blinking, and sweating.

[0109] The feedback unit 2085 outputs the human feedback and the biological feedback to the loss calculation unit 2086. Note that the human feedback and the biological feedback may also be referred to as Fine-Tuning.

[0110] The loss calculation unit 2086 calculates a loss between the feedback from the feedback unit 2085 and the action from the agent 2082, and outputs the calculated loss to the reward model 2084. In this embodiment, the loss calculation unit 2086 calculates, as the loss, the difference between the action input from the reward model 2084 and the feedback value from the feedback unit 2085.

[0111] The reward model 2084 inputs an action from the agent 20 and calculates a reward for the action. Then, the reward model 2084 calculates and learns a reward for the action, reflecting feedback from a feedback unit 2085. Specifically, the reward model 2084 inputs a loss calculated by a loss calculation unit 2086, calculates a reward by reflecting the loss, and outputs the calculated reward to the agent 2082.

[0112] Next, the rewards that the agent 2082 can obtain will be described. Fig. 8 is a diagram showing an example of a reward that the agent 2082 according to the embodiment can obtain. In the example of Fig. 8, there are roll, roll + steering angle, pitch, heave, roll + pitch, and roll + heave + pitch as vehicle state quantities, and the magnitude of the reward changes depending on the magnitude of each vehicle state quantity.

[0113] For example, in Fig. 8, when the vehicle state variable is roll, if the amount of roll, speed, and acceleration are large, the reward is large, and if the amount of roll, speed, and acceleration are small, the reward is small. Also, the phase lag between roll and the total steering angle affects the ride comfort, and the example in Fig. 8 shows that if the phase lag is large, the reward is large, and if the phase lag is small, the reward is small.

[0114] Next, human feedback based on human senses among the fine-tuning performed by the feedback unit 2085 will be described. FIG. 9 is a diagram illustrating a specific example of human feedback according to the embodiment.

[0115] As shown in FIG. 9, human feedback includes feedback based on the senses of a human being such as a driver, a passenger, or a third party for each of roll, heave, roll+pitch, and roll+heave+pitch related to ride comfort.

[0116] In Fig. 9, for example, feedback of sensations such as large or small, fast or slow, and delay in steering can be performed with respect to the amount of roll, speed, and acceleration. In the case of heave, feedback of sensations such as lumpy, bumpy, trembling, etc. can be performed with respect to frequency. Note that these are only examples and are not limited to the example in Fig. 9.

[0117] In the case of human feedback, the feedback unit 2085 acquires information on the ride comfort of the vehicle 10 as shown in the example of FIG. 8 from the driver, the passenger, or a third party by inputting words or text. Specifically, the feedback unit 2085 can cause the input unit 32 of the monitoring device 30 to input words uttered by the driver, the passenger, or a third party by voice, and perform human feedback using information obtained by voice recognition of the words inputted by voice. In addition, the input unit 32 of the monitoring device 30 can input words uttered by the driver, the passenger, or a third party by voice, and the input unit 32 of the monitoring device 30 can cause the driver, the passenger, or a third party to input text, and perform human feedback using the text-input information.

[0118] In addition, in the case of human feedback, the feedback unit 2085 displays operation buttons, etc. that function as the input unit 32 on the display unit 31 of the monitor device 30, and allows the driver, passengers, or a third party to touch the operation buttons, etc. to obtain information regarding the ride comfort of the vehicle 10.

[0119] 10 and 11 are diagrams showing examples of screens when human feedback is performed by inputting operation buttons according to the embodiment.

[0120] Fig. 10 shows an example in which the index of ride comfort is indicated by the magnitude of the control current. In the example of Fig. 10, an Up button for increasing the magnitude of the index by increasing the control current and a Down button for decreasing the magnitude of the index by decreasing the control current are displayed as input units 32 on the display unit 31 of the monitor device 30. The driver, passenger, or a third party inputs human feedback information by touching the buttons.

[0121] Fig. 11 shows an example for giving an instruction to directly increase or decrease the magnitude of the index of ride comfort. In the example of Fig. 11, Up buttons for increasing and Down buttons for decreasing the indices of ride comfort, such as flatness, bumpiness, roughness, and vibration, are displayed as input section 32 on display section 31 of monitor device 30. The driver, passenger, or a third party inputs human feedback information by touching the buttons.

[0122] Next, biofeedback based on bioinformation, which is part of the Fine-Tuning performed by the feedback unit 2085, will be described. FIG. 12 is a diagram showing a specific example of biofeedback according to the embodiment.

[0123] As shown in Fig. 12, biofeedback provides quantitative feedback close to the sensations of the occupant, with feedback ranging from good to bad for each of the following: the alpha / beta wave ratio of the brainwaves, which indicates the degree of relaxation; the LF / HF ratio related to the heart rate or pulse rate; the respiratory rate, blink rate, and EDA related to sweating. Here, EDA is an indirect measurement value that indicates the influence of the nerve's mediation function on the activity of the sweat glands.

[0124] In Fig. 12, for example, it is possible to feed back that the larger the α wave / β wave ratio of the brain wave is, the better, and that the smaller the LF / HF ratio of the heart rate or pulse is, the better, and that the larger the LF / HF ratio is, the worse. Note that these are only examples, and are not limited to the example of Fig. 12.

[0125] Fig. 13 is a diagram showing an example of electroencephalograms according to an embodiment, in which the detected amounts of α waves, σ waves, θ waves, and β waves are shown as types of electroencephalograms, and the waveforms of each are shown. However, it is difficult to directly measure the brain waves of the driver, passengers, or third parties while the vehicle 10 is traveling. For this reason, in this embodiment, the relationship between the brain waves and the indices that can be measured based on the images captured by the inner camera 14 while the vehicle 10 is traveling is measured and obtained in advance through experiments, and the feedback unit 2085 measures the measurable biological information while the vehicle 10 is traveling and obtains the brain wave indices based on the relationship.

[0126] Fig. 14 is a graph showing an example of the relationship between the electroencephalogram and bio-information measurable during running according to the embodiment. In the graph of Fig. 14, the horizontal axis indicates body temperature, skeletal movement, pulse rate, etc. as bio-information measurable during running. The vertical axis indicates indices related to the electroencephalogram, specifically, theta wave dominance, alpha wave / beta wave ratio, etc.

[0127] While driving, the feedback unit 2085 acquires the body temperature, skeletal movement, pulse rate, etc. of the driver, passenger, or third party, and performs biofeedback by identifying theta wave dominance, alpha wave / beta wave ratio, etc. from the relationship shown in Figure 14.

[0128] In addition, a trained model may be generated by machine learning the relationship between brain waves as shown in FIG. 14 and bio-information measurable while driving, and the feedback unit 2085 may be configured to acquire and input measurable bio-information into the trained model while driving, output indicators related to brain waves, and perform bio-feedback while learning.

[0129] (Example of vehicle control processing) Next, a vehicle control process performed by the vehicle control device 20 according to this embodiment configured as above will be described. FIG. 15 is a flowchart illustrating an example of a procedure of the vehicle control process according to the embodiment.

[0130] First, the acquisition unit 204 acquires a surrounding image captured from the outer camera 16, which is an image of the surroundings of the vehicle 10, and also acquires an occupant image captured from the inner camera 14. Next, the learning unit 208 executes a learning process to obtain optimal vehicle characteristic information (signals) (S120).

[0131] Here, the learning process of S120 will be described. FIG. 16 is a flowchart illustrating an example of a procedure of the learning process according to the embodiment. FIG. 17 is a diagram illustrating a specific example of the flow of the learning process according to the embodiment. In the learning unit 208, the large scale language model (LLM) 2081 inputs the captured image and profile data acquired in S110, and outputs road surface conditions such as cobblestone roads, concrete joint roads, etc. (S201).

[0132] Next, the agent 2082 inputs the road surface conditions from the large-scale language model 2081, determines an action based on a policy, and outputs the determined action (S202). Here, the relationship between the control current and the index (parameter) of the ride comfort used in the strategy for determining the behavior will be described.

[0133] Fig. 18 is a graph showing an example of the relationship between the index (parameter) of ride comfort and the control current in the embodiment. In Fig. 18, the horizontal axis shows the parameter, and the vertical axis shows the target of the control current. As shown in Fig. 18, the target value of the control current is determined for each of the parameters of ride comfort, namely, flat, bumpy, and trembling.

[0134] Fig. 19 is a diagram showing an example of a reward model and a policy according to an embodiment. As shown in Fig. 19(b), in the reward model, ranks and probabilities are determined for each of the parameters of ride comfort, namely, flat, bumpy, and trembling.

[0135] Here, it is assumed that a stone-paved road is output as the road surface condition from the large-scale language model 2081. The agent 2082 outputs 0.52 A, which is the average of the control current shown in Fig. 19(a), as an action for the stone-paved road surface condition based on the policy. The average of the control current indicated by this action is calculated by the following formula (1) using Fig. 19(b) and Fig. 18.

[0136] Average current = (control current target of rank [1]) * (probability of rank [1]) + (control current target of rank [2]) * (probability of rank [2]) + (control current target of rank [3]) * (probability of rank [3]) + (control current target of rank [4]) * (probability of rank [4]) (1)

[0137] As shown in FIG. 17, the action output from the agent 2082 (0.52A in the example of FIG. 17) is input to the environment 2083, the reward model 2084, and the feedback section 2085.

[0138] Returning to FIG. 16, in the environment 2083, each part of the vehicle characteristic calculation unit 210 is made to execute the following processes. First, the vehicle state estimation unit 205 estimates vibrations applied to the vehicle 10 due to undulations and steps on the road surface based on the surrounding image from the outer camera 16. The occupant state estimation unit 206 estimates vibrations of each part of the occupant, such as the head, chest, abdomen, and legs, relative to the vehicle 10 based on the occupant image from the inner camera 14. The occupant state estimation unit 206 also estimates attributes of the occupant in the vehicle cabin, such as the gender, age, and physique. The occupant state estimation unit 206 also detects the occupant's heart rate, breathing, pulse, blinking, and the like, and estimates the occupant's state, such as fatigue level and discomfort level, based on these (S1204).

[0139] The vibration calculation unit 202 calculates the vibration actually applied to each body part of the occupant based on the vibration applied to the vehicle 10 estimated by the vehicle state estimation unit 205 and the vibration of the occupant relative to the vehicle 10 estimated by the occupant state estimation unit 206 (S205).

[0140] The signal generating unit 203 generates a signal, i.e., vehicle characteristic information, that controls at least one of the vibration control system 40, the lateral movement control system 50, and the steering system 60 so as to suppress vibrations that the occupant finds uncomfortable, while referring to the vibration characteristic data, based on the attributes and state of the occupant estimated by the occupant state estimating unit 206 and the vibrations applied to the occupant calculated by the vibration calculating unit 202 (S206).

[0141] Then, the environment 2083 outputs the generated vehicle characteristic information as a state to the agent 2082 (S207).

[0142] The reward model 2084 inputs the action from the agent 2082 and calculates the reward (S208). Here, in the example of the action 052A shown in Fig. 17, the reward model 2084 is as shown in Fig. 19(b).

[0143] The feedback unit 2085 inputs the behavior from the agent 2082, and performs human feedback and biofeedback by the above-mentioned method (S203). Here, as shown in the example of FIG. 17, it is assumed that the driver, passenger, or third party has given feedback as a specific behavior that the bouncing is bothersome and that the driver would like to suppress the bouncing. More specifically, it is assumed that the feedback has been given to reduce the input behavior of 0.52A to 0.46A. The feedback unit 2085 outputs the feedback value to the loss calculation unit 2086.

[0144] Next, the loss calculation unit 2086 calculates the difference between the action from the agent 2082 and the feedback value from the feedback unit 2085 as a loss (S209). In the example of Fig. 17, the difference between the action of 0.52A and the feedback value of 0.46A is the loss of 0.06A. The loss calculation unit 2086 outputs the calculated loss to the reward model 2084.

[0145] Next, the reward model 2084 inputs the loss from the loss calculation unit 2086 and corrects the reward based on the loss (S210). In the example of Fig. 17, the reward model 2084 reviews and corrects the rank and probability for the ride comfort parameter from the difference 0.06A, which is the loss.

[0146] FIG. 20 is a diagram showing an example of a reward model and a behavior decision policy based on the corrected rank and probability in the embodiment. FIG. 20(a) shows a reward model, and FIG. 20(b) shows a behavior decision policy. FIG. 20 is a modified version of the example in FIG. 19. In the example in FIG. 20, the probability is also modified along with the rank. Specifically, in the example in FIG. 19(b), the rank of the feeling of flatness was [1] and the probability was 0.7, but as shown in FIG. 20(a), the reward model 2084 corrects the rank of the feeling of flatness to [2] and the probability to 0.4. Also, in the example in FIG. 19(b), the rank of Hyokohyoko was [2] and the probability was 0.2, but as shown in FIG. 20(a), the reward model 2084 corrects the rank of Hyokohyoko to [1] and the probability to 0.5.

[0147] Furthermore, the reward model 2084 calculates the average current on the cobblestone road to be 0.46 A using the target value of the control current shown in Fig. 18 and the corrected probability in Fig. 20(a) according to formula (1) above, and corrects the policy as shown in Fig. 20(b). The reward model 2084 then returns the correction result to the agent 2082 as a reward (S211).

[0148] Next, the agent 2082 inputs the state output from the environment 2083 and the reward output from the reward model 2084, and determines optimal vehicle characteristic information based on these (S212). Then, the process returns to the caller.

[0149] Returning to FIG. 15, once the learning process of S120 has been performed as described above and the optimal vehicle characteristic information has been determined, vehicle control is performed by the vibration control system 40, the lateral sway control system 50 and the steering system 60 based on this optimal vehicle characteristic information (S130).

[0150] Fig. 21 is a graph showing the change in sprung acceleration of the suspension 41 in the embodiment with respect to frequency. In Fig. 21, the horizontal axis shows frequency, and the vertical axis shows sprung acceleration. Graph 2201 shows the output results in the conventional technology as a comparative example, and graph 2202 shows the output results in the vehicle control device 20 according to the present embodiment. As shown in Fig. 21, it can be seen that the output results of the present embodiment have a smaller amount of change in sprung acceleration and are superior in ride comfort.

[0151] (Overview) As described above, in this embodiment, the vehicle control device 20 includes a vehicle characteristic calculation unit 210 that determines vehicle characteristic information indicating the characteristics of the vehicle 10 based on an image captured around the vehicle 10 and a detection value of the state of the vehicle 10, a learning unit 208 that determines optimal vehicle characteristic information by reinforcement learning, and a control unit 207 that controls the vehicle 10 based on the optimal vehicle characteristic information. The learning unit 208 includes a feedback unit 2085 that provides feedback based on the senses of the driver or passenger of the vehicle 10 or a third party in response to an action based on a predetermined measure for the output value of a large-scale language model 2081, which is a trained model that inputs an image captured around the vehicle 10 and outputs an output value including the road surface condition, a reward model 2084 that reflects the feedback, calculates and learns a reward for the action, and a determination unit 2080 that determines the vehicle characteristic information that maximizes the reward as the optimal vehicle characteristic information.

[0152] According to this embodiment, a large-scale language model is used in the learning process for obtaining optimal vehicle characteristic information, and feedback is provided in the learning process based on the senses of the driver or passengers of the vehicle 10 or a third party. Therefore, according to this embodiment, a huge amount of data is not required, and the vehicle control state can be adapted to human senses without taking a lot of time for learning, and the optimal vehicle control state can be reliably achieved.

[0153] Moreover, the feedback unit 2085 of the vehicle control device 20 of this embodiment further acquires bio-information on the biological state of the driver, occupant, or third party of the vehicle 10, and performs feedback based on the bio-information. According to this embodiment, in the learning process, feedback is further performed using bio-information on the biological state of the driver, occupant, or third party of the vehicle 10. Therefore, even if the instructions in the human feedback of an occupant such as a small child or an elderly person are not appropriate, the vehicle control state can be adjusted to match human senses and the optimal vehicle control state can be more reliably achieved without requiring a large amount of time for learning.

[0154] In addition, the determination unit 2080 of the vehicle control device 20 of this embodiment includes an agent 2082 that outputs an action related to vehicle characteristic information based on a predetermined measure for an output value, and an environment 2083 that inputs an action, causes the vehicle characteristic calculation unit 210 to calculate vehicle characteristic information, and outputs the calculated vehicle characteristic information to the agent as a state, and the learning unit 208 further includes a loss calculation unit 2086 that calculates a loss between the feedback from the feedback unit 2085 and the action determined by the agent 2082, and the reward model 2084 calculates a reward by reflecting the loss calculated by the loss calculation unit 2086 and outputs the calculated reward to the agent 2082, and the agent 2082 determines optimal vehicle characteristic information based on the state calculated by the environment 2083 and the reward calculated by the reward model 2084. Therefore, according to this embodiment, the loss between the feedback from the feedback unit 2085 and the action determined by the agent 2082 is calculated, a reward is calculated by reflecting the calculated loss, and optimal vehicle characteristic information is determined based on the reward reflecting the loss and the state which is the calculated vehicle characteristic information, so that the vehicle control state can be adapted to human senses and biological information, and an optimal vehicle control state can be more reliably achieved.

[0155] Furthermore, the feedback unit 2085 of the vehicle control device 20 of this embodiment acquires information related to the ride comfort of the vehicle 10 from the driver, passenger, or third party by inputting words or text into the input unit 32. Therefore, according to this embodiment, more accurate human feedback is possible, and the vehicle control state can be adjusted to match human senses to improve the ride comfort, thereby more reliably achieving an optimal vehicle control state.

[0156] Furthermore, the feedback unit 2085 of the vehicle control device 20 of this embodiment acquires information related to the ride comfort of the vehicle 10 through an operation input to the input unit 32 by the driver, a passenger, or a third party. Therefore, according to this embodiment, human feedback is possible with a simpler operation, and the ride comfort is improved by adjusting the vehicle control state to human senses, so that an optimal vehicle control state can be more reliably achieved.

[0157] Furthermore, in the vehicle control device 20 of this embodiment, the biological information includes information on at least one of brain waves, heart rate, pulse rate, breathing, blinking, and sweating, and when the biological information uses brain waves, the feedback unit 2085 performs biological feedback based on brain waves based on a predetermined relationship between the biological information and brain waves that can be measured during measurement of the vehicle 10. Therefore, according to this embodiment, biological feedback can be reliably performed based on brain waves that cannot be directly measured, and the vehicle control state can be matched to the biological information to more reliably achieve an optimal vehicle control state.

[0158] In the above embodiment, the feedback unit 2085 performs both human feedback and biofeedback, but this is not limiting, and the feedback unit 2085 may be configured to perform only human feedback.

[0159] In the above embodiment, the vehicle characteristic information is a signal that controls at least one of the vibration control system 40, the lateral movement control system 50, and the steering system 60 to suppress vibrations that the occupants feel uncomfortable, but the present invention is not limited to this. Anything related to the vehicle characteristics can be used as the vehicle characteristic information.

[0160] In the above-described embodiment, the CPU 21 reads out and executes a program stored in a storage device such as the ROM 25 or the SSD 24, thereby implementing various functional modules such as the display control unit 201, the acquisition unit 204, the vehicle characteristic calculation unit 210 (vibration calculation unit 202, signal generation unit 203, vehicle state estimation unit 205, and occupant state estimation unit 206), the learning unit 208, and the control unit 207. However, this is not limited to this. For example, the various functional modules such as the display control unit 201, the acquisition unit 204, the vehicle characteristic calculation unit 210 (vibration calculation unit 202, signal generation unit 203, vehicle state estimation unit 205, and occupant state estimation unit 206), the learning unit 208, and the control unit 207 can also be implemented by independent hardware such as a circuit including an ASIC (Application Specific Integrated Circuit).

[0161] The vehicle control program executed by the vehicle control device 20 according to the above embodiment is provided in a state that it is pre-installed in a ROM or the like.

[0162] The vehicle control program executed by the vehicle control device 20 in the above embodiment may be configured to be provided by recording it on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a DVD (Digital Versatile Disk) in an installable or executable format file.

[0163] Furthermore, the vehicle control program executed by the vehicle control device 20 according to the above embodiment may be stored in a computer connected to a network such as the Internet and provided by being downloaded via the network. Also, the vehicle control program executed by the vehicle control device 20 according to the above embodiment may be provided or distributed via a network such as the Internet.

[0164] The vehicle control program executed by the vehicle control device 20 in the above-mentioned embodiment has a modular configuration including each of the functional units mentioned above (i.e., the display control unit 201, the acquisition unit 204, the vehicle characteristic calculation unit 210 (vibration calculation unit 202, signal generation unit 203, vehicle state estimation unit 205, occupant state estimation unit 206), the learning unit 208, the control unit 207, etc.), and in terms of actual hardware, the CPU reads out the vehicle control program from the above-mentioned ROM and executes it, thereby loading each of the above-mentioned units onto the main memory device, and the display control unit 201, the acquisition unit 204, the vehicle characteristic calculation unit 210 (vibration calculation unit 202, signal generation unit 203, vehicle state estimation unit 205, occupant state estimation unit 206), the learning unit 208, the control unit 207, etc. are generated on the main memory device.

[0165] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0166] 10...vehicle, 14...inner camera, 16...outer camera, 20...vehicle control device, 30...monitor device, 31...display unit, 32...input unit, 40...vibration control system, 50...rolling control system, 60...steering system, 201...display control unit, 202...vibration calculation unit, 203...signal generation unit, 204...acquisition unit, 205...vehicle state estimation unit, 206...occupant state estimation unit, 207...control unit, 208...learning unit, 210...vehicle characteristic calculation unit, 2080...decision unit, 2081...large-scale language model, 2082...agent, 2083...environment, 2084...reward model, 2085...feedback unit, 2086...loss calculation unit.

Claims

1. a vehicle characteristic calculation unit that calculates vehicle characteristic information indicating characteristics of the vehicle based on an image of the surroundings of the vehicle and a detection value of a state of the vehicle; a learning unit that obtains optimal vehicle characteristic information by reinforcement learning; a control unit that controls the vehicle based on the optimal vehicle characteristic information, The learning unit is a feedback unit that provides feedback based on a sense of a driver, a passenger, or a third party of the vehicle with respect to an action based on a predetermined measure for an output value of a large-scale language model that is a trained model that inputs the captured image and outputs an output value including a road surface condition; A reward model that calculates and learns a reward for the action by reflecting the feedback; a determination unit that determines vehicle characteristic information that maximizes the reward as the optimal vehicle characteristic information; A vehicle control device comprising:

2. The feedback unit further acquires biological information regarding a biological state of a driver, a passenger, or a third party of the vehicle, and performs the feedback based on the biological information. The vehicle control device according to claim 1.

3. The determination unit is an agent that outputs an action related to the vehicle characteristic information based on a predetermined measure for the output value; The action is input, the vehicle characteristic calculation unit is caused to calculate the vehicle characteristic information, and the calculated and an environment for outputting vehicle characteristic information as a state to the agent, The learning unit is a loss calculation unit that calculates a loss between the feedback from the feedback unit and an action determined by the agent, the reward model calculates the reward by reflecting the loss calculated by the loss calculation unit, and outputs the calculated reward to the agent; The agent determines the optimal vehicle characteristic information based on the state calculated in the environment and the reward calculated by the reward model. The vehicle control device according to claim 2.

4. The feedback unit acquires information regarding the ride comfort of the vehicle from the driver, the passenger, or the third party by inputting words or text. The vehicle control device according to claim 3.

5. The feedback unit acquires information regarding the ride comfort of the vehicle through an operation input to a predetermined input device by the driver, the passenger, or the third party. The vehicle control device according to claim 3.

6. The biological information includes at least one of information regarding brain waves, heart rate, pulse rate, breathing, blinking, and sweating, When the biological information is an electroencephalogram, the feedback unit performs the feedback based on the electroencephalogram based on a predetermined relationship between the biological information measurable during measurement of the vehicle and the electroencephalogram. The vehicle control device according to claim 3.

Images