Driving reproduction system and driving reproduction program

Abstract

To provide a driving simulation system that can accurately reproduce driving conditions. [Solution] The driving reproduction device 10, as a driving reproduction system, is equipped with at least one processing unit, a CPU 10a, and reproduces the driver's driving in the vehicle 1. The CPU 10a performs the following: acquires driving characteristic data of driving scenes in which predetermined characteristics appear in the driver's driving, and reproduces the driver's driving by autonomous driving control of the vehicle, in a manner in which adjustments have been made based on the driving characteristic data so that the predetermined characteristics can be properly perceived by the driver who performed the driving.

Description

Technical Field

[0001] The disclosure according to this specification relates to a technology for experiencing the characteristics of a driver's driving.

Background Art

[0002] Patent Document 1 discloses a technology for detecting a driver's dangerous driving and reproducing the dangerous driving by autonomous driving control of the vehicle itself. This enhances the driver's understanding of dangerous driving and improves the driver's awareness of safe driving.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, when the dangerous driving is faithfully reproduced as it is on a road such as a public road, even if the driving is reproduced in a situation where it is determined that there is no possibility of an accident, the reproduced driving itself is determined to be dangerous. Therefore, there remains a possibility that the reproduction action may trigger an unexpected situation not only for oneself but also for others on the road. In addition, faithful reproduction of driving may not necessarily enable the driver to effectively experience the characteristics. Therefore, there is a need to reproduce driving more appropriately.

[0005] One of the purposes of the disclosure of this specification is to provide a driving reproduction system and a driving reproduction program that can reproduce driving more appropriately.

Means for Solving the Problems

[0006] The driving reproduction system disclosed herein is a driving reproduction system comprising at least one processing unit (10a) that reproduces the driving of a driver in a vehicle (1), At least one processing unit, To acquire driving characteristic data of driving scenes in which predetermined characteristics appear in the driver's driving, Based on driving characteristic data, the system reproduces the driver's driving through autonomous vehicle driving control, with adjustments made to ensure that the driver performing the driving can appropriately experience predetermined characteristics.

[0007] Furthermore, the disclosed driving reproduction program is a driving reproduction program that reproduces the driver's driving in vehicle (1), At least one processing unit (10a) To acquire driving characteristic data of driving scenes in which predetermined characteristics appear in the driver's driving, Based on driving characteristic data, the system reproduces the driver's driving through the vehicle's autonomous driving control in a manner that allows the driver to properly experience predetermined characteristics.

[0008] According to these methods, for the driving of the driver to be reproduced, predetermined characteristics to focus on are set in advance, and driving characteristic data is acquired for driving scenes in which those characteristics appear. Then, when the driver's driving is reproduced by the vehicle's autonomous driving control, adjustments are made to ensure that the driver can properly experience the predetermined characteristics. Such adjustments make it possible to avoid the disadvantages of fidelity to the reproduction and to prioritize the effective experience of those characteristics. Therefore, it becomes possible to reproduce driving appropriately.

[0009] The symbols in parentheses included in the claims, etc., are illustrative examples illustrating the correspondence with the embodiments described later, and are not intended to limit the technical scope. [Brief explanation of the drawing]

[0010] [Figure 1] This is a schematic diagram showing the overall configuration of the vehicle. [Figure 2] This is a configuration diagram showing the functional configuration of the driver assistance system and the driver simulation system. [Figure 3] This figure shows an example of the displayed content. [Figure 4] This figure shows an example of the displayed content. [Figure 5] This flowchart shows an example of the mode selection process. [Figure 6] This flowchart shows an example of the processing in diagnostic mode. [Figure 7] This flowchart shows an example of the processing in the driving simulation mode. [Modes for carrying out the invention]

[0011] In this disclosure or claims, the term "processor" means one or more hardware processors configured to execute processing defined by computer program code (i.e., one or more instructions of a computer program) contained in a computer program by reading the code each time. In other words, a "processor" is a hardware device that executes one or more programmed processes. Therefore, computer program code can also be considered software that can define the processing of the processor according to its content. For example, a "processor" may be a general-purpose or specific-purpose processor and may be, but is not limited to, a CPU, microprocessor, GPU, and DFP (Data Flow Processor).

[0012] In this disclosure or claims, the term “memory” means one or more hardware memories that are non-transitional tangible recording media configured to record computer program code and / or data in a manner accessible from a processor. “Memory” can be implemented by memory technology such as SRAM, SDRAM, non-volatile flash memory, or other types of memory. The computer program code that constitutes the program is recorded in memory and executed by a processor, thereby enabling the processor to perform the various functions described above.

[0013] In this disclosure or claims, the term “circuit” refers to one or more logic circuits as hardware, configured to perform specific processing defined based on a pre-designed circuit configuration. In other words (and, in contrast to “processor”), “circuit” in this disclosure or claims refers to a hardware device that performs specific processing based on a circuit configuration, rather than processing defined by software such as the computer program code described above. For example, “circuit” may include custom ICs such as ASICs (Application Specific Integrated Circuits) and FPGAs (Field Programmable Gate Arrays) designed with a Hardware Description Language (HDL). That is, “circuit” in this disclosure or claims includes all hardware circuits except for the processor described above, which performs processing by reading computer program code.

[0014] In this disclosure or in the claims, the expression “at least one circuit and processor” should be interpreted as a disjunctive (logical OR) and not as at least one circuit and at least one processor.

[0015] In the present disclosure or the claims, the term "processing unit" means a hardware device that executes processing by a "processor", "circuit", or a combination thereof. When its function is interpreted as being impossible to be realized by a "circuit" and possible to be realized by a "processor", the "processing unit" may mean the "processor" itself.

[0016] An embodiment will be described based on the drawings.

[0017] (First Embodiment) The driving reproduction application disclosed in the first embodiment is executed to let the driver experience the characteristics of his / her own driving using the vehicle 1 actually driven by the driver. The driving characteristics here may be preset characteristics, for example, characteristics such that a passenger feels discomfort with the driver's driving. That is, a scene where a passenger feels discomfort with the driver's driving is reproduced by autonomous driving control in a state where the driver is on board the vehicle 1, for example, in the passenger seat or the rear seat. Thereby, the driver can objectively recognize his / her own driving from the perspective of a passenger.

[0018] As shown in FIG. 1, the vehicle 1 used for the driver's driving and the reproduction of the driving includes a vehicle state sensor 31, an external environment sensor 32, a map database (hereinafter, map DB) 33, a driving control device 34, a display device 35, a speaker 36, a vibration device 37, an application operation switch (hereinafter, application operation SW) 38, a driving support device 20, a driving reproduction device 10, and the like. These are connected to be communicable with each other through an in-vehicle network system such as CAN (registered trademark).

[0019] Vehicle 1 of this embodiment can switch between Level 2 and Level 0 driving, as defined in SAE J3016. Here, Level 0 is no driving automation, and the driver performs all dynamic driving tasks. Levels 1 and 2 are categorized as driving assistance. In particular, Level 2 is partial driving automation, in which the system continuously performs longitudinal and lateral control of dynamic driving tasks within a specific Operational Design Domain. In this case, the driver is required to perform subtasks, monitor the system, etc. Levels 3 to 5 are categorized as fully automated driving. Here, "autonomous driving control" in this embodiment refers to the system's control of longitudinal and lateral dynamic driving tasks at Level 2 and above.

[0020] The vehicle state sensor 31 detects the state of the vehicle 1. This state may be the internal state of the vehicle 1. Typically, multiple types of vehicle state sensors 31 are installed. The vehicle state sensor 31 includes at least one, more preferably, of the following: acceleration sensors, jerk sensors, vehicle speed sensors, yaw rate sensors, driver state sensors, and steering wheel grip force sensors. The detected vehicle state sensor data is provided to the driving control device 34, the driving assistance device 20, and the driving reproduction device 10, etc.

[0021] The external environment sensor 32 detects the external environment of the vehicle 1. The external environment sensor 32 includes, for example, a camera, LiDAR (Light Detection and Ranging / Laser imaging Detection and Ranging), laser radar, millimeter-wave radar, ultrasonic sonar, acoustic sensor, etc. The external environment sensor data, which is the detection result, is provided to the driver assistance system 20, etc.

[0022] Map DB33 is a database that stores map data available for use in vehicle 1. Map DB33 is composed of at least one type of non-transitory tangible storage medium, such as semiconductor memory, magnetic media, and optical media. Map DB33 may store map data used in a navigation system that navigates the vehicle 1 to its destination. Map DB33 may store PD map data generated using probe data (PD) collected from each vehicle traveling on public roads by downloading it. Map DB44 may store high-precision map data with a high level of accuracy used for autonomous driving control of vehicle 1.

[0023] The driving control device 34 is an electronic control unit (ECC) primarily composed of a computer. The driving control device 34 can control the movement of the vehicle 1 via a driving actuator based on input signals received from an operating device or driving assistance device operated by the driver, such as the driving assistance device 20 or the driving reproduction device 10. The operating device is, for example, an accelerator pedal, a brake pedal, or a steering wheel. The drive-type motion actuator is a powertrain including at least one of the following: an internal combustion engine, a drive motor, etc. The braking-type motion actuator is, for example, a brake actuator. The steering-type motion actuator is, for example, a steering wheel. The driving control device 34 converts the target speed, target acceleration, target trajectory, or control amount of the driving actuator in the input signal into a control signal for the driving actuator and outputs it to the driving actuator.

[0024] The display device 35 is mounted inside the vehicle 1 and can display information to the driver or passengers through image display or analog display such as indicator lights. Examples of the display device 35 include a meter display, navigation system, CID (center information display), HUD (head-up display), etc.

[0025] The speaker 36 is configured to output sound into the vehicle cabin in conjunction with the display device 35, or independently of the display on the display device 35. The speaker 36 generates guidance voices, notification sounds, warning sounds, etc., according to control signals input from the driver assistance device 20 or the driving simulation device 10, etc. The vibration device 37 vibrates the steering wheel by motor drive in conjunction with the display device 35, or independently of the display on the display device 35.

[0026] The application control SW38 is a switch located, for example, on the spokes of the steering wheel or on the instrument panel in vehicle 1. The application control SW38 may be a push-button switch or another type of switch such as a wheel switch. The application control SW38 is configured to switch between the on and off states of ADAS applications, driving simulation applications, etc., as described later. Switching between the on and off states of ADAS applications is essentially equivalent to switching the level of automation.

[0027] The driver assistance system 20 is an electronic control unit primarily composed of a computer. The computer comprising the driver assistance system 20 has at least one CPU (Central Processing Unit) 20a and at least one memory 20b. The memory 20b is a non-transitional, tangible storage medium that non-temporarily stores computer programs and data that can be read by the CPU 20a. Furthermore, the computer may be provided with a rewritable, volatile storage medium such as RAM (Random Access Memory) 20c. The RAM 20c is used for temporary storage of data being processed. The CPU 20a can execute various processes according to the computer programs stored in the memory 20b. Furthermore, the computer includes an interface 20d for communicating data with the outside world.

[0028] The driver assistance device 20 may include a driver assistance unit X10 as a functional unit whose functions are realized by the CPU 20a executing a computer program stored in memory 20b. The driver assistance unit X10 executes various ADAS (Advanced Driver Assistance Systems) applications as a computer program implemented in the driver assistance device 20. Here, ADAS applications include, for example, ACC (Adaptive Cruise Control), PCS (Pre-Crash Safety system), LTA (Lane Tracing Assist), BSM (Blind Spot Monitor), etc. Through ADAS applications, the driver assistance unit X10 can assist the driver's driving as Level 2 automated driver assistance. As shown in Figure 2, the driver assistance unit X10 may include a target recognition unit X11, a scene recognition unit X12, and an assist function unit X13 in order to perform driver assistance.

[0029] The target identification unit X11 identifies targets present around vehicle 1 based on external environmental sensor data. Targets include other vehicles, pedestrians, animals, roads, buildings, road signs, etc. Target identification here may include at least one of the following: target type, target size, target position, and target speed.

[0030] The scene determination unit X12 determines the scene currently being encountered by vehicle 1 based on external environmental sensor data, target information determined by target determination unit X11, and map data from map DB33, etc. Scenes include, for example, a scene of turning a curve, a scene of turning right at an intersection, or a scene of passing through a highway exit. Here, a scene may also be called a scenario. Based on the determined scene, the operating state of the ADAS application can be determined.

[0031] The assist function unit X13 assists the driver's operation based on vehicle condition sensor data, external environment sensor data, target information determined by the target recognition unit X11, and scene information determined by the scene recognition unit X12.

[0032] The assist function unit X13 calculates the amount of assistance for the driver's manual operation and may cause the driving control device 34 to control the driving actuator based on this amount of assistance. The amount of assistance refers to the control amount that corrects any deficiency or excess in the amount of operation performed by the driver during manual operation compared to the ideal amount of operation, through assistance (support) from the vehicle 1. In other words, the assist function unit X13 calculates the amount of assistance by calculating the ideal amount of operation and taking the difference between that and the actual amount of operation performed by the driver during manual operation. The calculated amount of assistance is output to the driving actuator, thereby reducing the amount of manual operation performed by the driver or providing support for driver errors.

[0033] The assist amount may include at least one of the following: drive assist amount, brake assist amount, and steering assist amount. The ideal control amount should be calculated based on the ideal behavior of vehicle 1. The ideal behavior can also be described as the target behavior pre-set in the ADAS application. The target behavior may include the time-series target position (also known as the target trajectory) of vehicle 1, and may also include the target speed and target acceleration.

[0034] For example, in ACC, the steering assist amount is calculated and reflected in the control system in order to drive vehicle 1 along the lane markings (e.g., white lines) of the vehicle's own lane as determined by the target recognition unit X11. In addition, in ACC, the drive assist amount and braking assist amount are calculated and reflected in the control system in order to maintain an appropriate distance from other vehicles ahead as determined by the target recognition unit X11.

[0035] The assist function unit X13 may perform driving assistance using the display device 35, along with or instead of calculating the amount of assistance for controlling the driving actuator. For example, in BSM, if the target recognition unit X11 has identified another vehicle that is located diagonally behind vehicle 1 and is difficult to see using the door mirrors of vehicle 1, the presence of the other vehicle is notified to the driver using the display device 35 and speaker.

[0036] Furthermore, the driver assistance unit X10 of this embodiment continues calculations regardless of whether the ADAS application is on or off. When the ADAS application is off, these calculations are performed in shadow mode, which is a mode that performs undercurrent processing in which, for example, the calculation results are not reflected in the actual control of vehicle 1. For example, the assist function unit X13 continues to perform the calculations for the ideal behavior described above, regardless of whether the ADAS application is on or off.

[0037] The driving reproduction device 10 is an electronic control device mainly composed of a computer. The computer comprising the driving reproduction device 10 has at least one CPU 10a and one memory 10b. The memory 10b is a non-transitional tangible storage medium that non-temporarily stores computer programs and data that can be read by the CPU 10a. Furthermore, the computer may be provided with a rewritable volatile storage medium such as RAM 10c. RAM 10c is used for temporary storage of data being processed. The CPU 10a can execute various processes according to the computer programs stored in memory 10b. Furthermore, the computer includes an interface 10d for communicating data with the outside.

[0038] The driving reproduction device 10 is a device that reproduces the driver's driving. In this embodiment, the driving reproduction device 10 constitutes the driving reproduction system. The driving reproduction device 10 may include a driving characteristic data acquisition unit F10 and a driving reproduction unit F20 as functional units whose functions are realized when the CPU 10a executes a driving reproduction application implemented in the form of a computer program (e.g., a driving reproduction program) stored in memory 10b. The processing by the driving characteristic data acquisition unit F10 is performed when autonomous driving control is not functioning and the driver is driving manually, and the mode in which this processing is performed corresponds to the diagnostic mode described later. On the other hand, the processing by the driving reproduction unit F20 is performed while autonomous driving control is functioning, and the mode in which this processing is performed corresponds to the driving reproduction mode described later. This mode may be switchable by operation of the application operation SW37. The mode switching may be linked to the switching of the automation level.

[0039] The driving characteristic data acquisition unit F10 acquires driving characteristic data from the driver assistance device 20 or the driving control device 34. Along with acquiring the driving characteristic data, the driving characteristic data acquisition unit F10 performs a diagnosis to make the driving reproducible. As shown in Figure 2, the driving characteristic data acquisition unit F10 includes a discomfort calculation unit F11, a scene selection unit F12, and a data recording unit F13.

[0040] The discomfort calculation unit F11 calculates the estimated level of discomfort that a passenger would feel in response to the driver's driving. Specifically, the discomfort calculation unit F11 acquires vehicle state sensor data and the ideal behavior calculated by the driver assistance device 20, and compares these. The discomfort calculation unit F11 calculates the deviation of the actual behavior based on the vehicle state sensor data from the ideal behavior. In this embodiment, this deviation is considered the level of discomfort. Here, the deviation may be expressed as a G value with standard gravitational acceleration set to 1G. For example, the G value may be calculated separately for the longitudinal and lateral directions of the vehicle 1.

[0041] The scene selection unit F12 selects scenes to be reproduced as scenes that would cause discomfort to passengers. For example, as a selection method, the scene selection unit F12 counts the reproduction target count value based on the discomfort level calculated by the discomfort level calculation unit F11. A reproduction target count value is set for each scene, and the values ​​for each scene are accumulated to count them using an accumulation method.

[0042] A scene for which a reproducible count value is set (hereinafter referred to as a reproducible scene) is a scene in which the driving simulation application is implemented as a scene in which driving can be simulated. For example, in this embodiment, scenes for which a reproducible count value is set include "curves," "left turns at intersections," and "highway exits."

[0043] Here, we will explain in detail how the count of the reproducible count value is calculated. First, the scene selection unit F12 acquires the scene information determined by the scene discrimination unit X12 in the driving support device 20, and determines whether the scene for which discomfort was calculated corresponds to a reproducible scene, and which reproducible scene it is classified into.

[0044] Next, the scene selection unit F12 converts the level of discomfort into a reproducible count value corresponding to a classified reproducible scene. For example, the conversion converts the numerically calculated level of discomfort into three levels of discomfort: "high," "medium," and "low." If the deviation amount is used as the level of discomfort, for example, if the lateral deviation amount is 0.5G or more, the level is set to "high," if it is 0.3G or more but less than 0.5G, the level is set to "medium," and if it is less than 0.3G, the level is set to "low." In this embodiment, the calculation of this level of discomfort corresponds to diagnosing the driver's driving in order to reproduce the driving.

[0045] If the value is "small," it is determined that the passenger will not experience discomfort, and no count is performed. If the value is "large" or "medium," the scene selection unit F12 performs the count by referring to a pre-set addition table for each scene to be reproduced. For example, in the case of "curve," a value of 3 is added to the reproduced count value of "curve" if it is "large," and a value of 1 is added if it is "medium." In the case of "left turn at intersection," a value of 5 is added to the reproduced count value of "left turn at intersection" if it is "large," and a value of 4 is added if it is "medium." In the case of "highway exit," a value of 2 is added to the reproduced count value of "highway exit" if it is "large," and a value of 1 is added if it is "medium." This is because the conditions under which a passenger experiences discomfort and the degree of discomfort differ depending on the scene.

[0046] At this time, the counted value and the date and time may be stored together in memory 10b. The scene selection unit F12 may then refer to the date and time of each stored past count and exclude counts that have exceeded a predetermined set period. For example, if the set period is one month, the scene selection unit F12 may subtract the values ​​of counts that have exceeded one month but have not yet been excluded for each reproducible count value.

[0047] Subsequently, the scene selection unit F12 determines whether the cumulative count value of the reproducible target in each scene has reached a predetermined amount. The predetermined amount may be, for example, 5. The scene selection unit F12 changes the reproducible target scene that has reached the predetermined amount from the reproducible preparation state to the reproducible state. The scene selection unit F12 may also use the display device 35 to notify the driver that the reproducible target scene that has reached the predetermined amount is now reproducible.

[0048] The data recording unit F13 records driving characteristic data for scenes judged by the scene selection unit F12 to have a "high" or "medium" discomfort level into memory 10b. Here, driving characteristic data is data that shows the characteristics of the driver's driving in the scene to be reproduced. The driving characteristic data is configured so that the driver's driving and / or driving characteristics can be reproduced retrospectively. Typically, the driving characteristic data consists of multiple sets of data.

[0049] The driving characteristic data of this embodiment consists of a set of multiple data, including data indicating the classification of the scene, data indicating the location where the scene occurred, time-series data of vehicle condition sensor data, and data indicating the degree of discomfort or level of discomfort of the scene. The data recording unit F13 may also appropriately delete driving characteristic data corresponding to counts that have been excluded due to the elapsed of the set period, as described above. By doing so, it becomes possible to collect driving characteristic data within the set period necessary for reproduction and to easily secure memory capacity for recording more necessary driving characteristic data.

[0050] The driving reproduction unit F20 reproduces the driver's driving in the target scene based on the driving characteristic data of the target scene that has become reproducible, by autonomous driving control of vehicle 1. The driving reproduction unit F20 includes a control variable calculation unit F21 and an adjustment unit F22.

[0051] In this embodiment, the driving reproduction by the driving reproduction unit F20 only needs to reproduce the driving in a manner that allows the driver to appropriately experience the characteristics of their own driving (e.g., discomfort), and it is not necessary for the driver's driving to be faithfully reproduced. In other words, the reproduction does not need to be in the same place where the driver actually drove, nor is it required that the same road surface conditions or the same positions of other vehicles or pedestrians be present. The reproduction can be carried out in an environment different from the environment in which the driver actually drove, as long as it can reproduce the same target scene in terms of classification and the characteristics can be experienced by the driver during the reproduction, and the behavior of vehicle 1 in the reproduction may differ from the behavior of the driver's actual driving. In order to realize such a driving reproduction method, an adjustment unit F22 is provided in this embodiment.

[0052] The control quantity calculation unit F21 calculates control quantities to reproduce the driver's driving through autonomous driving control of vehicle 1, based on driving characteristic data. Specifically, the control quantity calculation unit F21 uses time-series data of vehicle state sensor data to estimate and reconstruct the behavior of vehicle 1 based on the driver's driving in the relevant scene and location. If the reproduction of the driver's driving is performed by an ADAS application at automation level 2, these control quantities may be referred to as the assist quantities described above.

[0053] The control variable calculation unit F21 may use the maximum value of the degree of the driving characteristics that were counted as the feature to be reproduced. That is, the driving characteristic data corresponding to the driving that is estimated to have been the most unpleasant for the passenger among the driving characteristic data that were counted may be used as the feature to be reproduced. Alternatively, the control variable calculation unit F21 may use the average value of the degree of the driving characteristics that were counted as the feature to be reproduced. That is, the averaged parameters of the driving characteristic data that were counted may be used as the feature to be reproduced.

[0054] The adjustment unit F22 adjusts the reproduction mode in the reproduction of driving so that the characteristics can be appropriately perceived by the driver who performed the driving. This adjustment may be referred to as modification, deformation, or alteration (partial change). Specifically, the adjustment of the reproduction mode includes at least one of the following: exaggeration or simplification of characteristics, limitation of kinetic physical quantities during reproduction, and realization through cooperation with HMI. Here, HMI is an abbreviation for Human Machine Interface.

[0055] The exaggeration or minimization of characteristics may involve increasing or decreasing the level of discomfort caused by the reproduced driving compared to the actual level of discomfort caused by the driver's driving. For example, if a driver causes discomfort to a passenger by swerving with a left-to-right amplitude W, the level of discomfort can be increased or decreased by exaggerating the amplitude to 1.2 × W or decreasing it to 0.8 × W. Such exaggeration or minimization makes it possible to appropriately convey the characteristics to the driver depending on the situation.

[0056] The limitations on kinetic quantities during simulation may be limitations on the range of kinetic quantities that are permissible for vehicle 1 during driving simulation. For example, kinetic quantities may include velocity, acceleration, jerk, yaw rate, etc. For example, if the driver's actual driving is a concern for a certain reason, the limitations on kinetic quantities prevent the exact simulation of that driving from being reproduced.

[0057] The specified reasons here may be at least one of the following: compliance with traffic rules, consideration of unforeseen circumstances, hardware performance in vehicle 1, and software performance in vehicle 1. Safety may also be determined by acquiring scene information and target information from the driver assistance system 20 and calculating the risk of collision with other vehicles or pedestrians, etc.

[0058] The permissible range here may be set by referring to the permissible range in the ADAS application. For example, if the speed limit in ACC is 135 km / h, then the speed limit in driving simulation will also be set to 135 km / h. That is, even if the maximum speed in the driver's actual driving is 140 km / h, adjustments will be made so that the maximum speed in driving simulation is 135 km / h. Note that when the kinetic physical quantity is a scalar quantity, the permissible range may be defined by both an upper and lower limit, or it may be defined by either an upper or lower limit.

[0059] The realization through integration with the HMI may involve adjustments that allow the driver to more appropriately experience the characteristics of the vehicle by coordinating the autonomous driving control of vehicle 1 with the HMI during driving simulation. In other words, the adjustment unit F22 generates display content that is linked to the autonomous driving control and displays it on the display device 35.

[0060] The realization of the driving experience through integration with the HMI may also involve the HMI providing information about the differences between the reproduced driving and driving that is less likely to cause discomfort to passengers (hereinafter referred to as "comfortable driving"). As an example of realization through integration with the HMI, Figure 3 shows the display on screen 35a of the display device 35. The display content DC1 included in this display shows, in a comparable manner, the trajectory T1 of vehicle 1 in the currently reproduced driving (solid line in Figure 3) and the trajectory T2 in comfortable driving (dashed line in Figure 3). The curvature of trajectory T1 is greater than the curvature of trajectory T2. By visualizing the difference between trajectory T1 and trajectory T2 along with the reproduction of their driving, the driver can become aware that their driving is causing discomfort to passengers due to sudden steering.

[0061] Alternatively, the realization through integration with the display may be a way to compensate for differences in the display when discrepancies arise between the driver's actual driving and the displayed driving due to exaggeration or minimization of features, or limitations on kinetic physical quantities. Figure 4 shows an example of the display on screen 35a of the display device 35 in the "curve" scene. Here, it is assumed that the driver's actual driving generates a G value of approximately 0.5G due to sudden steering, while the driving reproduced by the limitations on kinetic physical quantities during reproduction is limited to a G value of approximately 0.3G. Furthermore, this reproduction is achieved by the output of the assist amount in the driving assistance at automation level 2.

[0062] The display content DC2 included in Figure 4 shows the difference between the driving reproduced in the circular chart image and the driver's actual driving. Specifically, in the circular chart, point P1, which represents the G-value generated in the reproduced driving, and point P2, which represents the G-value generated in the driver's actual driving, are plotted. The distance from the center of the circular chart to points P1 and P2 represents the absolute value of the G-value. The direction from the center of the circular chart to points P1 and P2 represents the direction in which the G-value is generated. By visualizing the difference between point P1 and point P2 along with the reproduction of their driving, the driver can become aware that their driving is even more unpleasant than the reproduced driving.

[0063] The display content DC3 included in the display in Figure 4 is display content that instructs manual additional input to make the driving, which is reproduced with limitations on the amount of assistance, closer to the driver's actual driving. Specifically, the display content DC3 includes an operating device image IM1 and an instruction image IM2 that instructs manual additional input to the operating device. The operating device here may be a steering wheel. The instruction image IM2 may be, for example, an arrow. The direction of the arrow may indicate the direction of operation on the operating device, and the length of the arrow may indicate the amount of operation.

[0064] This display content DC3 is designed to simulate a scenario where, for example, the driver who performed the simulated driving maneuver is seated in the passenger or rear seat, while a simulation assistant is seated in the driver's seat. By having the simulation assistant perform additional inputs according to the instruction image IM2, the simulated driving maneuver can be reproduced more faithfully. This allows the driver who performed the simulated driving maneuver to experience the discomfort from the perspective of a passenger.

[0065] If an additional input instructed in display content DC3 is performed by the reproduction assistant, point P1 in display content DC2 may be configured to move in accordance with the change in G-value caused by that input. That is, point P1 may indicate the G-value generated by the sum of the amount of assistance in the reproduced driving and the amount of control due to the reproduction assistant's additional input. By adjusting the degree of the additional input while checking the position of the moving point P1, the reproduction assistant can reproduce a situation that is closer to the driver's actual driving.

[0066] Furthermore, the realization of the experience through collaboration with the HMI may be achieved by linking at least one of the following with the display: sound from the speaker 36 and vibration from the vibration device 37. For example, along with the display of the display content DC3, the adjustment unit F22 may use the speaker 36 to give instructions for additional voice input. Also, for example, if the driving being reproduced approaches the target control value or target behavior of the driving being reproduced based on the additional input from the reproduction assistant as described above, the adjustment unit F22 may use the vibration device 37 to vibrate the steering wheel. The determination of whether or not the driving is approaching the target control value or target behavior may be, for example, determined by whether or not the distance between point P1 and point P2 is below a preset threshold.

[0067] Next, an example of how the driving simulation device 10 can reproduce the driver's operation will be explained using the flowcharts in Figures 5-7. The series of processes in this flowchart may be realized by the CPU 10a executing a computer program stored in memory 10b.

[0068] The flowchart in Figure 5 shows an example of how to select a mode. The first step, S1, is initiated when the driver or reproduction assistant turns on the driving reproduction application (launches the application) via the application operation SW38. In S1, the driver or reproduction assistant obtains information about the mode selected via the application operation SW38. In S2, based on the results obtained in S1, it is determined whether a driving reproduction mode has been selected. If yes, proceed to S3. If no, proceed to S6.

[0069] In S3, it is determined whether the collection of driving characteristic data is complete. For example, completion of driving characteristic data collection may mean that the accumulated reproducible count value has reached a predetermined amount. If yes, proceed to S4. If no, proceed to S5. In S4, the driving reproduction mode processing begins.

[0070] In S5, a diagnostic mode is proposed using the display device 35 and the speaker 36. After processing in S5, the process proceeds to S6.

[0071] In S6, it is determined whether a diagnostic mode is selected based on the results obtained in S1. If yes, proceed to S7. If no, the driving reproduction application is terminated.

[0072] The flowchart in Figure 6 shows an example of the processing in diagnostic mode. In the first step, S101, the driving characteristic data acquisition unit F10 confirms the driver currently seated in the driver's seat based on the driver's application operation SW38 or the vehicle status sensor 31 (particularly the driver status sensor). After processing in S101, the process proceeds to S102.

[0073] In S102, the driving characteristic data acquisition unit F10 determines whether the confirmed driver is already registered as a target driver for the driving reproduction application. If yes, proceed to S105. If no, proceed to S103.

[0074] In S103, the driving characteristic data acquisition unit F10 uses the display device 35 and the application operation SW38 to query the current driver whether or not to register it as a target driver for the driving reproduction application. The driving characteristic data acquisition unit F10 determines whether or not to register the driver based on the query result. If yes, proceed to S104. If no, terminate the series of processes.

[0075] In S104, the driving characteristic data acquisition unit F10 registers the current driver as the target driver. Upon registration, a new reproduction target count value is created that corresponds individually to that driver. After processing in S104, the process proceeds to S105.

[0076] In S105, the driving characteristic data acquisition unit F10 determines whether a cancellation operation was performed in the application operation SW38. If yes, the series of processes ends. If no, proceed to S106.

[0077] In S106, the driving characteristic data acquisition unit F10 determines whether the engine switch (also called the ignition switch) of vehicle 1 is in the OFF state. If Yes, the series of processes ends. If No, proceed to S107.

[0078] In S107, the driving characteristic data acquisition unit F10 begins acquiring driving characteristic data for the current driver. That is, the processing of the discomfort level calculation unit F11, the scene selection unit F12, and the data recording unit F13 begins. After processing in S107, the process proceeds to S108.

[0079] In S108, the scene selection unit F12 determines whether the driving characteristic data for a specific scene has reached a predetermined amount and whether it is possible to reproduce the driving. If yes, proceed to S109. If no, return to S105.

[0080] In S109, the scene selection unit F12 notifies the driver via the display device 35 and speaker 36 that the system is ready to reproduce the driving for a specific scene. The series of processes ends in S109.

[0081] The flowchart in Figure 7 shows an example of the processing in the driving reproduction mode. In the first step, S201, the driving reproduction unit F20 determines whether or not a cancellation operation was performed in the application operation SW38. If yes, the series of processes ends. If no, proceed to S202.

[0082] In S202, the driving simulation unit F20 determines whether the engine switch of vehicle 1 is turned off. If yes, the series of processes ends. If no, proceed to S202.

[0083] In S203, the driving simulation unit F20 determines whether it is ready to simulate driving. Specifically, it determines whether the scene currently encountered by vehicle 1 corresponds to the scene to be simulated, whether the probability of a collision during the simulation is lower than a preset threshold, and whether autonomous driving control can be performed normally. If yes, proceed to S204. If no, proceed to S205.

[0084] In S204, the driving reproduction unit F20 starts reproducing driving using autonomous driving control. That is, the processing of the control variable calculation unit F21 and the adjustment unit F22 begins.

[0085] Meanwhile, in S205, the driving reproduction unit F20 temporarily stops the reproduction of driving by autonomous driving control. After processing in S205, the process returns to S203. That is, after a predetermined time has elapsed, or based on a predetermined trigger, the decision in S203 will be executed again.

[0086] According to the first embodiment described above, predetermined characteristics to be focused on are set in advance for the driving of the driver to be reproduced, and driving characteristic data is acquired for driving scenes in which those characteristics appear. When the driver's driving is reproduced by the autonomous driving control of vehicle 1, adjustments are made to enable the driver to properly experience the predetermined characteristics. Depending on the manner in which adjustments are made, it becomes possible to avoid the disadvantages based on fidelity of reproduction and to prioritize the effective experience of those characteristics. Therefore, it becomes possible to reproduce driving appropriately.

[0087] Furthermore, according to the first embodiment, the driving scene in which the predetermined characteristics appear is a scene in which the passenger experiences discomfort due to the driver's driving. The modified embodiment is a modified embodiment that allows the driver who performed the driving to actually experience the discomfort. By reproducing the driver's driving, the driver can more objectively experience the passenger's discomfort.

[0088] Furthermore, according to the first embodiment, the modified form includes the exaggeration or dwarfing of a predetermined feature. By exaggerating or dwarfing, the predetermined feature can be appropriately perceived by the driver.

[0089] Furthermore, according to the first embodiment, the modified configuration includes limitations on the kinetic physical quantities in autonomous driving control during reproduction. Therefore, it becomes possible to appropriately reproduce driving within the range permitted for autonomous driving control.

[0090] Furthermore, according to the first embodiment, the modified configuration includes an embodiment that enables immersion through the coordination of autonomous driving control and HMI. By combining driving control and HMI, it is possible to more effectively enable users to experience the features.

[0091] Furthermore, according to the first embodiment, the modified aspect is an information presentation realized using an HMI, which includes the presentation of information about the difference between the reproduced driving and the driver's actual driving. By recognizing the difference through the information presentation, the driver can become aware of their actual driving in reverse.

[0092] Furthermore, according to the first embodiment, the differences presented in the information, which are the differences between the reproduced driving and the driver's actual driving, are differences caused by the exaggeration or simplification of predetermined characteristics, or by the limitation of kinetic physical quantities in autonomous driving control during reproduction. By supplementing these differences caused by reasons on the driving control side through information presentation, the driver's understanding of predetermined characteristics can be further promoted.

[0093] Furthermore, according to the first embodiment, the modified configuration includes limiting the permissible range of kinetic physical quantities in autonomous driving control, and providing instructions for additional manual inputs, implemented by the HMI, to reduce the difference between the reproduced driving resulting from the limit and the driver's actual driving. By manually performing additional inputs based on the instructions, it becomes possible to bring the reproduction of driving with discrepancies closer to the driver's actual driving. Therefore, it becomes possible to reproduce a state close to the driver's actual driving while suppressing excessive load on the autonomous driving control.

[0094] Furthermore, according to the first embodiment, the driving reproduction system further includes a memory 10b as a storage medium for recording driving characteristic data. The system then makes a judgment regarding the degree of a predetermined characteristic (e.g., discomfort level) in relation to the driver's driving, and if the degree of the characteristic is above a preset level, the driving characteristic data acquired for that driving is recorded in the memory 10b. Since the driving characteristic data necessary for reproduction is recorded on the premise of reproducing the predetermined characteristics in a realistic manner, the load on hardware such as the storage medium can be reduced while achieving the purpose of reproduction.

[0095] Furthermore, according to the first embodiment, a count value corresponding to the degree of a predetermined characteristic, which corresponds to the driving characteristic data recorded in memory 10b, is accumulated over a preset period. When the count value accumulated over the preset period reaches a preset amount, the driver's driving is changed from a ready-to-reproduce state to a reproducible state. By limiting the driver's driving to be reproduced to a period close to the present, the predetermined characteristics can be effectively experienced in relation to the driver's current driving.

[0096] Furthermore, according to the first embodiment, the driver's driving scenes are classified into multiple reproducible scenes. The degree of a predetermined characteristic is determined for each reproducible scene, and it is determined whether or not the driver's driving in the reproducible scene should be made reproducible. Reproducibility is then performed for the reproducible scenes that are determined to be reproducible. Since the driving reproduction is performed specifically on driving scenes in which the predetermined characteristics are prominently displayed, the predetermined characteristics can be effectively observed by the driver.

[0097] Furthermore, according to the first embodiment, the driving characteristic data is configured to allow for the retrospective reproduction of either or both of the driver's driving and predetermined characteristics. By using such driving characteristic data, driving can be appropriately reproduced.

[0098] (Other embodiments) Although one embodiment has been described above, this disclosure is not limited to that embodiment and can be applied to various embodiments without departing from the gist of this disclosure.

[0099] In other embodiments, various methods may be employed for calculating the reproducible count value corresponding to the reproducible scene. The threshold values ​​for the deviation amount, such as 0.3G and 0.5G, used when assigning discomfort levels may be changed as appropriate. Furthermore, not only the deviation amount but also the amount of control performed by the driver's manual operation may be included as a condition for assigning discomfort levels. For example, the control amount may be assigned to three control amount levels: "large," "medium," and "small," and when the control amount level is "large," the discomfort level may be set to "large" regardless of the deviation amount. Alternatively, the reproducible count may be calculated directly from the deviation amount without going through the discomfort level.

[0100] In another embodiment, the degree of discomfort or discomfort level may be calculated using gripping information obtained from a gripping force sensor on the steering wheel. For example, if the driver is driving with one hand or hands off the wheel, it may be determined that the passenger is experiencing discomfort.

[0101] In another embodiment, the degree of discomfort or discomfort level may be calculated using information about the driver's gaze obtained from a driver state sensor. For example, if the driver is distracted or drowsy, it may be determined that the passenger is experiencing discomfort.

[0102] In another embodiment, the degree of discomfort or discomfort level felt by the passenger may be calculated using the results of direct measurements of the passenger. For example, a passenger state sensor may be provided to measure the passenger's condition, and the degree of discomfort or discomfort level may be calculated using the measured condition of the passenger. More specifically, the passenger state sensor is configured to measure the passenger's facial expression using a camera and to analyze the passenger's emotions based on that expression. If the analysis determines that the passenger is experiencing discomfort, the discomfort or discomfort level may be set to a higher value.

[0103] In another embodiment, the reproducible count value corresponding to each reproducible scene may be configured to be reset at any time by the driver using the application operation SW38. This reset means returning the reproducible count value to its initial value of 0. The system may also be configured to delete the driving characteristic data corresponding to the count from memory 10b in conjunction with the reset.

[0104] In other embodiments, the characteristics of the driving to be reproduced may not be those that cause discomfort to the occupants. For example, the characteristics of the driving to be reproduced may be dangerous or uneconomical (fuel-inefficient) driving. Conversely, the characteristics may be those that are generally considered exemplary.

[0105] In another embodiment, the driving characteristic data only needs to be configured to allow for the retrospective reproduction of either or both of the driver's driving and predetermined characteristics, and various data or datasets can be used.

[0106] For example, the driving characteristic data may include time-series data of the position of vehicle 1 calculated by a locator mounted on vehicle 1 using a self-position estimation program. The self-position of vehicle 1 can be calculated by combining, for example, GPS data indicating the latitude and longitude of vehicle 1 obtained from the Global Positioning System (GPS), and sensor data from external environmental sensors 32. The control variable calculation unit F21 can estimate the behavior of vehicle 1 as driven by the driver based on the time-series data of the position of vehicle 1 and calculate the control variable of vehicle 1.

[0107] For example, the driving characteristic data may include time-series data showing the driver's driving operation history. This time-series data showing the driving operation history may include, for example, time-series data of the accelerator pedal opening angle operated by the driver, or time-series data of the brake fluid pressure corresponding to the brake pedal operated by the driver. The control quantity calculation unit F21 can calculate the control quantities of the vehicle 1 to reproduce the driver's driving operations based on the time-series data showing the driver's driving operation history.

[0108] In another embodiment, the application operation switch 38 may be a switch displayed on the screen of a touch panel display device 35.

[0109] In another embodiment, the vibration device 37 does not have to vibrate the steering wheel, but may, for example, vibrate the driver's seat.

[0110] In another embodiment, the computer comprising at least one of the driving reproduction device 10 and the driving support device 20 may include another type of processor (e.g., a GPU) instead of, or in conjunction with, the CPU 10a.

[0111] In another embodiment, the computer comprising at least one of the driving reproduction device 10 and the driving support device 20 may incorporate circuits such as FPGAs. Then, a portion of the processing performed by at least one of the driving reproduction device 10 and the driving support device 20 may be realized by the processor controlling the circuit based on a computer program, or by the circuit operating independently.

[0112] In another embodiment, the driving simulation system may be wirelessly connected to a mobile terminal such as a smartphone used by the driver or simulation assistant. The driving simulation device 10 included in the driving simulation system may, together with or in place of the display device 35 mounted on the vehicle, send a display request to the mobile terminal and cause the mobile terminal to display the display content DC1, DC2, DC3.

[0113] In other embodiments, the driving reproduction system may employ various configurations. For example, a device having the functions of a driving characteristic data acquisition unit F10 and a device having the functions of a driving reproduction unit F20 may be provided in a manner that allows them to communicate with each other. Alternatively, the driving reproduction system may consist of a device having the functions of a driving characteristic data acquisition unit F10 and a device that adds the functions of the driving reproduction unit F20 to the functions of the driving support device 20. The driving reproduction system may also be configured by an integrated ECU that integrates all the functions of the driving support device 20 and the functions of the driving reproduction device 10. The driving reproduction system may include devices other than in-vehicle devices, such as the aforementioned portable terminal. [Explanation of Symbols]

[0114] 1: Vehicle, 10: Driving simulation device (driving simulation system), 10a: CPU (processing unit)

Claims

1. A driving reproduction system comprising at least one processing unit (10a) for reproducing the driving of a driver in a vehicle (1), The at least one processing unit is, To acquire driving characteristic data of driving scenes in which predetermined characteristics appear in the driving of the aforementioned driver, A driving reproduction system that reproduces the driving of a driver by autonomous driving control of the vehicle, in a manner in which adjustments have been made to enable the driver who performed the driving to properly experience the predetermined characteristics based on the aforementioned driving characteristic data.

2. The driving scene in which the aforementioned predetermined characteristics appear is a scene in which a passenger experiences discomfort due to the driver's driving. The driving reproduction system according to claim 1, wherein the adjusted embodiment is an embodiment in which adjustments are made so that the driver who performed the driving can feel the discomfort.

3. The operational reproduction system according to claim 1 or 2, wherein the modified embodiment includes an exaggeration or simplification of the predetermined features.

4. The driving reproduction system according to claim 1 or 2, wherein the modified embodiment includes limiting of kinetic physical quantities in the autonomous driving control during reproduction.

5. The driving reproduction system according to claim 1 or 2, wherein the modified embodiment includes the realization of the autonomous driving control through cooperation with HMI (35, 36, 37).

6. The modified mode is an information presentation implemented using an HMI, which includes the presentation of information about the difference between the reproduced driving and the driver's actual driving, according to claim 5, the driving reproduction system.

7. The driving reproduction system according to claim 6, wherein the difference between the reproduced driving and the actual driving of the driver is due to the exaggeration or simplification of the predetermined characteristics as adjustment, or the limitation of kinetic physical quantities in the autonomous driving control during reproduction as adjustment.

8. The embodiment with the aforementioned adjustments is: Limitation of the permissible range of kinetic physical quantities in the autonomous driving control, The driving reproduction system according to claim 1 or 2, comprising instructions for additional manual inputs, which are implemented by HMIs (35, 36, 37), to reduce the difference between the reproduced driving resulting from the limitations and the actual driving of the driver.

9. The system further includes a storage medium (10b) for recording the aforementioned driving characteristic data, The at least one processing unit is, The driving reproduction system according to claim 1 or 2, further comprising determining the degree of the predetermined characteristics in relation to the driving of the driver, and if the degree of the characteristics is greater than or equal to a preset degree, recording the driving characteristic data acquired for that driving in the storage medium.

10. The at least one processing unit is, A count value corresponding to the degree of the predetermined characteristic, which corresponds to the driving characteristic data recorded on the storage medium, is accumulated over a predetermined set period. The operation reproduction system according to claim 9, further comprising changing the operation of the driver from a reproduction preparation state to a reproductionable state when the count value accumulated during the aforementioned set period reaches a predetermined amount.

11. The at least one processing unit is, The aforementioned driver's driving scenes are classified into multiple reproducible scenes, Further, the process involves determining the degree of the predetermined features for each of the scenes to be reproduced, and deciding whether or not to make the driver's operation in the scene to be reproduced reproducible. The operation reproduction system according to claim 1 or 2, wherein the reproduction is performed in the reproduction target scene which is determined to be in a reproducible state.

12. The driving characteristic data is data configured to allow for the retrospective reproduction of either or both of the driver's driving and the predetermined characteristics, according to claim 1 or 2.

13. A driving simulation program that reproduces the driver's actions in a vehicle (1), At least one processing unit (10a) To acquire driving characteristic data of driving scenes in which predetermined characteristics appear in the driving of the aforementioned driver, A driving reproduction program that reproduces the driving of the driver by autonomous driving control of the vehicle, in a manner in which adjustments have been made to enable the driver who performed the driving to properly experience the predetermined characteristics based on the aforementioned driving characteristic data.

Images