Notification control device, method, and program, and driving support system

Abstract

This notification control device includes a controller that sets, on the basis of input data (Din) and using estimation AI (100), notification parameters (Pout) to be used when notifying a user of target information. The controller sets the notification parameters on the basis of the input data and the reaction index of the user to a past notification.

Description

Notification control device, method and program, and driver assistance system

[0001] The present invention relates to a notification control device, method and program, and a driver assistance system.

[0002] Various devices that notify users of information are widely used. For example, devices that provide guidance and notifications for right and left turn points and for detecting approaching obstacles are installed in automobiles and other vehicles. Information can be notified through the user's sight, hearing, or touch. Furthermore, a method for issuing warnings to individuals who perform predetermined actions using a model is disclosed in Patent Document 1.

[0003] Japanese Patent Publication No. 2023-109396

[0004] Users may be dissatisfied with the way information is notified. As a countermeasure, using AI (artificial intelligence) to determine parameters for information notification is considered effective. However, the AI ​​installed in the device may not be tailored to the individual user's characteristics.

[0005] The present invention aims to provide a notification control device, method, and program, as well as a driving assistance system, that contribute to realizing notifications tailored to the individual characteristics of users.

[0006] The notification control device according to the present invention includes a controller that sets notification parameters when notifying a user of target information using estimation AI based on input data, and the controller sets the notification parameters based on the input data and the user's evaluation index for past notifications.

[0007] User evaluation metrics for past notifications reflect the user's personal characteristics (preferences, etc.). Therefore, by setting notification parameters based on user evaluation metrics for past notifications, it is possible to reflect the user's personal characteristics in the settings of the notification parameters. As a result, it becomes possible to implement notifications in a manner that is appropriate to the user's personal characteristics.

[0008] (a) to (d) are explanatory diagrams of multiple types of vibration output methods according to embodiments of the present invention. (a) to (e) are diagrams illustrating the relationship between a person to receive notification and other components (a) to (d) are explanatory diagrams of the method for acquiring FB information for notification of target information, according to an embodiment of the present invention. (b) is a hardware configuration diagram related to pre-machine learning, according to an embodiment of the present invention. (c) is a flowchart related to the preparation and operation of an in-vehicle device, according to an embodiment of the present invention. (d) is a diagram showing the internal configuration of the notification parameter setting unit and its peripheral configuration, according to Example EX_1A belonging to an embodiment of the present invention. (c) is an operation flowchart related to the controller's information notification processing, according to Example EX_1A belonging to an embodiment of the present invention. (c) is a conceptual diagram of input data correction, according to Example EX_1A belonging to an embodiment of the present invention.Figures (a) to (d) relate to Example EX_1A, which belongs to an embodiment of the present invention, and show specific examples of correction content adjustment processing based on FB information. Figures (a) to (d) relate to Example EX_1A, which belongs to an embodiment of the present invention, and show specific examples of correction content adjustment processing based on FB information. Figures (a) to (d) relate to Example EX_1A, which belongs to an embodiment of the present invention, and show specific examples of correction content adjustment processing based on FB information. Figures (a) to (d) relate to Example EX_1A, which belongs to an embodiment of the present invention, and show specific examples of correction content adjustment processing based on FB information. Figures (a) to (d) relate to Example EX_1B, which belongs to an embodiment of the present invention, and show the internal configuration of the notification parameter setting unit and its peripheral configuration. Figures (a) to (d) relate to Example EX_1B, which belongs to an embodiment of the present invention, and show the relationship between the output parameter, notification parameter and correction amount. Figures (a) to (d) relate to Example EX_1B, which belongs to an embodiment of the present invention, and show a conceptual diagram of output parameter correction. Figures (a) to (d) relate to Example EX_1B, which belongs to an embodiment of the present invention, and show specific examples of correction content adjustment processing based on FB information. Figures (a) to (d) relate to Example EX_1B, which belongs to an embodiment of the present invention, and show specific examples of correction content adjustment processing based on FB information. Figures (a) to (d) relate to Example EX_1B, which belongs to an embodiment of the present invention, and show specific examples of correction content adjustment processing based on FB information. Figures (a) to (d) relate to Example EX_1B, which belongs to an embodiment of the present invention, and show specific examples of correction content adjustment processing based on FB information. Figures (a) to (d) relate to Example EX_1C, which belongs to an embodiment of the present invention, and show an explanatory diagram of the configuration of the FB set. Figures (a) to (d) relate to Example EX_2A, which belongs to an embodiment of the present invention, and show an explanatory diagram of the mode switching of the notification parameter setting unit. Figures (a) to (d) relate to Example EX_2A, which belongs to an embodiment of the present invention, and show an operation flowchart related to the controller's information notification processing. Figures (a) to (d) relate to Example EX_2A, which belongs to an embodiment of the present invention, and show a diagram of the relationship between the first mode period and the second mode period. This figure relates to Example EX_2C, which belongs to an embodiment of the present invention, and illustrates a method for setting notification parameters after specific FB information has been obtained. This figure relates to Example EX_3A, which belongs to an embodiment of the present invention, and shows the internal configuration of the notification parameter setting unit employing a pre-correction configuration and the peripheral configuration related to retraining.This diagram relates to Example EX_3A, which belongs to an embodiment of the present invention, and shows the internal configuration of the notification parameter setting unit employing a post-stage correction configuration and the peripheral configuration related to retraining. This diagram relates to Example EX_3A, which belongs to an embodiment of the present invention, and is an explanatory diagram of the method for generating unit training data based on positive FB information. This diagram relates to Example EX_3A, which belongs to an embodiment of the present invention, and is an explanatory diagram of the method for generating unit training data based on negative FB information. This diagram relates to Example EX_3B, which belongs to an embodiment of the present invention, and is an operation flowchart related to the controller's information notification processing. This diagram relates to Example EX_3C, which belongs to an embodiment of the present invention, and is an operation flowchart related to the controller's information notification processing.

[0009] Hereinafter, examples of embodiments of the present invention will be specifically described with reference to the drawings. In each of the referenced figures, the same parts are denoted by the same reference numerals, and redundant descriptions relating to the same parts are omitted as a general rule. In this specification, for the sake of simplification of the description, symbols or reference numerals that refer to information, signals, physical quantities, functional parts, circuits, elements, or components may be used, and the names of the information, signals, physical quantities, functional parts, circuits, elements, or components corresponding to such symbols or reference numerals may be omitted or abbreviated. For example, the parameter estimation AI referred to by "100" described later (see Figure 15) may be written as parameter estimation AI 100, or abbreviated as estimation AI 100 or AI 100, but all of these refer to the same thing.

[0010] Figure 1 shows the relationship between person U1 and other components assumed in an embodiment of the present invention. Person U1 is the recipient of the notification and the user of the driver assistance system SYS and the in-vehicle device 10, and is hereinafter referred to as user U1. Vehicle V1 is any type of vehicle. Here, vehicle V1 is assumed to be an automobile or the like that travels on the road surface. User U1 is an occupant of vehicle V1. In this embodiment, it is assumed that user U1 is the driver of vehicle V1. Hereafter, when simply referred to as the driver, it refers to the driver of vehicle V1 (and therefore user U1). However, user U1 may be an occupant other than the driver.

[0011] A driver assistance system SYS is installed in vehicle V1. The driver assistance system SYS has a notification function that provides various notifications to user U1, and when focusing on the notification function, the driver assistance system SYS functions as an in-vehicle notification system. In this embodiment, unless otherwise specified, "notification" refers to a notification to user U1.

[0012] A seat ST1 is installed inside the vehicle V1. User U1 sits in seat ST1. Figure 2 is an external perspective view of seat ST1. Here, it is assumed that user U1 is the driver, so seat ST1 is the driver's seat. Hereafter, when simply referred to as "vehicle interior," it refers to the vehicle interior of vehicle V1 unless otherwise specified. Also, hereafter, when simply referred to as "inside the vehicle," it refers to the interior area of ​​vehicle V1 unless otherwise specified, and when simply referred to as "outside the vehicle," it refers to the exterior area of ​​vehicle V1 unless otherwise specified.

[0013] The direction from the driver's seat of vehicle V1 towards the steering wheel is defined as "forward," and the direction from the steering wheel of vehicle V1 towards the driver's seat is defined as "rear." The direction perpendicular to the longitudinal direction and parallel to the road surface on which vehicle V1 travels is defined as the left-right direction. The direction perpendicular to both the longitudinal direction and the left-right direction is defined as the up-down direction. User U1 is assumed to be seated in seat ST1 facing forward. The longitudinal, left-right, and up-down directions correspond to the longitudinal, left-right, and up-down directions as seen from the perspective of user U1. Unless otherwise specified below, the direction of travel of vehicle V1 is assumed to be forward.

[0014] To further elaborate on the explanation, the relationship between the mutually orthogonal X, Y, and Z axes and the front-to-back, left-to-right, and up-and-down directions is defined as follows: The X-axis direction is parallel to the left-to-right direction. The Y-axis is parallel to the front-to-back direction. The Z-axis is parallel to the up-and-down direction. The direction from left to right coincides with the direction from the negative side to the positive side of the X-axis. The direction from rear to front coincides with the direction from the negative side to the positive side of the Y-axis. The direction from bottom to top coincides with the direction from the negative side to the positive side of the Z-axis.

[0015] As shown in Figure 2, the seat ST1 comprises a seat surface ST1a and a backrest ST1b. When user U1 sits on the seat ST1, the backs of user U1's thighs and buttocks come into contact with the seat surface ST1a, and user U1's back comes into contact with the backrest ST1b. More specifically, the seat surface ST1a has a seat surface that is approximately parallel to the X and Y axes, and when user U1 sits on the seat ST1, the backs of user U1's thighs and buttocks come into contact with the seat surface. The backrest ST1b has a backrest surface that is approximately parallel to the X and Z axes, and when user U1 sits on the seat ST1, user U1's back comes into contact with the backrest surface.

[0016] Figure 3 shows a schematic block diagram of the driver assistance system SYS. The driver assistance system SYS comprises an in-vehicle device 10, a vehicle control device 20, an actuator unit 30, a sensing unit 40, and an information output unit 50. Each component of the driver assistance system SYS is capable of transmitting and receiving arbitrary signals and information from one another through an in-vehicle network formed within the vehicle V1. The in-vehicle network includes, for example, a CAN (Controller Area Network) and an AVCLAN (Audio Visual Communication Local Area Network).

[0017] The in-vehicle device 10 performs various controls to realize notifications to user U1. For this reason, the in-vehicle device 10 has a notification control device. Alternatively, the in-vehicle device 10 can also be referred to as the notification control device. Under the control of the in-vehicle device 10, the information output unit 50 is activated to actually provide notifications. That is, various notifications to user U1 are provided through the cooperation of the in-vehicle device 10 and the information output unit 50. It can also be thought that the in-vehicle device 10 and the information output unit 50 form an information notification device. As will be described in detail later, the information output unit 50 includes a display device and a speaker, etc. The in-vehicle device 10 may also have functions other than those of a notification control device. Functions other than notification functions include navigation functions and drive recorder functions, as well as entertainment functions that play back and output arbitrary video and audio signals.

[0018] The vehicle control device 20 controls the movement of the vehicle V1 using the actuator unit 30. The actuator unit 30 has various drive components, such as motors, that enable the movement of the vehicle V1. Specifically, the actuator unit 30 includes an engine and motor that generate the driving force of the vehicle V1, a steering actuator that drives the steering of the vehicle V1, and a brake actuator that drives the brakes of the vehicle V1. Here, the on-board device 10 and the vehicle control device 20 are shown separately, but the vehicle control device 20 may be included within the on-board device 10. In addition, the vehicle V1 may also be equipped with on-board devices other than the on-board device 10 and the vehicle control device 20 (hereinafter referred to as other on-board devices). Other on-board devices may or may not be components of the driver assistance system SYS.

[0019] The sensing unit 40 includes sensors for detecting driving operations of the vehicle V1 by the driver of the vehicle V1, sensors for detecting various states of the vehicle V1, and sensors for detecting conditions inside and outside the vehicle. Sensing information is output from the sensing unit 40. The sensing information includes various information and signals generated or detected by each component of the sensing unit 40 (see Figure 5). The sensing information is supplied to the in-vehicle device 10 and the vehicle control device 20. The vehicle control device 20 can drive and control the actuator unit 30 based on the sensing information. The in-vehicle device 10 can use the information output unit 50 to notify the user U1 according to the sensing information, and can also notify the user U1 in a manner independent of the sensing information.

[0020] Figure 4 shows the internal configuration of the in-vehicle device 10. The in-vehicle device 10 comprises a controller 11, memory 12, communication unit 13, recording medium 14, and operation input unit 15.

[0021] The controller 11 includes a arithmetic processing unit 11a, including a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit), as hardware resources. The controller 11 may implement any functions, operations, and processes to be realized by the controller 11 by executing a program recorded in the memory 12, the recording medium 14, or any other recording medium (not shown). All or part of the operations mainly performed by the controller 11 described below may be understood as operations performed by the arithmetic processing unit 11a. In the following description, when the controller 11 notifies the user U1, it means that the controller 11 notifies the user U1 using the information output unit 50, and in this case, the controller 11 controls the information output unit 50 so that the information to be notified is output from the information output unit 50 to the user U1.

[0022] Memory 12 is composed of non-volatile memory such as ROM (Read-only memory) or flash memory, and volatile memory such as RAM (Random Access Memory). Memory 12 stores various data that the controller 11 refers to, as well as various programs that should be executed by the controller 11.

[0023] The communication unit 13 is a communication circuit (communication module) that transmits and receives arbitrary signals between the in-vehicle device 10 and a different other device. The communication unit 13 may be a communication circuit (communication module) located outside the in-vehicle device 10. The communication circuit located outside the in-vehicle device 10 may be a device installed in the vehicle V1 and may be a communication circuit shared by the in-vehicle device 10 and other devices besides the in-vehicle device 10. The communication unit 13 may also be a communication circuit (communication module) installed in an information terminal device (smartphone, etc.) brought into the vehicle V1. The information terminal device brought into the vehicle V1 is an information terminal device owned by user U1 and located inside the vehicle V1, and is hereinafter referred to as the user terminal device. The other device for the communication unit 13 includes components other than the in-vehicle device 10 among the components of the driver assistance system SYS shown in Figure 3. The communication unit 13 can communicate with the other device via the in-vehicle network formed in the vehicle V1. For the communication unit 13, the other party's device may include an external device connected to an external network (such as a server device located outside the vehicle). The external network includes mobile communication networks, the internet, and intranets. The controller 11 can send and receive arbitrary information with the other party's device using the communication unit 13, but the description of the communication unit 13 may be omitted below.

[0024] The recording medium 14 is a non-volatile recording medium consisting of a magnetic disk or flash memory, and stores (records) arbitrary information non-volatilely. The controller 11 can record arbitrary information on the recording medium 14 and can also read arbitrary information recorded on the recording medium 14. The recording medium 14 may be detachably attached to the in-vehicle device 10. The recording medium 14 may also be located outside the in-vehicle device 10 and installed within the driver assistance system SYS. If the in-vehicle device 10 has the function of a drive recorder, the controller 11 can record image data of images captured by the camera 41 or 42 (see Figure 5), which will be described later, on the recording medium 14.

[0025] The operation input unit 15 receives arbitrary operations from user U1. User U1 can input arbitrary operations to the in-vehicle device 10, and operations to the in-vehicle device 10 by user U1 are received by the operation input unit 15. Hereinafter, operations to the in-vehicle device 10 by user U1 may be referred to as user operations. Information indicating the content of user operations will be referred to as user operation information. When a user operation is input to the in-vehicle device 10, user operation information indicating the content of the input user operation is transmitted from the operation input unit 15 to the controller 11. The operation input unit 15 has a touch panel, and the user operation may be an operation on the touch panel. The touch panel may be provided on the display screen 51a (see Figure 6), which will be described later. The operation input unit 15 may also have operation components other than the touch panel (such as mechanical push buttons), in which case the user operation may be an operation on such operation component. Furthermore, user operations may be voice operations. Voice operation refers to an operation in which user U1 inputs the operation intended by user U1 to the in-vehicle device 10 through user U1's speech. A microphone located in the vehicle V1 (for example, the in-vehicle microphone 44 described later; Figure 5) converts the user U1's speech into an electrical signal, and the controller 11 can recognize the content of the voice operation through speech recognition of the electrical signal representing the speech content. Speech recognition may be performed by the controller 11 or by the operation input unit 15.

[0026] Figure 5 shows the internal configuration of the sensing unit 40. The sensing unit 40 includes an external camera 41, an internal camera 42, an external microphone 43, and an internal microphone 44.

[0027] The external camera 41 consists of one or more cameras that capture images of the outside of the vehicle V1. The external camera 41 has a shooting area set outside the vehicle V1, and generates an external camera image by capturing images of the area within the shooting area. The external camera image is an image of the shooting area captured by the external camera 41. Image information representing the external camera image is called external image information. External image information is included as a component of sensing information. The external camera 41 captures images at a predetermined frame rate.

[0028] The in-vehicle camera 42 consists of one or more cameras that capture images of the interior of the vehicle V1. The in-vehicle camera 42 has a shooting area set inside the vehicle V1 (i.e., the passenger compartment of the vehicle V1), and generates an in-vehicle camera image by capturing images of the shooting area. The in-vehicle camera image is an image of the shooting area captured by the in-vehicle camera 42. Image information indicating the in-vehicle camera image is called in-vehicle image information. In-vehicle image information is included as a component of sensing information. The in-vehicle camera 42 takes images at a predetermined frame rate. User U1 is located within the shooting area of ​​the in-vehicle camera 42, and therefore the image of user U1 is included in the in-vehicle camera image. User U1's face is included within the shooting area of ​​the in-vehicle camera 42, and therefore the in-vehicle camera image includes an image of user U1's face (an image of user U1's face). Furthermore, when other passengers are present in the vehicle, they are also assumed to be located within the shooting area of ​​the in-vehicle camera 42.

[0029] The external microphone 43 picks up ambient sounds around its installation location and converts the picked-up sounds into electrical signals. The electrical signals obtained by the conversion of the external microphone 43 are called external microphone signals. The external microphone signals are included as components of sensing information. The external microphone 43 is installed at a predetermined location on the vehicle body of the vehicle V1 and picks up sounds generated outside the vehicle V1.

[0030] The in-vehicle microphone 44 picks up ambient sounds around its installation location and converts the picked-up sounds into electrical signals. The electrical signals obtained by the conversion of the in-vehicle microphone 44 are called in-vehicle microphone signals. The external microphone signals are included as components of sensing information. The in-vehicle microphone 44 is installed inside the vehicle V1, and therefore, the in-vehicle microphone signals represent the electrical signals of sounds inside the vehicle. The content of user U1's speech is picked up by the in-vehicle microphone 44. The in-vehicle microphone 44 may also be a microphone provided on the user terminal device.

[0031] The sensing unit 40 further includes sensors 45a to 45k. Sensors 45a, 45b, and 45c are the accelerator pedal sensor, brake pedal sensor, and steering wheel sensor, respectively. The vehicle V1 is equipped with operating components that receive driving operations from the driver, and these operating components include the accelerator pedal, brake pedal, and steering wheel. Sensors 45d, 45e, 45f, and 45g are the vehicle speed sensor, steering angle sensor, G sensor, and distance sensor, respectively. Sensors 45h, 45i, 45j, and 45k are the GPS sensor, temperature sensor, rainfall sensor, and illuminance sensor, respectively. The information generated by sensors 45a to 45k is included as a component of the sensing information.

[0032] The accelerator pedal sensor 45a detects the operation of the accelerator pedal of vehicle V1 by the driver of vehicle V1 and generates and outputs accelerator pedal operation information indicating the operation of the accelerator pedal. The brake pedal sensor 45b detects the operation of the brake pedal of vehicle V1 by the driver of vehicle V1 and generates and outputs brake pedal operation information indicating the operation of the brake pedal. The steering wheel sensor 45c detects the operation of the steering wheel of vehicle V1 by the driver of vehicle V1 and generates and outputs steering wheel operation information indicating the operation of the steering wheel.

[0033] The vehicle speed sensor 45d detects the speed of the vehicle V1 and generates and outputs vehicle speed information (vehicle speed pulse) representing the detected speed. The steering angle sensor 45e detects the steering angle of the vehicle V1 and generates and outputs steering angle information representing the detected steering angle. The G sensor 45f detects the acceleration applied to the vehicle V1 in a predetermined axial direction and generates and outputs the acceleration detection result as acceleration information. The G sensor 45f may detect acceleration in two mutually orthogonal axial directions, or it may detect acceleration in three mutually orthogonal axial directions.

[0034] The distance measuring sensor 45g generates and outputs distance measurement information by measuring distance. In the distance measuring sensor 45g, the distance between the vehicle V1 and a three-dimensional object located within the distance measuring area around the vehicle V1 is detected, as well as the orientation of the object relative to the vehicle V1. These detection results are included in the distance measurement information. The distance measuring sensor 45g may be composed of a LIDAR (Light Detection and Ranging) that uses light to measure distance, or it may be composed of a radar that uses radio waves to measure distance. The distance measuring sensor 45g may also be composed of a combination of LIDAR and radar.

[0035] The GPS sensor 45h receives signals from multiple GPS satellites that form the GPS (Global Positioning System), and generates and outputs vehicle position information based on the received results. The vehicle position information generated by the GPS sensor 45h represents the current location of vehicle V1 in terms of longitude and latitude, or represents the current location of vehicle V1 in terms of longitude, latitude and altitude.

[0036] The temperature sensor 45i detects the temperature inside the vehicle V1 and generates and outputs temperature information representing the detected temperature. The rainfall sensor 45j detects whether it is raining outside the vehicle V1 and generates and outputs rainfall information representing the detection result. The rainfall sensor 45j may also detect whether it is raining by detecting whether water droplets are adhering to the windshield of the vehicle V1. The illuminance sensor 45k detects the illuminance outside the vehicle V1 and the illuminance inside the vehicle V1 and generates and outputs illuminance information representing the detected illuminance.

[0037] The sensing unit 40 further includes a biosensor 46. The biosensor 46 detects the biometric information of user U1. The biosensor 46 may have an electroencephalogram (EEG) sensor. The EEG sensor detects the brainwaves of user U1 and generates EEG data indicating the EEG detection result. The biosensor 46 may also have a heart rate sensor. The heart rate sensor detects the heart rate of user U1 and generates heart rate data indicating the heart rate detection result. The biometric information of user U1 may include the EEG data and heart rate data of user U1. The biometric information of user U1 is included as a component of the sensing information. Based on the biometric information of user U1, the controller 11 (or other components within the driver assistance system SYS) may estimate the emotions of user U1.

[0038] Other sensors (for example, a shift lever sensor and a door lock sensor) may also be provided in the sensing unit 40.

[0039] Figure 6 shows the internal configuration of the information output unit 50. The information output unit 50 includes a display device 51, a speaker 52, and a vibration device 53. Under the control of the in-vehicle device 10, the information output unit 50 outputs information to the user U1 that should be notified to the user U1.

[0040] The display device 51 has a display screen 51a made of a liquid crystal display panel or the like, and displays any image (in other words, picture) according to the control of the in-vehicle device 10, the vehicle control device 20, or a display control device not shown. The display device 51 is installed in an appropriate place in the passenger compartment of the vehicle V1 so that each occupant of the vehicle V1 (at least the user U1) can see the display content of the display device 51. Multiple display devices 51 may be installed in the passenger compartment of the vehicle V1. The display device 51 may be a component of a car navigation system. The car navigation system is installed in the vehicle V1. The car navigation system may be included in a driver assistance system SYS. The display device 51 may be a display device provided in the user terminal device described above. In the following description, "display" refers to the display on the display device 51 unless otherwise specified. More specifically, "display on the display device 51" refers to the display on the display screen 51a provided on the display device 51.

[0041] Speaker 52 outputs any sound (message, music, etc.) under the control of the in-vehicle device 10, the vehicle control device 20, or an audio device (not shown). Speaker 52 is installed in an appropriate location in the vehicle's interior so that each occupant of the vehicle V1 (at least user U1) can hear the output sound from speaker 52. Speaker 52 may be a speaker provided in the user terminal device described above. Multiple speakers 52 may be installed in the vehicle's interior. In the following description, unless otherwise specified, sound and voice output refers to the sound and voice output from speaker 52.

[0042] The vibration device 53 generates vibrations perceptible to the user U1 under the control of the controller 11. The vibration device 53 comprises a plurality of vibrators and a drive circuit that drives each vibrator. The arrangement of the vibrators will be explained with reference to Figure 7. Figure 7 is a plan view of the seat ST1 as seen from the rear. As shown in Figure 7, the backrest ST1b has a central region Cb, a left region Lb located to the left of the central region Cb, and a right region Rb located to the right of the central region Cb. The plurality of vibrators provided in the vibration device 53 include a left vibrator, a central vibrator, and a right vibrator. The left vibrator, central vibrator, and right vibrator are arranged in the left region Lb, central region Cb, and right region Rb, respectively. The vibration device 53 can vibrate one or more of the left vibrator, central vibrator, and right vibrator according to the control of the controller 11. The controller 11 can individually and arbitrarily adjust the vibration intensity and vibration frequency of the left vibrator, central vibrator, and right vibrator.

[0043] The vibration device 53 can produce left-side vibration output, central vibration output, right-side vibration output, or overall vibration output according to the control of the controller 11 (see Figures 8(a) to (d)). In left-side vibration output, only the left vibrator vibrates, and as a result, vibration from the left vibrator is applied only to the left side of the user U1's body. In central vibration output, only the central vibrator vibrates, and as a result, vibration from the central vibrator is applied only to the central side of the user U1's body. In right-side vibration output, only the right vibrator vibrates, and as a result, vibration from the right vibrator is applied only to the right side of the user U1's body. In overall vibration output, the left, central, and right vibrators all vibrate, and as a result, vibration from the left, central, and right vibrators is applied to the entire left, central, and right side of the user U1's body. Note that the left side of the user U1's body refers to the part of the user U1's body to the left of the midline. Similarly, the right side of the user U1's body refers to the part of the user U1's body to the right of the midline. The central part of user U1's body refers to the area between the left and right parts described above, and is the part that encloses the midline. Here, an example is given in which the vibrator is placed on the backrest ST1b, but the vibrator may also be placed on the seat ST1a instead of the backrest ST1b, or in addition to the backrest ST1b.

[0044] [Information Notification Processing] The controller 11 performs information notification processing as a process for notifying the user U1 of various information. In the information notification processing, the controller 11 notifies the user U1 of one or more of the first to n types of information in one or more notification formats from the first to m types. m and n each represent any integer of 2 or more. However, in this embodiment, it is assumed that "m = 5", and Figure 9 shows an overview of the first to fifth notification formats. The i-th type of information, which is any one of the first to n types of information, is called the i-th notification information. The i-th notification information is information that can be expressed in language. i represents any integer.

[0045] The first notification format is a character notification format in which characters representing the i-th notification information are displayed on the display device 51. In the first notification format, an image of the characters representing the i-th notification information is displayed on the display device 51. The image of the characters (an image representing the characters) will be referred to as the character image below. The second notification format is a graphic notification format in which a graphic representing the i-th notification information is displayed on the display device 51. The graphic includes illustrations and pictograms. In the second notification format, an image of the graphic associated with the i-th notification information is displayed on the display device 51. The image of the graphic (an image representing the graphic) will be referred to as the graphic image below. The first and second notification formats are notification formats that affect the user U1's vision. While character display is superior in terms of accuracy of information transmission, graphic display may be more advantageous in terms of speed of information transmission. By providing the first and second notification formats, information can be transmitted (notification of the i-th notification information) in an appropriate notification format according to the user U1's situation and the content of the message.

[0046] The third notification format is an audio notification format in which an audio signal of words indicating the i-th notification information is output as sound from speaker 52. In the third notification format, the words indicating the i-th notification information themselves are reproduced and output as sound from speaker 52. The fourth notification format is an audio notification format in which an acoustic signal associated with the i-th notification information, but which is not words (for example, an acoustic signal whose melody, tempo, etc., differs depending on the information content), is output as sound from speaker 52. In the fourth notification format, the acoustic signal associated with the i-th notification information is reproduced and output as sound from speaker 52. The third and fourth notification formats are notification formats that affect the hearing of user U1. Audio output is superior in accuracy of information transmission, but in terms of speed of information transmission, non-word acoustic output may be more advantageous. By providing the third and fourth notification formats, information can be transmitted (notification of the i-th notification information) in an appropriate notification format according to the user U1's situation and the content of the message.

[0047] The fifth notification format is a vibration notification format in which a vibration device 53 generates vibrations corresponding to the i-th notification information. In the fifth notification format, vibrations corresponding to the i-th notification information are generated by the vibration device 53, and the generated vibrations are transmitted to the user U1. The fifth notification format is a notification format that acts on the user U1's sense of touch. A notification format that acts on the sense of touch does not need to rely on sight or hearing, and is therefore less likely to affect the user U1's driving operations, and in this respect may be more advantageous than a notification format that acts on sight or hearing.

[0048] Figures 10(a) to 10(e) show the situations in which left-turn guidance information is notified to user U1 in the first to fifth notification formats. Left-turn guidance information is information that instructs vehicle V1 to turn left at the next intersection it will reach during the vehicle's journey, and is one of the first to nth notification formats. Left-turn guidance information is information that is notified to user U1 in accordance with the navigation operation. Navigation operation is an operation that guides vehicle V1 to a destination set according to the user U1's wishes, etc., and is performed by the in-vehicle device 10 (controller 11) or other in-vehicle devices. In the notification of left-turn guidance information in the first notification format, the string "Turn left at the next intersection" 611 is displayed on the display screen 51a. In the notification of left-turn guidance information in the second notification format, a figure 612 associated with "Turn left at the next intersection" is displayed on the display screen 51a. In the notification of left-turn guidance information in the third notification format, the words "Please turn left at the next intersection" 613 are output by speaker 52. In the fourth notification format for left-turn guidance information, a sound 614 (here, a sound that sounds like "buzzing") which is not a word and is associated with the left-turn guidance information is output from the speaker 52. In the fifth notification format for left-turn guidance information, a vibration 615 (here, a vibration output on the left side that is perceived as "buzzing") associated with the left-turn guidance information is generated by the vibration device 53 and transmitted to the user U1.

[0049] Figures 11(a) to (e) show the situations in which forward obstacle information is notified to user U1 in the first to fifth notification formats. Forward obstacle information is information that alerts user U1 to the presence of an obstacle in front of vehicle V1, and is one of the first to nth notification formats. In the first notification format for forward obstacle information, the string "Caution: Obstacle ahead" 621 is displayed on the display screen 51a. In the second notification format for forward obstacle information, a graphic 622 associated with "Caution: Obstacle ahead" is displayed on the display screen 51a. In the third notification format for forward obstacle information, the words "Please be careful of the obstacle ahead" 623 are output from the speaker 52. In the fourth notification format for forward obstacle information, a sound 624 that does not correspond to words but is associated with forward obstacle information (here, a sound that sounds like "buzz, buzz") is output from the speaker 52. In the notification of forward obstacle information using the fifth notification format, a vibration 625 (here, a vibration with a central vibration output that is perceived as "buzz, buzz") associated with the forward obstacle information is generated by the vibration device 53 and transmitted to the user U1.

[0050] Left-turn guidance information and forward obstacle information are two different notification pieces included in the first to nth notification pieces. The controller 11 can notify user U1 of left-turn guidance information using one or more of the first to fifth notification formats. In notifications using two notification methods, for example, the string "Turn left at the next intersection" 611, which is a notification of left-turn guidance information using the first notification format, is displayed on the display screen 51a, and the words "Please turn left at the next intersection" 613, which is a notification of left-turn guidance information using the third notification format, are output as voice from the speaker 52. The controller 11 can also notify user U1 of forward obstacle information using one or more of the first to fifth notification formats. The same applies to each notification piece other than left-turn guidance information and forward obstacle information. The string 611, graphic 612, word 613, sound 614, and vibration 615 related to left-turn guidance information differ from the string 621, graphic 622, word 623, sound 624, and vibration 625 related to forward obstacle information, respectively. Therefore, user U1 can distinguish and recognize left-turn guidance information and forward obstacle information. The same applies to each notification information other than left-turn guidance information and forward obstacle information. In other words, the notification mode of each notification information is set so that user U1 can distinguish and recognize the first to nth notification information. By making it possible to selectively notify user U1 of one notification information using one or more of the first to fifth notification formats, information can be transmitted (notification of the ith notification information) in an appropriate notification format according to user U1's situation and the content to be conveyed. Note that here we are focusing only on the first to fifth notification formats, but other notification formats (for example, notification formats that affect the sense of smell) may be included in the first to mth notification formats.

[0051] [Notification Parameters; Figure 12] Among the first to nth notification information, the notification information that is actually notified to user U1 is referred to as the target information. The manner in which the target information is notified to user U1 during the information notification process (hereinafter may be referred to as the notification manner) is represented by the notification parameters. The notification parameters are set by the controller 11. The notification parameters consist of multiple parameters. Figure 12 shows the structure of the notification parameters. The notification parameters are parameter PR Ａ１ ~PR Ａ４ PR Ｂ１ ~PR Ｂ３, PR Ｃ１ ~PR Ｃ２ and PR Ｄ１ It has. Each parameter that constitutes the notification parameter is classified into any one of a visual-related parameter, an auditory-related parameter, a tactile-related parameter, and a common-related parameter. The parameter PR Ａ１ ~PR Ａ４ belongs to the visual-related parameter. The parameter PR<000001​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​=5”, “PR Ａ１ =4”, “PR Ａ１ =2”, “PR Ａ１ When = 1”, the notification image will be displayed at display sizes 1.5 times, 1.2 times, 0.8 times, and 0.5 times the standard display size, respectively. See the parameter PR below. Ａ２ PR Ａ４ PR Ｂ１ ~PR Ｂ３ PR Ｃ１ ~PR Ｃ２ and PR Ｄ１ The same can be said for the same thing.

[0054] Parameter PR Ａ２ This parameter specifies the contrast of the notification image on the display screen 51a, and has one of several discrete values. Here, as an example, parameter PR Ａ２ The value of shall be set to 5, 4, 3, 2, or 1, and parameter PR Ａ２ As the value of increases, the contrast of the notification image will increase. The contrast of the notification image can be understood as referring to the brightness of the notification image on the display screen 51a.

[0055] Parameter PR Ａ３ This parameter specifies whether or not the notification image on the display screen 51a flashes and the flashing pattern. Here, as an example, parameter PR Ａ３ The value of shall be set to 3, 2, or 1. Parameter PR of “3” Ａ３ This corresponds to no blinking, and parameter PR is "2". Ａ３ This corresponds to 0.5 seconds, and the parameter PR is "1". Ａ３ This corresponds to 0.3 seconds. Parameter PR Ａ３ If the value is "3", the notification image will not flash on the display screen 51a (i.e., the notification image will be displayed continuously for a predetermined display time). Parameter PR Ａ３ If the value of is "2" or "1", the notification image on the display screen 51a will blink (i.e., the state in which the notification image is displayed and the state in which the notification image is hidden will occur alternately). At this time, the parameter PR Ａ３ If the value is "2", the blinking period in the notification image is 0.5 seconds, and the parameter PRＡ３ If the value is "1", the blinking period in the notification image is 0.3 seconds.

[0056] Parameter PR Ａ４ This parameter specifies the display time of the notification image on the display screen 51a, and has one of several discrete values. Here, as an example, parameter PR Ａ４ The value of shall be set to 5, 4, 3, 2, or 1, and parameter PR Ａ４ As the value of increases, the display time of the notification image will also increase. The display time of the notification image refers to the length of time the notification image is displayed.

[0057] Auditory parameters are (therefore parameter PR Ｂ１ ~PR Ｂ３ (a) is a parameter that is set and enabled when the target information is notified to user U1 using a third or fourth notification format that acts on the hearing of user U1. The hearing-related parameter represents the notification format when the target information is notified to user U1 using the third or fourth notification format. When the target information is not notified to user U1 using the third or fourth notification format, the hearing-related parameter is not set (it may be set as a variation, but even if set, it will be invalid). For convenience, the sound (sound of words or other sounds) output from speaker 52 to notify user U1 of the target information using the third or fourth notification format is referred to as the notification sound.

[0058] Parameter PR Ｂ１ This parameter specifies the volume of the notification sound and can take one of several discrete values. Here, as an example, the parameter PR Ｂ１ The value of shall be set to 5, 4, 3, 2, or 1, and parameter PR Ｂ１ The volume of the notification sound will increase as the value of increases. Parameter PR Ｂ２ This parameter specifies the frequency of the notification sound and can have one of several discrete values. Here, as an example, the parameter PR Ｂ２ The value of shall be set to 5, 4, 3, 2, or 1, and parameter PR Ｂ２ As the value of increases, the frequency of the notification sound will increase. Parameter PR Ｂ３This parameter specifies the stimulus length of the notification sound and can have one of several discrete values. Here, as an example, the parameter PR Ｂ３ The value of shall be set to 5, 4, 3, 2, or 1, and parameter PR Ｂ３ As the value of increases, the stimulated duration of the notification sound will increase. The stimulated duration of the notification sound refers to the length of time the notification sound is output from speaker 52.

[0059] Tactile-related parameters (therefore parameter PR Ｃ１ ~PR Ｃ２ (a) is a parameter that is set and enabled when target information is notified to user U1 using a fifth notification format that acts on the user U1's sense of touch. The tactile-related parameter represents the notification format when target information is notified to user U1 using the fifth notification format. When target information is not notified to user U1 using the fifth notification format, the tactile-related parameter is not set (it may be set as a variation, but even if set, it will be invalid). For convenience, the vibration output from the vibration device 53 to notify user U1 of target information using the fifth notification format (i.e., the vibration generated by the vibration device 53) is referred to as the notification vibration.

[0060] Parameter PR Ｃ１ This parameter specifies the intensity of the notification vibration and has one of several discrete values. Here, as an example, the parameter PR Ｃ１ The value of shall be set to 5, 4, 3, 2, or 1, and parameter PR Ｃ１ As the value of increases, the intensity of the notification vibration increases. The intensity of the notification vibration can be adjusted by adjusting the vibration amplitude of the oscillator that generates the notification vibration. The intensity of the notification vibration may be referred to as vibration intensity below. Parameter PR Ｃ２ This parameter specifies the frequency of the notification vibration and can have one of several discrete values. Here, as an example, the parameter PR Ｃ２ The value of shall be set to 5, 4, 3, 2, or 1, and parameter PR Ｃ２ As the value of increases, the frequency of the notification vibration will increase.

[0061] Common related parameters are (therefore parameter PRＤ１ (a) is a parameter that is always set and enabled, regardless of which of the first to fifth notification formats is used to notify user U1 of the target information.

[0062] Parameter PR Ｄ１ This parameter specifies the timing for initiating notifications of the target information and can have one of several discrete values. Here, as an example, the parameter PR Ｄ１ The value of shall be set to 5, 4, 3, 2, or 1, and parameter PR Ｄ１ As the value of increases, the notification start timing will be delayed. Parameter PR Ｄ１ Depending on the value of this parameter, the multiplier for the actual notification start time of the target information, relative to the reference start time, is determined.

[0063] For example, suppose the notification management unit F1 (see Figure 13), described later, decides to notify user U1 of the i-th notification information as target information 1.0 seconds after a certain attention time. In this case, 1.0 seconds after the attention time is the reference start timing. In this case, “PR Ｄ１ When = 3”, the controller 11 starts notifying the target information 1.0 second after the attention time. In contrast, “PR Ｄ１ =5”, “PR Ｄ１ =4”, “PR Ｄ１ =2”, “PR Ｄ１ When = 1”, the controller 11 starts notifying the target information 1.5 seconds, 1.2 seconds, 0.8 seconds, and 0.5 seconds after the attention time, respectively. Starting to notify the target information means starting to display a notification image, output a notification sound, or output a notification vibration.

[0064] Alternatively, for example, suppose the notification management unit F1 (see Figure 13), described later, decides to notify user U1 of left-turn guidance information as target information at a reference start timing. Here, the reference start timing is the timing when the distance between vehicle V1 and the left-turn point is 300m. The left-turn point refers to the intersection where the navigation system instructs vehicle V1 to turn left. In this case, “PR Ｄ１When = 3”, the controller 11 starts notifying the target information (left turn guidance information) when the distance between the vehicle V1 and the left turn point becomes 300m. In response to this, “PR Ｄ１ =5”, “PR Ｄ１ =4”, “PR Ｄ１ =2”, “PR Ｄ１ When the value is 1, the controller 11 starts notifying the target information (left-turn guidance information) at the timing when the above distance is 450m, 360m, 240m, and 150m, respectively.

[0065] The notification parameters include parameters for visual transmission intensity, auditory transmission intensity, and tactile transmission intensity. The visual transmission intensity parameter represents the intensity of transmission when target information is transmitted to user U1 through the user U1's vision, and at least parameter PR Ａ１ This corresponds to the visual transmission intensity parameter. The auditory transmission intensity parameter is a parameter that represents the intensity of transmission when target information is transmitted to user U1 through the user U1's hearing, and at least parameter PR Ｂ１ This corresponds to the parameter of auditory transmission intensity. The parameter of tactile transmission intensity is a parameter that represents the intensity of transmission when target information is transmitted to user U1 through the user U1's sense of touch, and at least parameter PR Ｃ１ This corresponds to a parameter of tactile transmission intensity.

[0066] Note that while examples were given here where the values ​​of each parameter constituting the notification parameters are set to be variable in three or five stages, the number of variable stages for those values ​​is arbitrary.

[0067] [Functional Blocks and Input Data Din for Notification Parameter Setting] Figure 13 shows a functional block diagram of the controller 11 involved in information notification processing. The controller 11 is provided with functional blocks F1 to F6. The controller 11 (arithmetic processing unit 11a) may execute a program recorded in the memory 12, recording medium 14, or any other recording medium (not shown) to realize each function of functional blocks F1 to F6. Functional block F1 is the notification parameter setting unit. Functional block F2 is the information acquisition unit. Functional block F3 is the notification management unit. Functional block F4 is the priority setting unit. Functional block F5 is the notification format determination unit. Functional block F6 is the output control unit.

[0068] The notification parameter setting unit F1 sets the above-mentioned notification parameters based on the input data Din. In the following, the notification parameters set by the notification parameter setting unit F1 will be referred to as notification parameters Pout as appropriate.

[0069] Figure 14 shows the structure of input data Din. Input data Din consists of multiple data related to multiple input items. In this embodiment, it is assumed that there are first to twenty-third input items as the multiple input items. That is, input data Din includes data from the first to twenty-third input items. Of the data from the first to twenty-third input items, some data is generated and acquired by the information acquisition unit F2 based on sensing information from the sensing unit 40 (see Figure 5) or user operation information from the operation input unit 15 (see Figure 4).

[0070] The data in the first input field specifies which notification format to use to notify the target information. Notification format information containing the data in the first input field is generated by the notification format determination unit F5. Based on the data in the first input field (and therefore the notification format information), it is specified for each notification format whether or not to use the first to fifth notification formats to notify the target information.

[0071] When the data of the second input item notifies the target information in the first or second notification format, it specifies the display position of the target information. The display position of the target information is the display position of a notification image (character image or graphic image). The data of the second input item is generated by the notification format determination unit F5.

[0072] The data of the third input item represents the priority of the target information when notifying the target information. The priority represents the degree of urgency of the notification, so the priority may be read as the urgency. The data of the third input item is generated by the priority setting unit F4. In the data of the third input item, the priority is quantized into L ３ levels. That is, the data of the third input item has an integer value of 1 or more and L ３ or less, and the larger the value of the data of the third input item, the higher the priority. L ３ has an arbitrary integer value of 2 or more. For example, "L ３ = 5".

[0073] The data of the fourth input item is data for specifying the content of the target information. The data of the fourth input item is generated by the notification management unit F3. The data of the fourth input item has an integer value of 1 or more and L ４ or less, and it is specified which of the first to n notification information the target information is based on the data of the fourth input item. L ４ has an arbitrary integer value of 2 or more. Since the number of types of target information is n in this embodiment, "L ４ = n" may be used.

[0074] The data of the fifth input item is data corresponding to TTC. TTC is an abbreviation of "Time to Collision". TTC is the time TTC1 required for the vehicle V1 to reach the target object, or the time TTC2 required for the vehicle V1 to reach the target position. The target object is a three-dimensional object existing in the traveling direction of the vehicle V1. For example, in a situation where there is a concern about a collision between the vehicle V1 and a vehicle in front of the vehicle V1, the vehicle in front is the target object. The information acquisition unit F2 can estimate and acquire the time TTC1 based on the relative speed between the target object and the vehicle V1 and the distance between the target object and the vehicle V1. The target position is a position existing in the traveling direction of the vehicle V1. For example, in a situation where the left-turn guidance information is the target information, the left-turn point is the target position. The information acquisition unit F2 can estimate and acquire the time TTC2 based on the speed of the vehicle V1 and the distance between the target position and the vehicle V1. Note that TTC may be derived by ADAS. ADAS is an "Advanced Driver Assistance System" mounted on the vehicle V1. In the data of the fifth input item, TTC is quantized into L ５ levels. That is, the data of the fifth input item has an integer value of 1 or more and L ５ or less, and the larger the value of the data of the fifth input item, the longer the TTC. L ５ has an arbitrary integer value of 2 or more. For example, "L ５ = 5".

[0075] The data of the sixth input item represents the visibility outside the vehicle. The visibility outside the vehicle represents the height of the user U1's visibility of the situation outside the vehicle. The visibility outside the vehicle depends on the illuminance outside the vehicle and the presence or absence of rainfall outside the vehicle, etc. Therefore, the information acquisition unit F2 can estimate the visibility outside the vehicle based on the illuminance information and the rainfall information (see FIG. 5), etc., and generate and acquire the data of the sixth input item. Note that the visibility outside the vehicle may be detected by a drive recorder. The drive recorder is mounted on the vehicle V1. In the data of the sixth input item, the visibility outside the vehicle is quantized into L ６ levels. That is, the data of the sixth input item has an integer value of 1 or more and L ６ or less, and the larger the value of the data of the sixth input item, the higher the visibility outside the vehicle. L ６ has an arbitrary integer value of 2 or more. For example, "L６ = 4''.

[0076] The data for the seventh input item represents the interior brightness of the vehicle. Interior brightness represents the brightness (illuminance inside the vehicle) of the vehicle V1. The information acquisition unit F2 can generate and acquire data for the seventh input item based on illuminance information (see Figure 5). Note that the interior brightness may also be detected by the DMS. DMS is an abbreviation for the driver monitoring system installed in vehicle V1. In the data for the seventh input item, the interior brightness is L ７ It is quantized in stages. That is, the data of the seventh input item is 1 or greater than L. ７ The following integer values ​​are used, and the larger the value of the data in the 7th input item, the brighter the interior of the car. L ７ It can be any integer value greater than or equal to 2, for example, "L ７ = 4''.

[0077] The data for the eighth input item indicates whether or not vehicle V1 is stationary. The information acquisition unit F2 can generate the data for the eighth input item by determining whether or not vehicle V1 is stationary based on vehicle speed information (see Figure 5). Alternatively, whether or not vehicle V1 is stationary may be detected by the ADAS installed in vehicle V1. The data for the eighth input item is 1 or greater, L. ８ The following integer values ​​are available: ８ = 2". A data value of "1" in the eighth input item indicates that vehicle V1 is not stopped, and a data value of "2" in the eighth input item indicates that vehicle V1 is stopped. However, the data for the eighth input item may actually be represented as "0" or "1".

[0078] The data for the ninth input item indicates the level of in-vehicle noise. The level of in-vehicle noise represents the noise level inside the vehicle V1. The information acquisition unit F2 can detect and acquire the level of in-vehicle noise based on the in-vehicle microphone signal (see Figure 5). In the data for the ninth input item, the level of in-vehicle noise is L. ９ It is quantized in stages. That is, the data of the ninth input item is 1 or greater than L. ９ The following integer values ​​are used, and the larger the value of the data in the 9th input item, the higher the level of noise inside the vehicle. L ９ It can be any integer value greater than or equal to 2, for example, "L ９ = 5".

[0079] The data in the 10th input item indicates the magnitude of vehicle vibration. The magnitude of vehicle vibration represents the vibration level of vehicle V1 and may be the average intensity of the vibration of vehicle V1 over a certain period of time. The information acquisition unit F2 can detect and acquire the magnitude of vehicle vibration based on acceleration information (see Figure 5). In the data of the 10th input item, the magnitude of vehicle vibration is L １０ It is quantized in stages. That is, the data of the 10th input item is 1 or greater than L. １０ The following integer values ​​are used, and the larger the value of the data in the 10th input item, the greater the magnitude of vehicle vibration. L １０ It can be any integer value greater than or equal to 2, for example, "L １０ = 5".

[0080] The data for the 11th input item is data corresponding to the gaze of user U1. The data for the 11th input item has a value corresponding to the degree to which user U1's gaze is directed toward the display screen 51a or away from the display screen 51a. The information acquisition unit F2 can generate the data for the 11th input item by detecting the direction of user U1's gaze from the in-vehicle camera image. It is assumed that the information acquisition unit F2 already knows the information necessary to generate the data for the 11th input item from the in-vehicle camera image, such as the positional relationship between seat ST1, in-vehicle camera 42 and display screen 51a, and the direction of the optical axis of in-vehicle camera 42. It is also possible that user U1's gaze is detected by a DMS installed in vehicle V1. In the data for the 11th input item, user U1's gaze is L １１ It is quantized in stages. That is, the data of the 11th input item is 1 or greater than L. １１ The following integer values ​​are used, and the larger the value of the data in the 11th input item, the greater the degree to which user U1's gaze deviates from the display screen 51a. １１ It can be any integer value greater than or equal to 2, for example, "L １１ = 4''.

[0081] The data for input items 12, 13, 14, 15, 17, and 20 represent user U1's visual acuity, hearing, height, weight, age, and gender, respectively. User U1's visual acuity, hearing, height, weight, age, and gender are identified by the personal information self-reported by user U1. User U1 can input their personal information into the in-vehicle device 10 through the operation input unit 15, and based on the user operation information including said personal information, the information acquisition unit F2 may generate and acquire the data for input items 12 to 15, 17, and 20. Alternatively, the data for input items 12 to 15, 17, and 20 may be acquired by the information acquisition unit F2 when user U1's personal information registered in the user terminal device is provided to the in-vehicle device 10. In the data for input item 12, user U1's visual acuity is L １２ It is quantized in stages. That is, the data of the 12th input item is 1 or greater than L. １２ The following integer values ​​are used, and the larger the value of the data in the 12th input item, the higher the eyesight of user U1. １２ It can be any integer value greater than or equal to 2, for example, "L １２ = 5". In the data for the 13th input item, user U1's hearing is L １３ It is quantized in stages. That is, the data of the 13th input item is 1 or greater than L. １３ The following integer values ​​are used, and the larger the value of the data in the 13th input item, the higher the hearing of user U1. １３ It can be any integer value greater than or equal to 2, for example, "L １３ = 5". In the data for the 14th input item, the height of user U1 is L １４ It is quantized in stages. That is, the data of the 14th input item is 1 or greater than L. １４ The following integer values ​​are used, and the larger the value of the data in the 14th input item, the taller user U1 is assumed to be. １４ It can be any integer value greater than or equal to 2, for example, "L １４ = 7". In the data for the 15th input item, the weight of user U1 is L １５ It is quantized in stages. That is, the data of the 15th input item is 1 or greater than L. １５ The following integer values ​​are used, and the larger the value of the data in the 15th input item, the greater the weight of user U1. １５It can be any integer value greater than or equal to 2, for example, "L １５ = 7". In the data for the 17th input item, the age of user U1 is L １７ It is quantized in stages. That is, the data of the 17th input item is 1 or greater than L. １７ The following integer values ​​are used, and the larger the value of the data in the 17th input item, the older the user U1 is considered to be. １７ It can be any integer value greater than or equal to 2, for example, "L １７ = 7". The data for the 20th input item is 1 or greater. ２０ The following integer values ​​are used, and the value of the data in the 20th input item represents the gender of user U1. ２０ = 2”, but “L ２０ It can be expressed as =3".

[0082] The data for the 16th input item is data corresponding to the thickness of the clothing worn by user U1. The information acquisition unit F2 can generate the data for the 16th input item by estimating the thickness of the clothing worn by user U1 from the in-vehicle camera image. Alternatively, the thickness of the clothing worn by user U1 may be detected by the DMS installed in the vehicle V1. In the data for the 16th input item, the thickness of the clothing worn by user U1 is L. １６ It is quantized in stages. That is, the data of the 16th input item is 1 or greater than L. １６ The following integer values ​​are used, and the larger the value of the data in the 16th input item, the thicker the clothing worn by user U1 becomes. １６ It can be any integer value greater than or equal to 2, for example, "L １６ = 5".

[0083] The data for the 18th input item is data corresponding to the emotions of user U1. The information acquisition unit F2 can estimate user U1's emotions based on user U1's biometric information, user U1's facial image in the in-vehicle camera image, or user U1's speech content included in the in-vehicle microphone signal, or a combination thereof. The information acquisition unit F2 can generate the data for the 18th input item by quantifying the estimated emotions of user U1. Note that user U1's emotions may also be estimated by the DMS installed in vehicle V1. In the data for the 18th input item, user U1's emotions are L １８It is quantized in stages. That is, the data of the 18th input item is 1 or greater than L. １８ The following integer values ​​are used, and the value of the data in the 18th input item represents the emotions of user U1. １８ It can be any integer value greater than or equal to 2, for example, "L １８ = 3". A known method can be used as a method for estimating emotions based on biometric information, for example, the method proposed by the applicant of this application in International Publication No. 2023 / 127930 may be used. As the emotions of user U1, the degree of discomfort and the degree of anxiety felt by user U1 may be estimated individually, in which case the data of the 18th input item will be two-dimensional data consisting of a numerical value indicating the degree of discomfort and a numerical value indicating the degree of anxiety.

[0084] The data for the 19th input item represents the continuous driving time of vehicle V1 by user U1. The continuous driving time of vehicle V1 represents the length of time that user U1 continuously drives vehicle V1, and in detail, for example, it may be the continuous time that user U1 wears a seat belt while seated in user U1's seat ST1. The information acquisition unit F2 can detect the continuous driving time of vehicle V1 by measuring the continuous seat belt wearing time or based on the in-vehicle camera image, and can generate and acquire the data for the 19th input item from the detection result. In the data for the 19th input item, the continuous driving time is L １９ It is quantized in stages. That is, the data of the 19th input item is 1 or greater than L. １９ The following integer values ​​are used, and the larger the value of the data in the 19th input item, the longer the continuous operation time will be. １９ It can be any integer value greater than or equal to 2, for example, "L １９ = 4''.

[0085] The data for the 21st input item indicates the presence or absence of passengers. Passengers refer to occupants of vehicle V1 other than user U1. The information acquisition unit F2 can generate the data for the 21st input item by determining the presence or absence of passengers based on in-vehicle camera images or by using the function of a seatbelt reminder installed in vehicle V1. Alternatively, the presence or absence of passengers may be detected by a DMS (or passenger monitoring system) installed in vehicle V1. The data for the 21st input item is 1 or more L ２１The following integer values ​​are available: ２１ = 2". A value of "1" in the 21st input field indicates that there are no passengers, and a value of "2" in the 21st input field indicates that there are passengers. However, the data for the 21st input field may actually be represented as "0" or "1".

[0086] The data in input item 22 represents the safe driving score. The safe driving score represents the degree of safety of the driving performed by user U1, and the higher the degree of safety, the higher the value of the data in input item 22. In the data for input item 22, the degree of safety is L ２２ It is quantized in stages. That is, the data of the 22nd input item is 1 or greater than L. ２２ It has the following integer value: L ２２ It can be any integer value greater than or equal to 2, for example, "L ２２ = 4". The information acquisition unit F2 detects the frequency of sudden braking, sudden steering, sudden acceleration, sudden deceleration, and speeding when user U1 is driving vehicle V1, based on sensing information, and can derive a safe driving score based on these detection results. The safe driving score may be derived by the navigation system installed in vehicle V1, or it may be derived and managed by the user terminal device.

[0087] The data in input item 23 indicates driver skill. Driver skill represents the degree of user U1's driving proficiency (driving skill level), and the higher the degree of proficiency, the higher the value of the data in input item 23. In the data for input item 23, the degree of proficiency is L ２３ It is quantized in stages. That is, the data of the 23rd input item is 1 or greater than L. ２３ It has the following integer value: L ２３ It can be any integer value greater than or equal to 2, for example, "L ２３= 4". The above safe driving score may also be used as a driver skill. The information acquisition unit F2 may identify the driver skill based on user operation information self-reported by user U1 (such as the number of years of driving experience of user U1 or the total distance driven). The information acquisition unit F2 may also identify the driver skill based on the number of years elapsed since user U1 obtained their driver's license.

[0088] Some of the input fields from 1 to 23 are classified into several categories. Input field 6 is classified into a category representing external vehicle conditions. Input fields 7, 9, 10, and 21 are classified into a category representing the interior vehicle environment. Input fields 11 to 20, 22, and 23 are classified into a category representing user information (user's personal characteristics information).

[0089] Individuals' sensitivity to notifications varies. User U1's sensitivity is their reaction to notifications of target information and represents how easily they recognize notifications. User U1's sensitivity includes visual sensitivity, auditory sensitivity, and tactile sensitivity. Visual sensitivity represents User U1's reaction to notifications when target information is presented to User U1 in a notification format that acts on User U1's vision (i.e., the first or second notification format). Auditory sensitivity represents User U1's reaction to notifications when target information is presented to User U1 in a notification format that acts on User U1's hearing (i.e., the third or fourth notification format). Tactile sensitivity represents User U1's reaction to notifications when target information is presented to User U1 in a notification format that acts on User U1's sense of touch (i.e., the fifth notification format).

[0090] Some of the input items 1 through 23 are particularly related to the user U1's vision, hearing, or touch. Specifically, the data in input items 6, 7, 11, 12, and 17 are first-order sensitivity-related information that affects visual sensitivity. This first-order sensitivity-related information affects how easily the user U1 recognizes the notification when the target information is notified in the first or second notification format. The data in input items 9, 13, and 17 are second-order sensitivity-related information that affects auditory sensitivity. This second-order sensitivity-related information affects how easily the user U1 recognizes the notification when the target information is notified in the third or fourth notification format. The data in input items 10 and 14 through 17 are third-order sensitivity-related information that affects tactile sensitivity. This third-order sensitivity-related information affects how easily the user U1 recognizes the notification when the target information is notified in the fifth notification format. Based on the input data Din having the characteristics described above, the notification parameter setting unit F1 sets an appropriate notification parameter Pot, taking into consideration how easily the user U1 recognizes the notification (see Figures 13 and 14).

[0091] Let's return to the explanation of each functional block in Figure 13. The information acquisition unit F2 generates and acquires the information group DD based on sensing information, user operation information, and other information. The other information here is provided to the in-vehicle device 10 from ADAS or DMS, from a user terminal device, or from an external device connected to an external network (such as a server device located outside the vehicle). The information supplied to the in-vehicle device 10 from the external device includes traffic congestion information, weather forecast information, and information from broadcast waves, and includes any information that the external device can acquire on the external network. Each data in the information group DD is shared among the functional blocks in the controller 11.

[0092] The information group DD includes data from the 5th to 23rd input items (see Figure 14). The information group DD may also include various other data besides the data from the 5th to 23rd input items. For example, data showing the distance and positional relationship between vehicle V1 and any obstacle located around vehicle V1 may be included in the information group DD. Also, for example, data showing the distance and positional relationship between vehicle V1 and any point on the map (such as the left-turn point mentioned above) may be included in the information group DD. In addition, any data that should be included in the notification determination data, priority determination data, or format determination data described later is acquired by the information acquisition unit F2 and included in the information group DD.

[0093] Notification determination data is input to the notification management unit F3. Based on the notification determination data, the notification management unit F3 performs a notification determination process to determine whether or not to send any notification to user U1 according to a predetermined algorithm. The notification determination data includes all or part of the information group DD.

[0094] When the notification management unit F3 determines that it will issue a notification for any of the first to nth notification information described above, the notification information that has been determined to be issued is set as the target information. The content of the target information is transmitted from the notification management unit F3 to the priority setting unit F4, the notification format determination unit F5, and the notification parameter setting unit F1. In some cases, emergency notification information may be input to the notification management unit F3 from ADAS or the like. In this case, an emergency notification command is output from the notification management unit F3 to the notification parameter setting unit F1, instructing that the emergency notification information be immediately notified to user U1. The emergency notification information is information that should be notified to user U1 with extremely high urgency. When the notification parameter setting unit F1 receives the emergency notification command, it immediately controls the output control unit F6 so that the emergency notification information is notified to user U1 in a pre-set notification format and notification parameters. The emergency notification information is separate from the target information, and in this embodiment, the existence of the emergency notification information and the emergency notification command will be ignored below.

[0095] Target information is input to the priority setting unit F4 (information identifying whether the target information is one of the first to nth notification information is input). Priority determination data is also input to the priority setting unit F4. The priority setting unit F4 performs priority setting processing to estimate and set the priority of the target information based on the priority determination data. The priority setting unit F4 may be composed of a priority determination AI that estimates the priority of the target information based on the priority determination data. In this embodiment, AI refers to artificial intelligence formed through machine learning. However, the priority setting unit F4 may also estimate and set the priority of the target information in a rule-based manner based on the priority determination data. As described above, priority represents the priority (in other words, urgency) of the target information when notifying the target information. Priority determination data includes all or part of the information group DD.

[0096] Priority determination data may include data specifying the priority relationship between the first to nth notification information. Based on this priority relationship data, for example, forward obstacle information may be assigned a higher priority than left-turn guidance information (see Figures 10(a) to (e) and 11(a) to (e)), and left-turn guidance information may be assigned a higher priority than entertainment-related information. Entertainment-related information is information related to entertainment that does not involve the driving operation of vehicle V1. For example, information that guides viewers to recommended broadcast programs and information that introduces stores located near the vehicle V1's current location fall under the category of entertainment-related information.

[0097] When Time to Traffic (TTC) is defined in the notification of target information, the priority setting unit F4 can estimate and set the priority of the target information based on the TTC. For example, in a situation where there is concern about a collision between vehicle V1 and the vehicle in front of vehicle V1, if forward obstacle information is set as target information, the priority setting unit F4 can estimate and set the priority of the target information based on the time TTC1 required for vehicle V1 to reach the target object. Also, for example, in a situation where left-turn guidance information is designated as target information, the priority setting unit F4 can estimate and set the priority of the target information based on the time TTC2 required for vehicle V1 to reach the left-turn point.

[0098] Of the first to nth notification information, the priority corresponding to notification information unrelated to TTC may be a predetermined, unchangeable priority. That is, for example, if the target information represents the information "Caution: Icy Road Surface," then the priority set by the priority setting unit F4 may estimate and set a predetermined priority for the information "Caution: Icy Road Surface." The priority estimated and set by the priority setting unit F4 is transmitted to the notification format determination unit F5 and the notification parameter setting unit F1.

[0099] The notification format determination unit F5 performs notification format determination processing based on the priority estimated and set by the priority setting unit F4, the target information set by the notification management unit F3, and the format determination data. In the notification format determination processing, the notification format determination unit F5 determines, according to a predetermined algorithm, which notification format to use to notify the user U1 of the target information, and generates notification format information indicating the determined content. The notification format information is transmitted to the notification parameter setting unit F1. In the notification format determination processing, it is individually determined whether or not to notify the target information using the first notification format, the second notification format, the third notification format, the fourth notification format, and the fifth notification format.

[0100] Hereinafter, when the i-th notification format is ON, it means that it has been decided to notify the target information using the i-th notification format, and when the i-th notification format is OFF, it means that it has been decided not to notify the target information using the i-th notification format (where i is an integer between 1 and 5). Only when the i-th notification format is ON, the output control unit F6 and the information output unit 50 work together to notify the user U1 of the i-th notification format. In the notification format determination process, ON or OFF is set individually for the first to fifth notification formats, and the results of these settings are included in the notification format information. The notification format determination unit F5 can also set two or more of the first to fifth notification formats to ON.

[0101] The format determination data includes all or part of the information group DD. Notification history data indicating which notification information was notified to user U1 in the past using which notification format may be stored in the recording medium 14, and the notification history data may be included in the format determination data. The information acquisition unit F2 can estimate the degree of familiarity with the i-th notification information being notified in a specific notification format based on the notification history data as the notification learning level, and the notification learning level may be included in the format determination data. The notification learning level may also be included in the input data Din.

[0102] When the notification format determination unit F5 sets the first or second notification format to ON during the notification format determination process, it also determines the display position of the target information on the display screen 51a. The determined display position is transmitted to the notification parameter setting unit F1 and the output control unit F6. The display position of the target information is determined according to the content of the target information (depending on whether the target information is the first to nth notification information). When the notification format determination unit F5 sets the third or fourth notification format to ON during the notification format determination process, it also determines the audible position of the target information. The determined audible position is transmitted to the output control unit F6. The audible position of the target information is determined according to the content of the target information (depending on whether the target information is the first to nth notification information). The audible position of the target information refers to the position of the sound source that outputs the target information as sound. The speaker 52 can be configured with multiple speakers installed at multiple locations within the vehicle V1. In this process, the output control unit F6 can set the acoustic position of the target information by controlling which of the multiple speakers outputs the acoustic signal for the target information. When the notification format determination unit F5 sets the fifth notification format to ON during the notification format determination process, it also determines the vibration position for notifying the target information. The determined vibration position is transmitted to the output control unit F6. The vibration position is determined according to the content of the target information (depending on whether the target information is the first to the nth notification information). Based on the determination of the vibration position, it is determined for each vibrator whether to vibrate the left vibrator, the center vibrator, and the right vibrator in the notification of the target information.

[0103] Input data Din is supplied to the notification parameter setting unit F1. The notification parameter setting unit F1 performs parameter setting processing to generate and set notification parameters Pout based on the input data Din. The data of the first input item in input data Din is represented by the notification format information from the notification format determination unit F5 (see Figure 14). The data of the second input item in input data Din is represented by the display position determined by the notification format determination unit F5 (see Figure 14). The data of the third input item in input data Din is represented by the priority estimated and set by the priority setting unit F4 (see Figure 14). The data of the fourth input item in input data Din is represented by the content of the target information set by the notification management unit F3 (see Figure 14). The data of the 5th to 23rd input items included in the information group DD is supplied to the notification parameter setting unit F1 as part of input data Din.

[0104] The notification parameter Pout is input to the output control unit F6. The output control unit F6 controls the information output unit 50 so that the target information is notified to the user U1 according to the characteristics of the notification parameter Pout. As a result, the information output unit 50 outputs the target information to the user U1 according to the notification parameter Pout. When the first or second notification format is ON, the output control unit F6 controls the parameter PR in the notification parameter Pout. Ａ１ ~PR Ａ４ The notification image corresponding to the target information is displayed on the display device 51 accordingly (see Figures 6 and 12). In this case, the display position of the notification image follows the display position determined by the notification format determination unit F5. When the third or fourth notification format is ON, the output control unit F6 determines the parameter PR in the notification parameter Out. Ｂ１ ~PR Ｂ３ Accordingly, a notification sound corresponding to the target information is output from speaker 52 (see Figures 6 and 12). In this case, the acoustic position of the notification sound follows the acoustic position determined by notification format determination unit F5. When the fifth notification format is ON, the output control unit F6 outputs the parameter PR in notification parameter Out. Ｃ１ ~PR Ｃ２Accordingly, the vibration device 53 outputs a notification vibration corresponding to the target information (see Figures 6 and 12). At this time, the vibration position of the notification vibration follows the vibration position determined by the notification format determination unit F5. In addition, the output control unit F6 determines the parameter PR in the notification parameter Pot. Ｄ１ The timing of the start of notification of the target information is controlled accordingly. Multiple notification formats may be set to ON. For example, if the first, third, and fifth notification formats are ON, the output control unit F6 controls the parameter PR in the notification parameter Out. Ａ１ ~PR Ａ４ The notification image corresponding to the target information is displayed on the display device 51 accordingly. In addition, when the first, third, and fifth notification formats are ON, the output control unit F6 displays the notification image and, at the same time, the parameter PR in the notification parameter Pot Ｂ１ ~PR Ｂ３ Accordingly, a notification sound corresponding to the target information is output from speaker 52. In addition, when the first, third, and fifth notification formats are ON, the output control unit F6 displays the notification image and outputs the notification sound, and simultaneously outputs the parameter PR in the notification parameter Pot. Ｃ１ ~PR Ｃ２ Accordingly, the vibration device 53 outputs a notification vibration corresponding to the target information.

[0105] User U1's response to receiving notification of the target information may be fed back to the notification parameter setting unit F1. Information indicating User U1's response is referred to as FB information. FB information is an abbreviation for feedback information. The method of using FB information will be described later.

[0106] Here are some examples of the relationship between the input data Din and the notification parameter Pout (see Figures 12 and 14).

[0107] For example, the larger the data for the third input item (i.e., the higher the priority) or the smaller the data for the fifth input item (i.e., the shorter the TTC), the larger the display size of the notification image, the louder the notification sound, or the stronger the vibration of the notification vibration in the notification parameter Out. The larger the data for the third input item (i.e., the higher the priority) or the smaller the data for the fifth input item (i.e., the shorter the TTC), the earlier the notification start timing may be set in the notification parameter Out. Also, for example, the smaller the data for the sixth input item (the worse the visibility from outside the vehicle), the larger the display size of the notification image, the louder the notification sound, or the stronger the vibration of the notification vibration in the notification parameter Out. Also, for example, the smaller the data for the seventh input item (the darker the interior of the vehicle), the higher the contrast of the notification image in the notification parameter Out.

[0108] Furthermore, for example, if the data for the 8th input item indicates that vehicle V1 is stationary, it is easier to set the 1st or 2nd notification format to ON compared to when vehicle V1 is in motion. This is because a visual notification while stationary is less likely to hinder safe driving and is less bothersome to user U1. Also, for example, the larger the data for the 9th input item (in-vehicle noise), the higher the volume of the notification sound in the notification parameter Pout. Also, for example, the larger the data for the 10th input item (vehicle vibration), the higher the vibration intensity of the notification vibration in the notification parameter Pout. Also, for example, if the data for the 11th input item (gaze) indicates that user U1's gaze is away from the display screen 51a, the higher the volume of the notification sound or the vibration intensity of the notification vibration in the notification parameter Pout.

[0109] For example, in the data for the 12th input item, the lower the user U1's eyesight, the more likely it is that the 3rd, 4th, or 5th notification format will be set to ON preferentially over the 1st or 2nd notification format, or the display size and contrast of the notification image will be increased in the notification parameter Pot. For example, in the data for the 13th input item, the lower the user U1's hearing, the more likely it is that the 1st, 2nd, or 5th notification format will be set to ON preferentially over the 3rd or 4th notification format, or the volume of the notification sound will be increased in the notification parameter Pot.

[0110] Furthermore, regarding the data for the 14th and 15th input items, the heavier the user U1's weight is relative to their height, the stronger the vibration intensity of the notification vibration in the notification parameter "Pout" will be increased. The vibration position may be determined considering the user U1's height. Also, regarding the data for the 16th input item, the thicker the clothing worn by user U1, the stronger the vibration intensity of the notification vibration in the notification parameter "Pout" will be increased. For example, regarding the data for the 17th input item, the older the user U1 is, the larger the display size of the notification image, the volume of the notification sound, or the vibration intensity of the notification vibration in the notification parameter "Pout" will be increased.

[0111] Regarding the data for the 18th input item, it is not uniquely determined what type of notification is desirable depending on the user U1's emotions. Through the machine learning performed later for notification parameter estimation, it is expected that the notification parameter Pout will be optimized according to emotions. For example, regarding the data for the 19th input item, if the continuous operation time is long, measures such as modulating the frequency of the notification vibration can be considered to suppress habituation to the vibration. For example, regarding the data for the 20th input item, if user U1 is female, the vibration intensity of the notification vibration in the notification parameter Pout will be weakened compared to when user U1 is male.

[0112] For example, with regard to the data for input item 21, if there is a passenger, the first, second, or fifth notification format will be preferentially set to ON compared to when there is no passenger (in this case, the third and fourth notification formats may be set to OFF). This is because the notification sound is likely to be unnecessary for the passenger. Also, for example, with regard to the data for input item 22 or 23, the higher the safe driving score or driver skill, the smaller the display size of the notification image, the volume of the notification sound, or the vibration intensity of the notification vibration in the notification parameter Pout will be.

[0113] [Notification Parameter Setting Unit F1] Figure 15 shows the internal functional block diagram of the notification parameter setting unit F1. The notification parameter setting unit F1 comprises functional blocks F11 to F14. Functional blocks F11, F12, F13, and F14 are the parameter generation unit, pre-processing unit, post-processing unit, and FB management unit, respectively. The pre-processing unit and post-processing unit can also be read as the pre-correction unit and post-correction unit, respectively.

[0114] Input data Din is input to the pre-processing unit F12. The pre-processing unit F12 generates input data CDin by executing a pre-correction process to correct the input data Din. Input data CDin is the input data Din after correction by the pre-correction process. In the following description, when emphasizing the correction in the pre-correction process, input data CDin may be referred to as corrected input data CDin. Input data CDin has the same structure as input data Din and therefore contains data for the 1st to 23rd input items (see Figure 14). In the pre-correction process, the data of one or more input items from the 1st to 23rd input items in input data Din can be corrected. However, there are cases where correction is not performed in the pre-correction process, in which case input data CDin will be a perfect match to input data Din.

[0115] Input data CDin is input to the parameter generation unit F11. The parameter generation unit F11 generates output parameter Dout from the input data CDin, which is the basis for the notification parameter Pout.

[0116] The output parameter Dout generated by the parameter generation unit F11 is input to the subsequent processing unit F13. The subsequent processing unit F13 generates the notification parameter Pout by executing a subsequent correction process to correct the output parameter Dout. The notification parameter Pout is the output parameter Dout after correction by the subsequent correction process. The output parameter Dout has the same configuration as the notification parameter Pout, and therefore the parameter PR Ａ１ ~PR Ａ４ PR Ｂ１ ~PR Ｂ３ PR Ｃ１ ~PR Ｃ２ and PR Ｄ１ It has (see Figure 12). In the subsequent correction process, the parameter PR in the output parameter Dout Ａ１ ~PR Ａ４ PR Ｂ１ ~PR Ｂ３ PR Ｃ１ ~PR Ｃ２ and PR Ｄ１One or more of these parameters can be corrected. However, in subsequent correction processing, correction may not be performed, in which case the notification parameter Put will exactly match the output parameter Dout.

[0117] Either the pre-processing unit F12 or the post-processing unit F13 may not be provided in the notification parameter setting unit F1. For convenience, a configuration in which both the pre-processing unit F12 and the post-processing unit F13 are provided in the notification parameter setting unit F1 is called a double-sided correction configuration (see Figure 15). In contrast, as shown in Figure 16, a configuration in which the notification parameter setting unit F11 is provided with the pre-processing unit F12 but not the post-processing unit F13 is called a pre-correction configuration. As shown in Figure 17, a configuration in which the notification parameter setting unit F11 is provided with the post-processing unit F13 but not the pre-processing unit F12 is called a post-correction configuration. In all configurations, the parameter generation unit F11 and the FB management unit F14 are provided in the notification parameter setting unit F1.

[0118] In the pre-correction configuration, the output parameter Dout itself always becomes the notification parameter Poout. In the pre-correction configuration, the output parameter Dout and the notification parameter Poout are understood to refer to the same thing. In the post-correction configuration, the input data Din itself is always input to the parameter generation unit F11 as the input data CDin. In the post-correction configuration, the input data Din and the input data CDin are understood to refer to the same thing. In practice, a double-sided correction configuration can be adopted, and the notification parameter setting unit F1 can individually control whether or not the pre-correction process and the post-correction process are executed. Hereinafter, when the pre-correction process is ON, it means that the pre-correction process is executed, and when the pre-correction process is OFF, it means that the pre-correction process is not executed. Similarly, when the post-correction process is ON, it means that the post-correction process is executed, and when the post-correction process is OFF, it means that the post-correction process is not executed. A configuration in which the pre-correction process is ON and the post-correction process is OFF is a pre-correction configuration, and a configuration in which the pre-correction process is OFF and the post-correction process is ON is a post-correction configuration. Furthermore, a double-sided correction configuration when the pre-correction process is OFF is equivalent to a post-correction configuration, and a double-sided correction configuration when the post-correction process is OFF is equivalent to a pre-correction configuration.

[0119] The parameter generation unit F11 is composed of the parameter estimation AI 100. The parameter generation unit F11 can be understood as the parameter estimation AI 100 itself. The parameter estimation AI 100 estimates appropriate notification parameters based on the input data CDin. Hereinafter, the parameter estimation AI 100 will often be abbreviated as estimation AI 100. The estimation by estimation AI 100 is called inference. The notification parameters obtained by inference (i.e., the notification parameters estimated by estimation AI 100) are output from the parameter generation unit F11 as output parameters Dout. The notification parameters obtained by inference (i.e., the notification parameters estimated by estimation AI 100) have the same configuration as the notification parameters Put described above, but if a subsequent processing unit F13 is provided, they can be corrected by subsequent correction processing.

[0120] As described above, the user U1's response to receiving notification of the target information may be fed back to the notification parameter setting unit F1 as FB information. The FB management unit F14 acquires the FB information (feedback information) indicating the user U1's response. The content of the pre-correction process or post-correction process is adjusted and set by the FB management unit F14 according to the acquired FB information. The controller 11 repeatedly performs the information notification process. In each information notification process, the target information is notified to the user U1 through the cooperation of the functional blocks F1 to F6 in Figure 13. Therefore, multiple pieces of target information are sequentially notified to the user U1 within a reasonable time. The FB management unit F14 attempts to acquire FB information each time target information is notified in the information notification process.

[0121] The FB management unit F14 may attempt to acquire FB information using the touch panel on the display device 51. The operation that user U1 inputs to the touch panel is specifically referred to as a touch panel operation. The content of the touch panel operation may be transmitted to the controller 11 as user operation information (see Figure 4). For example, immediately after notification of target information, the FB management unit F14 receives user U1's response to the notification of target information via the touch panel. User U1's response is, in other words, user U1's evaluation (user U1's evaluation of the notification of target information), such as user U1's level of comfort or satisfaction. In practice, this evaluation is converted into data, which is then acquired and processed by the controller 11. The evaluation index is data (e.g., numerical data or text data) that shows the content of user U1's response and evaluation to the notification of target information. FB information corresponds to the evaluation index; in other words, the evaluation index is shown by the FB information. User U1's intentions are expressed in user U1's response. User U1's response, user U1's evaluation, and user U1's intention can be expressed as user response, user evaluation, and user intention, respectively. In the following explanation, the response of user U1 or user response handled by controller 11 can be understood as referring to the evaluation indicator. The same applies to user U1's evaluation, user evaluation, user U1's intention, and user intention.

[0122] If a user response is indicated by a touch panel operation within the predetermined FB reception time, the content of the user response indicated by the touch panel operation is acquired as FB information. For example, the FB management unit F14 displays multiple FB buttons on the display screen 51a, notifies the user of the target information, and then accepts a touch panel operation to select one of the multiple FB buttons within the predetermined FB reception time. The user evaluation is indicated by which FB button is selected by the touch panel operation. The FB buttons on the display screen 51a are button-shaped icons on the display screen 51a. If there is no touch panel operation within the predetermined FB reception time, FB information is not acquired (the FB management unit F14 determines that the acquisition of FB information has failed). The touch panel operation here indicates the user U1's positive or negative response to the notification of the target information. A positive response is a positive evaluation indicating that user U1 was satisfied with the notification of the target information. A negative response is a negative evaluation indicating that user U1 was dissatisfied with the notification of the target information.

[0123] When FB information is acquired, the FB management unit F14 classifies the FB information into positive FB information or negative FB information according to its content. FB information that shows a positive response to notification of certain target information is classified as positive FB information. FB information that shows a negative response to notification of certain target information is classified as negative FB information. In other words, positive FB information is FB information that indicates that user U1 was satisfied with the notification of the target information, and negative FB information is FB information that indicates that user U1 was dissatisfied with the notification of the target information.

[0124] Let's look at a specific example. Referring to Figure 18(a), the case in which the target information 710 is notified is referred to as Case CS1. In Case CS1, the target information 710 is notified using all or part of the first to fifth notification formats. Immediately after the notification of the target information 710, the FB management unit F14 displays the message "How was the notification just now?" along with four FB buttons 711 to 714 on the display screen 51a, as shown in Figure 18(a). Corresponding to the FB buttons 711, 712, 713, and 714, the words "Good," "Too strong," "Too weak," and "Notification unnecessary" are displayed, respectively. The FB management unit F14 displays the FB buttons 711 to 714 only for a predetermined FB reception time and accepts touch panel operations to select one of the FB buttons 711 to 714 within the FB reception time. If a touch panel operation is performed to select FB buttons 711, 712, 713, or 714 within the FB reception time, the content associated with each FB button 711, 712, 713, or 714 will be acquired as FB information 718. If there is no touch panel operation within the FB reception time, FB information 718 will not be acquired.

[0125] A touch panel operation selecting the FB button 711 indicates a positive reaction (evaluation) by user U1 to the notification of the target information 710. Therefore, the FB information 718 acquired when the touch panel operation selecting the FB button 711 occurs is classified as positive FB information, indicating the user's intention that the notification of the target information 710 and the notification method (notification parameter Pout) of the target information 710 are appropriate for user U1. A touch panel operation selecting the FB buttons 712, 713, or 714 indicates a negative reaction (evaluation) by user U1 to the notification of the target information 710. Therefore, the FB information 718 acquired when the touch panel operation selecting the FB buttons 712, 713, or 714 occurs is classified as negative FB information. The FB information 718 acquired when the touch panel operation selecting the FB button 712 occurs indicates the user's intention that the notification of the target information 710 is too strong for user U1 (the display size, volume, or vibration intensity is excessive). When a touch panel operation is performed to select the FB button 713, the FB information 718 obtained indicates that the user U1 considers the notification of the target information 710 to be too weak (display size, volume, or vibration intensity is insufficient). When a touch panel operation is performed to select the FB button 714, the FB information 718 obtained indicates that the user U1 considers notification of the target information 710 unnecessary.

[0126] Referring to Figure 18(b), the case in which the target information 720 is notified is referred to as Case CS2. In Case CS2, the target information 720 is notified using at least one of the first or second notification formats. Immediately after the notification of the target information 720, the FB management unit F14 displays the message "How was the notification?" along with four FB buttons 721 to 724 on the display screen 51a, as shown in Figure 18(b). Corresponding to the FB buttons 721, 722, 723, and 724, the words "Good," "Display too large," "Display too small," and "Notification unnecessary" are displayed, respectively. The FB management unit F14 displays the FB buttons 721 to 724 only for a predetermined FB reception time and accepts touch panel operations to select one of the FB buttons 721 to 724 within the FB reception time. If a touch panel operation is performed to select FB buttons 721, 722, 723, or 724 within the FB reception time, the content associated with each FB button 721, 722, 723, or 724 will be acquired as FB information 728. If there is no touch panel operation within the FB reception time, FB information 728 will not be acquired.

[0127] A touch panel operation selecting the FB button 721 indicates a positive response (evaluation) by user U1 to the notification of the target information 720. Therefore, the FB information 728 acquired when a touch panel operation selecting the FB button 721 occurs is classified as positive FB information, indicating the user's intention that the notification of the target information 720 and the notification method (notification parameter Pout) of the target information 720 are appropriate for user U1. A touch panel operation selecting the FB buttons 722, 723, or 724 indicates a negative response (evaluation) by user U1 to the notification of the target information 720. Therefore, the FB information 728 acquired when a touch panel operation selecting the FB buttons 722, 723, or 724 occurs is classified as negative FB information. When a touch panel operation is performed to select the FB button 722, the FB information 728 obtained is FB information indicating that "the display is too large," indicating the user's intention that the display size of the notification image of the target information 720 is excessive for user U1. When a touch panel operation is performed to select the FB button 723, the FB information 728 obtained is FB information indicating that "the display is too small," indicating the user's intention that the display size of the notification image of the target information 720 is insufficient for user U1. When a touch panel operation is performed to select the FB button 724, the FB information 728 obtained is FB information indicating the user's intention that notification of the target information 720 is unnecessary for user U1.

[0128] Referring to Figure 18(c), the case in which the target information 730 is notified is referred to as Case CS3. In Case CS3, the target information 730 is notified using at least one of the third or fourth notification formats. Immediately after the notification of the target information 730, the FB management unit F14 displays the message "How was the notification?" along with four FB buttons 731 to 734 on the display screen 51a, as shown in Figure 18(c). Corresponding to the FB buttons 731, 732, 733, and 734, the words "Good," "Too loud," "Too quiet," and "No notification needed" are displayed, respectively. The FB management unit F14 displays the FB buttons 731 to 734 only for a predetermined FB reception time and accepts touch panel operations to select one of the FB buttons 731 to 734 within the FB reception time. If a touch panel operation is performed to select FB buttons 731, 732, 733, or 734 within the FB reception time, the content associated with each FB button 731, 732, 733, or 734 will be acquired as FB information 738. If there is no touch panel operation within the FB reception time, FB information 738 will not be acquired.

[0129] A touch panel operation selecting the FB button 731 indicates a positive reaction (evaluation) by user U1 to the notification of the target information 730. Therefore, the FB information 738 acquired when the FB button 731 is selected is classified as positive FB information, indicating the user's intention that the target information 730 is notified and that the notification method (notification parameter Pout) of the target information 730 is appropriate for user U1. A touch panel operation selecting the FB buttons 732, 733, or 734 indicates a negative reaction (evaluation) by user U1 to the notification of the target information 730. Therefore, the FB information 738 acquired when the FB button 732, 733, or 734 is selected is classified as negative FB information. The FB information 738 acquired when the FB button 732 is selected is FB information stating that "the sound is too loud," indicating the user's intention that the notification sound of the target information 730 is excessive for user U1. When a touch panel operation is performed to select the FB button 733, the FB information 738 obtained is FB information indicating that "the sound is too quiet," indicating the user U1's intention that the notification sound for the target information 730 is too quiet. When a touch panel operation is performed to select the FB button 734, the FB information 738 obtained is FB information indicating the user U1's intention that notification for the target information 730 is unnecessary.

[0130] Referring to Figure 18(d), the case in which the target information 740 is notified is referred to as Case CS4. In Case CS4, the target information 740 is notified using the fifth notification format. Immediately after the notification of the target information 740, the FB management unit F14 displays the message "How was the notification?" along with four FB buttons 741 to 744 on the display screen 51a, as shown in Figure 18(d). Corresponding to the FB buttons 741, 742, 743, and 744, the words "Good," "Vibration too strong," "Vibration too weak," and "No notification needed" are displayed, respectively. The FB management unit F14 displays the FB buttons 741 to 744 only for a predetermined FB reception time and accepts touch panel operations to select one of the FB buttons 741 to 744 within the FB reception time. If a touch panel operation is performed to select FB buttons 741, 742, 743, or 744 within the FB reception time, the content associated with each FB button 741, 742, 743, or 744 will be acquired as FB information 748. If there is no touch panel operation within the FB reception time, FB information 748 will not be acquired.

[0131] A touch panel operation selecting the FB button 741 indicates a positive response (evaluation) by user U1 to the notification of the target information 740. Therefore, the FB information 748 acquired when the FB button 741 is selected is classified as positive FB information, indicating the user's intention that the target information 740 is notified and that the notification method (notification parameter Pout) of the target information 740 is appropriate for user U1. A touch panel operation selecting the FB buttons 742, 743, or 744 indicates a negative response (evaluation) by user U1 to the notification of the target information 740. Therefore, the FB information 748 acquired when the FB button 742, 743, or 744 is selected is classified as negative FB information. The FB information 748 acquired when the FB button 742 is selected is FB information stating "the vibration is too strong," indicating the user's intention that the vibration intensity of the target information 740 is excessive for user U1. When a touch panel operation is performed to select the FB button 743, the FB information 748 obtained is FB information indicating that "the vibration is too weak," indicating the user U1's intention that the vibration intensity of the target information 740 is insufficient. When a touch panel operation is performed to select the FB button 744, the FB information 748 obtained indicates the user U1's intention that notification of the target information 740 is unnecessary.

[0132] The total number and contents of the FB buttons described here are merely examples, and FB information can be acquired in various ways using touch panel operation. Touch panel operation is a type of feedback operation by user U1 in response to notification of target information, and the feedback operation may also be an operation on any operating component (such as a mechanical push button) provided on the operation input unit 15.

[0133] In the following description, the selection operation of a certain FB button means an operation input to the in-vehicle device 10 by user U1 and a touch panel operation that selects the FB button. That is, for example, the selection operation of FB button 711 refers to the touch panel operation in which user U1 selects FB button 711, and the selection operation of FB button 712 refers to the touch panel operation in which user U1 selects FB button 712.

[0134] FB information may also be acquired based on the content of user U1's speech after the target notification has been made. In this case, the FB management unit F14 can acquire FB information corresponding to the content of user U1's speech based on the in-vehicle microphone signal (see Figure 5) indicating the content of user U1's speech. For example, if user U1 speaks an evaluation of the notification of target information 710 within the FB reception time after the notification of the target information 710, the FB management unit F14 analyzes the content of that speech using speech recognition and natural language processing. Based on the results of this analysis, the FB management unit F14 can generate and acquire FB information 718 corresponding to the content of the speech. In this case, for example, if user U1 says "That notification was good" in response to the notification of target information 710, the same FB information 718 as when the selection operation of the FB button 711 was input will be acquired. Alternatively, for example, if user U1 says "This notification is too strong" in response to a notification of target information 710, the same FB information 718 is obtained as if the FB button 712 were selected. Similarly, for example, if user U1 says "This notification is too weak" in response to a notification of target information 710, the same FB information 718 is obtained as if the FB button 713 were selected. Similarly, for example, if user U1 says "I don't need this notification" in response to a notification of target information 710, the same FB information 718 is obtained as if the FB button 714 were selected. The same applies to obtaining FB information based on user U1's utterance after target information other than target information 710 (for example, target information 720, 730, or 740) has been notified.

[0135] FB information may be acquired based on the emotions of user U1 estimated by the information acquisition unit F2. That is, FB management unit F14 may acquire FB information by estimating the user's evaluation of the target information based on the estimated emotions of user U1 immediately after notification of the target information. In addition, FB information may be acquired using any method that can identify the user's reaction to the notification of the target information. For example, FB management unit F14 may acquire FB information as an evaluation index by analyzing the facial image of user U1. In this case, FB management unit F14 may determine in the analysis whether user U1's facial expression indicates difficulty in viewing the notification image, difficulty in hearing the notification sound, surprise, etc., and set the numerical data of the determination result as FB information.

[0136] [Pre-machine learning to obtain estimated AI 100] Estimated AI 100 is generated by machine learning a learning model using pre-prepared basic training data. The machine learning to obtain estimated AI 100 is specifically called pre-machine learning. The machine learning to obtain estimated AI 100 is supervised machine learning, and therefore the training data can also be read as training data. The above operations described for the driver assistance system SYS and the in-vehicle device 10 are operations in the actual operation process after pre-machine learning. Figure 19 shows the hardware configuration involved in pre-machine learning. The data collection device 300, the database 310, and the learning device 320 are connected to a communication network including the Internet and an intranet. Each of the data collection device 300 and the learning device 320 consists of one or more computer devices. The database 310 is a large-capacity recording medium. The database 310 may be built into the data collection device 300 or the learning device 320. Estimated AI 100 is created by machine learning (pre-machine learning) performed by the learning device 320.

[0137] Figure 20 shows a flowchart related to the preparation and operation of the in-vehicle device 10. First, in step S1, a data collection process is performed using the data collection device 300, and a large amount of unit training data UDa is collected in the data collection process. Each unit training data UDa is stored in the database 310. After the data collection process, the machine learning process in step S2 is performed. After that, the process proceeds to step S3, and the actual operation process is performed. In the actual operation process, the operation of the above-mentioned driver assistance system SYS is performed using the in-vehicle device 10 into which the estimated AI 100 is incorporated.

[0138] This section describes a method for generating one set of unit training data UDa during the data collection process. Multiple cooperating vehicles are prepared, and the data collection process is carried out with the cooperation of multiple subjects. Each subject acts as the driver of a cooperating vehicle and drives it. Each cooperating vehicle has the same configuration as vehicle V1, and a driver assistance system equivalent to the driver assistance system SYS is installed in each cooperating vehicle. However, the notification parameter setting unit F1 in the driver assistance system of each cooperating vehicle is not provided with estimated AI 100. In the driver assistance system of each cooperating vehicle, the notification parameter setting unit F1 generates notification parameter Pot according to a predetermined algorithm based on the input data Din. Alternatively, in the driver assistance system of each cooperating vehicle, the notification parameter setting unit F1 generates notification parameter Pot from the input data Din using an estimated AI different from estimated AI 100.

[0139] The focus is on one cooperating vehicle. Target information is set one after another in the cooperating vehicle, and the set target information is notified to the corresponding subject through information notification processing. Each time the subject receives notification of target information, they provide evaluation information to the driving assistance system in the cooperating vehicle. The evaluation information indicates whether the notification of target information was good or bad for the subject. The evaluation information may be provided on a scale of three or more levels. The data acquisition device 300 is configured to enable bidirectional communication with the in-vehicle equipment in the cooperating vehicle. In the data acquisition process, the data acquisition device 300 classifies the pairs of input data Din and notification parameter Pout that are evaluated as good notifications for the subject into correct answer sets. It is expected that many correct answer sets will be set from one cooperating vehicle. In the data acquisition process, many correct answer sets will be set in each of the multiple cooperating vehicles. One set of unit training data UDa is formed from the input data Din and notification parameter Pout in one correct answer set. In one set of unit training data UDa, the input data Din and the notification parameter Pout are associated with each other.

[0140] In the data collection process, a large amount of unit training data UDa is collected by the data collection device 300. The large amount of unit training data UDa collected in the data collection process is called basic learning data LDa. The basic learning data LDa is stored in the database 310. After the data collection process, the machine learning process is performed by the learning device 320. In the machine learning process, the notification parameter Pout in the unit training data UDa is used as the correct answer data. The learning device 320 has a learning model 321. The learning model 321 is formed by the arithmetic processing unit of the learning device 320. The learning model 321 is artificial intelligence based on a deep neural network. The deep neural network will be denoted as DNN below.

[0141] In the machine learning process, the learning device 320 obtains each unit of training data UDa from the database 310. For each unit of training data UDa, the learning device 320 extracts input data Din from UDa and inputs the extracted input data Din into the learning model 321. In the machine learning process, the learning model 321 performs inference similar to that performed by the estimation AI 100 based on the input data Din, thereby generating a notification parameter Pout. In the machine learning process, the learning device 320 derives the error between the notification parameter Pout generated by the learning model 321 and the correct data for each unit of training data UDa, and performs machine learning (pre-machine learning) to update the parameters of the learning model 321 so that the error is reduced. The parameters of the learning model 321 include the weights and biases of the DNN that constitute the learning model 321. Machine learning is performed until the error becomes sufficiently small, or for a specified number of epochs. A learning model having the same configuration as the learning model 321 after machine learning (pre-machine learning) is incorporated into the estimated AI 100 in the in-vehicle device 10. Then, the process proceeds to step S3 and the above-described actual operation process is carried out (see Figure 20). Unless otherwise specified, each operation shown below is an operation in the actual operation process.

[0142] The following describes several specific operational examples, application techniques, and modification techniques related to the driver assistance system SYS and the in-vehicle device 10 (particularly the notification parameter setting unit F1) within the context of multiple practical examples. Unless otherwise specified and without contradiction, the matters described above apply to each of the following examples. In the event of any inconsistency between the above and the above in any example, the description in that example may take precedence. Furthermore, unless contradictory, the matters described in any of the following examples can be applied to any other example (i.e., any two or more examples from the multiple examples can be combined).

[0143] <<Example EX_1A>> Example EX_1A will now be described. In Example EX_1A, a pre-correction configuration is adopted (see Figure 16). Figure 21 shows the internal configuration of the notification parameter setting unit F1 in Example EX_1A when the pre-correction configuration is adopted, and its surrounding configuration. A learning model 110 is provided for the estimated AI 100 in Example EX_1A and any other example described later. The learning model 110 is a learning model having the same configuration as the learning model 321 after the above-mentioned pre-machine learning, and therefore is a learning model that has been trained by the machine learning process in step S2 (see Figures 19 and 20). The learning model 110 in the pre-correction configuration receives the input data CDin for the estimated AI 100 as input data to itself. The learning model 110 in the pre-correction configuration outputs the notification parameters inferred based on the input data to itself as the output parameter Dout.

[0144] Input data Din and notification parameter Pout are input to the FB management unit F14. Correction input data CDin may also be input to the FB management unit F14 in addition to input data Din. The FB reflection table 141 is stored in the recording medium 14 (see Figure 4). The FB reflection table 141 may also be built into the FB management unit F14.

[0145] The FB management unit F14 has the function of adjusting the content of the pre-processing correction process performed by the pre-processing unit F12 according to the FB information. The FB reflection table 141 stores reflection method information in advance, which indicates how the content of the pre-processing correction process should be adjusted and set according to the content of the FB information. The FB management unit F14 adjusts and sets the content of the pre-processing correction process by referring to the reflection method information based on the FB information.

[0146] Figure 22 is an operation flowchart of the controller 11, relating to embodiment EX_1A, focusing on information notification processing. The on-board device 10 is activated in conjunction with the start of the engine mounted on the vehicle V1 by operating the ignition switch of the vehicle V1. Once the on-board device 10 is activated, the process proceeds to step S11.

[0147] After the in-vehicle device 10 is started, the notification management unit F3 continuously performs the notification determination process described above, and continuously and repeatedly determines whether any information (any of the first to nth notification information) should be notified to user U1 (see Figure 13). If it is determined in step S11 that any information should be notified to user U1 (Y in step S11), the system proceeds to step S12. Otherwise, the system returns to step S11 and repeats the determination of whether any information (any of the first to nth notification information) should be notified to user U1.

[0148] In step S12, the notification management unit F3 sets the target information according to the result of the notification determination process. In the following step S13, the priority setting unit F4 sets the priority and the notification format determination unit F5 sets the notification format information (see Figure 13). In step S13, the notification format determination unit F5 also determines the display position, sound position, and vibration position related to the target information (see Figure 13). Subsequently, in step S14, input data Din, including the settings from steps S12 and S13, is supplied to the notification parameter setting unit F1. In step S14, the notification parameter setting unit F1 sets the notification parameter Pout by parameter setting processing based on the supplied input data Din. In step S14 according to embodiment EX_1A, corrected input data CDin is generated from the input data Din through a pre-correction process, and the corrected input data CDin is input to the estimation AI 100. Then, the output parameter Dout derived from the corrected input data CDin by the estimation AI 100 becomes the notification parameter Pout.

[0149] Subsequently, in step S15, the output control unit F6 controls the information output unit 50 so that the target information is notified to the user U1 in accordance with the characteristics of the notification parameter Pout, thereby notifying the user U1 of the target information. At this time, if the first or second notification format is ON as described above, the output control unit F6 controls the parameter PR in the notification parameter Pout. Ａ１ ~PR Ａ４The notification image corresponding to the target information is displayed on the display device 51 accordingly (see Figures 6 and 12). When the third or fourth notification format is ON, the output control unit F6 checks the parameter PR in the notification parameter Out. Ｂ１ ~PR Ｂ３ Accordingly, a notification sound corresponding to the target information is output from speaker 52 (see Figures 6 and 12). When the fifth notification format is ON, the output control unit F6 outputs the parameter PR in the notification parameter Out. Ｃ１ ~PR Ｃ２ Accordingly, the vibration device 53 outputs a notification vibration corresponding to the target information (see Figures 6 and 12). In addition, the output control unit F6 outputs the parameter PR in the notification parameter Poout. Ｄ１ The timing of notification of the target information is controlled accordingly.

[0150] In parallel with or after the notification in step S15, the FB management unit F14 performs the FB information acquisition process in step S16. However, as described above, FB information may not be acquired. In step S17 following step S16, it is confirmed whether FB information has been acquired. If FB information has been acquired (step S17 Y), the process proceeds to step S18; if FB information has not yet been acquired (step S17 N), the process proceeds to step S19. In step S19, the FB management unit F14 checks whether the elapsed time since the notification of the most recent target information has reached a predetermined FB reception time. If the elapsed time has not reached the FB reception time (step S19 N), the process returns to step S17. If the elapsed time reaches the FB reception time without FB information being acquired (step S19 Y), the FB management unit F14 determines that it has failed to acquire FB information for the most recent target information and triggers a transition to step S11. For example, when attempting to acquire FB information by having the user select one of the multiple FB buttons displayed on the display screen 51a using a touch panel, if the user makes the selection via the touch panel before the elapsed time reaches the FB reception time, the FB information will be acquired. If the elapsed time reaches the FB reception time without any such selection via the touch panel, it will be determined that the acquisition of FB information has failed.

[0151] In step S18, the FB management unit F14 performs a correction content adjustment process. In the example EX_1A, the correction content adjustment process is a correction content adjustment process for the preceding correction process. In the correction content adjustment process for the preceding correction process, the FB management unit F14 adjusts (in other words, sets) the correction content in the preceding correction process based on the acquired FB information. After step S18, the process returns to step S11. In steps S14 and S15, which follow step S18, the input data Din is corrected according to the correction content adjusted in the correction content adjustment process of step S18. The corrected input data CDin obtained by this correction is then input to the estimated AI 100, and the notification parameter Pot is determined.

[0152] --Adjustment Processing for Correction Content in Pre-Stage Correction Processing-- The adjustment processing for correction content in pre-stage correction processing will be explained. In the pre-stage processing unit F12, a correction amount ΔCin is defined for each of the 3rd, 5th to 23rd input items in the input data Din (see Figure 14). Among the 3rd and 5th to 23rd input items, the correction amount ΔCin for the ith input item is sometimes specifically written as correction amount ΔCin[i]. However, it is not necessary to define a correction amount ΔCin for all of the 3rd and 5th to 23rd input items. The data of the ith input item for which a correction amount ΔCin[i] is defined is corrected by the correction amount ΔCin[i] in the pre-stage correction processing.

[0153] Referring to Figure 23, the data of the i-th input item in the input data Din is specifically denoted as din[i], and the data of the i-th input item in the corrected input data CDin is specifically denoted as Cdin[i]. Then, "Cdin[i] = din[i] + ΔCin[i]".

[0154] The correction amount ΔCin[i] can be positive, negative, or zero. If the correction amount ΔCin[i] is zero, then no correction is applied to the data din[i], and "Cdin[i] = din[i]". If the correction amount ΔCin[i] is positive, then the data din[i] is increased by the magnitude of the correction amount ΔCin[i] in the preceding correction process. However, the upper limit of the values ​​that the data Cdin[i] can take (L) ｉThe amount of increase is limited so as not to exceed (see Figure 14). In other words, when "ΔCin[i] > 0", the sum (din[i] + ΔCin[i]) is limited to the upper limit L. ｉ If it exceeds this, then "Cdin[i] = L ｉ If the correction amount ΔCin[i] is negative, the data din[i] is reduced by the magnitude of the correction amount ΔCin[i] in the preceding correction process. However, the amount of reduction is limited so that the data Cdin[i] does not fall below the lower limit (1) of the values ​​that data din[i] can take. In other words, if the sum (din[i] + ΔCin[i]) is below the lower limit of 1 when "ΔCin[i] < 0", then "ΔCin[i] = 1".

[0155] In the initial state of the in-vehicle device 10, all correction amounts ΔCin are set to zero. The initial state of the in-vehicle device 10 refers to the in-vehicle device 10 immediately after the start of the actual operation process (the in-vehicle device 10 that has never performed information notification processing). The FB processing unit F14 can initialize the pre-stage correction processing. Even immediately after the initialization of the pre-stage correction processing, all correction amounts ΔCin are zero. In the correction content adjustment processing, the correction amount ΔCin[i] for one or more input items is adjusted in the direction of increasing or decreasing.

[0156] Let's consider a specific example. In Case CS2 shown in Figure 18(b), we assume a situation where target information 720 is notified and corresponding FB information 728 is acquired. In Case CS2, when the FB button 722 is selected with the message "Display is too large," the display size set by the estimated AI 100 is considered to be excessive for user U1. On the other hand, it is presumed that the estimated AI 100 derives the output parameter Dout so that the display size of the target information 720 becomes smaller the higher the user U1's visual acuity is. Therefore, in Case CS2a of Case CS2, where FB information 728 is acquired due to the selection of the FB button 722, the FB management unit F14 increases the correction amount ΔCin

[12] corresponding to visual acuity based on the FB information 728 by the reference change amount, as shown in Figure 24(a). Here, we assume that the reference change amount is set to 1, but the reference change amount may have an integer value greater than 1.

[0157] Conversely, in case CS2, when the FB button 723 is selected with the message "Display is too small," the display size set by the estimated AI 100 is considered to be too small for user U1. On the other hand, it is presumed that the estimated AI 100 derives the output parameter Dout so that the display size of the target information 720 increases as user U1's visual acuity decreases. Therefore, in case CS2b of case CS2, where the FB information 728 is acquired when the FB button 723 is selected, the FB management unit F14 reduces the correction amount ΔCin

[12] corresponding to visual acuity by the reference change amount (here, 1) based on the FB information 728, as shown in Figure 24(b).

[0158] In case CS2c, one of the cases in CS2, where FB information 728 is acquired due to the selection of the FB button 721 indicating "good," the FB management unit F14 does not change the correction amount ΔCin

[12] corresponding to visual acuity based on the FB information 728, as shown in Figure 24(c).

[0159] On the other hand, in case CS2, when the FB button 724 is selected, it is assumed that user U1's driving proficiency (driver skill) is high enough that notification of target information 720 is unnecessary. Conversely, it is presumed that the higher user U1's driver skill, the less conspicuous the display of target information 720 becomes (less bothersome to user U1), and the estimated AI 100 derives the output parameter Dout accordingly. Therefore, in case CS2d of case CS2, where FB information 728 is acquired due to the selection of the FB button 724, as shown in Figure 24(d), the FB management unit F14 increases the correction amount ΔCin

[23] corresponding to the driver skill by the reference change amount (here, 1) based on the FB information 728.

[0160] In the case CS3 shown in Figure 18(c), where target information 730 is notified and corresponding FB information 738 is acquired, the situation can be considered in the same way as in case CS2. That is, in case CS3a, where FB information 738 is acquired due to the selection of the FB button 732 indicating "sound is too loud," the FB management unit F14 increases the correction amount ΔCin

[13] corresponding to hearing based on the FB information 738 by the reference change amount (here, 1), as shown in Figure 25(a). In case CS3b, where FB information 738 is acquired due to the selection of the FB button 733 indicating "sound is too quiet," the FB management unit F14 decreases the correction amount ΔCin

[13] corresponding to hearing based on the FB information 738 by the reference change amount (here, 1), as shown in Figure 25(b). In case CS3c, one of the cases in CS3, where FB information 738 is acquired when the FB button 731 indicating "good" is selected, the FB management unit F14 does not change the correction amount ΔCin

[13] corresponding to hearing based on the FB information 738, as shown in Figure 25(c). In case CS3d, one of the cases in CS3, where FB information 738 is acquired when the FB button 734 is selected, the FB management unit F14 increases the correction amount ΔCin

[23] corresponding to driver skill by the reference change amount (here, 1) based on the FB information 738, as shown in Figure 25(d).

[0161] In the case CS4 shown in Figure 18(d), where target information 740 is notified and corresponding FB information 748 is acquired, the same considerations apply as in case CS2. Specifically, in case CS4a of case CS4, where FB information 748 is acquired due to the selection of the FB button 742 indicating "vibration is too strong," the FB management unit F14 reduces the correction amount ΔCin

[10] corresponding to the vehicle vibration by the reference change amount (here, 1) based on the FB information 748, as shown in Figure 26(a). This is because it is considered more likely that a lower vibration intensity will be set in the estimated AI 100 if the vehicle vibration data is corrected to a lower value. In case CS4b, one of the cases in CS4, where FB information 748 is acquired due to the selection of the FB button 743 indicating "vibration is too weak," the FB management unit F14 increases the correction amount ΔCin

[10] corresponding to the vehicle vibration by the reference change amount (here, 1) based on the FB information 748, as shown in Figure 26(b). This is because increasing the vehicle vibration data is considered to increase the likelihood of a higher vibration intensity being set in the estimated AI 100. In case CS4c, one of the cases in CS4, where FB information 748 is acquired due to the selection of the FB button 741 indicating "good," the FB management unit F14 does not change the correction amount ΔCin

[10] corresponding to the vehicle vibration based on the FB information 748, as shown in Figure 26(c). In case CS4d, one of the cases CS4, where FB information 748 is acquired due to the selection of the FB button 744, the FB management unit F14 increases the correction amount ΔCin

[23] corresponding to the driver skill by the reference change amount (here, 1) based on the FB information 748, as shown in Figure 26(d).

[0162] In Case CS1 shown in Figure 18(a), the situation in which the target information 710 is notified and the corresponding FB information 718 is acquired can be considered in the same way as in Cases CS2, CS3, or CS4.

[0163] Specifically, in case CS1a, where FB information 718 is acquired due to the selection of the FB button 712 indicating "too strong" within case CS1, the FB management unit F14 performs the necessary processing based on the FB information 718 as shown in Figure 27(a). In this case, if the target information 710 was notified using the first or second notification format in case CS1a, the processing to increase the correction amount ΔCin

[12] corresponding to visual acuity by the reference change amount (here, 1) is sufficient. If the target information 710 was notified using the third or fourth notification format in case CS1a, the processing to increase the correction amount ΔCin

[13] corresponding to hearing by the reference change amount (here, 1) is sufficient. If the target information 710 was notified using the fifth notification format in case CS1a, the processing to decrease the correction amount ΔCin

[10] corresponding to vehicle vibration by the reference change amount (here, 1) is sufficient. The selection of the FB button 712 indicating "too strong" means that the intensity of the sensory (visual, auditory, or tactile) stimulus in the notification of the target information 710 is too strong for user U1.

[0164] Conversely, in case CS1b, where FB information 718 is acquired due to the selection of the FB button 713 indicating "too weak" within case CS1, the FB management unit F14 performs the necessary processing based on the FB information 718 as shown in Figure 27(b). In this case, if the target information 710 was notified using the first or second notification format in case CS1b, the processing to reduce the correction amount ΔCin

[12] corresponding to visual acuity by the reference change amount (here, 1) is sufficient. If the target information 710 was notified using the third or fourth notification format in case CS1b, the processing to reduce the correction amount ΔCin

[13] corresponding to hearing by the reference change amount (here, 1) is sufficient. If the target information 710 was notified using the fifth notification format in case CS1b, the processing to increase the correction amount ΔCin

[10] corresponding to vehicle vibration by the reference change amount (here, 1) is sufficient. The operation of selecting the FB button 713 indicating "too weak" means that the intensity of the sensory (visual, auditory, or tactile) stimulus in the notification of the target information 710 is too weak for user U1.

[0165] Furthermore, in case CS1c, within case CS1, where FB information 718 is acquired due to the selection of the FB button 711 indicating "good," the FB management unit F14 does not change each correction amount ΔCin based on the FB information 718, as shown in Figure 27(c). In case CS1d, within case CS1, where FB information 718 is acquired due to the selection of the FB button 714, the FB management unit F14 increases the correction amount ΔCin

[23] corresponding to the driver skill by the reference change amount (here, 1) based on the FB information 718, as shown in Figure 27(d).

[0166] Here, in cases CS2a and CS2b (see Figures 24(a) and (b)), it is illustrated that the correction amount ΔCin

[12] for data corresponding to visual acuity is adjusted based on FB information 728. However, the correction amount ΔCin adjusted in the correction content adjustment process for cases CS2a and CS2b only needs to be the correction amount ΔCin for data related to visual sensitivity (for example, 1 or more of the correction amounts ΔCin for the data of the 6th, 7th, 11th, 12th and 17th input items). The same applies in cases CS1a and CS1b (see Figures 27(a) and (b)) when the target information 710 is notified using the first or second notification format.

[0167] Similarly, in cases CS3a and CS3b (see Figures 25(a) and (b)), it is illustrated that the correction amount ΔCin

[13] for the data corresponding to hearing is adjusted based on the FB information 738. However, the correction amount ΔCin adjusted in the correction content adjustment process for cases CS3a and CS3b only needs to be the correction amount ΔCin for data related to auditory sensitivity (for example, 1 or more of the correction amounts ΔCin for the data of the 9th, 13th and 17th input items). The same applies in cases CS1a and CS1b (see Figures 27(a) and (b)) when the target information 710 is notified using the third or fourth notification format.

[0168] Similarly, in cases CS4a and CS4b (see Figures 26(a) and (b)), it is illustrated that the correction amount ΔCin

[10] for data corresponding to vehicle vibration is adjusted based on FB information 748. However, the correction amount ΔCin adjusted in the correction content adjustment process for cases CS4a and CS4b only needs to be a correction amount ΔCin for data related to tactile sensitivity (for example, 1 or more of the correction amounts ΔCin for the data of the 10th and 14th to 17th input items). The same applies in cases CS1a and CS1b (see Figures 27(a) and (b)) when the target information 710 is notified using the 5th notification format.

[0169] As already mentioned, it is possible to obtain FB information without using the FB button selection operation, for example, by using natural language processing to obtain FB information based on the content of user U1's utterance. Even when FB information is obtained without using the FB button selection operation, the necessary correction amount ΔCin can be adjusted based on the content of the FB information.

[0170] The FB reflection table 141 specifies how each correction amount ΔCin should be adjusted according to the input data Din and the contents of the FB information. The FB management unit F14 should perform the correction content adjustment process according to the provisions of the FB reflection table 141.

[0171] As described above, an example is given in which one of the correction amounts ΔCin is increased or decreased by 1, which corresponds to the reference change amount. However, depending on the acquired FB information, the corresponding correction amount ΔCin may be increased or decreased by 2 or more. For example, in case CS2 corresponding to Figure 18(b), in addition to the FB buttons 721 to 724, a fifth FB button (not shown) indicating the user's intention that "the display is too large" may be displayed on the display screen 51a. When FB information 728 is obtained by selecting the fifth FB button, the FB management unit F14 may increase the correction amount ΔCin

[12] corresponding to visual acuity by 2 or 3 all at once. The same considerations apply when a sixth FB button indicating the user's intention that "the display is too small" is provided. The same applies to cases CS3, CS4, and CS1.

[0172] <<Example EX_1B>> Example EX_1B will now be explained. In Example EX_1B, a post-stage correction configuration is adopted (see Figure 17). Figure 28 shows the internal configuration of the notification parameter setting unit F1 in Example EX_1B when the post-stage correction configuration is adopted, and its surrounding configuration. The learning model provided in the estimated AI 100 is the learning model 110 as already described, and the matters described above for the learning model 110 also apply to Example EX_1B. In addition, for matters not specifically described in Example EX_1B, the matters described in Example EX_1A may be applied to Example EX_1B as long as there is no contradiction. The learning model 110 in the post-stage correction configuration receives the input data Din for the estimated AI 100 as input data to itself. The learning model 110 in the post-stage correction configuration outputs the notification parameters inferred based on the input data to itself as output parameters Dout.

[0173] In the notification parameter setting unit F1 of Figure 21, the notification parameter setting unit F1 of Figure 28 is formed by replacing the preceding processing unit F12 with the succeeding processing unit F13. Following this replacement, the FB management unit F14 of Example EX_1B adjusts and sets the content of the subsequent correction processing performed by the succeeding processing unit F13 according to the FB information. In conjunction with this, the FB reflection table 141 of Example EX_1B pre-stores reflection method information indicating how the content of the subsequent correction processing should be adjusted and set according to the content of the FB information. Input data Din and notification parameter Pout are input to the FB management unit F14. Based on the FB information, the FB management unit F14 refers to the reflection method information and adjusts and sets the content of the subsequent correction processing.

[0174] In Example EX_1B, the operation flowchart of the controller 11 related to the information notification process is the same as that shown in Figure 22. However, in step S14 of Example EX_1B, the notification parameter Pout is obtained by applying a subsequent correction process to the output parameter Dout, which is derived by inputting the input data Din to the estimation AI 100. Also, the correction content adjustment process in step S18 of Example EX_1B is a correction content adjustment process for the subsequent correction process. In the correction content adjustment process for the subsequent correction process, the FB management unit F14 adjusts the content of the subsequent correction process by the subsequent processing unit F13, taking into account the FB reflection table 141 based on the acquired FB information.

[0175] --Correction content adjustment process for subsequent correction process-- The correction content adjustment process for subsequent correction process will be explained. In the subsequent processing unit F13, the parameter PR that constitutes the output parameter Dout Ａ１ ~PR Ａ４ PR Ｂ１ ~PR Ｂ３ PR Ｃ１ ~PR Ｃ２ and PR Ｄ１ A correction amount ΔCout is defined for this. For the sake of explanation, the parameter PR is as shown in Figure 29. Ａ１ ~PR Ａ４ These are referred to as the 1st to 4th parameters, respectively. Similarly, parameter PR Ｂ１ ~PR Ｂ３ These are referred to as the 5th to 7th parameters, respectively, and parameter PR Ｃ１ PR Ｃ２ and PR Ｄ１ These are referred to as the 8th, 9th, and 10th parameters, respectively. The correction amount ΔCout for the i-th parameter in the output parameter Dout is sometimes specifically written as correction amount ΔCout[i] (where i is an integer between 1 and 10). However, it is not necessary to define a correction amount ΔCout for all of the 1st to 10th parameters. The value (data) of a parameter for which a correction amount ΔCout[i] is defined will be corrected by the correction amount ΔCout[i] in the subsequent correction process.

[0176] In addition, the value (data) of the i-th parameter in the output parameter Dout is specifically denoted as dout[i], and the value (data) of the i-th parameter in the notification parameter Pout is specifically denoted as Cdout[i]. Then, as shown in Figure 30, "Cdout[i] = dout[i] + ΔCout[i]".

[0177] The correction amount ΔCout[i] can be positive, negative, or zero. If the correction amount ΔCout[i] is zero, no correction is applied to the data dout[i], and "Cdout[i] = dout[i]". If the correction amount ΔCout[i] is positive, the data dout[i] is increased by the magnitude of the correction amount ΔCout[i] in the subsequent correction process. However, the amount of increase is limited so that the data Cdout[i] does not exceed the upper limit of the value that data dout[i] can take. In other words, if "ΔCout[i] > 0" and the sum (dout[i] + ΔCout[i]) exceeds a specified upper limit, that upper limit is set for the data Cdout[i]. If the correction amount ΔCout[i] is negative, the data dout[i] is reduced by the magnitude of the correction amount ΔCout[i] in the subsequent correction process. However, the reduction amount is limited so that the data Cdout[i] does not fall below the lower limit of the values ​​that data dout[i] can take. In other words, if the sum (dout[i] + ΔCout[i]) falls below a specified lower limit when "ΔCout[i] < 0", that lower limit is set to the data Cdout[i].

[0178] In the initial state of the in-vehicle device 10, all correction amounts ΔCout are set to zero. As already mentioned, the initial state of the in-vehicle device 10 refers to the in-vehicle device 10 immediately after the start of the actual operation process (the in-vehicle device 10 that has never performed information notification processing). The FB processing unit F14 can initialize the subsequent correction processing. Even immediately after the initialization of the subsequent correction processing, all correction amounts ΔCout are zero. In the correction content adjustment processing according to Example EX_1B, one or more correction amounts ΔCout are adjusted in the direction of increasing or decreasing.

[0179] Let's consider a specific example. In Case CS2 shown in Figure 18(b), we assume a situation where target information 720 is notified and corresponding FB information 728 is acquired. In Case CS2, when the FB button 722 is selected with the message "Display is too large," the display size set by the estimated AI 100 is considered to be excessive for user U1. Therefore, in Case CS2a of Case CS2, where FB information 728 is acquired due to the selection of the FB button 722, the FB management unit F14 reduces the correction amount ΔCout[1] corresponding to the display size by the reference change amount, as shown in Figure 31(a), based on the FB information 728. Similar to Example EX_1A, here we assume that the reference change amount is set to 1, but the reference change amount may have an integer value greater than 1. Conversely, in case CS2, when the FB button 723 is selected with the message "Display is too small," the display size set by the estimated AI 100 is considered to be too small for user U1. Therefore, in case CS2b, where FB information 728 is acquired due to the selection of the FB button 723, the FB management unit F14 increases the correction amount ΔCout[1] corresponding to the display size by the base change amount (here, 1) based on the FB information 728, as shown in Figure 31(b). In case CS2c, where FB information 728 is acquired due to the selection of the FB button 721 with the message "Good," the FB management unit F14 does not change the correction amount ΔCout[1] corresponding to the display size based on the FB information 728, as shown in Figure 31(c). Furthermore, in Case CS2, if the user selects the FB button 724 indicating "No notification required," the method shown in Example EX_1A may be applied.

[0180] In the case CS3 shown in Figure 18(c), where target information 730 is notified and corresponding FB information 738 is acquired, the situation can be considered in the same way as in case CS2. That is, in case CS3a, where FB information 738 is acquired due to the selection of the FB button 732 indicating "the sound is too loud", the FB management unit F14 decreases the correction amount ΔCout[5] corresponding to the volume by a reference change amount (here, 1) based on the FB information 738, as shown in Figure 32(a). In case CS3b, where FB information 738 is acquired due to the selection of the FB button 733 indicating "the sound is too quiet", the FB management unit F14 increases the correction amount ΔCout[5] corresponding to the volume by a reference change amount (here, 1) based on the FB information 738, as shown in Figure 32(b). In case CS3c, where FB information 738 is acquired due to the selection of the FB button 731 indicating "good," the FB management unit F14 does not change the correction amount ΔCout[5] corresponding to the volume based on the FB information 738, as shown in Figure 32(c). Furthermore, in case CS3, if the FB button 734 indicating "no notification needed" is selected, the method shown in Example EX_1A may be applied.

[0181] In the case CS4 shown in Figure 18(d), where target information 740 is notified and corresponding FB information 748 is acquired, the situation can be considered in the same way as in case CS2. That is, in case CS4a, where FB information 748 is acquired due to the selection of the FB button 742 indicating "vibration is too strong", the FB management unit F14 decreases the correction amount ΔCout[8] corresponding to the vibration intensity by the reference change amount (here, 1) based on the FB information 748, as shown in Figure 33(a). In case CS4b, where FB information 748 is acquired due to the selection of the FB button 743 indicating "vibration is too weak", the FB management unit F14 increases the correction amount ΔCout[8] corresponding to the vibration intensity by the reference change amount (here, 1) based on the FB information 748, as shown in Figure 33(b). In case CS4c, where FB information 748 is acquired due to the selection of the FB button 741 indicating "good," the FB management unit F14 does not change the correction amount ΔCout[8] corresponding to the vibration intensity based on the FB information 748, as shown in Figure 33(c). Furthermore, in case CS4, if the FB button 744 indicating "no notification needed" is selected, the method shown in Example EX_1A may be applied.

[0182] In Case CS1 shown in Figure 18(a), the situation in which the target information 710 is notified and the corresponding FB information 718 is acquired can be considered in the same way as in Cases CS2, CS3, or CS4.

[0183] Specifically, in case CS1a, where FB information 718 is acquired due to the selection of the FB button 712 indicating "too strong" within case CS1, the FB management unit F14 performs the necessary processing based on the FB information 718 as shown in Figure 34(a). In this case, if the target information 710 was notified using the first or second notification format in case CS1a, the processing to reduce the correction amount ΔCout[1] corresponding to the display size by the reference change amount (here, 1) is sufficient. If the target information 710 was notified using the third or fourth notification format in case CS1a, the processing to reduce the correction amount ΔCout[5] corresponding to the volume by the reference change amount (here, 1) is sufficient. If the target information 710 was notified using the fifth notification format in case CS1a, the processing to reduce the correction amount ΔCout[8] corresponding to the vibration intensity by the reference change amount (here, 1) is sufficient. The selection of the FB button 712 indicating "too strong" means that the intensity of the sensory (visual, auditory, or tactile) stimulus in the notification of the target information 710 is too strong for user U1.

[0184] Similarly, in case CS1b, where FB information 718 is acquired due to the selection of the FB button 713 indicating "too weak" within case CS1, the FB management unit F14 performs the necessary processing based on the FB information 718 as shown in Figure 34(b). In this case, if the target information 710 was notified using the first or second notification format in case CS1b, the processing to increase the correction amount ΔCout[1] corresponding to the display size by the reference change amount (here, 1) is sufficient. If the target information 710 was notified using the third or fourth notification format in case CS1b, the processing to increase the correction amount ΔCout[5] corresponding to the volume by the reference change amount (here, 1) is sufficient. If the target information 710 was notified using the fifth notification format in case CS1b, the processing to increase the correction amount ΔCout[8] corresponding to the vibration intensity by the reference change amount (here, 1) is sufficient. The operation of selecting the FB button 713 indicating "too weak" means that the intensity of the sensory (visual, auditory, or tactile) stimulus in the notification of the target information 710 is too weak for user U1.

[0185] Similarly, in case CS1c, where FB information 718 is acquired due to the selection of the FB button 711 indicating "good" within case CS1, the FB management unit F14 does not change each correction amount ΔCout based on the FB information 718, as shown in Figure 34(c). Furthermore, in case CS4, when the FB button 714 indicating "no notification needed" is selected, the method shown in Example EX_1A may be applied.

[0186] The in-vehicle device 10 may be provided with various types of FB buttons to allow user evaluations of the notification of target information from various angles. For example, suppose an FB button indicating "notification is late" is displayed on the display screen 51a, and FB information is acquired when the user selects the FB button indicating "notification is late". In this case, during the correction content adjustment process, the FB management unit F14 may reduce the correction amount ΔCout

[10] corresponding to the notification start timing by the base change amount (here, 1). As described above, the parameter PR corresponding to the notification start timing Ｄ１ As the value of decreases, the notification start timing becomes earlier. Conversely, for example, suppose a button for FB indicating "early notification" is displayed on the display screen 51a, and FB information is acquired when the FB button indicating "early notification" is selected. In this case, in the correction content adjustment process, the FB management unit F14 may increase the correction amount ΔCout

[10] corresponding to the notification start timing by the base change amount (here, 1). Parameter PR corresponding to the notification start timing Ｄ１ As the value of (parameter 10) increases, the notification start timing becomes later.

[0187] As already mentioned, it is also possible to obtain FB information without using the FB button selection operation, such as by using natural language processing to obtain FB information based on the content of the user U1's utterance. Even when FB information is obtained without using the FB button selection operation, the necessary correction amount ΔCout can be adjusted according to the user evaluation based on the content of the FB information. By using natural language processing, the FB management unit F14 can understand the user U1's intention from the content of the user U1's utterance with a high degree of freedom, making it easier to realize correction content adjustment processing in line with the user U1's intention. For example, consider the case where user U1 says "Make the display brighter" in response to notification of certain target information. In this case, the FB management unit F14 can increase the correction amount ΔCout[2] corresponding to the contrast in the correction content adjustment processing by the reference change amount (here, 1) from the FB information indicating the content of the utterance (see Figure 29).

[0188] As described above, an example is given in which one of the correction amounts ΔCout is increased or decreased by 1, which corresponds to the base change amount. However, depending on the acquired FB information, the corresponding correction amount ΔCout may be increased or decreased by 2 or more. For example, in case CS2 corresponding to Figure 18(b), in addition to the FB buttons 721 to 724, a fifth FB button (not shown) indicating the user's intention that "the display is too large" may be displayed on the display screen 51a. When FB information 728 is obtained by selecting the fifth FB button, the FB management unit F14 may suddenly decrease the correction amount ΔCout[1] corresponding to the display size by 2 or 3. The same considerations apply when a sixth FB button indicating the user's intention that "the display is too small" is provided. The same applies to cases CS3, CS4, and CS1.

[0189] <<Example EX_1C>> Example EX_1C will now be described. Example EX_1C includes supplementary information regarding the configuration and operation of the controller 11 described up to this point.

[0190] When providing information notifications to user U1, user U1 may be dissatisfied with the notification method or other aspects. To address this, the in-vehicle device 10 uses AI to determine the parameters (notification parameter Pout) for information notification. Specifically, the controller 11 (notification parameter setting unit F1) sets the notification parameter Pout when notifying user U1 of target information using estimated AI 100 based on the input data Din (see Figures 13, 21, and 28). However, the AI ​​initially incorporated into the in-vehicle device 10 may not be suited to user U1's personal characteristics (preferences, etc.). Taking this into consideration, the controller 11 sets the notification parameter Pout based on the input data Din and user U1's response (FB information) to previously executed notifications (see Figures 21 and 28). Previously executed notifications refer to past notifications (notifications made before the setting of the notification parameter Pout), and the controller 11 refers to evaluation indicators as user U1's response. Specifically, the controller 11 sets the notification parameter Pout based on the input data Din and the evaluation index of user U1 for past notifications. User U1's response to completed notifications reflects user U's personal characteristics (preferences, etc.). Therefore, by setting the notification parameter Pout based on user U's response to completed notifications, user U1's personal characteristics can be reflected in the setting of the notification parameter Pout. As a result, it becomes possible to realize notifications in a manner suitable for user U1's personal characteristics.

[0191] A pre-correction configuration can be adopted for the usage of user U1's response (see Figures 16 and 21). In this case, the controller 11 generates corrected input data CDin by correcting the input data Din based on an evaluation index (an evaluation index indicating user U1's response to a notification that has already been executed) (see Figure 21). Then, by inputting the corrected input data CDin to the estimated AI 100, the output parameter Dout output from the estimated AI 100 is set to the notification parameter Pout (see Figure 21). It is expected that by correcting the input data Din in accordance with user U1's response, the notification parameter Pout will be changed to one that is in line with user U1's personal characteristics (preferences, etc.).

[0192] A post-processing correction configuration can be adopted regarding the usage of user U1's response (see Figures 17 and 28). In this case, the controller 11 inputs the input data Din to the estimated AI 100, and then corrects the output parameter Dout output from the estimated AI 100 based on an evaluation index (an evaluation index indicating user U1's response to the executed notification) to set the notification parameter Pout (see Figure 28). It is expected that by correcting the output parameter Dout in accordance with user U1's response, the notification parameter Pout will be changed to one that is in line with user U1's personal characteristics (preferences, etc.).

[0193] It is also possible to combine a pre-correction configuration and a post-correction configuration. That is, a bidirectional correction configuration can be adopted regarding the usage pattern of user U1's response (see Figure 15). In this case, the controller 11 generates corrected input data CDin by correcting the input data Din based on an evaluation index (an evaluation index showing user U1's response to a completed notification). Then, the controller 11 inputs the corrected input data CDin to the estimated AI 100 and sets the notification parameter Pout by correcting the output parameter Dout output from the estimated AI 100 based on the evaluation index (an evaluation index showing user U1's response to a completed notification). This allows the advantages of both the pre-correction configuration and the post-correction configuration to be enjoyed, and it is expected that the notification parameter Pout will be changed to match user U1's personal characteristics (preferences, etc.). Specific examples of the use of the bidirectional correction configuration will be described later.

[0194] Furthermore, when FB information is obtained, the FB management unit F14 can generate an FB set containing the FB information. The term "FB set" may be read as "FB set data." The FB set will be explained with reference to Figure 35. Suppose that the i-th piece of FB information is obtained through an FB information acquisition process for an information notification process performed at a certain arbitrary timing of interest. The i-th piece of FB information is specifically referred to as FB information I. ＦＢ It is referred to as [i]. FB Information I ＦＢ[i] is the i-th FB information acquired from the initial state of the in-vehicle device 10. Each time FB information is acquired, the FB management unit F14 generates an FB set that includes that FB information. FB Information I ＦＢ FB sets that include [i] are specifically FB set S ＦＢ It is called [i]. FB Set S ＦＢ [i] is FB information I ＦＢ [i] and FB Information I ＦＢ It consists of input data Din and notification parameter Put corresponding to [i]. The FB information I is obtained by the FB information acquisition process for the information notification process performed at the attention timing. ＦＢ Suppose [i] is obtained. Then, FB information I ＦＢ The input data Din and notification parameter Put corresponding to [i] are the input data Din and notification parameter Put from the information notification process performed at the attention timing.

[0195] The FB management unit F14 stores and retains the generated FB set in a non-volatile memory area provided in the controller 11 or the recording medium 14. The FB management unit F14 can refer to the retained FB set at any time. Based on the referenced FB set, the FB management unit F14 can recognize, at a desired time, what kind of FB information was obtained through the information notification processing using what kind of input data Din and notification parameter Pout.

[0196] <<Example EX_2A>> Example EX_2A will now be explained. Before explaining the technology shown in Example EX_2A, we will briefly compare the pre-correction configuration and the post-correction configuration. For example, when target information is notified using the third or fourth notification format, if there is a user response that "the notification sound is too quiet," the post-correction configuration allows for increasing the volume through post-correction processing. In other words, correction that directly responds to the user response that "the notification sound is too quiet" is achieved. However, it is also possible that the user response that "the notification sound is too quiet" may occur due to high in-car noise. In this case, if the volume is increased through post-correction processing, and then the volume is uniformly increased through post-correction processing even when the in-car noise is low, the notification sound may become excessively loud.

[0197] In contrast, if a user responds that the notification sound is too quiet when the pre-correction process is used, the input data Din will be corrected. At this time, the estimated AI 100 estimates appropriate notification parameters by comprehensively considering not only the correction content of the input data Din in accordance with the user response (FB information) (for example, correction for a decrease in the hearing value), but also various data in the input data Din (for example, in-car noise). For this reason, it is expected that the pre-correction process will make it easier to set appropriate notification parameters according to the user U1's surrounding environment, etc., compared to the post-correction configuration.

[0198] However, the pre-correction process is less beneficial until a certain amount of FB information has been obtained. Considering this, the controller 11 in Example EX_2A operates first in a post-correction configuration, as shown in Figure 36, and then switches to a pre-correction configuration after the required amount of FB information has been obtained (however, it can also operate in a double-sided correction configuration). Figure 36 shows the mode flag F managed by the controller 11. ＭＯＤＥ This is shown. Mode flag F ＭＯＤＥ This is stored in a non-volatile memory provided within the controller 11. Mode flag F ＭＯＤＥ This is 1 bit of digital data, and the first or second value is the mode flag F ＭＯＤＥIt is set to this. Here, for the sake of explanation, the first value is assumed to be "1" and the second value to be "2". Mode flag F in the initial state of the in-vehicle device 10 ＭＯＤＥ It has a value of "1". The FB management unit F14 monitors whether the predetermined mode switching conditions are met, and when the mode switching conditions are met, the mode flag F ＭＯＤＥ Switch the value from "1" to "2", and thereafter, mode flag F ＭＯＤＥ Maintain the value at "2".

[0199] There are two operating modes for setting the notification parameter Poout: a first mode and a second mode. The operating mode for setting the notification parameter Poout will be hereinafter referred to as the operating mode of the controller 11, or simply as the operating mode. The operating mode is "F ＭＯＤＥ When = 1'' (i.e., mode flag F ＭＯＤＥ When the value of is "1", it is the first mode, and "F ＭＯＤＥ When = 2'' (i.e., mode flag F ＭＯＤＥ The second mode is when the value of is "2". The FB management unit F14 sets the mode flag F ＭＯＤＥ By managing the value of , the operating mode is set to either the first or second mode. When the operating mode is the first mode, the controller 11 is sometimes referred to as the controller 11 in the first mode. When the operating mode is the second mode, the controller 11 is sometimes referred to as the controller 11 in the second mode.

[0200] In the first mode, the controller 11 operates in a post-processing configuration. In the first mode, the pre-processing unit F12 does not function. That is, in the first mode, the controller 11 inputs the input data Din to the estimated AI 100 and sets the notification parameter Pout by performing a post-processing operation on the output parameter Dout output from the estimated AI 100. In the second mode, the controller 11 operates in a pre-processing configuration. That is, in the second mode, the controller 11 generates corrected input data CDin by performing a pre-processing operation on the input data Din and inputs the corrected input data CDin to the estimated AI 100. Then, in the second mode, the controller 11 sets the notification parameter Pout according to the output parameter Dout output from the estimated AI 100 based on the corrected input data CDin. However, the controller 11 in the second mode may operate in a bi-sided correction configuration. Therefore, in the second mode, the notification parameter Pout may be generated by applying a post-processing operation to the output parameter Dout (details will be described later). By switching modes to toggle pre-correction processing and post-correction processing on or off, it becomes possible to perform appropriate corrections depending on the situation.

[0201] Figure 36 also shows the cumulative number of acquisitions SUM managed by the controller 11. ＦＢ This is shown. Cumulative number of acquisitions SUM ＦＢ This is stored in a non-volatile memory provided within the controller 11. Cumulative acquisition count SUM ＦＢ This represents the total number of FB information acquired by the FB management unit F14. Therefore, in the initial state of the in-vehicle device 10, the cumulative number acquired is SUM. ＦＢ It is "0".

[0202] Figure 37 is an operation flowchart of the controller 11 focusing on the information notification process according to Embodiment EX_2A. The on-board device 10 starts in conjunction with the start of the engine mounted on the vehicle V1 by operating the ignition switch of the vehicle V1. Once the on-board device 10 starts, the process proceeds to step S10. In step S10, the FB management unit F14 sets the current mode flag F ＭＯＤＥ Refer to mode flag F ＭＯＤＥBased on the value, the operating mode is set to either the first mode or the second mode. After step S10, the process proceeds to step S11. After proceeding to step S11, the notification management unit F3 continuously performs the notification determination process described above, continuously and repeatedly determining whether any information (any of the first to nth notification information) should be notified to user U1 (see Figure 13). If it is determined in step S11 that any information should be notified to user U1 (Y in step S11), the process proceeds to step S12. Otherwise, the process returns to step S11 and repeats the determination of whether any information (any of the first to nth notification information) should be notified to user U1.

[0203] After proceeding to step S12, each process from steps S12 to S16 is executed before proceeding to step S17. The contents of each process from steps S12 to S17 are as shown in Example EX_1A (see Figure 22). However, in step S14, “F ＭＯＤＥ When = 1, the notification parameter Put is set by the controller 11 in the first mode. In step S14, "F ＭＯＤＥ When the value is 2, the notification parameter Put is set by the controller 11 in the second mode.

[0204] In step S16, the FB management unit F14 performs the FB information acquisition process, and in the following step S17, it is confirmed whether the FB information has been acquired. If the FB information has been acquired (step S17 Y), the process proceeds to step S18; however, if the FB information has not yet been acquired (step S17 N), the process proceeds to step S19. In step S19, the FB management unit F14 checks whether the elapsed time since the notification of the most recent target information has reached a predetermined FB reception time. If the elapsed time has not reached the FB reception time (step S19 N), the process returns to step S17. If the elapsed time reaches the FB reception time without the FB information being acquired (step S19 Y), the FB management unit F14 determines that it has failed to acquire the FB information for the most recent target information and initiates a transition to step S21.

[0205] In step S18, the FB management unit F14 performs correction content adjustment processing. The correction content adjustment processing in step S18 is the same as that shown in Example EX_1A or EX_1B. ＭＯＤＥ The correction content adjustment process when = 1" is the correction content adjustment process for the subsequent correction process described in Example EX_1B. ＭＯＤＥ The correction content adjustment process when = 2" is the correction content adjustment process for the preceding correction process described in Example EX_1A. In steps S14 and S15, which follow step S18, the notification parameter Pout is set by correcting the input data Din or output parameter Dout according to the correction content adjusted in the correction content adjustment process of step S18.

[0206] In Example EX_2A, after step S18, the process proceeds to step S20. In step S20, the FB management unit F14 calculates the cumulative number of acquired FB information SUM ＦＢ Add "1" to it. After step S20, proceed to step S21. In step S21, the FB management unit F14 checks whether the current operating mode is the first mode, i.e., "F ＭＯＤＥ Check if it is = 1". If the current operating mode is the first mode (Y in step S21), proceed from step S21 to step S22. If the current operating mode is not the first mode but the second mode (N in step S21), return from step S21 to step S11.

[0207] In step S22, the FB management unit F14 determines whether the predetermined mode switching conditions are met. Here, the cumulative number acquired SUM ＦＢ The success or failure of the mode switching condition is determined based on a comparison with the threshold TH1. The threshold TH1 has a predetermined positive integer value (for example, 100). The FB management unit F14 says, "SUM ＦＢ When the condition ≥TH1 is met, it is determined that the mode switching condition is met, and "SUM ＦＢWhen ≥TH1” is not met, it is determined that the mode switching condition is not met. If it is determined that the mode switching condition is met (Y in step S22), the process proceeds from step S22 to step S23. If it is determined that the mode switching condition is not met (N in step S22), the process returns from step S22 to step S11. In step S23, the FB management unit F14 sets the mode flag F ＭＯＤＥ By setting this to "2", the operating mode of the controller 11 is switched from the first mode to the second mode. Once the operating mode of the controller 11 is switched to the second mode, the operating mode of the controller 11 is fixed in the second mode. Also in step S23, the FB management unit F14 performs FB reflection processing.

[0208] In the FB reflection process, FB information during the first mode period is reflected in the preceding correction process. This will be explained further. For convenience, as shown in Figure 38, the period during which the operation mode of the controller 11 is set to the first mode is referred to as the first mode period, and the period during which the operation mode of the controller 11 is set to the second mode is referred to as the second mode period. The mode switches from the first mode period to the second mode period at the time the mode switching condition is met. During the first mode period, each time FB information is acquired, an FB set (Figure 35) is generated that includes the FB information and the corresponding input data Din and notification parameter Pout. Based on each FB set, the FB management unit F14 can recognize what kind of FB information was obtained through what kind of information notification processing using what kind of input data Din and notification parameter Pout. In the FB reflection process of step S23, the FB management unit F14 performs correction content adjustment processing for the preceding correction process based on each FB set generated during the first mode period. In other words, in the FB reflection process, FB information acquired when the operating mode is the first mode is treated as FB information acquired when the operating mode is the second mode, and then adjustment processing for the correction content of the preceding correction process is performed based on each generated FB set.

[0209] For example, during the first mode period, in case CS2 shown in Figure 18(b), the target information 720 is notified, and when the FB button 722 is selected, FB information 728 indicating "display is too large" is sent FB information I ＦＢ Let's consider the case where it is acquired as [1]. In this case, in the FB reflection process, as shown in Figure 24(a), the FB management unit F14 increases the correction amount ΔCin

[12] corresponding to visual acuity by the reference change amount. Conversely, for example, in the first mode period, if the target information 720 is notified in case CS2 and the FB button 723 is selected, FB information 728 indicating "the display is too small" is FB information I ＦＢ Let's consider the case where it is acquired as [1]. In this case, in the FB reflection process, as shown in Figure 24(b), the FB management unit F14 reduces the correction amount ΔCin

[12] corresponding to visual acuity by the reference change amount. Also, for example, in the first mode period, when the target information 720 is notified in case CS2 and the FB button 724 is selected, FB information 728 indicating "no notification needed" is sent to FB information I ＦＢ Let's consider the case where it is acquired as [1]. In this case, in the FB reflection process, as shown in Figure 24(d), the FB management unit F14 increases the correction amount ΔCin

[23] corresponding to the driver skill by the reference change amount. FB Information I ＦＢ The same applies when [1] is FB information other than FB information 728 (for example, the FB information 718, 738 or 748 mentioned above) (see Figures 18(a), (c) and (d)). FB information acquired during the first mode period and FB information I ＦＢ The same applies to FB information other than [1].

[0210] After the processing in step S23 is completed, the process returns to step S11. In step S14, which follows the processing in step S23, the notification parameter Poout is set through a pre-correction process, and the content of this pre-correction process is based on each FB set generated during the first mode period. In other words, the pre-correction process that reflects the FB information from the first mode period is performed during the second mode period. ＦＢThe same pre-correction processing as in the case where the system had been operating continuously in the second mode since the point of =0" is performed during the second mode period. Note that in the flowchart of Figure 37, the FB reflection processing is performed after the mode switching condition is met. However, it is also possible to maintain the operating mode in the first mode (see Figure 36) before the mode switching condition is met and perform only the FB reflection processing.

[0211] Thus, the FB management unit F14 is in a state where at least FB information has not been acquired (i.e., "SUM" ＦＢ When the state is = 0'', the operating mode is set to the first mode. After that, the FB management unit F14 calculates the cumulative number of acquired FB information SUM ＦＢ The operating mode is switched from the first mode to the second mode accordingly. This allows the cumulative number of acquired SUM ＦＢ When the value is too low and the benefits of the pre-correction process are not easily increased, the post-correction process can be used in the first mode to achieve correction that directly responds to the user's reaction. And the cumulative number acquired SUM ＦＢ Once this increases to a certain extent, the benefits of the pre-processing correction (optimization of notification parameter Out according to the user U1's surrounding environment, etc.) can be enjoyed by switching to the second mode. Cumulative acquisition count SUM ＦＢ Furthermore, the comparison between thresholds TH1 can be used to determine the switching between the first mode and the second mode (step S22).

[0212] Furthermore, the FB reflection process in step S23 determines the content of the pre-correction process during the second mode period based on the FB set (FB information, input data Din, and notification parameter Pot) during the first mode period. In other words, after switching to the second mode, the pre-correction process is executed with the previous FB information taken into account. Therefore, after switching to the second mode, it becomes possible to enjoy the benefits of the pre-correction process while simultaneously setting notification parameter Pot that is suitable for the user U1's personal characteristics (preferences, etc.) immediately after switching to the second mode. Even after switching to the second mode, if FB information is acquired, it is possible to perform correction content adjustment processing based on the FB information.

[0213] <<Example EX_2B>> Example EX_2B will be explained. The FB management unit F14 is the cumulative acquisition count SUM ＦＢ In addition, other information may be considered to determine whether the above mode switching conditions are met (see step S22 in Figure 37). Other information includes the frequency of FB information acquisition. The frequency of FB information acquisition refers to the number of FB information acquisitions per unit time. The unit time has a fixed length. A period of time equal to the length of the unit time is called a unit period.

[0214] The FB management unit F14 counts the number of FB information acquired during each unit period, that is, it repeatedly counts the number of FB information acquired per unit time. Acquisition frequency data representing the result of this counting is generated by the FB management unit F14 and stored in the non-volatile memory of the controller 11. For example, when the first unit period to the Mth unit period occur sequentially (M is an integer of 2 or more), the FB management unit F14 counts the number of FB information acquired in each of the first to Mth unit periods. The number of FB information acquired in the ith unit period is represented by the symbol "E[i]". Then, at the end of the Mth unit period, the number of FB information acquired from the first to the Mth unit periods, E[1] to E[M], are stored in the acquisition frequency data. The acquisition number E[i] corresponds to the acquisition frequency of FB information in the ith unit period, and the acquisition numbers E[1] to E[M] indicate the time-series change in the acquisition frequency of FB information. Cumulative acquisition count SUM ＦＢ If this represents the total number of FB pieces of information acquired during the first period, then each of the first to the Mth period is a part of the first period.

[0215] In step S22, the FB management unit F14 calculates the cumulative number of acquisitions SUM ＦＢ Based on the acquired frequency data, it is possible to determine whether the mode switching condition has been met. As methods for processing step S22 that can be executed after the end of the Mth unit period and before the end of the (M+1)th unit period, methods MTD_2Ba to MTD_2Bc are listed.

[0216] The FB management unit F14 related to method MTD_2Ba is “SUM ＦＢThe mode switching condition is determined to be met only if ≥TH1 is true and the number of acquired E[M] is greater than or equal to a predetermined value. The FB management unit F14 related to method MTD_2Bb is “SUM ＦＢ The condition ≥TH1 is met and the number of acquired items E[M-k] is met. Ａ The mode switching condition is determined to be met only if the average or median of ] to E[M] is greater than or equal to a predetermined value. Here, k Ａ represents an integer greater than or equal to 1 and “M-k”. Ａ Assume that ≥ 2'' holds true. As an extreme example, the cumulative number of acquisitions SUM ＦＢ Consider the case where the next FB information is obtained 5 years after the value (TH1-1) has been reached. In this case, “SUM ＦＢ Although ≥TH1” is true, there is a high possibility that the FB information acquired five years ago does not match the current personal characteristics of user U1. By using method MTD_2Ba or MTD_2Bb, the acquisition information of FB information around the time the processing in step S22 is executed is referenced, making a reasonable transition to the second mode possible.

[0217] Furthermore, the FB management unit F14 related to method MTD_2Bc is “SUM ＦＢ The mode switching condition is determined to be met only if ≥TH1" is true and the average or median of the acquired values ​​E[1] to E[M] is greater than or equal to a predetermined value. This makes it possible to suppress the transition to the second mode in usage patterns where FB information is acquired infrequently. In usage patterns where FB information is acquired infrequently, it cannot be said that the acquisition of FB information is sufficient, and it is considered that the post-processing correction is likely to work more favorably than the pre-processing correction.

[0218] <<Example EX_2C>> Example EX_2C will now be described. In the FB management unit F14, FB information (hereinafter referred to as specific FB information) indicating the user U1's response to a notification of specific conditions may be acquired. The specific FB information represents the user U1's opinion to the notification of specific conditions.

[0219] For example, refer to Figure 39, timing t Ａ１In the information notification process, consider case CS_2Ca in which forward obstacle information is notified to user U1 as target information 810 using a third or fourth notification format. Here, timing t Ａ１ Let be any timing during the second mode period. However, timing t Ａ１ This timing may occur during the first mode period. In case CS_2Ca, suppose user U1 inputs the opinion that the "sound is too loud" regarding the notification of forward obstacle information to the in-vehicle device 10 via touch panel operation or voice operation. In case CS_2Ca, the FB management unit F14, timing t Ａ１ Specific FB information 818 is acquired for the information notification process. In the specific FB information 818 in case CS_2Ca, the notification of specific conditions is the notification of forward obstacle information. For example, if user U1 says "That sound is too loud" immediately after speaker 52 outputs a notification sound representing forward obstacle information, specific FB information 818 is acquired. Also, for example, if user U1 inputs information to the in-vehicle device 10 indicating that the volume of the notification sound representing forward obstacle information is excessive, specific FB information 818 is acquired. Specific FB information 818 indicates that the volume of the notification sound representing forward obstacle information is excessive.

[0220] In case CS_2Ca, timing t Ａ１ later timing t Ａ２ At this point, an information notification process is executed to notify user U1 of the target information 820. Timing t Ａ２ This is the timing during the second mode period. When user U1 expresses an opinion after specifying some conditions for the notification, and a second notification is made under the specified conditions, it is more effective to perform parameter correction in a subsequent correction process to align with the user's opinion. Therefore, in case CS_2Ca, depending on whether the notification of target information 820 falls under the notification of specific conditions, the subsequent correction process is executed or not executed when setting the notification parameter Poout of target information 820. Note that timing t Ａ１ and t Ａ２ Each of these is understood to be a concept with a length of approximately the time required to notify the target information. Ａ１We consider this to be the first period, timing t Ａ２ This can also be considered a second period, following the first period.

[0221] Specifically, in case CS_2Ca, if the target information 820 is forward obstacle information and the target information 820 is notified to user U1 through the output of a notification sound, the notification of target information 820 is a notification that meets a specific condition. If the notification of target information 820 is a notification that meets a specific condition, the notification parameter Put is set by a bilateral correction configuration. At this time, based on the specific FB information 818, under the control of the FB management unit F14, the notification parameter Put of target information 820 is set by performing a subsequent correction process with the correction amount ΔCoout[5] corresponding to the volume set to "-1". In other words, timing t Ａ２ The output parameter Dout, derived by the estimated AI 100 based on the correction input data CDin, is corrected by "ΔCout[5] = -1" to determine the notification parameter Poout of the target information 820. If the FB information 818 was a specific FB information stating "the current sound is too quiet," then the polarity of the correction amount ΔCout[5] would be the opposite of that described above.

[0222] In case CS_2Ca, if the target information 820 is different from the forward obstacle information, the notification of target information 820 is a notification that does not meet the specific conditions. In case CS_2Ca, if the notification of target information 820 is not notified to user U1 through the output of a notification sound (when the third and fourth communication formats are OFF), the notification of target information 820 is also a notification that does not meet the specific conditions. If the notification of target information 820 is a notification that does not meet the specific conditions, the notification parameter Put is set in the pre-correction configuration as per the principle of the second mode. Therefore, if the notification of target information 820 is a notification that does not meet the specific conditions, timing t Ａ２ Based on the corrected input data CDin, the output parameter Dout derived by the estimated AI 100 is set directly as the notification parameter Poout of the target information 820.

[0223] According to the method shown in Example EX_2C, when a notification matching the conditions specified by user U1 is received, the parameter correction can be reliably performed in a manner consistent with user U1's intentions during the subsequent correction process.

[0224] In the above-described case CS_2Ca, it is assumed that the target information 810 is forward obstacle information, but the content of the target information 810 is arbitrary. In the above-described case CS_2Ca, timing t Ａ１ It is assumed that the information notification process in this case will be carried out using the third or fourth notification format, but timing t Ａ１ The information notification process in this context may be carried out using the first, second, or fifth notification format.

[0225] For example, timing t Ａ１ When information notification processing is performed using the first or second notification format, if user U1 says "The current display is too large," specific FB information 818 indicating that the notification image of the forward obstacle information is too large is acquired. In this case, the notification of the target information 820 meets the specific condition only if the target information 820 is forward obstacle information and the target information 820 is notified to user U1 through the display of the notification image. If the notification of the target information 820 meets the specific condition, the notification parameter Pot of the target information 820 is set by performing a subsequent correction process based on the specific FB information 818, under the control of the FB management unit F14, with the correction amount ΔCout[1] corresponding to the display size set to "-1".

[0226] Also, for example, timing t Ａ１When the information notification process is performed using the fifth notification format, if user U1 says "The vibration is too strong now," specific FB information 818 indicating that the notification vibration for forward obstacle information is too strong is acquired. In this case, the notification of target information 820 meets the specific conditions only if the target information 820 is forward obstacle information and the target information 820 is notified to user U1 through the output of the notification vibration. If the notification of target information 820 meets the specific conditions, the notification parameter Pout of the target information 820 is set by performing a subsequent correction process based on the specific FB information 818, under the control of the FB management unit F14, with the correction amount ΔCout[8] corresponding to the vibration intensity set to "-1".

[0227] Furthermore, after performing multiple information notification processes using the display of a notification image, output of a notification sound, and output of a notification vibration, timing t Ａ１ If user feedback to the effect of "all notification strengths are strong" is obtained as specific FB information, then a bilateral correction configuration may be permanently adopted thereafter. In this case, timing t Ａ２ In the information notification processing, the notification parameter Pout of the target information 820 may be set by performing a subsequent correction process with each of the correction amounts ΔCoout[1], ΔCoout[5], and ΔCoout[8] set to "-1".

[0228] <<Example EX_2D>> Example EX_2D will be described. The operating mode of the controller 11 may be fixed to the second mode at all times without switching the operating mode of the controller 11. In this case, the method shown in Example EX_2C may be implemented. That is, as in case CS_2Ca of Figure 39, timing t Ａ１ After specific FB information 818 is obtained in response to the notification of target information 810, timing t Ａ２When notifying the target information 820, it is determined whether the notification of the target information 820 meets specific conditions. If the notification of the target information 820 meets the specific conditions, the notification parameter setting unit F1 sets the notification parameter Put for the target information 820 with both the pre-correction process and the post-correction process turned ON. In this case, the content of the post-correction process performed based on the specific FB information 818 is as shown in Example EX_2C. Conversely, if the notification of the target information 820 does not meet the specific conditions, the notification parameter setting unit F1 sets the notification parameter Put for the target information 820 with the pre-correction process turned ON and the post-correction process turned OFF.

[0229] Alternatively, after specific FB information has been acquired, if a notification is to be issued that meets specific conditions, the pre-correction process may be turned OFF while the post-correction process is turned ON. If a notification is to be issued that does not meet specific conditions, the pre-correction process may be turned ON while the post-correction process is turned OFF.

[0230] <<Example EX_3A>> Example EX_3A will be described. The controller 11 can retrain the learning model 110 using FB information, thereby reconstructing the estimated AI 100 to suit the individual characteristics of user U1. Retraining can be achieved using the basic learning data LDa (Figure 19) described above and additional learning data based on FB information.

[0231] Figure 40 shows the internal configuration of the notification parameter setting unit F1 employing a pre-correction configuration, and the peripheral configuration related to relearning. Figure 41 shows the internal configuration of the notification parameter setting unit F1 employing a post-correction configuration, and the peripheral configuration related to relearning.

[0232] The storage units 142 and 143 are provided on the recording medium 14. The storage unit 142 stores the same basic learning data LDa as the basic learning data LDa stored in the database 310 in Figure 19. The storage unit 143 is a storage area for storing additional learning data LDb.

[0233] The FB management unit F14 can generate unit training data UDb based on the FB information it has acquired, and can store one or more unit training data UDb in the storage unit 143. As the total number of acquired FB information increases during the actual operation process, the total number of unit training data UDb increases. Additional learning data LDb is formed from one or more unit training data UDb stored in the storage unit 143. In the initial state of the in-vehicle device 10, no unit training data UDb is stored in the storage unit 143, and unit training data UDb is added to the storage unit 143 during the actual operation process. Note that the storage unit 142 may be provided in the database 310 (Figure 19). In this case, the controller 11 can freely read basic learning data LDa from the database 310 via the communication unit 13.

[0234] In addition to the above-mentioned functional blocks, the notification parameter setting unit F1 is provided with a functional block F15. Functional block F15 is a retraining processing unit. The retraining processing unit F15 can perform retraining processing after the unit training data UDb has been stored in the memory unit 143. In the retraining processing, the retraining processing unit F15 retrains the learning model 110 (and therefore the estimated AI 100) using the basic learning data LDa and additional learning data LDb stored in the memory units 142 and 143.

[0235] Regardless of whether a pre-correction configuration or a post-correction configuration is adopted for the notification parameter setting unit F1, or whether a bilateral correction configuration is adopted, the method for generating unit training data UDb is the same. However, the method for generating unit training data UDb when the FB information is positive FB information and the method for generating unit training data UDb when the FB information is negative FB information are different. These generation methods will be explained separately.

[0236] --Method for generating unit training data UDb based on positive FB information-- First, with reference to Figure 42, the method for generating unit training data UDb based on positive FB information will be explained. Assume a situation in which target information 700p is notified to user U1 during information notification processing. To make the explanation more concrete, the notification parameter Pout of the target information 700p will be called notification parameter 707p, and the input data Din used to set notification parameter 707p will be called input data 706p. That is, when input data 706p is input to the notification parameter setting unit F1 as input data Din, notification parameter 707p is set as notification parameter Pout in the notification parameter setting unit F1, and target information 700p is notified to user U1 according to notification parameter 707p.

[0237] It is assumed that in response to the notification of target information 700p in accordance with notification parameter 707p, positive FB information 708p was acquired by the FB management unit F14. For example, if target information 700p is target information 710, then FB information 718 in case CS1c corresponds to positive FB information 708p (see Figures 27(c) and 34(c)). Similarly, if target information 700p is target information 720, then FB information 728 in case CS2c corresponds to positive FB information 708p (see Figures 24(c) and 31(c)). Similarly, if target information 700p is target information 730, then FB information 738 in case CS3c corresponds to positive FB information 708p (see Figures 25(c) and 32(c)). Similarly, for example, if the target information 700p is the same as the target information 740, then the FB information 748 in case CS4c corresponds to the positive FB information 708p (see Figure 26(c)).

[0238] As shown in Figure 42, the FB management unit F14 generates unit training data 709p based on the input data Din corresponding to the positive FB information 708p, and the notification parameter Pout, which is input data 706p and notification parameter 707p. Unit training data 709p is unit training data UDb generated based on the positive FB information 708p, and is added to the additional training data LDb (see Figures 40 and 41). Unit training data 709p has input data 706p and notification parameter 707p. In the retraining process, the learning model 110 that constitutes the estimated AI 100 is retrained. The input data 706p in the unit training data 709p functions as input data (training input data) to the learning model 110 in the retraining process. The notification parameter 707p in the unit training data 709p functions as correct data for the inference result of the learning model 110 in the retraining process. Therefore, the retraining process makes it possible to reconstruct the estimated AI 100 in line with the evaluation (preferences) of user U1.

[0239] As the information notification process is executed repeatedly, the FB management unit F14 may generate one unit of training data UDb each time positive FB information is acquired and add it to the additional training data LDb. The timing of generating the unit of training data UDb based on positive FB information is arbitrary. The unit of training data UDb may be generated immediately upon obtaining the positive FB information that will serve as the basis for the unit of training data UDb, or it may be generated at any subsequent timing.

[0240] --Method for generating unit training data UDb based on negative FB information-- Next, with reference to Figure 43, the method for generating unit training data UDb based on negative FB information will be explained. Assume a situation in which target information 700n is notified to user U1 in the information notification process. For the sake of detail, the notification parameter Pout of the target information 700n will be called notification parameter 707n, and the input data Din used to set notification parameter 707n will be called input data 706n. That is, when input data 706n is input to the notification parameter setting unit F1 as input data Din, notification parameter 707n is set as notification parameter Pout in the notification parameter setting unit F1, and target information 700n is notified to user U1 according to notification parameter 707n.

[0241] It is assumed that in response to the notification of target information 700n in accordance with notification parameter 707n, negative FB information 708n is acquired by the FB management unit F14. For example, if target information 700n is target information 710, then FB information 718 in case CS1a, CS1b, or CS1d corresponds to negative FB information 708n (see Figures 27(a), (b), and (d) and Figures 34(a) and (b)). Similarly, for example, if target information 700n is target information 720, then FB information 728 in case CS2a, CS2b, or CS2d corresponds to negative FB information 708n (see Figures 24(a), (b), and (d) and Figures 31(a) and (b)). Similarly, for example, when the target information 700n is the target information 730, the FB information 738 in case CS3a, CS3b, or CS3d corresponds to the negative FB information 708n (see Figures 25(a), (b), and (d) and Figures 32(a) and (b)). Similarly, for example, when the target information 700n is the target information 740, the FB information 748 in case CS4a, CS4b, or CS4d corresponds to the negative FB information 708n (see Figures 26(a), (b), and (d) and Figures 33(a) and (b)).

[0242] As shown in Figure 43, the FB management unit F14 generates unit training data 709n based on the input data Din corresponding to the negative FB information 708n, and the input data 706n and notification parameter 707n, which are notification parameter Pout. The unit training data 709n is unit training data UDb generated based on the negative FB information 708n, and is added to the additional training data LDb (see Figures 40 and 41). The unit training data 709n has input data 706n and notification parameter 707n'. The FB management unit F14 processes the notification parameter Pout as notification parameter 707n according to the negative FB information 708n. The processed notification parameter Pout is called notification parameter Pout'. Notification parameter Pout' is stored in the unit training data 709n as notification parameter 707n'. In the retraining process, the learning model 110 that constitutes the estimated AI 100 is retrained. The input data 706n in the unit training data 709n functions as input data (training input data) to the learning model 110 during the retraining process. The notification parameters 707n' (processed notification parameters) in the unit training data 709n function as correct data for the inference results of the learning model 110 during the retraining process. Therefore, the retraining process makes it possible to reconstruct the estimated AI 100 in line with the evaluation (preferences) of user U1.

[0243] An example of a processing process for obtaining notification parameter Pout' from notification parameter Pout is given. For example, if target information 700n is target information 720 and FB information 728 in case CS2a is negative FB information 708n, then in the processing process, the parameter PR in notification parameter 707n Ａ１ The value of is reduced by 1 (see Figures 24(a), 31(a), and 12). Conversely, for example, if the target information 700n is target information 720 and the FB information 728 in case CS2b is negative FB information 708n, the parameter PR in the notification parameter 707n is processed in the processing step. Ａ１The value of is increased by 1 (see Figures 24(b), 31(b), and 12). Also, for example, if the target information 700n is target information 730 and the FB information 738 in case CS3a is negative FB information 708n, the parameter PR in the notification parameter 707n is processed in the processing step. Ｂ１ The value of is reduced by 1 (see Figures 25(a), 32(a), and 12). Conversely, for example, if the target information 700n is target information 730 and the FB information 738 in case CS3b is negative FB information 708n, the parameter PR in the notification parameter 707n is processed in the processing step. Ｂ１ The value of is increased by 1 (see Figures 25(b), 32(b), and 12). Also, for example, if the target information 700n is target information 740 and the FB information 748 in case CS4a is negative FB information 708n, the parameter PR in the notification parameter 707n is processed in the processing step. Ｃ１ The value of is reduced by 1 (see Figures 26(a), 33(a), and 12). Conversely, for example, if the target information 700n is target information 740 and the FB information 748 in case CS4b is negative FB information 708n, the parameter PR in the notification parameter 707n is processed in the processing step. Ｃ１ The value of is increased by 1 (see Figures 26(b), 33(b), and 12). The same processing should be performed if the target information 700n is the target information 710 (see Figure 27(a), etc.).

[0244] However, if the value of the parameter to be processed exceeds the upper limit or falls below the lower limit of the parameter after processing, the processing will not be performed (it will be prohibited). For example, the parameter PR to be processed Ａ１ Since the upper limit is 5 (see Figure 12), the parameter PR in the notification parameter 707n Ａ１ If the value of is 5, then the parameter PR Ａ１ Processing that increases the value of is not performed. If processing is not performed, the desired unit training data 709n cannot be generated, and therefore the generation of unit training data 709n is prohibited.

[0245] In addition, the notification parameters 707n can be modified according to the negative FB information 708n so that the preferences of user U1 are reflected. For example, if negative FB information 708n is obtained indicating a user opinion that the notification was too late, the parameter PR in the notification parameters 707n can be modified during the processing. Ｄ１ It is possible to decrease the value of by 1.

[0246] As the information notification process is executed repeatedly, the FB management unit F14 may generate one unit of training data UDb each time negative FB information is acquired and add it to the additional training data LDb. The timing of generating the unit of training data UDb based on negative FB information is arbitrary. The unit of training data UDb may be generated immediately upon obtaining the negative FB information that will serve as the basis for the unit of training data UDb, or it may be generated at any subsequent timing.

[0247] --Retraining Process-- The retraining process will now be explained. Retraining the learning model 110 means that the learning model 110, which was formed by pre-machine learning using the basic learning data LDa, is subjected to machine learning again using the basic learning data LDa and the additional learning data LDb. The model that is the target of the retraining process is called the target model. The target model before the retraining process is a model for which pre-machine learning has not been performed, and corresponds to the learning model 110 with initialized parameters (weights and biases). Therefore, strictly speaking, retraining the learning model 110 means that the target model is subjected to machine learning using the basic learning data LDa and the additional learning data LDb. Thus, in the retraining process, the input data to the learning model 110 (training input data) refers to the input data to the target model, and the correct data for the inference results of the learning model 110 refers to the correct data for the inference results of the target model. The target model before the retraining process is formed by a DNN with predetermined initial parameters (weights and biases).

[0248] In the retraining process, the retraining processing unit F15 extracts input data Din from each unit of training data (UDa or UDb) and inputs the extracted input data Din into the target model. The pre-processing unit F12 does not function during the retraining process. During the retraining process, the target model estimates appropriate notification parameters for notifying user U1 of target information based on the input data Din. This estimation by the target model is called inference. The notification parameters obtained by the target model's inference have the same structure as the notification parameter Put. During the retraining process, the target model's inference is performed for each unit of training data UDa in the basic training data LDa and for each unit of training data UDb in the additional training data LDb. During the retraining process, the retraining processing unit F15 derives the error between the notification parameters derived by the target model's inference and the corresponding ground truth data for each unit of training data (UDa, UDb), and performs machine learning to update the target model's parameters to reduce the error. The parameters of the target model include the weights and biases of the DNN that constitute the target model. Machine learning is performed until the error becomes sufficiently small, or for a specified number of epochs. The target model after the completion of machine learning by the retraining process is the machine-learned model 110 (hereinafter referred to as the retrained machine-learned model 110 or the machine-learned model 110 after the retraining process, etc.).

[0249] After the retraining process, the estimated AI 100 is constructed using the retrained learning model 110, and the notification parameters are inferred using the retrained learning model 110, with the inference results derived as the output parameter Dout (the same applies to any embodiment described later). Ideally, the retrained learning model 110 is expected to derive notification parameters Pout that match the preferences of user U1 as the output parameter Dout. Therefore, ideally, the pre-correction process and post-correction process should not be necessary immediately after the retraining process. For this reason, the FB management unit F14 should initialize the pre-correction process and post-correction process when the retraining process is executed.

[0250] <<Example EX_3B>> Example EX_3B will now be described. Example EX_3B is implemented in combination with any of the above-mentioned Examples EX_2A to EX_2D, and the retraining process shown in Example EX_3A is applied to any of Examples EX_2A to EX_2D.

[0251] Figure 44 is an operation flowchart of the controller 11 involved in information notification processing in Example EX_3B. The operation of the controller 11 in Example EX_3B includes the processing in steps S10 to S23 and the processing in steps S31 to S33. The processing content of steps S10 to S23 in Example EX_3B is the same as that shown in Example EX_2A (Figure 37). However, in step S11 in Example EX_3B, if it is determined that some information should be notified to user U1 (Y in step S11), the process proceeds to step S12; otherwise, it proceeds to step S31. Note that in step S20, the cumulative number of acquired FB information SUM ＦＢ "1" is added to the value, and unit training data UDb is generated according to the acquired FB information. The method for generating unit training data UDb is as described above. The generated unit training data UDb is added to the additional training data LDb as described above (see Figures 40 and 41).

[0252] In step S31, the retraining processing unit F15 checks whether the predetermined retraining conditions are met. If the retraining conditions are met (Y in step S31), the process proceeds to step S32; if the retraining conditions are not met (N in step S31), the process returns to step S11. In step S32, the retraining processing unit F15 executes the retraining process. The retraining processing unit F15, which is involved in the retraining process, retrains the learning model 110 (and therefore the estimated AI 100) using the basic learning data LDa and additional learning data LDb stored in the memory units 142 and 143. During the execution period of retraining, the controller 11 is unable to perform information notification processing. For this reason, the fulfillment of the retraining conditions may be required, such as the user U1 getting out of the vehicle V1. Also, meaningful retraining cannot be achieved unless a certain amount of FB information is secured. For this reason, the acquisition of a certain amount of FB information may also be required for the fulfillment of the retraining conditions.

[0253] Specifically, the relearning condition is met only when both the following temporal and numerical conditions are satisfied. The temporal condition is that user U1 disembarks from vehicle V1. The relearning processing unit F15 can determine whether user U1 has disembarked from vehicle V1 based on the engine operating status and the detection result of the seating sensor. The seating sensor here is a sensor installed on seat ST1 that detects whether user U1 is sitting on seat ST1 or not. However, the condition that the engine of vehicle V1 stops may also be the temporal condition. The numerical condition is the cumulative number of acquired FB information SUM ＦＢ The condition may be that is greater than or equal to the threshold TH2. The threshold TH2 has a predetermined positive integer value. The threshold TH2 is greater than the threshold TH1 referenced in step S22. However, it may also be "TH1 = TH2". By defining thresholds TH1 and TH2 such that "TH1 ≤ TH2" holds, the relearning condition is met only after the mode switching condition is met.

[0254] Furthermore, after the retraining process has been performed at least once in the past, it is desirable that the numerical condition be deemed satisfied when the total number of FB information acquired by the FB management unit F14 since the last retraining process is equal to or greater than the threshold TH2. If the retraining process can be completed in a sufficiently short time, it is also possible to remove the time condition and set the numerical condition itself as the retraining condition.

[0255] After the relearning process in step S32, in step S33, the controller 11 (e.g., the relearning processing unit F15) initializes at least the pre-correction process. Initialization of the pre-correction process returns it to a state where correction is not performed, and therefore, immediately after the initialization of the pre-correction process, the input data CDin perfectly matches the input data Din. Immediately before the relearning process in step S32, the post-correction process may be functioning effectively (i.e., the controller 11 may be operating in a bi-sided correction configuration). In this case, in step S33, the controller 11 (e.g., the relearning processing unit F15) also initializes the post-correction process. Initialization of the post-correction process returns it to a state where correction is not performed, and therefore, immediately after the initialization of the post-correction process, the notification parameter Pout perfectly matches the output parameter Dout. Note that the execution order of steps S32 and S33 when relearning is performed is arbitrary, and steps S32 and S33 may be performed simultaneously. Once steps S32 and S33 are completed, the flowchart in Figure 44 ends. Then, the next time the engine of vehicle V1 is started, the flowchart in Figure 44 resumes from step S10.

[0256] In step S32, the estimated AI 100 is reconstructed by referring to the additional training data LDb during the retraining process. In the operation flowchart of Figure 44, the retraining process is executed only after the operation mode of the controller 11 switches from the first mode to the second mode. Therefore, after the execution of the retraining process, the reconstructed estimated AI 100 (the retrained learning model 110) performs inference of notification parameters in the second mode, and the inference result is derived as the output parameter Dout.

[0257] By retraining using the basic training data LDa and the additional training data LDb generated considering FB information, the user U1's preferences are reflected in the estimated AI 100 during the retraining process. As a result, it is expected that the estimated AI 100 will estimate notification parameters that match the user U1's preferences after retraining. Since the content of the acquired FB information is reflected in the estimated AI 100 through retraining, ideally, correction in the pre-correction process should be unnecessary. For this reason, the pre-correction process is initialized when the retraining process is executed (in addition, the post-correction process may also be initialized). This suppresses the implementation of unnecessary corrections (corrections beyond what is necessary).

[0258] <<Example EX_3C>> Example EX_3C will now be described. Example EX_3C is carried out in combination with any of the above-mentioned Examples EX_2A to EX_2D, similar to Example EX_3B, and the retraining process shown in Example EX_3A is applied to any of Examples EX_2A to EX_2D.

[0259] Figure 45 is an operation flowchart of the controller 11 involved in information notification processing according to Example EX_3C. The operation of the controller 11 according to Example EX_3C includes the processing of steps S10 to S22 and S23a. The operation flowchart of Figure 45 is obtained by replacing step S23 with step S23a based on the operation flowchart of Figure 37. The processing content of steps S10 to S22 in Example EX_3C is the same as that shown in Example EX_2A (Figure 37). Note that in step S20, the cumulative number of acquired FB information SUM ＦＢ "1" is added to the value, and unit training data UDb is generated according to the acquired FB information. The method for generating unit training data UDb is as described above. The generated unit training data UDb is added to the additional training data LDb as described above (see Figures 40 and 41). In Example EX_3C, when it is determined in step S22 that the mode switching condition is met (Y in step S22), the process proceeds to step S23a.

[0260] In step S23a, the FB management unit F14 sets the mode flag F ＭＯＤＥBy setting this to "2", the operating mode of the controller 11 is switched from the first mode to the second mode. Once the operating mode of the controller 11 is switched to the second mode, the operating mode of the controller 11 is fixed in the second mode. Also, in step S23a, the relearning processing unit F15 executes the relearning process described above. The relearning processing unit F15, which is involved in the relearning process, reconstructs the learning model 110 (and therefore the estimated AI 100) by relearning the learning model 110 using the basic learning data LDa and additional learning data LDb stored in the memory units 142 and 143. Although the controller 11 cannot execute information notification processing during the relearning execution period, if the time required for relearning can be shortened sufficiently, the operation flowchart in Figure 45 can be used without any problems.

[0261] Furthermore, in step S23a, the FB management unit F14 may initialize the subsequent correction processing. However, if the controller 11 always adopts the pre-correction configuration and never adopts the double-sided correction configuration after the operating mode is switched to the second mode, initialization of the subsequent correction processing in step S23a is unnecessary. Immediately after switching the operating mode to the second mode, all correction amounts ΔCin in the pre-processing unit F12 are zero, so it is not necessary to initialize the pre-correction processing in step S23a (however, it may be done). Once the processing in step S23a is completed, the process returns to step S11.

[0262] In step S23a, the retraining process reconstructs the estimated AI 100 by referring to the additional training data LDb. After the retraining process is executed, the reconstructed estimated AI 100 (the retrained training model 110) performs inference of notification parameters in the second mode, and the inference result is derived as the output parameter Dout. In the operation flowchart of Figure 45, the retraining process is executed when the operation mode of the controller 11 switches from the first mode to the second mode. Therefore, the next information notification process is executed after the switch to the second mode and the retraining process have been completed.

[0263] By retraining using the basic training data LDa and the additional training data LDb, which is generated considering FB information, the user U1's preferences are reflected in the estimated AI 100 during the retraining process. As a result, it is expected that the estimated AI 100 will be able to estimate notification parameters that match the user U1's preferences after retraining.

[0264] <<Example EX_3D>> Example EX_3D will be explained. In any of the above or below examples, the cumulative number of acquired SUM ＦＢ Methods MTD_3Da, MTD_3Db, or MTD_3Dc can be used for the calculation method.

[0265] Method MTD_3Da: Cumulative number of acquisitions SUM ＦＢ This is the sum of the cumulative number of positive FB information obtained and the cumulative number of negative FB information obtained. When using method MTD_3Da, regardless of whether the FB information obtained through the acquisition process in step S16 is classified as positive FB information or negative FB information, the cumulative number obtained in step S20 is SUM ＦＢ Simply add 1 to it (see Figure 37, etc.).

[0266] SUM of cumulative number of acquisitions in method MTD_3Db ＦＢ This is the cumulative number of positive FB information acquired. When using method MTD_3Db, the cumulative number acquired SUM is calculated in step S20 only if the FB information acquired through the acquisition process in step S16 is classified as positive FB information. ＦＢ Simply add 1 to it (see Figure 37, etc.).

[0267] SUM of cumulative acquisitions in method MTD_3Dc ＦＢ This is the cumulative number of negative FB information acquired. When using method MTD_3Dc, the cumulative number acquired SUM is calculated in step S20 only if the FB information acquired through the acquisition process in step S16 is classified as negative FB information. ＦＢ Simply add 1 to it (see Figure 37, etc.).

[0268] Negative feedback from user U1 regarding notifications is useful in improving the notification parameter "Pout" to suit user U1's personal characteristics (preferences, etc.) based on FB information. For this reason, the adoption of method MTD_3Da or MTD_3Dc is preferred, but method MTD_3Db can also be adopted.

[0269] <<Example EX_4A>> Example EX_4A will be explained.

[0270] The learning model 110 may consist of a visual learning model, an auditory learning model, and a tactile learning model, each composed of a DNN. The visual, auditory, and tactile learning models are formed in the machine learning process of step S2 in Figure 20 by machine learning (pre-machine learning) the corresponding DNNs. In the actual operation process, the correction input data CDin is input to one or more of the visual, auditory, and tactile learning models (however, if a post-stage correction configuration is adopted, the input data Din is input).

[0271] In actual operation, the visual learning model operates only when target information is notified to user U1 using the first or second notification format, and derives only visual-related parameters and common-related parameters (see Figure 12). The common-related parameters derived by the visual learning model specify the start timing of target information notification by notification image (text image or graphic image). In actual operation, the auditory learning model operates only when target information is notified to user U1 using the third or fourth notification format, and derives only auditory-related parameters and common-related parameters (see Figure 12). The common-related parameters derived by the auditory learning model specify the start timing of target information notification by notification sound. In actual operation, the haptic learning model operates only when target information is notified to user U1 using the fifth notification format, and derives only haptic-related parameters and common-related parameters (see Figure 12). The common-related parameters derived by the haptic learning model specify the start timing of target information notification by notification vibration.

[0272] The learning model 110 may consist of a first to fifth learning model, each comprising a DNN. The first to fifth learning models are formed in the machine learning process of step S2 in Figure 20 by performing machine learning (pre-machine learning) on ​​the corresponding DNNs. In the actual operation process, the correction input data CDin is input to one or more of the first to fifth learning models (however, if a post-stage correction configuration is adopted, the input data Din is input). The first to fifth learning models each correspond to the first to fifth notification formats.

[0273] In the actual operation process, the first learning model operates only when the target information is notified to user U1 using the first notification format, and derives only visual-related parameters and common-related parameters (see Figure 12). The common-related parameters derived by the first learning model specify the start timing of notification of the target information using character images. In the actual operation process, the second learning model operates only when the target information is notified to user U1 using the second notification format, and derives only visual-related parameters and common-related parameters (see Figure 12). The common-related parameters derived by the second learning model specify the start timing of notification of the target information using graphic images. In the actual operation process, the third learning model operates only when the target information is notified to user U1 using the third notification format, and derives only auditory-related parameters and common-related parameters (see Figure 12). The common-related parameters derived by the third learning model specify the start timing of notification of the target information using word notification sounds. In the actual operation process, the fourth learning model operates only when the target information is notified to user U1 using the fourth notification format, and derives only auditory-related parameters and common-related parameters (see Figure 12). The common-related parameters derived by the fourth learning model specify the start timing of notification of the target information by non-verbal notification sounds. In the actual operation process, the fifth learning model operates only when the target information is notified to user U1 using the fifth notification format, and derives only tactile-related parameters and common-related parameters (see Figure 12). The common-related parameters derived by the fifth learning model specify the start timing of notification of the target information by notification vibrations.

[0274] The learning model 110 may be a single learning model common to vision, hearing, and touch (hereinafter referred to as the common learning model). In this case, in the actual operation process, input data Din or CDin is always input to the common learning model, and the common learning model derives vision-related parameters, hearing-related parameters, touch-related parameters, and common-related parameters. When target information is notified to user U1 using the first or second notification format, the common-related parameters derived by the common learning model specify the start timing of notification of target information by notification image. When target information is notified to user U1 using the third or fourth notification format, the common-related parameters derived by the common learning model specify the start timing of notification of target information by notification sound. When target information is notified to user U1 using the fifth notification format, the common-related parameters derived by the common learning model specify the start timing of notification of target information by notification vibration.

[0275] <<Example EX_4B>> Example EX_4B will now be described. The estimated AI 100 may be provided on a server device (not shown) different from the in-vehicle device 10. The server device is an external device connected to an external network, and bidirectional communication is possible between the in-vehicle device 10 and the server device. The server device is formed by one or more computer devices. When the estimated AI 100 is provided on the server device, the controller 11 uses the communication unit 13 to send a request signal including input data (Din or CDin) to the server device. When the server device receives the request signal, it inputs the input data in the request signal to the estimated AI 100 to obtain the output parameter Dout from the estimated AI 100, and sends a response signal including the obtained output parameter Dout to the in-vehicle device 10. When the in-vehicle device 10 receives the response signal, the controller 11 sets the notification parameter Pout using the output parameter Dout in the response signal, and notifies the user U1 of the target information according to the notification parameter Pout.

[0276] <<Example EX_4C>> Example EX_4C will be explained.

[0277] In the information notification process, the controller 11 may use a combination of two or more of the first to fifth notification formats to provide the necessary notifications. For example, when notifying user U1 of a certain document as target information, the controller 11 may display the document on the display screen 51a and output sound from the speaker 52, while simultaneously generating vibrations in the vibration device 53 in a manner associated with the document.

[0278] While passenger cars or trucks traveling on public roads are primarily envisioned as vehicle V1, the type of vehicle V1 is arbitrary. Vehicle V1 may also be an industrial vehicle such as a forklift, or an agricultural vehicle such as a tractor. The on-board device 10 corresponds to or incorporates the notification control device according to the present invention. The notification control device according to the present invention may also be installed on an airplane, ship, or train, in which case the user U1 will be a crew member of the airplane, ship, or train. In addition, the notification control device according to the present invention can be applied to any other use.

[0279] A program that causes a computer device to execute any method described in each embodiment of the present invention, and a non-volatile recording medium on which such program is recorded, are included within the scope of the embodiments of the present invention. The program that causes a computer (computer device) to execute any method described in the embodiments of the present invention may be a subprogram incorporated into any main program or called by any main program. The in-vehicle device 10 is equipped with a computer capable of executing any program. The arithmetic processing unit 11a provided in the in-vehicle device 10 may be considered to be the computer. The method executed by the notification control device according to the present invention may be called a notification control method, and the program that causes a computer to execute such notification control method may be called a notification control program. Any processing in the embodiments of the present invention may be realized by hardware such as a semiconductor integrated circuit, software corresponding to the above program, or a combination of hardware and software.

[0280] The embodiments of the present invention can be modified in various ways as appropriate within the scope of the technical idea set forth in the claims. The embodiments described above are merely examples of embodiments of the present invention, and the meaning of the terms of the present invention or each constituent element is not limited to those described above. The specific numerical values ​​shown in the above description are merely examples and can, of course, be changed to various numerical values.

[0281] <<Note>> Embodiments of the present invention also include the following distinctive technical concepts (see Examples EX_3A to EX_3D in particular).

[0282] A notification control device according to one aspect of the present invention includes a controller that sets notification parameters when notifying a user of target information using an estimation AI based on input data, the controller is capable of performing a pre-correction process that corrects the input data based on the user's evaluation index for past notifications, or a post-correction process that corrects the output parameters output from the estimation AI based on the evaluation index, the controller generates corrected input data by correcting the input data through the pre-correction process, and then inputs the corrected input data to the estimation AI to set the output parameters output from the estimation AI as the notification parameters, or the The first configuration involves inputting input data into the estimation AI, correcting the output parameters output from the estimation AI in the subsequent correction process to set the notification parameters, the estimation AI being constructed using a machine learning model, the controller sequentially notifying the user of multiple target information, attempting to obtain feedback information indicating the evaluation index each time the target information is notified, executing a retraining process after the group of feedback information has been obtained, and in the retraining process, retraining the learning model based on the input data and notification parameters corresponding to each piece of feedback information to reconstruct the estimation AI.

[0283] In the notification control device according to the first configuration described above, the learning model is generated by machine learning using basic learning data before the retraining process is executed, the controller generates additional learning data based on the feedback information, the input data and the notification parameters before the retraining process is executed, the estimated AI is rebuilt by retraining the learning model using the basic learning data and the additional learning data during the retraining process, the notification parameters are set using the rebuilt estimated AI after the retraining process is executed, and the controller initializes the pre-correction process or the post-correction process in conjunction with the execution of the retraining process (second configuration).

[0284] In the notification control device according to the second configuration described above, the controller classifies each piece of feedback information into positive feedback information or negative feedback information according to the content of the feedback information, the positive feedback information is feedback information of the evaluation index in which the user shows a positive reaction to the notification of the corresponding target information, and the negative feedback information is feedback information of the evaluation index in which the user shows a negative reaction to the notification of the corresponding target information, and the controller may be configured to execute the retraining process after a predetermined number or more of the positive feedback information has been acquired (third configuration).

[0285] In the notification control device according to the third configuration described above, the controller may generate unit training data for each positive feedback piece of information, having the input data corresponding to the positive feedback piece of information as training input data for the learning model and the notification parameter corresponding to the positive feedback piece of information as correct answer data, and the additional learning data may be formed from multiple unit training data (fourth configuration).

[0286] In the notification control device according to the second configuration described above, the controller classifies each piece of feedback information into positive feedback information or negative feedback information according to the content of the feedback information, the positive feedback information is feedback information of the evaluation index in which the user shows a positive reaction to the notification of the corresponding target information, and the negative feedback information is feedback information of the evaluation index in which the user shows a negative reaction to the notification of the corresponding target information, and the controller may be configured to execute the retraining process after a predetermined number or more of the negative feedback information has been acquired (fifth configuration).

[0287] In the notification control device according to the fifth configuration described above, the controller generates unit training data for each negative feedback information, having the input data corresponding to the negative feedback information as training input data for the learning model and the processed notification parameter corresponding to the negative feedback information as correct data. The controller generates the processed notification parameter for each negative feedback information by processing the notification parameter corresponding to the negative feedback information based on the negative feedback information, and the additional learning data is formed by multiple unit training data (sixth configuration).

[0288] In the notification control device according to the second configuration described above, the controller classifies each piece of feedback information into positive feedback information or negative feedback information according to the content of the feedback information, the positive feedback information is feedback information of the evaluation index in which the user shows a positive reaction to the notification of the corresponding target information, and the negative feedback information is feedback information of the evaluation index in which the user shows a negative reaction to the notification of the corresponding target information, and the controller may also be configured to execute the retraining process after the sum of the total number of positive feedback information and the total number of negative feedback information exceeds a predetermined threshold number (seventh configuration).

[0289] In the notification control device according to the seventh configuration, for each piece of the positive feedback information, the controller generates first unit training data having the input data corresponding to the positive feedback information as learning input data for the learning model and having the notification parameter corresponding to the positive feedback information as correct data, and for each piece of the negative feedback information, the controller generates second unit training data having the input data corresponding to the negative feedback information as other learning input data for the learning model and having the processed notification parameter corresponding to the negative feedback information as other correct data. For each piece of the negative feedback information, the controller generates the processed notification parameter by processing the notification parameter corresponding to the negative feedback information based on the negative feedback information. A configuration (eighth configuration) may be adopted in which the additional learning data is formed by one or more pieces of the first unit training data and one or more pieces of the second unit training data.

[0290] In the notification control device according to any one of the first to eighth configurations, the input data includes at least sensitivity-related information that affects the sensitivity of the user to the notification, and the notification parameter includes at least a parameter of the transmission intensity when transmitting the target information to the user through the user's vision, hearing, or touch. A configuration (ninth configuration) may be adopted.

[0291] A driving support system according to one aspect of the present invention is a driving support system mounted on a vehicle, including a notification control device according to any one of the first to ninth configurations, and an information output unit that outputs the target information to a user who is a passenger of the vehicle according to the notification parameter set by the notification control device.

[0292] A notification control method according to one aspect of the present invention is a notification control method executed by a notification control device, comprising a parameter setting step in which notification parameters for notifying a user of target information are set using an estimated AI based on input data, wherein the parameter setting step can execute a pre-correction process that corrects the input data based on the user's evaluation index for past notifications, or a post-correction process that corrects the output parameters output from the estimated AI based on the evaluation index. In the parameter setting step, after generating corrected input data by correcting the input data with the pre-correction process, the output parameters output from the estimated AI are set as the notification parameters by inputting the corrected input data to the estimated AI, or the notification parameters are set by correcting the output parameters output from the estimated AI with the post-correction process by inputting the input data to the estimated AI. In this notification control method, the estimated AI is constructed using a machine learning model, sequentially notifies the user of multiple pieces of target information, attempts to obtain feedback information indicating the evaluation index each time the target information is notified, and executes a retraining process after the group of feedback information has been obtained. In this notification control method, the estimated AI is rebuilt in the retraining process by retraining the learning model based on the input data and notification parameters corresponding to each piece of feedback information.

[0293] A notification control program can be formed that causes a computer to execute the notification control method relating to the above aspect.

[0294] SYS Driving Assistance System V1 Vehicle U1 User ST1 Seat 10 In-vehicle device 11 Controller 11a Calculation processing unit 12 Memory 13 Communication unit 14 Recording medium 15 Operation input unit 20 Vehicle control device 30 Actuator unit 40 Sensing unit 50 Information output unit 51 Display device 51a Display screen 52 Speaker 53 Vibration device F1 Notification parameter setting unit F11 Parameter generation unit F12 Pre-processing unit F13 Post-processing unit F14 FB management unit F141 Correction content adjustment unit F2 Information acquisition unit F3 Notification management unit F4 Priority setting unit F5 Notification format determination unit F6 Output control unit 100 Parameter estimation AI 110 Learning model Din Input data CDin Correction input data Dout Output parameter Poout Notification parameter

Claims

1. A notification control device comprising a controller that sets notification parameters for notifying a user of target information using estimation AI based on input data, wherein the controller sets the notification parameters based on the input data and the user's evaluation indicators for past notifications.

2. The notification control device according to claim 1, wherein the controller generates corrected input data by correcting the input data based on the evaluation index, and sets the output parameters output from the estimation AI to the notification parameters by inputting the corrected input data to the estimation AI.

3. The notification control device according to claim 1, wherein the controller sets the notification parameters by inputting the input data to the estimation AI and correcting the output parameters output from the estimation AI based on the evaluation index.

4. The notification control device according to claim 1, wherein the controller generates corrected input data by correcting the input data based on the evaluation index, and sets the notification parameter by inputting the corrected input data to the estimation AI and correcting the output parameter output from the estimation AI based on the evaluation index.

5. The notification control device according to claim 1, wherein the controller is capable of performing a pre-correction process to correct the input data based on the evaluation index, and a post-correction process to correct the output parameters output from the estimated AI based on the evaluation index, the controller sets an operating mode for setting the notification parameters to a first mode or a second mode, in the first mode, the controller sets the notification parameters by performing the post-correction process on the output parameters output from the estimated AI by inputting the input data to the estimated AI, and in the second mode, the controller generates corrected input data by performing the pre-correction process on the input data, and sets the notification parameters based on the output parameters output from the estimated AI by inputting the corrected input data to the estimated AI.

6. The notification control device according to claim 5, wherein the controller sequentially notifies the user of multiple target information, attempts to acquire the evaluation index each time the target information is notified, sets the operation mode to the first mode when the evaluation index has not been acquired, and then switches the operation mode from the first mode to the second mode according to the cumulative number of acquired evaluation indexes.

7. The notification control device according to claim 6, wherein the controller sets the operation mode to the first mode when the cumulative number of acquired data is less than a threshold, and sets the operation mode to the second mode when the cumulative number of acquired data is equal to or greater than the threshold.

8. The notification control device according to claim 6, wherein when the controller switches the operating mode from the first mode to the second mode, it determines the content of the pre-correction processing during the second mode period when the operating mode is set to the second mode, based on the evaluation index, the input data, and the notification parameters during the first mode period when the operating mode was set to the first mode.

9. The notification control device according to claim 6, wherein the controller repeatedly counts the number of times the evaluation index is acquired per unit time, and switches the operating mode from the first mode to the second mode based on the result of the count and the cumulative number acquired.

10. After specific feedback information indicating a user evaluation index for notifications of specific conditions is obtained, if a notification corresponding to the specific conditions is made in the second mode, the controller sets the notification parameters by performing the subsequent correction process corresponding to the specific feedback information on the output parameters, and if a notification not corresponding to the specific conditions is made in the second mode, the controller sets the output parameters to the notification parameters, as described in claim 6.

11. The notification control device according to claim 6, wherein the estimation AI is constructed from a machine learning model, the controller switches the operating mode from the first mode to the second mode and then performs a retraining process according to the cumulative number of acquisitions, the learning model is generated by machine learning using basic learning data before the execution of the retraining process, the controller generates additional learning data based on the evaluation index, the input data and the notification parameters before the execution of the retraining process, the estimation AI is reconstructed by retraining the learning model using the basic learning data and the additional learning data in the retraining process, the notification parameters are set in the second mode using the reconstructed estimation AI after the execution of the retraining process, the controller initializes the pre-correction process in conjunction with the execution of the retraining process, and the correction input data matches the input data immediately after the initialization.

12. The notification control device according to claim 6, wherein the estimation AI is constructed from a machine learning model, the controller performs a retraining process when switching the operating mode from the first mode to the second mode, the machine learning model is a model that has been machine learning using basic training data before the execution of the retraining process, the controller generates additional training data based on the evaluation index, the input data and the notification parameters before the execution of the retraining process, reconstructs the estimation AI by retraining the machine learning model using the basic training data and the additional training data in the retraining process, and sets the notification parameters in the second mode using the reconstructed estimation AI after the execution of the retraining process.

13. The notification control device according to any one of claims 1 to 12, wherein the input data includes at least sensitivity-related information that affects the user's sensitivity to notifications, and the notification parameters include at least parameters of transmission intensity when the target information is transmitted to the user through the user's sight, hearing, or touch.

14. A driver assistance system installed in a vehicle, comprising: a notification control device according to any one of claims 1 to 12; and an information output unit that outputs the target information to a user, who is an occupant of the vehicle, according to the notification parameters set by the notification control device.

15. A notification control method executed by a notification control device, comprising a parameter setting step of setting notification parameters for notifying a user of target information using estimation AI based on input data, wherein in the parameter setting step, the notification parameters are set based on the input data and the user's evaluation index for past notifications.

16. A notification control program that causes a computer to execute the notification control method described in claim 15.

Images