A presentation system that adapts to the nature of the problem and the user's situation.

Abstract

In a problem-solving and user-responsive presentation system, when a problem occurs while the user is driving, the system provides appropriate suggestions and responses based on the problem and user situation. [Solution] The adaptive presentation system for the problem and user situation includes: a response-to-response estimation unit (211) that estimates the urgency and complexity of a problem when a problem occurs while driving a vehicle; an acceptance-to-response estimation unit (212) that estimates the acceptance level of the vehicle user; a selection unit (213) that selects presentation content and methods that are estimated to increase the probability that the user will accept the presentation content and methods, according to the estimated response-to-response level and acceptance level; and a presentation unit (214) that presents the selected presentation content and methods.

Description

Technical Field

[0001] The present invention relates to a presentation system that utilizes an LLM to generate answers when a problem occurs during driving or when a user makes an inquiry in, for example, an owners manual Q&A system for a voice agent or vehicle agent mounted on a vehicle. More broadly, it relates to the technical field of a presentation system that uses an LLM to make presentations corresponding to problems regardless of whether there is an inquiry.

Background Art

[0002] As this type of system, a system has been proposed that changes the summary length of the answer text using an LLM according to the content of the answer (see Non-Patent Document 1).

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, according to the above-described background art, it is impossible to consider the urgency or complexity of the problem or the user's acceptance, and it is also impossible to change the modality related to the presentation or answer. For this reason, there is a technical problem that it is still difficult to make presentations and answers that are more adapted to the problem content and user situation.

[0005] An object of the present invention is to provide a presentation system adapted to problem content and user situation that can make presentations and answers adapted to problem content and user situation when a problem occurs during user driving or when the user makes an inquiry about the problem. [Means for solving the problem]

[0006] One embodiment of the present invention, which is a presentation system adapted to the problem content and user situation, in order to solve the above problem, comprises: a response-to-response estimation unit that estimates the degree to which action should be taken in response to a problem that occurs while a problem occurs while driving a vehicle, including at least one of the urgency and complexity of the problem; an acceptance-to-response estimation unit that estimates the acceptance level of the vehicle user when the problem occurs; a selection unit that selects presentation content and methods that are estimated to increase the probability that the user will accept the presentation content and methods, according to the estimated response-to-response level and the estimated acceptance level, and a presentation unit that presents the selected presentation content and methods to the user using LLM. [Effects of the Invention]

[0007] According to one embodiment of the problem-responding and user-response-based presentation system of the present invention, when a problem occurs while the user is driving, or when the user makes an inquiry in response to the occurrence of a problem or any question, the system can provide presentations and answers that are appropriate to the problem and the user's situation.

[0008] The effects and benefits of the present invention will be further clarified by the embodiments of the invention described below. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram showing the overall configuration of the presentation system according to the embodiment. [Figure 2] This is a schematic conceptual diagram illustrating how the presentation system according to the embodiment provides presentations and answers that are adapted to the problem content and user situation. [Figure 3] A flowchart illustrating an example of processing in the system according to the embodiment. [Figure 4] A flowchart showing another example of processing in the presentation system according to the embodiment. [Modes for carrying out the invention]

[0010] First, with reference to Figure 1, the overall configuration of the presentation system (hereinafter simply referred to as the "presentation system") adapted to the problem content and user situation according to the embodiment will be described.

[0011] In this embodiment, as will be described in detail below, for example, in an owner's manual Q&A system in an LLM voice dialogue system, LLM voice agent, or LLM vehicle agent installed in a vehicle, when a problem occurs while the user is driving the vehicle, or when the user makes an inquiry in response to the occurrence of a problem or any question, the system is configured to provide answers and suggestions that are adapted to the content of the problem and the user's situation, using LLM so that the user can accept the answers and suggestions.

[0012] In this embodiment, the "LLM (Single Modal LLM or Multimodal LLM) learning," "machine learning," or "AI learning" employed may include traditional AI learning systems such as supervised learning, unsupervised learning, or reinforcement learning, as well as novel generative AI or LLM technologies that have recently been put into practical use, are currently under development, or will be developed in the future. For example, the AI ​​learning or LLM learning described here may be configured using a neural network that performs efficient learning through representation learning, transfer learning, feature selection, fine tuning or hyperparameter tuning, ensemble learning, etc.

[0013] As shown in Figure 1, the presentation system according to this embodiment is configured to include an on-board unit 101 and a server device 200 mounted on a vehicle 100. The on-board unit 101 and the server device 200 are connected to a communication network 10, such as the Internet or a dedicated network line. Other multiple or numerous vehicles 100 are also connected to the communication network 10 in the same way.

[0014] The communication network 10 also includes a user data collection unit 306 that collects data specific to the driver or user of the vehicle 100 (i.e., "user data"). The user data collection unit 306 collects user data specific to the user or driver, such as driver behavior data and driving scene data related to the user or driver.

[0015] In addition to the user data collection unit 306, the communication network 10 may also house a general knowledge collection unit (not shown) that collects "common sense" useful when making estimations based on verbalized or documented data when utilizing LLM, and a map data collection unit (not shown) that collects road maps, multipurpose maps, etc. useful when making estimations based on data related to the state and conditions of the vehicle 100's operation. Furthermore, the communication network 10 may house an external related knowledge collection unit (not shown) that collects information obtained outside the vehicle 100 that can be used to impart domain knowledge by performing fine-tuning or hyper-tuning in the presentation system (i.e., "external related knowledge").

[0016] Such a user data collection unit 306, or the general knowledge collection unit and map data collection unit (not shown) described above, may be provided at least partially within the server device 200 or within the facility where the server device 200 is located, or within the vehicle-mounted unit 101 or within the vehicle 100.

[0017] A database (DB) 300, which stores various data including the data used in the presentation system, is connected to the server device 200. The DB 300 may be connected to the server device 200 or the vehicle-mounted unit 101 via the communication network 10. The server device 200 is composed of various computer-equipped devices and various computer devices that perform centralized or distributed processing. In other words, the presentation system is constructed as a system that performs centralized or distributed processing using the large amount of data in the DB 300.

[0018] In FIG. 1, the vehicle 100 may be, for example, a so-called HEV (Hybrid Electric Vehicle), PHEV (Plugin HEV), FCEV (Fuel Cell EV), BEV (Battery Electric Vehicle), etc., or may be a vehicle with an internal combustion engine as the main power source, but is constructed as a so-called connected car.

[0019] The vehicle-mounted unit 101 is configured to include an interface unit 20, a vehicle behavior and driving operation (CAN) unit 21, a position information unit 22, an in-vehicle and out-of-vehicle camera moving image unit 23, a biological data unit 24, a processing unit 26, and a communication unit 28. Among these, mainly the interface unit 20 and the processing unit 26, and from the server device 200 connected via the communication network 10, for example, a "voice agent" or a "vehicle agent" mounted on the vehicle 100 is constructed, and further, an "owners manual Q&A system" is constructed as one function of these agents.

[0020] The interface unit 20 is configured to include a microphone 20m, a speaker 20s, and a display unit 20d so that communication can be carried out by voice or image between the driver or user inside the vehicle, such as receiving an inquiry and providing an answer or a presentation. The interface unit 20 is configured to enable communication, for example, when using the "owners manual Q&A system" provided in the vehicle 100, or when using functions such as a navigation function, an AV function, an automatic driving function, etc., by voice input or a predetermined operation on an image.

[0021] Regarding such various functions, all or part of them may be configured to be executed on the vehicle-mounted unit 101 side such as the processing unit 26 and the interface unit 20, or part or all of them may be configured to be executed on the server device 200 side such as the processing unit 210 (in other words, the vehicle-mounted unit 101 side solely serves as a browser function).

[0022] The interface unit 20 is further configured to be able to appropriately output the presentation data and response data adapted to the problem content and user situation, which are passed from the server device 200 side, in the form of images or sounds of some form via the display unit 20d and the speaker 20s.

[0023] The vehicle behavior and driving operation (CAN) unit 21 includes various sensors such as a vehicle speed sensor, an acceleration sensor, a distance sensor, an engine speed sensor, a temperature sensor, an altitude sensor, a pressure sensor, a battery remaining amount sensor, etc., which are respectively arranged at predetermined positions in the vehicle 100. The vehicle behavior and driving operation (CAN) unit 21 is configured to be able to appropriately transmit CAN (Controlled Area Network) data and the like related to the current driving state of the vehicle 100 to the server device 200 side via the communication network 10 from the communication unit 28 under the control of the processing unit 26.

[0024] The position information unit 22 includes a GPS device, an inertial navigation positioning device, etc., and detects the current position of the vehicle 100. The position information unit 22 is further configured to generate raw position information data indicating, for example, latitude and longitude or coordinates, and be able to appropriately transmit it to the server device 200 side via the communication network 10 from the communication unit 28 under the control of the processing unit 26.

[0025] The in-vehicle and out-of-vehicle camera moving image unit 23 includes one or more cameras such as CCDs, and functions as a drive recorder that captures the conditions inside and outside the vehicle. The in-vehicle and out-of-vehicle camera moving image unit 23 is further configured to be able to appropriately transmit the drive recorder video to the server device 200 side via the communication network 10 from the communication unit 28 as captured image data under the control of the processing unit 26.

[0026] The biometric data unit 24 directly or indirectly detects the biometric data of the driver or user by, for example, various vital sensors arranged at predetermined positions in the vehicle 100 and image analysis devices, voice analysis devices, etc. that analyze the appearance and voice of the driver or user. The biometric data unit 24 is further configured to be able to appropriately transmit the detected biometric data to the server device 200 side via the communication network 10 from the communication unit 28 under the control of the processing unit 26.

[0027] The processing unit 26 has a CPU, memory, etc., that controls the interface unit 20, the vehicle behavior / driving operation (CAN) unit 21, the location information unit 22, the in-vehicle / out-of-vehicle camera video unit 23, the biometric data unit 24, and the communication unit 28. The processing unit 26 appropriately transmits various data detected or generated at each part of the vehicle 100 and its driver from the communication unit 28 to the server device 200 in a predetermined format. The processing unit 26 further appropriately receives data that has been estimated and generated from the server device 200 side, such as data related to problems that have occurred in the vehicle 100 and answers or presentations in response to user inquiries, via the communication unit 28. The processing unit 26 is configured to control the in-vehicle unit 101 related to the in-vehicle "voice agent," "vehicle agent," and "owner's manual Q&A system."

[0028] The communication unit 28 includes a modem or the like that is configured to communicate with the outside of the vehicle via the communication network 10. The communication unit 28 is configured to transmit various types of data collected by the vehicle 100 to the server device 200 via the communication network 10 in a predetermined format, under the control of the processing unit 26.

[0029] In Figure 1, the server device 200 is configured to include a communication unit 201 and a processing unit 210.

[0030] The communication unit 201, under the control of the processing unit 210, appropriately receives various data collected by the vehicle 100 via the communication network 10, and on the other hand, appropriately receives user data collected by the user data collection unit 306 via the communication network 10, and is configured to pass the received data to the processing unit 210.

[0031] The processing unit 210 comprises a response-to-response-to-response-to-response-to-response-to-acceptance-to-response

[0032] The Action Needed Estimation Unit 211 is configured to estimate the action needed when a problem occurs while the vehicle 100 is in operation. Here, "action needed" includes at least one of the urgency and complexity of the problem and indicates the degree to which action should be taken regarding the problem. Specifically, for example, the action needed is estimated when the occurrence of any problem with the vehicle 100 is detected, observed, or predicted, or when a user makes an inquiry using the "voice agent," "vehicle agent," or "owner's manual Q&A system" built on the in-vehicle unit 101. The user makes such an inquiry, for example, when they themselves recognize or are notified that a problem has occurred with the vehicle 100, or when they have any questions regarding the vehicle 100. Here, the Action Needed Estimation Unit 211 is configured to estimate such action needed using LLM.

[0033] The acceptance level estimation unit 212 is configured to estimate the acceptance level of the vehicle 100 user when a problem occurs or an inquiry is made, either before, after, or in parallel with the estimation of the degree of need for action described above. Here, "acceptance level" refers to the degree to which the user can accept the vehicle subjectively or objectively, and its degree is determined by various factors such as whether the user is driving or not, whether the user is in conversation or not, the level of the user's fatigue, the level of the user's hearing, the level of noise inside the vehicle 100, whether it is day or night, the level of the user's knowledge and experience, the quality of the weather, and the traffic congestion and congestion conditions on the road. Here, the acceptance level estimation unit 212 is configured to estimate such acceptance level using LLM.

[0034] The selection unit 213 is configured to select, as the presentation content and method for addressing the problem, those that are estimated to increase the probability that the user will accept the presentation content and method, according to the degree of need for action and the degree of acceptance estimated by the need for action and the degree of acceptance estimated by the need for action and the degree of acceptance estimated by the acceptance unit 211 and the acceptance unit 212, respectively. Preferably, the selection unit 213 performs such estimation using LLM or multimodal LLM. The selection unit 213 is configured to perform selections that satisfy the LLM by setting the condition for selection that "increases the probability that the user will accept the presentation content and method" for LLM or multimodal LLM.

[0035] The presentation unit 214 is configured to present information to the user using the presentation content and method selected by the selection unit 213, via LLM. The presentation unit 214 makes the presentation by creating data in a predetermined format that defines such presentation content and method, and the created data in the predetermined format is delivered to the user's sight and hearing as a response or presentation via the communication network 10 through the display unit 20d and speaker 20s of the interface unit 20.

[0036] DB300 is configured to include a large-scale and high-speed data input / output storage device that stores various data received by the server device 200 via the communication network 10, in particular various data necessary for the response level estimation process and acceptance level estimation process using machine learning, and data related to the estimation results or intermediate results generated by the processing unit 210.

[0037] In the system configuration described above, the interface unit 20 and processing unit 26 on the vehicle unit 101 side primarily function as an LLM voice dialogue system, LLM voice agent, or LLM vehicle agent owner's manual Q&A system with various functions. Alternatively, in addition to the interface unit 20 and processing unit 26 on the vehicle unit 101 side, the processing unit 210 on the server device 200 side primarily functions as such an owner's manual Q&A system.

[0038] Next, referring to the block diagram in Figure 1 and the schematic conceptual diagram in Figure 2, we will explain the process, mainly performed by the processing unit 210, that provides adaptive suggestions and answers to the problem content and user situation according to this embodiment.

[0039] In Figure 2, when a problem occurs or when a user makes an inquiry regarding vehicle 100, the user acceptance level 6 is estimated by the acceptance level estimation unit 212 (see Figure 1) using LLM, based on the user type 1, in-vehicle conversation 2, and road traffic conditions 3.

[0040] Here, "User Type 1" is acquired, for example, via the User Data Collection Unit 306 (see Figure 1). User Type 1 includes categories such as "Impatient person (or not)" and "Knowledgeable about cars?". "In-car conversation 2" is acquired, for example, via the microphone 20m of the in-vehicle unit 101. In-car conversation 2 includes categories such as "In conversation (or not)". "Road traffic conditions 3" is acquired, for example, via the location information unit 22, vehicle behavior driving operation (CAN) unit 21, in-vehicle and exterior camera video unit 23 of the in-vehicle unit 101. "Road traffic conditions 3" include categories such as "Driving (or not)", "Intersection?", "Traffic signal display?", "Congested?", "Highway?", and "Near junction / merging?".

[0041] On the other hand, when a problem occurs or when a user makes an inquiry regarding vehicle 100, the "urgency and complexity of the problem 7," which is a type of "level of need for action," is estimated by the level of need for action estimation unit 211 (see Figure 1) using LLM, either before or after the user acceptance level estimation process 6 described above, based on the inquiry 4 and the vehicle status 5.

[0042] Here, "Inquiry 4" is acquired, for example, via the microphone 20m located in the interface unit 20 of the in-vehicle unit 101. "Inquiry 4" can be categorized as, for example, "Is this an important problem (or not)?" or "Is the solution complicated?". "Vehicle Status 5" is acquired, for example, via the Vehicle Behavior / Driving Operation (CAN) unit 21. "Vehicle Status 5" can be categorized as, for example, "Is something abnormal (or not)?".

[0043] As described above, once the user acceptance level 6 and the urgency and complexity level 7 of the problem are estimated, the selection unit 213 (see Figure 1) selects an answer method selection model 8 in LLM based on the prompt template 11, image template 12, and voice template 13.

[0044] Then, the presentation unit 214 (see Figure 1) estimates and presents the answer 500 using the LLM 9, in accordance with the selected answer method model 8. The specific content of the answer 500 is as shown in Figure 2 as answer content 15. That is, as answer content 15, text-based answers are presented, such as "Please stay calm and pull over to the side of the road immediately," "It is okay to continue driving," and "We will give you a step-by-step lecture on how to use the ○○ function." Alternatively, image / video-based answers are presented, such as "Display the relevant page of the manual on the navigation system," "Explain with a video (for example, the three block-shaped images in answer content 15)," and "Adjust the display size of the images." Alternatively, audio answers are presented, such as "Sound an alarm."

[0045] Next, with reference to the block diagram in Figure 1 and the schematic conceptual diagram in Figure 2, as well as the flowchart in Figure 3, an example of processing in the presentation system according to this embodiment (particularly the processing performed by the processing unit 210 in the server device 200) will be further explained. In this example, when a problem occurs, a series of processes are performed to provide presentations and answers that are appropriate to the nature of the problem and the user's situation.

[0046] In Figure 3, the processing unit 26 on the vehicle unit 101 side and the processing unit 210 on the server device 200 side (see Figure 1) determine whether or not a problem has occurred in the vehicle 100 (Step S1). If a problem has occurred (Step S1: YES), the action requirement estimation unit 211 (see Figure 1) estimates an example of the action requirement, "Problem urgency / complexity level 7" (see Figure 2) (Step S2).

[0047] Simultaneously with or in parallel with the estimation of the degree of need for action (step S2), the acceptance estimation unit 212 (see Figure 1) estimates an example of acceptance, "User Acceptance 6" (see Figure 2) (step S3). For example, the LLM estimates "User Acceptance 6" which is expressed in text or numerically.

[0048] Subsequently, the selection unit 213 (see Figure 1) selects the presentation content and method (step S4). The selection here is made by constructing a response method selection model 8 using LLM9 with prompt template 11, image template 12, and voice template 13 (see Figure 2).

[0049] Subsequently, the presentation unit 214 (see Figure 1) executes a presentation, such as the presentation content 15 (see Figure 2), at the interface unit 20 on the vehicle 100 side (step S5), and the series of processes is completed. On the other hand, if no problem is found as a result of the determination in step S1 (step S1: No), the processing of the response and presentation is not necessary, and the process is terminated as is. This series of processes may be called periodically or irregularly as a subroutine that can be executed repeatedly in a short time and executed as appropriate.

[0050] In this example, in the selection process (step S4), as part of the selection of presentation content and method, the modality of the presentation may be changed, and in the presentation process (step S5), the presentation may be carried out using the changed modality. By changing the modality, it becomes possible to create a presentation that is easier for the user to understand and that is easier for the user to pay attention to.

[0051] In this case, the presentation unit 214 (see Figure 1) may further be configured to present text, images, videos, audio, or a combination thereof in response to the changed modality when performing the selection process (step S4). If the estimation result by the selection unit indicates that it is unclear whether the probability will be improved or the probability is below a predetermined threshold, the presentation may be made using such a combination. With this configuration, it becomes possible to use the change in modality to create a situation that is easier for the user to understand or to attract the user's attention. It is also possible to first change the modality and present the information as a trial to determine whether the user will accept it.

[0052] In the embodiments described above, the response requirement estimation unit 211 may be configured to estimate the response requirement at a future point in time advanced by the amount of time it takes for the user to understand the presented content, the acceptance level estimation unit 212 may estimate the acceptance level at the future point in time, the selection unit 213 may select what is estimated to improve the probability at the future point in time, and the presentation unit 214 may be configured to make the presentation at the future point in time. With this configuration, the time required for the user to understand the presentation by the presentation unit 214 when the interface unit 20 receives it is taken into consideration. The "time it takes for the user to understand the presented content" here may be estimated by LLM based on information such as the length of the presented text and the user's reading comprehension and speed reading ability, or it may be determined by a predetermined standard. As a result, it becomes possible to create a situation in which the user can understand more easily and a situation in which the user can pay more attention.

[0053] In the example processing shown in Figure 3, if a problem occurs while the user is driving vehicle 100, LLM is used to provide adaptive responses and suggestions to the user, whether the user contacts the system or not, so that the user accepts the answers and suggestions. In this way, when a problem occurs while the user is driving, adaptive suggestions and responses can be provided to the user, making it very practical.

[0054] Next, with reference to the block diagram in Figure 1 and the schematic conceptual diagram in Figure 2, as well as the flowchart in Figure 4, other examples of processing in the presentation system according to this embodiment (particularly processing performed by the processing unit 210 in the server device 200) will be further explained. In this example, when a user makes an inquiry in response to a problem occurring in the vehicle 100, or when a user makes an inquiry because they have any questions related to the vehicle 100, the system provides presentations and answers that are appropriate to the nature of the problem and the user's situation. In Figure 4, steps similar to those in Figure 3 are denoted by the same step numerals, and their explanations are omitted as appropriate.

[0055] In other words, as shown in Figure 4, the processing unit 26 on the vehicle unit 101 side and the processing unit 210 on the server device 200 side (see Figure 1) determine whether a user has made an inquiry on the Owner's Manual Q&A system in the LLM voice dialogue system, LLM voice agent, or LLM vehicle agent, regardless of whether a problem has occurred in the vehicle 100 or not; that is, triggered by the occurrence of a problem or the occurrence of a question (step S21). If an inquiry has been made (step S21: YES), the processing from step S2 onwards, which is the estimation of the degree to which action is required regarding the problem or question related to the inquiry, is executed in the same manner as in the example in Figure 3, and the series of processes is completed. On the other hand, if the determination in step S21 indicates that no inquiry has been made (step S21: No), the processing of providing the answer and presenting it is not necessary, and the process is completed as is.

[0056] Thus, in this example, the processing units 26 and 210, which are configured to function as, for example, an owner's manual Q&A system, can provide appropriate suggestions and answers to users when they contact them in response to a problem occurring during operation or (without a problem occurring) when a question arises, thus proving to be very practically useful.

[0057] As described in detail above, according to this embodiment, when a problem occurs while the vehicle 100 is in operation, or when a user makes an inquiry in response to such a problem, or even when a user makes an inquiry even if no particular problem has occurred, the response can be modified in terms of content and method, or modality, to better suit the user, depending on the urgency and complexity of the problem, and the user's level of acceptance.

[0058] Note The following additional information is disclosed regarding the embodiments described above.

[0059] [Note 1] The present invention, as described in Appendix 1, is a presentation system that is adaptive to the problem content and user situation, comprising: a response level estimation unit that estimates the degree to which action should be taken in response to a problem that occurs while a problem occurs while driving a vehicle, including at least one of the urgency and complexity of the problem; an acceptance level estimation unit that estimates the acceptance level of the vehicle user when the problem occurs; a selection unit that selects presentation content and methods that are estimated to increase the probability that the user will accept the presentation content and methods, according to the estimated response level and the estimated acceptance level, as presentation content and methods to respond to the problem; and a presentation unit that presents the selected presentation content and methods to the user using LLM.

[0060] According to the presentation system described in Appendix 1, when a problem occurs while driving the vehicle, the degree of need for action and the degree of acceptance are estimated, and accordingly, the presentation content and method that are estimated to increase the probability that the user will accept the presentation content and method are selected. Here, "when a problem occurs" may mean not only when some kind of problem actually occurs with respect to the vehicle, but also when the user recognizes the problem and makes an inquiry about the vehicle (i.e., when the problem or question is posed to the presentation system). Then, the LLM presents the information to the user using the selected presentation content and method. Thus, it becomes possible to provide presentations and answers that are adaptive to the nature of the problem and the user's situation.

[0061] [Note 2] The presentation system described in Appendix 2 of the present invention is characterized in that the response level estimation unit estimates the response level when the user makes an inquiry to the vehicle agent of the vehicle (for example, in its owner's manual Q&A system) when the problem occurs, the acceptance level estimation unit estimates the acceptance level when the user makes an inquiry when the problem occurs, the selection unit selects the content and method of the response to the inquiry that is estimated to increase the probability that the user will accept the answer to the inquiry, and the presentation unit provides the answer to the user using the selected response content and method via LLM.

[0062] According to the presentation system described in Appendix 2 of the present invention, when a user makes an inquiry in response to a problem occurring during vehicle operation or when a user has a question, the degree to which the inquiry needs to be addressed and the degree to which it is acceptable are estimated, and accordingly, the presentation content and method that are estimated to increase the probability that the user will accept the presentation content and method are selected as the response to the problem. Furthermore, the LLM presents the selected presentation content and method to the user.

[0063] [Note 3] The presentation system described in Appendix 3 of the present invention is the presentation system described in Appendix 1 or 2, characterized in that the selection unit selects to change the modality related to the presentation as part of the selection of the presentation content and method, and the presentation unit performs the presentation with the changed modality.

[0064] According to the presentation system described in Appendix 3 of the present invention, since the presentation is carried out with modalities changed by the selection unit, it is possible to create a situation that is easier for the user to understand and a situation that is easier for the user to draw attention to.

[0065] [Note 4] The presentation system described in Appendix 4 of the present invention is the presentation system described in Appendix 3, characterized in that the presentation unit presents text, images or videos, audio, or a combination thereof in response to the modified modality, and if the estimation result by the selection unit is unclear or the probability is below a predetermined threshold, the presentation is made using the combination.

[0066] According to the presentation system described in Appendix 4 of the present invention, presentations are made in the form of text, images, videos, audio, or a combination thereof, corresponding to the changed modality. In particular, if the estimation result by the selection unit does not show that the probability will be improved or the probability is below a predetermined threshold, the presentation is made in the aforementioned combination. Therefore, by utilizing modality changes, it becomes possible to create situations that are easier for users to understand or that are easier for users to pay attention to.

[0067] [Note 5] The presentation system according to Appendix 5 of the present invention is a presentation system according to any one of Appendix 1 to 4, characterized in that the acceptance level estimation unit estimates the acceptance level using LLM from information that is a factor in increasing or decreasing the acceptance level, including at least one of the user type relating to the user, the conversation in the vehicle, and the road / traffic conditions on which the vehicle is traveling, and the need for action estimation unit estimates the need for action using LLM from information that is a factor in increasing or decreasing the need for action, including at least one of the content of the problem and the state of the vehicle.

[0068] According to the presentation system described in Appendix 5 of the present invention, acceptance is estimated using LLM based on information that increases or decreases acceptance, such as user type, and the degree of need for action is estimated using LLM based on information that increases or decreases the degree of need for action, making it possible to provide appropriate presentations efficiently and with high accuracy.

[0069] [Note 6] The presentation system according to Appendix 6 of the present invention is a presentation system according to any one of Appendix 1 to 5, characterized in that the response requirement estimation unit estimates the response requirement at a future point in time advanced by the amount of time it takes for the user to understand the presentation content, the acceptance level estimation unit estimates the acceptance level at the future point in time, the selection unit selects what is estimated to improve the probability at the future point in time, and the presentation unit makes the presentation at the future point in time.

[0070] According to the presentation system described in Appendix 6 of the present invention, the time required for the user to understand the presentation is taken into consideration, making it possible to create a situation in which the user can understand more easily and is more likely to pay attention.

[0071] [Note 7] The present invention provides a method for presenting information that is adapted to the nature of the problem and the user situation as described in Appendix 7, comprising the steps of: estimating the degree to which action should be taken in response to a problem that occurs while a problem occurs while driving a vehicle, including at least one of the urgency and complexity of the problem; estimating the degree of acceptance of the vehicle user when the problem occurs; selecting, in accordance with the estimated degree to which action should be taken and the estimated degree of acceptance, presentation content and methods that are estimated to increase the probability that the user will accept the presentation content and methods; and presenting the selected presentation content and methods to the user using LLM.

[0072] According to the presentation method described in Appendix 7 of the present invention, similar to the presentation system described in Appendix 1, This makes it possible to provide suggestions and answers that are adapted to the nature of the problem and the user's situation.

[0073] The present invention may be modified as appropriate, without contradicting the gist or spirit of the invention as can be inferred from the claims and the specification as a whole, and presentation systems and methods involving such modifications are also included in the technical concept of the present invention. [Explanation of Symbols]

[0074] Interface section...20 Microphone... 20m Speaker... 20s Display section...20d Vehicle behavior / driving operation (CAN) section...21 Location information department...22 In-car and exterior camera video unit...23 Biometric Data Department...24 Server equipment...200 Processing unit...210 Need for Action Estimation Unit...211 Acceptance estimation unit...212 Selection section...213 Presentation section...214 DB...300 User Data Collection Department...306

Claims

1. A unit for estimating the degree of need for action, which indicates the degree to which action should be taken in response to a problem that occurs while the vehicle is in operation, including at least one of the urgency and complexity of the problem. When the aforementioned problem occurs, the acceptance estimation unit estimates the acceptance level of the vehicle's users, A selection unit that, in accordance with the estimated degree of need for action and the estimated degree of acceptance, selects, as the content and method of presentation to address the problem, those that are estimated to increase the probability that the user will accept the content and method of presentation. A presentation unit that presents the selected presentation content and method to the user via LLM. A presentation system characterized by comprising a feature that is adaptable to the content of the problem and the user's situation.

2. The aforementioned action-requiring degree estimation unit estimates the action-requiring degree when the vehicle agent of the vehicle in the event that the problem occurs, and when the user makes an inquiry to the vehicle agent. The acceptance estimation unit estimates the acceptance level when the user makes an inquiry, as in the case where the problem has occurred. The selection unit selects the content and method of the response to the inquiry that are estimated to increase the probability that the user will accept the response to the inquiry, as the probability, The presentation unit provides the user with the selected response content and method via LLM. A presentation system that is adaptable to the problem content and user situation described in feature 1.

3. The selection unit, as part of the selection of the presentation content and method, selects to change the modality related to the presentation. The presentation unit performs the presentation using the modified modality. A presentation system that is adaptable to the problem content and user situation as described in 1 or 2.

4. The aforementioned display unit is, In accordance with the modified modality, the presentation is made using text, images, videos, audio, or a combination thereof. If the estimation result by the selection unit indicates that it is unclear whether the probability can be improved or if the probability is below a predetermined threshold, the combination is presented. A presentation system that is adaptable to the problem content and user situation described in feature 3.

5. The acceptance estimation unit estimates the acceptance level using LLM based on information that is a factor in increasing or decreasing the acceptance level, including at least one of the user type relating to the user, the conversation inside the vehicle, and the road and traffic conditions on which the vehicle is traveling. The aforementioned response level estimation unit estimates the response level using LLM based on information that is a factor in increasing or decreasing the response level, including at least one of the content of the problem and the condition of the vehicle. A presentation system that is adaptable to the problem content and user situation described in any one of items 1 to 4.

6. The aforementioned action requirement estimation unit estimates the action requirement at a future point in time advanced by the amount of time it takes for the user to understand the presented content. The acceptance estimation unit estimates the acceptance at the future point in time, The selection unit selects those which are estimated to improve the probability at the future point in time. The display unit makes the display at the future point in time. A presentation system that is adaptable to the problem content and user situation described in any one of items 1 to 5.

7. A step of estimating the degree to which action should be taken in response to a problem that occurs while a vehicle is in operation, including at least one of the urgency and complexity of the problem, When the aforementioned problem occurs, the steps include: estimating the user's acceptance of the vehicle; The steps include selecting, in accordance with the estimated degree of need for action and the estimated degree of acceptance, the content and methods of presentation that are estimated to increase the probability that the user will accept the content and methods of presentation to address the problem, The LLM performs the presentation to the user using the selected presentation content and method. A method of presenting a problem that is adapted to the content of the problem and the user's situation, characterized by comprising the following features.

Images