Vehicle and user status / condition scenario generation system

Abstract

In a vehicle / user status / situation scenario generation system, even with input data acquired at uneven frequencies and intervals, it generates integrated scenarios that represent multiple states and situations in which the vehicle and user are placed. [Solution] The scenario generation system includes a scenario estimation unit (211) that uses machine learning to integrate multiple types of data that affect the state and situation of the input vehicle (100) and user, estimates a scenario for the state and situation, and further estimates a scenario for an arbitrary time for at least one of the input data if the acquisition frequency and interval of at least one of the data are uneven and not time-synchronized, and a learning model update unit (212) that updates the machine learning model.

Description

Technical Field

[0001] The present invention relates to the technical field of a scenario generation system that generates a scenario of at least one of a state and a situation (referred to as "state / situation" in the present application) related to at least one of a vehicle and its user (referred to as "vehicle / user" in the present application). In particular, it relates to the technical field of a scenario generation system that generates a scenario of a state / situation related to a vehicle / user using data in which at least one of an acquisition frequency and an acquisition interval (referred to as "acquisition frequency / interval" in the present application) is non-uniform.

Background Art

[0002] As technologies related to this type of system, for example, a driver state estimation device, a driver state estimation program, a daily operation report generation method, and a technology for detecting the state of a user driving a vehicle in particular have been proposed for the purpose of detecting a driver's distracted driving (see Patent Document 1). Also, for example, a driver state estimation device that can accurately estimate a driver's drowsiness level has been proposed (see Patent Document 2).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, according to the aforementioned background technology, it is not possible to estimate the state and circumstances of a vehicle or user based on data acquired at uneven frequencies and intervals. If an attempt were to make such an estimation, it would require a review of the method for estimating the state and circumstances, which presents a technical problem as it is not easily accomplished. In addition, according to the aforementioned background technology, there is a technical problem in that it is difficult or impossible to integrate multiple states.

[0005] The present invention aims to provide a vehicle and user state and situation scenario generation system that can generate integrated scenarios of multiple states and situations in which the vehicle and user are placed, even with input data acquired at uneven frequencies and intervals. [Means for solving the problem]

[0006] One aspect of the vehicle / user state / situation scenario generation system according to the present invention solves the above problems by integrating multiple types of data that affect the input vehicle / user state / situation using machine learning to estimate the state / situation scenario, and further, if the acquisition frequency and interval of at least one type of data among the input data are uneven and not time-synchronized, the scenario estimation unit estimates a scenario relating to an arbitrary time for the at least one type of data, the probability estimation unit estimates the probability of the estimated scenario, the predicted value estimation unit estimates predicted value data that has not been acquired at the time of scenario estimation for the at least one type of data, and the learning model update unit updates the machine learning model based on the estimated predicted value data, the measured values ​​corresponding to the estimated predicted value data among the input data, the estimated probabilities, and the input data. [Effects of the Invention]

[0007] According to one embodiment of the vehicle / user state / situation scenario generation system of the present invention, even for input data with uneven acquisition frequency and intervals and that is not time-synchronized, a scenario can be generated that integrates multiple states / situations in which the vehicle / user is placed, based on the scenario estimated by the scenario estimation unit, the probability of the scenario estimated by the probability estimation unit, the predicted value data estimated by the predicted value estimation unit, etc.

[0008] The effects and benefits of the present invention will be further clarified by the embodiments of the invention described below. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram showing the overall configuration of the scenario generation system according to the embodiment. [Figure 2] This is a schematic conceptual diagram illustrating the updating process of the scenario generation and scenario estimation methods in the scenario generation system according to the embodiment. [Figure 3] This flowchart shows an example of processing in the scenario generation system according to the embodiment. [Modes for carrying out the invention]

[0010] First, with reference to Figure 1, the overall configuration of the vehicle / user state / situation scenario generation system according to the embodiment (hereinafter simply referred to as the "scenario generation system") will be described.

[0011] In this embodiment, as will be explained in detail below, in the scenario generation of vehicle and user states and situations, even if the frequency and interval of input data acquisition are uneven and not time-synchronized, scenario estimation at an arbitrary time is performed by machine learning estimation, and prediction or estimation of data not acquired during scenario estimation is also performed simultaneously. Furthermore, the system is configured to update the scenario estimation method based on these predicted values, their measured values, the probability of the estimated scenario, and the observed data.

[0012] In this embodiment, the "machine learning" employed can include traditional AI learning systems such as supervised learning, unsupervised learning, or reinforcement learning, as well as novel generative AI or LLM (i.e., single-modal LLM or multimodal LLM) technologies that have recently been put into practical use, are currently under development, or will be developed in the future. For example, the AI ​​learning or LLM learning here may be configured using a neural network that performs efficient learning through representation learning, transfer learning, feature selection, fine-tuning or hyperparameter tuning, ensemble learning, etc.

[0013] As shown in Figure 1, the scenario generation system according to this embodiment is configured to include an on-board unit 101 mounted on a vehicle 100 and a server device 200. The on-board unit 101 and the server device 200 are connected to a communication network 10, such as the Internet or a dedicated network line. Other multiple or numerous vehicles 100 are also connected to the communication network 10 in the same way.

[0014] The communication network 10 also includes a user data collection unit 306 that collects data specific to the driver or user of the vehicle 100 (i.e., "user data"). In addition, the communication network 10 may also include a general knowledge collection unit (not shown) that collects "common sense" useful when performing estimations based on verbalized or documented data when employing LLM or multimodal LLM as machine learning, and a map data collection unit (not shown) that collects road maps, multipurpose maps, etc., useful when performing estimations based on data related to the state and conditions of the vehicle 100's operation. Furthermore, the communication network 10 may also include an external related knowledge collection unit (not shown) that collects information obtained outside of the vehicle 100 that can be used to impart domain knowledge by performing fine-tuning or hyper-tuning in the scenario generation system (i.e., "external related knowledge").

[0015] Such a user data collection unit 306, or the general knowledge collection unit and map data collection unit (not shown) described above, may be provided at least partially within the server device 200 or within the facility where the server device 200 is located, or within the vehicle-mounted unit 101 or within the vehicle 100.

[0016] A database (DB) 300, which stores various data including data used in the scenario generation system, is connected to the server device 200. The DB 300 may be connected to the server device 200 or the vehicle-mounted unit 101 via the communication network 10. The server device 200 is composed of various computer-equipped devices and various computer devices that perform centralized or distributed processing. In other words, the scenario generation system is constructed as a system that performs centralized or distributed processing using the large-scale data in the DB 300.

[0017] In Figure 1, vehicle 100 may be, for example, a so-called HEV (Hybrid Electric Vehicle), PHEV (Plugin HEV), FCEV (Fuel Cell EV), BEV (Battery Electric Vehicle), etc., or it may be a vehicle primarily powered by an internal combustion engine, but it is constructed as a so-called connected car.

[0018] The in-vehicle unit 101 comprises an interface unit 20, a vehicle behavior / driving operation (CAN) unit 21, a location information unit 22, an in-vehicle / out-of-vehicle camera video unit 23, a biometric data unit 24, a processing unit 26, and a communication unit 28.

[0019] The interface unit 20 is configured to enable communication with the driver or user inside the vehicle through voice and images. The interface unit 20 is configured to enable communication, for example, through voice input or a predetermined operation on an image, when using functions such as the navigation function, AV function, and automatic driving function provided in the vehicle 100. Regarding these functions, all or part of them may be configured to be executed by the processing unit 26, or part or all of them may be configured to be executed by the processing unit 210 on the server device 200 side (in other words, the in-vehicle unit 101 side solely serves as a browser function). The interface unit 20 is further configured to be able to appropriately output, in some form, voice or images, the data indicating the estimation result by machine learning passed from the server device 200 side.

[0020] The vehicle behavior and driving operation (CAN) unit 21 includes various sensors such as a vehicle speed sensor, an acceleration sensor, a distance sensor, an engine speed sensor, a temperature sensor, an altitude sensor, a pressure sensor, and a battery remaining amount sensor, which are respectively arranged at predetermined positions inside the vehicle 100. The vehicle behavior and driving operation (CAN) unit 21 is configured to be able to appropriately transmit, under the control of the processing unit 26, CAN (Controlled Area Network) data related to the current driving state of the vehicle 100 to the server device 200 side via the communication network 10 from the communication unit 28.

[0021] The position information unit 22 includes a GPS device, an inertial navigation positioning device, etc., and detects the current position of the vehicle 100. The position information unit 22 is further configured to generate raw position information data indicating, for example, latitude and longitude or coordinates, and be able to appropriately transmit it to the server device 200 side via the communication network 10 from the communication unit 28 under the control of the processing unit 26.

[0022] The in-vehicle and external camera moving image unit 23 includes one or more cameras such as CCDs and functions as a drive recorder for shooting the situation inside and outside the vehicle. The in-vehicle and external camera moving image unit 23 is further configured to be able to appropriately transmit the drive recorder video as image data to the server device 200 side via the communication network 10 from the communication unit 28 under the control of the processing unit 26.

[0023] The biometric data unit 24 directly or indirectly detects the driver's or user's biometric data using, for example, various vital sensors deployed at predetermined locations within the vehicle 100, as well as image analysis devices, voice analysis devices, etc., that analyze the driver's or user's appearance and voice. The biometric data unit 24 is further configured to transmit the detected biometric data to the server device 200 via the communication network 10 from the communication unit 28 as appropriate, under the control of the processing unit 26.

[0024] The processing unit 26 includes a CPU, memory, etc., that controls the interface unit 20, the vehicle behavior / driving operation (CAN) unit 21, the location information unit 22, the in-vehicle / out-of-vehicle camera video unit 23, the biometric data unit 24, and the communication unit 28. The processing unit 26 appropriately transmits data detected or generated at various parts of the vehicle 100 and its driver from the communication unit 28 to the server device 200 as data in a predetermined format. Furthermore, it is configured to appropriately receive data indicating estimation results from the server device 200 via the communication unit 28.

[0025] The communication unit 28 includes a modem or the like that is configured to communicate with the outside of the vehicle via the communication network 10. The communication unit 28 is configured to transmit various raw data collected by the vehicle 100 to the server device 200 via the communication network 10 as input data of type 1, under the control of the processing unit 26.

[0026] In Figure 1, the server device 200 is configured to include a communication unit 201 and a processing unit 210.

[0027] The communication unit 201, under the control of the processing unit 210, appropriately receives various data collected by the vehicle 100 via the communication network 10, and on the other hand, appropriately receives user data collected by the user data collection unit 306 via the communication network 10, and is configured to pass the received data to the processing unit 210.

[0028] The processing unit 210 comprises a user / vehicle state / situation scenario generation unit 211 (hereinafter simply referred to as "scenario generation unit 211" as appropriate) and a scenario estimation method update unit 212 (hereinafter simply referred to as "update unit 212" as appropriate). The various processes performed here will be described in detail later with reference to Figures 2 and 3.

[0029] DB300 is configured to include a large-scale and high-speed data input / output storage device that stores various data received by the server device 200 via the communication network 10, in particular various data necessary for estimation processing using machine learning, and data related to the estimation results or intermediate results generated by the processing unit 210.

[0030] Next, with reference to the block diagram in Figure 1 and the schematic conceptual diagram in Figure 2, we will explain the scenario generation process and the scenario estimation method update process, which are mainly performed in the processing unit 210, according to this embodiment.

[0031] In Figure 2, the scenario generation unit 211 receives input from the vehicle behavior / driving operation (CAN) unit 21, location information unit 22, in-vehicle / out-of-vehicle camera video unit 23, and biometric data unit 24 on the vehicle 100 side, in that order: CAN data from the vehicle 100, location information indicating the current position (e.g., raw location information data showing latitude, longitude, and coordinates), raw video data, and raw biometric data. The input here may be constant or periodic, but in this embodiment in particular, even if such input data is acquired at uneven frequencies or intervals, it is possible to generate scenarios with high accuracy and efficiency using machine learning.

[0032] The scenario generation unit 211 constitutes an example of a "scenario estimation unit," and uses machine learning, such as single-modal LLM (Large Language Models) learning, multimodal LLM learning, other AI learning, or generative AI learning, to integrate multiple types of data that affect the state and situation of the vehicle and user, as input as described above, and estimate the scenario of the state and situation. Furthermore, if the acquisition frequency and interval of at least one type of data among the input data are uneven and not time-synchronized, the scenario generation unit 211 uses machine learning to estimate the scenario for an arbitrary time point of the at least one type of data.

[0033] The scenario generation unit 211 also constitutes an example of a "probability estimation unit," which estimates the probability (or "confidence level") of the estimated scenarios, for example, using machine learning. As a result, a tabular data 15, as shown in Figure 2, is constructed by machine learning estimation, with each estimated scenario associated with each estimated probability.

[0034] The scenario generation unit 211 also constitutes an example of a "predicted value estimation unit," and in parallel with or preceding the scenario estimation process and the corresponding probability estimation process described above, it estimates, for example, machine learning, predicted value data for at least one type of data that has not been obtained at the time of scenario estimation. As a result, a tabular data 16 with predicted values ​​associated with each time point, as shown in Figure 2, is constructed by estimation using machine learning.

[0035] Once the data 15 and 16 have been constructed in this manner, the update unit 212 updates the machine learning model based on the estimated predicted value data, the actual values ​​from the input data corresponding to the estimated predicted value data, the estimated probabilities, and the input data.

[0036] Next, with reference to the block diagram in Figure 1 and the schematic conceptual diagram in Figure 2, as well as the flowchart in Figure 3, an example of processing in the scenario generation system according to this embodiment (in particular, processing performed by the processing unit 210 in the server device 200) will be explained.

[0037] In Figure 3, first, the scenario generation unit 211, functioning as a "scenario estimation unit," estimates the user and vehicle state and condition (step S1). Subsequently, it is determined whether the data input for estimation has uneven acquisition intervals (step S2).

[0038] If the result of this determination is that the data is non-uniform (Step S2: Yes), the scenario generation unit 211, using its function as a "prediction value estimation unit," estimates the predicted values ​​for future times (Step S3), and tabular data 16 (see Figure 2) is generated. Subsequently, the estimated predicted values ​​are added to the input data for machine learning, and the scenario generation unit 211, using its function as a "scenario estimation unit," generates scenarios using machine learning. At this time, the scenario generation unit 211, using its function as a "probability estimation unit," estimates the probability of each estimated scenario, and tabular data 15 (see Figure 2) is generated (Step S4).

[0039] On the other hand, if the determination in step S2 is not that the data is heterogeneous (step S2: No), the scenario is generated without going through step S3 (step S4). The scenario generation in step S4 generates data 15 (see Figure 2) in which various scenarios such as "The current location is the intersection in front of Tokyo Tower. Tokyo Tower is visible on the right," "Currently, the user is waiting at a traffic light at the intersection in front of Tokyo Tower," "User A is driving on the highway at a speed of 80 kph," and "User A is fatigued and is advised to take a break at the next service area B" are associated with their respective probabilities.

[0040] Next, the scenario generation unit 211 determines again whether the data acquisition intervals are uneven (step S5). If the data is uneven (step S5: Yes), the vehicle behavior / driving operation (CAN) unit 21, position information unit 22, in-vehicle / out-of-vehicle camera video unit 23, biometric data unit 24, etc., are used to acquire actual values ​​for future times (step S6).

[0041] Next, based on the discrepancy between the predicted values ​​in the data 16 estimated in step S3 and the actual values ​​obtained in step S6, the update unit 212, functioning as a "learning model update unit," updates the machine learning state / situation estimation model in the scenario generation unit 211 (step S7).

[0042] Subsequently, when the final time for data acquisition is reached (Step S8: Yes), the series of processes related to scenario generation are terminated. Alternatively, unless the final time for data acquisition has been reached (Step S8: No), the series of processes from Step S1 onward related to scenario generation are repeatedly executed.

[0043] On the other hand, if the result of the determination in step S5 is that the data acquisition interval is not uneven (step S5: No), the process proceeds to step S8, where it is determined whether or not the final time for data acquisition has been reached (step S8), and the series of processes is either repeated or terminated. Note that such a series of processes may be called periodically or irregularly as a subroutine that can be executed repeatedly in a short time and executed as appropriate.

[0044] As described in detail above, according to this embodiment, even with input data acquired at uneven frequencies and intervals, it is possible to generate a scenario that integrates multiple states and situations in which the vehicle and user are placed.

[0045] In the scenario generation unit 211, which performs various estimation processes, for example, in order to efficiently execute the machine learning described above, the scenario may be converted into text using LLM (Language Language Modeling) at least partially, and then vectorized. Furthermore, in each process using the machine learning described above, a large-scale language model may be fine-tuned using LLM, for example, using a large amount of text data. This makes it possible to adapt to various natural language processing (NLP) tasks such as text classification, sentiment analysis, information extraction, text summarization, text generation, and question answering.

[0046] In this embodiment, the scenario generation unit 211 and the update unit 212 described above are represented as separate parts, function blocks that perform different processing. However, they may be composed of a single processor or the like as hardware, or they may be configured to be functionally treated separately in software.

[0047] In the scenario generation system according to this embodiment, the scenario generation unit 211 and the update unit 212 may be configured to update existing or future improved autoregressive generative models, in addition to models such as multimodal LLM, chat GPT, or Transformer. Alternatively, they may be configured to update a model that generates scenarios as text after estimating individual states and situations using a time series model. The model here may be, for example, one estimated using a state-space model, latent variable model, or time series prediction model (e.g., existing Kalman filters, particle filters, recurrent neural networks, LTSM, HiPPO, LSSL, S4, Mamba, etc.), and obtained by inputting its high-dimensional output value or embedding vector as input to an autoregressive generative model. In any case, the effect of this embodiment, "the ability to generate scenarios even for input data with non-uniform acquisition frequencies and intervals," is appropriately demonstrated.

[0048] Note The following additional information is disclosed regarding the embodiments described above.

[0049] [Note 1] The vehicle / user state / situation scenario generation system described in Appendix 1 of the present invention uses machine learning to integrate multiple types of data that affect the input vehicle / user state / situation to estimate the state / situation scenario, and further includes a scenario estimation unit that estimates a scenario for an arbitrary time for at least one of the input data if the acquisition frequency and interval of at least one of the input data are uneven and not time-synchronized, a probability estimation unit that estimates the probability of the estimated scenario, a predictive value estimation unit that estimates predictive value data that has not been acquired at the time of scenario estimation for at least one of the data, and a learning model update unit that updates the machine learning model based on the estimated predictive value data, the measured values ​​corresponding to the estimated predictive value data from the input data, the estimated probabilities, and the input data.

[0050] According to the scenario generation system described in Appendix 1, the scenario estimation unit uses machine learning to integrate multiple types of data that affect the state and situation of the input vehicle and user, and estimates a scenario for the said state and situation. Furthermore, if the acquisition frequency and interval of at least one type of data among the input data are uneven and not time-synchronized, the scenario estimation unit uses machine learning to estimate a scenario for an arbitrary time for the at least one type of data. Then, the probability estimation unit estimates the probability of the estimated scenario, for example, using machine learning. In parallel with or around this time, the prediction value estimation unit uses, for example, machine learning to estimate prediction value data for the at least one type of data that has not been acquired at the time of scenario estimation. Then, based on this estimated prediction value data, the actual values ​​corresponding to the estimated prediction value data among the input data, the estimated probability, and the input data, the learning model update unit updates the machine learning model. Therefore, even for input data with uneven acquisition frequency and interval, it is possible to generate a scenario that integrates multiple states and situations in which the vehicle and user are placed.

[0051] [Note 2] The scenario generation system described in Appendix 2 of the present invention is the scenario generation system described in Appendix 1, characterized in that the input data includes vehicle behavior and driving operation data including CAN data, location information data, in-vehicle and out-of-vehicle camera video data, and biometric data related to the vehicle and user.

[0052] According to the scenario generation system described in Appendix 2 of the present invention, the scenario estimation unit estimates a scenario based on vehicle behavior / driving operation data, location information data, in-vehicle / out-of-vehicle camera video data, biometric data, etc., and uses this to update the machine learning model. Therefore, even if the input data has uneven acquisition frequency and intervals, it is possible to generate a scenario that integrates multiple states and situations in which the vehicle and user are placed.

[0053] [Note 3] The scenario generation system described in Appendix 3 of the present invention is a scenario generation system described in Appendix 1 or 2, characterized in that the scenario estimation unit estimates each of the scenarios as a single sentence, or estimates each of the predefined items in addition to or instead of thereto.

[0054] According to the scenario generation system described in Appendix 3 of the present invention, the scenario estimation unit estimates each of the scenarios as a single sentence using machine learning such as LLM or multimodal LLM, or in addition to or instead of this, estimates each of the predefined items. In other words, the scenario is estimated by generating sentence-like or verbalized data from non-sentenced or non-linguistic data. Therefore, by including the content of sentences such as common sense, general conventions, papers, and related literature related to the sentence-like or verbalized data generated by LLM or multimodal LLM as the basis for estimation, it is possible to improve the accuracy and efficiency of the estimation process. In addition to or instead of this, by estimating each of the predefined items, it is also possible to appropriately improve the accuracy and efficiency of the estimation process compared to randomly performing machine learning without any definitions.

[0055] [Note 4] The scenario generation system described in Appendix 4 of the present invention is a scenario generation system described in any one of Appendix 1 to 3, characterized in that the predicted value estimation unit estimates the unacquired predicted value data by imputation of missing values ​​or time series forecasting.

[0056] According to the scenario generation system described in Appendix 4 of the present invention, the predicted value estimation unit uses existing or future improved methods for imputing missing values ​​or time series forecasting, making it possible to estimate predicted value data with high accuracy or efficiency.

[0057] [Note 5] The scenario generation method described in Appendix 5 of the present invention comprises: a scenario estimation step in which machine learning integrates multiple types of data that affect the state and situation of the input vehicle and user to estimate a scenario of the state and situation; further, if the acquisition frequency and interval of at least one type of data among the input data are uneven and not time-synchronized, a scenario estimation step in which a scenario relating to an arbitrary time for the at least one type of data is estimated; a probability estimation step in which the probability of the estimated scenario is estimated; a prediction value estimation step in which prediction value data that has not been acquired at the time of scenario estimation for the at least one type of data; and a learning model update step in which the machine learning model is updated based on the estimated prediction value data, the actual values ​​from the input data corresponding to the estimated prediction value data, the estimated probability, and the input data.

[0058] According to the scenario generation method described in Appendix 5 of the present invention, similar to the scenario generation system described in Appendix 1, it is possible to generate a scenario that integrates multiple states and situations in which the vehicle and user are placed, even for input data with uneven acquisition frequencies and intervals.

[0059] The present invention may be modified as appropriate, without contradicting the gist or spirit of the invention as can be inferred from the claims and the specification as a whole, and scenario generation systems and methods involving such modifications are also included in the technical concept of the present invention. [Explanation of symbols]

[0060] Vehicle behavior / driving operation (CAN) section...21 Location information department...22 In-car and exterior camera video unit...23 Biometric Data Department...24 Server equipment...200 Processing unit...210 User / Vehicle Status / Condition Scenario Generation Unit...211 Update section for scenario estimation method...212 DB...300 User Data Collection Department...306

Claims

1. A scenario estimation unit that uses machine learning to integrate multiple types of data that affect the state and situation of the input vehicle and user, estimates a scenario for the said state and situation, and further estimates a scenario for an arbitrary time for at least one of the input data if the acquisition frequency and interval of at least one of the input data are uneven and not time-synchronized, A probability estimation unit that estimates the probability of the aforementioned estimated scenario, A predictive value estimation unit that estimates predictive value data that has not been obtained at the time of scenario estimation with respect to at least one of the aforementioned data, A learning model update unit updates the machine learning model based on the estimated predicted value data, the actual values ​​from the input data corresponding to the estimated predicted value data, the estimated probability, and the input data. A scenario generation system for vehicle and user status and conditions, characterized by comprising the following features.

2. The vehicle / user state / condition scenario generation system according to claim 1, characterized in that the input data includes vehicle behavior / driving operation data including CAN (Controller Area Network) data, location information data, in-vehicle and out-of-vehicle camera motion image data, and biometric data related to the vehicle / user.

3. The scenario generation system for vehicle and user status and conditions according to claim 1 or 2, characterized in that the scenario estimation unit estimates each of the scenarios as a single sentence, or estimates each of the scenarios in addition to or instead of each of the predefined items.

4. The vehicle / user status / condition scenario generation system according to any one of claims 1 to 3, characterized in that the predicted value estimation unit estimates the unacquired predicted value data by imputation of missing values ​​or time series forecasting.

5. The machine learning process involves integrating multiple types of data that influence the state and situation of the input vehicle and user to estimate a scenario for the said state and situation, and further, if the acquisition frequency and interval of at least one type of data among the input data are uneven and not time-synchronized, the scenario estimation step estimates a scenario for an arbitrary time point of the said at least one type of data. A probability estimation step for estimating the probability of the aforementioned estimated scenario, A predictive value estimation step for estimating predictive value data that has not been obtained at the time of scenario estimation with respect to at least one of the aforementioned data, A learning model update step that updates the machine learning model based on the estimated predicted value data, the actual values ​​from the input data corresponding to the estimated predicted value data, the estimated probability, and the input data. A method for generating scenarios of the state and situation in which a vehicle and user are placed, characterized by comprising the following features.

Images