Vehicle information processing system and vehicle information processing method

Abstract

A vehicle information processing system according to the present invention for processing time-series data collected in time series from a vehicle is constituted by a computer that has an arithmetic unit for executing prescribed processing and a storage device connected to the arithmetic unit. The arithmetic unit receives, from a user via an input device, an inquiry to a vehicle system for controlling the vehicle, determines a type and a range of time-series data related to the inquiry from among a plurality of types of time-series data collected by the vehicle system, extracts time-series data to be processed, generates a time-series data description text for describing, in a natural language, a processing result of the extracted time-series data, and outputs the time-series data description text.

Description

Vehicle information processing system and vehicle information processing method Incorporation by Reference

[0001] The present application claims priority from Japanese Patent Application No. 2025-52134 filed on March 26, 2025 (Reiwa 7), the content of which is incorporated into the present application by reference thereto.

[0002] The present invention relates to a vehicle information processing system.

[0003] Nowadays, the functions of vehicles are diversified and operations are becoming more complicated, thus leading to advanced inquiries from users.

[0004] As background art in the present technical field, there is the following prior art. Patent Document 1 (Japanese Unexamined Patent Application Publication No. 2019-145002) discloses an information processing method executed by an information processing apparatus, the method comprising the steps of: acquiring user input information including a question sentence about a vehicle; and outputting an answer sentence in accordance with a detail level, which is an indicator of the detail of a sentence and is determined based on a detection result of a characteristic value of the question sentence.

[0005] As described above, since vehicle functions are diversified and operations are becoming more complicated, it may be difficult to provide appropriate answers to advanced inquiries about vehicles from users (such as occupants, mechanics, and developers).

[0006] Therefore, there is a demand for limiting data analysis to the scope of the problem, improving analysis accuracy, and providing flexible and highly accurate answers to inquiries from users.

[0007] A representative example of the invention disclosed in this application is as follows: A vehicle information processing system for processing time-series data collected from a vehicle in a time-series manner, comprising a computer having an arithmetic unit that performs predetermined processing and a storage device connected to the arithmetic unit, wherein the arithmetic unit receives a query request from a user via an input device to a vehicle system that controls the vehicle, determines the type and range of time-series data related to the query request from among a plurality of types of time-series data collected by the vehicle system, extracts the time-series data to be processed, generates a time-series data description that explains the processing results of the extracted time-series data in natural language, and outputs the time-series data description.

[0008] According to one aspect of the present invention, data analysis can be limited to the scope of the problem and the accuracy of the analysis can be improved. Problems, configurations, and effects other than those described above will be clarified by the following description of embodiments.

[0009] This is a diagram showing the configuration of the vehicle information processing system of Embodiment 1. This is a diagram showing the hardware configuration of the server constituting the vehicle information processing system of Embodiment 1. This is a flowchart of the processing executed by the request receiving unit of Embodiment 1. This is a diagram showing an example of a prompt generated by the request receiving unit of Embodiment 1. This is a diagram showing an example of a prompt generated by the request receiving unit of Embodiment 1. This is a diagram showing the input to the request receiving unit of Embodiment 1. This is a diagram showing an example of a prompt generated by the request receiving unit of Embodiment 1. This is a diagram showing the configuration of the output unit of Embodiment 1. This is a diagram showing an example of the configuration of the application database of Embodiment 1. This is a diagram showing another configuration of the output unit of Embodiment 1. This is a diagram showing an example of the configuration of the problem analysis database of Embodiment 1. This is a diagram showing an example of the output unit of Embodiment 1 changing the data collection conditions. This is a diagram showing an example of the output unit of Embodiment 1 changing the data collection conditions. This is a diagram showing an example of the output unit of Embodiment 1 changing the data collection conditions. This is a diagram showing another configuration of the output unit of Embodiment 1. This is a diagram showing the content presented to the driver for the problem of Embodiment 1. This is a diagram showing another configuration of the vehicle information processing system of Embodiment 1. This is a diagram showing an example of the configuration of vehicle data of Embodiment 1. This is a diagram showing the use of vehicle data of Embodiment 1. This is a diagram showing the configuration of the vehicle information processing system of Embodiment 2. This is a diagram showing the configuration of the vehicle information processing system of Embodiment 3. This is a diagram showing the hardware configuration of the electronic control unit constituting the vehicle information processing system of Embodiment 3.

[0010] <First Embodiment> Figure 1 is a diagram showing the configuration of the vehicle information processing system 10 of Embodiment 1. In the vehicle information processing system 10 of Embodiment 1, the functions for receiving user input and transmitting vehicle data are installed on the in-vehicle side, while the other functions are installed on the server.

[0011] The vehicle information processing system 10 of Embodiment 1 includes a vehicle data storage unit 11, a request reception unit 12, a vehicle data extraction unit 13, a vehicle data processing unit 14, a time-series data explanatory text generation unit 15, and an output unit 16.

[0012] The vehicle data storage unit 11 stores vehicle data input to the vehicle information processing system 10. Vehicle data includes, for example, data related to the behavior of the vehicle and on-board devices, such as vehicle speed, acceleration, driving force, engine output, steering angle, accelerator operation, driver sensing data, and external environment sensing data, and is collected in a time series while the vehicle is in operation.

[0013] The request receiving unit 12 is a functional block that receives inquiries entered by the user and is composed of a machine learning model that has learned the relationship between user inquiries and vehicle-related data. More specifically, the machine learning model used by the request receiving unit 12 has learned inquiries annotated with information about vehicle behavior, and the relationship between inquiries and environmental performance, safety, malfunctions, and comfort. Based on the inquiries received from the user, the request receiving unit 12 outputs issues related to those inquiries. The detailed processing content of the request receiving unit 12 will be described later. Input to the request receiving unit 12 may be via a terminal operated by the user (smartphone, personal computer, etc.) as the input device 20, or via an in-vehicle terminal (navigation system, control unit provided on the instrument panel or steering wheel) as the input device 30, as shown in Figure 6. By accepting information input not only from in-vehicle devices but also from terminals outside the vehicle, the accuracy of the analysis can be improved. The request receiving unit 12 can accept various inquiries entered by the user via the input device 30. The request receiving unit 12 receives inquiries regarding issues that occur in the vehicle, such as safety issues or malfunctions that occur while the vehicle is in operation, issues related to passenger comfort, issues related to environmental performance, etc.

[0014] Furthermore, images displayed on the navigation system screen, images taken inside or outside the vehicle by external devices such as smartphones, and sounds collected by in-vehicle devices or external devices such as smartphones may also be input to the request receiving unit 12. For example, if the request receiving unit 12 has a voice recognition function, it may recognize a user's natural language statement such as "What is this sound?" and collect ambient sounds heard by the user. Details of the processing performed by the request receiving unit 12 will be explained with reference to Figure 3A. Including images and sounds in the query can improve the accuracy of the analysis.

[0015] The vehicle data extraction unit 13 is a functional block that extracts vehicle data (for example, physical quantities such as throttle operation information and brake operation information collected from the vehicle in time series, scene information, and images) related to a user inquiry that the request reception unit 12 has determined to accept, from the vehicle data storage unit 11. The vehicle data extraction unit 13 can be composed of, for example, a machine learning model that has learned the relationship between words or sentences related to vehicle behavior and vehicle data related to those words or sentences. The vehicle data extraction unit 13 inputs the user inquiry that the request reception unit 12 has determined to accept into the machine learning model and outputs vehicle data related to the inquiry from the machine learning model. Here, the vehicle data extraction unit 13 can determine the type and range of vehicle data related to the inquiry and extract the vehicle data to be processed from the vehicle data storage unit 11. The type of vehicle data includes information such as the type of physical quantity such as throttle operation information and brake operation information, and the type of data such as images and point clouds. The vehicle data range is information that determines the target of vehicle data to be extracted from the vehicle data storage unit 11, based on specifications such as the time the vehicle data was collected and the driving scene. For example, if a machine learning model has already learned the relationship between a sentence about vehicle behavior, such as "acceleration," and vehicle data such as "vehicle acceleration is 0.1G or higher," and the vehicle data extraction unit 13 receives an inquiry such as "the vehicle vibrated when accelerating," it can extract vehicle data collected during a time period when the vehicle's acceleration was above a certain value, based on the sentence "when accelerating." In this way, the vehicle data extraction unit 13 limits the range of data analysis processed by the subsequent vehicle data processing unit 14 to vehicle data related to the inquiry, thereby improving the accuracy of the analysis results.

[0016] The vehicle data processing unit 14 is a functional block that determines whether vehicle data is normal or abnormal, and can be composed of a machine learning model that has learned whether the physical quantities necessary to analyze the content of user inquiries are normal, abnormal, or unanalyzable. In response to an inquiry, the vehicle data processing unit 14 performs analysis processing of the vehicle data extracted by the aforementioned vehicle data extraction unit 13. For example, when the request reception unit 12 receives an inquiry that "the vehicle vibrated when the vehicle was accelerating," the vehicle data extraction unit 13 extracts vehicle data related to vehicle vibration (e.g., acceleration, angular velocity, vibration velocity, etc.) and data related to the vehicle's driving scene (e.g., images captured by an onboard camera, etc.) from the vehicle data storage unit 11, which are vehicle data collected when the vehicle was accelerating (e.g., time periods when acceleration of 0.1G or more was measured as a threshold). Next, the vehicle data processing unit 14 analyzes the vibration state of the vehicle from the extracted vehicle data, identifies the direction of vibration of the vehicle body (vertical direction, rotational direction, etc.) and the driving scene at the time of vibration (e.g., driving at 40 km / h through an intersection where it is snowing), and outputs information regarding the direction of vibration of the vehicle body and the driving scene at the time of vibration to the subsequent time-series data description generation unit 15. Note that the content of the vehicle data analysis by the vehicle data processing unit 14 is not limited to this example.

[0017] The time-series data description generation unit 15 is a functional block that generates a description (time-series data description) that includes the analysis results, including the causes of issues related to the user's inquiry. It consists of a machine learning model that has learned the relationship between the methods and physical quantities necessary to analyze the content of the inquiry, whether it is normal, abnormal, or unanalyzable, and the causes of issues related to the inquiry. The time-series data description generation unit 15 generates a time-series data description that explains the processing results of the time-series data by the vehicle data processing unit 14 in natural language. The time-series data description generation unit 15 inputs the processing results of the time-series data by the vehicle data processing unit 14 and a prompt instructing the machine learning model to explain the processing results in natural language, and obtains the time-series data description output from the machine learning model. For example, when the time-series data description generation unit 15 receives information on the direction of vibration of the vehicle body and the driving scene during vibration as processing results from the vehicle data processing unit 14, it inputs the processing results and a prompt instructing the machine learning model to generate a time-series data description. The time-series data description generation unit 15 acquires the time-series data description "When accelerating at 40 km / h through an intersection where it is snowing, the vehicle vibrates vertically, and the user is concerned about this issue." which is generated by the machine learning model. Note that the vehicle data processing unit 14 and the time-series data description generation unit 15 may be configured with the same machine learning model. In this case, the vehicle information processing system 10 can input the inquiry received from the user, the vehicle data extracted by the vehicle data extraction unit 13, and the prompt to output the time-series data description into the machine learning model, and acquire the time-series data description corresponding to the prompt generated by the machine learning model.

[0018] The output unit 16 is a functional block that outputs analysis results to be presented to the user. The output unit 16 outputs the time-series data explanation generated by the time-series data explanation generation unit 15 to the vehicle. The output unit 16 can present the time-series data explanation to the user, for example, via an output device mounted on the vehicle or via the user's terminal. The output unit 16 can also output to the vehicle an association between the cause of the problem explained by the time-series data explanation and inspection items that the user can visually inspect. In this case, the output unit 16 can be configured, for example, with a machine learning model that has learned the relationship between the cause of the problem related to the user's inquiry and the inspection items that the user can visually inspect. Furthermore, the output unit 16 can output to the vehicle information regarding means (applications) for solving the problem explained by the time-series data explanation. Details of the output unit 16 will be explained with reference to Figures 8 and 10.

[0019] The machine learning models comprising the request reception unit 12, the vehicle data extraction unit 13, the vehicle data processing unit 14, the time-series data description generation unit 15, and the output unit 16 may be large-scale language models that have learned information obtainable in the internet space.

[0020] Figure 2 shows the hardware configuration of the server that makes up the vehicle information processing system 10.

[0021] The server constituting the vehicle information processing system 10 is composed of a computer having a processor (CPU: Central Processing Unit) 21, memory 22, auxiliary storage device 23, and communication interface 24. The server may also have an input interface 25 and an output interface 26.

[0022] The processor 21 is an arithmetic unit that executes programs stored in the memory 22. By executing various programs, the processor 21 operates as each functional unit of the vehicle information processing system 10 (for example, the request reception unit 12, the vehicle data extraction unit 13, the vehicle data processing unit 14, the time-series data explanation text generation unit 15, and the output unit 16). Note that some of the processing performed by the processor 21 when executing programs may be executed by other arithmetic units (for example, hardware such as ASICs, FPGAs, or GPUs).

[0023] The memory 22 includes a non-volatile memory element called ROM and a volatile memory element called RAM. The ROM stores immutable programs (e.g., BIOS). The RAM is a high-speed, volatile memory element such as DRAM (Dynamic Random Access Memory) and temporarily stores programs executed by the processor 21 and data used during program execution.

[0024] The auxiliary storage device 23 is, for example, a high-capacity, non-volatile storage device such as a magnetic storage device (HDD) or flash memory (SSD). The auxiliary storage device 23 also stores data used by the processor 21 when executing a program and the program that the processor 21 executes. In other words, the program is read from the auxiliary storage device 23, loaded into memory 22, and executed by the processor 21, thereby realizing each function of the vehicle information processing system 10.

[0025] The communication interface 24 is a network interface device that controls communication with other devices according to a predetermined protocol.

[0026] The input interface 25 is an interface to which input devices such as keyboards and mice are connected and to which input is received from the operator. The output interface 26 is an interface to which output devices such as display devices and printers are connected and to which the execution results of the program are output in a format that can be viewed by the operator.

[0027] The program executed by the processor 21 is provided to the vehicle information processing system 10 via removable media (such as a CD-ROM or flash memory) or a network, and is stored in a non-volatile auxiliary storage device 23, which is a non-temporary storage medium. For this reason, the vehicle information processing system 10 should have an interface for reading data from the removable media.

[0028] The vehicle information processing system 10 is a computer system that operates on a single physical computer or on multiple computers configured logically or physically, and may operate on a virtual computer built on multiple physical computer resources. For example, each functional unit may operate on a separate physical or logical computer, or multiple units may be combined and operate on a single physical or logical computer.

[0029] Figure 3 is a flowchart of the process executed by the request receiving unit 12 in Embodiment 1.

[0030] The request receiving unit 12 determines whether the user's input query has been verbalized by checking whether the query contains a phrase indicating at least one of the following: the name of an in-vehicle device, an object, and vehicle behavior (121). For example, the prompt shown in Figure 7, "Determine whether the query contains information indicating the name of an in-vehicle device, an object, or vehicle behavior," may be input to a machine learning model to obtain a determination result on whether the query contains a phrase indicating the name of an in-vehicle device, a phrase indicating an object, or a phrase indicating vehicle behavior. Here, an object refers to an object related to a problem occurring in a vehicle, such as a car, a person, a bicycle, a traffic light, a guardrail, etc.

[0031] Furthermore, the auxiliary storage device 23 may store a phrase indicating at least one of the following: the name of an in-vehicle device, an object, and the vehicle behavior. The request receiving unit 12 may then determine the similarity between the inquiry entered by the user and the stored phrase indicating at least one of the in-vehicle device name, object, and vehicle behavior, and determine that the inquiry request has been verbalized if the similarity is higher than a predetermined threshold. A known determination algorithm such as cosine similarity can be used to determine the similarity of the phrases.

[0032] The request receiving unit 12 then determines that if the problem is not clearly articulated, it will be difficult to identify the cause of the inquiry entered by the user, and provides the user with advice to clarify the problem in the inquiry (125). For example, it may provide advice to inquire about the name of the in-vehicle equipment, the object, information about the vehicle's behavior, or the location where the event occurred. By providing the user with advice to clarify the problem, the final analysis accuracy can be improved even if an ambiguous inquiry is entered.

[0033] On the other hand, the request receiving unit 12 determines whether to accept the inquiry if it determines that the problem has been articulated (122). The request receiving unit 12 decides to accept the inquiry if it is an inquiry about the behavior of the vehicle. For example, the request receiving unit 12 may input the prompt shown in Figure 4, "#For this inquiry, please determine whether it is a vehicle malfunction or something the driver found unusual, or something unrelated to vehicle information, and then answer. #Example inquiry: When I was driving on the highway earlier, it was hard to see because of the glare," into a machine learning model to obtain a result of determining whether the inquiry entered by the user is an inquiry about the behavior of the vehicle. For example, in response to the prompt shown in Figure 4, the answer obtained is, "The problem that the driver felt was 'hard to see because of the glare' is not a malfunction of the vehicle itself, but is determined to be due to environmental factors. Specifically, it is caused by a decrease in visibility due to the angle of sunlight and light reflection, and is not directly related to vehicle information or performance." Then, the request receiving unit 12 accepts the inquiry if it is an inquiry about the behavior of the vehicle. On the other hand, if the inquiry entered by the user is not related to the vehicle's behavior, the request reception unit 12 terminates processing. In this case, the user may be informed that the inquiry cannot be accepted.

[0034] Figure 5 shows another example of determining whether the inquiry in step 122 concerns vehicle behavior. The prompt shown in Figure 5 can be input into a machine learning model to obtain a result indicating whether the user's input inquiry concerns vehicle behavior. For example, in response to the prompt "#Please determine whether this inquiry concerns vehicle behavior or not #Example inquiry) When I was driving on the highway earlier, the deceleration felt different than usual when I applied the brakes," the answer obtained is "This inquiry concerns 'vehicle behavior.' Specifically, it concerns 'differences in deceleration when applying the brakes,' which relates to the vehicle's braking system and behavior." In another example, in response to the prompt "#Please determine whether this inquiry concerns vehicle behavior or not #Example inquiry) When I was driving on the highway earlier, the navigation screen went completely black," the answer obtained is "This inquiry does not concern vehicle behavior, but rather the operation of the navigation system (navigator). The problem of the screen going black is a technical issue related to the navigation or display, not to the vehicle's control or behavior itself."

[0035] Furthermore, a problem description document explaining the issues related to the vehicle system may be stored in the auxiliary storage device 23, and the request reception unit 12 may use a predetermined algorithm to determine the similarity between the inquiry entered by the user and the stored problem description document, and may accept inquiries whose similarity is higher than a predetermined threshold.

[0036] By limiting user inquiries to those related to vehicles, the data analysis scope is narrowed to vehicle data only, and analysis processing is not performed for other data, thereby improving the accuracy of the analysis results.

[0037] If the request receiving unit 12 determines that it will accept the inquiry, it determines the processing priority (123). For example, the request receiving unit 12 inputs the following prompt "#Please assign one of the following classifications to the inquiry: "safety", "defect", "comfort", or "environmental performance" #Example inquiry: When I was driving on the highway earlier, the deceleration felt different than usual when I applied the brakes" into the machine learning model to identify the category to which the inquiry belongs. The priority of the categories included in the prompt is predetermined, and the inquiry has the priority of the identified category. In the prompt mentioned above, the inquiry is classified as unclassifiable in addition to the four categories, but it may be classified into five or more categories by adding other categories. As described above, the request receiving unit 12 classifies the inquiry into one of the categories of safety, defect, comfort, or environmental performance. In subsequent functional blocks, inquiries classified as safety or defect are processed with priority over inquiries classified as comfort or environmental performance. In other words, inquiries classified as safety or defects are given a higher priority than inquiries classified as comfort or environmental performance, and when multiple inquiries are received, the order of subsequent processing (124) is determined according to priority. By analyzing high-priority inquiries early and providing solutions to the issues early, the accuracy of the analysis can be improved.

[0038] Then, the request receiving unit 12 outputs the tasks related to the inquiry according to the determined processing priority and terminates processing (124). The request receiving unit 12 can be composed of a machine learning model that has learned the inquiry and the tasks related to the inquiry. The request receiving unit 12 inputs the inquiry into the machine learning model and outputs the tasks generated by the machine learning model. The request receiving unit 12 may output a task description that explains the content of the task in natural language, or it may output it in another data format.

[0039] Figure 8 shows an example of the configuration of the output unit 16 in Embodiment 1.

[0040] The output unit 16 shown in Figure 8 includes an application search unit 161 and an application database 162. It searches for in-vehicle applications from time-series data descriptions input from the time-series data description generation unit 15 and outputs them to the vehicle.

[0041] As shown in Figure 9, the application database 162 stores problems in association with applications that are effective in solving those problems. For example, to solve the problem "I want to be notified when I deviate from the white line," a lane departure warning application is effective, so the application "Lane Departure Warning" is stored as an association with the problem "I want to be notified when I deviate from the white line." Similarly, applications that are effective in solving other problems are stored in the database.

[0042] The application search unit 161 determines the similarity between the input time-series data description and the issues stored in the application database 162, retrieves the application with the highest similarity from the application database 162, and outputs the retrieved application to the vehicle. For example, if the time-series data description input from the time-series data description generation unit 15 contains words or phrases such as "white line" or "deviation," and the application search unit 161 determines that the time-series data description has a high similarity to the issue "I want to be notified when I deviate from the white line" stored in the application database 162, the output unit 16 searches the application "Lane Departure Warning" from the application database 162 and outputs it to the vehicle. The application search unit 161 can use a machine learning model or a known similarity determination algorithm, as described later. The application search unit 161 may also select an application corresponding to the issue with the highest similarity to the time-series data description from among multiple issues, or it may select an application corresponding to one or more issues with a similarity to the time-series data description that exceeds a threshold from among multiple issues. The vehicle installs the received application into the electronic control unit to improve or add functionality for responding to user-inputted inquiries. The vehicle may also be configured to present the received application to the user and install it only after obtaining user permission.

[0043] The application search unit 161 may use a machine learning model to identify applications that correspond to the input time-series data descriptions. The machine learning model used by the application search unit 161 has learned the correspondence between application functions and applications, and outputs applications that have functions similar to the time-series data descriptions.

[0044] The application search unit 161 may input a prompt including a time-series data description into a machine learning model (e.g., a large language model) to obtain information on an application effective for solving the problem described in the time-series data description. This machine learning model has learned the relationship between a problem in a vehicle and an application effective for solving the problem, and outputs an application effective for solving the problem described in the time-series data description.

[0045] As described above, the output unit 16 not only outputs a description of the problem, but also provides an application effective for solving the problem. Therefore, by comparing the behavior of the vehicle before and after the introduction of the provided application, the analysis accuracy of the problem can be improved.

[0046] FIG. 10 is a diagram showing another configuration of the output unit 16 according to the first embodiment.

[0047] The output unit 16 shown in FIG. 10 includes a data collection condition updating unit 163 and a problem analysis database 164, retrieves data collection conditions from the time-series data description input from the time-series data description generating unit 15, and outputs the data collection conditions to the vehicle.

[0048] As shown in FIG. 11, the problem analysis database 164 stores types of problems in association with parameters (vehicle data) necessary for analyzing the problem. For example, in order to analyze a problem related to acceleration performance, vehicle speed, acceleration, driving force, and prime mover output are required as vehicle data. Similarly, vehicle data necessary for analyzing other problems is stored in correspondence with each of the other problems.

[0049] The data collection condition update unit 163 searches the problem analysis database 164 to determine the similarity between the input time-series data description and the types of problems stored in the problem analysis database 164. It then retrieves the vehicle data types with the highest similarity from the problem analysis database 164 and outputs the retrieved vehicle data types to the vehicle as data collection conditions. The vehicle sets the received data collection conditions to the electronic control unit and collects data to resolve the inquiry entered by the user. The data collection condition update unit 163 may determine the similarity between the time-series data description and the types of problems stored in the problem analysis database 164 and select the vehicle data type with the highest similarity as the data collection condition, or it may select one or more vehicle data types with similarity exceeding a threshold as the data collection condition.

[0050] The data collection condition update unit 163 may input a prompt containing a time-series data description to a machine learning model (e.g., a large-scale language model) to identify the type of vehicle data that is effective for analyzing the cause of the problem described in the time-series data description. This machine learning model has learned the relationship between the type of problem in a vehicle and the type of vehicle data that is effective for analyzing the cause of that type of problem, and outputs the type of vehicle data that is effective for analyzing the cause of the problem described in the time-series data description.

[0051] In this way, the output unit 16 can improve the accuracy of its analysis by changing the data acquisition conditions and analyzing the vehicle data.

[0052] Figures 12, 13, 14, and 15 show examples where the output unit 16 shown in Figure 10 changes the data acquisition cycle as a data acquisition condition.

[0053] As shown in Figure 12, parameters A, B, and C are collected. If the data collection conditions shown in Figure 13 are stored in the problem analysis database 164, depending on the pre-set data collection conditions, it may be difficult to analyze user inquiries due to insufficient data volume or other reasons. For this reason, as shown in Figure 14, the output unit 16 generates a new collection condition setting file (see Figure 15). For example, as shown in Figure 13, before changing the data collection conditions, data for parameter B was collected every 100 milliseconds, and data for parameter D was not collected. After changing the data collection conditions, as shown in Figure 15, data for parameter B is collected frequently every 10 milliseconds, and data for parameter D is collected every 10 milliseconds.

[0054] In this way, the data collection condition update unit 163 can change not only the type of data to be collected, but also the timing of data collection, and collect data for analyzing vehicle data based on user requests.

[0055] Figure 16 shows an alternative configuration of the output unit 16 of Embodiment 1.

[0056] The output unit 16 shown in Figure 16 includes a driver presentation determination unit 165 and a presentation content database 166. It searches for content to present to the occupant from the time-series data explanation text input from the time-series data explanation text generation unit 15 and outputs it to the vehicle's presentation device or the user's terminal.

[0057] As shown in Figure 17, the presentation content database 166 stores the factors of a problem and the causes that lead to those factors in association. Here, a "factor" of a problem refers to an element involved in the occurrence of the problem. A "cause" refers to the specific reason or problem that causes that factor. For example, for a problem factor "the driver's operation or settings are not appropriate (inconsistency between the driver's intention and the operation or settings)," causes such as "settings that suppress acceleration, such as eco mode, are enabled," "there is a delay or error in the driver's pedal operation," and "the shift lever position is not appropriate (e.g., set to a low gear)" are stored in association. Similarly, other problem factors and the causes that lead to those factors are stored in association. Other factors include functional degradation due to temporary vehicle limitations, inconsistency between control specifications and the driver's intention, and not being used as intended. Candidate factors and causes of problems are not limited to these examples.

[0058] The driver presentation determination unit 165 determines the similarity between the input time-series data description and the factors stored in the presentation content database 166. If there are factors with a high degree of similarity, it determines that instructions or warnings to the driver are necessary. Next, it retrieves one or more causes associated with the factors with a high degree of similarity from the presentation content database 166 and outputs the retrieved causes as presentation content to the driver to the vehicle's presentation device. The vehicle outputs the received causes as audio, text, images, etc., from the presentation device (for example, the navigation system, the instrument panel display device, or the speaker).

[0059] The driver suggestion determination unit 165 may use a machine learning model to identify the cause corresponding to the input time-series data description. The machine learning model used by the driver suggestion determination unit 165 has learned the correspondence between factors and causes and outputs the cause that triggers factors similar to the time-series data description. Alternatively, a known similarity determination algorithm may be used.

[0060] Figure 18 shows a modified example of the vehicle information processing system 10 of Embodiment 1. In the vehicle information processing system 10 shown in Figure 18, supplementary information is fed back from the output unit 16 to the vehicle data storage unit 11.

[0061] The vehicle information processing system 10 shown in Figure 18 includes a vehicle data storage unit 11, a request reception unit 12, a vehicle data extraction unit 13, a vehicle data processing unit 14, a time-series data explanatory text generation unit 15, and an output unit 16. The functions of each unit, other than the output unit 16 feeding back supplementary information to the vehicle data storage unit 11, are the same as those of the vehicle information processing system 10 shown in Figure 1, so their explanations are omitted.

[0062] Figure 19 shows an example of the configuration of the vehicle data storage unit 11, which includes supplementary information that is fed back by the output unit 16 of Embodiment 1.

[0063] In the vehicle data storage unit 11 shown in Figure 19, items 1 to 5 are vehicle data input as vehicle information, and items 6 to 9 are analysis results and analysis result-related information fed back as supplementary information from the output unit 16. For example, the time-series data analysis result for the task in item 8 is the result of comparing the time changes of the data and analyzing the correlation between multiple data. Also, for example, the image analysis result for the task in item 9 is the result of extracting a specific object through image analysis.

[0064] Figure 20 shows the use of the vehicle data storage unit 11 in Embodiment 1.

[0065] In this modified example of the vehicle data storage unit 11, the user can access the vehicle data storage unit 11, which includes supplementary information, to obtain analyzed waveforms and image analysis results. Furthermore, the accuracy of the analysis can be improved by reviewing the supplementary information.

[0066] As described above, according to the embodiments of the present invention, the accuracy of the analysis can be improved by limiting the data analysis to the scope of the problem, and a flexible and highly accurate response can be provided to user inquiries.

[0067] <Second Embodiment> Figure 21 is a diagram showing the configuration of the vehicle information processing system 10 of Embodiment 2. In the vehicle information processing system 10 of Embodiment 2, the functions for receiving user input, transmitting vehicle data, and storing vehicle data 11 are mounted on the vehicle side, while the other functions are mounted on the server.

[0068] The vehicle information processing system 10 of Embodiment 2 includes a vehicle data storage unit 11, a request reception unit 12, a vehicle data extraction unit 13, a vehicle data processing unit 14, a time-series data explanation unit 15, and an output unit 16. The functions of each unit are the same as those of Embodiment 1 described above, so their descriptions will be omitted.

[0069] The vehicle information processing system 10 of Embodiment 2 extracts and analyzes information necessary for analysis from vehicle data based on the processing results of the request reception unit 12. The vehicle information processing system 10 of Embodiment 2 transfers only the vehicle data necessary for analysis, rather than all vehicle data, from the vehicle to the server, thereby reducing the communication load.

[0070] <Third Embodiment> Figure 22 shows the configuration of the vehicle information processing system 10 of Embodiment 3. In Embodiment 3, all functions of the vehicle information processing system 10 are mounted on the vehicle side.

[0071] The vehicle information processing system 10 of Embodiment 3 includes a vehicle data storage unit 11, a request reception unit 12, a vehicle data extraction unit 13, a vehicle data processing unit 14, a time-series data explanatory text generation unit 15, and an output unit 16. The functions of each unit are the same as those of Embodiment 1 described above, so their descriptions will be omitted.

[0072] Figure 23 is a diagram showing the hardware configuration of the electronic control unit that constitutes the vehicle information processing system 10 of Embodiment 3.

[0073] The electronic control unit constituting the vehicle information processing system 10 consists of an Electronic Control Unit (ECU) having a processor (CPU: Central Processing Unit) 31, a memory 32, a communication device 33, an input / output device 34, and a storage device 35. The processor 31, the memory 32, the communication device 33, the input / output device 34, and the storage device 35 are connected to each other via an internal signal line 39 such as a bus.

[0074] The processor 31 is an arithmetic unit that executes programs stored in the memory 32. By executing predetermined arithmetic processes, the processor 31 operates as a functional block that provides various functions of the vehicle information processing system 10 (for example, a request reception unit 12, a vehicle data extraction unit 13, a vehicle data processing unit 14, a time-series data explanation unit 15, and an output unit 16). The memory 32 has a volatile storage area that temporarily stores data used by the processor 31 when executing programs. The communication device 33 connects to other vehicle control devices via an in-vehicle network. The input / output device 34 is an interface for inputting and outputting data to the vehicle information processing system 10. The storage device 35 is accessible by the processor 31 and has a non-volatile storage area that includes a program area for storing programs executed by the processor 31 and a data area for storing data used by the processor 31 when executing programs.

[0075] The vehicle information processing system 10 of Embodiment 3 performs processing using an electronic control unit mounted on the vehicle without going through a network, thus reducing the communication load.

[0076] It should be noted that the present invention is not limited to the embodiments described above, but includes various modifications and equivalent configurations within the spirit of the attached claims. For example, the embodiments described above are described in detail for the purpose of clearly illustrating the present invention, and the present invention is not necessarily limited to having all the configurations described. Furthermore, some of the configurations of one embodiment may be replaced with those of another embodiment. Furthermore, configurations of other embodiments may be added to the configuration of one embodiment. Furthermore, some of the configurations of each embodiment may be added, deleted, or replaced with those of other embodiments.

[0077] Furthermore, each of the aforementioned configurations, functions, processing units, and processing means may be implemented in hardware, for example, by designing them as integrated circuits, or they may be implemented in software by having a processor interpret and execute programs that realize each function.

[0078] Information such as programs, tables, and files that implement each function can be stored in memory, hard disks, SSDs (Solid State Drives), or recording media such as IC cards, SD cards, and DVDs.

[0079] Furthermore, the control lines and information lines shown are those deemed necessary for explanation purposes and do not necessarily represent all control lines and information lines required for implementation. In reality, it can be assumed that almost all components are interconnected.

Claims

1. A vehicle information processing system for processing time-series data collected from a vehicle in a time-series manner, comprising a computer having an arithmetic unit that performs predetermined processing and a storage device connected to the arithmetic unit, wherein the arithmetic unit receives a query from a user via an input device to a vehicle system that controls the vehicle, determines the type and range of time-series data related to the query from among a plurality of types of time-series data collected by the vehicle system, extracts the time-series data to be processed, generates a time-series data description that explains the processing results of the extracted time-series data in natural language, and outputs the time-series data description.

2. A vehicle information processing system according to claim 1, wherein the storage device stores a problem description statement that describes a problem relating to a vehicle system, and the computing device determines whether to accept the query based on the similarity between the query and the stored problem description statement.

3. A vehicle information processing system according to claim 1, wherein the computing device generates a prompt that includes at least the query and outputs a problem description statement explaining the relationship between the query and the behavior of the vehicle, inputs the prompt to a machine learning model to obtain a problem description statement corresponding to the prompt, and determines whether to accept the query based on the obtained problem description statement.

4. A vehicle information processing system according to claim 1, wherein the computing device receives the inquiry from at least one of an input device mounted on a vehicle and a terminal device not mounted on a vehicle.

5. A vehicle information processing system according to claim 1, characterized in that it receives at least one of the following as an inquiry: natural language speech uttered by a user, an image displayed on an in-vehicle device or captured by a terminal device not mounted on the vehicle, and sound collected by an in-vehicle device or a terminal device not mounted on the vehicle.

6. A vehicle information processing system according to claim 1, wherein the computing device determines that the problem has not been articulated in the query, and provides the user with advice to clarify the problem in the query.

7. A vehicle information processing system according to claim 6, wherein the computing device determines that the problem has not been articulated if the query does not include at least one of the following: the name of an in-vehicle device, an object, or information relating to the vehicle behavior, and provides the user with advice to query for the name of an in-vehicle device, an object, information relating to the vehicle behavior, or the location where the problem occurred.

8. A vehicle information processing system according to claim 1, wherein the computing device classifies the query into one of safety, malfunction, comfort, and environmental performance, and processes queries classified as safety or malfunction with priority over queries classified as comfort or environmental performance.

9. A vehicle information processing system according to claim 1, wherein the computing device outputs an application if there is an application that is effective in solving the problem corresponding to the query.

10. A vehicle information processing system according to claim 1, wherein the storage device stores a task analysis database that stores tasks in association with the type of task and the vehicle data necessary for analyzing the task, and the computing device searches the task analysis database using the time-series data description and outputs parameters effective for analyzing the task described in the time-series data description as data collection conditions.

11. A vehicle information processing system according to claim 1, wherein the calculation device outputs the collection period of parameters necessary for problem analysis as a data collection condition.

12. A vehicle information processing system according to claim 1, wherein the storage device stores presentation content that associates the factors of a problem with the causes that bring about the factors, and the computing device retrieves the presentation content using the time-series data description and outputs the causes that bring about the factors of the problem as described in the time-series data description.

13. A vehicle information processing system according to claim 12, wherein the presented content includes, as factors of the problem, inconsistency between the driver's intent and operation or settings, functional degradation due to temporary constraints of the vehicle, inconsistency between control specifications and the driver's intent, and differences from the intended use, and the arithmetic unit determines whether any of the following apply: inconsistency between driver operation or settings, functional degradation due to temporary constraints of the vehicle, inconsistency between control specifications and the driver's intent, and differences from the intended use.

14. A vehicle information processing system according to claim 1, wherein the calculation device registers the analysis results of a problem described in the time-series data description in the vehicle data, and the analysis results include at least one of a description of the cause of the problem, image processing results, time-series data, and correlation analysis results.

15. A vehicle information processing system according to claim 1, characterized in that the arithmetic unit and the storage device are provided by a computer in an electronic control unit mounted on a vehicle.

16. A vehicle information processing method executed by a vehicle information processing system that processes time-series data collected from a vehicle in a time-series manner, comprising a computer having an arithmetic unit that performs predetermined processing and a storage device connected to the arithmetic unit, wherein the vehicle information processing method is characterized in that the arithmetic unit receives a query from a user via an input device to a vehicle system that controls a vehicle, the arithmetic unit determines the type and range of time-series data related to the query from among a plurality of types of time-series data collected by the vehicle system and extracts the time-series data to be processed, the arithmetic unit generates a time-series data description that explains the processing result of the extracted time-series data in natural language, and the arithmetic unit outputs the time-series data description.

Images