Learning system

Abstract

This contributes to the realization of a dialogue system that makes it easier for users to obtain the answers they expect. [Solution] The learning system (1) comprises an acquisition means (11) for acquiring driver information relating to the actions of the vehicle driver, a generation means (12) for generating learning data based on the driver information, and a learning means (13) for fine-tuning a base model relating to a dialogue system that performs voice-based dialogue with the driver using the learning data.

Description

Technical Field

[0001] The present invention relates to the technical field of learning systems.

Background Art

[0002] As an example of this type of system, there is an in-vehicle dialogue system described in Patent Document 1.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Patent Document 1 describes a dialogue system using a language model (e.g., a large language model). In such a dialogue system, an answer expected by the user may not be obtained. Note that a large language model (LLM) is a language model constructed using a very large dataset and deep learning technology.

[0005] The present invention has been made in view of the above circumstances, and an object thereof is to provide a learning system that can contribute to the realization of a dialogue system in which it is easy to obtain an answer expected by the user.

Means for Solving the Problems

[0006] A learning system according to an aspect of the present invention includes an acquisition unit that acquires driver information related to the operation of a driver of a vehicle, a generation unit that generates learning data based on the driver information, and a learning unit that performs fine-tuning of a base model related to a dialogue system that conducts a dialogue with the driver using voice using the learning data.

Brief Description of the Drawings

[0007] [Figure 1] This is a block diagram showing the configuration of the learning system according to the embodiment. [Modes for carrying out the invention]

[0008] An embodiment of the learning system will be described with reference to Figure 1. The learning system 1 according to this embodiment is a system for fine-tuning a basic model of a dialogue system that performs voice-based dialogue with a vehicle driver. In Figure 1, the learning system 1 includes an acquisition unit 11, a generation unit 12, a learning unit 13, a specification unit 14, and an extraction unit 15, as well as databases DB1 and DB2.

[0009] The learning system 1 may be implemented by a single computer. The learning system 1 may be implemented by multiple computers. In this case, the acquisition unit 11, generation unit 12, learning unit 13, identification unit 14, and extraction unit 15, as well as parts of databases DB1 and DB2, may be implemented by one computer, while the remaining parts may be implemented by one or more other computers. The computer may be a cloud server. The computer may also be an in-vehicle device mounted on a vehicle.

[0010] The acquisition unit 11 acquires driver information relating to the operation of the vehicle's driver. The acquisition unit 11 acquires driver information for each driver. In other words, the acquisition unit 11 treats the driver information of one driver as different information from the driver information of other drivers. The acquisition unit 11 may associate the driver information of one driver with a vehicle used by that driver. In other words, the acquisition unit 11 may treat the driver information when one driver uses a vehicle as different information from the driver information when that driver uses another vehicle.

[0011] Driver information may include at least one of the following: the driver's voice, an image of the driver, and CAN (Controller Area Network) data related to the vehicle. The driver's voice may be collected by a microphone in the vehicle's onboard equipment. An image of the driver may be generated by an onboard camera capturing images of the interior of the vehicle. The CAN data may include at least one of the following: data related to the control of the drive system (e.g., engine, motor, transmission, etc.), data related to the control of the braking system, and data related to the control of the steering system. The CAN data may also include data related to lights, turn signals, wipers, power windows, door locks, air conditioners, etc. The CAN data may also include data related to diagnostic results obtained through diagnostics. Furthermore, the CAN data may be generated by the driver operating the vehicle. For this reason, the CAN data can be said to be information that indirectly indicates the driver's actions. In other words, the CAN data can be said to be information related to the driver's actions.

[0012] The generation unit 12 generates training data based on the driver information acquired by the acquisition unit 11. The generation unit 12 registers the generated training data in the database DB1 in association with the driver. In other words, training data is registered in the database DB1 for each driver. For example, the generation unit 12 may generate training data based on audio as driver information. In this case, the generation unit 12 may use speech recognition technology to generate text data from the audio. The generation unit 12 may generate training data based on the generated text data. For example, the generation unit 12 may generate training data based on an image as driver information. In this case, the generation unit 12 may use image caption generation technology to generate a descriptive text (i.e., text data) about the image. The generation unit 12 may generate training data based on the generated text data. For example, the generation unit 12 may generate training data based on CAN data as driver information. In this case, the generation unit 12 may use image caption generation technology to generate a descriptive text (i.e., text data) about a graph based on the CAN data. Furthermore, the generation unit 12 may use a large-scale multimodal model (LMM) to generate character data.

[0013] The identification unit 14 identifies the driver's attributes based on the driver information acquired by the acquisition unit 11. For example, when driver information is input to the identification unit 14, it may use a learning model (in other words, AI (Artificial Intelligence)) to identify the driver's attributes. The attributes identified by the identification unit 14 are not limited to one, but may be multiple. For example, the driver's attributes may be "male," "careful driver," and "prone to getting lost." Thus, the driver's attributes may include information indicating the driver's driving characteristics. The driver's attributes do not necessarily have to include information indicating the driver's gender.

[0014] The extraction unit 15 extracts person information from the database DB2 that has the same or similar attributes as the driver identified by the identification unit 14. Here, the database DB2 contains person information for multiple people (for example, other drivers of other vehicles), associated with the attributes of multiple people. The person information may be the same training data as the training data registered in the database DB1.

[0015] The learning unit 13 fine-tunes the base model using at least one of the training data registered in the database DB1 and the person information extracted by the extraction unit 15. For example, if there is a relatively large amount of training data registered in the database DB1, the learning unit 13 may fine-tune the base model using only that training data. For example, if there is a relatively small amount of training data registered in the database DB1, the learning unit 13 may fine-tune the base model using that training data and the person information extracted by the extraction unit 15.

[0016] The learning system 1 may repeatedly perform fine-tuning of the base model. For example, if the acquisition unit 11 acquires new driver information regarding the operation of a driver, the generation unit 12 may generate new learning data based on the newly acquired driver information. If a predetermined amount of new learning data (i.e., learning data not used for fine-tuning the base model) is registered in the database DB1, the learning unit 13 may perform fine-tuning of the base model using at least the new learning data. In this case, the amount of learning data registered in the database DB1 increases over time. Therefore, the learning unit 13 may change the amount of person information extracted by the extraction unit 15 used for fine-tuning the base model according to the amount of learning data registered in the database DB1. For example, as the amount of learning data registered in the database DB1 increases, the learning unit 13 may reduce the amount of person information extracted by the extraction unit 15 used for fine-tuning the base model (ultimately, the person information may not be used at all).

[0017] Furthermore, the underlying model for the dialogue system may be a large-scale language model (LLM) or a language model (i.e., a small model smaller than a large-scale language model). If the underlying model is a large-scale language model, the dialogue system may be implemented by a data center or cloud server. When a vehicle driver uses the dialogue system, the driver's speech may be transmitted to the data center or cloud server via the in-vehicle device and network. The response to the driver's speech, generated by the large-scale language model, is transmitted to the in-vehicle device via the network. As a result, dialogue may take place between the driver and the dialogue system. In this case, the learning unit 13 of the learning system 1 may be implemented by a data center or cloud server. If the underlying model is the language model described above, the dialogue system may be implemented by the vehicle's in-vehicle device. In this case, the learning unit 13 of the learning system 1 may be implemented by the in-vehicle device.

[0018] (Technical effects) According to Learning System 1, the foundational model of the dialogue system is fine-tuned using training data based on driver information regarding the operation of a single driver. As a result, the dialogue system is modified according to the characteristics of the single driver. The dialogue system modified according to the characteristics of the single driver is expected to be more likely to output the response that the single driver expects. Therefore, Learning System 1 can contribute to the realization of a dialogue system that makes it easier for the driver (in other words, the user) to obtain the response they expect.

[0019] The driver information may include the history of interaction with the dialogue system (i.e., the driver's voice), vehicle behavior (i.e., CAN data), and location (e.g., vehicle position, map information). For example, if learning (in this case, fine-tuning) reveals that the driver is slow to operate the vehicle in a particular location, the dialogue system may refrain from speaking to the driver in that location (in other words, the dialogue system may refrain from emitting voice in that location). The learning unit 13 may also perform learning using the learning data registered in the database DB1 to highlight the areas where the driver is weak.

[0020] Various aspects of the invention derived from the embodiments described above are described below.

[0021] A learning system according to one aspect of the invention comprises an acquisition means (11) for acquiring driver information relating to the actions of a vehicle driver, a generation means (12) for generating learning data based on the driver information, and a learning means (13) for fine-tuning a base model relating to a dialogue system that engages in voice-based dialogue with the driver using the learning data. In the above embodiment, the "acquisition unit 11" corresponds to an example of the "acquisition means," the "generation unit 12" corresponds to an example of the "generation means," and the "learning unit 13" corresponds to an example of the "learning means."

[0022] In the learning system of the above aspect, the acquisition means acquires driver information regarding the operation of the driver of the vehicle. The driver information may include at least one of information directly indicating the driver's operation and information indirectly indicating the driver's operation. The generation means generates learning data based on the driver information. The learning means performs fine-tuning of the base model regarding the dialogue system that conducts a dialogue using the driver and voice using the learning data. The base model may mean a learning model pre-trained using a large amount of data. Each driver has an individuality. If the learning model regarding the dialogue system is uniform regardless of the driver, there is a possibility that the answer expected by the driver cannot be obtained from the dialogue system. In the learning system, fine-tuning of the base model regarding the dialogue system is performed using the learning data based on the driver information. Therefore, according to the learning system, it can contribute to the realization of a dialogue system in which it is easy to obtain the answer expected by the driver.

[0023] In the learning system according to the above aspect, the driver information may include at least one of the driver's voice, an image in which the driver is reflected, and CAN data related to the vehicle.

[0024] In the learning system according to the above aspect, the generation means may generate character data based on the driver information.

[0025] The learning system according to the above embodiment may include an identification means for identifying the attributes of a driver based on the driver information, and an extraction means for extracting person information for one or more of the multiple persons from a database in which person information for multiple persons is registered in association with the attributes of the multiple persons, based on the attributes of the driver, and the learning means may perform fine tuning of the base model using the learning data and the extracted person information. With this configuration, even if the learning data based on the driver information is relatively small, the base model can be fine-tuned appropriately. In the above embodiment, the "identification unit 14" corresponds to an example of the "identification means," and the "extraction unit 15" corresponds to an example of the "extraction means."

[0026] In the learning system according to the above embodiment, the learning means may be mounted on the vehicle.

[0027] The present invention is not limited to the embodiments described above, and can be modified as appropriate without contradicting the gist or idea of ​​the invention as can be read from the claims and specification as a whole. Learning systems with such modifications are also included within the technical scope of the present invention. [Explanation of symbols]

[0028] 1...Learning system, 11...Acquisition unit, 12...Generation unit, 13...Learning unit, 14...Specification unit, 15...Extraction unit

Claims

1. A means for acquiring driver information related to the actions of the vehicle's driver, A generation means for generating training data based on the aforementioned driver information, A learning means for fine-tuning a basic model of a dialogue system that performs dialogue with the driver using voice, using the aforementioned training data, A learning system equipped with the following features.

2. The driver information includes at least one of the driver's voice, an image in which the driver is captured, and CAN data related to the vehicle. The learning system according to claim 1.

3. The generation means generates character data based on the driver information. The learning system according to claim 1.

4. A means for identifying the attributes of the driver based on the driver information, An extraction means for extracting person information for one or more of the multiple persons from a database in which person information for multiple persons is registered in association with the attributes of the multiple persons, based on the attributes of the driver, Equipped with, The learning means uses the learning data and the extracted person information to fine-tune the base model. The learning system according to claim 1.

5. The learning means is mounted on the vehicle. The learning system according to claim 1.

Images