Vehicle dialogue system

Abstract

To provide a vehicle dialogue system capable of delivering a more natural dialogue experience to users.SOLUTION: A vehicle dialogue system comprises: a storage unit that stores, for each user, information for identifying the user and a character pre-selected by the user in association with the user; a voice input unit that inputs voice utterances made by the user; a voice recognition unit that converts the voice input by the voice input unit into first text data; a character information acquisition unit that identifies the user and acquires information about the character associated with that user; a character information generation unit that generates second text data representing the characteristics of the character acquired by the character information acquisition unit; and a control unit that inputs the first text data and the second text data as input information to a generative AI.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a dialogue system for a vehicle. [Background technology]

[0002] A proposed voice dialogue system for automobiles is one in which, when a user speaks to the system, the system selects an appropriate response from pre-registered dialogue scenarios and converts it into a voice response (Patent Document 1).

[0003] Such voice dialogue systems select appropriate responses from pre-recorded dialogue scenarios even when the user's utterances are somewhat vague. However, because such voice dialogue systems output responses selected from dialogue scenarios in the same character and tone of voice regardless of the user, the user may feel uncomfortable with the responses. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2018-54790 Summary of the Invention [Problem to be solved by the invention]

[0005] The present invention has been made in view of the above-mentioned circumstances, and its object is to provide a vehicle dialogue system that can provide a more natural dialogue experience to a user of a vehicle. [Means for solving the problem]

[0006] In order to achieve the above-mentioned object, the vehicle dialogue system according to the present invention has the following features. A vehicle dialogue system using a generation AI that is mounted on a vehicle and that outputs response information made up of text data when input information made up of text data is input, a storage unit that stores, for each user, information for identifying the user and a character selected in advance by the user in association with the user; a speech recognition unit that receives speech uttered by a user and converts it into first text data; a character information acquisition unit that identifies a user and acquires information about a character associated with the user; a character information generation unit that generates second text data representing characteristics of the character acquired by the character information acquisition unit; a control unit that inputs the first text data and the second text data as the input information to the generation AI; a speech synthesis unit that converts the response information from the generation AI into speech; a voice output unit that outputs the voice converted by the voice synthesis unit; a display control unit that displays the character acquired by the character information acquisition unit on a display unit. Vehicle dialogue system. [Effects of the Invention]

[0007] According to the present invention, it is possible to provide a vehicle dialogue system that can provide a more natural dialogue experience to a user of a vehicle.

[0008] The present invention has been briefly described above. The details of the present invention will become clearer by reading the following detailed description of the invention (hereinafter referred to as "embodiments") with reference to the accompanying drawings. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing a vehicle dialogue system according to a first embodiment. [Figure 2]FIG. 2 is a flowchart showing a processing procedure of the microcomputer shown in FIG. [Figure 3] FIG. 3 is a flowchart showing the driver authentication process. [Figure 4] FIG. 4 is an explanatory diagram showing an example of how a character is displayed. [Figure 5] FIG. 5 is a flowchart showing the procedure for changing a character. [Figure 6] FIG. 6 is a flowchart showing the procedure of the voice recognition process. DETAILED DESCRIPTION OF THE INVENTION

[0010] Specific embodiments of the present invention will be described below with reference to the accompanying drawings. 1 is a block diagram showing an embodiment of the vehicle dialogue system of the present invention. In the following description, the case where the driver is the user and speaks is taken as an example, but the present invention can also be applied to cases where any occupant in the vehicle, such as a passenger, speaks.

[0011] The vehicle dialogue system 1 of this embodiment is a system that is mounted on a vehicle and that dialogues with the driver using a generation AI (Artificial Intelligence) 10. The generation AI 10 is configured, for example, by ChatGPT, and when input information S1 made up of text data is input, it outputs response information S2 made up of text data.

[0012] The vehicle dialogue system 1 includes a microphone 2 as a voice input unit, a communication module 3, a microcomputer 4 (hereinafter abbreviated as "microcomputer 4") that functions as an agent, a speaker 5 as a voice output unit, a display 6, a memory unit 13 that stores user information corresponding to each driver, and a mobile terminal 8 carried by the user. The microphone 2 inputs the voice uttered by the driver to the microcomputer 4. The communication module 3 is for communicating with the generation AI 10 via an Internet communication network (not shown) and is composed of circuits, an antenna, etc. for connecting to the Internet communication network. Note that while FIG. 1 depicts a display 6 for setting by screen operation and a display 6 for outputting images, these displays 6 may be integrated or separate.

[0013] The microcomputer 4 has a memory such as a RAM (Random Access Memory) or a ROM (Read Only Memory), and a CPU (Central Processing Unit) that operates according to a program stored in the memory, and controls the entire vehicle dialogue system 1.

[0014] The microcomputer 4 has a voice recognition unit 41, a voice dialogue unit 42 as a control unit that controls input to the generation AI 10 and the character management server 12, controls connected devices, and controls output to these devices, a voice synthesis unit 44, and a drawing processing unit 45. The voice recognition unit 41 converts a voice signal input by the microphone 2 into text data (first text data) and inputs it to the voice dialogue unit 42. The voice dialogue unit 42 adds text data (second text data) representing the characteristics of the character selected by the driver to the text data converted by the voice recognition unit 41 and inputs it to the generation AI 10 as input information S1. The voice dialogue unit 42 may further add text data (third text data) representing the state of the driver or the vehicle driven by the driver at the time the driver spoke, or a command corresponding to the state.

[0015] Furthermore, vehicle information S3 is input to the voice dialogue unit 42 from a plurality of sensors mounted on the vehicle. Possible sensors include an illuminance sensor that measures the illuminance outside the vehicle, a temperature sensor that measures the outside air temperature, a GPS (Global Positioning System) that detects the vehicle position, a seating sensor that detects whether a person is seated in a seat, and a speed sensor that detects the vehicle speed. The vehicle information S3 input from the sensors may be not only measurement values ​​measured by the sensors, but also moving averages or weighted averages of the measurement values, or values ​​calculated using the measurement values ​​or the moving averages or weighted averages of the measurement values.

[0016] Furthermore, human information S4, which is a detection result from a driver monitor that detects the driver's state (facial expression, whether the driver is dozing, careless driving, or distracted driving) based on an image of the driver's face, is input to the voice dialogue unit 42. Furthermore, warning information, which is a detection result from an abnormality detection unit that detects a vehicle abnormality (engine abnormality, oil pressure abnormality, water temperature abnormality, charging abnormality, etc.) and turns on a warning lamp, may be input to the voice dialogue unit 42.

[0017] The voice dialogue unit 42 is also connected to vehicle equipment 11 mounted on the vehicle and can control the vehicle equipment 11. The vehicle equipment 11 may be, for example, an air conditioner mounted on the vehicle, a display such as a head-up display, or an audio device.

[0018] The voice dialogue unit 42 receives the response information S2 from the generation AI 10 and outputs the received response information S2 to the voice synthesis unit 44 and the drawing processing unit 45. The voice synthesis unit 44 converts the response information S2 into a voice signal and outputs it to the speaker 5. The speaker 5 outputs the voice converted by the voice synthesis unit 44. The display 6 is configured with a means having a screen capable of displaying information to the driver, such as a monitor or a head-up display (HUD) installed near the driver's seat. When the speaker 5 outputs voice, the microcomputer 4 displays a character linked to the voice on the screen.

[0019] The storage unit 13 stores, for each occupant, characteristics included in the voice uttered by the occupant and a character selected in advance by the occupant in association with each other. As the characteristics included in the voice uttered, for example, voice information indicating the intensity and pitch of the sound, such as a voiceprint, is used.

[0020] The storage unit 13 also stores character characteristic information that indicates the characteristics of each of a plurality of characters that can be displayed on the display 6. The character characteristic information stores, for example, the approximate age of the character, the personality that the passengers would like for the character, etc. The character characteristic information may be stored in the character management server 12.

[0021] The mobile terminal 8 has an app that can change the character. The mobile terminal 8 connects to the character management server 12 via a line such as the Internet, and when the occupant changes the character by purchasing a new character or selecting another registered character using the app, the mobile terminal 8 transmits information that identifies the occupant and information that represents the changed character to the character management server 12. The character management server 12 updates the character associated with the occupant to the changed character.

[0022] Next, the operation of the vehicular dialogue system 1 configured as described above will be described with reference to the flowchart shown in Figure 2. When the microcomputer 4 detects that the ignition is turned on (IG-ON), it starts up the system (Sp1). In the case of an electric vehicle, when it detects that the power supply to the vehicle is turned on and that the entire vehicle system has started up, it starts up the vehicular dialogue system 1.

[0023] Then, when the ignition is turned on (IG-ON) or the vehicle power is turned on, the microcomputer 4 performs driver authentication for the occupant seated in the driver's seat (Sp2). The driver authentication method uses the characteristics of the voice input by the occupant. Alternatively, the microcomputer 4 may perform authentication by wirelessly or wiredly connecting to the mobile terminal 8 that the occupant has brought into the vehicle, or may perform authentication by using an image of the occupant input from the driver monitor, fingerprint authentication of the occupant using a sensor installed on the steering wheel, or a seating sensor installed on the seat.

[0024] The details of driver authentication will be described later. When the occupant has not yet set a character, such as when purchasing a new car, the microcomputer 4 starts up with the character and the way of speaking of that character set by default.

[0025] Next, the microcomputer 4 transmits user information identifying the authenticated occupant to the character management server 12 (Sp3). The character management server 12 returns information indicating the character selected by the occupant to the microcomputer 4. If the character selected by the occupant stored in the storage unit 13 differs from the character stored in the character management server 12, the microcomputer 4 updates the information stored in the storage unit 13 with information acquired from the character management server 12. As a result, for example, if the occupant changes the character using the mobile terminal 8 outside the vehicle, the changed information is stored in the character management server 12. Therefore, when the occupant gets into the vehicle, the changed character is automatically displayed on the display 6 without the occupant having to perform a new character change operation.

[0026] Next, the microcomputer 4 determines whether or not there is a character change request (Sp4). The determination is made based on whether or not the occupant has performed a change operation on the display 6, the character change history stored in the occupant's mobile terminal 8, etc. If it is determined that there is a character change request, the microcomputer 4 executes a character change process (Sp5). The character change process will be described in detail later.

[0027] The microcomputer 4 determines whether or not the user has spoken (Sp6), and if so, executes voice recognition (Sp7). The voice recognition process will be described in detail later.

[0028] Next, the microcomputer 4 detects whether or not there has been a change in the occupant (Sp8). In this step, it detects whether or not there has been a change in the occupant while the vehicle engine and power are not turned off. If a change in the occupant is detected, driver authentication is performed (Sp9). This driver authentication is the same as the driver authentication in Sp2.

[0029] Next, the microcomputer 4 determines whether the entire vehicle system has been turned off by turning off the ignition (IG-OFF) or the power (Sp10), and if it has been turned off, ends operation. If it determines that it has not been turned off, it determines whether a specified time has elapsed (Sp11). If the specified time has elapsed, it transmits character setting information to the character management server 12 (Sp12). By periodically storing the character setting information, users such as passengers can receive services using the set character through the mobile terminal 8 even after getting out of the vehicle. For example, the user can use the mobile terminal 8 to interact with the set character or search for information. If the specified time has not elapsed, the process returns to Sp4.

[0030] Next, the procedure for the driver authentication described above will be described with reference to the flowchart shown in Fig. 3. When the microcomputer 4 detects an occupant based on signals from various sensors in the vehicle, such as the seat occupancy sensor (Sp21), it compares the occupant's voice characteristics, such as a voiceprint, images acquired by the driver monitor, fingerprints, etc., with the occupant's information registered in the storage unit 13 (Sp22). If the detected occupant information matches the registered occupant information (Yes in Sp23), it acquires the occupant's name and character setting information associated with the occupant (Sp25). If the detected occupant information does not match the registered occupant information, it sets the occupant's name to a temporary name such as "guest," and resets the character setting information to the initial setting information (Sp24).

[0031] In this way, the microcomputer 4 acquires character setting information corresponding to each occupant, and therefore, as shown in FIG. 4, can have a conversation with users A, B, and C using different characters.

[0032] Next, the processing procedure for the character change described above will be explained with reference to the flowchart shown in FIG. 5. When the occupant requests to change the character in Sp4, the microcomputer 4 displays a character list on the display 6 (Sp31) and also displays a message prompting the occupant to select a character, such as "Please select a character" (Sp32). The microcomputer 4 then determines whether or not a character has been selected by the occupant operating the display 6 (Sp33). The occupant's selection may be made by voice recognition. If a character has not been selected, the process returns to Sp32. The microcomputer 4 may notify the occupant by voice from the speaker 5 in Sp32, instead of displaying a message on the display 6.

[0033] When the microcomputer 4 determines in S33 that a character has been selected by the occupant, it determines whether information about the selected character is stored in the storage unit 13 (Sp34), and if not stored in the storage unit 13, it downloads the character information and customer character characteristic information from the character management server 12 (Sp35) and stores them in the storage unit 13. The microcomputer 4 then uses this character information to change the character (Sp36).

[0034] Next, the processing procedure of the above-mentioned voice recognition will be described with reference to the flowchart shown in Fig. 6. When the microcomputer 4 detects the occupant's speech in Sp6 of Fig. 2, it enters a voice recognition state (Sp41), and when it determines that the occupant's speech has ended (Sp42), it converts the occupant's speech into text data (Sp43).

[0035] The microcomputer 4 also determines whether the setting for displaying the occupant's speech content is ON (Sp44), and if the result is Yes, the occupant's speech content is displayed on the display 6 (Sp45), and if the result is No, the occupant's speech content is not displayed (Sp46). Whether this display setting is ON or OFF is selected in advance by the occupant via a setting screen displayed on the display 6, for example.

[0036] The microcomputer 4 then determines whether the occupant's speech includes a request to change the character (Sp46), and if so, proceeds to the character change process (Sp5) of FIG. 2. The presence or absence of a request to change the character is determined, for example, by whether the speech includes a word related to change, such as "change character." If a request to change the character is not included, the microcomputer 4 proceeds to a response generation state (Sp48), and adds text data representing the characteristics of the character to the text data generated in Sp43, and inputs the added text to the generation AI 10 (Sp49). At this time, the microcomputer 4 may further add text data indicating the vehicle status, etc.

[0037] The microcomputer 4 waits for a response from the generation AI 10 (Sp50). Upon receiving the response, the microcomputer 4 transitions to a voice output state (Sp51), reads out the response from the generation AI, and outputs it to the speaker 5 (Sp52). At this time, the display 6 displays a character selected by the occupant, and displays the character in accordance with the voice output from the speaker 5. The microcomputer 4 also determines whether the display setting for the spoken content representing the response from the generation AI 10 is ON (Sp53). If the answer is Yes, the read-out content is displayed on the display 6 (Sp54). If the answer is No, the read-out content is not displayed (Sp55). The ON / OFF setting for this display is selected in advance by the occupant, for example, via a setting screen displayed on the display 6.

[0038] Then, the microcomputer 4 determines whether the reading of the generated AI10 answer has been completed (Sp56), and if No, returns to Sp51, and if Yes, ends the process.

[0039] The present invention is not limited to the above-described embodiments, and can be appropriately modified, improved, etc. Furthermore, the material, shape, size, number, location, etc. of each component in the above-described embodiments are arbitrary and not limited as long as they can achieve the present invention.

[0040] For example, the microcomputer 4 may change the character's facial expression, tone of voice, etc. depending on the driving scene of the vehicle. For example, even when the same character is displayed, the microcomputer 4 may change the costume depending on the driving scene, such as summer and winter, daytime and nighttime, sunny and rainy weather, etc.

[0041] In addition, the voice dialogue unit 42 may infer the emotions of the occupant from the input voice and change the character's tone of voice depending on the emotion, or may accumulate information on what character or tone of voice to use when the driver is feeling certain emotions to make the driver feel positive, and change the character settings depending on the situation.

[0042] Here, the features of the vehicle dialogue system according to the embodiment of the present invention will be briefly summarized and listed below in [1] to [4].

[0043] [1] A vehicle dialogue system (1) that uses a generation AI to input input information consisting of text data and output response information consisting of text data, a storage unit (13) that stores, for each user, information for identifying the user and a character previously selected by the user in association with the user; a speech recognition unit (41) that receives speech uttered by a user and converts it into first text data; a character information acquisition unit (42) that identifies a user and acquires information about a character associated with the user; a character information generation unit (42) that generates second text data representing the characteristics of the character acquired by the character information acquisition unit; a control unit (42) that inputs the first text data and the second text data as the input information to the generation AI; a voice synthesis unit (44) that converts the response information from the generation AI into voice; a voice output unit (5) that outputs the voice converted by the voice synthesis unit; a display control unit (42) that displays the character acquired by the character information acquisition unit on a display unit. Vehicle dialogue system (1).

[0044] According to the configuration of [1] above, the vehicle dialogue system can provide dialogue services using a desired character for each user, thereby providing a vehicle dialogue system that can give vehicle users a more natural dialogue experience. Furthermore, even if the vehicle occupant changes, the character can be changed to a character that corresponds to the occupant without the occupant having to perform any character change operations. Furthermore, the vehicle dialogue system adds information representing the character's characteristics to information representing the user's utterance content and inputs this to the generation AI, thereby enabling the generation AI to obtain a response using expressions appropriate for the character selected. Therefore, unlike conventional dialogue systems that provide uniform responses, user satisfaction with the dialogue can be increased.

[0045] [2] The storage unit (13) stores features included in the voice uttered by the user as information for identifying the user, the character information acquisition unit (42) compares the input voice with the voice characteristics of each user stored in the storage unit (13) to identify the user who made the utterance; The vehicle dialogue system according to [1] above.

[0046] According to the configuration [2] above, the vehicle dialogue system can automatically identify the user without requiring any setting operation by the user, and can execute dialogue services using the identified user's preferred character.

[0047] [3] A character management server (12) that stores character information; When the characteristics of the character acquired by the character information acquisition unit (42) are not included in the storage unit, the character information generation unit (42) acquires the characteristics from the character management server (12) and generates the second text data. The vehicle dialogue system according to [1] or [2] above.

[0048] According to the configuration [3] above, the vehicle dialogue system can increase the number of selectable characters without increasing the capacity of the storage unit installed in the vehicle.

[0049] [4] A mobile terminal (8) connectable to the character management server (12), The character management server (12) stores the character selected by the user in association with the user, The mobile terminal (8) has a change program for allowing the user to change the character, When the user inputs information indicating a change in character from the mobile terminal (8), the mobile terminal (8) transmits information identifying the user and information indicating the changed character to the character management server (12); The character management server (12) updates the character associated with the user based on the received information. The vehicle dialogue system according to [3] above.

[0050] According to the configuration of [4] above, even when the user is outside the vehicle, the user can change the character using the mobile terminal. Therefore, the vehicle dialogue system can provide seamless services using the same character both inside and outside the vehicle. [Explanation of symbols]

[0051] 1. Vehicle dialogue system 2. Microphone 3. Communication Module 4. Microcomputer 5 speakers 6. Display 8. Mobile devices 10 Generation AI 11 Vehicle equipment 12 Character Management Server 13 Storage section 41 Voice Recognition Unit 42 Voice dialogue unit (control unit, character information acquisition unit, character information generation unit, display control unit) 44 Speech synthesis unit S1 Input information S2 Response Information S3 vehicle information S4 People information

Claims

1. A vehicle dialogue system using a generation AI that is mounted on a vehicle and that outputs response information made up of text data when input information made up of text data is input, a storage unit that stores, for each user, information for identifying the user and a character selected in advance by the user in association with the user; a speech recognition unit that receives speech uttered by a user and converts it into first text data; a character information acquisition unit that identifies a user and acquires information about a character associated with the user; a character information generation unit that generates second text data representing characteristics of the character acquired by the character information acquisition unit; a control unit that inputs the first text data and the second text data as the input information to the generation AI; a voice synthesis unit that converts the response information from the generation AI into voice; a voice output unit that outputs the voice converted by the voice synthesis unit; a display control unit that displays the character acquired by the character information acquisition unit on a display unit. Vehicle dialogue system.

2. the storage unit stores features included in a voice uttered by a user as information for identifying the user; the character information acquisition unit compares the input voice with voice characteristics of each user stored in the storage unit to identify the user who made the utterance; The vehicle dialogue system according to claim 1 .

3. A character management server that stores character information is provided, when the characteristics of the character acquired by the character information acquisition unit are not included in the storage unit, the character information generation unit acquires the characteristics from the character management server and generates the second text data. The vehicle dialogue system according to claim 1 .

4. a mobile terminal connectable to the character management server; the character management server stores the character selected by the user in association with the user; the mobile terminal has a change program for allowing a user to change a character; When the user inputs information indicating a change in character from the mobile terminal, the mobile terminal transmits information identifying the user and information indicating the changed character to the character management server; The character management server updates the character associated with the user based on the received information. The vehicle dialogue system according to claim 3 .

Images