Display control device and display control method

Abstract

A display control device according to the present embodiment is provided with a controller 100 comprising a speech recognition unit 111 that detects input of speech of a passenger, and recognizes the speech by speech processing for the detected speech, and a display control unit 130 that causes a display 10 to display a main screen including an operating state image indicating the operating state of a dialog system and a category screen including a category image in which commands are grouped. The controller 100 causes the display 10 to display the main screen before the speech recognition unit 110 detects the speech, causes the display 10 to display the category screen when the speech processing is not normally completed, and causes a display screen to make a transition from the category screen to another screen when a category is selected in a state where the category screen is displayed.

Description

Technical Field

[0001] The present invention relates to a display control device and a display control method.

Background Art

[0002] An agent device that provides a service including a response in accordance with the speech of a vehicle occupant is known (for example, Patent Document 1). This agent device recognizes a command included in the speech of the occupant, outputs the recognized command to a plurality of agent function units, and selects, based on the results made by each of the plurality of agent function units, an agent function unit that responds to the speech of the occupant from among the plurality of agent function units, and executes a response to the speech of the occupant on the selected agent function unit.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the above-described conventional agent device, the agent function unit is selected based on the command included in the speech content, and there is a problem that the operability is poor for a user who does not understand what kind of command should be included in the speech content.

[0005] The problem to be solved by the present invention is to provide a display control device and a display control method with improved operability.

Means for Solving the Problems

[0006] Before detecting voice, the main screen including the operation state image indicating the operation state of the dialogue system is displayed on the display. When the voice processing is not completed normally, the category screen including the category image grouping commands is displayed on the display. When a category is selected while the category screen is being displayed, the above problem is solved by transitioning the display screen from the category screen to another screen.

Effect of the Invention

[0007] According to the present invention, since the display screen of the display transitions according to the result of the voice recognition process and the display screen of the display also transitions when the voice recognition process does not proceed normally, the operability can be improved.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3A

Figure 3B

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Best Mode for Carrying Out the Invention

[0009] Hereinafter, an embodiment of the dialogue service system according to the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a dialogue service system according to an embodiment of the present invention. The dialogue service system is a system mounted on a vehicle and includes a display 10, a communicator 20, a steering switch 30, an ECU 40, and a controller 100. The dialogue service system according to the present embodiment is a system that uses a voice processing function to interact with passengers and provide services to passengers, and includes a system for controlling the display screen of the display.

[0010] The display 10 is a touch panel type display and is mounted on the vehicle. The touch panel type display 10 is provided with a pressure sensor and detects a touch operation by a passenger by measuring the pressing force applied to the display 10 during a contact operation from a resistance value, a voltage, or the like. The mechanism of the pressure sensor is not particularly limited, and a method known at the time of filing can be appropriately used. Further, the touch panel is not limited to the pressure sensitive type, and other types such as an electrostatic type may be used. FIG. 2 is a front view near a dashboard provided with a touch panel. As shown in FIG. 2, the display 10 is a display (center display) on the front part of the dashboard and is disposed between the driver's seat and the passenger seat. The display 10 is provided at a position where a passenger sitting in the driver's seat and a passenger sitting in the passenger seat can each touch it. Further, the display 10 does not necessarily need to be disposed at a position where both the driver and the passenger in the passenger seat can touch it, and may be disposed at a position where only the driver can touch it, for example. Further, the display 10 is not limited to the dashboard, and may be disposed at a position where a passenger in the rear seat can touch it, for example.

[0011] When the occupant selects an icon, button, etc. from the menu screen by touching the display 10 with a finger or an operating device substituting for the finger, the screen display of the display 10 changes. In the following description, the touch operation mainly using a finger will be described, but in this embodiment, a touch operation using an operating device instead of a finger may also be used. The touch operation is a pointing operation executed by a gesture using a finger, such as touching a finger to an icon or button displayed on the display 10, or bringing a finger close to an icon or button. The touch operation includes a tap (touching the screen once), a double-tap (touching the screen twice), a long-tap (long-pressing the screen), a swipe (sliding a finger that has touched the screen as it is on the screen (tracing the finger)), a flick (moving a finger that has touched the screen quickly and elastically on the screen), a pinch-in / pinch-out (touching the screen with two fingers and bringing the two fingers closer / farther apart), etc. The touch operation is not limited to the first gesture of touching a finger at one or multiple points on the screen, and may also include the second gesture of moving the contact point of the finger while touching the screen. Also, the touch operation does not necessarily need to directly touch the display screen, and so-called hovering, which brings a finger or the like close to the display screen, may also be used. Note that the operation method of the touch operation is not limited to these and other methods may also be used.

[0012] The communication device 20 is a communication device capable of communicating with an external server 2. The communication device 20 transmits and receives signals between the controller 100 and the server 2. The handle switch (steering switch) 30 is a switch for starting (activating) the dialogue system, and is provided on the steering wheel as shown in FIG. 2. When the occupant (driver) wants to receive services from the dialogue system, the occupant operates the handle switch 30 to activate the dialogue system. As will be described later, the dialogue system includes a first dialogue system 112 that mainly performs voice processing using a voice recognition engine provided in the vehicle 1 and provides services, and a second dialogue system 32 that mainly performs voice processing using a voice recognition engine included in the server 2 and provides services from the server 2 to the occupant. The handle switch 30 is assigned to activate either one of the dialogue systems, and operating the handle switch 30 activates either the first dialogue system or the second dialogue system.

[0013] The ECU 40 is a control unit that controls control targets such as the air conditioning equipment 41 and the sunroof 42, and controls the control targets and the controller 100 via the CAN communication network. The ECU 40 is not limited to one, but is a number of control units, and the ECU 40 is connected for each control target.

[0014] The controller 100 is composed of a computer equipped with hardware and software, and has a memory for storing programs and a CPU for executing the programs stored in this memory. Further, the controller 100 has, as functional blocks, an interactive system control unit 110, a startup unit 120, and a display control unit 130, and programs for realizing the functions of the interactive system control unit 110, the startup unit 120, and the display control unit 130 are stored in the memory. Then, by the computer included in the controller 100 executing the program, each function of the functional block is realized. Further, the controller 100 is connected to the display 10, the communication device 20, the handle switch 30, and the ECU 40 via a CAN communication network or the like. The controller 100 has control blocks for controlling various systems in the vehicle interior, such as a function for controlling an audio system, not limited to the interactive system control unit 110, the startup unit 120, and the display control unit 130, and controls other in-vehicle devices. Note that the device having the controller 100 corresponds to the "display control device" of the present invention.

[0015] The interactive system control unit 110 has a function of recognizing the voice of an occupant and controls the interactive system. The interactive system control unit 110 has a voice recognition unit 111 and a first interactive system 112. The voice recognition unit 111 detects the voice input of the occupant and recognizes the voice by performing voice processing on the detected voice using the voice recognition engine included in the first interactive system 112. When the first interactive system 112 is activated, when the occupant speaks to the microphone in the vehicle, the voice recognition unit 11 detects the input voice. Voice detection is executed from the start of voice input until a predetermined time has elapsed after the end of voice input. For example, when the occupant generates a message such as "Set the temperature of the air conditioner (air conditioner) to 25°C", the voice recognition unit 11 detects the voice from the beginning to the end of the message, and after a silent period of a predetermined time from the end of the message, ends voice detection. The voice recognition unit 11 recognizes the voice by performing voice processing on the detected voice using the voice recognition engine.

[0016] When the voice recognition unit 111 recognizes a wake-up word for activating the dialogue system, it outputs a control signal indicating that the wake-up word has been recognized to the second interface 122 described later. The wake-up word is, for example, a calling message such as "Hello ○○ (name of the car manufacturer, compatibility of the dialogue system, etc.)" and is predetermined. Note that the wake-up word may be changed according to the settings of the occupant.

[0017] The first dialogue system 112 is an in-vehicle voice processing system mainly aimed at cooperation with in-vehicle devices. Using a voice recognition engine, it conducts a dialogue between the occupant and the controller 100 and provides services to the occupant according to the dialogue content. The first dialogue system 112 is connected to the ECU 40 to cooperate with in-vehicle devices such as the air conditioner 41, sunroof 42, navigation system, audio system, and hands-free system. The first dialogue system 112 is equipped with a communication function for communicating with various ECUs. For example, when the occupant speaks to the dialogue system to change the temperature setting of the air conditioner 41, the voice recognition unit 111 performs voice processing. Based on the voice data recognized by the voice recognition unit 111, after identifying the air conditioner 41 to be controlled, it generates a control command and outputs the control command to the ECU 40 through the in-vehicle communication path (CAN communication network). When sending a control signal from the first dialogue system 112 to in-vehicle devices such as the air conditioner 41 via the ECU 40, the in-vehicle communication path through which the control signal passes does not include the communication path connecting the controller 100 and the communication device 20. The ECU 40 controls the air conditioner 41 according to the control command and changes the set temperature of the air conditioner 41. Thereby, the first dialogue system 112 provides services to the occupant through the dialogue. The services provided by the first dialogue system 112 are not limited to the setting of the air conditioner 41, and may also include the adjustment of the air direction and air volume of the air conditioner 41. In other examples of services, there are operations of in-vehicle devices such as route guidance by the navigation system, setting of the audio system, telephone operation and telephone number search by the hands-free system, short message service (SMS), opening and closing operations of the sunroof 42, window, back door, sliding door, etc.

[0018] On the other hand, the second dialogue system 32 included in the server 2 is a system that performs voice processing using an external voice recognition engine. When the vehicle 1 connects to the server 2 to access external information and provide services to the passengers. The services provided by the second dialogue system 32 are a variety of services such as shopping using the Internet and remote control of home appliances at home. Different from the first dialogue system 112, the second dialogue system 32 is not intended for cooperation with in-vehicle devices. Also, regarding the difference between the first dialogue system 112 and the second dialogue system 32, the first dialogue system and the second dialogue system are separate dialogue systems. The first dialogue system sends control commands to in-vehicle devices through the in-vehicle communication path and has a voice recognition engine (corresponding to the "first voice recognition engine" of the present invention). The second dialogue system has a voice recognition engine different from the voice recognition engine included in the first dialogue system (corresponding to the "second voice recognition engine" of the present invention). Also, from another perspective, the difference between the first dialogue system 112 and the second dialogue system 32 is that the first dialogue system 112 inputs or outputs control commands to in-vehicle devices through the in-vehicle communication path, while the second dialogue system 32 may be a system that does not input and output control commands to in-vehicle devices other than the communication device 20 through the in-vehicle communication path. From yet another perspective, the difference between the first dialogue system 112 and the second dialogue system 32 is that the first dialogue system outputs control commands to the in-vehicle devices through an in-vehicle communication path other than the communication path connecting the communication device 20 and the first dialogue system 112, and does not output control commands through an in-vehicle communication path other than the communication path connecting the communication device 20 and the first dialogue system 112.

[0019] When the voice recognition unit 111 conducts a dialogue with the passenger using the second dialogue system 32, it detects the voice input of the passenger and transmits the detected voice data to the server 2 via the communication device 20. The second dialogue system 32 of the server 2 performs voice processing on the input voice data and transmits the processing result to the vehicle 1. The voice recognition unit 111 outputs the processing result by the second dialogue system 32 through the voice output from the in-vehicle speaker or the display screen of the display 10.

[0020] The first dialogue system 112 may be connected not only to the in-vehicle ECU 40 but also to the server 2 outside the vehicle. For example, the first dialogue system 112 may be connected to the server 2 outside the vehicle and perform voice recognition processing using the voice recognition engine outside the vehicle. For example, since there is a lot of information such as the name of a store, it may be better to use the voice recognition engine outside the vehicle rather than the in-vehicle voice recognition engine. In such a case, the first dialogue system 112 performs voice recognition processing using the voice recognition engine outside the vehicle.

[0021] The activation unit 120 is a functional block for activating the first dialogue system 112 and the second dialogue system 32, and has a first interface 121 and a second interface 122. The first interface 121 transmits an activation command to the dialogue system by operating the steering wheel switch 30. That is, the first interface 121 serves as a connection unit that connects between the steering wheel switch 30 and the first dialogue system 112 and between the steering wheel switch 30 and the second dialogue system 32, respectively. The first interface 121 transmits an activation command to either one of the first dialogue system 112 and the second dialogue system 32. The first interface 121 does not transmit control commands to both the first dialogue system 112 and the second dialogue system 32 at the same time. The destination of the control command by the first interface 121 is determined by the setting of the occupant and / or the initial setting at the time of factory shipment. When the steering wheel switch 30 is assigned to activate the first dialogue system 112, the first interface 121 transmits an activation command to the first dialogue system 112 when it receives an operation command by operating the steering wheel switch 30. When the steering wheel switch 30 is assigned to activate the second dialogue system 32, the second interface 122 transmits an activation command to the second dialogue system 32. Note that the activation command for the second dialogue system 32 is a command for making the vehicle 1 and the server 2 communicable and making the dialogue system of the server 2 available for the vehicle 1.

[0022] The second interface 122 transmits a startup command to the dialogue system using a wake-up word. The second interface 122 is a connection part between the voice recognition unit 111 and the dialogue system. The second interface 122 transmits a startup command to either the first dialogue system 112 or the second dialogue system 32. The first interface 121 does not simultaneously transmit control commands to both the first dialogue system 112 and the second dialogue system 32. In the example of FIG. 1, the second interface 122 transmits a startup command to the first dialogue system 112. The destination of the control command transmitted by the second interface 122 is determined by the initial settings at the time of factory shipment. In the example of FIG. 1, since the wake-up word is assigned to the startup of the first dialogue system 112, when the second interface 122 recognizes the wake-up word by the voice recognition unit 111, the second interface 122 transmits a startup command to the first dialogue system 112. That is, the voice recognition engine included in the first dialogue system 112 waits in a state where it can recognize at least the wake-up word, and when the second interface 122 recognizes the wake-up word by the voice recognition unit 111, the second interface 122 transmits a startup command to the first dialogue system 112, and the first dialogue system 112 starts up the voice recognition engine so that various words other than the wake-up word can be recognized. Note that when the wake-up word is assigned to the startup of the second dialogue system 32, the second interface 122 transmits a startup command to the second dialogue system 32.

[0023] In addition, the activation unit 120 sets the destination of the activation command to either the first dialogue system 112 or the second dialogue system 32 based on the operation of the occupant. The operation of the occupant for setting the destination of the activation command includes touch operations on the display 10, switch operations other than the steering wheel switch 30, and the like. For example, in the case of a touch operation on the display 10, the controller 100 displays a setting screen for the dialogue system on the display 10, and the occupant touches a selection menu for selecting either the first dialogue system 112 or the second dialogue system 32 from the displayed setting screen, and selects the dialogue system to be used on the selection menu. The activation unit 120 sets the destination of the activation signal of the first interface 121 in order to transmit an activation signal to the dialogue system selected by the touch operation on the display 10. For example, when the occupant changes the target system to be activated during the operation of the steering wheel switch 30 from the first dialogue system 112 to the second dialogue system 32 by a touch operation on the display 10, the activation unit 120 sets the destination of the activation signal by the first interface 121 to the second dialogue system 32. Thereby, the occupant can select the dialogue system to be activated during the operation of the steering wheel switch 30 from the first dialogue system 112 and the second dialogue system 32 according to preference.

[0024] Also, when the settings of the dialogue system can be changed through dialogue with the passengers using the dialogue system, if applicable, the activation unit 120 may set the destination of the activation command by the first interface 121 to either the first dialogue system 112 or the second dialogue system 32 based on the voice recognized by the dialogue system control unit 110. For example, assume that the passenger is in a state of having a dialogue with the controller 100 through the first dialogue system 112, and the passenger expresses the intention to change the assignment of the steering switch 30 from the first dialogue system 112 to the second dialogue system 32. The dialogue system control unit 110 recognizes the voice of the passenger and sends a control command to the activation unit 120 for assigning the steering switch 30 to activate the second dialogue system 32. Based on the control command, the activation unit 120 sets the destination of the activation signal by the first interface 121 to the second dialogue system 32. As a result, the passenger can select, through the dialogue using the dialogue system, the target system to be activated when operating the steering switch 30 from either the first dialogue system 112 or the second dialogue system 32.

[0025] In this embodiment, for the dialogue system that can be activated by operating the steering switch 30, the passenger can select either the first dialogue system 112 or the second dialogue system 32. However, for the dialogue system that can be activated by a wake-up word, it is fixed to the first dialogue system 112. Since the first dialogue system 112 is mainly for the purpose of cooperation with in-vehicle devices, it has a higher usage frequency than the second dialogue system 32. Therefore, by fixing the activation by the wake-up word to the first dialogue system 112, the convenience for the passenger can be enhanced. Also, since the assignment of the steering switch 30 can be selected by the passenger, the degree of freedom in selecting the dialogue system can also be ensured.

[0026] Figure 3A is a table for explaining whether the activation of the first / second dialogue system can be switched. As shown in Figure 3A, in the dialogue service system according to this embodiment, the "wake-up word" can activate the first dialogue system 112 but cannot activate the second dialogue system 32. The "handle switch" can activate either the first dialogue system 112 or the second dialogue system 32 according to the setting of the occupant. That is, the activation of the dialogue system by the handle switch 30 can be exclusively selected. When the handle switch 30 is assigned to the first dialogue system 112, the second dialogue system 32 cannot be activated by the operation of the handle switch 30 and the wake-up word, but the second dialogue system 32 can be activated, for example, by selection on the menu screen displayed on the display 10 or by the operation of a switch other than the handle switch 30.

[0027] Figure 3B is a table for explaining the correlation with the activation of the first / second dialogue system in the dialogue service system according to a modification of this embodiment. In the dialogue service system according to the modification, the dialogue system that can be activated by the wake-up word can be exclusively selected according to the user setting. The activation unit 120 sets the destination of the activation command when the wake-up word is recognized to either the first dialogue system 112 or the second dialogue system 32. The occupant can select the dialogue system to be activated from the first dialogue system 112 and the second dialogue system 32 by a touch operation on the display 10, an operation of a switch other than the handle switch 30, or a dialogue with the controller 100 using the dialogue system. Thereby, the situation where a plurality of dialogue systems start up when the wake-up word is uttered is avoided, and the dialogue system to be activated when the handle switch 30 is operated can be selected from the first dialogue system 112 and the second dialogue system 32 according to the user's preference.

[0028] In addition, the dialogue system may have its own standard regarding the startup of the system. The own standard may be, for example, "when a specific dialogue system is in a state where it can be started, other dialogue systems must be turned off (invalidated)". When such a standard is used for the first dialogue system 112, as in this embodiment, the assignment of the handle switch 30 may be made changeable by user settings, or the dialogue system may be selectable exclusively. Also, when making the startup of the dialogue system by the wake-up word changeable by user settings, as in the modified example, the assignment of the startup by the wake-up word may be made changeable by user settings, or the dialogue system may be selectable exclusively. That is, according to the standard of the first dialogue system, it is necessary to start the first dialogue system 112 and the second dialogue system 32 exclusively. Therefore, as in this embodiment, by making the assignment of the handle switch 30 and the assignment of the startup by the wake-up word changeable by user settings, it is possible to ensure the degree of freedom in selecting the dialogue system while complying with the standard of the dialogue system.

[0029] In this embodiment, when the transmission destination of the startup command by the first interface 121 and / or the second interface 122 is changed by the operation of the occupant or the dialogue between the occupant and the dialogue system, the dialogue system control unit 110 may output the changed content to the occupant by voice.

[0030] The display control unit 130 controls the display screen of the display 10. Also, when the display control unit 130 detects a touch operation by the display, it accepts an operation command according to the position of the touch operation and the type of the touch operation on the display screen. The display control unit 130 causes the display 10 to display an operation screen for operating the dialogue system, a setting screen of the dialogue system, a screen for displaying information provided from the dialogue system, etc. Also, the display control unit 130 outputs the operation command accepted by the touch operation of the display 10 to the dialogue system control unit 110 and the startup unit 120.

[0031] The control of the dialogue system by the dialogue system control unit 110 and the control of the display screen by the display control unit 130 are associated by an application (software). The operating state of the dialogue system by the dialogue system control unit 110 is displayed on the display 10. Also, in response to the dialogue by the dialogue system, the display control unit 130 controls the display screen of the display 10. When a passenger advances a dialogue with the controller 100 through the dialogue system, the dialogue system control unit 110 transmits a control command corresponding to the content of the dialogue to the display control unit 130, and the display control unit 130 causes the display screen of the display 10 to transition according to the control command.

[0032] Next, referring to FIG. 4, a control method of the controller 100 and the display screen of the display 10 will be described. FIG. 4 is a display screen of the display 10, and shows a top screen of the entire system including the dialogue service system.

[0033] When the crew turns on the main switch of the display 10, the display control unit 130 displays the top screen on the display 10. As shown in FIG. 4, the top screen is a menu screen for selecting a navigation system, music, the first dialogue system 112, and the second dialogue system 32. Each menu screen includes an icon representing the content. For example, the menu image of the first dialogue system 112 includes the first icon 101, and the menu image of the second dialogue system 32 includes the second icon 102. The first icon 101 is an icon that enables a touch operation for activating the first dialogue system 112, and the second icon 102 is an icon that enables a touch operation for activating the second dialogue system 32. When the crew touches the first icon 101, the activation unit 120 transmits an activation signal to the first dialogue system 112. When the crew touches the second icon 102, the activation unit 120 transmits an activation signal to the second dialogue system 32. Thereby, the first dialogue system 112 and the second dialogue system 32 can be exclusively selected by a touch operation on the display 10.

[0034] Next, a display screen that is displayed according to the operating state of the first dialogue system 112 will be described. FIGS. 5 to 8 show display screens displayed on the display 10 for each operating state of the first dialogue system 112. On the top screen shown in FIG. 4, when the passenger touches the first icon 101 or the area around the first icon 101, the display control unit 130 displays the screen shown in FIG. 5. The operating states of the first dialogue system 112 are four states: "initial state", "voice reception enabled state", "voice detection state", and "voice processing state". The initial state is a state until the voice recognition engine is activated based on the activation signal and becomes a state where it can detect voice. The voice recognition unit 111 performs an initialization process before voice processing in the initial state. When the initialization process is completed, the operating state of the first dialogue system 112 changes from the initial state to the voice reception enabled state. The voice reception enabled state is a state where voice can be received and indicates a state where the passenger is not speaking. The voice detection state is a state where voice can be received and the passenger is speaking and the voice is being detected. That is, when voice is not input in a state where voice can be detected, the operating state of the first dialogue system 112 becomes the voice input reception state, and when voice is input, the operating state of the first dialogue system 112 becomes the voice detection state. Then, when the voice detection state ends and a predetermined time elapses, the operating state of the first dialogue system 112 becomes the voice processing state. The voice processing state is a state where voice processing is being performed by the voice recognition engine.

[0035] The display screens in FIGS. 5 to 8 represent the main image in the service through the dialogue system and include an operating state image indicating the operating state of the dialogue system. The operating state image is an image representing to the passenger what state (status) the dialogue system is in and corresponds to text, animation, patterns, etc. In the present embodiment, the main message 50 and / or the animation 70 correspond to the operating state image.

[0036] FIG. 5 shows a display screen displayed on the display 10 when the operating state of the first dialogue system 112 is the "initial state". As shown in FIG. 5, the display screen in the "initial state" includes a main message 50, a sample message 60, and an animation 70. The main message 50 is a message from the first dialogue system 112 to the occupant, and is a notation prompting the occupant for voice input. By confirming the main message 50, the occupant can confirm that the dialogue system has been activated. That is, the main message 50 corresponds to an operation state image indicating that the dialogue system has been activated. The sample message 60 represents an example of the voice that the occupant should utter in order to use the first dialogue system 112, and includes a command 61 and a setting value 62. The command 61 is a word representing the operation or state of the service that the occupant desires to receive through the dialogue system, and represents an operation command from the occupant to the dialogue system. The setting value 62 is a phrase representing the target of the service, and is represented by a noun such as a numerical value or a name. For example, in the message "Play station AAA Radio" on the second line of the sample message 60, "Play" indicates the command and "station AAA Radio" indicates the setting value. The command "Play" indicates operating the audio system, and "station AAA Radio" indicates the target to be played on the audio system (the name of the radio station). Note that the sample message 60 also serves as a function button. For example, when the occupant touches the message "Enter Number" on the third line, the screen transitions to a screen for inputting a telephone number.

[0037] Animation 70 is an image that represents the status in a predetermined function with periodic movement. In the example of FIG. 5, the predetermined function corresponds to the voice detection processing function in the dialogue system, and the status corresponds to the operating state of the dialogue system. The periodic movement is represented by the movement of a waveform. That is, there are four animations 71 to 74 in the animation 70 to represent the four operating states of the first dialogue system 112. That is, the progress of the voice processing by the first dialogue system 112 is represented by the four animations 71 to 74. The animation 70 is distinguished for each operating state so that a passenger viewing the animation can understand it by the shape of the wave, the color of the wave, the size of the wave, etc. Then, the controller 100 operates the animation 70 in response to the voice input of the passenger, and thereby displays the progress of the voice processing on the display screen of the display 10. The animation 71 in FIG. 5 represents the "initial state". The animation 71 is a gray waveform and has a wave with a relatively small amplitude. The animation 70 is displayed at a position between the main message 50 and the sample message 60. Note that the operating state image indicating the operating state of the dialogue system may include at least one of the images of the main message 50 and the animation 70. Similarly, in the main screens shown in FIGS. 6 to 8 described later, the operating state image may include at least one of the images of the main message 50 and the animation 70.

[0038] FIG. 6 shows a display screen displayed on the display 10 when the operating state of the first dialogue system 112 is the "voice reception enabled state". As shown in FIG. 6, the display screen in the "voice reception enabled state" includes a main message 50, a sample message 60, and an animation 70. The display positions and display images of the main message 50 and the sample message 60 are the same as those in FIG. 5. The animation 72 in FIG. 6 represents the "voice reception enabled state". The animation 72 is a blue waveform, and the amplitude of the animation 72 is larger than the amplitude of the wave of the animation 71. When voice is detected while the "voice reception enabled state" is being displayed, the display screen of the display 10 switches to the screen in FIG. 7. When no voice is detected while the "voice reception enabled state" is being displayed, the display screen of the display 10 is maintained in the state of FIG. 6.

[0039] FIG. 7 shows a display screen displayed on the display 10 when the operating state of the first dialogue system 112 is the "voice detection state". As shown in FIG. 7, the display screen in the "voice detection state" includes a main message 50, a sample message 60, and an animation 70. The display positions and display images of the main message 50 and the sample message 60 are the same as those in FIGS. 5 and 6. The animation 73 in FIG. 7 represents the "voice detection state". The animation 73 is a blue waveform, and the amplitude of the wave of the animation 73 is larger than the amplitude of the wave of the animation 72. By making the amplitude of the wave of the animation 73 larger than the amplitude of the wave of the animation 72, the occupant can confirm from the display screen that voice has been detected.

[0040] Note that the display control unit 130 may change the amplitude of the animation 73 according to the detected volume of the voice. For example, the display control unit 130 controls the amplitude of the animation 73 according to the voice input so that the larger the detected volume of the voice, the larger the amplitude of the animation 73. The occupant can confirm from the display screen that the voice is detected by the voice recognition system by seeing the movement of the amplitude of the animation 73 while speaking. Also, the animation 73 may be an animation in which a plurality of waves are superimposed, and the movement may have a sense of bounce.

[0041] FIG. 8 shows a display screen displayed on the display 10 when the operating state of the first dialogue system 112 is the "voice processing state". As shown in FIG. 8, the display screen in the "voice processing state" includes the main message 50, the sample message 60, and the animation 70. The display positions and display images of the main message 50 and the sample message 60 are the same as those in FIGS. 5 to 7. The animation 74 in FIG. 8 represents the "voice processing state". The animation 74 is a waveform of red, white, and green, and the waves are divided by color in the area. For example, at a certain time, the animation 74 is displayed on the paper surface of FIG. 8 from left to right in the order of red, white, and green. Then, the amplitude of the wave and the colored parts change with the passage of time. The amplitude of the animation 74 is smaller than the amplitude of the wave of the animation 73. The occupant can confirm from the display screen that the voice processing has started by seeing that the color of the wave has changed from "blue" to "red·white·green". The animation 74 may be an animation in which a plurality of waves are superimposed, and the movement may have a sense of bounce. Also, not limited to the animation 74, the animations 71 to 73 may also be those in which a plurality of waves are superimposed.

[0042] Also, the number of colors assigned to the animation 74 is three, which is more than the number of colors assigned to the animations 71 to 73. As a result, the occupant can more clearly distinguish from the display form of the animation 70 that the operating state of the first dialogue system 112 is the "voice processing state" and that the operating state of the first dialogue system 112 is a state other than the "voice processing state". In this embodiment, the animation 40 is divided into four types according to the operating state of the voice processing function. However, the number of types of animations is not limited to four, and may be two, three, or five or more. Also, the movement displayed in the animation is not limited to waves, and may be, for example, a pattern change, a change in the height of a bar graph, etc., as long as it has at least periodicity.

[0043] FIG. 9 is a graph for explaining the time transition of the animation 73 representing the "voice detection state". The animation 73 changes its wave form in one cycle (2π). The wave shown in the animation 73 includes characteristic points of periodic movement. The characteristic points are represented by, for example, the position of the antinode / note of the wave, the position of the peak of the wave, or the point of color change. The characteristic points are points that serve as indicators for correspondence with the other animations 71, 72, 74. In FIG. 9, (a) represents the animation 73 at time (0), and (b), (c), and (d) represent the animation 73 at times (π / 2, π, 3π / 2). The left side of FIG. 9 is an image of the wave displayed on the display screen. The right side of FIG. 9 conceptually represents the movement of the characteristic points included in the waveform of the animation 73. The movement of the characteristic points may be represented, for example, by a standing wave obtained by superimposing a wave (periodic function) that represents changes in the shape, color, hue, coordinates, etc. of the wave. For example, the characteristic points at the center point (position = λ) of the wave representing the animation 73 become zero at times (0, π, 2π ···), maximum at times (π / 2, 5π / 2, 9π / 2 ···), and minimum at times (3π / 2, 7π / 2, 11π / 2 ···). That is, in accordance with the periodic movement of the animation 73, the characteristic points also move at the same period. Note that the change in the characteristic points does not necessarily have to be represented by a movement or color that can be visually confirmed by the occupant, and may be a point calculated in terms of arithmetic operations.

[0044] FIG. 10 is a graph for explaining the relationship between the still images of animations 71 to 74 and the feature points in each of the animations 71 to 74 at a point in time during the period. In FIG. 10, (a) represents the still image of animation 71, and (b) to (d) represent the still images of animations 72 to 74. (e) is a graph of the feature points in animations 71 to 74. Even at the same time in the period, the waveform, color of the wave, etc. of animations 71 to 74 are different according to each animation. Animations 71 to 74 are in the same period. For example, the waveforms of animation 71 and animation 72 at time (π / 2) have different colors and different shapes. On the other hand, the periodic motion feature points of animations 71 to 74 are represented by the same waveform if they are at the same time in the period.

[0045] Next, the image configuration for displaying animations 71 to 74 and the screen transition of the display 10 will be described with reference to FIG. 11. FIG. 11 is a conceptual diagram for explaining the image configuration for displaying animations 71 to 74 and the still images reproduced before and after the screen transition. In FIG. 11, the first and second display screens are display screens displayed according to the respective operation states of the first dialogue system 112. The first display screen shows the screen before the transition, and the second display screen shows the screen after the transition. For example, when the operation state of the first dialogue system 112 switches from the "initial state" to the "voice reception enabled state" and the display screen of the display 10 transitions, the first display screen corresponds to the display screen of the "initial state", and the second display screen corresponds to the display screen of the "voice reception enabled state".

[0046] Animations 71 to 74 are displayed by continuously playing a plurality of still images. The images for displaying Animations 71 to 74 are composed of 60 still images, and each image is assigned an index from 0 to 59. The number of still images is the same for each of Animations 71 to 74. The index is an identifier of the still image and also represents the playback order. The index is assigned to the still images so that the characteristic points of the periodic motion correspond between the images included in Animations 71 to 74. For example, in the example of FIG. 10, since the still images of Animations 71 to 74 represent the images at the same time within the period, the same index is assigned. In other words, when comparing the still images between Animations 71 to 74, the same index is assigned to the still images in which the characteristic points correspond.

[0047] When the display control unit 130 displays Animations 71 to 74, it sends the images to the display 10 at a playback rate of playing 30 still images per second. That is, the time slot (display interval) for each still image is 20 msec. The display control unit 130 starts playing the still images in the order in which the indexes are assigned. Thus, in this embodiment, since Animations 71 to 74 are displayed by playing a plurality of still images, the processing load on the CPU can be reduced. Also, when executing a screen transition from the first display screen to the second display screen, the display control unit 130 displays up to the transition previous image, which is the still image immediately before the screen transition, and selects a second index corresponding to the first index assigned to the transition previous image. Then, the display control unit 130 starts playing the second still image from the selected second index.

[0048] In the example of FIG. 11, in order to display the "initial state" of the first dialogue system 112 with animation 71, the display control unit 130 starts image playback from a still image assigned with the index "♯0". Then, assuming that the operating state of the first dialogue system 112 has changed from the "initial state" to the "voice reception enabled state" while the still image assigned with the index "♯9" is being displayed. The display control unit 130 selects a still image assigned with the same index "♯9" as that assigned to the pre-transition still image from among the 60 still images for displaying animation 72. The display control unit 130 causes the pre-transition still image to be displayed on the display 10 until the time slot of the pre-transition still image has elapsed, and at the time when the time slot of the pre-transition still image has elapsed, starts image playback for displaying animation 72 from the selected index "♯9". That is, when the display control unit 130 executes a screen transition, it selects an index so that the characteristic points of the periodic movement displayed in animation 71 and the characteristic points of the periodic movement displayed in animation 72 maintain continuity, and starts playback of the still image from the selected index.

[0049] By the way, when the operating state of the first dialogue system 112 changes and the display screen of the display 10 transitions, if image playback is started from the first index "♯0" at the timing of the screen transition, the display will be such that the movement of the wave is interrupted in the middle, and the movement of the animation will seem to be interrupted. That is, in the example of FIG. 11, if the playback order is reset without inheriting the index at the timing of transitioning from the first display screen to the second display screen, the characteristic points will not correspond between the still images before and after the transition, resulting in an unnatural presentation of the animation.

[0050] In this embodiment, indexes are assigned so that the characteristic points of the periodic movement correspond among the images included in each of the animations 71 to 74. When the screen is switched, the still image is reproduced while inheriting the index. As a result, even if the method of animating changes, the continuity of the display of the animation can be maintained, so that the animation can be given a natural effect before and after the screen transition. Further, in this embodiment, when the screen is switched, the state in which the still image before the transition is displayed on the display 10 is maintained, and when the time slot of the still image before the transition has elapsed, the screen is switched to the second display screen. Thereby, the continuity of the animation can be maintained.

[0051] Next, the display screen after the voice recognition process by the first dialogue system 112 is normally completed will be described. FIG. 12 shows the display screen of the display 10 after the first dialogue system 112 recognizes the voice regarding the POI. For example, after the first dialogue system 112 is activated, it is assumed that the passenger uttered "Coffee Shop" as the voice regarding the POI while the display 10 is displaying the display screen of FIG. 6. The voice recognition unit 111 detects the voice "Coffee Shop" and executes a voice recognition process. When the voice recognition process is normally completed, the dialogue system control unit 110 determines from the word regarding the POI (Point of Interest) for which the voice was input that the passenger is searching for a "Coffee Shop" near the current location, and outputs a control command to the display control unit 130 so as to display the guidance image 80 regarding the POI and the map 90 around the current location on the display 10. As shown in FIG. 12, the guidance image 80 includes a character string 81 representing the result of voice recognition and POI information 82. The character string 81 is the character string of the voice recognized by the first dialogue system 112. In the example of FIG. 12, "Coffee Shop" is displayed. Thereby, the passenger can confirm what voice was recognized by the system. The POI information 82 indicates information regarding the POI, such as the name of the POI near the current location and the distance from the current location.

[0052] When a predetermined time elapses from the state of the display screen of FIG. 12 where the character string 81 is displayed, the display screen of the display 10 changes to the screen of FIG. 13. When the first dialogue system 112 finishes the speech recognition process based on the voice "Coffee Shop", it becomes possible to receive voice again. On the display screen of the display 10, the animation 72 indicating the "voice reception enabled state" is displayed at the position where the character string 81 was displayed. That is, the passenger can confirm that it is sufficient to utter the next voice of "Coffee Shop" by looking at the display of the animation 72 at the position where the character string 81 of "Coffee Shop" is displayed.

[0053] When the passenger reads out, for example, the store name included in the POI information 82 from the state of the display screen of FIG. 13, the first dialogue system 112 detects the voice of the passenger. As shown in FIG. 14, on the display screen of the display 10, the display of the animation 70 becomes the animation 73 indicating the "voice detection state" along with the voice detection. After that, when the voice detection is finished and the voice processing state is entered, as shown in FIG. 15, the display of the animation 70 becomes the animation 74 indicating the "voice processing state". Thereby, the passenger can confirm the processing status of the first dialogue system on the display of the display 10 while looking at the guidance information and map information of the POI.

[0054] When the voice processing does not complete normally from the state of the display screen of FIG. 14, the first dialogue system 112 outputs a control signal indicating that the speech recognition has failed to the display control unit 130. The display control unit 130 displays the gray waveform animation 71 at the position of the animation 73 on the display screen of FIG. 14. Thereby, the passenger can confirm that the speech recognition has failed on the display of the display 10. In addition, when the speech recognition fails, a voice such as "Please input voice again" may be output to prompt the passenger to speak.

[0055] As described above, in the present embodiment, the display control unit 130 switches the display screen according to the status of the first dialogue system 112. That is, the display control unit 130 switches between a screen including an image (corresponding to the "first information display image" of the present invention) that displays the main message 50 and the sample message as shown in FIGS. 5 to 8, and a screen including an image (corresponding to the "second information display image" of the present invention) that displays information about the POI and map information as shown in FIGS. 12 to 15, and causes the display 10 to display them. At this time, each display screen includes an image of the animation 70. When comparing the pre-transition screen (corresponding to the display screen shown in FIGS. 5 to 8) and the post-transition screen (corresponding to the display screen shown in FIGS. 12 to 15), the display of the animation 70 included in the pre-transition image and the display of the animation 70 included in the post-transition image are different in terms of position and size. Thereby, it is possible to achieve both information presentation using the first dialogue system 112 and display of the operating state of the first dialogue system 112.

[0056] Next, the transition of the display screen of the display 10 when the voice process does not complete normally from the "voice receivable state" of the first dialogue system 112 will be described. The case where the voice process does not complete normally is the case where a predetermined time has elapsed in the "voice receivable state". For example, after the display screen of the display 10 becomes the screen of FIG. 6, if the passenger does not speak, the voice recognition unit 111 does not detect the voice. When such a state continues for a predetermined time, the voice recognition unit 111 determines that the voice process has not completed normally. In addition, the case where the voice process does not complete normally also includes the case where an abnormality occurs in the voice process. The voice recognition unit 111 executes voice recognition processing on the detected voice. Then, when the input voice cannot be analyzed or there is no corresponding command 61 for the analyzed voice, the voice recognition unit 111 determines that there is an abnormality in the voice process. Such a case of abnormal determination of the voice process also corresponds to the case where the voice process does not complete normally.

[0057] As described above, when the first dialogue system 112 is in the "voice reception enabled state", the display 10 displays the main screen as shown in FIG. 6. And when it is determined that the voice processing is not completed normally, the display screen of the display 10 becomes the screen shown in FIG. 16. FIG. 16 shows the display screen of the display 10 when the voice processing is not completed normally starting from the "voice reception enabled state".

[0058] When the voice processing is not completed normally, the display control unit 130 displays a category screen as shown in FIG. 16. The category screen includes a category image that groups commands. The category image represents the group name with text or an icon after grouping a plurality of commands. The category image includes the category name 201 and the category icon 202. For example, as an example of commands, a plurality of commands related to the phone such as "call and / or dial", "redial", "telephone", etc. are grouped into one category "Phone". Also, the category icon 202 is displayed next to the category name 201. The grouping of the command 61 is not limited to the phone, and there are also navigation, audio, etc., and the names and icons of the grouped categories are included in the category screen. Also, the image including the category name 201 and the icon 202 represent an area where the touch operation is enabled. That is, the image including the category name 201 and the icon 202 also serves as a button function for allowing the occupant to select a category. For example, when the occupant touches the image including the name 201 of the category "Phone" or the icon of the category "Phone", the display control unit 130 causes the display screen to transition from the category screen to a list screen described later. Note that the category image displayed on the main screen may include at least one of the images of the name 201 and the icon 202.

[0059] In addition to the category name 201 and the icon 202, the category screen includes a main message 50 and an animation 70. The animation 70 is an animation 72 representing the "voice detection state". The voice recognition process by the voice recognition unit 111 functions effectively even when the category screen is being displayed. The passenger can confirm from the display of the animation 72 that the operating state of the first dialogue system 112 is the "voice reception enabled state".

[0060] When the passenger does not know the voice for operating the first dialogue system, the passenger can proceed with the dialogue system from the display of the category name 201 and the icon 202. Also, even when the voice cannot be processed normally due to noise or the like, the passenger can operate the dialogue system by displaying a category image (icon) that also serves as a button function. Moreover, since the category images with button functions that are frequently used are displayed, convenience is also improved. Furthermore, since the category screen includes the main message 50 and the animation 72, the passenger can confirm from the screen display that the system can be advanced through the dialogue with the controller 100.

[0061] Note that the selection of a category and / or a list by the passenger may be by voice input or other switch operations. That is, the controller 100 may determine the presence or absence of category selection based on any one of the passenger's voice, the touch operation of the display 10 by the passenger, and the switch operation by the passenger in a state where the category screen is displayed on the display 10.

[0062] When the passenger selects a category on the category screen, the display control unit 130 causes the list screen as shown in FIG. 17 to be displayed on the display 10. FIG. 17 shows the list screen displayed on the display 10.

[0063] The list screen is an image containing grouped commands and includes a list 300 having a plurality of commands. The list 300 has a plurality of syntaxes 310, and the syntax 310 has at least command 61 and, in addition to the command, has sample text 311. The list screen does not include an animation 70. The syntax 310 is a text display of a combination of command 61 and setting value 62. The sample text 311 represents arbitrary text of the setting value 62 following the command and indicates information of either a character string or a numeric string assigned to the syntax 310. For example, the topmost syntax in the list is the command "Call" and the sample text " <name>」 is composed of. 「 <name>」 indicates that after the command, a setting value 62 such as a person's name or a store name is entered. When a passenger touches the display of the syntax 310 in the list, an example of the setting value 62 that enters the sample text 311 is displayed on the right side of the list 300. Further, when the passenger touches the display of the syntax 310 in the list by double-tapping or the like, a setting list that enters the sample text 311 as shown in FIG. 18 is displayed. For example, on the display screen of FIG. 17, the topmost syntax in the list "Call <name>When "」 is selected, either the string or the numerical sequence information assigned to syntax 310 as shown in FIG. 18 is displayed. The occupant can select command 61 and the sample assigned to syntax 310 by touching the number display on the list screen of FIG. 18. In the example of FIG. 18, the topmost syntax in the list "Call <name>When "」" is selected, the controller 100 may call the phone number touched in the list, or may display a notification to make a phone call on the display 10.

[0064] Next, an example of the model flow of the dialogue in the dialogue system will be described. FIGS. 19, 20, 22 to 25 show response messages that the controller 100 returns to the voice uttered by the passenger. In FIGS. 19, 20, 22 to 25, "Customer" shows an example of a message issued by the passenger, and "INI" shows an example of a message generated by the dialogue system. Note that FIGS. 19, 20, 22 to 25 are only examples, and the dialogue system can execute the dialogue under other model flows.

[0065] FIG. 19 is a model flow of the dialogue regarding the temperature setting of the air conditioning equipment 41. For example, when the passenger utters "Hey ○○", the controller 100 emits a Japanese message "Please select whether to input the command by voice" in a mechanical voice. The model flow of FIG. 19 can advance the dialogue system and change the set temperature of the air conditioning equipment 41 by uttering the message shown in the "Customer" column of FIG. 19 in a state where the display screen of FIG. 6 is displayed on the display 10, for example.

[0066] Figure 20 is a model flow of a dialogue regarding POI search. For example, when a passenger utters a wake-up word and the voice is processed normally, the controller 100 returns a message "Please select whether to input a command by voice or not". Next, the passenger utters a message "I want to go to a café near the destination" including a word that can identify a POI such as the genre of a store. After recognizing this message, the controller 100 performs a search using the navigation system and returns a message "Please input the item number by voice from the displayed list or select another tab". At this time, a display screen shown in Figure 21 is displayed on the display 10. The controller 100 causes a guidance image 80 regarding the POI to be displayed on the display 10. The controller 100 may also display a map together. The guidance image 80 includes an animation 72 representing "voice reception enabled state" and POI information 82 obtained from the search results. Item numbers are assigned to the POI information. The passenger can select a POI by uttering the item number. In the example of Figure 21, when the passenger utters "One", the POI "AAA" is selected. The controller 100 repeats the voice "One" uttered by the passenger and then returns the next selection message such as "Do you want to change the destination?".

[0067] Figure 22 is a model flow of a dialogue regarding destination setting. Figure 23 is a model flow of a dialogue regarding voice playback. Figure 24 is a model flow of a dialogue regarding SMS (Short Message Service) transmission. Figure 25 is a model flow of a dialogue regarding SMS (Short Message Service) transmission and reception.

[0068] As described above, in the present embodiment, the controller 100 includes a voice recognition unit 111 that detects a passenger's voice input and recognizes the voice through voice processing on the detected voice, and a main screen including an operation state image indicating the operation state of the dialogue system including a sample message having a command, and a display control unit 130 that causes the display 10 to display a category screen including a category image obtained by grouping commands. Then, the controller 100 causes the display 10 to display the main screen before the voice recognition unit 111 detects the voice, causes the display 10 to display the category screen when the voice processing does not complete normally, and causes the display screen of the display 10 to transition from the category screen to another screen when a category is selected while the category screen is being displayed. As described above, in the present embodiment, the main screen has a simple screen configuration. Further, even when the voice processing in the dialogue system does not complete normally and the voice processing does not proceed, the main screen transitions to the next screen. As a result, even when the passenger is not familiar with how to use the dialogue system and the voice processing does not proceed, the screen transition proceeds. This can avoid a complicated operation for a passenger who is familiar with how to use the dialogue system, and the screen can be advanced so that a passenger who is not familiar with how to use the dialogue system can receive service without operating the dialogue system. As a result, the operability can be improved.

[0069] Also, in this embodiment, the display control method executed by the controller 100 activates an interactive system that interacts with the occupant to provide a service. Before the interactive system detects voice, a control command for displaying a main screen including an operation state image indicating the operation state of the interactive system is output to the display 10. When the voice input of the occupant is detected and voice processing is executed on the detected voice, if the voice processing is not completed normally, a control command for displaying a category screen including a category image obtained by grouping commands is output to the display 10. When a category is selected while the category screen is being displayed, a control command for transitioning to another screen is output to the display 10. As a result, it is possible to avoid complicated operations for occupants who are familiar with the usage of the interactive system, and it is possible to advance the screen so that occupants who are not familiar with the usage of the interactive system can receive service provision without operating the interactive system. As a result, the operability can be improved.

[0070] In this embodiment, the communication device 20 may be a part of the controller 100, and the communication path between the voice recognition unit 111 and the communication device 20 may be a communication path within the voice recognition unit 111 instead of an in-vehicle communication path. Further, the second interactive system 32 may perform voice processing using a voice recognition engine (corresponding to the "second voice recognition engine" of the present invention).

[0071] Note that the embodiments described above are described to facilitate understanding of the present invention and are not described to limit the present invention. Therefore, each element disclosed in the above embodiments is intended to include all design changes and equivalents belonging to the technical scope of the present invention.

Description of Reference Numerals

[0072] 1 Vehicle 2 Server 20 Communication Device 30 Steering Switch 32 Second Interactive System 70 Animation 100 Controller 110 Dialogue system control unit 111 Speech recognition unit 112 First dialogue system 120 Startup unit 121 First interface 122 Second interface 130 Display control unit< / name> < / name> < / name> < / name>

Claims

1. A display control device that controls a display screen based on the voice of a vehicle occupant, comprising: a voice recognition unit that detects the voice input of the occupant and recognizes the voice by performing voice processing on the detected voice; a controller having a display control unit that causes the display to display a main screen including an operation state image indicating an operation state of a dialogue system and a category screen including a category image grouping commands; The controller: Before the voice recognition unit detects the voice, causes the main screen to be displayed on the display; In the main screen, when the voice processing is not completed normally, causes the category screen to be displayed on the display; A display control device that, when a category is selected while the category screen is being displayed, causes the display screen to transition from the category screen to another screen.

2. The display control device according to claim 1, wherein: The display control unit causes the display to display a list screen including a list of the grouped commands.

3. The display control device according to claim 1 or 2, wherein: The operation state image includes at least one of an image of a message indicating the operation state of the dialogue system and an animation that moves according to the operation state of the dialogue system; The category screen includes an image of either the name of the category or the icon of the category.

4. The display control device according to any one of claims 1 to 3, wherein: The category screen includes an animation that moves according to the operation state of the dialogue system.

5. The display control device according to any one of claims 1 to 4, wherein: The controller: When a predetermined time has elapsed in a state where the voice input can be received before the voice processing is completed normally, causes the category screen to be displayed on the display.

6. The display control device according to any one of claims 1 to 5, wherein: The controller: When the command cannot be recognized from the utterance content of the occupant, determines that there is an abnormality in the voice processing; When it is determined that there is an abnormality in the voice processing, causes the category screen to be displayed on the display.

7. The display control device according to any one of claims 1 to 6, wherein: The controller: A display control device that determines whether a category is selected based on any one of the voice of the occupant, a touch operation on the display by the occupant, and a switch operation by the occupant, with the category screen being displayed on the display.

8. The display control device according to claim 2, wherein the controller determines whether a list is selected based on any one of the voice of the occupant, a touch operation on the display by the occupant, and a switch operation by the occupant, with the list screen being displayed on the display.

9. The display control device according to claim 2 or 8, wherein the list screen includes sample text, the list is a list that groups a plurality of syntaxes having at least commands, and the sample text indicates information of either a character string or a numerical string assigned to the syntax.

10. In the display control device according to claim 9, wherein the controller displays, on the display, a screen that selects sample text of either a character string or a numerical string assigned to the syntax when the syntax is selected.

11. In the display control device according to any one of claims 1 to 10, the main screen includes a sample message having the command.

12. In the display control device according to any one of claims 2, 8, and 9, the list screen does not include an animation.

13. A display control device that controls a display screen of a display based on the voice of a vehicle occupant, comprising a voice recognition unit that detects the voice input of the occupant and recognizes the voice by voice processing on the detected voice, and a controller having a display control unit that displays on the display a main screen including an operation state image indicating the operation state of the dialogue system, a category screen including at least one of the name and icon of a category that groups commands, and a list screen including a list that groups a plurality of syntaxes having at least the commands, wherein the controller displays the main screen on the display before the voice recognition unit detects the voice, and displays the category screen on the display when the voice processing is not completed normally. A display control device that, when a category is selected while the category screen is being displayed, causes the display screen to transition from the category screen to the list screen.

14. In a display control method for controlling a display screen of a display based on the voice of a vehicle occupant, activating an interactive system that provides a service by interacting with the occupant, outputting, to the display, a control command to display, on the display, a main screen including an operation state image indicating the operation state of the interactive system, in a state before the voice is detected by the interactive system, detecting the voice input of the occupant and performing voice processing on the detected voice, outputting, to the display, a control command to display, on the main screen, a category screen including a category image in which commands are grouped, when the voice processing is not completed normally, A display control method that outputs, to the display, a control command to transition from the category screen to another screen when a category is selected while the category screen is being displayed.

Images