Information processing apparatus, vehicle, non-transitory computer-readable medium, and control method

Abstract

The invention provides an information processing device, a vehicle, a non-transitory computer-readable medium, and a control method that proactively suggests a driver to perform an operation by voice input. An information processing device (20) is provided with a control unit that controls a specific function on the basis of an operation that can be performed by both manual input and voice input, and that, when it is determined that the operation is performed by manual input by an occupant of a vehicle (12), controls the specific function on the basis of the operation that can be performed by manual input by the occupant of the vehicle (12). In the case where the occupant is the driver, the operation can be performed in the form of voice input at a higher frequency than in the case where the occupant is not the driver.

Description

Technical Field

[0001] The present disclosure relates to an information processing device, a vehicle, a program, and a control method. Background Art

[0002] Patent Document 1 discloses a device that, when a command that can be input by voice is manually input, guides a user to the fact that voice input is possible.

[0003] Prior Art Literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2003-114698 Summary of the invention

[0006] Problems to be solved by the invention

[0007] From the perspective of safety, it is better to actively suggest to the driver to input the command by voice.

[0008] An object of the present disclosure is to actively suggest to the driver to perform an operation by voice input.

[0009] Means for solving problems

[0010] The information processing device involved in the present disclosure includes a control unit, which controls a specific function based on an operation that can be performed by both manual input and voice input. When the control unit determines that the operation is performed by a vehicle occupant in the form of manual input, the control unit notifies the occupant at a higher frequency that the operation can be performed by voice input when the occupant is a driver than when the occupant is not a driver.

[0011] The control method involved in the present disclosure includes the following contents, namely:

[0012] The computer controls specific functions based on operations that can be performed by both manual input and voice input;

[0013] When the computer determines that the operation is performed by a vehicle occupant through manual input, the computer notifies the occupant more frequently that the operation can be performed through voice input when the occupant is a driver than when the occupant is not a driver.

[0014] Effects of the Invention

[0015] According to the present disclosure, by actively suggesting to the driver to perform operation by voice input, it is possible to contribute to improvement of safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a diagram showing the configuration of a system according to an embodiment of the present disclosure.

[0017] Figure 2 It is a block diagram showing the structure of an information processing device involved in an embodiment of the present disclosure.

[0018] Figure 3 This is a flowchart showing the operation of the information processing device according to the embodiment of the present disclosure.

[0019] Figure 4 To express Figure 3 A flowchart of a modified example of the operation shown.

[0020] Figure 5 This is a flowchart showing additional operations of the information processing device according to the embodiment of the present disclosure.

[0021] Figure 6 To express Figure 5 A flowchart of a modified example of the operation shown. DETAILED DESCRIPTION

[0022] Hereinafter, one embodiment of the present disclosure will be described with reference to the drawings.

[0023] In each of the drawings, the same or corresponding parts are denoted by the same reference numerals. In the description of the present embodiment, the description of the same or corresponding parts will be appropriately omitted or simplified.

[0024] Reference Figure 1 , the structure of the system 10 involved in this embodiment is described.

[0025] The system 10 according to the present embodiment includes an information processing device 20 and a server device 30 . The information processing device 20 can communicate with the server device 30 via a network 40 .

[0026] The information processing device 20 is a computer equipped in the vehicle 12 and having a voice recognition function. The information processing device 20 is used by a user 11. The user 11 is a passenger of the vehicle 12.

[0027] The server device 30 is a computer that is installed in a data center or other facilities and belongs to a cloud computing system or other computing systems. The server device 30 is operated by a service provider that provides network services or the like.

[0028] The vehicle 12 is any type of vehicle, such as a gasoline vehicle, a diesel vehicle, a hydrogen vehicle, an HEV, a PHEV, a BEV, or a FCEV. "HEV" is an abbreviation for a hybrid electric vehicle. "PHEV" is an abbreviation for a plug-in hybrid electric vehicle. "BEV" is an abbreviation for a battery electric vehicle. "FCEV" is an abbreviation for a fuel cell electric vehicle. Although the vehicle 12 is driven by a driver, the driving can also be automated at a specific level. The level of automation is, for example, any level from level 1 to level 4 in the SAE classification. "SAE" is an abbreviation for the Society of Automotive Engineers. The vehicle 12 can also be a MaaS dedicated vehicle. "MaaS" is an abbreviation for Mobility as a Service.

[0029] The network 40 includes the Internet, at least one WAN, at least one MAN, or any combination thereof. "WAN" is short for wide area network. "MAN" is short for metropolitan area network. The network 40 may also include at least one wireless network, at least one optical network, or any combination thereof. The wireless network is, for example, an ad hoc network, a cellular network, a wireless LAN, a satellite communication network, or a terrestrial microwave network. "LAN" is short for local area network.

[0030] Reference Figure 1 , an overview of this embodiment is described.

[0031] The information processing device 20 controls a specific function Fp based on an operation that can be performed by both manual input and voice input. When the information processing device 20 determines that the operation is performed by the user 11 in the form of manual input, the user 11 is notified of the operation that can be performed by voice input at a higher frequency when the user 11 is a driver than when the user 11 is not a driver.

[0032] According to this embodiment, it is possible to actively suggest to the user to perform the operation by voice input, thereby contributing to the improvement of safety.

[0033] Reference Figure 2, the structure of the information processing device 20 involved in this embodiment is described.

[0034] The information processing device 20 includes a control unit 21 , a storage unit 22 , a communication unit 23 , an input unit 24 , an output unit 25 , and a positioning unit 26 .

[0035] The control unit 21 includes at least one processor, at least one programmable circuit, at least one dedicated circuit, or any combination thereof. The processor is a general-purpose processor such as a CPU or a GPU, or a dedicated processor specifically used for specific processing. "CPU" is the abbreviation of central processing unit. "GPU" is the abbreviation of graphics processing unit. The programmable circuit is, for example, FPGA. "FPGA" is the abbreviation of field-programmable gate array. The dedicated circuit is, for example, ASIC. "ASIC" is the abbreviation of application specific integrated circuit. The control unit 21 controls each part of the information processing device 20 and performs processing related to the action of the information processing device 20.

[0036] The storage unit 22 includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or any combination thereof. The semiconductor memory is, for example, RAM, ROM, or flash memory. "RAM" is the abbreviation of random access memory. "ROM" is the abbreviation of read only memory. RAM is, for example, SRAM or DRAM. "SRAM" is the abbreviation of static random access memory. "DRAM" is the abbreviation of dynamic random access memory. ROM is, for example, EEPROM. "EEPROM" is the abbreviation of electrically erasable programmable read only memory. Flash memory is, for example, SSD. "SSD" is the abbreviation of solid-state drive. Magnetic memory is, for example, HDD. "HDD" is the abbreviation of hard disk drive. The storage unit 22 functions as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 22 stores information used in the operation of the information processing device 20 and information obtained by the operation of the information processing device 20 .

[0037] The communication unit 23 includes at least one communication module. The communication module is, for example, a module corresponding to a mobile communication standard such as LTE, 4G standard, or 5G standard, or a wireless LAN communication standard such as IEEE802.11. "LTE" is an abbreviation for Long Term Evolution. "4G" is an abbreviation for 4th generation (fourth generation mobile communication technology). "5G" is an abbreviation for 5th generation (fifth generation mobile communication technology). "IEEE" is an abbreviation for Institute of Electrical and Electronics Engineers. The communication unit 23 communicates with the server device 30. The communication unit 23 receives information used in the operation of the information processing device 20, and sends information obtained through the operation of the information processing device 20.

[0038] The input unit 24 includes at least one input device. The input device is, for example, a physical key, an electrostatic capacitance key, a pointing device, a touch screen integrated with a display, a visible light camera, a depth camera, LiDAR, or a microphone. "LiDAR" is the abbreviation of light detection and ranging. The input unit 24 accepts an operation to input information used in the operation of the information processing device 20. For the input unit 24, instead of being set in the information processing device 20, it can also be connected to the information processing device 20 as an external input device. As a connection interface, an interface corresponding to standards such as USB, HDMI (registered trademark), or Bluetooth (registered trademark) can be used. "USB" is the abbreviation of Universal Serial Bus. "HDMI (registered trademark)" is the abbreviation of High-Definition Multimedia Interface.

[0039] The output unit 25 includes at least one output device. The output device is, for example, a display or a speaker. The display is, for example, an LCD or an organic EL display. "LCD" is the abbreviation of liquid crystal display. "EL" is the abbreviation of electro luminescent. The output unit 25 outputs information obtained by the operation of the information processing device 20. For the output unit 25, instead of being set in the information processing device 20, it can also be connected to the information processing device 20 as an external output device for displaying audio, etc. As a connection interface, an interface corresponding to standards such as USB, HDMI (registered trademark), or Bluetooth (registered trademark) can be used.

[0040] The positioning unit 26 includes at least one GNSS receiver. "GNSS" is the abbreviation of global navigation satellite system. GNSS is, for example, GPS, QZSS, BDS, GLONASS, or Galileo. "GPS" is the abbreviation of Global Positioning System. "QZSS" is the abbreviation of Quasi-Zenith Satellite System. The satellites of QZSS are called quasi-zenith satellites. "BDS" is the abbreviation of BeiDou Navigation Satellite System. "GLONASS" is the abbreviation of Global Navigation Satellite System. The positioning unit 26 measures the position of the information processing device 20.

[0041] The functions of the information processing device 20 are realized by executing the program involved in this embodiment by the processor as the control unit 21. That is, the functions of the information processing device 20 are realized by software. The program causes the computer to perform the operation of the information processing device 20, thereby causing the computer to function as the information processing device 20. That is, the computer functions as the information processing device 20 by performing the operation of the information processing device 20 according to the program.

[0042] The program can be pre-stored in a non-temporary computer-readable medium. The non-temporary computer-readable medium is, for example, a flash memory, a magnetic recording device, an optical disc, a magneto-optical recording medium, or a ROM. The circulation of the program is implemented, for example, by selling, transferring or lending a removable medium such as an SD card, DVD or CD-ROM that stores the program. "SD" is the abbreviation for Secure Digital. "DVD" is the abbreviation for digital versatile disc. "CD-ROM" is the abbreviation for compact disc read only memory. The program can also be circulated by storing the program in a storage device of a server in advance and transmitting the program from the server to other computers. The program can also be provided as a program product.

[0043] The computer temporarily stores a program stored in a removable medium or a program transmitted from a server in a main storage device, for example. Then, the computer uses a processor to read the program stored in the main storage device, and uses the processor to execute processing according to the read program. The computer can also read the program directly from the removable medium and execute processing according to the program. The computer can also execute processing according to the received program in sequence each time a program is transmitted from the server to the computer. It is also possible not to implement the transmission of the program from the server to the computer, but to execute processing through a so-called ASP-type service that implements functions only based on execution instructions and results. "ASP" is the abbreviation of application service provider. The program is information for processing implemented by an electronic computer and includes content based on the program. For example, data that is not a direct instruction to the computer but has the nature of regulating the processing of the computer is equivalent to "content based on the program."

[0044] A part or all of the functions of the information processing device 20 may be realized by a programmable circuit or a dedicated circuit as the control unit 21. In other words, a part or all of the functions of the information processing device 20 may be realized by hardware.

[0045] Reference Figure 3 , the operation of the information processing device 20 involved in this embodiment is described. The operation described below is equivalent to the control method involved in this embodiment. That is, the control method involved in this embodiment includes Figure 3 The steps S101 to S105 are shown.

[0046] In S101, the control unit 21 receives a first operation via the input unit 24. The first operation corresponds to an operation that can be performed by both manual input and voice input. For example, at least some of the operations of the devices installed on the vehicle 12, such as a navigation device, an audio device, an information terminal device, or an air conditioning device, can be performed by both manual input and voice input. Alternatively, at least some of the operations of the vehicle 12 itself can be performed by both manual input and voice input.

[0047] When the first operation is performed by manual input, the control unit 21 receives the first operation via a touch screen as the input unit 24 or other input device directly touched by the user 11 as the input unit 24. When the first operation is performed by voice input, the control unit 21 receives the first operation via a microphone as the input unit 24.

[0048] In parallel with the steps after S102, the control unit 21 controls the specific function Fp according to the first operation. The specific function Fp is a function of a device mounted on the vehicle 12. Alternatively, the specific function Fp may be a function of the vehicle 12 itself. For example, when the control unit 21 accepts the operation of a navigation device as the first operation, the functions of the navigation device such as the search for the destination are controlled. When the control unit 21 accepts the operation of an audio device as the first operation, the functions of the audio device such as the playing of music are controlled. When the control unit 21 accepts the operation of an information terminal device as the first operation, the functions of the information terminal device such as the prompting of information such as news or weather are controlled. When the control unit 21 accepts the operation of an air-conditioning device as the first operation, the functions of the air-conditioning device such as the adjustment of the temperature or air volume are controlled. When the control unit 21 receives an operation of the vehicle 12 itself as a first operation, it controls functions of the vehicle 12 itself, such as opening and closing of windows or sunroofs equipped on the vehicle 12 or displaying information related to the vehicle 12, such as fuel consumption.

[0049] In S102, the control unit 21 determines whether the first operation received in S101 is implemented by manual input or by voice input. If it is determined that the first operation is implemented by voice input, Figure 3 The illustrated flow ends. If it is determined that the first operation is performed by manual input, step S103 is executed.

[0050] In S103, the control unit 21 determines whether the user 11 is a driver. Specifically, the control unit 21 analyzes the image inside the vehicle 12 captured by a visible light camera, a depth camera, or a sensor such as LiDAR, thereby identifying the position and action of the user 11. As a method of image analysis, a known method can be used. Machine learning such as deep learning can also be used. When it is recognized that the user 11 has performed the first operation in a seat other than the driver's seat, such as the co-pilot seat, the control unit 21 determines that the user 11 is not a driver. On the other hand, when it is recognized that the user 11 has performed the first operation in the driver's seat, the control unit 21 determines that the user 11 is a driver. Alternatively, the control unit 21 may also determine that the user 11 is a driver when it is recognized that there is only one occupant in the vehicle 12. If it is determined that the user 11 is not a driver, the step S104 is executed. If it is determined that the user 11 is a driver, the step S105 is executed.

[0051] In S104, the control unit 21 notifies the user 11 that the first operation can be performed by voice input at a lower frequency than when the user 11 is the driver. Specifically, the control unit 21 stores in advance in the storage unit 22 the number of times N1a that the operation can be performed by voice input to passengers other than the driver in the recent period such as today, this week, or this month. When T1a is set to an integer greater than 0, if the number obtained by adding 1 to the number of times N1a stored in the storage unit 22 does not exceed the threshold value T1a, that is, if N1a+1<T1a or N1a+1=T1a, the control unit 21 notifies the user 11 that the first operation can be performed by voice input. For example, the control unit 21 outputs a message indicating that the first operation can be performed by voice input from the speaker as the output unit 25. When notifying the user 11 that the first operation can be performed by voice input, the control unit 21 may also display a message indicating that the first operation can be performed by voice input on the screen. That is, the control unit 21 may display a message indicating that the first operation can be performed by voice input on the display as the output unit 25. When the number obtained by adding 1 to the number of times N1a stored in the storage unit 22 exceeds the threshold value T1a, that is, when N1a+1>T1a, the control unit 21 does not notify the user 11 that the first operation can be performed by voice input.

[0052] The control unit 21 may store in advance in the storage unit 22 the number of times N1b of the operations that can be performed by both manual input and voice input by passengers other than the driver so far and the number of times N1c of the notification to passengers other than the driver that the operations can be performed by voice input, instead of the number of times N1a in the most recent period. In such an example, when T1b is a real number greater than 0, if the ratio of the number obtained by adding 1 to the number of times N1c stored in the storage unit 22 to the number obtained by adding 1 to the number of times N1b stored in the storage unit 22 does not exceed the threshold value T1b, that is, if (N1c+1) / (N1b+1)<T1b or (N1c+1) / (N1b+1)=T1b, the control unit 21 notifies the user 11 that the first operation can be performed by voice input. When the ratio of the number obtained by adding 1 to the number of times N1c stored in the storage unit 22 to the number obtained by adding 1 to the number of times N1b stored in the storage unit 22 exceeds the threshold value T1b, that is, if (N1c+1) / (N1b+1)>T1b, the control unit 21 does not notify the user 11 that the first operation can be performed by voice input.

[0053] The number of times N1a, N1b, and N1c may be stored in the storage unit 22 for each user, or may be stored in the storage unit 22 as the total number of times for all users without distinguishing between users. Instead of being stored in the storage unit 22, the number of times N1a, N1b, and N1c may be stored in the server device 30 or a storage connected to the server device 30 and acquired and updated via the communication unit 23.

[0054] In S105, the control unit 21 notifies the user 11 that the first operation can be performed by voice input at a higher frequency than when the user 11 is not a driver. Specifically, the control unit 21 stores in advance in the storage unit 22 the number of times N2a in which the driver has been notified that the operation can be performed by voice input in the recent period. When T2a is set to an integer greater than T1a, when the number obtained by adding 1 to the number of times N2a stored in the storage unit 22 does not exceed the threshold value T2a, that is, when N2a+1<T2a or N2a+1=T2a, the control unit 21 notifies the user 11 that the first operation can be performed by voice input. For example, the control unit 21 outputs a message indicating that the first operation can be performed by voice input from the speaker as the output unit 25. For the control unit 21, when notifying the user 11 that the first operation can be performed by voice input, the message indicating that the first operation can be performed by voice input can also be displayed on the screen. That is, the control unit 21 may display a message indicating that the first operation can be performed by voice input on the display as the output unit 25. When the number obtained by adding 1 to the number of times N2a stored in the storage unit 22 exceeds the threshold value T2a, that is, when N2a+1>T2a, the control unit 21 does not notify the user 11 that the first operation can be performed by voice input.

[0055] The control unit 21 may store in advance in the storage unit 22 the number of times N2b of the operation that can be performed by both manual input and voice input that has been performed by the driver so far by manual input, and the number of times N2c of the operation that the driver has been notified that can be performed by voice input, instead of the number of times N2a in the most recent period. In such an example, when T2b is set to a real number greater than T1b, if the ratio of the number obtained by adding 1 to the number of times N2c stored in the storage unit 22 to the number obtained by adding 1 to the number of times N2b stored in the storage unit 22 does not exceed the threshold value T2b, that is, if (N2c+1) / (N2b+1)<T2b or (N2c+1) / (N2b+1)=T2b, the control unit 21 notifies the user 11 that the first operation can be performed by voice input. When the ratio of the number obtained by adding 1 to the number of times N2c stored in the storage unit 22 to the number obtained by adding 1 to the number of times N2b stored in the storage unit 22 exceeds the threshold value T2b, that is, if (N2c+1) / (N2b+1)>T2b, the control unit 21 does not notify the user 11 that the first operation can be performed by voice input.

[0056] The number of times N2a, N2b, and N2c may be stored in the storage unit 22 for each user, or may be stored in the storage unit 22 as the total number of times for all users without distinguishing between users. Instead of being stored in the storage unit 22, the number of times N2a, N2b, and N2c may be stored in the server device 30 or a storage connected to the server device 30 and acquired and updated via the communication unit 23.

[0057] As described above, in this embodiment, when an operation that can be replaced by voice input is performed in a manual input manner, the control unit 21 notifies the user 11 that the operation can also be performed in a voice input manner. The control unit 21 makes the frequency of notification to the driver higher than the frequency of notification to passengers other than the driver. Therefore, safety can be improved. The control unit 21 can also make the frequency of notification to the driver who is driving higher than the frequency of notification to the driver who is not driving.

[0058] Reference Figure 4 ,right Figure 3 However, regarding steps S201 to S204, since they are respectively Figure 3 The steps S101 to S104 are the same as those shown in FIG. 1 , and thus the description thereof is omitted. Figure 3 The step of S105 shown is the same, so the description is omitted.

[0059] If it is determined in S203 that the user 11 is a driver, step S205 is executed.

[0060] In S205, the control unit 21 determines whether the user 11 is performing the second operation by manual input or by voice input. Although the second operation is different from the first operation, it is equivalent to an operation that can be performed by both manual input and voice input like the first operation. For example, when the first operation is an operation of a navigation device, the second operation can be any operation other than the operation of the navigation device, such as the operation of an audio device.

[0061] Specifically, the control unit 21 determines whether the user 11 is performing the second operation by manual input or by voice input, based on which of the frequencies of the user 11 performing the second operation by manual input and the user 11 performing the second operation by voice input is greater. More specifically, the control unit 21 stores in advance in the storage unit 22 the number of times N3a, which the user 11 has performed the second operation by manual input in the recent period such as today, this week, or this month, or the number of times N3b, which the user 11 has performed the second operation by voice input. When the number of times N3a stored in the storage unit 22 is greater than the number of times N3b stored in the storage unit 22, that is, when N3a>N3b or N3a=N3b, the control unit 21 determines that the user 11 is performing the second operation by manual input. In the case where the number of times N3a stored in the storage unit 22 is less than the number of times N3b stored in the storage unit 22, that is, if N3a<N3b, the control unit 21 determines that the user 11 is performing the second operation by voice input. Alternatively, the control unit 21 may also determine whether the user 11 is performing the second operation by manual input or by voice input based on whether the user 11 performed the second operation most recently by manual input or by voice input. In the case where it is determined that the user 11 is performing the second operation by manual input, step S206 is executed. In the case where it is determined that the user 11 is performing the second operation by voice input, step S207 is executed.

[0062] Instead of being stored in the storage unit 22 , the number of times N3 a and the number of times N3 b may be stored in the server device 30 or a storage connected to the server device 30 , and may be acquired and updated via the communication unit 23 .

[0063] In S207, the control unit 21 notifies the user 11 that the first operation can be performed by voice input at a higher frequency than when the user 11 is performing the second operation by manual input. Specifically, similarly to the step of S205, the control unit 21 stores in advance in the storage unit 22 the number of times N2a in which the driver has been notified that the operation can be performed by voice input in the recent period. When T3a is set to an integer greater than T2a, when the number obtained by adding 1 to the number of times N2a stored in the storage unit 22 does not exceed the threshold value T3a, that is, when N2a+1<T3a or N2a+1=T3a, the control unit 21 notifies the user 11 that the first operation can be performed by voice input. For example, the control unit 21 outputs a message indicating that the first operation can be performed by voice input from the speaker as the output unit 25. For the control unit 21, when notifying the user 11 that the first operation can be performed by voice input, it can also display a message indicating that the first operation can be performed by voice input on the screen. That is, the control unit 21 may display a message indicating that the first operation can be performed by voice input on the display as the output unit 25. When the number obtained by adding 1 to the number of times N2a stored in the storage unit 22 exceeds the threshold value T3a, that is, when N2a+1>T3a, the control unit 21 does not notify the user 11 that the first operation can be performed by voice input.

[0064] As in step S205, the control unit 21 may store in advance in the storage unit 22 the number of times N2b of the operation that can be performed by both manual input and voice input by the driver so far and the number of times N2c of the driver being notified that the operation can be performed by voice input, instead of the number of times N2a in the most recent period. In such an example, when T3b is set to a real number greater than T2b, if the ratio of the number obtained by adding 1 to the number of times N2c stored in the storage unit 22 to the number obtained by adding 1 to the number of times N2b stored in the storage unit 22 does not exceed the threshold value T3b, that is, if (N2c+1) / (N2b+1)<T3b or (N2c+1) / (N2b+1)=T3b, the control unit 21 notifies the user 11 that the first operation can be performed by voice input. When the ratio of the number obtained by adding 1 to the number of times N2c stored in the storage unit 22 to the number obtained by adding 1 to the number of times N2b stored in the storage unit 22 exceeds the threshold value T3b, that is, if (N2c+1) / (N2b+1)>T3b, the control unit 21 does not notify the user 11 that the first operation can be performed by voice input.

[0065] As described above, when an operation replaceable by voice input is performed by manual input, the control unit 21 may make the frequency of notification to the driver who usually performs other operations by voice input higher than the frequency of notification to the driver who usually performs other operations by manual input. The control unit 21 may also make the frequency of notification to passengers other than the driver who usually performs other operations by voice input higher than the frequency of notification to passengers other than the driver who usually performs other operations by manual input.

[0066] exist Figure 3 S105 and Figure 4 In S206 of FIG. 1 , the control unit 21 may notify the user 11 that the first operation can be performed by voice input regardless of the number of times N2a, the number of times N2b, and the number of times N2c. That is, when the first operation is performed by the driver by manual input, the control unit 21 may always notify the driver that the first operation can be performed by voice input. Alternatively, when the first operation is performed by the driver who is performing the second operation by voice input by manual input, the control unit 21 may always notify the driver that the first operation can be performed by voice input.

[0067] Reference Figure 5 , the additional actions of the information processing device 20 involved in this embodiment are described. The actions described below can also be additionally applied to the control method involved in this embodiment. That is, the control method involved in this embodiment can also further include Figure 5 The steps S301 to S303 are shown.

[0068] In S301, the control unit 21 receives at least one operation of the multiple stages of operations via the input unit 24. Similar to the first operation, the multiple stages of operations correspond to operations that can be performed by both manual input and voice input. The multiple stages of operations are, for example, operations of selecting options included in a hierarchical menu in sequence, such as first selecting an artist or a genre, then selecting an album, and finally selecting a song in an audio device. The multiple stages of operations may also include the first operation, the second operation, or both.

[0069] When at least one stage of operation is performed by manual input, the control unit 21 receives at least one stage of operation via the touch screen as the input unit 24 or other input device directly touched by the user 11 as the input unit 24. When at least one stage of operation is performed by voice input, the control unit 21 receives at least one stage of operation via the microphone as the input unit 24.

[0070] In parallel with the steps after S302, the control unit 21 controls a specific function Fp according to at least one stage of operation. For example, when the control unit 21 receives an operation of selecting an artist in the audio device as at least one stage of operation, it controls functions of the audio device such as searching for an album of the selected artist.

[0071] In S302, the control unit 21 determines whether the at least one stage of operation received in S301 is implemented by manual input or by voice input. If it is determined that the at least one stage of operation is implemented by voice input, Figure 5 The illustrated flow ends. If it is determined that at least one stage of operation is performed by manual input, step S303 is executed.

[0072] In S303, the control unit 21 notifies the user 11 that the operation of the next stage can be performed by voice input. For example, the control unit 21 outputs a message through the speaker as the output unit 25 indicating that the operation of selecting the album of the artist selected in the previous stage of operation, etc., and the operation of the next stage can be performed by voice input. The control unit 21 may also display a message on the display as the output unit 25 indicating that the operation of the next stage can be performed by voice input.

[0073] Reference Figure 6 ,right Figure 5 However, regarding the steps S311 and S312, since they are respectively Figure 5 The steps S301 and S302 shown are the same, so the description is omitted.

[0074] In S313, the control unit 21 outputs a voice message requesting the user 11 to perform a voice input corresponding to the operation of the next stage. For example, the control unit 21 outputs a voice message from the speaker as the output unit 25, such as "Please tell me the album name" or "Which album do you choose?", which urges the user 11 to tell the album name of the artist selected in the previous stage of operation by voice.

[0075] The present disclosure is not limited to the above-mentioned embodiments. For example, two or more blocks recorded in the block diagram may be merged, or one block may be divided. Instead of executing two or more steps recorded in the flowchart in a time series according to the description, they may be executed in parallel or in a different order according to the processing capacity of the device executing each step or as needed. In addition, changes can be made within the scope of the subject matter of the present disclosure.

[0076] Hereinafter, a part of the embodiments of the present disclosure will be exemplified, but it should be noted that the embodiments of the present disclosure are not limited to these cases.

[0077] [Note 1]

[0078] An information processing device comprises a control unit, which controls a specific function based on an operation that can be performed by both manual input and voice input. When the control unit determines that the operation is performed by a vehicle occupant by manual input, the control unit notifies the occupant that the operation can be performed by voice input at a higher frequency when the occupant is a driver than when the occupant is not a driver.

[0079] [Note 2]

[0080] An information processing device as described in Note 1, wherein, when the control unit determines that the first operation and the second operation respectively corresponding to the operations are performed by the occupant by manual input, the control unit notifies the occupant that the first operation can be performed by voice input at a higher frequency than when the occupant is performing the second operation by manual input when at least the occupant is a driver.

[0081] [Note 3]

[0082] An information processing device as described in Note 2, wherein the control unit determines whether the occupant is performing the second operation by manual input or by voice input based on which of the two is greater: the frequency with which the occupant performs the second operation by manual input or the frequency with which the occupant performs the second operation by voice input.

[0083] [Note 4]

[0084] An information processing device as described in Note 2, wherein the control unit determines whether the occupant is performing the second operation by manual input or by voice input based on whether the occupant performed the second operation by manual input or by voice input when the occupant most recently performed the second operation.

[0085] [Note 5]

[0086] An information processing device as described in any one of Notes 1 to 4, wherein, when the control unit determines that at least one stage of operations corresponding to multiple stages of the operations has been performed by the occupant in the form of manual input, the control unit notifies the occupant that the next stage of operation can be performed by voice input.

[0087] [Note 6]

[0088] An information processing device as described in Supplement 5, wherein the operations in the multiple stages are operations of sequentially selecting options included in a hierarchical menu.

[0089] [Note 7]

[0090] An information processing device as described in any one of Notes 1 to 4, wherein the control unit, when determining that at least one of the operations corresponding to multiple stages of the operation has been performed by the occupant in the form of manual input, voice outputs a message requesting the occupant to perform a voice input corresponding to the next stage of the operation.

[0091] [Note 8]

[0092] An information processing device as described in Supplement 7, wherein the operations in the multiple stages are operations of sequentially selecting options included in a hierarchical menu.

[0093] [Note 9]

[0094] An information processing device as described in any one of Notes 1 to 8, wherein the control unit displays a message indicating that the operation can be performed by voice input on the screen when notifying the occupant that the operation can be performed by voice input.

[0095] [Note 10]

[0096] An information processing device as described in any one of Notes 1 to 9, wherein the specific function is a function possessed by the vehicle or a device mounted on the vehicle.

[0097] [Note 11]

[0098] A vehicle comprising the information processing device according to any one of Supplementary Notes 1 to 10.

[0099] [Note 12]

[0100] A program for causing a computer to function as the information processing device described in any one of Supplementary Notes 1 to 10.

[0101] [Note 13]

[0102] A control method, comprising: a computer controls a specific function according to an operation that can be performed by both manual input and voice input;

[0103] When the computer determines that the operation is performed by a vehicle occupant in the form of manual input, the computer notifies the occupant more frequently that the operation can be performed in the form of voice input when the occupant is a driver than when the occupant is not a driver.

[0104] [Note 14]

[0105] A control method as described in Note 13, which further includes the following content, namely, when the computer determines that the first operation and the second operation respectively corresponding to the operations are performed by the occupant by manual input, the computer notifies the occupant that the first operation can be performed by voice input at a higher frequency than a case where the occupant is performing the second operation by voice input.

[0106] [Note 15]

[0107] The control method as described in Note 14 further includes the following content, namely, the computer determines whether the occupant is performing the second operation by manual input or by voice input based on which of the frequencies of the occupant performing the second operation by manual input and the occupant performing the second operation by voice input is greater.

[0108] [Note 16]

[0109] The control method as described in Note 14 further includes the following content, namely, the computer determines whether the occupant is performing the second operation by manual input or by voice input based on whether the occupant performed the second operation by manual input or by voice input when the occupant most recently performed the second operation.

[0110] [Note 17]

[0111] A control method as described in any one of Notes 13 to 16, which further includes the following content, namely, when the computer determines that at least one stage of the operations corresponding to multiple stages of the operations has been performed by the occupant in the form of manual input, the computer notifies the occupant that the next stage of the operation can be performed by voice input.

[0112] [Note 18]

[0113] A control method as described in Note 17, wherein the operations in the multiple stages are operations of sequentially selecting options included in a hierarchical menu.

[0114] [Note 19]

[0115] A control method as described in any one of Notes 13 to 16, which further includes the following content, namely, when the computer determines that at least one operation of multiple stages of operations respectively corresponding to the operation has been performed by the occupant in the form of manual input, the computer voice outputs a message requesting the occupant to perform a voice input equivalent to the next stage of the operation.

[0116] [Note 20]

[0117] A control method as described in Appendix 19, wherein the operations in the multiple stages are operations of sequentially selecting options included in a hierarchical menu.

[0118] Explanation of symbols

[0119] 10. System;

[0120] 11. User;

[0121] 12. Vehicles;

[0122] 20. Information processing device;

[0123] 21. Control department;

[0124] 22. Storage department;

[0125] 23. Ministry of Communications;

[0126] 24. Input unit;

[0127] 25. Output unit;

[0128] 26. Positioning unit;

[0129] 30. Server device;

[0130] 40. Internet.

Claims

1. An information processing device comprising a control unit, wherein the control unit controls a specific function based on an operation that can be performed both manually and by voice input, and when the control unit determines that the operation is performed by a vehicle occupant by manual input, the control unit notifies the occupant that the operation can be performed by voice input at a higher frequency when the occupant is a driver than when the occupant is not a driver.

2. The information processing device according to claim 1, wherein: When the control unit determines that the first operation corresponding to the first operation and the second operation respectively corresponding to the operations are performed by the occupant by manual input, when at least the occupant is a driver, the control unit notifies the occupant at a higher frequency that the first operation can be performed by voice input than when the occupant is performing the second operation by manual input.

3. The information processing device according to claim 2, wherein: The control unit determines whether the occupant is performing the second operation by manual input or by voice input based on which of the frequencies of the occupant performing the second operation by manual input and the occupant performing the second operation by voice input is greater.

4. The information processing device according to claim 2, wherein: The control unit determines whether the occupant is performing the second operation by manual input or by voice input based on whether the occupant performed the second operation by manual input or by voice input when the occupant most recently performed the second operation.

5. The information processing device according to claim 1, wherein: When the control unit determines that at least one operation of the operations corresponding to the plurality of stages of the operation has been performed by the occupant by manual input, the control unit notifies the occupant that the next stage of the operation can be performed by voice input.

6. The information processing device according to claim 5, wherein: The operations in the multiple stages are operations of sequentially selecting the options included in the hierarchical menu.

7. The information processing device according to claim 1, wherein: The control unit outputs a voice message requesting the occupant to perform a voice input corresponding to a next stage of operation when it is determined that at least one stage of operation corresponding to a plurality of stages of the operation has been performed by the occupant in the form of manual input.

8. The information processing device according to claim 7, wherein: The operations in the multiple stages are operations of sequentially selecting the options included in the hierarchical menu.

9. The information processing device according to claim 1, wherein: When notifying the occupant that the operation can be performed by voice input, the control unit displays a message on the screen indicating that the operation can be performed by voice input.

10. The information processing device according to claim 1, wherein: The specific function is a function possessed by the vehicle or a device mounted on the vehicle.

11. A vehicle comprising the information processing device according to any one of claims 1 to 10. 12 . A non-transitory computer-readable medium having stored thereon a program for causing a computer to function as the information processing device according to claim 1 .

13. A control method, comprising the following contents: The computer controls specific functions based on operations that can be performed by both manual input and voice input; When the computer determines that the operation is performed by a vehicle occupant in the form of manual input, the computer notifies the occupant more frequently that the operation can be performed in the form of voice input when the occupant is a driver than when the occupant is not a driver.

14. The control method according to claim 13, wherein: It also includes the following content, namely, when the computer determines that the first operation and the second operation respectively corresponding to the operations are performed by the occupant by manual input, the computer notifies the occupant that the first operation can be performed by voice input at a higher frequency than the case where the occupant is performing the second operation by manual input.

15. The control method according to claim 14, wherein: It also includes the following content, namely, the computer determines whether the occupant is performing the second operation by manual input or by voice input based on which of the frequencies of the occupant performing the second operation by manual input and the occupant performing the second operation by voice input is greater.

16. The control method according to claim 14, wherein: It also includes the following content, namely, the computer determines whether the occupant is performing the second operation by manual input or by voice input based on whether the occupant performed the second operation by manual input or by voice input when the occupant most recently performed the second operation.

17. The control method according to claim 13, wherein: It also includes the following content, namely, when the computer determines that at least one stage of the operations corresponding to multiple stages of the operations has been performed by the occupant in the form of manual input, it notifies the occupant that the next stage of the operation can be performed by voice input.

18. The control method according to claim 17, wherein: The operations in the multiple stages are operations of sequentially selecting the options included in the hierarchical menu.

19. The control method according to claim 13, wherein: It also includes the following content, namely, when the computer determines that at least one operation of multiple stages of the operation has been performed by the occupant in the form of manual input, the computer voice outputs a message requesting the occupant to perform a voice input equivalent to the next stage of the operation.

20. The control method according to claim 19, wherein: The operations in the multiple stages are operations of sequentially selecting the options included in the hierarchical menu.

Images