Information processing device and control method

Abstract

To make return operation more in line with user intention.SOLUTION: An information processing device 20 includes a control unit that controls a specific function in accordance with operation performed by a speech input from an occupant in a vehicle 12, and monitors a reaction of the occupant after a state of an apparatus having the specific function transitions from a first state to a second state different from the first state as a result of controlling the specific function, and determines whether or not to return the state of the apparatus from the second state to the first state according to the reaction.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus and a control method.

Background Art

[0002] Patent Document 1 discloses a system that, when a return operation is instructed by voice, restores the state before the canceled operation instruction was executed, and when a screen switch was performed by the canceled operation instruction, displays the screen before the canceled operation instruction was executed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a conventional system, a return operation that does not conform to the user's intention may be performed.

[0005] An object of the present disclosure is to make the return operation more in line with the user's intention.

Means for Solving the Problems

[0006] The information processing apparatus according to the present disclosure is a control unit that controls a specific function in response to an operation performed by voice input from an occupant of a vehicle, and monitors the reaction of the occupant after the state of a device having the specific function has transitioned from a first state to a second state different from the first state as a result of controlling the specific function, and determines whether to return the state of the device from the second state to the first state based on the reaction.

[0007] The control method according to the present disclosure is A computer controls a specific function in response to an operation performed by voice input from an occupant of a vehicle, after the state of the device having the specific function transitions from a first state to a second state different from the first state as a result of the computer controlling the specific function, the computer monitors the reaction of the occupant, and the computer determines whether to return the state of the device from the second state to the first state based on the reaction. It includes.

Effect of the Invention

[0008] According to the present disclosure, the return operation can be made more in line with the user's intention.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Mode for Carrying Out the Invention

[0010] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings.

[0011] In each figure, the same or corresponding parts are denoted by the same reference numerals. In the description of the present embodiment, the description of the same or corresponding parts will be omitted or simplified as appropriate.

[0012] With reference to FIG. 1, the configuration of the system 10 according to the present embodiment will be described.

[0013] The system 10 according to this embodiment includes an information processing device 20 and a server device 30. The information processing device 20 can communicate with the server device 30 via a network 40.

[0014] The information processing device 20 is a computer with a voice recognition function mounted on the vehicle 12. The information processing device 20 is used by the user 11. The user 11 is a passenger in the vehicle 12.

[0015] The server device 30 is a computer belonging to a cloud computing system or other computing system installed in a facility such as a data center. The server device 30 is operated by a service provider that provides services such as web services.

[0016] The vehicle 12 is, for example, an automobile of any type such as a gasoline vehicle, a diesel vehicle, a hydrogen vehicle, an HEV, a PHEV, a BEV, or an FCEV. "HEV" is an abbreviation for hybrid electric vehicle. "PHEV" is an abbreviation for plug-in hybrid electric vehicle. "BEV" is an abbreviation for battery electric vehicle. "FCEV" is an abbreviation for fuel cell electric vehicle. The vehicle 12 may be driven by the user 11 or may have its driving automated at any level. The level of automation is, for example, any one of levels 1 to 5 in the SAE level classification. "SAE" is the abbreviation of Society of Automotive Engineers. The vehicle 12 may also be a vehicle dedicated to MaaS. "MaaS" is an abbreviation for Mobility as a Service.

[0017] Network 40 includes the Internet, at least one WAN, at least one MAN, or any combination thereof. "WAN" is an abbreviation for wide area network. "MAN" is an abbreviation for metropolitan area network. Network 40 may include at least one wireless network, at least one optical network, or any combination thereof. The wireless network is, for example, an ad hoc network, a cellular network, a wireless LAN, a satellite communication network, or a terrestrial microwave network. "LAN" is an abbreviation for local area network.

[0018] Referring to FIG. 1, the outline of this embodiment will be described.

[0019] The information processing apparatus 20 controls a specific function Fp in response to an operation performed by voice input from the user 11. As a result of controlling the specific function Fp, after the state of the device having the specific function Fp transitions from the first state T1 to the second state T2 different from the first state T1, the information processing apparatus 20 monitors the reaction of the user 11. The information processing apparatus 20 determines whether to return the state of the device from the second state T2 to the first state T1 based on the reaction of the user 11.

[0020] According to this embodiment, the operation performed by voice input can be restored. And the restoration operation for restoration can be made more in line with the intention of the user 11. For example, it is possible to reduce the possibility that a restoration operation contrary to the intention of the user 11 is performed due to misrecognition even though the user 11 did not really give an instruction.

[0021] Referring to FIG. 2, the configuration of the information processing apparatus 20 according to this embodiment will be described.

[0022] The information processing apparatus 20 includes a control unit 21, a storage unit 22, a communication unit 23, an input unit 24, an output unit 25, and a positioning unit 26.

[0023] The control unit 21 includes at least one processor, at least one programmable circuit, at least one dedicated circuit, or any combination thereof. The processor is a general-purpose processor such as a CPU or GPU, or a dedicated processor specialized for specific processing. "CPU" is an abbreviation for central processing unit. "GPU" is an abbreviation for graphics processing unit. The programmable circuit is, for example, an FPGA. "FPGA" is an abbreviation for field-programmable gate array. The dedicated circuit is, for example, an ASIC. "ASIC" is an abbreviation for application specific integrated circuit. While controlling each part of the information processing device 20, the control unit 21 executes processing related to the operation of the information processing device 20.

[0024] The storage unit 22 includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or any combination thereof. The semiconductor memory is, for example, a RAM, ROM, or flash memory. "RAM" is an abbreviation for random access memory. "ROM" is an abbreviation for read only memory. The RAM is, for example, an SRAM or DRAM. "SRAM" is an abbreviation for static random access memory. "DRAM" is an abbreviation for dynamic random access memory. The ROM is, for example, an EEPROM. "EEPROM" is an abbreviation for electrically erasable programmable read only memory. The flash memory is, for example, an SSD. "SSD" is an abbreviation for solid-state drive. The magnetic memory is, for example, an HDD. "HDD" is an abbreviation for hard disk drive. The storage unit 22 functions as, for example, a main memory device, an auxiliary storage device, or a cache memory. Information used for the operation of the information processing device 20 and information obtained by the operation of the information processing device 20 are stored in the storage unit 22.

[0025] The communication unit 23 includes at least one communication module. The communication module is a module corresponding to a mobile communication standard such as LTE, 4G standard, or 5G standard, or a wireless LAN communication standard such as IEEE802.11. "LTE" is an abbreviation for Long Term Evolution. "4G" is an abbreviation for 4th generation. "5G" is an abbreviation for 5th generation. "IEEE" is an abbreviation for Institute of Electrical and Electronics Engineers. The communication unit 23 communicates with the server device 30. The communication unit 23 receives information used for the operation of the information processing device 20 and transmits information obtained by the operation of the information processing device 20.

[0026] The input unit 24 includes at least one input device. The input device is, for example, a physical key, a capacitive key, a pointing device, a touch screen provided integrally with a display, a visible light camera, a depth camera, LiDAR, or a microphone. "LiDAR" is an abbreviation for light detection and ranging. The input unit 24 receives an operation of inputting information used for the operation of the information processing device 20. Instead of being provided in the information processing device 20, the input unit 24 may be connected to the information processing device 20 as an external input device. As the connection interface, an interface corresponding to a standard such as USB, HDMI (registered trademark), or Bluetooth (registered trademark) can be used. "USB" is an abbreviation for Universal Serial Bus. "HDMI (registered trademark)" is an abbreviation for High-Definition Multimedia Interface.

[0027] The output unit 25 includes at least one output device. The output device is, for example, a display or a speaker. The display is, for example, an LCD or an organic EL display. "LCD" is an abbreviation for liquid crystal display. "EL" is an abbreviation for electro luminescent. The output unit 25 outputs information obtained by the operation of the information processing device 20. Instead of being provided in the information processing device 20, the output unit 25 may be connected to the information processing device 20 as an external output device such as a display audio. As the connection interface, an interface corresponding to a standard such as USB, HDMI (registered trademark), or Bluetooth (registered trademark) can be used.

[0028] The positioning unit 26 includes at least one GNSS receiver. "GNSS" is an abbreviation for global navigation satellite system. GNSS is, for example, GPS, QZSS, BDS, GLONASS, or Galileo. "GPS" is an abbreviation for Global Positioning System. "QZSS" is an abbreviation for Quasi-Zenith Satellite System. The satellites of QZSS are called quasi-zenith satellites. "BDS" is an abbreviation for BeiDou Navigation Satellite System. "GLONASS" is an abbreviation for Global Navigation Satellite System. The positioning unit 26 measures the position of the information processing device 20.

[0029] The functions of the information processing device 20 are realized by executing the program according to this embodiment by a processor as the control unit 21. That is, the functions of the information processing device 20 are realized by software. The program causes a computer to execute the operations of the information processing device 20, causing the computer to function as the information processing device 20. That is, the computer functions as the information processing device 20 by executing the operations of the information processing device 20 according to the program.

[0030] The program can be stored in a non-transitory computer-readable medium. Examples of non-transitory computer-readable media include flash memory, magnetic recording devices, optical discs, magneto-optical recording media, or ROM. The distribution of the program can be carried out, for example, by selling, transferring, or lending a portable medium such as an SD card, DVD, or CD-ROM that stores the program. "SD" is an abbreviation for Secure Digital. "DVD" is an abbreviation for digital versatile disc. "CD-ROM" is an abbreviation for compact disc read only memory. The program can also be distributed by storing it in the server's storage and transferring the program from the server to other computers. The program may be provided as a program product.

[0031] The computer stores, for example, a program stored in a portable medium or a program transferred from a server, once, in the main memory device. Then, the computer reads the program stored in the main memory device with the processor and executes the processing according to the read program with the processor. The computer may directly read the program from the portable medium and execute the processing according to the program. The computer may sequentially execute the processing according to the received program each time a program is transferred from the server to the computer. The processing may be executed by a so-called ASP-type service that realizes the function only by execution instructions and result acquisition without transferring the program from the server to the computer. "ASP" is an abbreviation for application service provider. The program is information for use in processing by an electronic computer and includes those conforming to the program. For example, data that is not a direct instruction to the computer but has the property of defining the computer's processing corresponds to "those conforming to the program".

[0032] Some or all of the functions of the information processing apparatus 20 may be realized by a programmable circuit or a dedicated circuit as the control unit 21. That is, some or all of the functions of the information processing apparatus 20 may be realized by hardware.

[0033] Referring to FIG. 3, the operation of the information processing apparatus 20 according to the present embodiment will be described. The operations described below correspond to the control method according to the present embodiment. That is, the control method according to the present embodiment includes the steps of S101 to S105 shown in FIG. 3.

[0034] In S101, the control unit 21 receives an operation performed by voice input from the user 11 via a microphone as the input unit 24. For example, assume that at least some operations of the device Di mounted on the vehicle 12, such as a navigation device, an audio device, or an air conditioner, can be performed by voice input.

[0035] In S102, the control unit 21 controls a specific function Fp according to the operation received in S101. The specific function Fp is a function of the device Di mounted on the vehicle 12. For example, when the control unit 21 receives an operation of the navigation device in S101, it controls functions of the navigation device, such as changing the map scale or route. When the control unit 21 receives an operation of the audio device in S101, it controls functions of the audio device, such as changing the volume or track. When the control unit 21 receives an operation of the air conditioner in S101, it controls functions of the air conditioner, such as changing the temperature or air volume.

[0036] As a result of controlling the specific function Fp in S102, the state of the device Di mounted on the vehicle 12 transitions from the first state T1 to a second state T2 different from the first state T1. Then, in S103, the control unit 21 monitors the reaction of the user 11. Specifically, the control unit 21 analyzes the sound inside the vehicle 12 collected by the microphone as the input unit 24 to recognize the reaction of the user 11 by voice. As a method of voice analysis, a known method can be used. Machine learning such as deep learning may be used. The control unit 21 may analyze the image inside the vehicle 12 captured by a sensor such as a visible light camera, a depth camera, or LiDAR as the input unit 24 to recognize the reaction of the user 11 by expression or movement. As a method of image analysis, a known method can be used. Machine learning such as deep learning may be used.

[0037] In S104, the control unit 21 determines whether to return the state of the device Di mounted on the vehicle 12 from the second state T2 to the first state T1 according to the reaction of the user 11. Specifically, when the control unit 21 recognizes a negative reaction of the user 11 within a certain period in S103, it determines to return the state of the device Di from the second state T2 to the first state T1. For example, if the control unit 21 causes the navigation device to reduce the map scale in S102 and then recognizes the user 11's statement "It's too small and hard to see!" in S103, it determines to return the state of the navigation device to the state before the change of the map scale. If the control unit 21 causes the audio device to increase the music volume in S102 and then recognizes the user 11's statement "It's too noisy!" in S103, it determines to return the state of the audio device to the state before the change of the volume. Alternatively, the control unit 21 may recognize an unhappy expression or a movement of covering the ears of the user 11 and determine to return the state of the audio device to the state before the change of the volume. If the control unit 21 causes the air conditioner to lower the cooling temperature in S102 and then recognizes the user 11's statement "It's cold!" in S103, it determines to return the state of the air conditioner to the state before the change of the temperature. Alternatively, the control unit 21 may recognize an unhappy expression or a shivering movement of the user 11 and determine to return the state of the air conditioner to the state before the change of the temperature. On the other hand, when the control unit 21 recognizes a positive reaction of the user 11 within a certain period in S103, it determines not to return the state of the device Di from the second state T2 to the first state T1. When there is no reaction from the user 11 for a certain period, the control unit 21 also determines not to return the state of the device Di from the second state T2 to the first state T1.

[0038] If it is determined in S104 to return the state of the device Di from the second state T2 to the first state T1, the step of S105 is executed. On the other hand, if it is determined in S104 not to return the state of the device Di from the second state T2 to the first state T1, the flow shown in FIG. 3 ends.

[0039] In S105, the control unit 21 refers to the history of one or more operations performed by voice input and returns the state of the device Di mounted on the vehicle 12 from the second state T2 to the first state T1. The operation history is stored in the storage unit 22 and is updated by the control unit 21 at any time. Alternatively, the operation history may be stored in the device Di and accessed by the information processing device 20 at any time. After S105, the flow shown in FIG. 3 ends.

[0040] According to the present embodiment, a return operation can be immediately performed without waiting for an instruction from the user 11. For example, as a result of an operation to decrease the scale of the map, even if the scale of the map has decreased too much, it is possible to immediately respond to the negative reaction of the user 11 and return the scale of the map to its original state. As a result of an operation to increase the volume of the music, even if the volume of the music has increased too much, it is possible to immediately respond to the negative reaction of the user 11 and return the volume of the music to its original state. As a result of an operation to decrease the temperature of the air conditioner, even if the temperature of the air conditioner has decreased too much, it is possible to immediately respond to the negative reaction of the user 11 and return the temperature of the air conditioner to its original state.

[0041] When the control unit 21 determines to return the state of the device Di from the second state T2 to the first state T1, in the flow shown in FIG. 3, the control unit 21 automatically returns the state, but as shown below, the control unit 21 may confirm with the user 11 before returning the state.

[0042] Referring to FIG. 4, a modified example of the operation shown in FIG. 3 will be described. However, since the steps of S201 to S204 are the same as the steps of S101 to S104 shown in FIG. 3, respectively, the description thereof will be omitted. The step of S207 is also the same as the step of S105 shown in FIG. 3, so the description thereof will be omitted.

[0043] When it is determined in S204 to return the state of the device Di from the second state T2 to the first state T1, the step of S205 is executed. On the other hand, when it is determined in S204 not to return the state of the device Di from the second state T2 to the first state T1, the flow shown in FIG. 4 ends.

[0044] In S205, the control unit 21 outputs a confirmation request to the user 11 via the output unit 25. For example, when the control unit 21 determines in S204 to return the state of the navigation device to the state before the change in the map scale, it outputs an audio confirmation message "Shall we return the map scale to the original?" from the speaker as the output unit 25. When the control unit 21 determines in S204 to return the state of the audio device to the state before the change in the volume, it outputs an audio confirmation message "Shall we return the music volume to the original?" from the speaker as the output unit 25. When the control unit 21 determines in S204 to return the state of the air conditioner to the state before the change in the temperature immediately after reducing the cooling temperature of the air conditioner in S102, it outputs an audio confirmation message "Shall we return the cooling temperature to the original?" from the speaker as the output unit 25. The confirmation message may be displayed on the display as the output unit 25.

[0045] In S206, the control unit 21 receives the response from the user 11 via the input unit 24. For example, the control unit 21 receives an affirmative reply such as "Yes" or a negative reply such as "No" via the microphone as the input unit 24. Alternatively, the control unit 21 may display a "Return to original" button on the display as the output unit 25 and receive an operation of tapping the button via the touch screen as the input unit 24. If an affirmative reply is received or an operation of tapping the "Return to original" button is received within the specified time, it means that confirmation has been obtained. If a negative reply is received or an operation of tapping the "Return to original" button is not received within the specified time, it means that confirmation has not been obtained.

[0046] If confirmation is obtained in S206, the step of S207 is executed. On the other hand, if confirmation is not obtained in S206, the flow shown in FIG. 4 ends.

[0047] The present disclosure is not limited to the above-described embodiments. For example, two or more blocks described in the block diagram may be integrated, or one block may be divided. Instead of executing two or more steps described in the flowchart in time series according to the description, depending on the processing capabilities of the device that executes each step, or as necessary, they may be executed in parallel or in a different order. In addition, changes can be made without departing from the spirit of the present disclosure.

Explanation of Signs

[0048] 10 System 11 User 12 Vehicle 20 Information Processing Device 21 Control Unit 22 Storage Unit 23 Communication Unit 24 Input Unit 25 Output Unit 26 Positioning Unit 30 Server Device 40 Network

Claims

1. A control unit that controls a specific function in response to an operation performed by voice input from a vehicle occupant, and as a result of controlling the specific function, after the state of the device having the specific function transitions from a first state to a second state different from the first state, monitors the reaction of the occupant, and determines whether to return the state of the device from the second state to the first state based on the reaction. An information processing apparatus comprising the control unit.

2. The information processing apparatus according to claim 1, wherein the reaction includes a reaction by voice of the occupant.

3. The information processing apparatus according to claim 1, wherein the control unit refers to a history of one or more operations performed by voice input when returning the state of the device from the second state to the first state.

4. The information processing apparatus according to claim 1, wherein when the control unit determines to return the state of the device from the second state to the first state, it outputs a confirmation request to the occupant, and after the confirmation is obtained, returns the state of the device from the second state to the first state.

5. A computer controls a specific function in response to an operation performed by voice input from a vehicle occupant; after the computer controls the specific function and as a result, the state of the device having the specific function transitions from a first state to a second state different from the first state, the computer monitors the reaction of the occupant; and the computer determines whether to return the state of the device from the second state to the first state based on the reaction A control method including.

Images