Information processing device and information processing method

Abstract

To enable a user to perform voice operations satisfactorily. The situation determination unit determines the situation. The state control unit controls the voice command acceptance state to match the determined situation. For example, the user is notified of the voice command that is being accepted by display or voice output. The user can utter a voice command without taking any user action, such as uttering a wake word to prevent misrecognition, thereby reducing the user's hassle and burden.

Description

[Technical Field]

[0001] The present technology relates to an information processing device and an information processing method, and more particularly to an information processing device and an information processing method suitable for application to voice-operated devices such as smart TVs and smart speakers. [Background technology]

[0002] Generally, in voice-operated devices such as smart TVs and smart speakers, accepting a voice command from the user requires user action such as speaking a specific wake word (sometimes called a "hot word" or "activation word") or pressing a specific button.

[0003] The reason why such user actions are necessary is to prevent users from misrecognizing voice commands when they converse in their daily lives by treating only the voice immediately after a user action as a voice command. However, forcing a user to take a user action every time they utter a voice command is bothersome and a heavy burden for the user.

[0004] For example, Patent Document 1 discloses a technology that starts accepting voice commands when it is determined that a user has looked at a predetermined object based on information about the user's gaze position on a display screen. Here, the user's act of looking at the predetermined object is an alternative to the above-mentioned act of uttering a specific wake word or pressing a specific button, which is also cumbersome and a significant burden for the user. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-055718 Summary of the Invention [Problem to be solved by the invention]

[0006] To reduce the inconvenience and burden on users, it is desirable to be able to issue voice commands without requiring user action.

[0007] An object of the present technology is to enable a user to perform voice operations satisfactorily. [Means for solving the problem]

[0008] The concept of this technology is: A control unit controls a process for determining a situation and controls the state of accepting voice commands appropriate for the determined situation. It is located in the information processing device.

[0009] In the present technology, a control unit controls a process of determining a situation. Then, the control unit controls a state of accepting a voice command appropriate for the determined situation. For example, when an input voice command is appropriate for the situation, the present technology may further include a command execution unit that executes the voice command.

[0010] For example, the control unit may determine the status of the application function, and the status may be the determined status of the application function. This automatically controls the voice command acceptance state to be appropriate for the status of the application function. In this case, for example, the application function may be the application function of a smart TV or a smart speaker.

[0011] Also, for example, a sensor unit may be further provided, and the situation may be the situation acquired by the sensor unit. As a result, the voice command acceptance state is automatically controlled to be appropriate for the situation acquired by the sensor unit. In this case, for example, the sensor unit may include a camera. Also, in this case, for example, the situation may be the user's situation.

[0012] Furthermore, for example, the device may further include a communication unit that communicates with an external device, and the situation may be a situation acquired by the external device, thereby automatically controlling the state of acceptance of a voice command to be suitable for the situation acquired by the external device.

[0013] In this way, this technology controls the voice command acceptance state to suit the determined situation, allowing the user to utter a voice command without taking any user action to prevent misrecognition, thereby reducing the user's inconvenience and burden.

[0014] In addition, the present technology may further include, for example, a command notification unit that notifies the user of a voice command appropriate for the situation, thereby enabling the user to appropriately utter a voice command appropriate for the situation. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a block diagram showing an overview of an example of an information processing system according to an embodiment; [Figure 2] FIG. 1 is a block diagram showing an example of the configuration of an information processing device included in a voice-operation compatible device. [Figure 3] FIG. 10 is a diagram illustrating a configuration example in which the voice-operated device is a smart TV or a smart speaker. [Figure 4] 10 is a flowchart illustrating an example of a processing procedure in a control unit of the information processing device. [Figure 5] FIG. 10 is a block diagram showing an example of the configuration of an information processing device corresponding to Case 1. [Figure 6] FIG. 10 is a diagram showing an example of a list of acceptable voice commands displayed on a display. [Figure 7] FIG. 10 is a block diagram showing an example of the configuration of an information processing device corresponding to Case 2. [Figure 8] FIG. 10 is a block diagram showing an example of the configuration of an information processing device corresponding to Case 3. [Figure 9]FIG. 10 is a block diagram showing an example of the configuration of an information processing device corresponding to Case 4. [Figure 10] 1 is a block diagram showing an example of a schematic configuration of a vehicle control system; [Figure 11] FIG. 2 is an explanatory diagram showing an example of the installation positions of an outside-vehicle information detection unit and an imaging unit. DETAILED DESCRIPTION OF THE INVENTION

[0016] The following describes modes for carrying out the invention (hereinafter referred to as "embodiments") in the following order: 1. Embodiment 2. Mobile application examples 3. Variations

[0017] <1. Embodiment> [Example of information processing system configuration] 1 shows an overview of an example of an information processing system 10 according to an embodiment. In this example, the information processing system 10 includes a voice-operated device 100 and also includes an external device 200, such as an IoT (Internet of Things) device, that can communicate with the voice-operated device 100.

[0018] The voice-operated device 100 is an electronic device that can be operated by voice commands uttered by the user 300, and specifically includes, for example, a smart TV, a smart speaker, and the like. Here, a smart TV is a smart version of a conventional TV. For example, a smart TV has features such as the ability to connect to the Internet to obtain various information and to connect to other devices via a network for inter-device collaboration. A smart speaker is a smart version of a conventional speaker, and is a speaker that can use an AI (Artificial Intelligence) assistant that supports interactive voice operation. Note that an AI assistant that supports interactive voice operation can also be used with a smart TV alone.

[0019] The external device 200 is, for example, a security camera, lighting equipment, or even a microwave oven, vacuum cleaner, air conditioner, washing machine, refrigerator, etc. Communication between the voice-operated device 100 and the external device 200 is performed wirelessly or via a wire. Known wireless methods include "Wi-Fi" and "Bluetooth." Note that "Wi-Fi" and "Bluetooth" are registered trademarks.

[0020] The voice-operated device 100 determines the situation, enters a state where it can accept a voice command appropriate for the determined situation, and executes the voice command input by the user 300 if the voice command is appropriate for the situation. In this case, the user 300 can utter the voice command without taking any user action to prevent erroneous recognition, such as uttering a wake word, thereby reducing the user's inconvenience and burden.

[0021] The voice-operated device 100 can, for example, determine the status of its own application function unit and enter a state where it accepts voice commands appropriate to the determined status. For example, if the voice-operated device 100 is a smart TV, the application function units include a TV reception function unit, an email function unit, an internet connection function unit, an information display function unit, etc. Also, for example, if the voice-operated device 100 is a smart speaker, the application function units include a voice output function unit, an email function unit, an internet connection function unit, an information display function unit, etc.

[0022] Furthermore, the voice-operated device 100 can enter a state of accepting voice commands suited to the situation (information about the user and the external environment) acquired by, for example, its own sensor unit. The sensor unit includes a temperature sensor, humidity sensor, human presence sensor, distance sensor, proximity sensor, etc., as well as a camera (image sensor) and microphone. For example, the user's situation can be acquired by a camera, microphone, human presence sensor, etc.

[0023] Furthermore, the voice-operation compatible device 100 can enter a state in which it accepts a voice command suited to the situation (information on the user or the external environment) acquired by the external device 200. The external device 200 may be a device for acquiring the situation, a device controlled by executing a voice command, or a device having both of these functions.

[0024] In the above description, the situation is exemplified by the situation of the application function unit, the situation acquired by the device's own sensor unit, and the situation acquired by an external device, but the situation is not limited to these and may be other situations. Furthermore, the voice-operated device 100 can be in a state where it accepts voice commands appropriate for each of these situations, as well as in a state where it accepts voice commands appropriate for a comprehensive situation that combines some or all of the situations.

[0025] Furthermore, when the voice-operated device 100 is ready to accept a voice command appropriate for the situation, it notifies the user 300 of which voice command it is ready to accept. In this case, it notifies the user by displaying a message on the display so that the user can visually confirm it, or by outputting a voice message from the speaker so that the user can audibly confirm it, or by both. By notifying the user of the voice command it is ready to accept in this way, the user can appropriately utter a voice command appropriate for the situation.

[0026] In addition, there may be cases where the user's intended voice command is not included in the voice commands that are notified and are in a state of being accepted. In such cases, the user can perform the desired voice operation by uttering the intended voice command after performing a user action such as uttering a wake word.

[0027] FIG. 2 shows a configuration example of an information processing device 110 included in the voice-operated device 100. The information processing device 110 has a control unit 111, a communication unit 112, a sensor unit 113, a microphone 114, a speaker 115, and a display 116. This example has the display 116, and assumes that the voice-operated device 100 is a smart TV, for example, as shown in FIG. 3(a). Note that, as shown in FIG. 3(b), if the voice-operated device 100 is a smart speaker, a configuration is conceivable in which the function of the display 116 is provided to an external TV receiver (or monitor or projector) 400. Note that there are also cases in which the smart speaker 100 itself has the function of the display 116 of a projector or the like.

[0028] Returning to FIG. 2 , communication unit 112 communicates with external device 200. Sensor unit 113 includes a camera, a microphone, and various other sensors. It is possible to use microphone 114 instead of the microphone included in sensor unit 113, or to use a microphone provided on an operating device such as a remote controller or a smartphone. For example, by pressing a specific button on a remote controller capable of operating voice-operated device 100 via infrared light, a voice command reception process can be initiated by a microphone on the remote controller or microphone 114, or by selecting a specific icon image displayed on the display unit of a smartphone. A configuration without a camera or microphone is also conceivable.

[0029] The control unit 111 controls the entire information processing device 110. The control unit 111 has a situation determination unit 117 and a voice recognition control unit 118. Situation information acquired by the external device 200 is provided to the situation determination unit 117 of the control unit 111 through the communication unit 112. Furthermore, situation information acquired by the sensor unit 113 is provided to the situation determination unit 117 of the control unit 111. The control unit 111 constantly keeps track of the situation of the application function unit of the voice-operation-compatible device 100, and provides the situation information to the situation determination unit 117.

[0030] The situation determination unit 117 determines the situation of the application function unit, the situation acquired by the sensor unit, and the situation acquired by the external device based on the situation information provided by each unit. The voice recognition control unit 118 selects a voice command suitable for the situation determined by the situation determination unit 117 and enters a state in which the voice command is accepted.

[0031] At this time, the voice recognition control unit 118 performs control to notify the user 300 of which voice command is currently being accepted. In this case, for example, the notification is displayed on the display 116 or output as voice from the speaker 115.

[0032] Furthermore, the voice recognition control unit 118 recognizes the utterance of a voice command input from the microphone 114, and if the voice command is appropriate for the situation, performs control to execute the voice command. In this case, control information is sent to the external device 200 via the communication unit 112 as necessary. Note that it is also possible to configure the information processing device 110 so that some of its processes, such as voice recognition processing, are performed by a cloud server (not shown).

[0033] 4 shows an example of a processing procedure in the control unit 111 of the information processing device 110. The control unit 111 periodically and repeatedly executes the processing of this flowchart. The control unit 111 starts the processing in step ST1.

[0034] Next, in step ST2, the control unit 111 acquires status information. In this case, the control unit 111 acquires at least one of status information from the application function unit, status information from the sensor unit 113, and status information from the external device 200.

[0035] Next, in step ST3, the control unit 111 determines the situation from the situation information. Next, in step ST4, the control unit 111 determines whether or not a voice command suitable for the situation exists based on the determined situation. If a voice command suitable for the situation exists, in step ST5, the control unit 111 enters a state in which a voice command suitable for the situation is accepted, and notifies the user by display or voice output.

[0036] Next, in step ST6, the control unit 111 determines whether or not a voice command has been uttered. When a voice command has been uttered, the control unit 111 determines in step ST7 whether the uttered voice command is appropriate for the situation, that is, whether or not the voice command is in an acceptable state.

[0037] If the uttered voice command is appropriate for the situation, the control unit 111 executes the voice command in step ST8. After the process of step ST8, the control unit 111 ends the process in step ST9.

[0038] Furthermore, if there is no voice command suitable for the situation in step ST4, if no voice command is uttered in step ST6, or if the voice command uttered in step ST7 is not suitable for the situation, the control unit 111 ends the process in step ST9. In these cases, the control unit 111 may also perform a process of notifying the user of a voice command suitable for the situation before ending the process in step ST9.

[0039] Next, we will explain specific cases. Here, we will explain the following four cases.

[0040] "Case 1" Case 1 is a case in which the voice-operated device 100 determines the situation based only on the information it holds. Case 1 is a case in which a change in the situation in the application function of the voice-operated device 100 is detected, and it is assumed that the user will execute the next action using a voice command. In Case 1, the information processing device 110 determines the situation of the application function unit, and enters a state in which it will accept a voice command appropriate to the determined situation.

[0041] Fig. 5 shows an example of the configuration of an information processing device 110 corresponding to Case 1, and the information processing device 110 has a control unit 111, a microphone 114, a speaker 115, and a display 116. In Fig. 5, parts corresponding to those in Fig. 2 are given the same reference numerals, and detailed description thereof will be omitted. In Case 1, the communication unit 112 and the sensor unit 113 in the information processing device 110 in Fig. 2 are not used.

[0042] In this case 1, the following use cases are possible: (1) When the arrival of a new email is detected (information obtained from the device's application function) and the user is expected to read the email body. (2) When it is detected that the time has reached 9 o'clock (information obtained from the device's application function), the TV program guide screen is displayed and the channel is changed.

[0043] In the case of use case (1), a voice command appropriate for the situation may be, for example, "Open new email." In the case of use case (2), a voice command appropriate for the situation may be, for example, "Show the program guide" → "Change to XX TV." Note that FIG. 6 shows an example of a list of acceptable voice commands displayed on the display 116 in the case of use case (1).

[0044] "Case 2" Case 2 is a case in which the voice-operated device 100 determines the situation using its own camera and various sensors. In Case 2, it is assumed that the user will perform the next action using a voice command when a change in the situation of the user or the external environment is detected from information from the camera and various sensors held by the voice-operated device 100. In Case 2, the information processing device 110 determines the situation acquired by the sensor unit 113 and enters a state in which it will accept a voice command appropriate to the determined situation.

[0045] 7 shows an example of the configuration of an information processing device 110 corresponding to Case 2, and the information processing device 110 has a control unit 111, a sensor unit 113, a microphone 114, a speaker 115, and a display 116. In Fig. 7, parts corresponding to those in Fig. 2 are given the same reference numerals, and detailed description thereof will be omitted. In Case 2, the communication unit 112 in the information processing device 110 in Fig. 2 is not used.

[0046] In this case 2, the following use cases are possible: (1) When it is assumed that the user will turn on the device after detecting that the user is sitting in front of the TV (based on camera information stored on the device itself). (2) When it is expected that the user will change the font size after detecting that the user has moved away from the device based on information from the distance sensor.

[0047] In the case of use case (1), a voice command appropriate to the situation might be "Turn on the TV," while in the case of use case (2), a voice command might be "Increase the text size."

[0048] "Case 3" Case 3 is a case in which the voice-operated device 100 determines the situation by communicating with an external IoT device or the like. Case 3 is a case in which it is assumed that the user will perform the next action by a voice command when a change in the situation of the user or the external environment is detected from information acquired from various IoT devices connected to the voice-operated device 100 via a network. In Case 3, the information processing device 110 determines the situation acquired by the external device 200 and enters a state in which it will accept a voice command appropriate to the determined situation.

[0049] Fig. 8 shows an example of the configuration of an information processing device 110 corresponding to Case 3, and the information processing device 110 has a control unit 111, a communication unit 112, a microphone 114, a speaker 115, and a display 116. In Fig. 8, parts corresponding to those in Fig. 2 are given the same reference numerals, and detailed description thereof will be omitted. In Case 3, the sensor unit 113 in the information processing device 110 in Fig. 2 is not used.

[0050] In this case 3, the following use cases are possible: (1) When someone visits the front door (information obtained from an IoT security camera), the image from this IoT security camera is expected to be displayed on a TV screen.

[0051] In the case of use case (1), a voice command appropriate to the situation might be something like "Show the front door camera."

[0052] In this case, the control unit 111 may perform processing such as outputting a notification to the display 116 based on information from an external IoT device or outputting a voice notification from the speaker 115, and may be configured to accept only voice commands related to the output notification. Similarly, in case 1, a notification based on a status change in the application function may be output, and only voice commands based on that notification may be accepted.

[0053] "Case 4" Case 4 is a case in which the voice-operated device 100 judges the situation using a combination of the information used in the above-mentioned cases 1 to 3. Case 4 is a case in which the voice-operated device 100 detects changes in the situation of the user or the external environment by integrating the functions of the voice-operated device 100 itself, information from cameras and various sensors held by the voice-operated device 100 itself, and information acquired from various IoT devices connected to the voice-operated device 100 via a network, and it is assumed that the user will then execute the next action using a voice command.

[0054] In this case 4, the information processing device 110 comprehensively judges the status of the application function unit, the status acquired by the sensor unit 113, and the status acquired by the external device 200, and enters a state where it will accept voice commands appropriate to the judged status.

[0055] 9 shows an example of the configuration of an information processing device 110 corresponding to Case 4, and the information processing device 110 has a control unit 111, a communication unit 112, a sensor unit 113, a microphone 114, a speaker 115, and a display 116. In this Fig. 9, parts corresponding to those in Fig. 2 are given the same reference numerals, and detailed description thereof will be omitted. In this Case 4, all parts in the information processing device 110 in Fig. 2 are used.

[0056] 1, the voice-operated device 100 automatically enters a state where it can accept a voice command appropriate for the determined situation. This allows the user 300 to utter a voice command without taking any user action, such as uttering a wake word to prevent erroneous recognition, thereby reducing the hassle and burden on the user 300.

[0057] 1, the voice-operated device 100 notifies the user 300 of the voice command appropriate for the situation, i.e., the voice command that can be accepted, by displaying or outputting voice, thereby enabling the user 300 to appropriately utter a voice command appropriate for the situation.

[0058] In each case, the control unit may control the information determination by the situation determination unit based on the user's prior settings, or may control the information determination based on information accumulated based on the user's past actions. For example, in case 1, if the user has previously scheduled a program to start at 9:00, the control unit may start voice reception processing at 9:00, or if it is confirmed based on sensor information that the user has sat down in front of the television at the time a program they regularly watch is about to start, the control unit may start voice reception processing. In addition, information such as the user's settings and past action history may be stored inside the voice-operated device 100, or may be recorded in an external recording device (not shown), such as a cloud server.

[0059] The effects described in this specification are merely examples and are not limiting, and additional effects may also be present.

[0060] < 2 .Application examples for mobile devices> The technology according to the present disclosure (the present technology) can be applied to various products. For example, the technology according to the present disclosure may be realized as a device mounted on any type of moving body, such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, personal mobility, an airplane, a drone, a ship, or a robot.

[0061] Figure 10 1 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile object control system to which the technology according to the present disclosure can be applied.

[0062] The vehicle control system 12000 comprises a plurality of electronic control units connected via a communication network 12001 . Figure 10 In the example shown in FIG. 1, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside-of-vehicle information detection unit 12030, an inside-vehicle information detection unit 12040, and an integrated control unit 12050. Also shown as functional components of the integrated control unit 12050 are a microcomputer 12051, an audio / video output unit 12052, and an in-vehicle network I / F (Interface) 12053.

[0063] The drivetrain control unit 12010 controls the operation of devices related to the drivetrain of the vehicle in accordance with various programs. For example, the drivetrain control unit 12010 functions as a control device for a drive force generating device for generating a drive force of the vehicle, such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, a braking device for generating a braking force of the vehicle, etc.

[0064] The body system control unit 12020 controls the operation of various devices equipped in the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window device, or various lamps such as headlamps, backup lamps, brake lamps, turn signals, and fog lamps. In this case, radio waves transmitted from a portable device that serves as a key or signals from various switches may be input to the body system control unit 12020. The body system control unit 12020 receives these radio waves or signals and controls the vehicle's door lock device, power window device, lamps, etc.

[0065] The outside-vehicle information detection unit 12030 detects information outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the outside-vehicle information detection unit 12030. The outside-vehicle information detection unit 12030 causes the imaging unit 12031 to capture images outside the vehicle and receives the captured images. The outside-vehicle information detection unit 12030 may perform object detection processing or distance detection processing for people, cars, obstacles, signs, characters on the road surface, etc., based on the received images.

[0066] The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal according to the amount of light received. The imaging unit 12031 can output the electrical signal as an image, or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0067] The in-vehicle information detection unit 12040 detects information inside the vehicle. For example, a driver state detection unit 12041 that detects the state of the driver is connected to the in-vehicle information detection unit 12040. The driver state detection unit 12041 includes, for example, a camera that captures an image of the driver, and the in-vehicle information detection unit 12040 may calculate the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 12041, or may determine whether the driver is dozing off.

[0068] The microcomputer 12051 can calculate control target values ​​for the driving force generating device, steering mechanism, or braking device based on the information inside and outside the vehicle acquired by the outside-vehicle information detection unit 12030 or the inside-vehicle information detection unit 12040, and output control commands to the drivetrain control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing the functions of an ADAS (Advanced Driver Assistance System), including avoiding or mitigating collisions between vehicles, following based on the distance between vehicles, maintaining vehicle speed, warning of vehicle collisions, or warning of vehicle lane departure.

[0069] In addition, the microcomputer 12051 can perform cooperative control for the purpose of autonomous driving, which allows the vehicle to travel autonomously without relying on driver operation, by controlling the driving force generating device, steering mechanism, braking device, etc. based on information about the surroundings of the vehicle obtained by the outside vehicle information detection unit 12030 or the inside vehicle information detection unit 12040.

[0070] Furthermore, the microcomputer 12051 can output a control command to the body system control unit 12030 based on the information about the outside of the vehicle acquired by the outside information detection unit 12030. For example, the microcomputer 12051 can control the headlamps according to the position of a preceding vehicle or an oncoming vehicle detected by the outside information detection unit 12030, and perform cooperative control for the purpose of preventing glare, such as switching from high beams to low beams.

[0071] The audio / video output unit 12052 transmits at least one of audio and video output signals to an output device capable of visually or audibly notifying information to passengers in the vehicle or to the outside of the vehicle. Figure 10 In this example, the output devices are exemplified by an audio speaker 12061, a display unit 12062, and an instrument panel 12063. The display unit 12062 may include, for example, at least one of an on-board display and a head-up display.

[0072] Figure 11 12 is a diagram showing an example of the installation position of the imaging unit 12031.

[0073] Figure 11 In the image capturing unit 1203, the image capturing unit 12031 includes image capturing units 12101, 12102, 12103, 12104, and 12105.

[0074] The imaging units 12101, 12102, 12103, 12104, and 12105 are provided, for example, at positions such as the front nose, side mirrors, rear bumper, back door, and the top of the windshield inside the vehicle cabin of the vehicle 12100. The imaging unit 12101 provided at the front nose and the imaging unit 12105 provided at the top of the windshield inside the vehicle cabin mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 provided at the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 provided at the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 provided at the top of the windshield inside the vehicle cabin is mainly used to detect preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, lanes, etc.

[0075] In addition, Figure 11 shows an example of the imaging ranges of the imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of the imaging unit 12101 provided on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of the imaging units 12102 and 12103 provided on the side mirrors, respectively, and imaging range 12114 indicates the imaging range of the imaging unit 12104 provided on the rear bumper or back door. For example, by overlaying the image data captured by the imaging units 12101 to 12104, a bird's-eye view image of the vehicle 12100 viewed from above can be obtained.

[0076] At least one of the imaging units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera made up of multiple imaging elements, or may be an imaging element having pixels for phase difference detection.

[0077] For example, the microcomputer 12051 can calculate the distance to each three-dimensional object within the imaging ranges 12111 to 12114 and the change in this distance over time (relative speed with respect to the vehicle 12100) based on the distance information obtained from the imaging units 12101 to 12104, thereby extracting as a preceding vehicle, in particular, the three-dimensional object that is the closest three-dimensional object on the path of the vehicle 12100 and traveling in approximately the same direction as the vehicle 12100 at a predetermined speed (for example, 0 km / h or higher). Furthermore, the microcomputer 12051 can set a vehicle-to-vehicle distance to be maintained in advance in front of the preceding vehicle, and perform automatic braking control (including follow-up stop control), automatic acceleration control (including follow-up start control), etc. In this way, cooperative control can be performed for the purpose of automatic driving, which runs autonomously without relying on driver operation.

[0078] For example, the microcomputer 12051 classifies and extracts three-dimensional object data regarding three-dimensional objects into two-wheeled vehicles, ordinary vehicles, large vehicles, pedestrians, utility poles, and other three-dimensional objects based on distance information obtained from the imaging units 12101 to 12104, and can use the data for automatic obstacle avoidance. For example, the microcomputer 12051 distinguishes obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. The microcomputer 12051 then determines the collision risk, which indicates the degree of risk of collision with each obstacle, and when the collision risk is equal to or greater than a set value and a collision is possible, the microcomputer 12051 can provide driving assistance for collision avoidance by outputting an alarm to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or avoidance steering via the drivetrain control unit 12010.

[0079] At least one of the image capturing units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether or not a pedestrian is present in the images captured by the image capturing units 12101 to 12104. The pedestrian recognition is performed, for example, by extracting feature points from the images captured by the image capturing units 12101 to 12104, which are infrared cameras, and then performing pattern matching on a series of feature points that indicate the outline of an object to determine whether or not the object is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the images captured by the image capturing units 12101 to 12104 and recognizes the pedestrian, the audio / image output unit 12052 controls the display unit 12062 to superimpose a rectangular outline on the recognized pedestrian for emphasis. The audio / image output unit 12052 may also control the display unit 12062 to display an icon or the like indicating the pedestrian at a desired position.

[0080] An example of a vehicle control system to which the technology according to the present disclosure can be applied has been described above. The technology according to the present disclosure can be applied to the integrated control unit 12050 and other components of the above-described configuration. By applying the technology according to the present disclosure to the integrated control unit 12050, when a user performs voice operations on the vehicle control system 12000, the user can utter a voice command without taking any user action, such as uttering a wake word to prevent erroneous recognition, thereby reducing the user's inconvenience and burden. Furthermore, for example, a car navigation system installed in a vehicle may exist as the external device 200, and the audio speaker 12061 and the display unit 12062 of the vehicle control system 12000 may be controlled based on situation determination processing by a control unit of the car navigation system.

[0081] <3. Modifications> Although the preferred embodiments of the present disclosure have been described in detail with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person skilled in the art of the present disclosure can conceive of various modified or altered examples within the scope of the technical ideas described in the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure.

[0082] The technology can also be configured as follows: (1) A control unit that controls the process of determining the situation and controls the state of accepting voice commands appropriate to the determined situation. Information processing device. (2) The control unit controls the determination of the status of the application function, The situation is the situation of the determined application function. The information processing device according to (1) above. (3) The above application functions are application functions of smart TVs or smart speakers. The information processing device according to (2) above. (4) Further comprising a sensor unit, The situation is acquired by the sensor unit. The information processing device according to any one of (1) to (3). (5) The sensor unit includes a camera. The information processing device according to (4) above. (6) The above situation is the user's situation. The information processing device according to (4) or (5). (7) A communication unit for communicating with an external device is further provided. The situation is acquired by the external device. The information processing device according to any one of (1) to (6). (8) The apparatus further includes a command execution unit that executes an input voice command when the input voice command is suitable for the situation. The information processing device according to any one of (1) to (7). (9) A command notification unit is further provided to notify the user of a voice command appropriate for the situation. The information processing device according to any one of (1) to (8). (10) Procedures for assessing the situation; and a procedure for controlling the state of accepting voice commands appropriate to the determined situation. Information processing methods. [Explanation of symbols]

[0083] 10. Information Processing Systems 100···Voice-operated devices 110 Information processing device 111 Control unit 112···Communications Department 113 Sensor section 114 Microphone 115···Speaker 116···Display 117···Situation Assessment Department 118···Voice recognition control unit 200...External device 300 users 400···Television receiver (TV)

Claims

1. The device includes a control unit that controls a process of determining a situation, a process of making a voice command suitable for the determined situation acceptable without being based on a user operation, and a process of notifying a user of the acceptable voice command before the user inputs the voice command. Information processing device.

2. The control unit controls the process of determining the situation based on a previous setting of the user or a behavior history of the user. The information processing device according to claim 1 .

3. The control unit further controls a process of making a voice command different from the voice command that has been made acceptable acceptable based on a user operation. The information processing device according to claim 1 .

4. The situation includes the situation of an application function unit of a voice-operated device including the information processing device. The information processing device according to claim 1 .

5. The above voice-operated devices include smart TVs and smart speakers. The information processing device according to claim 4 .

6. Further comprising a sensor unit, The situation includes the situation acquired by the sensor unit. The information processing device according to claim 1 .

7. The situation includes the situation of the user. The information processing device according to claim 6 .

8. Further comprising a communication unit for communicating with an external device; The situation is acquired by the external device. The information processing device according to claim 1 .

9. When the input voice command is the voice command that is determined to be acceptable, the control unit further controls a process of executing the voice command. The information processing device according to claim 1 .

10. How to assess the situation and a step of accepting a voice command suitable for the determined situation without relying on a user operation; and notifying the user of the voice command that has been determined to be acceptable before the user inputs the voice command. Information processing methods.

Images