Television, information processing method, and television operation method

Abstract

To enable users to perform voice commands effectively. [Solution] The situation determination unit determines the situation. The state control unit controls the system to accept voice commands appropriate to the determined situation. For example, the system notifies the user of which voice command is in the acceptance state through display or voice output. The user can speak voice commands without having to perform user actions such as uttering a wake word to prevent misrecognition, thus reducing hassle and burden.

Description

Technical Field

[0001] This technology relates to a television, an information processing method, and a television operation method.

Background Art

[0002] Generally, in voice operation-compatible devices such as smart TVs and smart speakers, in order to receive a voice command from a user, a user action such as uttering a specific wake word (sometimes referred to as a "hot word" or "activation word") or pressing a specific button is required.

[0003] The reason for such a user action is to prevent misrecognition of a voice command when the user is having a conversation in normal life by treating only the voice immediately after the user action as a voice command. However, forcing the user to perform a user action every time the user wants to utter a voice command is troublesome and a great burden for the user.

[0004] For example, Patent Document 1 discloses a technique for starting to receive a voice command when it is determined that the user has looked at a predetermined object based on information on the position of the user's line of sight on the display screen. Here, the act of the user looking at a predetermined object is a substitute for the act of uttering the specific wake word or pressing a specific button described above, and this is also troublesome and a great burden for the user.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] To reduce user inconvenience and burden, it is desirable to enable voice commands to be issued without requiring any user action.

[0007] The purpose of this technology is to enable users to perform voice control effectively. [Means for solving the problem]

[0008] The concept behind this technology is: The system includes a control unit that controls the process of determining the situation and controls the state to accept voice commands appropriate to the determined situation. It is located in the information processing unit.

[0009] In this technology, the control unit controls the process of determining the situation. The control unit then controls the system to accept voice commands appropriate to the determined situation. For example, the system may further include a command execution unit that executes a voice command when the input voice command is appropriate for the situation.

[0010] For example, the control unit may perform control to determine the status of an application function, and that status may be the status of the determined application function. This automatically controls the system to accept voice commands appropriate to the status of the application function. In this case, for example, the application function may be the application function of a smart TV or smart speaker.

[0011] Furthermore, for example, the system may include a sensor unit, where the "situation" is the situation acquired by the sensor unit. This allows the system to automatically control the state to accept voice commands appropriate to the situation acquired by the sensor unit. In this case, for example, the sensor unit may include a camera. Also, in this case, for example, the "situation" may be the user's situation.

[0012] Furthermore, for example, the system may be further equipped with a communication unit that communicates with an external device, and the status may be such that it is the status acquired by the external device. This allows the system to be automatically controlled to accept voice commands in a state appropriate to the status acquired by the external device.

[0013] In this technology, the system controls the voice command acceptance state to match the determined situation. Therefore, users can utter voice commands without having to take any user actions to prevent misrecognition, thereby reducing user inconvenience and burden.

[0014] Furthermore, this technology may include, for example, a command notification unit that notifies the user of a voice command appropriate to the situation. This would enable the user to appropriately utter a voice command appropriate to the situation. [Brief explanation of the drawing]

[0015] [Figure 1] This is a block diagram showing an overall overview of an example of an information processing system as an embodiment. [Figure 2] This is a block diagram showing an example configuration of an information processing device included in a voice-operated device. [Figure 3] This diagram illustrates an example configuration when the voice-controlled device is a smart TV or smart speaker. [Figure 4] This flowchart shows an example of a processing procedure in the control unit of an information processing device. [Figure 5] This is a block diagram showing an example configuration of an information processing device corresponding to Case 1. [Figure 6] This figure shows an example of a list of acceptable voice commands displayed on the screen. [Figure 7] This is a block diagram showing an example configuration of an information processing device corresponding to Case 2. [Figure 8] This is a block diagram showing an example configuration of an information processing device corresponding to Case 3. [Figure 9]It is a block diagram showing a configuration example of an information processing apparatus corresponding to Case 4. [Figure 10] It is a block diagram showing an example of a schematic configuration of a vehicle control system. [Figure 11] It is an explanatory diagram showing an example of an installation position of an external information detection unit and an imaging unit.

Embodiments for Carrying Out the Invention

[0016] Hereinafter, embodiments for carrying out the invention (hereinafter referred to as "embodiments") will be described. The description will be made in the following order. 1. Embodiments 2. Application Examples to Mobile Objects 3. Modification Examples

[0017] <1. Embodiments> [Configuration Example of Information Processing System] FIG. 1 shows an overall overview of an example of an information processing system 10 as an embodiment. In this example, the information processing system 10 has, in addition to a voice operation compatible device 100, an external device 200 capable of communicating with the voice operation compatible device 100, for example, an IoT (Internet of Things) device.

[0018] The voice operation compatible device 100 is an electronic device that can be operated by a voice command uttered by the user 300. Specifically, for example, a smart TV, a smart speaker, etc. are applicable. Here, the smart TV is a smart version of a conventional TV. For example, the smart TV has features such as being able to connect to the Internet to acquire various information and being able to connect to other devices via a network for device-to-device cooperation. Also, the smart speaker is a smart version of a conventional speaker and is a speaker that can utilize an AI (Artificial Intelligence) assistant corresponding to interactive voice operation. Note that a smart TV alone can also utilize an AI assistant corresponding to interactive voice operation.

[0019] External devices 200 include, for example, security cameras, lighting equipment, and even microwave ovens, vacuum cleaners, air conditioners, washing machines, and refrigerators. Communication between the voice-controlled device 100 and the external devices 200 is conducted wirelessly or via wired connection. Known wireless methods include "Wi-Fi" and "Bluetooth." Note that "Wi-Fi" and "Bluetooth" are registered trademarks.

[0020] The voice-operated device 100 assesses the situation and enters a state where it can accept voice commands appropriate to that situation. If the voice command entered by the user 300 is appropriate to that situation, the device executes that voice command. In this case, the user 300 can utter a voice command without having to perform any user actions to prevent misrecognition, such as uttering a wake word, thus reducing inconvenience and burden.

[0021] The voice-controlled device 100 can, for example, determine the status of its application function unit and enter a state where it can accept voice commands appropriate to that determined status. For example, if the voice-controlled device 100 is a smart TV, the application function unit may include a TV reception function unit, an email function unit, an internet connection function unit, an information display function unit, etc. Also, for example, if the voice-controlled device 100 is a smart speaker, it may include an audio output function unit, an email function unit, an internet connection function unit, an information display function unit, etc.

[0022] Furthermore, the voice-operated device 100 can, for example, enter a state where it is ready to accept voice commands appropriate to the situation (information about the user and the external environment) acquired by its sensor unit. The sensor unit includes temperature sensors, humidity sensors, motion sensors, distance sensors, proximity sensors, etc., as well as cameras (image sensors) and microphones. For example, the camera, microphone, motion sensor, etc., can be used to acquire information about the user's situation.

[0023] Furthermore, the voice-operated device 100 can, for example, enter a state where it is ready to receive voice commands appropriate to the situation (user and external environment information) acquired by the external device 200. The external device 200 may include devices for acquiring the situation, devices controlled by the execution of voice commands, or devices that have both functions.

[0024] In the above, the situations mentioned were the status of the application function unit, the status acquired by the unit's own sensors, and the status acquired by external devices. However, the situations are not limited to these and may include others. In addition, the voice-operated device 100 can be in a state to accept voice commands appropriate to each of these individual situations, or it can be in a state to accept voice commands appropriate to a combined situation that includes some or all of these situations.

[0025] Furthermore, when the voice-operated device 100 becomes ready to accept a voice command appropriate to the situation, it notifies the user 300 of which voice command is currently being accepted. In this case, the notification is displayed on the screen so that the user can visually confirm it, or it is output as sound from the speaker so that the user can audibly confirm it, or both are used. By notifying the user of which voice command is currently being accepted in this way, the user can appropriately utter a voice command that is appropriate to the situation.

[0026] It should be noted that there may be cases where the voice command the user intends to use is not among the voice commands that are notified and in an available state. In such cases, the user can perform a user action, such as uttering a wake word, and then utter the intended voice command to perform the desired voice operation.

[0027] Figure 2 shows an example configuration of an information processing device 110 included in a voice-operated device 100. The information processing device 110 includes a control unit 111, a communication unit 112, a sensor unit 113, a microphone 114, a speaker 115, and a display 116. This example includes a display 116 and assumes that the voice-operated device 100 is a smart television, as shown in Figure 3(a). As shown in Figure 3(b), if the voice-operated device 100 is assumed to be a smart speaker, a configuration in which the function of the display 116 is provided by an external television receiver (or monitor or projector) 400 is conceivable. In some cases, the smart speaker 100 itself may have the function of a display 116 such as a projector.

[0028] Returning to Figure 2, the communication unit 112 communicates with the external device 200. The sensor unit 113 includes a camera, microphone, and various other sensors. It is also possible to use a microphone 114 instead of the microphone included in this sensor unit 113, or to use a microphone provided on an operating device such as a remote controller or smartphone. For example, by pressing a specific button on a remote controller that can operate the voice control device 100 using infrared light, the process of receiving voice commands from the microphone on the remote controller or microphone 114 can be started, or by selecting a specific icon image displayed on the smartphone's display screen, the same process can be started. Furthermore, a configuration without a camera or microphone is also conceivable.

[0029] The control unit 111 controls the entire information processing device 110. This control unit 111 includes a status determination unit 117 and a voice recognition control unit 118. Status information acquired by the external device 200 is provided to the status determination unit 117 of the control unit 111 via the communication unit 112. In addition, status information acquired by the sensor unit 113 is also provided to the status determination unit 117 of the control unit 111. Furthermore, the control unit 111 constantly monitors the status of the application function unit of the voice-operated device 100, and this status information is provided to the status determination unit 117.

[0030] The status determination unit 117 determines the status of the application function unit, the status acquired by the sensor unit, and the status acquired by external devices based on the status information provided by each unit. The voice recognition control unit 118 selects a voice command appropriate to the status determined by the status determination unit 117 and sets the system to accept that voice command.

[0031] At this time, the speech recognition control unit 118 controls the system to notify the user 300 of which voice command is currently being accepted. In this case, for example, it may display the information on the display 116 or output audio from the speaker 115.

[0032] Furthermore, the speech recognition control unit 118 recognizes the utterance of a voice command input from the microphone 114, and controls the system to execute the voice command if it is appropriate for the situation. In this case, control information is sent to the external device 200 via the communication unit 112 as needed. It should be noted that some of the processing of the information processing device 110, such as speech recognition processing, may be configured to be performed on a cloud server (not shown).

[0033] The flowchart in Figure 4 shows an example of the processing procedure in the control unit 111 of the information processing device 110. The control unit 111 periodically repeats the processing shown in this flowchart. In step ST1, the control unit 111 starts processing.

[0034] Next, in step ST2, the control unit 111 acquires status information. In this case, the control unit 111 acquires at least one of the status information from the application function unit, the status information from the sensor unit 113, and the status information from the external device 200.

[0035] Next, in step ST3, the control unit 111 determines the situation from the situation information. Next, in step ST4, the control unit 111 determines whether there is a voice command suitable for the determined situation. If there is a voice command suitable for the situation, in step ST5, the control unit 111 sets the system to accept the voice command suitable for the situation and notifies the user by display or voice output.

[0036] Next, in step ST6, the control unit 111 determines whether or not a voice command has been spoken. If a voice command has been spoken, in step ST7, the control unit 111 determines whether the spoken voice command is appropriate for the situation, that is, whether or not it is a voice command that is in an accepted state.

[0037] If the spoken voice command is appropriate for the situation, the control unit 111 executes the voice command in step ST8. Then, after processing in step ST8, the control unit 111 terminates processing in step ST9.

[0038] Furthermore, if no voice command suitable for the situation exists in step ST4, if no voice command is spoken in step ST6, or if the voice command spoken in step ST7 is not suitable for the situation, the control unit 111 terminates processing in step ST9. In these cases, the control unit 111 may also perform a process to notify the user of a voice command suitable for the situation before terminating processing in step ST9.

[0039] Next, we will explain specific cases. Here, we will explain the following four cases.

[0040] "Case 1" Case 1 is a case in which the voice-operated device 100 makes a situational judgment based solely on the information it holds. In Case 1, it is assumed that the voice-operated device 100 detects a change in the status of its application function and that the user performs the next action by voice command. In Case 1, the information processing device 110 determines the status of the application function unit and enters a state where it can accept voice commands appropriate to the determined situation.

[0041] Figure 5 shows an example configuration of the information processing device 110 corresponding to this Case 1. The information processing device 110 includes a control unit 111, a microphone 114, a speaker 115, and a display 116. In Figure 5, parts corresponding to those in Figure 2 are denoted by the same reference numerals, and their detailed descriptions are omitted. In this Case 1, the communication unit 112 and sensor unit 113 of the information processing device 110 in Figure 2 are not used.

[0042] In this Case 1, the following use cases are possible. (1) When it is detected that a new email has arrived (information obtained from the device's application function) and it is expected that the user will read the email body. (2) When it is expected that the time will be detected as 9 o'clock (information obtained from the device's application function), the program guide screen will be displayed and the channel will be changed.

[0043] In use case (1), an appropriate voice command might be, "Open new emails." In use case (2), an appropriate command might be, "Show the TV guide" → "Switch to XX TV." Figure 6 shows an example of the list of acceptable voice commands displayed on display 116 in use case (1).

[0044] Case 2 Case 2 is a case in which the voice-operated device 100 makes a situation determination using its own camera and various sensors. In Case 2, when the voice-operated device 100 detects a change in the user's or the external environment's situation from the information of its camera and various sensors, it is assumed that the user will perform the next action by voice command. In Case 2, the information processing device 110 determines the situation acquired by the sensor unit 113 and enters a state where it can accept voice commands appropriate to the determined situation.

[0045] Figure 7 shows an example configuration of the information processing device 110 corresponding to this case 2. The information processing device 110 includes a control unit 111, a sensor unit 113, a microphone 114, a speaker 115, and a display 116. In Figure 7, parts corresponding to those in Figure 2 are denoted by the same reference numerals, and their detailed descriptions are omitted. In this case 2, the communication unit 112 in the information processing device 110 of Figure 2 is not used.

[0046] In this Case 2, the following use cases are possible. (1) When it is detected that the user is sitting in front of the TV (information from the camera held by the device itself), and it is expected that the user will turn on the power of the device. (2) When the distance sensor detects that the user has moved away from the device, and it is anticipated that the user will change the font size.

[0047] In use case (1), an appropriate voice command might be, "Turn on the TV." In use case (2), an appropriate command might be, "Increase the font size."

[0048] Case 3 Case 3 is a case in which the voice-operated device 100 communicates with external IoT devices to determine the situation. In this case 3, it is assumed that when the voice-operated device 100 detects a change in the user's or external environment's status from information acquired from various IoT devices connected to the network, the user will perform the next action by voice command. In this case 3, the information processing device 110 determines the situation acquired by the external device 200 and enters a state where it can accept voice commands appropriate to the determined situation.

[0049] Figure 8 shows an example configuration of the information processing device 110 corresponding to this case 3. The information processing device 110 includes a control unit 111, a communication unit 112, a microphone 114, a speaker 115, and a display 116. In Figure 8, parts corresponding to those in Figure 2 are denoted by the same reference numerals, and their detailed descriptions are omitted. In this case 3, the sensor unit 113 in the information processing device 110 of Figure 2 is not used.

[0050] In this Case 3, the following use cases are possible. (1) When a person visits the entrance (information obtained from the IoT security camera), it is expected that the video from this IoT security camera will be displayed on the TV screen.

[0051] In the use case (1), an appropriate voice command for the situation might be, "Show the front door camera."

[0052] In this case, the control unit 111 may perform processing such as outputting a notification to the display 116 or outputting an audio notification from the speaker 115 based on information from an external IoT device, and may accept only audio commands related to those output notifications. Similarly, in Case 1, notifications based on changes in the application function may be output, and only audio commands based on those notifications may be accepted.

[0053] Case 4 Case 4 is a case in which the voice-controlled device 100 makes a situational judgment by utilizing a combination of information used in Cases 1 to 3 described above. In Case 4, it is assumed that changes in the user's or external environment's situation are detected by integrating the functions of the voice-controlled device 100 itself, information from the camera and various sensors held by the voice-controlled device 100 itself, and information acquired from various IoT devices connected to the voice-controlled device 100 via the network, and that the user takes the next action by voice command.

[0054] In this case 4, the information processing device 110 comprehensively determines the status of the application function unit, the status acquired by the sensor unit 113, and the status acquired by the external device 200, and enters a state to accept voice commands appropriate to the determined status.

[0055] Figure 9 shows an example configuration of the information processing device 110 corresponding to this case 4. The information processing device 110 includes a control unit 111, a communication unit 112, a sensor unit 113, a microphone 114, a speaker 115, and a display 116. In Figure 9, parts corresponding to those in Figure 2 are denoted by the same reference numerals, and their detailed descriptions are omitted. In this case 4, all parts of the information processing device 110 in Figure 2 are used.

[0056] As explained above, in the information processing system 10 shown in Figure 1, the voice-operated device 100 automatically enters a state where it can accept voice commands appropriate to the determined situation. Therefore, the user 300 can issue voice commands without having to perform any user actions such as uttering a wake word to prevent misrecognition, thereby reducing the hassle and burden on the user 300.

[0057] Furthermore, in the information processing system 10 shown in Figure 1, the voice-operated device 100 notifies the user 300, either by display or voice output, of what voice commands are appropriate for the situation, that is, what voice commands are acceptable. As a result, the user 300 can appropriately utter voice commands that are appropriate for the situation.

[0058] In each case, the control unit may control the information judgment made by the situation judgment unit based on the user's prior settings, or it may control it based on information accumulated based on the user's actions up to that point. For example, in Case 1, if the user has made a reservation to watch a program that starts at 9:00, the system may start the voice reception process at 9:00, or, based on sensor information, it may start the voice reception when it is confirmed that the user is sitting in front of the television at the time when a program they regularly watch is about to start. Furthermore, information such as the user's settings and past behavior history may be stored inside the voice-operated device 100, or it may be recorded in an external recording device, such as a cloud server (not shown).

[0059] Furthermore, the effects described herein are merely illustrative and not limiting, and additional effects may also exist.

[0060] <2. Examples of applications to mobile devices> The technology disclosed herein (the Technology) can be applied to a variety of products. For example, the Technology disclosed herein may be implemented as a device mounted on any type of mobile vehicle, such as an automobile, electric vehicle, hybrid electric vehicle, motorcycle, bicycle, personal mobility device, airplane, drone, ship, or robot.

[0061] Figure 10 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile control system to which the technology described herein may be applied.

[0062] The vehicle control system 12000 comprises multiple electronic control units connected via a communication network 12001. In the example shown in Figure 10, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an external information detection unit 12030, an internal information detection unit 12040, and an integrated control unit 12050. The functional configuration of the integrated control unit 12050 is shown in the figure, which includes a microcomputer 12051, an audio / image output unit 12052, and an in-vehicle network interface 12053.

[0063] The drivetrain control unit 12010 controls the operation of devices related to the vehicle's drivetrain according to various programs. For example, the drivetrain control unit 12010 functions as a control device for a drivetrain generating device that generates driving force for the vehicle, such as an internal combustion engine or a drive motor; a drivetrain transmission mechanism that transmits driving force to the wheels; a steering mechanism that adjusts the steering angle of the vehicle; and a braking device that generates braking force for the vehicle.

[0064] The body system control unit 12020 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window system, or various lamps such as headlights, reverse lights, brake lights, turn signals, or fog lights. In this case, the body system control unit 12020 may receive radio waves transmitted from a portable device that replaces a key or signals from various switches. The body system control unit 12020 receives these radio waves or signals and controls the vehicle's door lock system, power window system, lamps, etc.

[0065] The external information detection unit 12030 detects information from outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the external information detection unit 12030. The external information detection unit 12030 causes the imaging unit 12031 to capture images of the outside of the vehicle and receives the captured images. Based on the received images, the external information detection unit 12030 may perform object detection processing such as detecting people, cars, obstacles, signs, or characters on the road surface, or distance detection processing.

[0066] The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal corresponding to the amount of light received. The imaging unit 12031 can output the electrical signal as an image or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0067] The in-vehicle information detection unit 12040 detects information inside the vehicle. The in-vehicle information detection unit 12040 is connected to, for example, a driver status detection unit 12041 that detects the driver's state. The driver status detection unit 12041 includes, for example, a camera that images the driver, and the in-vehicle information detection unit 12040 may calculate the driver's level of fatigue or concentration, or determine whether the driver is drowsy, based on the detection information input from the driver status detection unit 12041.

[0068] The microcomputer 12051 can calculate control target values ​​for the drive force generator, steering mechanism, or braking system based on information from inside and outside the vehicle acquired by the external information detection unit 12030 or the internal information detection unit 12040, and output control commands to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing ADAS (Advanced Driver Assistance System) functions, including collision avoidance or impact mitigation, following based on distance between vehicles, maintaining vehicle speed, vehicle collision warning, or vehicle lane departure warning.

[0069] Furthermore, the microcomputer 12051 can perform cooperative control for purposes such as autonomous driving, where the vehicle drives autonomously without driver intervention, by controlling the drive force generating device, steering mechanism, or braking device, etc., based on information about the vehicle's surroundings acquired by the external information detection unit 12030 or the internal information detection unit 12040.

[0070] Furthermore, the microcomputer 12051 can output control commands to the body system control unit 12030 based on external information acquired by the external information detection unit 12030. For example, the microcomputer 12051 can control the headlights according to the position of a preceding or oncoming vehicle detected by the external information detection unit 12030, and perform coordinated control aimed at reducing glare, such as switching from high beams to low beams.

[0071] The audio-image output unit 12052 transmits at least one of audio and image output signals to an output device capable of visually or audibly notifying information to the vehicle's occupants or to those outside the vehicle. In the example in Figure 10, the output devices are exemplified as an audio speaker 12061, a display unit 12062, and an instrument panel 12063. The display unit 12062 may include, for example, at least one of an onboard display and a head-up display.

[0072] Figure 11 shows an example of the installation position of the imaging unit 12031.

[0073] In Figure 11, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.

[0074] The imaging units 12101, 12102, 12103, 12104, and 12105 are installed, for example, on the front nose, side mirrors, rear bumper, back door, and the upper part of the windshield inside the vehicle 12100. The imaging unit 12101 installed on the front nose and the imaging unit 12105 installed on the upper part of the windshield inside the vehicle mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 installed on the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 installed on the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 installed on the upper part of the windshield inside the vehicle is mainly used for detecting preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, or lanes.

[0075] Figure 11 shows an example of the imaging range of imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of imaging unit 12101 located on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of imaging units 12102 and 12103 located on the side mirrors, respectively, and imaging range 12114 indicates the imaging range of imaging unit 12104 located on the rear bumper or back door. For example, by superimposing the image data captured by imaging units 12101 to 12104, an overhead view image of the vehicle 12100 can be obtained.

[0076] At least one of the imaging units 12101 to 12104 may have a function for acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera consisting of multiple image sensors, or an image sensor having pixels for phase difference detection.

[0077] For example, the microcomputer 12051, based on distance information obtained from imaging units 12101 to 12104, can determine the distance to each object within the imaging range 12111 to 12114 and the temporal change of this distance (relative speed to vehicle 12100). In particular, it can extract the nearest object on the vehicle 12100's path that is traveling in approximately the same direction as vehicle 12100 at a predetermined speed (e.g., 0 km / h or more) as the preceding vehicle. Furthermore, the microcomputer 12051 can set a predetermined distance to be maintained before the preceding vehicle and perform automatic braking control (including follow-and-stop control) and automatic acceleration control (including follow-and-start control), etc. In this way, cooperative control aimed at autonomous driving, where the vehicle drives autonomously without driver intervention, can be performed.

[0078] For example, the microcomputer 12051 can use distance information obtained from imaging units 12101 to 12104 to classify and extract three-dimensional object data related to three-dimensional objects, such as motorcycles, passenger cars, heavy vehicles, pedestrians, utility poles, and other three-dimensional objects, and use this data for automatic obstacle avoidance. For example, the microcomputer 12051 identifies obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. The microcomputer 12051 then determines the collision risk, which indicates the degree of risk of collision with each obstacle. If the collision risk is above a set value and there is a possibility of collision, the microcomputer 12051 can provide driving assistance to avoid collisions by outputting a warning to the driver via the audio speaker 12061 or display unit 12062, or by performing forced deceleration or evasive steering via the drive system control unit 12010.

[0079] At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared light. For example, the microcomputer 12051 can recognize pedestrians by determining whether or not pedestrians are present in the images captured by the imaging units 12101 to 12104. Such pedestrian recognition is performed, for example, by a procedure to extract feature points from the images captured by the imaging units 12101 to 12104 as infrared cameras, and a procedure to perform pattern matching on a series of feature points that indicate the contour of an object to determine whether or not it is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the images captured by the imaging units 12101 to 12104 and recognizes a pedestrian, the audio-image output unit 12052 controls the display unit 12062 to superimpose a rectangular contour line for emphasis on the recognized pedestrian. The audio-image output unit 12052 may also control the display unit 12062 to display an icon indicating a pedestrian at a desired position.

[0080] The above describes an example of a vehicle control system to which the technology described herein may be applied. The technology described herein can be applied to the integrated control unit 12050, etc., among the configurations described above. By applying the technology described herein to the integrated control unit 12050, when a user performs voice operation on the vehicle control system 12000, the user can utter voice commands without performing user actions such as uttering a wake word to prevent misrecognition, thereby reducing inconvenience and burden on the user. Furthermore, for example, a car navigation system installed in the vehicle may exist as an external device 200, or the audio speaker 12061 and display unit 12062 of the vehicle control system 12000 may be controlled based on the status judgment processing of the control unit of the car navigation system.

[0081] <3. Variant> While preferred embodiments of this disclosure have been described in detail with reference to the attached drawings, the technical scope of this disclosure is not limited to such examples. It is clear to any person with ordinary skill in the art of this disclosure that various modifications or alterations may be conceived within the scope of the technical idea set forth in the claims, and these too are understood to fall within the technical scope of this disclosure.

[0082] Furthermore, the technology can also take the following configurations. (1) The system includes a control unit that controls the process of determining the situation and controls the state to accept voice commands appropriate to the determined situation. Information processing device. (2) The control unit performs control to determine the status of the application function, The above situation describes the status of the application functionality as determined above. The information processing device described in (1) above. (3) The above application functions are application functions of a smart TV or smart speaker. The information processing device described in (2) above. (4) Further equipped with a sensor unit, The above situation is the situation acquired by the sensor unit mentioned above. An information processing device according to any one of (1) to (3) above. (5) The sensor unit includes a camera The information processing device described in (4) above. (6) The above situation is the user's situation. The information processing device described in (4) or (5) above. (7) Further comprising a communication unit for communicating with external devices, The above situation is the situation as captured by the external device mentioned above. An information processing device according to any one of (1) to (6) above. (8) When the input voice command is a voice command suitable for the above situation, the system further comprises a command execution unit that executes the voice command. An information processing device according to any one of (1) to (7) above. (9) The system further comprises a command notification unit that notifies the user of a voice command appropriate to the above situation. An information processing device according to any of (1) to (8) above. (10) Procedures for assessing the situation, The procedure includes controlling the state to accept voice commands in a manner appropriate to the situation determined above. Information processing methods. [Explanation of Symbols]

[0083] 10. Information Processing Systems 100...Voice-controlled devices 110... Information Processing Device 111... Control Unit 112... Communications Department 113...Sensor section 114... Microphone 115...Speaker 116...Display 117... Situation Assessment Department 118.. Voice Recognition Control Unit 200...External device 300 users 400...Television receiver (TV)

Claims

1. The system includes a circuit that uses situational information to determine the situation, prepares the system to accept voice commands appropriate to the determined situation, and notifies the user of the appropriate voice command visually through the display and audibly through the speaker. tv set.

2. The circuit acquires the status information from an external device. The television according to claim 1.

3. The circuit automatically makes the system capable of accepting voice commands appropriate only to the determined situation, without any user action on the television. The television according to claim 1.

4. The aforementioned TV user action is a button press operation. The television according to claim 3.

5. The aforementioned TV user action is the utterance of a wake expression for receiving a voice command. The television according to claim 3.

6. The circuit determines the status of the application function based on the status information of the application function, The determined circumstances include the determined circumstances of the application function. The television according to claim 1.

7. The aforementioned application functions include application functions for smart TVs or smart speakers. The television according to claim 6.

8. The receiving device uses the situation information to determine the situation, The receiving device is set to a state where it can accept voice commands appropriate to the determined situation. The television notifies the user of voice commands appropriate to the determined situation, visually through the display and audibly through the speaker. Information processing methods.

9. The aforementioned determination includes the television acquiring the status information from an external device. The information processing method according to claim 8.

10. The receiving device automatically becomes capable of receiving voice commands appropriate only to the determined situation, without any action from the television user. The information processing method according to claim 8.

11. The aforementioned TV user action is a button press operation. The information processing method according to claim 10.

12. The aforementioned TV user action is the utterance of a wake expression for receiving a voice command. The information processing method according to claim 10.

13. The aforementioned determination includes the receiving device determining the status of the application function based on the status information of the application function, The determined circumstances include the determined circumstances of the application function. The information processing method according to claim 8.

14. The aforementioned application functions include application functions for smart TVs or smart speakers. The information processing method according to claim 13.

15. Obtain status information from external devices, The situation is assessed using the acquired situational information. The system visually and audibly notifies the TV user of voice commands appropriate to the determined situation, and automatically makes the system ready to accept voice commands appropriate only to the determined situation without any action from the TV user. The visual notification to the television user is displayed on the television in a visually recognizable manner, and the auditory notification is made through the speaker in an auditory recognizable manner. The aforementioned TV user action is a button press operation or the utterance of a wake expression for receiving a voice command. How to operate a TV.

16. The determined situation includes the determined situation of the application function determined based on the status information of the application function. The television operation method according to claim 15.

17. The aforementioned application functions include application functions for smart TVs or smart speakers. The television operation method according to claim 16.

18. If the voice command is appropriate for the situation, the voice command will be executed. The television operation method according to claim 15.

19. The aforementioned external device is an IoT (Internet of Things) device. The television operation method according to claim 15.

20. We will adopt an interactive AI (Artificial Intelligence) assistant. The television operation method according to claim 15.

Images