Voice command reception device, voice command reception method and program

Abstract

To appropriately perform an operation by voice command.SOLUTION: A voice command reception device includes: a voice command reception unit that accepts a voice command; a detection unit that detects a direction of a face of a person uttering the voice command; and an execution control unit that executes a function for the accepted voice command when the voice command reception unit accepts the voice command. If the detection unit determines that the direction of the person's face faces the direction of a microphone from which utterance sound of the voice command is acquired, the voice command reception unit accepts the voice command with a recognition rate of the voice command acquired by the voice command reception unit equal to or higher than a first threshold value, and if the detection unit determines that the direction of the person's face does not face the microphone, the voice command reception unit accepts the voice command with the recognition rate of the voice command acquired by the voice command reception unit equal to or higher than a second threshold value lower than the first threshold value.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present invention relates to a voice command reception device, a voice command reception method, and a program.

Background Art

[0002] Devices that perform operations using voice commands are diversifying. For example, in a vehicle recording device, a so-called drive recorder, in addition to shock detection by an acceleration sensor, there are also those that perform event recording using voice commands (for example, Non-Patent Document 1). Event recording using voice commands can perform event recording safely without the need for operations on a touch panel or the like during driving, such as when recording an accident when one is not the party involved in the accident. Patent Document 1 discloses a drive recorder that performs event recording by giving an instruction by voice with respect to event detection by acceleration.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Non-Patent Documents

[0004]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] The voice commands used to instruct the dashcam to record events are pre-configured to accept commands such as "start recording." To prevent false detection by other voices, voice commands are required to consist of a certain number of syllables. For example, "start recording" has 6 syllables. Therefore, in order to accurately recognize the voice command, the speaker often faces the direction of the microphone that inputs the voice command, such as the direction of the dashcam. Since typical dashcams are installed in front of the vehicle from the perspective of the passenger who is speaking, voice commands entered while facing the direction of travel (the front of the vehicle) are properly recognized.

[0006] However, if a voice command is spoken in a situation where it cannot be properly recognized, the recognition rate of the voice command will be low, and there may be cases where the voice command instruction is not accepted. In such cases, voice commands for operations requiring urgency or immediate action, such as voice commands for recording events on a dashcam, may be delayed due to the need to repeat the voice command. Situations where a voice command cannot be properly recognized may occur, for example, when the person speaking the voice command is not facing the microphone that is capturing the voice of the voice command.

[0007] The present invention aims to provide a voice command receiving device, a voice command receiving method, and a program that can appropriately perform operations using voice commands. [Means for solving the problem]

[0008] The voice command receiving device of the present invention comprises a voice command receiving unit that receives voice commands, a detection unit that detects the orientation of the face of a person speaking the voice command, and an execution control unit that, when the voice command receiving unit receives a voice command, causes the voice command receiving unit to execute a function corresponding to the received voice command. The voice command receiving unit accepts the voice command if the detection unit determines that the orientation of the person's face is directed toward the direction of a microphone that acquires the voice of the voice command, and accepts the voice command if the recognition rate of the voice command acquired by the voice command receiving unit is at or above a first threshold, which is lower than the first threshold, when the detection unit determines that the orientation of the person's face is directed toward a direction other than the direction of the microphone.

[0009] The voice command receiving method of the present invention includes the steps of: detecting the orientation of the face of a person speaking a voice command; if it is determined that the person's face is facing the direction of a microphone that acquires the voice command speech, accepting the voice command if the recognition rate of the acquired voice command is at or above a first threshold; if it is determined that the person's face is facing a direction other than the direction of the microphone, accepting the voice command if the recognition rate of the acquired voice command is at or above a second threshold that is lower than the first threshold; and, when the voice command is accepted, executing a function corresponding to the accepted voice command.

[0010] The program of the present invention includes the step of detecting the orientation of the face of a person speaking a voice command, If it is determined that the person's face is facing the direction of the microphone that acquires the voice command, the voice command is accepted if the recognition rate of the acquired voice command is at or above a first threshold; if it is determined that the person's face is facing a direction other than the direction of the microphone, the voice command is accepted if the recognition rate of the acquired voice command is at or above a second threshold which is lower than the first threshold; and if the voice command is accepted, the computer is instructed to execute the function corresponding to the accepted voice command. [Effects of the Invention]

[0011] According to the present invention, operations can be performed appropriately using voice commands. [Brief explanation of the drawing]

[0012] [Figure 1] Figure 1 is a block diagram showing an example configuration of a recording device according to the first embodiment. [Figure 2] Figure 2 is a flowchart showing the processing flow of the control unit according to the first embodiment. [Figure 3] Figure 3 is a block diagram showing an example configuration of a recording device according to the second embodiment. [Figure 4] Figure 4 is a flowchart showing the processing flow of the control unit according to the second embodiment. [Figure 5] Figure 5 is a block diagram showing an example configuration of a voice command receiving device according to the third embodiment. [Figure 6] Figure 6 is a flowchart showing the processing flow of the voice command receiving device according to the third embodiment. [Modes for carrying out the invention]

[0013] Embodiments of the present invention will be described in detail below with reference to the attached drawings. However, the present invention is not limited by these embodiments, and in the following embodiments, the same parts are denoted by the same reference numerals to omit redundant explanations. Furthermore, the voice command receiving device according to the present invention is intended for various devices that are operated using voice commands, and the following embodiments do not limit the devices to which it is applied.

[0014] [First Embodiment] In the first embodiment, a recording device used in a vehicle will be described as an example of a voice command receiving device.

[0015] (Recording device) Using FIG. 1, a configuration example of the recording apparatus according to the first embodiment will be described. FIG. 1 is a block diagram showing a configuration example of the recording apparatus according to the first embodiment.

[0016] The recording apparatus 1 is a so-called drive recorder that records video and the like based on events occurring to a vehicle. The recording apparatus 1 may be a device mounted on the vehicle, or may be a portable device that can be used in the vehicle. The recording apparatus 1 may be realized including functions or configurations of a device or a navigation device etc. that are installed in the vehicle in advance. The recording apparatus 1 executes a process of changing the recognition rate for voice commands received according to whether or not the passengers including the driver of the vehicle are facing the traveling direction of the vehicle.

[0017] As shown in FIG. 1, the recording apparatus 1 includes a first camera 10, a second camera 12, a recording unit 14, a display unit 16, a microphone 18, an acceleration sensor 20, an operation unit 22, a GNSS (Global Navigation Satellite System) 24, and a control unit (recording control device) 26. The recording apparatus 1 may be a device that integrally includes the first camera 10, the second camera 12, and the microphone 18, or may be a device in which the first camera 10, the second camera 12, and the microphone 18 are configured separately.

[0018] The first camera 10 is a camera that photographs the periphery of the vehicle. As an example, the first camera 10 is a camera specific to the recording apparatus 1, or a plurality of cameras that photograph the front-back direction etc. of the vehicle respectively. In the first embodiment, the first camera 10 is constituted by, for example, a plurality of cameras arranged facing the front and the rear of the vehicle, and photographs the periphery centered on the front and the rear of the vehicle. The first camera 10 may be, for example, a single camera capable of photographing all-round or semi-all-round. The first camera 10 outputs the photographed first video data to the video data acquisition unit 30 of the control unit 26. The first video data is, for example, a moving image composed of images at 30 frames per second.

[0019] The second camera 12 is a camera that photographs the interior of the vehicle. The second camera 12 is positioned to photograph at least the faces of the vehicle's occupants. The vehicle's occupants may be only the driver, or they may include the driver and other occupants. The second camera 12 is positioned, for example, on the vehicle's instrument panel, or inside or around the vehicle's rearview mirror. The second camera 12 has a fixed or nearly fixed shooting range and shooting direction. The second camera 12 is composed of, for example, a visible light camera or a near-infrared camera. The second camera 12 may be composed of, for example, a combination of a visible light camera and a near-infrared camera. The second camera 12 outputs the captured second video data to the video data acquisition unit 30 of the control unit 26. The second video data is, for example, a moving image composed of 30 frames per second. When it is not necessary to distinguish between the first video data and the second video data, it is referred to as video data.

[0020] The first camera 10 and the second camera 12 may consist of a single camera capable of capturing a full or half-sky view. In this case, the first video data is defined as the entire video data, the area surrounding the vehicle, or the area in front of the vehicle, within the video data capturing a full or half-sky view. The second video data is defined as the area within the video data capturing a full or half-sky view that is capable of capturing the faces of passengers seated in the vehicle's seats. The entire video data capturing a full or half-sky view may also be treated as the first and second video data.

[0021] The recording unit 14 is used for temporary storage of data in the recording device 1. The recording unit 14 is, for example, a semiconductor memory element such as RAM (Random Access Memory) or flash memory, or a recording medium such as a memory card. Alternatively, it may be an external recording unit wirelessly connected via a communication device (not shown). The recording unit 14 records loop recording video data or event data based on control signals output from the recording control unit 36 ​​of the control unit 26.

[0022] The display unit 16 is, for example, a display device specific to the recording device 1, or a display device shared with other systems, including a navigation system. The display unit 16 may be formed integrally with the first camera 10. The display unit 16 is a display including, for example, a liquid crystal display (LCD) or an organic electro-luminescence (OLED) display. In the first embodiment, the display unit 16 is located in front of the driver of the vehicle, on the dashboard, instrument panel, center console, etc. The display unit 16 displays video based on the video signal output from the recording control unit 36 ​​of the control unit 26. The display unit 16 displays video being captured by the first camera 10, or video recorded in the recording unit 14.

[0023] Microphone 18 picks up sound from inside the vehicle's cabin. In the first embodiment, microphone 18 is positioned to pick up the voices of the vehicle's occupants, including the driver. Microphone 18 is located, for example, on the dashboard, instrument panel, or center console. Microphone 18 picks up sound related to voice commands to recording device 1. Microphone 18 outputs the sound related to the voice commands to voice command receiving unit 44. Microphone 18 outputs the picked-up sound to video data acquisition unit 30, so that recording control unit 36 ​​can record loop recording video data or event data including the sound.

[0024] The acceleration sensor 20 is a sensor that detects acceleration occurring in the vehicle. The acceleration sensor 20 outputs the detection result to the event detection unit 46 of the control unit 26. The acceleration sensor 20 is a sensor that detects acceleration in, for example, three axes. The three axes are the longitudinal direction, lateral direction, and vertical direction of the vehicle.

[0025] The operation unit 22 can accept various operations on the recording device 1. For example, the operation unit 22 can accept an operation to manually save the captured video data as event data to the recording unit 14. For example, the operation unit 22 can accept an operation to play back loop recording video data or event data recorded in the recording unit 14. For example, the operation unit 22 can accept an operation to erase event data recorded in the recording unit 14. For example, the operation unit 22 can accept an operation to terminate loop recording. The operation unit 22 outputs the operation information to the operation control unit 48 of the control unit 26.

[0026] The GNSS receiving unit 24 consists of a GNSS receiver and the like that receives GNSS signals from GNSS satellites. The GNSS receiving unit 24 outputs the received GNSS signals to the position information acquisition unit 50 of the control unit 26.

[0027] The control unit 26 controls each part of the recording device 1. The control unit 26 includes, for example, an information processing device such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), and a storage device such as RAM (Random Access Memory) or ROM (Read Only Memory). The control unit 26 executes a program that controls the operation of the recording device 1 according to the present invention. The control unit 26 may be implemented by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 26 may be implemented by a combination of hardware and software.

[0028] The control unit 26 includes a video data acquisition unit 30, a buffer memory 32, a video data processing unit 34, a recording control unit 36, a playback control unit 38, a display control unit 40, a detection unit 42, a voice command reception unit 44, an event detection unit 46, an operation control unit 48, and a location information acquisition unit 50, all of which are functional blocks realized by the configuration of the control unit 26 or by the execution of a program.

[0029] The video data acquisition unit 30 acquires first video data, which is footage of the area around the vehicle, and second video data, which is footage of the interior of the vehicle. Specifically, the video data acquisition unit 30 acquires first video data, which is footage of the first camera 10, and second video data, which is footage of the second camera 12. The video data acquisition unit 30 outputs the acquired first video data and second video data to the buffer memory 32. The first video data and second video data acquired by the video data acquisition unit 30 are not limited to video data only, but may also include video and audio. The video data acquisition unit 30 may acquire video data that captures a full or half-sky view as the first video data and second video data.

[0030] The buffer memory 32 is an internal memory of the recording device 1, and is a memory that temporarily records video data acquired by the video data acquisition unit 30 for a certain period of time, while updating it.

[0031] The video data processing unit 34 converts the video data temporarily stored in the buffer memory 32 into any file format, such as MP4, encoded using any codec, such as H.264 or MPEG-4 (Moving Picture Experts Group). The video data processing unit 34 generates a file of video data for a certain period of time from the video data temporarily stored in the buffer memory 32. As a specific example, the video data processing unit 34 generates a file of 60 seconds of video data in recording order from the video data temporarily stored in the buffer memory 32. The video data processing unit 34 outputs the generated video data to the recording control unit 36. The video data processing unit 34 outputs the generated video data to the display control unit 40. The duration of the video data generated as a file is 60 seconds as an example, but is not limited to this.

[0032] The recording control unit 36 ​​controls the recording unit 14 to record the video data that has been filed by the video data processing unit 34. When the vehicle's accessory power is ON, or during the period when loop recording processing is performed, the recording control unit 36 ​​records the video data that has been filed by the video data processing unit 34 to the recording unit 14 as overwritable video data. During the period when loop recording processing is performed, the recording control unit 36 ​​continues to record the video data generated by the video data processing unit 34 to the recording unit 14, and when the capacity of the recording unit 14 becomes full, it overwrites the oldest video data with new video data.

[0033] When the voice command receiving unit 44 receives an instruction to record an event via a voice command, the recording control unit 36 ​​saves the first video data, including the time when the instruction to record an event was received, as event data. The recording control unit 36 ​​saves the event data to the recording unit 14 as data that cannot be overwritten. For example, the recording control unit 36 ​​copies the first video data for a predetermined period of about 10 seconds before and after the time when the voice command receiving unit 44 received the event detection via a voice command from the buffer memory 32 and saves it as event data.

[0034] When the event detection unit 46 detects the occurrence of an event based on the output value of the acceleration sensor 20, the recording control unit 36 ​​saves the first video data, including the time when the event was detected, as event data. The recording control unit 36 ​​saves the event data to the recording unit 14 as data that cannot be overwritten. For example, the recording control unit 36 ​​copies the first video data for a predetermined period of about 10 seconds before and after the time when the event detection unit 46 detected the event from the buffer memory 32 and saves it as event data.

[0035] The playback control unit 38 plays back loop recording video data or event data recorded in the recording unit 14 based on the playback control signal output from the operation control unit 48, and controls the display control unit 40 to output the played-back video, etc., to the display unit 16.

[0036] The display control unit 40 controls the display of video data on the display unit 16. The display control unit 40 outputs a video signal that causes the video data to be output to the display unit 16. More specifically, the display control unit 40 outputs a video signal to be displayed by playing back the video captured by the first camera 10, or loop recording video data or event data recorded in the recording unit 14.

[0037] The detection unit 42 detects conditions in the environment in which a voice command is spoken that would result in the voice command not being properly recognized. In this embodiment, the detection unit 42 detects the orientation of the face of the person speaking the voice command. A condition in which the voice command is not properly recognized may be that the face of the person speaking the voice command is not facing the direction of the microphone that acquires the voice of the voice command. For this reason, the detection unit 42 recognizes the occupants of the vehicle from the second video data. In the first embodiment, the occupants include the driver and other passengers. The occupants may also consist only of the driver of the vehicle. The detection unit 42 recognizes the faces of people from the second video data and also detects the orientation of their faces. For example, the detection unit 42 detects a person's face based on the positional relationship of feature parts that make up a person's face, and detects the orientation of the face based on the positional relationship between the face centerline and each element that makes up the face. These detection methods can be known methods and are not limited to these.

[0038] The detection unit 42 detects the orientation of the vehicle occupant's face from the second video data and determines whether the vehicle occupant is facing in a direction that allows the microphone 18 to properly pick up the vehicle occupant's voice. For example, the detection unit 42 detects the orientation of the vehicle occupant's face from the second video data and determines whether the vehicle occupant is facing the direction of travel of the vehicle or not. The direction of travel of the vehicle refers to a direction that falls within a range of approximately ±30 degrees, where the angle when the occupant is facing forward of the vehicle is defined as 0 degrees.

[0039] In this embodiment, the detection unit 42 is described as determining whether the vehicle occupant is facing the direction of travel or not, but this disclosure is not limited thereto. The detection unit 42 may, for example, determine from the second video data whether the vehicle occupant is facing the direction of the microphone 18 or not.

[0040] The voice command receiving unit 44 accepts voice commands by recognizing the sound picked up by the microphone 18. For example, the voice command receiving unit 44 performs sound source separation processing and voice recognition processing on the sound picked up by the microphone 18 and recognizes a voice command to start event recording. A voice command to start event recording is, for example, "Start recording". When the voice command receiving unit 44 recognizes the six consecutive syllables "Ro·Ku·Ga·Ka·I·Shi" in the sound picked up by the microphone 18, it outputs a control signal to the recording control unit 36 ​​to start event recording processing. Alternatively, when the voice command receiving unit 44 recognizes the sound representing the word "RoKuGaKaIShi" in the sound picked up by the microphone 18, it outputs a control signal to the recording control unit 36 ​​to start event recording processing. The voice command receiving unit 44 changes the voice recognition rate for determining whether or not a voice command has been received, depending on whether or not the vehicle occupant is facing the direction of travel, that is, whether or not they are facing the direction of the microphone 18 for acquiring the voice command speech.

[0041] The voice command receiving unit 44 determines that a voice command has been acquired when all of the six consecutive syllables of "Ro·Ku·Ga·Ka·I·Shi" match, in situations where the voice command can be properly recognized, such as when the vehicle occupant is facing the direction of travel. For example, the voice command receiving unit 44 sets the first threshold for the recognition rate at which it determines that a voice command has been acquired to 90%. In this case, the voice command receiving unit 44 determines that a voice command has been acquired if it recognizes 90% or more of the six syllables of "Ro·Ku·Ga·Ka·I·Shi".

[0042] In situations where a voice command cannot be properly recognized, such as when a vehicle occupant is facing a direction other than the vehicle's direction of travel, the voice command receiving unit 44 determines that a voice command has been acquired if five or more of the six consecutive syllables "Ro·Ku·Ga·Ka·I·Shi" match. In this case, the voice command receiving unit 44 sets the recognition rate for determining that a voice command has been acquired to a second threshold, which is lower than the first threshold. For example, the voice command receiving unit 44 sets the second threshold to 80%. In this case, the voice command receiving unit 44 determines that a voice command has been acquired if it recognizes 80% or more of the six consecutive syllables "Ro·Ku·Ga·Ka·I·Shi". In other words, in situations where a voice command cannot be properly recognized, such as when a vehicle occupant is facing a direction other than the vehicle's direction of travel, the voice command is properly recognized by determining that a voice command has been spoken, even if the occupant's speech cannot be fully recognized.

[0043] Furthermore, when the vehicle occupant is facing the direction of travel, the voice command receiving unit 44 sets the matching rate between the acoustic model of the sound waveform representing the word "RoKuGaKaIShi" and the waveform of the input voice as the first threshold for determining whether a voice command has been acquired, for example, 90%. In this case, the voice command receiving unit 44 determines that a voice command has been acquired if the matching rate between the acoustic model of the sound waveform representing the word "RoKuGaKaIShi" and the waveform of the input voice is 90% or higher.

[0044] Furthermore, if the vehicle occupant is facing a direction other than the vehicle's direction of travel, the voice command receiving unit 44 sets the matching rate between the acoustic model of the sound waveform representing the word "RoKuGaKaIShi" and the waveform of the input sound as a second threshold, lower than the first threshold for determining whether a voice command has been acquired, for example, 80%. In this case, if the matching rate between the acoustic model of the sound waveform representing the word "RoKuGaKaIShi" and the waveform of the input sound is 80% or higher, the voice command receiving unit 44 determines that a voice command has been acquired. In other words, if the vehicle occupant is facing a direction other than the vehicle's direction of travel, the occupant's voice is more likely to be recognized as a voice command.

[0045] The event detection unit 46 detects events based on acceleration applied to the vehicle. The event detection unit 46 detects events based on the detection results of the acceleration sensor 20. The event detection unit 46 detects that an event has occurred if the acceleration information is above a preset threshold corresponding to a vehicle collision.

[0046] The operation control unit 48 acquires operation information for operations received by the operation unit 22. For example, the operation control unit 48 acquires save operation information indicating a manual save operation of video data, playback operation information indicating a playback operation, or erase operation information indicating an erase operation of video data, and outputs a control signal. For example, the operation control unit 48 acquires termination operation information indicating an operation to end loop recording, and outputs a control signal.

[0047] The operation control unit 48 accepts event recording operations based on voice commands recognized and received by the voice command reception unit 44.

[0048] The location information acquisition unit 50 acquires location information indicating the vehicle's current location. Based on the GNSS signal received by the GNSS receiving unit 24, the location information acquisition unit 50 calculates the vehicle's current location information using a known method.

[0049] (Processing of the control unit) The processing flow of the control unit according to the first embodiment will be explained using Figure 2. Figure 2 is a flowchart showing the processing flow of the control unit according to the first embodiment. The flowchart shown in Figure 2 starts when the power source, such as the engine of the vehicle to which the recording device 1 is installed, is started.

[0050] As processing begins, the control unit 26 starts normal recording and orientation detection (step S10). Specifically, the recording control unit 36 ​​transmits the video data captured by the first camera 10 and the second camera 12 to the buffer memory 32, generates a video file for each predetermined period of video, such as every 60 seconds, and has the recording unit 14 record it. The detection unit 42 starts detecting the orientation of the vehicle occupants' faces. Then, the process proceeds to step S12.

[0051] The detection unit 42 determines whether the occupant of the vehicle is facing a direction other than the direction of travel (step S12). Specifically, the detection unit 42 determines whether the occupant of the vehicle is facing a direction other than the direction of travel for a predetermined period of time or longer. The detection unit 42 determines that the occupant of the vehicle is facing a direction other than the direction of travel if the occupant is facing a direction other than the direction of travel for a predetermined period of time or longer. The predetermined period is, for example, 2 seconds or more, but is not limited to this. If it is determined that the occupant is facing a direction other than the direction of travel (step S12; Yes), the process proceeds to step S14. If it is not determined that the occupant is facing a direction other than the direction of travel (step S12; No), the process proceeds to step S18.

[0052] If the result in step S12 is Yes, the voice command receiving unit 44 determines whether or not it has received a voice command from the vehicle occupant via the microphone 18 (step S14). If it is determined that a voice command has been received (step S14; Yes), the process proceeds to step S16. If it is determined that a voice command has not been received (step S14; No), the process proceeds to step S24.

[0053] If the answer in step S14 is Yes, the voice command receiving unit 44 determines whether the recognition rate of the acquired voice command is above the second threshold (step S16). If it is determined that the recognition rate of the voice command is above the second threshold (step S16; Yes), the process proceeds to step S22. If it is determined that the recognition rate of the voice command is not above the second threshold (step S16; No), the process proceeds to step S24.

[0054] If the result in step S12 is No, the voice command receiving unit 44 determines whether or not it has received a voice command from the vehicle occupant via the microphone 18 (step S18). If it is determined that a voice command has been received (step S18; Yes), the process proceeds to step S20. If it is determined that a voice command has not been received (step S18; No), the process proceeds to step S24.

[0055] If the answer in step S18 is Yes, the voice command receiving unit 44 determines whether the recognition rate of the acquired voice command is above the first threshold (step S20). If it is determined that the recognition rate of the voice command is above the first threshold (step S20; Yes), the process proceeds to step S22. If it is determined that the recognition rate of the voice command is not above the second threshold (step S20; No), the process proceeds to step S24.

[0056] In steps S14 and S18, in addition to determining whether or not a voice command has been received, it may also be determined whether or not the received voice command is an urgent or immediate voice command. In other words, in steps S14 and S18, it is determined whether or not an urgent or immediate voice command has been received. An urgent or immediate voice command is a voice command that requests operation on a function that is required to start operating without delay once the voice command is received. For example, an urgent or immediate voice command in recording device 1 is a voice command that instructs event recording.

[0057] If the answer is Yes in step S16 or Yes in step S20, the recording control unit 36 ​​saves the event data to the recording unit 14 (step S22). Specifically, the recording control unit 36 ​​saves the first video data before and after the time when the voice command receiving unit 44 acquired the voice command as event data to the recording unit 14. Then, the process proceeds to step S24.

[0058] If the result in steps S14 to S20 is No, or after step S22, the control unit 26 determines whether or not to terminate the process (step S24). Specifically, the control unit 26 determines that the process is terminated when the operation unit 22 receives an operation to turn off the power or an operation to terminate the process, or when the power of the vehicle to which the recording device 1 is installed, such as the engine, is turned off. If it is determined that the process should be terminated (step S24; Yes), the process shown in Figure 2 is terminated. If it is not determined that the process should be terminated (step S24; No), the process proceeds to step S12.

[0059] As described above, the first embodiment saves event data by changing the recognition rate for recognizing voice as a voice command depending on whether the vehicle occupant is facing the direction of travel or not. In the first embodiment, when the occupant is facing not the direction of travel, the recognition rate is lowered compared to when the occupant is facing not the direction of travel, and the event data saving process is performed. As a result, the first embodiment can properly save event data via voice commands even when the microphone 18 has difficulty picking up the occupant's voice because the occupant is facing not the direction of travel.

[0060] [Second Embodiment] A second embodiment will now be described. The second embodiment differs from the first embodiment in that, when there are multiple passengers in the vehicle, the recognition rate of voice commands is changed based on the direction that each passenger is facing.

[0061] (Recording device) An example of the configuration of a recording device according to the second embodiment will be explained using Figure 3. Figure 3 is a block diagram showing an example of the configuration of a recording device according to the second embodiment.

[0062] As shown in Figure 3, the recording device 1A differs from the recording device 1 shown in Figure 1 in that the control unit 26A includes a ratio calculation unit 52.

[0063] The detection unit 42A detects the direction each of the multiple passengers is facing when there are multiple passengers in the vehicle. The detection unit 42A detects whether each of the multiple passengers is facing the direction of travel or not. The detection unit 42A determines that the passengers in the vehicle are facing a direction other than the direction of travel if a predetermined percentage or more of the multiple passengers are facing the same direction other than the direction of travel.

[0064] The ratio calculation unit 52 calculates the percentage of passengers facing a predetermined direction among multiple passengers. The ratio calculation unit 52 also calculates the percentage of passengers facing a direction other than the direction of travel. The ratio calculation unit 52 determines whether the percentage of passengers facing a direction other than the direction of travel is equal to or greater than a predetermined percentage. The predetermined percentage is, for example, 50% or more. Specifically, the ratio calculation unit 52 determines that a predetermined percentage or more of passengers are facing a direction other than the direction of travel if, when there are two passengers, one or more passengers are facing the same direction other than the direction of travel; when there are three passengers, two or more passengers are facing the same direction other than the direction of travel; and when there are four passengers, two or more passengers are facing the same direction other than the direction of travel. Note that the predetermined percentage is not limited to 50% and may be any other value.

[0065] (Processing of the control unit) The processing flow of the control unit according to the second embodiment will be explained using Figure 4. Figure 4 is a flowchart showing the processing flow of the control unit according to the second embodiment.

[0066] The control unit 26A starts normal recording and orientation detection (step S30). The normal recording process is the same as step S10 shown in Figure 2, so the explanation is omitted. The detection unit 42A detects the orientation of each of the multiple passengers in the vehicle. Then, the process proceeds to step S22.

[0067] The detection unit 42A determines whether a predetermined percentage or more of the passengers are facing a direction other than the direction of travel of the vehicle (step S32). Specifically, the percentage calculation unit 52 determines, based on the detection result of the detection unit 42A, whether a predetermined percentage or more of the passengers are facing a direction other than the direction of travel of the vehicle. The detection unit 42A then determines that the passengers of the vehicle are facing a direction other than the direction of travel of the vehicle if a predetermined percentage or more of the passengers among multiple passengers are facing the same direction other than the direction of travel of the vehicle. If it is determined that a predetermined percentage or more of the passengers are facing a direction other than the direction of travel of the vehicle (step S32; Yes), the process proceeds to step S34. If it is not determined that a predetermined percentage or more of the passengers are facing a direction other than the direction of travel of the vehicle (step S32; No), the process proceeds to step S38.

[0068] The processes from step S34 to step S44 are identical to the processes from step S14 to step S24 shown in Figure 2, so their explanation will be omitted.

[0069] As described above, the second embodiment saves event data by changing the recognition rate for recognizing voices as voice commands according to the proportion of passengers in the vehicle who are facing a direction other than the direction of travel. In the second embodiment, if the proportion of passengers facing a direction other than the direction of travel is greater than a predetermined proportion, the recognition rate is lowered and the event data saving process is executed. As a result, the second embodiment can appropriately save event data via voice commands even when the microphone 18 has difficulty picking up the voices of passengers because multiple passengers are facing a direction other than the direction of travel.

[0070] [Third Embodiment] A third embodiment will now be described. The third embodiment is a general-purpose device that is operated using voice commands and can be applied to, for example, household devices such as smart speakers and television receivers, information devices such as smartphones, tablet terminals and PCs, and navigation devices and infotainment systems used in vehicles.

[0071] An example of the configuration of the voice command receiving device according to the third embodiment will be explained using Figure 5. Figure 5 is a block diagram showing an example of the configuration of the voice command receiving device according to the third embodiment.

[0072] As shown in Figure 5, the voice command receiving device 100 comprises a voice command receiving unit 102, a detection unit 104, and an execution control unit 106. The voice command receiving device 100 includes, for example, an information processing device such as a CPU or MPU, and a storage device such as RAM or ROM. The voice command receiving device 100 executes a program according to the present invention. The voice command receiving device 100 may be implemented by an integrated circuit such as an ASIC or FPGA. The voice command receiving device 100 may be implemented by a combination of hardware and software.

[0073] The microphone 110 picks up the voice spoken by the speaker. The microphone 110 outputs the voice related to the picked-up voice to the voice command receiving device 100. The microphone 110 may be integrated with the voice command receiving device 100 or it may be configured as a separate unit.

[0074] Camera 120 photographs the speaker. Camera 120 photographs at least the speaker's face. Camera 120 outputs video data related to the captured video to the voice command receiving device 100. Camera 120 may be integrated with the voice command receiving device 100 or may be configured separately.

[0075] The voice command receiving unit 102 receives voice commands. The voice command receiving unit 102 receives voice commands, for example, by recognizing the sound picked up by the microphone 110.

[0076] The detection unit 104 detects conditions in the environment in which a voice command is spoken that would result in the voice command not being properly recognized. In this embodiment, the detection unit 104 detects the orientation of the face of the person speaking the voice command. For example, the detection unit 104 detects the orientation of the face of the person speaking the voice command based on video data captured by the camera 120.

[0077] When the voice command receiving unit 102 receives a voice command, the execution control unit 106 causes the function corresponding to the received voice command to be executed.

[0078] The voice command receiving unit 102 accepts voice commands by changing the recognition rate of the voice command depending on whether the detection unit 104 determines that the direction of the person's face is facing the direction of the microphone 110 that acquires the voice command speech. For example, if the voice command receiving unit 102 determines that the direction of the person's face is facing the direction of the microphone 110, it accepts the voice command at a recognition rate of the first threshold or higher. For example, if the voice command receiving unit 102 determines that the direction of the person's face is facing a direction other than the microphone 110, it accepts the voice command at a recognition rate of the second threshold or higher, which is lower than the first threshold.

[0079] The voice command receiving unit 102 accepts voice commands of high urgency or immediacy with a recognition rate of the second threshold or higher. In the third embodiment, voice commands of high urgency or immediacy are voice commands for functions where a delay from the time of operation is undesirable, or where a delay would have adverse effects or risks, such as emergency calls, emergency communications, instructions to start recording broadcast content, and instructions to stop functions with a high risk of continuation.

[0080] (Processing by the voice command reception device) The processing flow of the voice command receiving device according to the third embodiment will be explained using Figure 6. Figure 6 is a flowchart showing the processing flow of the voice command receiving device according to the third embodiment.

[0081] The detection unit 104 starts orientation detection (step S50). Specifically, the detection unit 104 starts detecting the orientation of the speaker's face. Then, the process proceeds to step S52.

[0082] The detection unit 104 determines whether the speaker is facing the microphone 110 (step S52). Specifically, the detection unit 42 determines whether the speaker is facing the microphone 110 for a predetermined time or longer. The detection unit 42 determines that the speaker is facing the microphone 110 if the speaker is facing the microphone 110 for a predetermined time or longer. The predetermined time is, for example, 2 seconds or more, but is not limited to this. If it is determined that the speaker is facing the microphone 110 (step S52; Yes), the process proceeds to step S54. If it is not determined that the speaker is facing the microphone 110 (step S52; No), the process proceeds to step S58.

[0083] If the result in step S52 is Yes, the voice command receiving unit 102 determines whether or not it has received a voice command from the speaker via the microphone 110 (step S54). If it is determined that a voice command has been received (step S54; Yes), the process proceeds to step S56. If it is determined that a voice command has not been received (step S54; No), the process proceeds to step S64.

[0084] If the answer in step S54 is Yes, the voice command receiving unit 102 determines whether the recognition rate of the acquired voice command is above the first threshold (step S56). If it is determined that the recognition rate of the voice command is above the first threshold (step S56; Yes), the process proceeds to step S62. If it is determined that the recognition rate of the voice command is not above the first threshold (step S56; No), the process proceeds to step S64.

[0085] If the result in step S52 is No, the voice command receiving unit 102 determines whether or not it has received a voice command from the speaker via the microphone 110 (step S58). If it is determined that a voice command has been received (step S58; Yes), the process proceeds to step S60. If it is determined that a voice command has not been received (step S58; No), the process proceeds to step S64.

[0086] If the answer in step S58 is Yes, the voice command receiving unit 102 determines whether the recognition rate of the acquired voice command is above the second threshold (step S60). If it is determined that the recognition rate of the voice command is above the second threshold (step S60; Yes), the process proceeds to step S62. If it is determined that the recognition rate of the voice command is not above the second threshold (step S60; No), the process proceeds to step S64.

[0087] In steps S54 and S58, in addition to determining whether or not a voice command has been acquired, it may also be determined whether or not the acquired voice command is an urgent or immediate voice command.

[0088] If the result is Yes in step S56 or Yes in step S60, the execution control unit 106 executes the function corresponding to the voice command (step S62). Then, the process proceeds to step S64.

[0089] If the result in steps S54 to S60 is No, or after step S62, the voice command receiving device 100 determines whether or not to terminate the process (step S64). Specifically, the voice command receiving device 100 determines to terminate the process when it receives an operation to turn off the power or an operation to terminate the process. If it is determined that the process should be terminated (step S64; Yes), the process shown in Figure 6 is terminated. If it is not determined that the process should be terminated (step S64; No), the process proceeds to step S52.

[0090] As described above, the third embodiment changes the recognition rate for recognizing speech as a voice command depending on whether the speaker is facing the microphone or not, and executes the function for voice commands. In the third embodiment, when the speaker is facing a direction other than the microphone, the recognition rate is lowered compared to when the speaker is facing the microphone, and the function for voice commands is executed. As a result, the third embodiment can appropriately execute the function for voice commands even when the speaker is facing a direction other than the microphone, making it difficult for the microphone to pick up the speaker's voice.

[0091] Although embodiments of the present invention have been described above, the present invention is not limited by the content of these embodiments. Furthermore, the aforementioned components include those that can be easily conceived by those skilled in the art, those that are substantially the same, and those that fall within the so-called equivalent range. Moreover, the aforementioned components can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the gist of the embodiments described above. [Explanation of Symbols]

[0092] 1.1A Recording Device 10 First Camera 12 Second Camera 14 Records Section 16 Display section 18,110 microphones 20 Accelerometer 22 Control section 24 GNSS receiver 26,26A Control Unit (Recording Control Device) 30 Video data acquisition unit 32 buffer memory 34 Video Data Processing Unit 36 Recording Control Unit 38 Regeneration Control Unit 40 Display Control Unit 42, 42A, 104 Detection unit 44 Voice Command Reception Unit 46 Event Detection Unit 48 Operation Control Unit 50 Location information acquisition section 52 Percentage Calculation Section 100 Voice command receiving device 102 Voice Command Reception Unit 106 Execution Control Unit 120 Cameras

Claims

1. A voice command reception unit that receives voice commands, A detection unit that detects conditions under which the voice command is not properly recognized in the environment where the voice command is uttered, An execution control unit that, when the voice command reception unit receives a voice command, executes a function for the received voice command, Comprising When the detection unit determines that the condition is not a condition under which the voice command is not properly recognized, the voice command reception unit receives the voice command when the recognition rate of the voice command acquired by the voice command reception unit is equal to or higher than a first threshold. When the detection unit determines that the condition is a condition under which the voice command is not properly recognized, the voice command reception unit receives the voice command when the recognition rate of the voice command acquired by the voice command reception unit is equal to or higher than a second threshold lower than the first threshold. A voice command reception device.

2. The detection unit detects the orientation of the face of the person uttering the voice command as a condition under which the voice command is not properly recognized. When the detection unit determines that the orientation of the person's face is facing the direction of the microphone that acquires the voice of the voice command, the voice command reception unit receives the voice command when the recognition rate of the voice command acquired by the voice command reception unit is equal to or higher than a first threshold. When the detection unit determines that the orientation of the person's face is facing a direction other than the direction of the microphone, the voice command reception unit receives the voice command when the recognition rate of the voice command acquired by the voice command reception unit is equal to or higher than a second threshold lower than the first threshold. The voice command reception device according to Claim 1.

3. For voice commands with high urgency or immediacy, the voice command reception unit receives the voice command when the recognition rate of the voice command acquired by the voice command reception unit is equal to or higher than a second threshold lower than the first threshold. The voice command reception device according to Claim 1.

4. The voice command reception device is a vehicle recording control device used in a vehicle, Comprising a video data acquisition unit that acquires first video data captured by a first imaging unit that captures the periphery of the vehicle and second video data captured by a second imaging unit that captures the interior of the vehicle cabin, The voice command reception unit receives an event recording instruction by the voice command, The detection unit detects the orientation of the face of the vehicle occupant from the second video data. When the execution control unit receives an event recording instruction by the voice command from the voice command reception unit, the execution control unit stores the first video data including the time point when the event recording instruction is received as event data. The voice command reception device according to claim 1.

5. When the passenger of the vehicle faces a direction other than the traveling direction of the vehicle for a predetermined time or more, the detection unit determines that the passenger of the vehicle faces a direction other than the direction of the microphone. The voice command reception device according to claim 4.

6. The detection unit detects the face orientations of a plurality of passengers of the vehicle from the second video data, and when a predetermined ratio or more of the plurality of passengers face the same direction other than the traveling direction of the vehicle, the detection unit determines that the passengers of the vehicle face a direction other than the traveling direction of the vehicle. The voice command reception device according to claim 4 or 5.

7. Detecting conditions under which the recorded voice command is not properly recognized in an environment where a voice command is uttered; When it is determined that the conditions are not those under which the voice command is not properly recognized, accepting the voice command when the recognition rate of the obtained voice command is equal to or higher than a first threshold value, and when it is determined that the conditions are those under which the voice command is not properly recognized, accepting the voice command when the recognition rate of the obtained voice command is equal to or higher than a second threshold value lower than the first threshold value; When the voice command is accepted, executing a function for the accepted voice command; A voice command reception method executed by a voice command reception device.

8. Detecting conditions under which the recorded voice command is not properly recognized in an environment where a voice command is uttered; When it is determined that the conditions are not those under which the voice command is not properly recognized, accepting the voice command when the recognition rate of the obtained voice command is equal to or higher than a first threshold value, and when it is determined that the conditions are those under which the voice command is not properly recognized, accepting the voice command when the recognition rate of the obtained voice command is equal to or higher than a second threshold value lower than the first threshold value; When the voice command is accepted, executing a function for the accepted voice command; A program that causes a computer to execute the above steps.

Images