Voice command receiving device and voice command receiving method

Abstract

A voice command receiving device according to the present application comprises: a voice command receiving unit for receiving a voice command; a detection unit that detects biological information of a person who issues the voice command; and an execution control unit that executes a function for the received voice command when the voice command reception unit receives the voice command, and that executes a function for the received voice command when the detection unit determines that the biological information of the person who issues the voice command indicates a stationary state. When the recognition rate of the voice command acquired by the voice command receiving unit is greater than or equal to a first threshold value, the voice command is received, and when the detection unit determines that the biological information of the person who issues the voice command indicates that the person is not in a stationary state, the voice command is received. The voice command receiving unit receives the voice command when the recognition rate of the voice command acquired by the voice command receiving unit is equal to or greater than a second threshold value lower than the first threshold value.

Description

Technical Field

[0001] The invention relates to a voice instruction accepting device and a voice instruction accepting method. Background Art

[0002] Devices that operate by voice commands are becoming more diverse. For example, in vehicle recording devices, so-called driving recorders, in addition to the impact detection of the acceleration sensor, there are also devices that record events by voice commands (for example, non-patent document 1). When recording an accident in which the driver is not a party to the accident, event recording based on voice commands does not require operation of a touch panel or the like while driving, and events can be recorded safely. Patent document 1 discloses a driving recorder that records events by providing voice instructions for event detection based on acceleration.

[0003] Prior art literature

[0004] Patent document 1: Japanese Patent Application Publication No. 2020-154904.

[0005] Non-patent literature

[0006] Non-patent document 1: DRV-MR760 [retrieved on December 20, 2019], Internet (URL: https: / / www.kenwood.com / jp / car / drive-recorders / products / drv-mr760 / ) Summary of the invention

[0007] The voice command for instructing the dash cam to record an event is pre-set to accept a voice command such as "Ro·Ku·Ga·Ka·I·Shi (start recording)". In order to prevent false detection caused by other voices, the voice command is required to consist of a certain number of syllables. For example, "Ro·Ku·Ga·Ka·I·Shi" consists of 6 syllables. Therefore, in order to accurately recognize the voice command, the speaker often speaks in the direction of the dash cam or the direction of the microphone that inputs the spoken voice of the voice command. A general dash cam is installed in front of the vehicle when viewed from the passenger who is the speaker, so it can appropriately recognize the voice command input in the state of being directed in the direction of travel in front of the vehicle.

[0008] However, when a voice command is issued under a condition where the voice command is not properly recognized, the recognition rate of the voice command becomes low, so the instruction based on the voice command is not accepted. In such a case, for example, voice commands for recording events in a driving recorder, or voice commands for instructing operations that require urgency or immediacy, are delayed due to the repetition of the voice command. The condition where the voice command is not properly recognized means that, for example, it may occur when the person issuing the voice command is in a state where he cannot issue the voice command properly.

[0009] An object of the present disclosure is to provide a voice command accepting device and a voice command accepting method that can appropriately perform an operation based on a voice command.

[0010] The voice instruction accepting device disclosed in the present invention includes: a voice instruction accepting unit for accepting voice instructions; a detection unit for detecting biological information of a person issuing the voice instruction; and an execution control unit for executing a function for the accepted voice instruction when the voice instruction accepting unit has accepted the voice instruction. The voice instruction accepting unit accepts the voice instruction when the recognition rate of the voice instruction obtained by the voice instruction accepting unit is above a first threshold value when the detection unit determines that the biological information of the person indicates a stable state; and accepts the voice instruction when the recognition rate of the voice instruction obtained by the voice instruction accepting unit is above a second threshold value when the detection unit determines that the biological information of the person indicates a non-steady state, and the second threshold value is lower than the first threshold value.

[0011] The voice command receiving method disclosed in the present invention comprises a voice command receiving device executing the following steps: detecting the biological information of a person issuing a voice command; accepting the voice command when the recognition rate of the voice command is above a first threshold value when it is determined that the biological information indicates a stable state, and accepting the voice command when the recognition rate of the voice command is above a second threshold value when it is determined that the biological information does not indicate a stable state; and executing a function for the accepted voice command when the voice command is received.

[0012] According to the present disclosure, operations based on voice instructions can be appropriately performed. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is a block diagram showing a configuration example of a recording device according to the first embodiment.

[0014] Figure 2 This is a flowchart showing the flow of processing by the control unit according to the first embodiment.

[0015] Figure 3 This is a block diagram showing a configuration example of a voice command accepting device according to the second embodiment.

[0016] Figure 4 This is a flowchart showing the processing flow of the voice command accepting device involved in the second embodiment. DETAILED DESCRIPTION

[0017] Hereinafter, the embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In addition, the present disclosure is not limited to the embodiments, and in the following embodiments, repeated descriptions are omitted by marking the same parts with the same figure marks. In addition, the voice command receiving device of the present disclosure envisions various devices that are operated using voice commands, and the devices applied are not limited to the following embodiments.

[0018] [First embodiment]

[0019] In the first embodiment, a recording device used in a vehicle is described as an example of a voice command accepting device.

[0020] (Recording device)

[0021] use Figure 1 , an example of the structure of the recording device involved in the first embodiment is described. Figure 1 This is a block diagram showing a configuration example of a recording device according to the first embodiment.

[0022] The recording device 1 is a so-called driving recorder that records images of events occurring in a vehicle. The recording device 1 may be a device mounted on a vehicle or a portable device that can be used in a vehicle. The recording device 1 may also be implemented by including functions or structures of a device pre-installed in a vehicle, a navigation device, etc. The recording device 1 performs processing to change the recognition rate of received voice commands based on whether the biological information of passengers including the driver of the vehicle is in a stable state.

[0023] like Figure 1 As shown, the recording device 1 includes: a first camera 10, a second camera 12, a recording unit 14, a display unit 16, a microphone 18, an acceleration sensor 20, an operation unit 22, a GNSS (Global Navigation Satellite System) receiving unit 24, and a control unit (recording control device) 26. The recording device 1 may be a device that includes the first camera 10, the second camera 12, and the microphone 18 in an integrated manner, or may be a device that includes the first camera 10, the second camera 12, and the microphone 18 separately.

[0024] The first camera 10 is a camera that captures the surroundings of the vehicle. As an example, the first camera 10 is a camera inherent to the recording device 1, or a plurality of cameras that capture the front and rear directions of the vehicle, etc. In the first embodiment, the first camera 10 is, for example, composed of a plurality of cameras arranged toward the front and rear of the vehicle, and captures the surroundings centered on the front and rear of the vehicle. The first camera 10 may also be, for example, a single camera that can capture omnidirectional or semi-directional images. The first camera 10 outputs the captured first image data to the image data acquisition unit 30 of the control unit 26. The first image data is, for example, a moving image composed of images at 30 frames per second.

[0025] The second camera 12 is a camera for photographing the interior of the vehicle. The second camera 12 is arranged at a position where it can photograph at least the face of the vehicle's passengers. The vehicle's passengers refer to: it can be only the driver of the vehicle, or it can also include other passengers in addition to the driver of the vehicle. The second camera 12 is arranged, for example, on the dashboard of the vehicle, or inside the rearview mirror of the vehicle or around the rearview mirror. The shooting range and shooting direction of the second camera 12 are fixed or approximately fixed. The second camera 12 includes, for example, a visible light camera or a near-infrared camera. The second camera 12 can also be composed of a combination of a visible light camera and a near-infrared camera. The second camera 12 outputs the captured second image data to the image data acquisition unit 30 of the control unit 26. The second image data is a moving image composed of, for example, 30 frames of images per second. In addition, when there is no need to distinguish between the first image data and the second image data, it is recorded as image data.

[0026] The first camera 10 and the second camera 12 may also be composed of a single camera capable of capturing images in all directions or in half directions. In this case, among the image data obtained by capturing images in all directions or in half directions, the entire image data or the range around the vehicle, the range in front of the vehicle, etc. are set as the first image data. In addition, among the image data obtained by capturing images in all directions or in half directions, the range capable of capturing the face of a passenger sitting on a seat in the vehicle is set as the second image data. The image data obtained by capturing images throughout the day or half a day can be regarded as the first image data and the second image data as a whole.

[0027] The recording unit 14 is used for temporary storage of data in the recording device 1. The recording unit 14 is, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a recording medium such as a memory card. Alternatively, it may be an external recording unit wirelessly connected via a communication device not shown. The recording unit 14 records the cyclic recording image data or event data according to the control signal output from the recording control unit 36 ​​of the control unit 26.

[0028] The display unit 16 is, for example, a display device inherent to the recording device 1, or a display device shared with other systems including a navigation system. The display unit 16 may also be formed integrally with the first camera 10. The display unit 16 is, for example, a display including a liquid crystal display (LCD: Liquid Crystal Display) or an organic EL (Organic Electro-Luminescence) display. In the first embodiment, the display unit 16 is arranged on a control panel (Dashboard), an instrument panel (Instrument Panel), a center console, etc. in front of the driver of the vehicle. The display unit 16 displays an image based on an image signal output from the recording control unit 36 ​​of the control unit 26. The display unit 16 displays an image captured by the first camera 10 or an image recorded in the recording unit 14.

[0029] The microphone 18 collects voices in the vehicle cabin. In the first embodiment, the microphone 18 is arranged at a position where it can pick up voices uttered by passengers of the vehicle including the driver. The microphone 18 is arranged, for example, on a control panel (Dashboard), an instrument panel (Instrument Panel), a center console, etc. The microphone 18 picks up voices related to voice instructions for the recording device 1. The microphone 18 outputs the voice related to the voice instructions to the voice instruction receiving unit 44. The microphone 18 can output the picked up voice to the image data acquisition unit 30, and the recording control unit 36 ​​can record the loop-recorded image data or event data including the voice.

[0030] The acceleration sensor 20 is a sensor that detects acceleration applied to the vehicle. The acceleration sensor 20 outputs the detection result to the event detection unit 46 of the control unit 26. The acceleration sensor 20 is, for example, a sensor that detects acceleration in three-axis directions. The three-axis directions are the front-rear direction, the left-right direction, and the up-down direction of the vehicle.

[0031] The operation unit 22 can accept various operations for the recording device 1. For example, the operation unit 22 can accept an operation of manually saving the captured image data as event data in the recording unit 14. For example, the operation unit 22 can accept an operation of reproducing the loop-recorded image data or event data recorded in the recording unit 14. For example, the operation unit 22 can accept an operation of deleting the event data recorded in the recording unit 14. For example, the operation unit 22 can accept an operation of ending the loop recording. The operation unit 22 outputs the operation information to the operation control unit 48 of the control unit 26.

[0032] The GNSS receiving unit 24 is composed of a GNSS receiver that receives a GNSS signal from a GNSS satellite, etc. The GNSS receiving unit 24 outputs the received GNSS signal to the position information acquiring unit 50 of the control unit 26 .

[0033] The control unit 26 is a recording control device that controls each part of the recording device 1. The control unit 26 has, for example, an information processing device such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) and a storage device such as a RAM (Random Access Memory) or a ROM (Read Only Memory). The control unit 26 executes a program that controls the actions of the recording device 1 involved in the present disclosure. The control unit 26 can also be implemented by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 26 can also be implemented by a combination of hardware and software.

[0034] The control unit 26 includes an image data acquisition unit 30, a buffer memory 32, an image data processing unit 34, a recording control unit 36, a reproduction control unit 38, a display control unit 40, a detection unit 42, a voice command receiving unit 44, an event detection unit 46, an operation control unit 48 and a position information acquisition unit 50 as the components of the control unit 26 or functional blocks realized by the execution of a program.

[0035] The image data acquisition unit 30 acquires first image data of the surrounding area of ​​the vehicle and second image data of the interior of the vehicle. Specifically, the image data acquisition unit 30 acquires the first image data captured by the first camera 10 and the second image data captured by the second camera 12. The image data acquisition unit 30 outputs the acquired first image data and second image data to the buffer memory 32. The first image data and second image data acquired by the image data acquisition unit 30 are not limited to image data, but may also be image data including images and voices. The image data acquisition unit 30 may acquire image data obtained by capturing a whole day or half a day as the first image data and the second image data.

[0036] The buffer memory 32 is an internal memory included in the recording device 1 , and is a memory for updating and temporarily recording the video data for a certain period of time acquired by the video data acquisition unit 30 .

[0037] The image data processing unit 34 converts the image data temporarily stored in the buffer memory 32 into any file format such as MP4 format, which is encoded by a codec of any method such as H.264 or MPEG-4 (Moving Picture Experts Group). The image data processing unit 34 generates image data as a file of a certain amount of time based on the image data temporarily stored in the buffer memory 32. As a specific example, the image data processing unit 34 generates 60 seconds of image data as a file according to the recording order of the image data temporarily stored in the buffer memory 32. The image data processing unit 34 outputs the generated image data to the recording control unit 36. The image data processing unit 34 outputs the generated image data to the display control unit 40. The period of the image data generated as a file is set to 60 seconds as an example, but is not limited to this.

[0038] The recording control unit 36 ​​controls the recording unit 14 to record the image data that has been file-processed by the image data processing unit 34. The recording control unit 36 ​​records the image data that has been file-processed by the image data processing unit 34 in the recording unit 14 as overwritable image data during the period of executing the loop recording process, such as when the auxiliary power of the vehicle is turned on. The recording control unit 36 ​​continuously records the image data generated by the image data processing unit 34 in the recording unit 14 during the period of executing the loop recording process, and when the capacity of the recording unit 14 is full, the oldest image data is overwritten with new image data for recording.

[0039] When the voice command receiving unit 44 receives an instruction to record an event based on a voice command, the recording control unit 36 ​​saves the first image data including the time point when the instruction to record the event is received as event data. The recording control unit 36 ​​saves the event data in the recording unit 14 as data that is prohibited from being overwritten. For example, the recording control unit 36 ​​copies the first image data of a predetermined period of about 10 seconds before and after the time point when the voice command receiving unit 44 receives the event detection of the voice command from the buffer memory 32, and saves it as event data. The recording control unit 36 ​​is equivalent to the execution control unit 150 in the second embodiment.

[0040] When the event detection unit 46 detects the occurrence of an event based on the output value of the acceleration sensor 20, the recording control unit 36 ​​saves the first video data including the time point at which the event was detected as event data. The recording control unit 36 ​​saves the event data as data that is prohibited from being overwritten in the recording unit 14. For example, the recording control unit 36 ​​copies the first video data of a predetermined period of about 10 seconds before and after the time point at which the event detection unit 46 detects the event from the buffer memory 32 and saves it as event data.

[0041] The reproduction control unit 38 performs control such as reproducing the loop-recorded video data or event data recorded in the recording unit 14 based on the control signal of the reproduction operation output from the operation control unit 48 and outputting the reproduced video on the display unit 16 via the display control unit 40 .

[0042] The display control unit 40 controls the display of the image data on the display unit 16. The display control unit 40 outputs an image signal for causing the display unit 16 to output the image data. More specifically, the display control unit 40 outputs an image signal for displaying an image captured by the first camera 10 or for displaying the image data or event data recorded in the recording unit 14 by reproducing the loop-recorded image data or event data.

[0043] The detection unit 42 detects conditions in the environment where the voice command is issued that lead to a situation where the voice command is not properly recognized. In this embodiment, the detection unit 42 detects the biological information of the person who issued the voice command. In the case of the recording device 1 used in a vehicle, the person who issued the voice command is a passenger of the vehicle or a driver of the vehicle.

[0044] The detection unit 42 acquires information related to the heartbeat of the vehicle passenger from the second image data. The detection unit 42 detects the increase or decrease in the brightness value of the green wavelength band corresponding to the absorption band of hemoglobin in the blood in the second image data based on the part corresponding to the skin of the face of the vehicle passenger in the second image data, thereby detecting the heartbeat of the vehicle passenger. The detection unit 42 can also replace the second image data, or acquire information related to the heartbeat from a smart watch worn by the vehicle passenger, a sensor provided on the steering wheel of the vehicle, etc., based on the second image data. The detection unit 42 determines whether the heart rate of the vehicle passenger represents a heart rate within a normal range. The heart rate within a normal range refers to: an average range of the heart rate inherent to the vehicle passenger in a stable state, and a general average heart rate range. The detection unit 42 determines that the state in which the heart rate of the vehicle passenger is not within the normal range is not a stable state. For example, when the heart rate is higher than the normal range, it is a state of tension, excitement, etc., and it is often a state in which voice commands cannot be issued appropriately. In addition, when the heart rate is lower than the normal range, it means that the speaker has some physical abnormality, and is often in a state where he cannot issue voice commands appropriately.

[0045] The detection unit 42 obtains information related to the emotions of the passengers of the vehicle from the second image data. The detection unit 42 detects the face of the passenger of the vehicle in the second image data, and detects the movement of the parts constituting the face, and infers the emotions of the passenger of the vehicle with reference to a learning model that performs machine learning on the relationship between the movement of the parts constituting the face, the overall facial expression, and the emotions. The detection unit 42 infers emotions such as "happy", "calm", "nervous", "surprised", and "angry" based on image analysis. In the present embodiment, the detection unit 42 only needs to be able to detect the states of "nervous", "surprised", and "angry" as unstable emotions.

[0046] In the present embodiment, the detection unit 42 detects biological information that can determine whether the biological information of the vehicle's passengers is in a stable state. The unstable state in the present embodiment refers to a state in which a voice command cannot be issued in a stable state, such as a nervous state, a frightened state, an angry state, etc. In such a case, the heart rate of the vehicle's passengers is on an upward trend. In addition, in such a case, it is also detected by detecting the emotions of the vehicle's passengers. In addition, the stable state in the present embodiment refers to a state other than the above-mentioned unstable state. In such a case, the heart rate of the vehicle's passengers is on a stable trend. In addition, in such a case, it is also detected by detecting the emotions of the vehicle's passengers.

[0047] The voice command receiving unit 44 receives the voice command by recognizing the voice collected by the microphone 18. For example, the voice command receiving unit 44 performs a sound source separation process and a voice recognition process on the voice collected by the microphone 18, and recognizes the voice command for starting the event recording. The voice command for starting the event recording is, for example, "Start recording (Ro·Ku·Ga·Ka·I·Shi)". When the voice command receiving unit 44 recognizes six consecutive syllables of "Ro·Ku·Ga·Ka·I·Shi" in the voice collected by the microphone 18, it outputs a control signal for starting the event recording process to the recording control unit 36. Alternatively, when the voice command receiving unit 44 recognizes the voice representing the word "Ro·Ku·Ga·Ka·I·Shi" in the voice collected by the microphone 18, it outputs a control signal for starting the event recording process to the recording control unit 36. The voice command receiving unit 44 changes the recognition rate of the voice for determining whether the voice command is acquired, depending on whether the biological information of the person who issues the voice command, that is, the passenger of the vehicle, indicates a stable state.

[0048] When the biological information of the vehicle passenger is in a stable state, if all the syllables of the continuous six syllables of "Ro·Ku·Ga·Ka·I·Shi" are consistent, the voice command receiving unit 44 determines that the voice command has been acquired. The voice command receiving unit 44 sets the first threshold of the recognition rate for determining that the voice command has been acquired to 90%, for example. In this case, the voice command receiving unit 44 determines that the voice command has been acquired when it can recognize more than 90% of the six syllables of "Ro·Ku·Ga·Ka·I·Shi".

[0049] When the biological information of the vehicle passenger is not in a stable state, the voice command receiving unit 44 determines that the voice command has been acquired when more than 5 syllables of the 6 consecutive syllables of "Ro·Ku·Ga·Ka·I·Shi" are consistent. In this case, the voice command receiving unit 44 sets the recognition rate for determining that the voice command has been acquired to a second threshold value lower than the first threshold value. The voice command receiving unit 44 sets the second threshold value to 80%, for example. In this case, the voice command receiving unit 44 determines that the voice command has been acquired when it can recognize more than 80% of the 6 consecutive syllables of "Ro·Ku·Ga·Ka·I·Shi". That is, in a situation where the voice command cannot be properly recognized, such as when the biological information of the vehicle passenger is not in a stable state, even if the passenger's words cannot be fully recognized, the voice command can be properly recognized by determining that the voice command has been issued.

[0050] In addition, when the biological information of the vehicle passenger is in a stable state, the voice command receiving unit 44 sets the first threshold of the recognition rate for determining that the voice command has been obtained, for example, to 90%, based on the consistency rate between the acoustic model of the voice waveform of the word "Ro·Ku·Ga·Ka·I·Shi" and the waveform of the input voice. In this case, the voice command receiving unit 44 determines that the voice command has been obtained when the consistency rate between the acoustic model of the voice waveform of the word "Ro·Ku·Ga·Ka·I·Shi" and the waveform of the input voice is 90% or more.

[0051] In addition, when the biological information of the vehicle passenger is not in a stable state, the voice command receiving unit 44 sets the consistency rate between the acoustic model of the voice waveform of the word "Ro·Ku·Ga·Ka·I·Shi" and the waveform of the input voice as a second threshold, for example, to 80%, and the second threshold is lower than the first threshold of the recognition rate for determining that the voice command has been acquired. In this case, the voice command receiving unit 44 determines that the voice command has been acquired when the consistency rate between the acoustic model of the voice waveform of the word "Ro·Ku·Ga·Ka·I·Shi" and the waveform of the input voice is 80% or more. That is, when the biological information of the vehicle passenger is not in a stable state, the passenger's voice is easily recognized as a voice command.

[0052] The event detection unit 46 detects an event based on the acceleration applied to the vehicle. The event detection unit 46 detects an event based on the detection result of the acceleration sensor 20. The event detection unit 46 detects that an event has occurred when the acceleration information is equal to or greater than a preset threshold value indicating a collision of the vehicle.

[0053] The operation control unit 48 acquires operation information of the operation received by the operation unit 22. For example, the operation control unit 48 acquires save operation information indicating a manual save operation of the video data, playback operation information indicating a playback operation, or delete operation information indicating a delete operation of the video data, and outputs a control signal. For example, the operation control unit 48 acquires end operation information indicating an operation of ending loop recording, and outputs a control signal.

[0054] The operation control unit 48 accepts an event recording operation based on the voice command recognized and accepted by the voice command accepting unit 44 .

[0055] The position information acquisition unit 50 acquires position information indicating the current position of the vehicle. The position information acquisition unit 50 calculates the position information of the current position of the vehicle based on the GNSS signal received by the GNSS receiving unit 24 by a known method.

[0056] (Processing by the control unit)

[0057] use Figure 2 , describing the processing flow of the control unit of the first embodiment. Figure 2 This is a flowchart showing the flow of processing by the control unit according to the first embodiment. Figure 2 The flowchart shown is started by starting the engine or the like of the vehicle on which the recording device 1 is mounted.

[0058] With the start of the process, the control unit 26 starts normal recording (step S10). Specifically, the recording control unit 36 ​​sends the image data captured by the first camera 10 and the second camera 12 to the buffer memory 32, and starts the process of generating an image file according to the image of a predetermined period such as every 60 seconds, and recording it in the recording unit 14, and then proceeds to step S12.

[0059] The voice command receiving unit 44 determines whether the biological information of the speaker of the voice command indicates a stable state (step S12). The speaker of the voice command may be limited to the driver of the vehicle, or may be a passenger other than the driver of the vehicle. Specifically, the voice command receiving unit 44 obtains the biological information of the speaker of the voice command from the detection unit 42, and determines whether the biological information indicates a stable state. When it is determined that the biological information of the speaker of the voice command indicates a stable state (step S12; yes), the process proceeds to step S14. When it is not determined that the biological information of the speaker of the voice command indicates a stable state (step S12; no), the process proceeds to step S18.

[0060] If the determination result is "yes" in step S12, the voice command receiving unit 44 determines whether a voice command is obtained from the vehicle passenger through the microphone 18 (step S14). If it is determined that a voice command is obtained (step S14; yes), the process proceeds to step S16. If it is not determined that a voice command is obtained (step S14; no), the process proceeds to step S24.

[0061] If the judgment is "yes" in step S14, the voice command receiving unit 44 determines whether the recognition rate of the acquired voice command is above the first threshold (step S16). If it is determined that the recognition rate of the voice command is above the first threshold (step S16; yes), the process proceeds to step S22. If it is not determined that the recognition rate of the voice command is above the first threshold (step S16; no), the process proceeds to step S24.

[0062] If the determination result in step S12 is "No", the voice command receiving unit 44 determines whether a voice command has been acquired from the vehicle passenger through the microphone 18 (step S18). If it is determined that a voice command has been acquired (step S18; Yes), the process proceeds to step S20. If it is not determined that a voice command has been acquired (step S18; No), the process proceeds to step S24.

[0063] If the judgment is yes in step S18, the voice command receiving unit 44 determines whether the recognition rate of the acquired voice command is greater than the second threshold value (step S20). If it is determined that the recognition rate of the voice command is greater than the second threshold value (step S20; yes), the process proceeds to step S22. If it is not determined that the recognition rate of the voice command is greater than the second threshold value (step S20; no), the process proceeds to step S24.

[0064] In step S14 and step S18, in addition to determining whether a voice instruction is obtained, it is also possible to determine whether the obtained voice instruction is an urgent or immediacy voice instruction. In other words, in step S14 and step S18, it is determined whether an urgent or immediacy voice instruction is obtained. An urgent or immediacy voice instruction refers to a voice instruction that requests an operation on a function that requires the action to be started without delay by accepting a voice instruction. For example, an urgent or immediacy voice instruction in the recording device 1 refers to a voice instruction that instructs event recording.

[0065] If the answer is "yes" in step S16 or "yes" in step S20, the recording control unit 36 ​​saves the event data in the recording unit 14 (step S22). Specifically, the recording control unit 36 ​​saves the first video data before and after the time when the voice command receiving unit 44 obtains the voice command as event data in the recording unit 14, and proceeds to step S24.

[0066] If the determination result is "No" in steps S14 to S20, or after step S22, the control unit 26 determines whether to terminate the process (step S24). Specifically, the control unit 26 determines that the process is to be terminated when the operation unit 22 receives an operation to turn off the power supply or an operation to terminate the process, or when the engine of the vehicle on which the recording device 1 is installed is turned off. If the determination result is "No" (step S24; Yes), the process is terminated. Figure 2 If it is not determined to be the end of the process (step S24; No), proceed to step S12.

[0067] As described above, the first embodiment changes the recognition rate for recognizing speech as a voice command and saves event data when the biological information of the vehicle passenger indicates a stable state and when it does not indicate a stable state. In the first embodiment, when the biological information of the vehicle passenger, that is, the speaker of the voice command, does not indicate a stable state, the recognition rate is lowered compared to the case of indicating a stable state to perform event data saving processing. Thus, the first embodiment can appropriately save event data based on voice commands even when the passenger is not in a state where the voice command can be issued appropriately.

[0068] exist Figure 2 In the process shown, after determining whether the biological information of the speaker of the voice instruction indicates a stable state (step S12), it is determined whether the voice instruction is acquired (step S14, step S18). The first embodiment is not limited to such processing, as long as it is possible to determine whether the biological information of the speaker of the voice instruction indicates a stable state when the voice instruction is acquired. In other words, it is sufficient to determine whether the voice instruction is uttered in a stable state. In addition, the judgment of whether the biological information of the speaker of the voice instruction indicates a stable state can also be determined based on the biological information of the time point when the voice instruction is acquired, the time immediately before the voice instruction is acquired, etc.

[0069] [Second embodiment]

[0070] The second embodiment is described. The voice command receiving device in the second embodiment is a general device that uses voice commands for operation, and can be applied to, for example, household devices such as smart speakers and television receivers, information devices such as smart phones, tablet terminals, and PCs, navigation devices used in vehicles, infotainment systems, etc.

[0071] use Figure 3 , an example of the structure of the voice command receiving device involved in the second embodiment is described. Figure 3 This is a block diagram showing a configuration example of a voice command accepting device according to the second embodiment.

[0072] like Figure 3 As shown, the voice instruction receiving device 100 includes a voice instruction receiving unit 144, a detection unit 142 and an execution control unit 150. The voice instruction receiving device 100 has, for example, an information processing device such as a CPU, an MPU and a storage device such as a RAM or a ROM. The voice instruction receiving device 100 executes the program according to the present disclosure. The voice instruction receiving device 100 can also be implemented by an integrated circuit such as an ASIC, an FPGA, etc. The voice instruction receiving device 100 can also be implemented by a combination of hardware and software.

[0073] The voice command receiving device 100 acquires voice and images from the microphone 118 and the camera 110. The microphone 118 and the camera 110 may also be components of the voice command receiving device 100.

[0074] The microphone 118 picks up the voice uttered by the speaker. The microphone 118 outputs the voice related to the picked up voice to the voice command receiving device 100. The microphone 118 may be integrally formed with the voice command receiving device 100 or may be separately formed.

[0075] The camera 110 captures a picture of the speaker who gives the voice command. The camera 110 captures at least the face of the speaker. The camera 110 outputs image data related to the captured image to the voice command receiving device 100. The camera 110 may be integrally formed with the voice command receiving device 100 or may be separately formed.

[0076] The voice instruction accepting unit 144 accepts a voice instruction. For example, the voice instruction accepting unit 144 accepts a voice instruction by recognizing a voice picked up by the microphone 118 .

[0077] The detection unit 142 detects conditions in the environment where the voice command is issued that lead to a situation where the voice command is not properly recognized. In this embodiment, the detection unit 142 detects biological information of the person who issued the voice command. The detection unit 142 obtains information related to the heartbeat and emotion of the person who issued the voice command based on the image data captured by the camera 110, for example.

[0078] When the voice command accepting unit 144 accepts a voice command, the execution control unit 106 executes a function corresponding to the accepted voice command.

[0079] The voice command receiving unit 144 determines whether the biometric information of the person who issued the voice command detected by the detection unit 142 indicates a stable state. The voice command receiving unit 144 changes the recognition rate of the voice command according to the fact that the biometric information of the speaker of the voice command is in a stable state based on the detection result of the detection unit 142 to receive the voice command. For example, when the voice command receiving unit 144 determines that the biometric information of the speaker of the voice command indicates a stable state, the voice command is accepted at a recognition rate above a first threshold. When the voice command receiving unit 144 determines that the biometric information of the speaker of the voice command does not indicate a stable state, the voice command is accepted at a recognition rate above a second threshold lower than the first threshold.

[0080] The voice command receiving unit 144 receives the voice command with a recognition rate of more than the second threshold value for the voice command with high urgency or immediacy. In the second embodiment, the voice command with high urgency or immediacy refers to the voice command for the function that does not want to be delayed from the operation time point relative to the start or end of the execution of the function or has adverse effects or risks due to the delay, such as emergency calls, emergency communications, and instructions for starting recording of broadcast content, and stopping instructions for functions with high continuous risks.

[0081] (Processing of Voice Command Accepting Device)

[0082] use Figure 4 , describing the processing flow of the voice command receiving device involved in the second embodiment. Figure 4 This is a flowchart showing the processing flow of the voice command accepting device involved in the second embodiment.

[0083] The detection unit 142 starts detecting the biological information of the speaker of the voice command (step S40), and the process proceeds to step S42. Specifically, the detection unit 142 starts detecting the biological information of a person existing in the shooting direction of the camera 110, such as in front of the voice command receiving device 100.

[0084] The voice command receiving unit 144 determines whether the speaker's biological information indicates a stable state (step S42). Specifically, the voice command receiving unit 144 detects the speaker's heartbeat and emotions, and determines whether the speaker's heartbeat and emotions indicate a stable state. When the speaker's heart rate is a normal heart rate and the speaker's emotion is not "nervous", "surprised" or "angry", the voice command receiving unit 144 determines that the speaker's biological information is in a stable state. When it is determined that the speaker's biological information indicates a stable state (step S42; yes), the process proceeds to step S44. When it is determined that the speaker's biological information does not indicate a stable state (step S42; no), the process proceeds to step S48.

[0085] If the determination result is "yes" in step S42, the voice command receiving unit 144 determines whether a voice command has been acquired from the speaker through the microphone 118 (step S44). If it is determined that a voice command has been acquired (step S44; yes), the process proceeds to step S46. If it is not determined that a voice command has been acquired (step S44; no), the process proceeds to step S54.

[0086] If the judgment is "yes" in step S44, the voice command receiving unit 144 determines whether the recognition rate of the acquired voice command is above the first threshold (step S46). If it is determined that the recognition rate of the voice command is above the first threshold (step S46; yes), the process proceeds to step S52. If it is not determined that the recognition rate of the voice command is above the first threshold (step S46; no), the process proceeds to step S54.

[0087] If the determination result in step S42 is "No", the voice command receiving unit 144 determines whether a voice command has been acquired from the speaker via the microphone 118 (step S48). If it is determined that a voice command has been acquired (step S48; Yes), the process proceeds to step S50. If it is determined that a voice command has not been acquired (step S48; No), the process proceeds to step S54.

[0088] If the judgment is "yes" in step S48, the voice command receiving unit 144 determines whether the recognition rate of the acquired voice command is greater than the second threshold value (step S50). If it is determined that the recognition rate of the voice command is greater than the second threshold value (step S50; yes), the process proceeds to step S52. If it is not determined that the recognition rate of the voice command is greater than the second threshold value (step S50; no), the process proceeds to step S54.

[0089] In step S44 and step S48, in addition to determining whether a voice instruction is acquired, it is also possible to determine whether the acquired voice instruction is a voice instruction with high urgency or immediacy.

[0090] If it is determined as "yes" in step S46 or "yes" in step S50, the execution control unit 150 executes the function for the voice command (step S52), and then proceeds to step S54.

[0091] If the result of the judgment in step S44 to step S50 is "No", or after step S52, the voice command receiving device 100 determines whether to end the process (step S54). Specifically, when the voice command receiving device 100 receives an operation to turn off the power or an operation to end the process, the voice command receiving device 100 determines that the process is to be ended. If the judgment is to end the process (step S54; Yes), the process is ended. Figure 4 If it is not determined to be the end of the processing (step S54; No), proceed to step S42.

[0092] As described above, the second embodiment changes the recognition rate for recognizing speech as a speech command when the biological information of the speaker of the speech command indicates a steady state and when it does not indicate a steady state, and executes the function for the speech command. In the second embodiment, when the biological information of the speaker does not indicate a steady state, the recognition rate is lowered compared to when it indicates a steady state to execute the function for the speech command. Thus, the second embodiment can appropriately execute the function for the speech command even when the biological information of the speaker of the speech command does not indicate a steady state, in other words, when the speaker is not in a state where the speech command can be issued appropriately.

[0093] The embodiments of the present disclosure are described above, but the present disclosure is not limited to the contents of these embodiments. In addition, the above-mentioned constituent elements include elements that can be easily thought of by those skilled in the art, substantially the same elements, and elements of the so-called equivalent scope. Moreover, the above-mentioned constituent elements can be appropriately combined. Moreover, various omissions, substitutions or changes of constituent elements can be made within the scope of the gist of the above-mentioned embodiments.

[0094] Possibility of industrial application

[0095] The voice command receiving device and voice command receiving method disclosed in the present invention can be applied to household devices such as smart speakers and television receivers, information devices such as smart phones, tablet terminals, PCs, navigation devices for vehicles, infotainment systems, etc.

[0096] Explanation of symbols

[0097] 1 Recording device

[0098] 10First Camera

[0099] 12. Second Camera

[0100] 14 Records Department

[0101] 16 Display unit

[0102] 18 Microphones

[0103] 20 Acceleration sensor

[0104] 22 Operation Department

[0105] 24GNSS receiving unit

[0106] 26 Control unit (recording control device)

[0107] 30 Image data acquisition unit

[0108] 32 Buffer memory

[0109] 34 Image Data Processing Department

[0110] 36 Recording Control Unit

[0111] 38 playback control unit

[0112] 40 Display control unit

[0113] 42 Detection Department

[0114] 44 Voice command receiving unit

[0115] 46 Incident Detection Department

[0116] 48 Operation Control Department

[0117] 50 Position information acquisition unit

[0118] 100 Voice command receiving device

[0119] 110 Camera

[0120] 118 Microphones

[0121] 144 Voice command receiving unit

[0122] 142 Detection Department

[0123] 150Execution Control Unit

Claims

1. A voice command receiving device, comprising: A voice command receiving unit, receiving a voice command; a detection unit for detecting biological information of a person issuing the voice command; as well as an execution control unit, which executes a function corresponding to the received voice command when the voice command receiving unit receives the voice command, The voice command receiving unit accepts voice commands when the recognition rate of voice commands acquired by the voice command receiving unit is above a first threshold value when the detection unit determines that the biological information of the person indicates a stable state; and accepts voice commands when the recognition rate of voice commands acquired by the voice command receiving unit is above a second threshold value when the detection unit determines that the biological information of the person indicates a non-steady state, and the second threshold value is lower than the first threshold value.

2. The voice command receiving device according to claim 1, wherein: The detection unit acquires information related to the person's heartbeat, When the voice command receiving unit determines that the information related to the person's heartbeat indicates a heart rate within the normal range, the voice command receiving unit accepts the voice command when the recognition rate of the voice command obtained by the voice command receiving unit is above a first threshold value; when the information related to the person's heartbeat is determined to indicate a heart rate outside the normal range, the voice command is accepted when the recognition rate of the voice command obtained by the voice command receiving unit is above a second threshold value, and the second threshold value is lower than the first threshold value.

3. The voice command receiving device according to claim 1, wherein: The detection unit acquires information related to the person's emotions, When the voice command receiving unit determines that the information related to the person's emotion indicates a relatively stable emotion, the voice command receiving unit accepts the voice command when the recognition rate of the voice command obtained by the voice command receiving unit is above a first threshold value; when the information related to the person's emotion is determined to indicate an unstable emotion, the voice command receiving unit accepts the voice command when the recognition rate of the voice command obtained by the voice command receiving unit is above a second threshold value, and the second threshold value is lower than the first threshold value.

4. The voice command receiving device according to claim 1, wherein: The voice command accepting unit accepts a voice command with high urgency or immediacy when a recognition rate of the voice command acquired by the voice command accepting unit is greater than a second threshold value, and the second threshold value is lower than the first threshold value.

5. The voice instruction accepting device according to any one of claims 1 to 3, wherein: The voice command receiving device is a vehicle recording control device used in a vehicle, The voice command receiving device includes an image data acquisition unit, the image data acquisition unit acquires first image data captured by a first imaging unit, the first imaging unit captures the periphery of the vehicle, The voice command receiving unit receives an event recording instruction based on a voice command, The execution control unit stores the first video data including a time point when the event recording instruction is received as event data when the voice instruction receiving unit receives the event recording instruction based on the voice instruction.

6. A method for accepting a voice command, the method being executed by a voice command accepting device, the method comprising the following steps: Detecting biological information of the person issuing the voice command; If it is determined that the biometric information indicates a stable state, the voice command is accepted when the recognition rate of the voice command is above a first threshold value, and if it is determined that the biometric information indicates a non-steady state, the voice command is accepted when the recognition rate of the voice command is above a second threshold value, the second threshold value being lower than the first threshold value; and When the voice command is received, a function corresponding to the received voice command is executed.

Images