Vehicle recording control device and recording method

Abstract

To enable appropriate event recording using a voice command.SOLUTION: A vehicle recording control device 200 according to the present invention, includes an imaging data acquisition unit 202 that acquires first imaging data captured by a first camera for imaging the periphery of a vehicle and second imaging data captured by a second camera for imaging a cabin of the vehicle, a visual line detection unit 214 that detects a direction of a visual line of a passenger of the vehicle from the second imaging data, a voice recognition unit 216 that recognizes a voice command for instructing event recording, an output control unit 212 that, when the visual line detection unit 214 detects that the visual line of the passenger is directed to a display unit 108a, outputs information indicating the voice command to the passenger, and a recording control unit 208 that, when the voice recognition unit 216 recognizes the voice command, stores the first imaging data as event data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a recording control device and a recording method for a vehicle, and more particularly to a recording control device and a recording method for a vehicle used in a vehicle such as an automobile. [Background technology]

[0002] Some vehicle recording devices, so-called drive recorders, record events using voice commands in addition to detecting impacts using acceleration sensors (for example, Non-Patent Document 1). Event recording using voice commands allows safe event recording without the need to operate a touch panel or the like while driving, such as when recording an accident in which the driver was not involved. Patent Document 1 discloses a drive recorder that records events by issuing voice instructions in response to event detection based on acceleration. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-154904 [Non-patent literature]

[0004] [Non-Patent Document 1] DRV-MR760 [searched September 24, 2021], Internet<https: / / www.kenwood.com / jp / car / drive-recorders / products / drv-mr760 / > Summary of the Invention [Problem to be solved by the invention]

[0005] Even if a drive recorder is equipped with a function for recording events via voice commands, because there are few opportunities to record events, the driver may not immediately remember what voice command to use to record an event. Alternatively, the driver may mistakenly utter a similar phrase, preventing the action specified by the voice command from being performed. In such cases, the timing to record an event may be missed, or the driver may have to use their hands to perform an operation to record an event while driving the vehicle, which may impair safety.

[0006] The present invention provides a vehicle recording control device and a recording method that can appropriately record events using voice commands. [Means for solving the problem]

[0007] The vehicle recording control device of the present invention comprises an imaging data acquisition unit that acquires first imaging data captured by a first camera that captures images of the area around the vehicle and second imaging data captured by a second camera that captures images of the interior of the vehicle, a gaze detection unit that detects the direction of the gaze of an occupant of the vehicle from the second imaging data, a voice recognition unit that recognizes a voice command that instructs event recording, an output control unit that, when the gaze detection unit detects that the occupant's gaze is directed toward the display unit, outputs information indicating the voice command to the occupant, and a recording control unit that, when the voice recognition unit recognizes the voice command, saves the first imaging data as event data.

[0008] The recording method of the present invention is performed by a vehicle recording control device, and includes the following steps: a photographing data acquisition step of acquiring first photographing data photographed by a first camera that photographs the area around the vehicle and second photographing data photographed by a second camera that photographs the interior of the vehicle; a gaze detection step of detecting the direction of the gaze of an occupant of the vehicle from the second photographing data; a voice recognition step of recognizing a voice command that instructs event recording; an output step of outputting information indicating the voice command to the occupant when it is detected that the occupant's gaze is directed toward the display unit; and a recording step of saving the first photographing data as event data when the voice command is recognized. [Effects of the Invention]

[0009] According to the present invention, event recording can be performed appropriately using voice commands. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a configuration of a vehicle-mounted recording device according to a first embodiment of the present invention. [Figure 2] 4 is a flowchart showing an example of processing performed by the vehicle recording control device according to the first embodiment of the present invention. [Figure 3] 10 is a flowchart showing an example of processing performed by a vehicle recording control device according to a second embodiment of the present invention. [Figure 4] 10 is a flowchart showing an example of processing performed by a vehicle recording control device according to a third embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the following description, a drive recorder used in a moving vehicle, such as an automobile, will be described as an example of a vehicle recording device according to this embodiment. However, the present invention is not limited to this. For example, the present invention can be applied to various moving objects, such as various vehicles, such as motorcycles and bicycles, trains, ships, robots, and even people. Furthermore, the present invention is not limited to the following embodiment.

[0012] A first embodiment of the present invention will be described below with reference to Figures 1 and 2. Figure 1 is a block diagram showing the configuration of a vehicle recording device 100 according to the first embodiment of the present invention. The vehicle recording device 100 is a so-called drive recorder. The vehicle recording device 100 is attached to the upper part of the windshield of the vehicle, detects an impact equivalent to an accident, that is, an acceleration equivalent to an accident applied to the vehicle, and saves video and other images taken when the accident is detected as event data.

[0013] The vehicular recording device 100 shown in FIG. 1 is not limited to a device mounted on a vehicle as a standalone device, but may be configured in various ways, such as a configuration realized as a function of a navigation device or a configuration pre-mounted on a vehicle.

[0014] 1, the vehicle recording device 100 includes a control unit 200 as a vehicle recording control device, a first camera 102, a second camera 104, a recording unit 106, an output unit 108, a microphone 110, an acceleration sensor 112, an operation unit 114, and a GNSS (Global Navigation Satellite System) receiving unit 116. The output unit 108 is made up of a display unit 108a and an audio output unit 108b.

[0015] The control unit 200 is composed of one or more central processing units (CPUs), graphics processing units (GPUs), random access memories (RAMs), etc. that perform various data processing, and executes various processes according to programs. The control unit 200 at least includes, as its configuration or realized functions, a captured image data acquisition unit 202, a buffer memory 204, a captured image data processing unit 206, a recording control unit 208, a playback control unit 210, an output control unit 212, a gaze detection unit 214, a voice recognition unit 216, an event detection unit 218, an operation control unit 220, and a position information acquisition unit 222. The output control unit 212 is composed of a display control unit 212a and an audio output control unit 212b.

[0016] The control unit 200 is a recording control device for a vehicle that executes the recording method according to the present invention, and is a computer that runs the program according to the present invention.

[0017] The first camera 102 captures images of the surroundings of a vehicle, which is a moving body. The first camera 102 may be provided integrally with the vehicle recording device 100, or may be connected as a separate device via a wired or wireless connection. The first camera 102 includes a lens, an imaging element, an AD (Analog to Digital) conversion element, etc., which are not shown. The first camera 102 outputs captured first image data to the image data acquisition unit 202.

[0018] The first camera 102 is provided inside the vehicle cabin above the windshield of the vehicle recording device 100, and if it is provided integrally with the vehicle recording device 100, it is provided on the windshield side of the vehicle recording device 100.

[0019] The first camera 102 is not limited to a single camera, but may be configured with multiple cameras. For example, the first camera 102 may be configured with multiple cameras in any combination that capture images of the front, rear, sides, interior, etc. of the vehicle, or may be all cameras.

[0020] The second camera 104 captures images of the interior of the vehicle, which is a moving body. The second camera 104 may be provided integrally with the vehicular recording device 100, or may be connected as a separate device via a wired or wireless connection. The second camera 104 includes a lens, an image sensor, an AD conversion element, etc., which are not shown. The second camera 104 outputs captured second captured data to the captured data acquisition unit 202.

[0021] The second camera 104 is provided at a position where it can photograph the face of the vehicle occupant and detect the line of sight of the vehicle occupant. The second camera 104 is provided on the interior side of the vehicle above the windshield of the vehicle recording device 100, and when the second camera 104 is provided integrally with the vehicle recording device 100, it is provided on the interior side of the vehicle recording device 100.

[0022] When second camera 104 is provided at a position where the line of sight of the driver, who is a vehicle occupant, can be detected, it is provided, for example, on the instrument panel of the vehicle so as to be able to capture an image in the direction of the face of the vehicle driver.When second camera 104 is provided at a position where the line of sight of other vehicle occupants in addition to the driver can be detected, it may be provided, for example, near or inside a rearview mirror of the vehicle so as to be able to capture an image of the entire interior of the vehicle.

[0023] Second camera 104 is not limited to a single camera and may be configured with multiple cameras. Second camera 104 may also be a camera that captures visible light, a camera that captures infrared light, or a combination of these.

[0024] 1, the first camera 102 and the second camera 104 are described as different cameras, but a single camera may be used if it can capture the range captured by the first camera 102 and the second camera 104. The single camera is, for example, an all-around camera capable of capturing images of 180 degrees or 360 degrees.

[0025] The image data captured by the first camera 102 and the second camera 104 is moving image data. The image data may include audio data in addition to the moving image data.

[0026] Recording unit 106 is a non-volatile memory, such as a memory card, for recording first captured data captured by first camera 102 under the control of recording control unit 208. The first captured data and event data recorded in recording unit 106 are played back under the control of playback control unit 210. Recording unit 106 may be replaced with a separate recording / playback device that includes recording control unit 208 and playback control unit 210 in addition to recording unit 106. Recording unit 106 may be provided as an integrated recording / playback device, or may be connected as a separate device via wire or wirelessly.

[0027] The output unit 108 outputs information indicating a voice command to a vehicle occupant. The output unit 108 is composed of a display unit 108a and a voice output unit 108b. When the display unit 108a operates as the output unit 108, the display unit 108a displays information such as text information indicating the voice command on the display unit 108a as an output of information indicating the voice command to the vehicle occupant. When the voice output unit 108b operates as the output unit 108, the voice output unit 108b outputs a voice indicating the voice command.

[0028] The display unit 108a is a display device that displays various information under the control of the display control unit 212a. The display unit 108a includes a display panel such as a liquid crystal panel or an organic EL (organic electro-luminescence) panel. The display unit 108a may be provided integrally with the vehicle recording device 100, or may be connected as a separate device by wire or wirelessly. The display unit 108a may be connected wirelessly to another device that includes the display control unit 212a. The display unit 108a is located in a position that is easily visible to vehicle occupants.

[0029] The display unit 108a is provided on the interior side of the vehicle above the windshield of the vehicle when the vehicle recording device 100 is mounted thereon, and when the display unit 108a is provided integrally with the vehicle recording device 100, the display unit 108a is provided on the interior side of the vehicle recording device 100. In this case, the display unit 108a is provided near the second camera 104.

[0030] The audio output unit 108b outputs various sounds under the control of the audio output control unit 212b. Specifically, the audio output unit 108b is configured with an audio amplifier, a speaker, etc. The audio output unit 108b outputs the same sound as the audio command as information indicating the audio command to the passengers of the vehicle.

[0031] When outputting information indicating a voice command, the voice output unit 108b outputs the voice so that the voice is not mistakenly recognized as a voice command. For this reason, the voice output unit 108b is configured to be easy for the vehicle occupants to hear and not input to the microphone 110. The voice output unit 108b is, for example, a speaker provided above the seat on which the vehicle occupant sits. Alternatively, the voice output unit 108b is a neck speaker worn by the vehicle occupant.

[0032] The microphone 110 acquires the voice uttered by a passenger such as a driver of the vehicle. The microphone 110 is provided in the second camera 104, and the voice uttered by the passenger such as a driver of the vehicle may be included in the second captured data. The microphone 110 is disposed in a position where it can acquire the voice uttered by the passenger such as a driver of the vehicle. The microphone 110 may have directionality that allows it to acquire the voice uttered by the passenger such as a driver of the vehicle.

[0033] The acceleration sensor 112 detects acceleration applied to the vehicle recording device 100 or the vehicle. The acceleration sensor 112 is, for example, a three-axis acceleration sensor that detects acceleration applied in the x-axis direction (front-rear direction of the vehicle), the y-axis direction (left-right direction of the vehicle), and the z-axis direction (up-down direction of the vehicle). The acceleration sensor 112 outputs the detected acceleration information to the event detection unit 218. The acceleration sensor 112 may be provided integrally with the vehicle recording device 100, or may be connected as a separate unit via a wire or wirelessly. Alternatively, the acceleration sensor 112 may be a separate unit including the acceleration sensor 112 and the event detection unit 218.

[0034] The operation unit 114 is an interface that accepts operations on the vehicular recording device 100 and outputs the accepted operation information to the operation control unit 220. The operation unit 114 is various buttons provided on the vehicular recording device 100 or a touch panel provided on the display unit 108a, and accepts operations by the user. The operation unit 114 may also accept operations from other wirelessly connected devices. For example, the operation unit 114 accepts an operation by the user to start event recording. The operation unit 114 also accepts an operation by the user to play back event data.

[0035] The GNSS receiver 116 is configured with, for example, a receiving device and an antenna that receives signals from GPS (Global Positioning System) satellites and the like. The GNSS receiver 116 outputs the received signals to the location information acquisition unit 222. The GNSS receiver 116 may be provided integrally as part of the vehicular recording device 100, or may be connected as a separate unit by wire or wirelessly. Furthermore, the GNSS receiver 116 may be a separate unit that includes the GNSS receiver 116 and the location information acquisition unit 222.

[0036] The photographing data acquisition unit 202 acquires first photographing data photographed by the first camera 102 and second photographing data photographed by the second camera 104. The photographing data acquisition unit 202 outputs the first photographing data acquired from the first camera 102 to the buffer memory 204. The photographing data acquisition unit 202 outputs the second photographing data acquired from the second camera 104 to the gaze detection unit 214.

[0037] The image capturing data acquisition unit 202 also functions as an image capturing control unit that controls image capturing by the first camera 102 and the second camera 104. The image capturing data acquisition unit 202 causes the first camera 102 and the second camera 104 to capture images as necessary while the vehicular recording device 100 is operating.

[0038] The buffer memory 204 is an internal memory provided in the control unit 200, and temporarily stores the photographic data for a certain period of time acquired by the photographic data acquisition unit 202 while updating it.

[0039] The shooting data processing unit 206 converts the shooting data temporarily stored in the buffer memory 204 into an arbitrary file format, such as MP4 format, encoded using an arbitrary codec, such as H.264 or MPEG-4 (Moving Picture Experts Group). The shooting data processing unit 206 generates shooting data as files for a certain period of time from the shooting data temporarily stored in the buffer memory 204. As a specific example, the shooting data processing unit 206 generates 60 seconds of shooting data as files in the order of shooting from the shooting data temporarily stored in the buffer memory 204. The shooting data processing unit 206 outputs the generated shooting data to the recording control unit 208. The shooting data processing unit 206 also outputs the generated shooting data to the display control unit 212a. The period of the shooting data generated as files is set to 60 seconds as an example, but is not limited to this.

[0040] The recording control unit 208 controls the recording unit 106 to record the shooting data filed by the shooting data processing unit 206. While the vehicular recording device 100 is operating, the recording control unit 208 records the shooting data filed by the shooting data processing unit 206 in the recording unit 106 as overwritable shooting data. When the recording capacity of the recording unit 106 reaches its upper limit, the recording control unit 208 overwrites and records new shooting data from a recording area in which old shooting data is recorded among the overwritable shooting data recorded in the recording unit 106. The shooting data recorded by the recording control unit 208 in the recording unit 106 is the first shooting data captured by the first camera 102.

[0041] The recording control unit 208 continues to record the image capture data based on the first image capture data in the recording unit 106 while the vehicular recording device 100 is operating. This type of recording process is called normal recording or loop recording.

[0042] When the event detection unit 218 determines that an event has occurred, the recording control unit 208 stores the first captured data for a predetermined period including the time point at which the event occurred as event data that is prohibited from being overwritten. The event data is, for example, captured data for 15 seconds before and after the time point at which the event occurred.

[0043] When the operation control unit 220 receives an operation to perform event recording, the recording control unit 208 saves the first captured data for a predetermined period including the time when the operation to perform event recording was performed as event data that is prohibited from being overwritten. The event data is, for example, captured data for 15 seconds before and after the time when the operation to perform event recording was performed.

[0044] When the voice recognition unit 216 recognizes a voice command to perform event recording, the recording control unit 208 saves the first captured data for a predetermined period including the time when the voice command to perform event recording was recognized as event data that is prohibited from being overwritten. The event data is, for example, captured data for 15 seconds before and after the time when the voice command to perform event recording was recognized.

[0045] The recording control unit 208 may store the event data in any manner. For example, an overwrite prohibition flag may be added to the header or payload of a section of the first captured data that is to be overwritten, and the data may be stored in the recording unit 106. Alternatively, the section of the first captured data that is to be overwritten may be stored in an overwrite prohibition area of ​​the recording unit 106. Alternatively, the section of the first captured data that is to be overwritten may be transmitted to another device and stored therein.

[0046] The playback control unit 210 controls playback of the shooting data and event data recorded in the recording unit 106. The playback control unit 210 plays back not only the overwritable shooting data recorded in the recording unit 106 but also the event data that is not overwritable, and outputs the data to the display control unit 212a.

[0047] The output control unit 212 performs processing to output various information to a vehicle occupant. Specifically, when the emotion detection unit 224 detects that the emotion of the vehicle occupant is negative, the output control unit 212 outputs information indicating a voice command to the occupant. The output control unit 212 is composed of a display control unit 202a and a voice output control unit 212b.

[0048] The display control unit 212a controls the display unit 108a to display various information on the display unit 108a. When the display control unit 212a functions as the output control unit 212, the display control unit 212a causes the display unit 108a to display information indicating a voice command instructing event recording when the gaze detection unit 214 detects that the gaze of the passenger is directed toward the display unit 108a. When the voice command instructing event recording is "start recording," the display control unit 212a causes the display unit 108a to display information indicating the voice command, such as "start recording" or "recording started."

[0049] When the gaze detection unit 214 detects that the gaze of the passenger is directed toward the vehicular recording device 100, the display control unit 212a may cause the display unit 108a to display information indicating a voice command instructing to record an event. This is effective, for example, when the vehicular recording device 100 is integrally provided with the display unit 108.

[0050] In addition to functioning as the output control unit 212, the display control unit 212a may cause the display unit 108a to display the first video data acquired by the image capture data acquisition unit 202. In this case, the video captured by the first camera 102 is displayed on the display unit 108a. The display control unit 212a also causes the display unit 108a to display the image capture data or event data played back by the playback control unit 210.

[0051] The audio output control unit 212b causes the audio output unit 108b to output various sounds. When the audio output control unit 212b functions as the output control unit 212, the audio output control unit 212b causes the audio output unit 108b to output a sound indicating a voice command instructing event recording when the gaze detection unit 214 detects that the passenger's gaze is directed toward the display unit 108a. When the voice command instructing event recording is "start recording," the audio output control unit 212b causes the audio output unit 108b to output the sound "start recording."

[0052] In addition to functioning as the output control unit 212, the audio output control unit 212b causes the audio output unit 108b to output the audio included in the shooting data or event data played back by the playback control unit 210.

[0053] The gaze detection unit 214 detects the gaze direction of the vehicle occupant from the second captured data acquired by the captured data acquisition unit 202. When the second camera 104 is positioned to capture an image of the vehicle driver, the gaze detection unit 214 detects the gaze direction of the vehicle driver captured in the second captured data. When the second camera 104 is positioned to capture an image of the entire interior of the vehicle, the gaze detection unit 214 detects the gaze direction of the vehicle occupant other than the driver in addition to detecting the gaze direction of the vehicle driver if there is a passenger other than the driver on board.

[0054] The gaze detection unit 214 detects a person's face from the second captured data and detects the position of the eyes in the detected face. The gaze detection unit 214 detects the face and eyes using any method such as pattern matching. The gaze detection unit 214 detects the gaze direction based on the positional relationship between the inner corner of the eye and the iris detected from the image of the eye, or the positional relationship between the corneal reflex and the pupil. The gaze detection method is not limited to the above-mentioned one, and any method can be applied.

[0055] The gaze detection unit 214 detects that the gaze of the vehicle occupant is directed toward the display unit 108a. The gaze detection unit 214 detects that the gaze of the vehicle occupant is directed toward the display unit 108a based on pre-stored data indicating the positional relationship between the second camera 104 and the display unit 108a. In other words, the gaze detection unit 214 detects that the vehicle occupant is gazing at the display unit 108a. Therefore, the gaze detection unit 214 may detect that the gaze of the vehicle occupant is directed toward the display unit 108a for a predetermined period of time or more. In this case, the predetermined period of time is set to, for example, 0.5 seconds, not simply when the direction of the occupant's gaze passes in the direction of the display unit 108a, but a period of time during which it can be determined that the occupant has looked at the display unit 108a.

[0056] The line-of-sight detection unit 214 may detect that the line of sight of the vehicle occupant is directed toward the vehicle recording device 100. For example, this is effective when the vehicle recording device 100 is integrally provided with the display unit 108a.

[0057] The voice recognition unit 216 analyzes the voice acquired by the microphone 110 and recognizes the content of the utterance. The voice recognition unit 216 performs acoustic model analysis for each phoneme or word on the voice acquired by the microphone 110 and compares the voice with a phoneme model or a language model to recognize the content of the utterance. The voice recognition unit 216 recognizes voice commands to the vehicular recording device 100. Voice commands recognized by the voice recognition unit 216 include, for example, "start recording," which is a voice command for recording an event, and "take a photo," which is a voice command for taking a still image.

[0058] When the voice recognition unit 216 determines through phoneme model analysis of the voice acquired by the microphone 110 that each phoneme in succession is "roku ga kai shi," it accepts the voice command "start recording" and outputs an instruction to the recording control unit 208 to record the event.

[0059] The event detection unit 218 acquires acceleration information detected by the acceleration sensor 112, and when acceleration corresponding to an event is detected, determines that an event has been detected. The acceleration corresponding to an event is acceleration corresponding to acceleration occurring when an object, such as another vehicle, collides with the vehicle on which the vehicular recording device 100 is installed. The detection of acceleration corresponding to an event may be weighted in each of the x-axis direction, the y-axis direction, and the z-axis direction. When it is determined that an event has been detected, the event detection unit 218 outputs an instruction to the recording control unit 208 to perform event recording.

[0060] The operation control unit 220 acquires operation information received by the operation unit 114, and outputs operation instructions based on the operation information to each component. When the operation control unit 220 receives an operation of a button or icon for performing event recording from the operation unit 114, it outputs an instruction to perform event recording to the recording control unit 208. In addition to the above, the operation control unit 220 acquires operation information indicating an operation for selecting event data or the like, a playback operation, an erasing operation, a pausing operation, various setting operations of the vehicular recording device 100, and the like, and issues instructions for processing corresponding to the acquired operation information.

[0061] The location information acquisition unit 222 calculates the current location information of the vehicle using a known method based on the radio waves received by the GNSS receiving unit 116. The location information calculated by the location information acquisition unit 222 is recorded by the recording control unit 208 together with the shooting data and event data.

[0062] Next, the flow of the recording process executed by the vehicle recording device 100 will be described with reference to Fig. 2. The recording process executed by the vehicle recording device 100 is executed by the control unit 200, which is a computer operating as a vehicle recording control device, based on a program.

[0063] The start of processing in FIG. 2 means that the vehicle recording device 100 starts operating when, for example, the engine or power supply of a vehicle, which is a moving body, is turned on.

[0064] When the process starts, the vehicular recording device 100 starts a normal recording process and a gaze detection process (step S10). Specifically, the recording control unit 208 starts a normal recording process of recording the image data based on the first image data in the recording unit 106, and the gaze detection unit 214 starts detecting the orientation of the vehicle occupant from the second image data.

[0065] When the process of step S10 starts, the vehicular recording device 100 determines whether or not an operation for performing event recording has been performed (step S11). Specifically, the operation control unit 220 determines whether or not an operation of a button or icon for performing event recording has been received from the operation unit 114. If it is determined that an operation for performing event recording has been performed (step S11: Yes), the process proceeds to step S16. If it is not determined that an operation for performing event recording has been performed (step S11: No), the process proceeds to step S12.

[0066] In step S12, the vehicular recording device 100 determines whether or not a voice command to perform event recording has been received. Specifically, the voice recognition unit 216 determines whether or not an utterance indicating a voice command to perform event recording has been included in the voice captured by the microphone 110. If it is determined that a voice command to perform event recording has been received (step S12: Yes), the process proceeds to step S16. If it is not determined that a voice command to perform event recording has been received (step S12: No), the process proceeds to step S14.

[0067] In step S13, the vehicular recording device 100 determines whether or not an event has been detected. Specifically, the event detection unit 218 determines whether or not an event has been detected based on the acceleration detected by the acceleration sensor 112. If it is determined that an event has been detected (step S13: Yes), the process proceeds to step S16. If it is not determined that an event has been detected (step S13: No), the process proceeds to step S14.

[0068] In step S14, the vehicular recording device 100 determines whether or not the vehicle occupant gazes at the display unit 108a. Specifically, the gaze detection unit 214 determines whether or not the gaze of the vehicle occupant is directed at the display unit 108a. The gaze detection unit 214 may determine whether or not the gaze of the vehicle occupant is directed at the display unit 108a for a predetermined period of time or longer. If it is determined that the vehicle occupant has gazed at the display unit 108a (step S14: Yes), the process proceeds to step S15. If it is not determined that the vehicle occupant has gazed at the display unit 108a (step S14: No), the process proceeds to step S17.

[0069] In step S15, the vehicular recording device 100 uses the output unit 108 to output information indicating a voice command to a vehicle occupant. As an example of step S15, information such as text indicating the voice command, for example, "Start recording" or "Start recording", is displayed on the display unit 108a operating as the output unit 108. The display unit 108a determined to be gazed at by the vehicle occupant in step S14 and the display unit 108a on which the information indicating the voice command is displayed in step S15 are the same display unit. As another example of step S15, the audio output unit 108b operating as the output unit 108 is made to output audio indicating the voice command, for example, "Start recording".

[0070] The processes of steps S11, S12, S13 and S14 are started simultaneously in parallel with the start of the process of step S10, and are executed until the process of FIG. 2 is completed.

[0071] In step S16, the vehicular recording device 100 saves the event data. Specifically, the recording control unit 208 saves the event data based on the first captured data as overwrite-protected. If it is determined in step S11 that an operation for performing event recording has been performed, step S16 saves the first captured data, such as for 15 seconds before and after the operation for performing event recording, as event data. If it is determined in step S12 that a voice command for performing event recording has been received, step S16 saves the first captured data, such as for 15 seconds before and after the voice command for performing event recording, as event data. If it is determined in step S13 that an event has been detected, step S16 saves the first captured data, such as for 15 seconds before and after the event occurrence, as event data.

[0072] If the determination in step S14 is No, after the processing of step S15 and after the processing of step S16, the vehicular recording device 100 determines whether or not the recording processing and the like have been completed (step S17). Specifically, the vehicular recording device 100 determines whether or not the processing started in step S10 has been completed. The completion of the processing started in step S10 means, for example, that the operation of the vehicular recording device 100 has been completed by turning off the engine or power supply of the vehicle, which is a moving body. If it is determined that the processing has been completed (step S17: Yes), the processing of FIG. 2 is completed. If it is not determined that the processing has been completed (step S17: No), the processing proceeds to step S11.

[0073] When a vehicle driver or other passenger wants to record as event data image data of an event such as an accident in which the vehicle is not involved, the passenger may want to save the event data using a voice command. Also, when the vehicle is being tailgated by a nearby vehicle, the passenger may want to save the event data using a voice command. In these cases, the event is not detected in the processing of step S13. It may also be unsafe for the driver to perform an event recording operation using the operation unit 114 while driving the vehicle. Even for passengers other than the driver, if the passenger cannot reach the operation unit 114 of the vehicular recording device 100, it may be difficult for the passenger to perform an event recording operation using the operation unit 113.

[0074] Even when using a voice command to instruct event recording, there are cases where the user cannot immediately remember the voice command to instruct event recording, such as when the frequency of using voice commands to instruct event recording is low, or when the user uses multiple vehicles and the voice command to instruct event recording in the vehicle recording device installed in each vehicle is different.

[0075] In this way, when a vehicle occupant such as a driver cannot immediately remember the correct voice command or utters an incorrect voice command, the vehicle occupant often focuses on the vehicular recording device 100, particularly the display unit 108a. In such a case, the process shown in Fig. 2 outputs a voice command instructing event recording, allowing the vehicle occupant to utter an appropriate voice command, minimizing delay in the timing of event recording and enabling appropriate event recording.

[0076] Next, a second embodiment of the present invention will be described with reference to Fig. 3. The configuration of the vehicular recording device 100 according to the second embodiment is the same as that of the first embodiment, and a description will be omitted except for the differences in processing.

[0077] The gaze detection unit 214 detects that the gaze of a vehicle occupant is directed toward the display unit 108a multiple times consecutively. "Multiple times consecutively" refers to, for example, the case where the gaze of the vehicle occupant is directed toward the display unit 108a two or more times within three seconds. "Multiple times consecutively" does not simply refer to the case where the gaze of the occupant passes through the direction of the display unit 108a, but may refer to the case where the gaze of the occupant is detected to be directed toward the display unit 108a multiple times for a period of time, such as 0.5 seconds, that is sufficient to determine that the occupant is looking at the display unit 108a. When there are multiple occupants in the vehicle, the gaze detection unit 214 detects that the gaze of the same occupant is directed toward the display unit 108a multiple times consecutively.

[0078] The processes of steps S20 to S23 and steps S25 to S27 shown in FIG. 3 are the same as the processes of steps S10 to S13 and steps S15 to S17 shown in FIG. 2, and therefore will not be described.

[0079] In step S24, the vehicular recording device 100 determines whether or not the vehicle occupant has gazed at the display unit 108a multiple times consecutively. Specifically, the gaze detection unit 214 determines whether or not the gaze of the vehicle occupant has been directed at the display unit 108a multiple times consecutively. In other words, it determines whether or not the gaze of the vehicle occupant has been directed at the display unit 108a multiple times within a predetermined period for a predetermined period of time or longer. If it is determined that the vehicle occupant has gazed at the display unit 108a multiple times consecutively (step S24: Yes), the process proceeds to step S25. If it is not determined that the vehicle occupant has gazed at the display unit 108a multiple times consecutively (step S24: No), the process proceeds to step S27.

[0080] When a vehicle driver or other passenger cannot immediately remember the correct voice command and utters an incorrect voice command, the vehicle passenger will often look at the vehicle recording device 100, particularly the display unit 108a, with the intention of recording an event, and then look at the vehicle recording device 100, particularly the display unit 108a, again to confirm whether the event recording is being performed, i.e., whether the voice command was correct.

[0081] In such a case, the process shown in Figure 3 outputs a voice command to instruct event recording, allowing the vehicle occupants to utter the appropriate voice command, minimizing delay in the timing of event recording and enabling appropriate event recording.

[0082] Next, a third embodiment of the present invention will be described with reference to Fig. 4. The configuration of the vehicular recording device 100 according to the third embodiment is the same as that of the first embodiment, and a description will be omitted except for the differences in processing.

[0083] The voice recognition unit 216 recognizes voice commands for the vehicular recording device 100 as well as voices other than voice commands. Recognition of voices other than voice commands only determines that some other utterance has occurred, rather than the utterance of a voice command. When the voice recognition unit 216 recognizes voice corresponding to a voice command, it outputs an instruction to the recording control unit 208 to perform event recording. When the voice recognition unit 216 recognizes voice other than a voice command, it outputs to the recording control unit 208 a notification that voice other than a voice command has been recognized.

[0084] The voice recognition unit 216 does not need to constantly output to the recording control unit 208 that a voice other than a voice command has been recognized. For example, if a voice other than a voice command is recognized during a period in which a vehicle occupant is gazing at the display unit 108a or during a period of, for example, 3 seconds after the vehicle occupant has gazed at the display unit 108a, the voice recognition unit 216 outputs to the recording control unit 208 that a voice other than a voice command has been recognized.

[0085] The processes of steps S30 to S34 and steps S36 to S37 shown in FIG. 4 are the same as the processes of steps S10 to S14 and steps S15 to S17 shown in FIG. 2, and therefore will not be described.

[0086] If it is determined in step S34 that the vehicle occupant has gazed at the display unit 108a (step S34: Yes), the process proceeds to step S35. In step S35, the vehicular recording device 100 determines whether or not a voice other than a voice command has been recognized (step S35). Specifically, the voice recognition unit 216 determines whether or not a voice other than a voice command has been recognized during the period in which the vehicle occupant has gazed at the display unit 108a or within a predetermined period, such as three seconds, after the vehicle occupant has gazed at the display unit 108a. If it is determined that a voice other than a voice command has been recognized (step S35: Yes), the process proceeds to step S36. If it is not determined that a voice other than a voice command has been recognized (step S35: No), the process proceeds to step S38.

[0087] As described above, there are many cases where a vehicle occupant, such as a driver, cannot immediately remember the correct voice command, or where the vehicle occupant makes some kind of utterance while gazing at the vehicular recording device 100. If the utterance is a voice command, step S31 becomes Yes. If the utterance is not a voice command, the process shown in FIG. 3 displays a voice command instructing event recording on the display unit 108a, allowing the vehicle occupant to utter an appropriate voice command, minimizing delay in the timing of event recording and enabling appropriate event recording.

[0088] As a modification of the third embodiment of the present invention, in step S35, it may be determined whether a voice similar to the voice command has been recognized. A voice similar to the voice command is an utterance of a phrase that does not match the voice command but partially matches the voice command.

[0089] In such a case, the voice recognition unit 216 compares the analysis result of the acquired voice with pre-stored voice commands to determine whether there is a phrase that matches the phrase indicated by the voice command. The voice recognition unit 216 may determine that a specific phrase matches. For example, if the voice command is "start recording," and there is an utterance that includes the phrase "recording," it is determined that a voice similar to the voice command has been recognized.

[0090] In step S35, the voice recognition unit 216 may determine whether a synonym has been recognized as a voice similar to the voice command. For example, if the voice command is "start recording," and an utterance including the word "record," such as "start recording" or "event recording," is received, it is determined that a voice similar to the voice command has been recognized.

[0091] In this way, when a vehicle occupant, such as a driver, cannot immediately recall the correct voice command and utters an incorrect voice command, the vehicle occupant often focuses on the vehicular recording device 100, particularly the display unit 108a. In such a case, the process shown in FIG. 3 displays a voice command instructing event recording on the display unit 108a, allowing the vehicle occupant to utter an appropriate voice command, minimizing delay in the timing of event recording and enabling appropriate event recording. Furthermore, uttering the correct voice command helps the occupant understand the voice command and further prevents unintentional event recording due to an incorrect voice command.

[0092] The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the present invention. Furthermore, the program for causing a computer to execute the above-described processes can be stored on various types of non-transitory computer-readable media and supplied to a computer operating as a vehicle recording control device. [Explanation of symbols]

[0093] 100 Vehicle recording device 102 First Camera 104 Second Camera 106 Recording Unit 108 Output section 108a Display section 108b Audio output section 110 Microphone 112 Acceleration Sensor 114 Operation section 116 GNSS receiver 200 Control unit (vehicle recording control device) 202 Shooting data acquisition unit 204 Buffer Memory 206 Shooting data processing section 208 Recording control section 210 Playback control unit 212 Output control section 212a Display control unit 212b Audio output control unit 214 Gaze detection unit 216 Voice Recognition Unit 218 Event detection unit 220 Operation control section 222 Location information acquisition unit

Claims

1. A vehicle recording control device that controls a vehicle recording device, an image data acquisition unit that acquires first image data captured by a first camera that captures an image of the periphery of a vehicle and second image data captured by a second camera that captures an image of the interior of the vehicle; a gaze detection unit that detects a gaze direction of an occupant of the vehicle from the second photographed data; a voice recognition unit that recognizes a voice command for instructing event recording; an output control unit that outputs information indicating the voice command to the occupant when the gaze detection unit detects that the occupant's gaze is directed toward a display unit provided inside the vehicle compartment of the vehicular recording device; a recording control unit that stores the first captured data as event data when the voice recognition unit recognizes the voice command; Equipped with the voice recognition unit recognizes voices other than the voice commands, the output control unit outputs information indicating the voice command to the passenger when the gaze detection unit detects that the passenger's gaze is directed toward the display unit and the voice recognition unit recognizes a voice other than the voice command. Vehicle recording control device.

2. the output control unit outputs information indicating the voice command to the passenger when the gaze detection unit detects that the passenger's gaze is directed to the display unit and the voice recognition unit recognizes a voice uttered other than the voice command during a period in which the passenger's gaze is directed to the display unit or within a predetermined period after the passenger's gaze is directed to the display unit. The recording control device for a vehicle according to claim 1.

3. the output control unit outputs information indicating the voice command to the passenger when the gaze detection unit detects that the passenger's gaze is directed toward the display unit and the voice recognition unit recognizes an utterance other than the voice command that partially matches the voice command or a synonym of the voice command. The recording control device for a vehicle according to claim 1.

4. A vehicle recording control device that controls a vehicle recording device, an image data acquisition unit that acquires first image data captured by a first camera that captures an image of the periphery of a vehicle and second image data captured by a second camera that captures an image of the interior of the vehicle; a gaze detection unit that detects a gaze direction of an occupant of the vehicle from the second photographed data; a voice recognition unit that recognizes a voice command for instructing event recording; an output control unit that outputs information indicating the voice command to the occupant when the gaze detection unit detects that the occupant's gaze is directed toward a display unit provided inside the vehicle compartment of the vehicular recording device; a recording control unit that stores the first captured data as event data when the voice recognition unit recognizes the voice command; Equipped with the output control unit is an audio output control unit that outputs audio to the passenger, and outputs audio indicating the voice command to the passenger from an audio output unit that is easy for the passenger to hear and that is not input to a microphone that acquires the voice command. Vehicle recording control device.

5. the output control unit outputs information indicating the voice command to the occupant when the gaze detection unit detects that the occupant's gaze is directed toward the display unit that is provided inside the vehicle compartment of the vehicular recording device and has a function of accepting an operation to start event recording by an operation by the occupant.

5. The recording control device for a vehicle according to claim 1 or 4.

6. the output control unit is a display control unit that causes information to be displayed on the display unit, and when the gaze detection unit detects that the gaze of the passenger is directed toward the display unit, displays information indicating the voice command on the display unit.

5. The recording control device for a vehicle according to claim 1 or 4.

7. A recording method executed by a vehicle recording control device that controls a vehicle recording device, comprising: an image data acquisition step of acquiring first image data captured by a first camera that captures an image of the periphery of a vehicle and second image data captured by a second camera that captures an image of the interior of the vehicle; a gaze detection step of detecting a gaze direction of the occupant of the vehicle from the second photographed data; a voice recognition step for recognizing a voice command instructing event recording; an output step of outputting information indicating the voice command to the occupant when it is detected that the occupant's line of sight is directed toward a display unit provided inside the vehicle compartment of the vehicular recording device; a recording step of storing the first captured data as event data when the voice command is recognized; Including, In the voice recognition step, a voice other than the voice command is recognized, In the output step, when it is detected in the gaze detection step that the gaze of the passenger is directed toward the display unit and when a voice other than the voice command is recognized in the voice recognition step, information indicating the voice command is output to the passenger. Recording method.

8. A recording method executed by a vehicle recording control device that controls a vehicle recording device, comprising: an image data acquisition step of acquiring first image data captured by a first camera that captures an image of the periphery of a vehicle and second image data captured by a second camera that captures an image of the interior of the vehicle; a gaze detection step of detecting a gaze direction of the occupant of the vehicle from the second photographed data; a voice recognition step for recognizing a voice command instructing event recording; an output step of outputting information indicating the voice command to the occupant when it is detected that the occupant's line of sight is directed toward a display unit provided inside the vehicle compartment of the vehicular recording device; a recording step of storing the first captured data as event data when the voice command is recognized; Including, In the output step, a voice indicating the voice command is output to the passenger in a voice that is easy for the passenger to hear and that is not input to a microphone that acquires the voice command. Recording method.

Images