Vehicle recording control device and recording method

Abstract

To enable event to be recorded properly using voice commands.SOLUTION: A vehicle recording control device 200 according to the present invention includes: a photography data acquisition unit 202 for acquiring first photography data captured by a first camera capturing images of the surroundings of a vehicle; an emotion detection unit 224 for detecting an emotion of an occupant of the vehicle; a voice recognition unit 216 for recognizing voice commands indicating event recording; an output control unit 108 for outputting information indicating the voice command to the occupant when the emotion of the occupant is detected by the emotion detection unit 224 to be a negative emotion; and a recording control unit 208 that saves the first photography data as event data when the voice recognition unit 216 recognizes the voice command.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a recording control device and a recording method for a vehicle, and more particularly to a recording control device and a recording method for a vehicle used in a vehicle such as an automobile. [Background technology]

[0002] Some vehicle recording devices, so-called drive recorders, record events using voice commands in addition to detecting impacts using acceleration sensors (for example, Non-Patent Document 1). Event recording using voice commands allows safe event recording without the need to operate a touch panel or the like while driving, such as when recording an accident in which the driver was not involved. Patent Document 1 discloses a drive recorder that records events by issuing voice instructions in response to event detection based on acceleration. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-154904 [Non-patent literature]

[0004] [Non-Patent Document 1] DRV-MR760 [searched September 24, 2021], Internet<https: / / www.kenwood.com / jp / car / drive-recorders / products / drv-mr760 / > Summary of the Invention [Problem to be solved by the invention]

[0005] Even if a drive recorder is equipped with a function for recording events via voice commands, because there are few opportunities to record events, the driver may not immediately remember what voice command to use to record an event. Alternatively, the driver may mistakenly utter a similar phrase, preventing the action specified by the voice command from being performed. In such cases, the timing to record an event may be missed, or the driver may have to use their hands to perform an operation to record an event while driving the vehicle, which may impair safety.

[0006] The present invention provides a vehicle recording control device and a recording method that can appropriately record events using voice commands. [Means for solving the problem]

[0007] The vehicle recording control device of the present invention comprises an image data acquisition unit that acquires first image data captured by a first camera that captures images of the area around the vehicle, an emotion detection unit that detects the emotions of an occupant of the vehicle, a voice recognition unit that recognizes a voice command that instructs event recording, an output control unit that, when the emotion detection unit detects that the occupant's emotion is a negative emotion, outputs information indicating the voice command to the occupant, and a recording control unit that, when the voice recognition unit recognizes the voice command, saves the first image data as event data.

[0008] The recording method of the present invention is carried out by a vehicle recording control device, and includes an imaging data acquisition unit step of acquiring first imaging data captured by a first camera that captures images of the area around the vehicle, an emotion detection step of detecting the emotions of an occupant of the vehicle, a voice recognition step of recognizing a voice command that instructs event recording, an output control step of outputting information indicating the voice command to the occupant if the emotion of the occupant is detected to be a negative emotion in the emotion detection step, and a recording control step of saving the first imaging data as event data if the voice command is recognized in the voice recognition step. [Effects of the Invention]

[0009] According to the present invention, event recording can be performed appropriately using voice commands. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a configuration of a vehicle-mounted recording device according to a first embodiment of the present invention. [Figure 2] 4 is a flowchart showing an example of processing performed by the vehicle recording control device according to the first embodiment of the present invention. [Figure 3] FIG. 10 is a block diagram showing the configuration of a vehicle-mounted recording device according to a second embodiment of the present invention. [Figure 4] 10 is a flowchart showing an example of processing performed by a vehicle recording control device according to a second embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the following description, a drive recorder used in a moving vehicle, such as an automobile, will be described as an example of a vehicle recording device according to this embodiment. However, the present invention is not limited to this. For example, the present invention can be applied to various moving objects, such as various vehicles, such as motorcycles and bicycles, trains, ships, robots, and even people. Furthermore, the present invention is not limited to the following embodiment.

[0012] A first embodiment of the present invention will be described below with reference to Figures 1 and 2. Figure 1 is a block diagram showing the configuration of a vehicle recording device 100 according to the first embodiment of the present invention. The vehicle recording device 100 is a so-called drive recorder. The vehicle recording device 100 is attached to the upper part of the windshield of the vehicle, detects an impact equivalent to an accident, that is, an acceleration equivalent to an accident applied to the vehicle, and saves video and other images taken when the accident is detected as event data.

[0013] The vehicular recording device 100 shown in FIG. 1 is not limited to a device mounted on a vehicle as a standalone device, but may be configured in various ways, such as a configuration realized as a function of a navigation device or a configuration pre-mounted on a vehicle.

[0014] 1, the vehicle recording device 100 includes a control unit 200 as a vehicle recording control device, a first camera 102, a second camera 104, a recording unit 106, an output unit 108, a microphone 110, an acceleration sensor 112, an operation unit 114, and a GNSS (Global Navigation Satellite System) receiving unit 116. The output unit 108 is made up of a display unit 108a and an audio output unit 108b.

[0015] The control unit 200 includes one or more central processing units (CPUs) that perform various data processing. Unit), GPU (Graphics Processing Unit), RAM (Random Access Memory), etc. The control unit 200 is configured to execute various processes according to the program. The functions realized include a photographing data acquisition unit 202, a buffer memory 204, a photographing data processing unit 206, and a memory card 208. The system includes at least a logic unit 206, a recording control unit 208, a playback control unit 210, an output control unit 212, a voice recognition unit 216, an event detection unit 218, an operation control unit 220, a position information acquisition unit 222, and an emotion detection unit 224. The output control unit 212 is made up of a display control unit 212a and an audio output control unit 212b.

[0016] The control unit 200 is a recording control device for a vehicle that executes the recording method according to the present invention, and is a computer that runs the program according to the present invention.

[0017] The first camera 102 captures images of the surroundings of a vehicle, which is a moving body. The first camera 102 may be provided integrally with the vehicle recording device 100, or may be connected as a separate device via a wired or wireless connection. The first camera 102 includes a lens, an imaging element, an AD (Analog to Digital) conversion element, etc., which are not shown. The first camera 102 outputs captured first image data to the image data acquisition unit 202.

[0018] The first camera 102 is provided inside the vehicle cabin above the windshield of the vehicle recording device 100, and if it is provided integrally with the vehicle recording device 100, it is provided on the windshield side of the vehicle recording device 100.

[0019] The first camera 102 is not limited to a single camera, but may be configured with multiple cameras. For example, the first camera 102 may be configured with multiple cameras in any combination that capture images of the front, rear, sides, interior, etc. of the vehicle, or may be all cameras.

[0020] The second camera 104 captures images of the interior of the vehicle, which is a moving body. The second camera 104 may be provided integrally with the vehicular recording device 100, or may be connected as a separate device via a wired or wireless connection. The second camera 104 includes a lens, an image sensor, an AD conversion element, etc., which are not shown. The second camera 104 outputs captured second captured data to the captured data acquisition unit 202.

[0021] The second camera 104 is provided at a position where it can capture the face of the vehicle occupant and detect the facial expression or gaze of the vehicle occupant. The second camera 104 is provided on the interior side of the vehicle above the windshield of the vehicle when the vehicle recording device 100 is provided thereon, and when the second camera 104 is provided integrally with the vehicle recording device 100, it is provided on the interior side of the vehicle recording device 100.

[0022] When second camera 104 is provided at a position where the facial expression or line of sight of the driver who is a vehicle occupant can be detected, it is provided, for example, on the instrument panel of the vehicle so as to be able to capture an image in the direction of the face of the vehicle driver.When second camera 104 is provided at a position where the facial expressions or line of sight of other vehicle occupants in addition to the driver can be detected, it may be provided, for example, near or inside a rearview mirror of the vehicle so as to be able to capture an image of the entire interior of the vehicle.

[0023] The second camera 104 is not limited to a single camera and may be configured with multiple cameras. The second camera 104 may be a camera that captures visible light, a camera that captures infrared light, or a combination of these.

[0024] 1, the first camera 102 and the second camera 104 are described as different cameras, but a single camera may be used if it can capture the range captured by the first camera 102 and the second camera 104. The single camera is, for example, an all-around camera capable of capturing images of 180 degrees or 360 degrees.

[0025] The image data captured by the first camera 102 and the second camera 104 is moving image data. The image data may include audio data in addition to the moving image data.

[0026] Recording unit 106 is a non-volatile memory, such as a memory card, for recording first captured data captured by first camera 102 under the control of recording control unit 208. The first captured data and event data recorded in recording unit 106 are played back under the control of playback control unit 210. Recording unit 106 may be replaced with a separate recording / playback device that includes recording control unit 208 and playback control unit 210 in addition to recording unit 106. Recording unit 106 may be provided as an integrated recording / playback device, or may be connected as a separate device via wire or wirelessly.

[0027] The output unit 108 outputs information indicating a voice command to a vehicle occupant. The output unit 108 is composed of a display unit 108a and a voice output unit 108b. When the display unit 108a operates as the output unit 108, the display unit 108a displays information such as text information indicating the voice command on the display unit 108a as an output of information indicating the voice command to the vehicle occupant. When the voice output unit 108b operates as the output unit 108, the voice output unit 108b outputs a voice indicating the voice command.

[0028] The display unit 108a is a display device that displays various information under the control of the display control unit 212. The display unit 108a includes a display panel such as a liquid crystal panel or an organic EL (organic electro-luminescence) panel. The display unit 108a may be provided integrally with the vehicle recording device 100, or may be connected as a separate device by wire or wirelessly. The display unit 108a may be connected wirelessly to another device that includes the display control unit 212a. The display unit 108a is located in a position that is easily visible to vehicle occupants.

[0029] The display unit 108a is provided on the interior side of the vehicle above the windshield of the vehicle when the vehicle recording device 100 is mounted thereon, and when the display unit 108a is provided integrally with the vehicle recording device 100, the display unit 108a is provided on the interior side of the vehicle recording device 100. In this case, the display unit 108a is provided near the second camera 104.

[0030] The audio output unit 108b outputs various sounds under the control of the audio output control unit 212b. Specifically, the audio output unit 108b is configured with an audio amplifier, a speaker, etc. The audio output unit 108b outputs the same sound as the audio command as information indicating the audio command to the passengers of the vehicle.

[0031] When outputting information indicating a voice command, the voice output unit 108b outputs the voice so that the voice is not mistakenly recognized as a voice command. For this reason, the voice output unit 108b is configured to be easy for the vehicle occupants to hear and not input to the microphone 110. The voice output unit 108b is, for example, a speaker provided above the seat on which the vehicle occupant sits. Alternatively, the voice output unit 108b is a neck speaker worn by the vehicle occupant.

[0032] The microphone 110 acquires the voice uttered by a passenger such as a driver of the vehicle. The microphone 110 is provided in the second camera 104, and the voice uttered by the passenger such as a driver of the vehicle may be included in the second captured data. The microphone 110 is disposed in a position where it can acquire the voice uttered by the passenger such as a driver of the vehicle. The microphone 110 may have directionality that allows it to acquire the voice uttered by the passenger such as a driver of the vehicle.

[0033] The acceleration sensor 112 detects acceleration applied to the vehicle recording device 100 or the vehicle. The acceleration sensor 112 is, for example, a three-axis acceleration sensor that detects acceleration applied in the x-axis direction (front-rear direction of the vehicle), the y-axis direction (left-right direction of the vehicle), and the z-axis direction (up-down direction of the vehicle). The acceleration sensor 112 outputs the detected acceleration information to the event detection unit 218. The acceleration sensor 112 may be provided integrally with the vehicle recording device 100, or may be connected as a separate unit via a wire or wirelessly. Alternatively, the acceleration sensor 112 may be a separate unit including the acceleration sensor 112 and the event detection unit 218.

[0034] The operation unit 114 is an interface that accepts operations on the vehicular recording device 100 and outputs the accepted operation information to the operation control unit 220. The operation unit 114 is various buttons provided on the vehicular recording device 100 or a touch panel provided on the display unit 108a, and accepts operations by the user. The operation unit 114 may also accept operations from other wirelessly connected devices. For example, the operation unit 114 accepts an operation by the user to start event recording. The operation unit 114 also accepts an operation by the user to play back event data.

[0035] The GNSS receiver 116 is configured with, for example, a receiving device and an antenna that receives signals from GPS (Global Positioning System) satellites and the like. The GNSS receiver 116 outputs the received signals to the location information acquisition unit 222. The GNSS receiver 116 may be provided integrally as part of the vehicular recording device 100, or may be connected as a separate unit by wire or wirelessly. Furthermore, the GNSS receiver 116 may be a separate unit that includes the GNSS receiver 116 and the location information acquisition unit 222.

[0036] The photographing data acquisition unit 202 acquires first photographing data captured by the first camera 102 and second photographing data captured by the second camera 104. The photographing data acquisition unit 202 outputs the first photographing data acquired from the first camera 102 to the buffer memory 204. When the emotion detection unit 224 detects emotions based on the facial expressions of vehicle occupants, the photographing data acquisition unit 202 outputs the second photographing data acquired from the second camera 104 to the emotion detection unit 224.

[0037] The image capturing data acquisition unit 202 also functions as an image capturing control unit that controls image capturing by the first camera 102 and the second camera 104. The image capturing data acquisition unit 202 causes the first camera 102 and the second camera 104 to capture images as necessary while the vehicular recording device 100 is operating.

[0038] The buffer memory 204 is an internal memory provided in the control unit 200, and temporarily stores the photographic data for a certain period of time acquired by the photographic data acquisition unit 202 while updating it.

[0039] The shooting data processing unit 206 converts the shooting data temporarily stored in the buffer memory 204 into an arbitrary file format, such as MP4 format, encoded using an arbitrary codec, such as H.264 or MPEG-4 (Moving Picture Experts Group). The shooting data processing unit 206 generates shooting data as files for a certain period of time from the shooting data temporarily stored in the buffer memory 204. As a specific example, the shooting data processing unit 206 generates 60 seconds of shooting data as files in the order of shooting from the shooting data temporarily stored in the buffer memory 204. The shooting data processing unit 206 outputs the generated shooting data to the recording control unit 208. The shooting data processing unit 206 also outputs the generated shooting data to the display control unit 212a. The period of the shooting data generated as files is set to 60 seconds as an example, but is not limited to this.

[0040] The recording control unit 208 controls the recording unit 106 to record the shooting data filed by the shooting data processing unit 206. While the vehicular recording device 100 is operating, the recording control unit 208 records the shooting data filed by the shooting data processing unit 206 in the recording unit 106 as overwritable shooting data. When the recording capacity of the recording unit 106 reaches its upper limit, the recording control unit 208 overwrites and records new shooting data from a recording area in which old shooting data is recorded among the overwritable shooting data recorded in the recording unit 106. The shooting data recorded by the recording control unit 208 in the recording unit 106 is the first shooting data captured by the first camera 102.

[0041] The recording control unit 208 continues to record the image capture data based on the first image capture data in the recording unit 106 while the vehicular recording device 100 is operating. This type of recording process is called normal recording or loop recording.

[0042] When the event detection unit 218 determines that an event has occurred, the recording control unit 208 stores the first captured data for a predetermined period including the time point at which the event occurred as event data that is prohibited from being overwritten. The event data is, for example, captured data for 15 seconds before and after the time point at which the event occurred.

[0043] When the operation control unit 220 receives an operation to perform event recording, the recording control unit 208 saves the first captured data for a predetermined period including the time when the operation to perform event recording was performed as event data that is prohibited from being overwritten. The event data is, for example, captured data for 15 seconds before and after the time when the operation to perform event recording was performed.

[0044] When the voice recognition unit 216 recognizes a voice command to perform event recording, the recording control unit 208 saves the first captured data for a predetermined period including the time when the voice command to perform event recording was recognized as event data that is prohibited from being overwritten. The event data is, for example, captured data for 15 seconds before and after the time when the voice command to perform event recording was recognized.

[0045] The recording control unit 208 may store the event data in any manner. For example, an overwrite prohibition flag may be added to the header or payload of a section of the first captured data that is to be overwritten, and the data may be stored in the recording unit 106. Alternatively, the section of the first captured data that is to be overwritten may be stored in an overwrite prohibition area of ​​the recording unit 106. Alternatively, the section of the first captured data that is to be overwritten may be transmitted to another device and stored therein.

[0046] The playback control unit 210 controls playback of the shooting data and event data recorded in the recording unit 106. The playback control unit 210 plays back not only the overwritable shooting data recorded in the recording unit 106 but also the event data that is not overwritable, and outputs the data to the display control unit 212a.

[0047] The output control unit 212 performs processing to output various information to a vehicle occupant. Specifically, when the emotion detection unit 224 detects that the emotion of the vehicle occupant is negative, the output control unit 212 outputs information indicating a voice command to the occupant. The output control unit 212 is composed of a display control unit 202a and a voice output control unit 212b.

[0048] The display control unit 212a controls the display unit 108a to display various information on the display unit 108a. When the display control unit 212a functions as the output control unit 212, if the emotion detection unit 224 detects that the emotion of the passenger is a negative emotion, the display control unit 212a causes the display unit 108a to display information indicating a voice command instructing to record an event. If the voice command instructing to record an event is "start recording," the display control unit 212 causes the display unit 108a to display information indicating the voice command, such as "start recording" or "recording started."

[0049] In addition to functioning as the output control unit 212, the display control unit 212a may cause the display unit 108a to display the first video data acquired by the image capture data acquisition unit 202. In this case, the video captured by the first camera 102 is displayed on the display unit 108a. The display control unit 212a also causes the display unit 108a to display the image capture data or event data played back by the playback control unit 210.

[0050] The audio output control unit 212b causes the audio output unit 108b to output various sounds. When the audio output control unit 212b functions as the output control unit 212, if the emotion detection unit 224 detects that the passenger's emotion is a negative emotion, the audio output control unit 212b causes the audio output unit 108b to output a sound indicating a voice command instructing event recording. If the voice command instructing event recording is "start recording," the audio output control unit 212b causes the audio output unit 108b to output the sound "start recording."

[0051] In addition to functioning as the output control unit 212, the audio output control unit 212b causes the audio output unit 108b to output the audio included in the shooting data or event data played back by the playback control unit 210.

[0052] The voice recognition unit 216 analyzes the voice acquired by the microphone 110 and recognizes the content of the utterance. The voice recognition unit 216 performs acoustic model analysis for each phoneme or word on the voice acquired by the microphone 110 and compares the voice with a phoneme model or a language model to recognize the content of the utterance. The voice recognition unit 216 recognizes voice commands to the vehicular recording device 100. Voice commands recognized by the voice recognition unit 216 include, for example, "start recording," which is a voice command for recording an event, and "take a photo," which is a voice command for taking a still image.

[0053] When the voice recognition unit 216 determines through phoneme model analysis of the voice acquired by the microphone 110 that each phoneme in succession is "roku ga kai shi," it accepts the voice command "start recording" and outputs an instruction to the recording control unit 208 to record the event.

[0054] The event detection unit 218 acquires acceleration information detected by the acceleration sensor 112, and when acceleration corresponding to an event is detected, determines that an event has been detected. The acceleration corresponding to an event is acceleration corresponding to acceleration occurring when an object, such as another vehicle, collides with the vehicle on which the vehicular recording device 100 is installed. The detection of acceleration corresponding to an event may be weighted in each of the x-axis direction, the y-axis direction, and the z-axis direction. When it is determined that an event has been detected, the event detection unit 218 outputs an instruction to the recording control unit 208 to perform event recording.

[0055] The operation control unit 220 acquires operation information received by the operation unit 114, and outputs operation instructions based on the operation information to each component. When the operation control unit 220 receives an operation of a button or icon for performing event recording from the operation unit 114, it outputs an instruction to perform event recording to the recording control unit 208. In addition to the above, the operation control unit 220 acquires operation information indicating an operation for selecting event data or the like, a playback operation, an erasing operation, a pausing operation, various setting operations of the vehicular recording device 100, and the like, and issues instructions for processing corresponding to the acquired operation information.

[0056] The location information acquisition unit 222 calculates the current location information of the vehicle using a known method based on the radio waves received by the GNSS receiving unit 116. The location information calculated by the location information acquisition unit 222 is recorded by the recording control unit 208 together with the shooting data and event data.

[0057] The emotion detection unit 224 performs processing to detect the emotion of the vehicle occupant. For example, the emotion detection unit 224 detects the emotion of the vehicle occupant by detecting the vehicle occupant from the second captured data and analyzing the facial expression of the detected occupant. The emotion detection unit 224 also detects the emotion of the vehicle occupant by analyzing the speech of the vehicle occupant acquired by the voice recognition unit 216. Publicly known methods can be applied to either method.

[0058] When detecting the emotion of a vehicle occupant from the second captured data, the emotion detection unit 224 acquires the movements of the eyes and mouth that constitute the occupant's face, and detects emotions such as joy, serenity, anger, fear, etc. by using a trained model that has undergone machine learning of the movements of various facial components, etc. When the emotion detected from the facial expression of the vehicle occupant is anger or fear, the emotion detection unit 224 determines that the emotion is a negative emotion.

[0059] When detecting the emotion of a vehicle occupant from the speech of the vehicle occupant, the emotion detection unit 224 acquires the content and intonation of the speech of the occupant, and detects emotions such as joy, serenity, anger, fear, surprise, etc. by using a trained model that has undergone machine learning of various speech contents and intonations, etc. When the emotion detected from the speech of the vehicle occupant is anger or fear, the emotion detection unit 224 determines that the emotion is a negative emotion.

[0060] The emotion detection unit 224 may determine that the emotion of the vehicle occupant is negative when the occupant shows anger, fear, or surprise for a predetermined period of time, such as 60 seconds or more.

[0061] Next, the flow of the recording process executed by the vehicle recording device 100 will be described with reference to Fig. 2. The recording process executed by the vehicle recording device 100 is executed by the control unit 200, which is a computer operating as a vehicle recording control device, based on a program.

[0062] The start of processing in FIG. 2 means that the vehicle recording device 100 starts operating when, for example, the engine or power supply of a vehicle, which is a moving body, is turned on.

[0063] When the process starts, the vehicular recording device 100 starts a normal recording process and a gaze detection process (step S10). Specifically, the recording control unit 208 starts a normal recording process of recording the image data based on the first image data in the recording unit 106, and the gaze detection unit 214 starts detecting the emotion of the vehicle occupant from the second image data.

[0064] When the process of step S10 starts, the vehicular recording device 100 determines whether or not an operation for performing event recording has been performed (step S11). Specifically, the operation control unit 220 determines whether or not an operation of a button or icon for performing event recording has been received from the operation unit 114. If it is determined that an operation for performing event recording has been performed (step S11: Yes), the process proceeds to step S16. If it is not determined that an operation for performing event recording has been performed (step S11: No), the process proceeds to step S12.

[0065] In step S12, the vehicular recording device 100 determines whether or not a voice command to perform event recording has been received. Specifically, the voice recognition unit 216 determines whether or not an utterance indicating a voice command to perform event recording has been included in the voice captured by the microphone 110. If it is determined that a voice command to perform event recording has been received (step S12: Yes), the process proceeds to step S16. If it is not determined that a voice command to perform event recording has been received (step S12: No), the process proceeds to step S14.

[0066] In step S13, the vehicular recording device 100 determines whether or not an event has been detected. Specifically, the event detection unit 218 determines whether or not an event has been detected based on the acceleration detected by the acceleration sensor 112. If it is determined that an event has been detected (step S13: Yes), the process proceeds to step S16. If it is not determined that an event has been detected (step S13: No), the process proceeds to step S14.

[0067] In step S14, the vehicular recording device 100 determines whether the emotion of the vehicle occupant is negative. Specifically, the emotion detection unit 224 determines whether the emotion of the vehicle occupant is negative based on the facial expression or speech of the vehicle occupant. Occupants who experience negative emotions while driving a vehicle, that is, occupants who experience anger, fear, or surprise, are often the vehicle driver. This is because the vehicle driver is in a position where they are more likely to recognize dangers while driving a vehicle. Also, voice commands for recording an event are often uttered by the vehicle driver. Therefore, determining whether the emotion of the vehicle occupant is negative can be rephrased as determining whether the emotion of the vehicle driver is negative. However, the term "vehicle occupant" also applies to occupants other than the vehicle driver. The emotion detection unit 224 may determine whether the emotion of the vehicle occupant has been negative for a predetermined period of time or longer.

[0068] In step S14, if it is determined that the emotion of the vehicle occupant is a negative emotion (step S14: Yes), the process proceeds to step S15. If it is not determined that the emotion of the vehicle occupant is a negative emotion (step S14: No), the process proceeds to step S17.

[0069] In step S15, the vehicular recording device 100 outputs information indicating the voice command to the vehicle occupant using the output unit 108. As an example of step S15, information such as text indicating the voice command, for example, "Start recording" or "Start recording", is displayed on the display unit 108a operating as the output unit 108. As another example of step S15, the voice output unit 108b operating as the output unit 108 is made to output a voice indicating the voice command, for example, "Start recording".

[0070] The processes of steps S11, S12, S13 and S14 are started simultaneously in parallel with the start of the process of step S10, and are executed until the process of FIG. 2 is completed.

[0071] In step S16, the vehicular recording device 100 saves the event data. Specifically, the recording control unit 208 saves the event data based on the first captured data as overwrite-protected. If it is determined in step S11 that an operation for performing event recording has been performed, step S16 saves the first captured data, such as for 15 seconds before and after the operation for performing event recording, as event data. If it is determined in step S12 that a voice command for performing event recording has been received, step S16 saves the first captured data, such as for 15 seconds before and after the voice command for performing event recording, as event data. If it is determined in step S13 that an event has been detected, step S16 saves the first captured data, such as for 15 seconds before and after the event occurrence, as event data.

[0072] If the determination in step S14 is No, after the processing of step S15 and after the processing of step S16, the vehicular recording device 100 determines whether or not the recording processing and the like have been completed (step S17). Specifically, the vehicular recording device 100 determines whether or not the processing started in step S10 has been completed. The completion of the processing started in step S10 means, for example, that the operation of the vehicular recording device 100 has been completed by turning off the engine or power supply of the vehicle, which is a moving body. If it is determined that the processing has been completed (step S17: Yes), the processing of FIG. 2 is completed. If it is not determined that the processing has been completed (step S17: No), the processing proceeds to step S11.

[0073] When a vehicle driver or other passenger wants to record as event data image data of an event such as an accident in which the vehicle is not involved, the passenger may want to save the event data using a voice command. Also, when the vehicle is being tailgated by a nearby vehicle, the passenger may want to save the event data using a voice command. In these cases, the event is not detected in the processing of step S13. It may also be unsafe for the driver to perform an event recording operation using the operation unit 114 while driving the vehicle. Even for passengers other than the driver, if the passenger cannot reach the operation unit 114 of the vehicular recording device 100, it may be difficult for the passenger to perform an event recording operation using the operation unit 113.

[0074] Even when using a voice command to instruct event recording, there are cases where the user cannot immediately remember the voice command to instruct event recording, such as when the frequency of using voice commands to instruct event recording is low, or when the user uses multiple vehicles and the voice command to instruct event recording in the vehicle recording device installed in each vehicle is different.

[0075] In this way, when an accident or other incident occurs near the vehicle and a vehicle occupant, such as a driver, experiences negative emotions such as fear or surprise, or when the driver experiences negative emotions such as anger or fear after being tailgated by another vehicle, there is a high need to record an event. In such cases, the driver may not be able to immediately recall the correct voice command or may utter an incorrect voice command. In such cases, the process shown in FIG. 2 outputs a voice command indicating event recording, allowing the vehicle occupant to utter the appropriate voice command, minimizing delay in the timing of event recording and enabling appropriate event recording.

[0076] Next, a second embodiment of the present invention will be described with reference to Fig. 3. The configuration of the vehicle recording device 100A according to the second embodiment differs from that of the vehicle recording device 100 according to the first embodiment in that the control unit 200 is replaced with a control unit 200A. The control unit 200A further includes a gaze detection unit 214 in addition to the control unit 200. In the configuration of the vehicle recording device 100A according to the second embodiment, the same configuration as that of the vehicle recording device 100 according to the first embodiment will not be described.

[0077] The gaze detection unit 214 detects the gaze direction of the vehicle occupant from the second captured data acquired by the captured data acquisition unit 202. When the second camera 104 is positioned to capture an image of the vehicle driver, the gaze detection unit 214 detects the gaze direction of the vehicle driver captured in the second captured data. When the second camera 104 is positioned to capture an image of the entire interior of the vehicle, the gaze detection unit 214 detects the gaze direction of the vehicle occupant other than the driver in addition to detecting the gaze direction of the vehicle driver if there is a passenger other than the driver on board.

[0078] The gaze detection unit 214 detects a person's face from the second captured data and detects the position of the eyes in the detected face. The gaze detection unit 214 detects the face and eyes using any method such as pattern matching. The gaze detection unit 214 detects the gaze direction based on the positional relationship between the inner corner of the eye and the iris detected from the image of the eye, or the positional relationship between the corneal reflex and the pupil. The gaze detection method is not limited to the above-mentioned one, and any method can be applied.

[0079] The gaze detection unit 214 detects whether the gaze of the vehicle occupant tends to be fixed on the outside of the vehicle. The gaze detection unit 214 detects whether the gaze of the vehicle occupant tends to be fixed on the outside of the vehicle based on pre-stored data indicating the positional relationship between the second camera 104 and the vehicle's windshield, side windows, etc. The outside of the vehicle also includes the direction ahead of the vehicle. For example, if an accident occurs near the vehicle, the vehicle occupant tends to focus on the accident.

[0080] The gaze detection unit 214 may also determine that the vehicle occupant is gazing outside the vehicle when it is detected that the occupant's gaze tends to be fixed on an inner mirror or a display device equipped with an inner mirror function provided in the vehicle. The gaze detection unit 214 detects that the occupant's gaze tends to be fixed on an outside the vehicle based on pre-stored data indicating the positional relationship between the second camera 104 and the vehicle's inner mirror or a display device equipped with an inner mirror function. For example, when being tailgated by a vehicle behind, the occupant of the vehicle tends to use an inner mirror or the like to gaze behind the vehicle, i.e., outside the vehicle.

[0081] The term "gazing" as used herein means, for example, that the gaze direction of a vehicle occupant is directed outside the vehicle for a predetermined period of time, such as five seconds or more. The gaze detection unit 214 may determine that the vehicle occupant tends to gaze outside the vehicle when the gaze direction of the vehicle occupant is directed outside the vehicle and there is little change in the gaze direction. Furthermore, the gaze direction when gazing outside the vehicle through the windshield or side window of the vehicle may be determined based on little change in the absolute gaze direction relative to the vehicle, or may be determined based on little change in the relative gaze direction relative to the vehicle, or when the positional relationship with the accident site fluctuates as the vehicle moves.

[0082] The gaze detection unit 214 may determine that a vehicle occupant has a tendency to gaze outside the vehicle when the gaze of the vehicle occupant is directed in the same direction outside the vehicle multiple times consecutively. "Multiple times consecutively" means, for example, when the gaze of the vehicle occupant is directed in the same direction outside the vehicle two or more times within three seconds. When there are multiple vehicle occupants, the gaze detection unit 214 detects that the gaze of the same occupant is directed in the same direction outside the vehicle multiple times consecutively.

[0083] Next, the flow of the recording process executed by the vehicle recording device 100A will be described with reference to Fig. 4. The recording process executed by the vehicle recording device 100A is executed by the control unit 200A, which is a computer operating as a vehicle recording control device, based on a program.

[0084] The processes of steps S20 to S24, step S26, and step S27 to S28 shown in FIG. 4 are the same as the processes of steps S10 to S14, step S15, and step S16 to S17 shown in FIG. 2, and therefore will not be described.

[0085] If it is determined in step S24 that the emotion of the vehicle occupant is a negative emotion (step S24: Yes), the process proceeds to step S25. In step S25, the vehicular recording device 100 detects whether the vehicle occupant tends to gaze outside the vehicle, that is, whether the line of sight of the vehicle occupant tends to gaze outside the vehicle. If it is determined that the line of sight of the vehicle occupant tends to gaze outside the vehicle (step S25: Yes), the process proceeds to step S26. If it is not determined that the line of sight of the vehicle occupant tends to gaze outside the vehicle (step S25: No), the process proceeds to step S28.

[0086] In this way, when an accident or other incident occurs near the vehicle and a vehicle occupant, such as a driver, experiences negative emotions such as fear or surprise, or when the driver or occupant experiences negative emotions such as anger or fear due to being tailgated by another vehicle and tends to focus on the target of the accident or tailgating, there is a high need to perform event recording. In such cases, the driver may not be able to immediately remember the correct voice command or may utter an incorrect voice command. In such cases, the process shown in FIG. 4 outputs a voice command indicating event recording, allowing the vehicle occupant to utter the appropriate voice command, resulting in less delay in the timing of event recording and appropriate event recording.

[0087] The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the present invention. Furthermore, the program for causing a computer to execute the above-described processes can be stored on various types of non-transitory computer-readable media and supplied to a computer operating as a vehicle recording control device. [Explanation of symbols]

[0088] 100, 100A Vehicle Recording Device 102 First Camera 104 Second Camera 106 Recording Unit 108 Output section 108a Display section 108b Audio output section 110 Microphone 112 Acceleration Sensor 114 Operation section 116 GNSS receiver 200, 200A control unit (vehicle recording control device) 202 Shooting data acquisition unit 204 Buffer Memory 206 Shooting data processing section 208 Recording control section 210 Playback control unit 212 Output control section 212a Display control unit 212b Audio output control unit 214 Gaze detection unit 216 Voice Recognition Unit 218 Event detection unit 220 Operation control section 222 Location information acquisition unit 224 Emotion Detection Unit

Claims

1. an image data acquisition unit that acquires first image data captured by a first camera that captures an image of the periphery of the vehicle; an emotion detection unit for detecting an emotion of an occupant of the vehicle; a voice recognition unit that recognizes a voice command for instructing event recording; an output control unit that outputs information indicating the voice command to the passenger when the emotion detection unit detects that the passenger's emotion is a negative emotion; a recording control unit that stores the first captured data as event data when the voice recognition unit recognizes the voice command; A recording control device for a vehicle comprising:

2. a gaze detection unit that detects the direction of gaze of a passenger in the vehicle; When the gaze detection unit detects that the gaze of the passenger tends to gaze outside the vehicle and that the passenger's emotion is negative, the output control unit outputs information indicating the voice command to the passenger. The recording control device for a vehicle according to claim 1.

3. the emotion detection unit detects the emotion of the vehicle occupant from a facial expression of the vehicle occupant, 3. A recording control device for a vehicle according to claim 1 or 2.

4. The emotion detection unit detects the emotion of the occupant of the vehicle from an utterance of the occupant of the vehicle.

3. The recording control device for a vehicle according to claim 1 or 2.

5. an image data acquisition unit step of acquiring first image data captured by a first camera that captures an image of the periphery of the vehicle; an emotion detection step of detecting an emotion of an occupant of the vehicle; a voice recognition step for recognizing a voice command instructing event recording; an output control step of outputting information indicating the voice command to the passenger when the emotion of the passenger is detected to be a negative emotion in the emotion detection step; a recording control step of storing the first captured data as event data when the voice command is recognized in the voice recognition step; A recording method executed by a vehicle recording control device.

Images