Image recording control device and image recording method

Abstract

A control device (100) is provided with: an imaging data acquisition unit (111) that controls a camera used in a vehicle and acquires imaging data captured by the camera; a biological information detection unit (114) that detects biological information of a passenger of the vehicle and changes in the biological information; an imaging data processing unit (113) that generates, on the basis of the imaging data, an imaging file that is an object to which the thumbnail images are associated; a thumbnail image generation unit (120) that, when the biological information detection unit (114) has detected a variation in the biological information, generates a thumbnail image on the basis of an image of a predetermined range that includes a point in time at which the biological information detection unit (114) has detected; and a recording control unit (122) that associates the imaging file generated by the imaging data processing unit (113) with the thumbnail image generated by the thumbnail image generation unit (120) and records the same in a recording unit (212).

Description

Technical Field

[0001] The present invention relates to an image recording control device and an image recording method. Background Art

[0002] In-vehicle recorders that mainly capture images outside the vehicle are becoming popular. For example, they generate and record capture files at a predetermined time interval of 60 seconds. In addition, when an event such as an accident is detected, images for a predetermined period including the event detection time point are saved as event data. For example, Patent Document 1 discloses an in-vehicle recorder that can select and reproduce event data by recording the event data and displaying a thumbnail image corresponding to the event data.

[0003] Prior Art Documents

[0004] Patent Documents

[0005] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2020-141170. Summary of the Invention

[0006] In an image recording device that continuously generates and records capture files, such as an in-vehicle recorder, it is sometimes used for the purpose of recording surrounding images while the vehicle is moving, in addition to images at the time of an accident detected as an event. Such thumbnail images are generated from the first frame image of the capture file or event data (event file), and in order to select each file, the thumbnail images are displayed in a list. In such a case, there may be a phenomenon where the object to be confirmed is not captured in the displayed thumbnail images, and even when the thumbnail images of each of the displayed files are shown, it may be difficult to find the desired image based on the display of the thumbnail images.

[0007] The present disclosure has been made in view of the above circumstances, and an object thereof is to facilitate retrieval of captured images.

[0008] In order to solve the above problems and achieve the object, an image recording control device according to the present disclosure includes: a capture data acquisition unit that controls a camera used in a vehicle and acquires capture data captured by the camera; a biological information detection unit that detects biological information of a passenger in the vehicle and changes in the biological information; a capture data processing unit that generates a capture file that is an object to which a thumbnail image is to be associated, based on the capture data; a thumbnail image generation unit that generates a thumbnail image based on images in a predetermined range before and after the detected time point, when the biological information detection unit detects a change in the biological information; and a recording control unit that records the capture file generated by the capture data processing unit and the thumbnail image generated by the thumbnail image generation unit in association with each other in a recording unit.

[0009] The image recording method according to the present disclosure is performed by an image recording device through the following steps: a shooting control step of controlling a camera used in a vehicle and acquiring shooting data captured by the camera; a biological information detection step of detecting biological information of a passenger in the vehicle and changes in the biological information; a shooting data processing step of generating a shooting file that becomes an object corresponding to a thumbnail image based on the shooting data; a thumbnail image generation step of generating a thumbnail image based on an image within a predetermined range before and after the detected time point when a change in the biological information is detected in the biological information detection step; and a recording control step of recording the shooting file generated in the shooting data processing step and the thumbnail image generated in the thumbnail image generation step in a recording unit in association with each other.

[0010] According to the present disclosure, there is an effect that it is possible to easily retrieve the captured images. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is a block diagram showing a structural example of an image recording device having a control device according to a first embodiment.

[0012] Figure 2 is a diagram for explaining a shooting file.

[0013] Figure 3 is a flowchart showing an example of a process for generating a general shooting file in a control device according to a first embodiment.

[0014] Figure 4 is a flowchart showing an example of a process for associating a thumbnail image in a control device according to a first embodiment.

[0015] Figure 5 is a diagram for explaining a shooting file and event data.

[0016] Figure 6 is a flowchart showing an example of a process for associating a thumbnail image in a control device according to a second embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] Hereinafter, embodiments of an image recording control device and an image recording method according to the present disclosure will be described in detail with reference to the drawings. In addition, the present invention is not limited to the following embodiments.

[0018] [First Embodiment]

[0019] <Image Recording Device>

[0020] Figure 1FIG. 0 is a block diagram showing a structural example of an image recording device 10 having an image recording control device (hereinafter referred to as "control device") 100 according to a first embodiment. The image recording device 10 is a so-called driving recorder that records events occurring in a vehicle. The image recording device 10 records thumbnail images in association with captured files.

[0021] In addition to being mounted on a vehicle, the image recording device 10 may be a portable device that can be used in a vehicle. Further, the image recording device 10 may include functions or structures such as a device pre-installed in a vehicle, a navigation system, etc. The image recording device 10 includes a first camera 210, a second camera 211, a recording unit 212, an operation unit 213, an acceleration sensor 214, a GNSS (Global Navigation Satellite System) receiving unit 215, a display unit 216, and a control device 100. In the present embodiment, the second camera 211 is not an essential structure. For example, it may be a sensor that can detect biometric information such as the heart rate, expression, or change in expression of a vehicle occupant, or a portable electronic device worn by the occupant.

[0022] The first camera 210 is a camera included in the vehicle. The first camera 210 is a camera that captures the surroundings of the vehicle. The first camera 210 may be a group of multiple cameras. For example, the first camera 210 is disposed at a position in the front of the vehicle interior where the front of the vehicle can be captured. In the present embodiment, the first camera 210 continuously captures images during the period when the auxiliary power supply of the vehicle is turned on. The first camera 210 outputs the captured data to the captured data acquisition unit 111 of the control device 100. The captured data is, for example, a moving image composed of images at 27.5 frames per second.

[0023] The second camera 211 is a camera included in the vehicle. The second camera 211 is a camera that captures the occupants of the vehicle. The second camera 211 is a camera that can detect and capture biometric information of the occupants. For example, the second camera 211 is a camera that can detect the heart rate of the occupant as biometric information. For example, the second camera 211 is a camera that can detect the expression of the occupant as biometric information. The second camera 211 may be a group of multiple cameras. For example, the second camera 211 is disposed at a position where the occupants of each seat in the vehicle can be captured. In the present embodiment, the second camera 211 continuously captures images during the period when the auxiliary power supply of the vehicle is turned on. The second camera 211 outputs the captured data to the captured data acquisition unit 111 of the control device 100. The captured data is, for example, a moving image composed of images at 27.5 frames per second.

[0024] The recording unit 212 is used for temporary storage of data in the image recording device 10 and the like. The recording unit 212 is, for example, a semiconductor memory element such as a RAM (Random Access Memory), a flash memory, or a recording unit such as a memory card. Alternatively, it may be an external recording unit wirelessly connected via a communication device (not shown). The recording unit 212 records a shooting file as always-recorded data (normal file) or event data according to a control signal output from the recording control unit 122 of the control device 100.

[0025] The operation unit 213 can accept various operations on the image recording device 10. The operation unit 213 is, for example, a touch panel included in the display unit 216. For example, the operation unit 213 can accept an operation of manually saving the captured shooting data as event data in the recording unit 212. For example, the operation unit 213 can accept an operation of reproducing the continuous recording data or event data recorded in the recording unit 212. For example, the operation unit 213 can accept an operation of deleting the event data recorded in the recording unit 212. For example, the operation unit 213 can accept an operation for ending continuous recording. The operation unit 213 outputs operation information to the operation control unit 116 of the control device 100.

[0026] The acceleration sensor 214 is a sensor that detects the acceleration generated on the vehicle. The acceleration sensor 214 outputs the detection result to the event detection unit 117 of the control device 100. The acceleration sensor 214 is, for example, a sensor that detects the acceleration in three axial directions. The three axial directions refer to the front-rear direction, left-right direction, and up-down direction of the vehicle.

[0027] The GNSS receiving unit 215 is composed of a GNSS receiver that receives GNSS signals from GNSS satellites and the like. The GNSS receiving unit 215 outputs the received position information signal to the position information acquisition unit 118 of the control device 100.

[0028] As an example, the display unit 216 is a display device inherent to the image recording device 10, or a display device shared with other systems including a navigation system, etc. The display unit 216 may also be integrally formed with the first camera 210. The display unit 216 is, for example, a display including a liquid crystal display or an organic EL (Electro-Luminescence) display. In the present embodiment, the display unit 216 is disposed on the instrument panel, instrument board, center console, etc. in front of the driver of the vehicle. The display unit 216 displays an image based on an image signal output from the display control unit 119 of the control device 100. The display unit 216 displays the image captured by the first camera 210 or the image recorded in the recording unit 212.

[0029] <Image Recording Control Device>

[0030] The control device 100 is an arithmetic processing device (control device) constituted by, for example, a CPU (Central Processing Unit). The control device 100 loads the stored program into the memory and executes the commands included in the program. The control device 100 includes an internal memory (not shown) for temporarily storing data in the control device 100. The control device 100 has a captured data acquisition unit 111, a buffer memory 112, a captured data processing unit 113, a biological information detection unit 114, an operation control unit 116, an event detection unit 117, a position information acquisition unit 118, a display control unit 119, a thumbnail image generation unit 120, a reproduction control unit 121, and a recording control unit 122.

[0031] The captured data acquisition unit 111 controls the first camera 210 and the second camera 211 used in the vehicle, and acquires the captured data captured by the first camera 210 and the second camera 211. The captured data acquisition unit 111 outputs the acquired captured data to the buffer memory 112.

[0032] The buffer memory 112 is an internal memory included in the control device 100, and is a memory that updates and temporarily records a certain amount of captured data acquired by the captured data acquisition unit 111.

[0033] The captured data processing unit 113 generates a captured file every predetermined time based on the captured data. The captured data processing unit 113 generates a captured file that is an object corresponding to the thumbnail image based on the captured data. The captured data processing unit 113 converts the captured data temporarily stored in the buffer memory 112 into an arbitrary file format such as an MP4 format encoded by an arbitrary codec such as H.264 or MPEG-4 (Moving Picture Experts Group). The captured data processing unit 113 generates a captured file as a file of a certain amount of time based on the captured data temporarily stored in the buffer memory 112. As a specific example, the captured data processing unit 113 generates 60 seconds of captured data as a captured file in the recording order for the captured data temporarily stored in the buffer memory 112. The captured data processing unit 113 outputs the generated captured file to the recording control unit 122. In addition, the captured data processing unit 113 decodes the image of the generated captured file through the reproduction control unit 121 and outputs it to the display control unit 119. The period of the generated captured file is set to 60 seconds as an example, but is not limited thereto. The captured data here may be data including sound in addition to the image captured by the first camera 210.

[0034] The biological information detection unit 114 acquires the biological information of the vehicle occupant and the changes in the biological information. The biological information detection unit 114 detects the change in the biological information when the biological information changes from the normal state to the abnormal state. In the present embodiment, the biological information detection unit 114 detects the change in the biological information when the biological information changes from the normal state to the abnormal state during the normal recording process, that is, during the normal recording period.

[0035] When the biological information detected by the biological information detection unit 114 is the heart rate, the heart rate of the vehicle occupant is detected based on the image of the second camera 211 that captures the vehicle occupant. For example, the biological information detection unit 114 detects the heart rate of the vehicle occupant based on the change in the wavelength of the green band region absorbed by hemoglobin in the blood in the face or the like of the vehicle occupant captured by the second camera 211. The biological information detection unit 114 can apply any other heart rate detection method. The biological information detection unit 114 detects the heartbeat information of the vehicle occupant, and as a change in the heart rate, it detects that the heart rate rising from the normal state heart rate becomes an abnormal state. For example, when the average heart rate of the vehicle occupant is detected to be between 60 beats per minute and 70 beats per minute, and rises to 100 beats per minute or more within a period of less than 5 seconds, etc., the biological information detection unit 114 detects that it has changed from the normal state to the abnormal state. In the present embodiment, the biological information detection unit 114 detects the situation where the heart rate rises from the normal state heart rate during the normal recording period. The biological information detection unit 114 can also pre-store the average heart rate in the normal state for each occupant of the vehicle, and detect the change from the normal state to the abnormal state based on the change in the heart rate from the average heart rate.

[0036] When the biological information detected by the biological information detection unit 114 is the expression, the expression of the vehicle occupant is detected based on the image of the second camera 211 that captures the vehicle occupant. The biological information detection unit 114 detects the captured face of the occupant, and detects the movements of each part that makes up the face, especially the opening degree of the eyes, etc., and uses an expression judgment model that has been machine-learned for various expressions to detect the change in the expression. The biological information detection unit 114 detects the expression of the vehicle occupant, and as a change in the expression, it detects that the situation where the expression changes from the normal state expression to the expression indicating fright becomes an abnormal state. As a change in the expression, for example, the biological information detection unit 114 detects that the change from an expression other than the expression indicating surprise to the expression indicating surprise, especially when the change occurs within a period of less than 1 second, is a change from the normal state to the abnormal state. In the present embodiment, the biological information detection unit 114 detects the change in the expression from the normal state expression to the expression indicating surprise during the normal recording period.

[0037] The operation control unit 116 acquires operation information of an operation received by the operation unit 213. For example, the operation control unit 116 acquires reproduction operation information indicating a reproduction operation or deletion operation information indicating a deletion operation and outputs a control signal. For example, the operation control unit 116 acquires end operation information indicating an operation for ending continuous recording and outputs a control signal.

[0038] The event detection unit 117 detects an event based on the acceleration applied to the vehicle. More specifically, the event detection unit 117 detects an event based on the detection result of the acceleration sensor 214. The event detection unit 117 detects that an event has occurred when the acceleration information acquired from the acceleration sensor 214 is equal to or greater than a threshold value.

[0039] The position information acquisition unit 118 acquires position information indicating the current position of the vehicle. The position information acquisition unit 118 calculates the position information of the current position of the vehicle by a known method based on the GNSS signal received by the GNSS receiver unit 215.

[0040] The display control unit 119 controls the display in the display unit 216. The display control unit 119 outputs an image signal for causing the display unit 216 to output continuous recording data, event data, or the like. More specifically, the display control unit 119 outputs an image being captured by the first camera 210, or an image signal displayed by reproducing the continuous recording data or event data recorded in the recording unit 212.

[0041] The thumbnail image generation unit 120 generates a thumbnail image from the images constituting the captured file.

[0042] The thumbnail image generation unit 120 assigns a thumbnail image to the captured file. More specifically, the thumbnail image generation unit 120 generally assigns a thumbnail image based on the image of the first frame in the video constituting the captured file.

[0043] The thumbnail image generation unit 120 generates a thumbnail image from the video at the detected time point when the biological information detection unit 114 detects a change in biological information. More specifically, when the biological information detection unit 114 detects a change in biological information from a normal state to an abnormal state, the thumbnail image generation unit 120 generates a thumbnail image from the video at the detected time point. The video for which the thumbnail image generation unit 120 generates a thumbnail image may also be set to any time point about 2 to 5 seconds before and after the time point when the change in biological information is detected.

[0044] The thumbnail image generation unit 120 generates a thumbnail image based on the image at the time of the change in biological information when there is a change in biological information during normal recording. More specifically, the thumbnail image generation unit 120 assigns the image of the frame captured at the time of the change in biological information as the thumbnail image of its normal file (normal recording data). In the thumbnail image generated in this way, it is highly likely that the phenomenon around the vehicle that causes the change in the biological information of the passenger is captured.

[0045] Use Figure 2 to explain the captured file as a normal file. Figure 2 It is a diagram explaining the captured file. In Figure 2 the example shown, a captured file is generated as a normal file every 60 seconds based on the captured data captured by the first camera 210. The normal file is continuously captured during the operation of the vehicle, and the normal file is generated and recorded. The normal recording file is a file that can be overwritten.

[0046] A captured file F-1 is generated based on the captured data during the period from time T-1 to time T. During the period from time T-1 to time T, no change in biological information is detected. In this case, the thumbnail image of the captured file F-1 is generated based on the image at time T-1, that is, the image of the first frame of the captured file F-1. The normal recording period of the captured file F-1 is the period from time T-1 to time T.

[0047] A captured file F is generated based on the captured data during the period from time T to time T+1. At time Tb1 during the period from time T to time T+1, a change in biological information is detected. In this case, the thumbnail image of the captured file F is not the image at time T, but is generated based on the image at time Tb1. The normal recording period of the captured file F is the period from time T to time T+1.

[0048] The reproduction control unit 121 controls based on the reproduction operation control signal output from the operation control unit 116 to reproduce the continuous recording data or event data recorded in the recording unit 212. The reproduction control unit 121 includes a decoder (not shown) and reproduces various data by decoding the provided compressed data.

[0049] The recording control unit 122 records the captured data acquired by the captured data acquisition unit 111. In the embodiment, the recording control unit 122 controls the recording unit 212 to record the captured file documented by the captured data processing unit 113. During the period of executing the continuous recording process when the auxiliary power supply of the vehicle is turned on or the like, the recording control unit 122 records the captured file documented by the captured data processing unit 113 as a rewritable captured file in the recording unit 212. More specifically, during the period of executing the continuous recording process, the recording control unit 122 continuously records the captured file generated by the captured data processing unit 113 in the recording unit 212. When the capacity of the recording unit 212 is full, a new captured file is recorded by overwriting the oldest captured file.

[0050] When the event detection unit 117 detects an event, the recording control unit 122 saves the captured data corresponding to the detection of the event, that is, event data. The event data refers to the captured file in a predetermined period, that is, the event recording period, in the captured file generated by the captured data processing unit 113. The recording control unit 122 saves the captured file corresponding to the detection of the event as event data that is prohibited from being overwritten in the recording unit 212. When the event detection unit 117 detects an event, for example, the recording control unit 122 copies the captured files in a predetermined period of about 10 seconds before and after the time point when the event is detected from the buffer memory 112 and saves them as event data.

[0051] The recording control unit 122 records the captured file generated by the captured data processing unit 113 and the thumbnail image generated by the thumbnail image generation unit 120 in correspondence with each other in the recording unit 212.

[0052] <Information Processing in the Image Recording Control Device>

[0053] Next, use Figure 3 、 Figure 4 to illustrate the processing flow in the control device 100. Figure 3 is a flowchart showing an example of the processing flow for generating a general captured file in the control device 100 according to the first embodiment. Figure 4 is a flowchart showing an example of the processing flow for associating thumbnail images in the control device 100 according to the first embodiment.

[0054] Use Figure 3 to illustrate the recording process. During the operation of the image recording device 10, the processing of the flowchart shown in Figure 3 is executed.

[0055] As the processing of Figure 3 starts, the control device 100 starts normal recording and event detection (step S101). When executing Figure 3During the process of [processing content], in other words, during the operation of the image recording device 10, the normal recording by the recording control unit 122 and the event detection by the event detection unit 117 are continuously performed. The control device 100 proceeds to step S102.

[0056] As the control device 100 follows Figure 3 the start of the [processing content], the detection of biological information is started (step S102). During the execution of Figure 3 the [processing content], in other words, during the operation of the image recording device 10, the detection of biological information is continuously performed. More specifically, the control device 100 acquires the biological information of the vehicle occupant and the changes in the biological information through the biological information detection unit 114. More specifically, the control device 100, through the biological information detection unit 114, for example, detects an increase in the heart rate from the normal state heart rate as an abnormal state. The control device 100, through the biological information detection unit 114, for example, as a change in expression, detects a change from the normal state expression to an expression indicating fright as an abnormal state. The control device 100 proceeds to step S103.

[0057] During the execution of Figure 3 the [processing content], the normal recording process is continuously performed (step S103). The control device 100 proceeds to step S104.

[0058] The control device 100 determines whether an event is detected (step S104). More specifically, when the acceleration information obtained from the acceleration sensor 214 is equal to or greater than the threshold value, the control device 100 detects that an event has occurred through the event detection unit 117. When the control device 100 determines that an event is detected (Yes in step S104), it proceeds to step S105. When the control device 100 does not determine that an event is detected (No in step S104), it proceeds to step S106.

[0059] When it is determined that an event is detected (step S104: Yes), the control device 100 performs the event recording process (step S105). The control device 100 proceeds to step S106.

[0060] In the event recording process in step S105, the control device 100 stores the captured image data as event data in the recording unit 212 without allowing overwriting through the recording control unit 122. The event data is the shooting data including the time before and after the event detection time point.

[0061] When it is not determined that an event is detected (step S104: No), the control device 100 proceeds to step S106 while continuously performing the normal recording process.

[0062] In normal recording processing, the control device 100 stores the captured shooting data as a normal file that can be overwritten in the recording unit 212 through the recording control unit 122.

[0063] The control device 100 determines whether to end the processing (step S106). For example, it determines to end the processing based on the disconnection of the vehicle's power supply and power, the operation of the operation unit 213, etc. When the control device 100 determines to end the processing (Yes in step S106), it ends this processing. When the control device 100 does not determine to end the processing (No in step S106), it proceeds to step S103. In other words, normal recording, event detection, and biological information detection are continuously performed.

[0064] Next, use Figure 4 , for Figure 3 the normal recording processing shown in step S103, that is, the processing of assigning a thumbnail image to the normal file when performing normal recording, will be described. Figure 4 The processing of the flowchart shown is executed in the normal recording processing.

[0065] The control device 100 determines whether there is a change in biological information during normal recording (step S111). More specifically, based on the detection result of the biological information detection unit 114, the control device 100 determines whether the biological information changes from the normal state to the abnormal state as the change in biological information during normal recording. More specifically, the control device 100 determines through the biological information detection unit 114 whether the heart rate rises from the normal state heart rate to the abnormal state during normal recording. The control device 100, through the biological information detection unit 114, during normal recording, determines whether the expression changes from the normal state expression to the expression indicating surprise as an abnormal state of the expression change. When the control device 100 determines that there is a change in biological information during normal recording (step S111: Yes), it proceeds to step S112. When the control device 100 does not determine that there is a change in biological information during normal recording (step S111: No), it proceeds to step S113.

[0066] When it is determined that there is a change in biological information during normal recording (step S111: Yes), the control device 100 generates a thumbnail image based on the image at the time of the change in biological information (step S112). More specifically, the control device 100 assigns the image of the frame captured at the time of the change in biological information as the thumbnail image of its normal file through the thumbnail image generation unit 120. The control device 100 proceeds to step S114. When applied to Figure 2In the example of [], a change in biological information is detected at time Tb1 during the normal recording of the captured file F as a normal file (from time T to time T+1). Therefore, the image of the frame captured at time Tb1 is assigned as the thumbnail image of the captured file F.

[0067] In the case where it is not determined that there is a change in biological information during the normal recording (step S111: No), the control device 100 generates a thumbnail image based on the initial image of the normal file (step S113). More specifically, the control device 100 assigns the image of the first frame of the captured file as the thumbnail image of the normal file through the thumbnail image generation unit 120. The control device 100 proceeds to step S114. When applied to Figure 2 In the example of [], a change in biological information is not detected during the normal recording of the captured file F-1 as a normal file (from time T-1 to time T). Therefore, the image of the first frame of the captured file F-1 captured at time T-1 is assigned as the thumbnail image of the captured file F-1.

[0068] The control device 100 associates the thumbnail image and records the normal file (step S114). More specifically, the control device 100 associates the captured file, which is the normal file generated by the captured data processing unit 113, with the thumbnail image generated by the thumbnail image generation unit 120 and records them in the recording unit 212 through the recording control unit 122.

[0069] In this way, the control device 100 generally assigns and records a thumbnail image from the image of the first frame in the video that constitutes the captured file. When there is a change in biological information during the normal recording, the control device 100 assigns and records a thumbnail image based on the image at the time of the change in biological information.

[0070] <Effect>

[0071] As described above, in this embodiment, in the case where a change in biological information is detected, it is possible to assign and record a thumbnail image as the image based on the time point when the change is detected. In the case where a change in biological information is detected during the normal recording, it is possible to assign and record a thumbnail image as the image based on the image at the time of the change in biological information. According to this embodiment, it is possible to assign and record a thumbnail image with a high possibility of capturing the phenomenon around the vehicle that causes the change in the biological information of the passenger. According to this embodiment, it is possible to easily retrieve the captured normal file.

[0072] In this embodiment, in the case where it is detected that the biological information changes from the normal state to the abnormal state, it is possible to generate a thumbnail image based on the video at the detected time point.

[0073] During the normal recording period of this embodiment, when the heart rate rises from the normal state, it is possible to assign and record a thumbnail image as an image based on the change in biological information.

[0074] In this embodiment, during the normal recording period, when the expression changes from the normal state expression to an expression indicating surprise, it is possible to assign and record a thumbnail image as an image based on the change in biological information.

[0075] [Second Embodiment]

[0076] Refer to Figure 5 、 Figure 6 to describe the image recording device 10 of this embodiment. Figure 5 It is a diagram for explaining the shooting file and event data. Figure 6 It is a flowchart showing an example of the process of associating thumbnail images in the control device 100 of the second embodiment. The basic structure of the image recording device 10 is the same as that of the image recording device 10 of the first embodiment. In the following description, the same reference numerals or corresponding reference numerals are assigned to the same constituent elements as those of the image recording device 10, and their detailed descriptions are omitted. In this embodiment, the processing in the biological information detection unit 114 is different from that of the first embodiment.

[0077] In this embodiment, the biological information detection unit 114 detects that the biological information changes from the normal state to the abnormal state during the event recording period, that is, the period before the event is detected.

[0078] In this embodiment, when the biological information detected by the biological information detection unit 114 is the heart rate, during the event recording period before the event is detected, it detects that the heart rate rises from the normal state heart rate.

[0079] In this embodiment, when the detected biological information is the expression, during the event recording period before the event is detected, it detects the case where the expression changes from the normal state expression to an expression indicating fright.

[0080] The thumbnail image generation unit 120 assigns a thumbnail image to the shooting file as event data. More specifically, the thumbnail image generation unit 120 usually assigns a thumbnail image from the image of the first frame in the video constituting the shooting file as event data.

[0081] When there is a change in biological information during the event recording period before the event is detected, the thumbnail image generation unit 120 generates a thumbnail image based on the image when the biological information changes. More specifically, the thumbnail image generation unit 120 assigns the image of the frame captured when the biological information changes as the thumbnail image of the event data.

[0082] Use Figure 5 To explain the event data and the captured files. The captured files F-1, F, and F+1 are the same as those in the first embodiment. The event data stores the captured data during a predetermined period before and after the time point when an event is detected as event data that cannot be overwritten.

[0083] For example, when an event is detected at time Ti, the captured data during the period from time Ti1 to time Ti2 is stored as event data If01. When an event is detected at time Ti, for example, if no change in biological information is detected during the period from time Ti1 to time Ti, a thumbnail image of the event data If01 is generated based on the image of the first frame of the event data If01, that is, the image at time Ti1. When an event is detected at time Ti and a change in biological information is detected at time Tb2 during the period from time Ti1 to time Ti, the thumbnail image of the event data If01 is not generated based on the image at time Ti1 but based on the image at time Tb2. The period before the event detection of the event data If01 is the period from time Ti1 to time Ti.

[0084] Next, use Figure 6 , to Figure 3 explain the event recording process shown in step S105 of Figure 4 , that is, the process of assigning a thumbnail image to the event data when event recording is performed. The process of step S122 performs the same process as step S112 of the flowchart shown in Figure 6 The process shown in the flowchart is executed during the event recording process.

[0085] The control device 100 determines whether there is a change in biological information during the event recording period before the event is detected (step S121). More specifically, based on the detection result of the biological information detection unit 114, the control device 100 determines whether the biological information changes from the normal state to the abnormal state as the change in biological information during the event recording period before the event is detected. More specifically, the control device 100 determines through the biological information detection unit 114 whether the heart rate increases from the heart rate in the normal state to the abnormal state during the event recording period before the event is detected. The control device 100 determines through the biological information detection unit 114 whether the expression changes from the expression in the normal state to the expression indicating surprise as the abnormal state during the event recording period before the event is detected. When the control device 100 determines that there is a change in biological information during the event recording period before the event is detected (step S121: Yes), it proceeds to step S122. When the control device 100 does not determine that there is a change in biological information during the event recording period before the event is detected (step S121: No), it proceeds to step S123.

[0086] When the control device 100 does not determine that there is a change in biological information during the event recording period before the event is detected (step S121: No), the control device 100 generates a thumbnail image of the initial image based on the event data (step S123). More specifically, the control device 100 assigns the image of the first frame of the event data as the thumbnail image of the event data through the thumbnail image generation unit 120. The control device 100 proceeds to step S124. When applied to Figure 5 the example of, when no change in biological information is detected during the event recording period before the event is detected in the event data If01 (from time Ti1 to time Ti), the image of the first frame of the event data If01 captured at time Ti1 is assigned as the thumbnail image of the event data If01.

[0087] When applying the processing of step S122 to Figure 5 the example of, a change in biological information is detected at time Tb2 during the event recording period (from time Ti1 to time Ti) before the event is detected in the event data If01. Therefore, the image of the frame captured at time Tb2 is assigned as the thumbnail image of the event data If01.

[0088] The control device 100 associates the thumbnail image and stores the event data (step S124). More specifically, the control device 100 records the captured file, which is the event data generated by the captured data processing unit 113, and the thumbnail image generated by the thumbnail image generation unit 120 in the recording unit 212 in association with each other through the recording control unit 122.

[0089] In this way, the control device 100 generally assigns and records a thumbnail image based on the image of the first frame in the video that constitutes the shooting file. When there is a change in biological information during the event recording period before the event is detected, the control device 100 assigns and records a thumbnail image based on the image at the time of the change in biological information.

[0090] <Effect>

[0091] As described above, according to this embodiment, it is possible to easily retrieve the captured event data.

[0092] The shooting recording device related to the present disclosure can be implemented in various different ways in addition to the above-described embodiment.

[0093] Each component of the illustrated video recording device is functionally conceptual, and physically it does not necessarily have to be configured as illustrated. That is, the specific form of each device is not limited to the illustrated form, and all or part of it can be functionally or physically dispersed or combined in any unit according to the processing load, usage situation, etc. of each device.

[0094] The structure of the video recording device is realized, for example, by a program loaded into the memory as software. In the above embodiment, the functional blocks realized by the cooperation of these hardware or software are described. That is, regarding these functional blocks, they can be realized in various forms only by hardware, only by software, or by a combination thereof.

[0095] The above-described components include components that can be easily conceived by those skilled in the art and substantially identical components. And the above structures can be appropriately combined. In addition, various omissions, substitutions, or changes in the structure can be made without departing from the gist of the present invention.

[0096] In the above, the case including the second camera 211 has been described, but the detection of the heart rate of the vehicle occupant is not limited thereto. The detection of the heart rate of the vehicle occupant is not limited to the detection based on the video of the second camera 211, and various methods such as various sensors included in the vehicle seat or a smart watch can be applied.

[0097] Industrial Applicability

[0098] The video recording control device and the video recording method of the present disclosure can be used for, for example, a dash cam.

[0099] Reference Signs

[0100] 10 Video recording device

[0101] 100 Control device (video recording control device)

[0102] 111 Shooting data acquisition unit

[0103] 112 Buffer Memory

[0104] 113 Shooting Data Processing Unit

[0105] 114 Biological Information Detection Unit

[0106] 116 Operation Control Unit

[0107] 117 Event Detection Unit

[0108] 118 Position Information Acquisition Unit

[0109] 119 Display Control Unit

[0110] 121 Reproduction Control Unit

[0111] 122 Recording Control Unit

[0112] 120 Thumbnail Image Generation Unit

[0113] 210 First Camera

[0114] 211 Second Camera

[0115] 212 Recording Unit

[0116] 213 Operation Unit

[0117] 214 Acceleration Sensor

[0118] 215 GNSS Receiver

[0119] 216 Display Unit

Claims

1. An image recording control device, comprising: A shooting data acquisition unit that controls a camera used in a vehicle and acquires shooting data captured by the camera; A biological information detection unit that detects the biological information of the vehicle occupant and changes in the biological information; A shooting data processing unit that generates a shooting file that is the object corresponding to the thumbnail image based on the shooting data; A thumbnail image generation unit that generates a thumbnail image based on an image within a predetermined range before and after the detected time point when the biological information detection unit detects a change in the biological information; and A recording control unit that records the shooting file generated by the shooting data processing unit and the thumbnail image generated by the thumbnail image generation unit in correspondence with each other in a recording unit.

2. The image recording control device according to claim 1, wherein, The biological information detection unit detects a change in the biological information from a normal state to an abnormal state, When the biological information detection unit detects a change in the biological information from a normal state to an abnormal state, the thumbnail image generation unit generates a thumbnail image based on the image at the detected time point.

3. The image recording control device according to claim 2, wherein, The biological information detection unit detects the heart rate information of the vehicle occupant and detects an increase in the heart rate from the normal state heart rate as the abnormal state.

4. The image recording control device according to claim 2, wherein, The biological information detection unit detects the expression of the vehicle occupant and detects a change in the expression from the normal state expression to an expression indicating surprise as the abnormal state.

5. The image recording control device according to claim 1, wherein, The shooting data processing unit generates a shooting file at every predetermined time based on the shooting data, When a change in the biological information is detected during the recording period of the shooting file, the thumbnail image generation unit generates a thumbnail image of the shooting file based on an image within a predetermined range including the detected time point.

6. The image recording control device according to claim 1, wherein, The image recording control device further includes an event detection unit that detects an event based on the acceleration applied to the vehicle, When an event is detected by the event detection unit, the recording control unit stores event data, which is image data corresponding to the detection of the event, When a change in the biological information is detected during the period before the event is detected in the event data, the thumbnail image generation unit generates a thumbnail image of the event data based on an image within a predetermined range including the detected time point.

7. An image recording method, wherein ， The following steps are performed by an image recording device: A shooting control step that controls a camera used in a vehicle and acquires shooting data captured by the camera; A biological information detection step that detects the biological information of the vehicle occupant and changes in the biological information; A shooting data processing step of generating a shooting file that becomes an object corresponding to a thumbnail image according to the shooting data; A thumbnail image generation step of generating a thumbnail image according to an image within a predetermined range before and after the detected time point when a change in the biological information is detected in the biological information detection step; And A recording control step of correspondingly recording the shooting file generated in the shooting data processing step and the thumbnail image generated in the thumbnail image generation step into a recording unit.

Images