Recording control device and recording system

Abstract

To appropriately store video.SOLUTION: A recording control device comprises: a photographed data acquisition unit acquiring photographed data that is photographed by a camera provided at a vehicle and is obtained by photographing at least the outside of the vehicle; a trend information acquisition unit acquiring trend information on a travel path of the vehicle; a detection unit that on the basis of positional information associated with the photographed data and positional information on an object included in the trend information, detects a highlight including the object from the photographed data; and a recording control unit associating highlight data with respect to the highlight with the positional information and recording the associated highlight data and positional information, in a recording unit.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a recording control device and a recording system. [Background technology]

[0002] 2. Description of the Related Art When traveling by vehicle, there is a technique for capturing images of the outside or inside of the vehicle using a terminal device such as a drive recorder to record memories in the form of images.

[0003] For example, Patent Document 1 discloses a technique that allows a user to relatively easily look back on memories of a trip. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-86372 A Summary of the Invention [Problem to be solved by the invention]

[0005] When traveling by vehicle, the scenery outside the vehicle and the excitement inside the vehicle change from moment to moment. Therefore, there is a need for a technology that can detect exciting scenes as highlights during a given period of time such as a trip and save the highlight images together so that fun memories can be properly preserved.

[0006] An object of the present invention is to provide a recording control device and a recording system that are capable of appropriately storing video images. [Means for solving the problem]

[0007] A recording control device according to one embodiment of the present invention includes an imaging data acquisition unit that acquires imaging data of at least the exterior of the vehicle captured by a camera installed in a vehicle; a trend information acquisition unit that acquires trend information on the route along which the vehicle is traveling; a detection unit that detects highlights including a subject from the imaging data based on location information associated with the imaging data and location information of the subject included in the trend information; and a recording control unit that associates highlight data related to the highlights with the location information and records them in a recording unit.

[0008] A recording system according to an aspect of the present invention includes a recording control device according to an aspect of the present invention, and a terminal device that accepts an evaluation of highlight data from a user. Effect of the Invention

[0009] According to the present invention, video can be appropriately stored. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating the configuration of a recording system according to an embodiment. [Diagram 2] FIG. 2 is a block diagram showing an example of the configuration of a recording apparatus according to an embodiment. [Diagram 3] FIG. 3 is a block diagram illustrating an example of the configuration of a server device according to the embodiment. [Figure 4] FIG. 4 is a block diagram showing an example of the configuration of a user terminal. [Diagram 5] FIG. 5 is a flowchart showing an example of the flow of processing performed by the recording control device according to the embodiment. [Figure 6] FIG. 6 is a diagram for explaining the photographic data temporarily recorded in the recording unit. [Figure 7] FIG. 7 is a diagram for explaining a method for detecting highlight data from temporarily recorded shooting data. [Figure 8]FIG. 8 is a flowchart showing an example of the flow of processing performed by a recording control device according to a modified example of the embodiment. [Figure 9] FIG. 9 is a diagram for explaining a method for detecting highlights. [Figure 10] FIG. 10 is a diagram for explaining a method for generating a time lapse image. [Figure 11] FIG. 11 is a flowchart showing a process for adjusting conditions for detecting highlights according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to this embodiment, and when there are multiple embodiments, the present invention also includes a configuration in which each embodiment is combined. In the following embodiments, the same parts are given the same reference numerals, and duplicated explanations will be omitted.

[0012] [System of Record] The configuration of a recording system according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram for explaining the configuration of a recording system according to an embodiment.

[0013] 1, the recording system 1 includes a recording device 10 and a server device 20. The recording device 10, the server device 20, and the user terminal 30 are communicatively connected via, for example, a wireless network NW.

[0014] The recording device 10 is mounted in a user's vehicle. The recording device 10 captures images of the interior and exterior of the vehicle, and records the captured image data. The recording device 10 detects, as highlights, the excitement of passengers in the vehicle, and the like, generates highlight data relating to the detected highlights, and records the highlight data in the server device 20 via the network NW. The highlight data may be still image data or video data. The recording device 10 is, for example, a drive recorder.

[0015] The server device 20 records the highlight data received from the recording device 10. The server device 20 receives evaluation information regarding evaluations of the highlight data from users. The server device 20 transmits the evaluation information received from the users to the recording device 10. The server device 20 is, for example, a PC (Personal Computer).

[0016] The user terminal 30 is a terminal device owned by a user. The user can view the highlight data recorded in the server device 20 by the user terminal 30. The user can evaluate the highlight data recorded in the server device 20 by the user terminal 30 and transmit the evaluation result to the server device 20. Examples of the user terminal 30 include, but are not limited to, a smartphone, a PC, a tablet terminal, and the like.

[0017] [Recording device] The configuration of a recording device according to an embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the configuration of a recording device according to an embodiment.

[0018] As shown in FIG. 2, the recording device 10 includes a camera 11, a recording unit 12, a display unit 13, an operation unit 14, a GNSS (Global Navigation Satellite System) signal receiving unit 15, a communication unit 16, a CAN (Controller Area Network) interface unit 17, and a control unit (recording control device) 100.

[0019] The camera 11 captures images of the interior and exterior of the vehicle. The camera 11 captures images of the scenery outside the vehicle. The camera 11 captures images of passengers inside the vehicle. The camera 11 captures images in all directions around the vehicle, for example. The camera 11 is arranged inside the vehicle so as to capture images in all directions around the vehicle. The camera 11 is, for example, an all-around camera that captures images in all directions around the vehicle. The camera 11 may be composed of multiple cameras. The camera 11 may be, for example, a front camera that captures images in front of the vehicle, a rear camera that captures images behind the vehicle, a left side camera that captures images on the left side, and a right side camera that captures images on the right side. The camera 11 outputs captured image data to the image data acquisition unit 120. The camera 11 has an optical system and an image sensor such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The camera 11 may be configured as an FIR (Far Infrared) camera or an IR (Infrared) camera.

[0020] The recording unit 12 records various data. The recording unit 12 temporarily records the shooting data. The shooting data temporarily recorded is also called temporary recording data. The recording unit 12 records, for example, loop recording data in which the shooting data is converted into a file. The recording time of one file is, for example, 5 seconds, but is not limited to this. For example, the recording time of one file may be 10 seconds or 60 seconds. The recording time of one file may be arbitrarily set by the user. The recording unit 12 can be realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or a solid state drive. The recording unit 12 may be, for example, a recording medium that the user can carry around, such as a Secure Digital (SD) card memory.

[0021] The display unit 13 displays various images. For example, the display unit 13 displays the shooting data recorded by the recording unit 12. The display unit 13 is, for example, a display including a liquid crystal display (LCD) or an organic EL (Electro-Luminescence) display.

[0022] The operation unit 14 accepts various operations on the recording device 10. For example, the operation unit 14 accepts an operation for generating highlight data related to highlights from the shooting data recorded in the recording unit 12. The operation unit 14 outputs an operation signal corresponding to the accepted operation to the operation control unit 123. The operation unit 14 can be realized by, for example, a physical switch or a touch panel provided on the display unit 13.

[0023] The GNSS signal receiving unit 15 is configured with a GNSS receiver that receives GNSS signals from GNSS satellites, etc. The GNSS signal receiving unit 15 outputs the received GNSS signals to the position information acquiring unit 124.

[0024] The communication unit 16 is a communication circuit that inputs and outputs signals to and from an external device. The communication unit 16 transmits and receives information to and from the server device 20.

[0025] The CAN interface unit 17 is an interface for acquiring various vehicle information via the CAN. The vehicle information includes, for example, information on whether the vehicle is running or stopped. The CAN interface unit 17 outputs the acquired vehicle information to the vehicle information acquisition unit 126.

[0026] The control unit 100 is realized, for example, by a central processing unit (CPU) or a micro processing unit (MPU) executing a program (for example, a program related to the present disclosure) stored in a storage unit (not shown) using a RAM or the like as a working area. The control unit 100 is a controller, and may be realized, for example, by an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). The control unit 100 may be realized by a combination of software and hardware.

[0027] The control unit 100 includes a photographing data acquisition unit 120, a recording control unit 121, a display control unit 122, an operation control unit 123, a position information acquisition unit 124, a communication control unit 125, a vehicle information acquisition unit 126, a trend information acquisition unit 127, a person recognition unit 128, a scenery recognition unit 129, a determination unit 130, a detection unit 131, and a detection condition adjustment unit 132. The photographing data acquisition unit 120, the recording control unit 121, the display control unit 122, the operation control unit 123, the position information acquisition unit 124, the communication control unit 125, the vehicle information acquisition unit 126, the person recognition unit 128, the scenery recognition unit 129, the determination unit 130, the detection unit 131, and the detection condition adjustment unit 132 are connected to each other via a bus 110.

[0028] The image capturing data acquisition unit 120 acquires various types of image capturing data from the outside. For example, the image capturing data acquires an image of the outside or the inside of the vehicle from the camera 11. In addition to the image capturing data, the image capturing data acquire unit 120 may acquire image capturing data including audio data acquired by a microphone (not shown) arranged at the camera 11 or at another position.

[0029] The recording control unit 121 stores various data in the recording unit 12. For example, the recording control unit 121 converts the shooting data into a file and controls the recording unit 12 to temporarily record it as loop recording data. The recording control unit 121 associates the vehicle position information acquired by the position information acquisition unit 124 with the loop recording data and records it in the recording unit 12. The vehicle position information may be associated with the shooting data file by being multiplexed. The recording control unit 121 records highlight data of the loop recording data in the recording unit 12. The recording control unit 121 may control the recording of the highlight data in the recording unit 22 of the server device 20 via the communication unit 16.

[0030] The display control unit 122 causes various images to be displayed on the display unit 13. The display control unit 122 causes images to be displayed on the display unit 13 by outputting a video signal to the display unit 13. The display control unit 122 causes images to be displayed on the display unit 13 by outputting a video signal relating to the photographed data or overwrite-prohibited recorded data recorded in the recording unit 12 to the display unit 13, for example.

[0031] The operation control unit 123 receives an operation signal related to an operation received from a user or the like via the operation unit 14. The operation control unit 123 receives, for example, an operation signal for detecting highlights from shooting data. The operation control unit 123 outputs a control signal corresponding to the received operation signal to the person recognition unit 128, the scenery recognition unit 129, the determination unit 130, and the detection unit 131.

[0032] The position information acquisition unit 124 acquires the GNSS signal from the GNSS signal receiving unit 15. The position information acquisition unit 24 calculates the current position of the vehicle based on the GNSS signal. The position information acquisition unit 124 calculates, for example, coordinates specified by latitude and longitude, altitude, and moving direction as the current position of the vehicle.

[0033] The communication control unit 125 controls communication via the communication unit 16. The communication control unit 125 communicates with the server device 20 via the communication unit 16.

[0034] The vehicle information acquisition unit 126 acquires vehicle information from the CAN interface unit 17. The vehicle information includes information related to the vehicle speed. That is, the vehicle information acquisition unit 126 acquires information related to the vehicle speed. The vehicle information acquisition unit 126 acquires information indicating that the power of the vehicle's engine, etc. has been turned off, that the accessory power source has been turned off, and that the parking brake has been selected, from the vehicle information. That is, the vehicle information acquisition unit 126 determines whether the vehicle is traveling or not, and whether the vehicle is stopped or not, based on the vehicle information.

[0035] The trend information acquisition unit 127 acquires trend information on trends. The trend information acquisition unit 127 acquires trend information on the location of the photographed data captured by the camera 11. The trend information acquisition unit 127 acquires trend information from, for example, a social network service (SNS). The trend information acquisition unit 127 acquires, for example, spots with a predetermined number of posts or more from the SNS, spots with a predetermined number of high ratings or more, spots with a rapidly increasing number of views, and the like, as trend information. The spots may include, for example, restaurants, product stores, famous places, historical sites, tourist spots, public facilities, accommodation facilities, and the like, but are not limited thereto. The spots may be, for example, places that are trending on the SNS. The trend information acquisition unit 127 may acquire trend information in various formats, such as text, still images, and videos. The trend information acquisition unit 127 may acquire trend information from, for example, a website that transmits tourist information on tourism.

[0036] The person recognition unit 128 recognizes the vehicle occupant from the captured image data of the vehicle interior acquired by the image data acquisition unit 120. The person recognition unit 128 performs image processing on the captured image data to recognize the state of the occupant. For example, the person recognition unit 128 recognizes the outline of the occupant. The person recognition unit 128 recognizes the face direction, gaze direction, posture, and facial expression of the occupant from the outline of the occupant. The recognition method by the person recognition unit 128 may be any well-known recognition processing technique, and is not particularly limited. For example, the person recognition unit 128 may recognize various states of the occupant using dictionary data (not shown). The person recognition unit 128 outputs the recognition result to the determination unit 130.

[0037] The person recognition unit 128 recognizes an excited state of the occupant from the photographed data. Specifically, the person recognition unit 128 recognizes an excited facial expression of the occupant from the photographed data. For example, the person recognition unit 128 recognizes a surprised facial expression, a laughing facial expression, a smiling facial expression, and the like of the occupant. In this case, the person recognition unit 128 recognizes each facial expression based on the outline of the occupant's eyes, the outline of the mouth, and the like. For example, the person recognition unit 128 may recognize the degree to which the occupant's mouth is open and recognize that the occupant is singing. For example, the person recognition unit 128 may recognize the facial expression of a child in the vehicle cabin, such as a smiling face.

[0038] When the shooting data includes a voice, the person recognition unit 128 may recognize that the passenger is excited based on the voice in the shooting data. The person recognition unit 128 may recognize that the passenger is excited by recognizing, for example, a loud voice, a voice of surprise, a laugh, or the like.

[0039] When the camera 11 is composed of an FIR camera and an IR camera, the person recognition unit 128 may recognize the excited state of the vehicle occupant based on the measurement results by the FIR camera and the IR camera. In this case, the person recognition unit 128 may recognize the excited state of the vehicle occupant based on the body temperature and heart rate of the occupant. The person recognition unit 128 may recognize the state based on various biological information of the occupant acquired by other biological sensors (not shown), for example.

[0040] The scenery recognition unit 129 recognizes the scenery outside the vehicle from the photographed data of the outside of the vehicle acquired by the photographed data acquisition unit 120. The scenery recognition unit 129 performs image processing on the photographed data to recognize the scenery outside the vehicle. The scenery recognition unit 129 recognizes natural scenery such as buildings, animals, plants, and mountains outside the vehicle, for example, based on dictionary data (not shown). The scenery recognition unit 129 may recognize a beach, a sunset, a starry sky, and the like, for example, based on brightness information of the photographed data. The recognition method by the scenery recognition unit 129 may be any well-known recognition processing technique, and is not particularly limited. The scenery recognition unit 129 outputs the recognition result to the determination unit 130.

[0041] The scenery recognition unit 129 may recognize the scenery outside the vehicle based on the current location information acquired by the location information acquisition unit 124. For example, the scenery recognition unit 129 may recognize a tourist spot based on the current location information. For example, the scenery recognition unit 129 may recognize a tourist spot based on the current location information and the shooting data. For example, the scenery recognition unit 129 may acquire information about the surroundings of the current location from a social networking site or the like, and recognize stores (e.g., restaurants) and spots that are currently popular and exist in the vicinity of the current location.

[0042] The determination unit 130 determines the surrounding information based on at least one of the recognition result by the person recognition unit 128 and the recognition result by the scenery recognition unit 129. The surrounding information means information including the scenery outside the vehicle and the state inside the vehicle cabin. The determination unit 130 determines whether the surrounding information has changed based on at least one of the recognition result by the person recognition unit 128 and the recognition result by the scenery recognition unit 129.

[0043] For example, when the person recognition unit 128 recognizes that the vehicle occupants are excited, the determination unit 130 determines that the surrounding information has changed. When the person recognition unit 128 recognizes that the vehicle occupants are excited, the determination unit 130 may determine the degree of excitement as the change in the surrounding information. For example, the determination unit 130 may determine the degree of excitement based on the number of excited occupants. The determination unit 130 determines that the degree of excitement is higher as the number of excited occupants increases. For example, the determination unit 130 may determine the degree of excitement based on the ratio of the number of excited occupants. The determination unit 130 determines that the degree of excitement is higher as the ratio of the excited occupants increases. The determination unit 130 may determine the degree of excitement based on the volume of the occupants' voices, for example.

[0044] The determination unit 130 determines that the surrounding information has changed, for example, when the scenery recognition unit 129 recognizes a natural landscape such as a building, an animal, a plant, or a mountain outside the vehicle. For example, the determination unit 130 determines that the surrounding information has changed, for example, when the scenery recognition unit 129 recognizes an animal while the vehicle is traveling. The determination unit 130 determines that the surrounding information has changed, for example, when a beach, a sunset, a starry sky, or the like is recognized while the vehicle is traveling. The determination unit 130 determines that the surrounding information has changed, for example, when the scenery recognition unit 129 recognizes a tourist spot, a famous tourist spot, or a store or spot that is currently a hot topic. The determination unit 130 determines that the surrounding information has changed, for example, when the outside of the vehicle is a photography spot, based on the position information acquired by the position information acquisition unit 124 and map information acquired from the outside.

[0045] The detection unit 131 detects highlights included in the shooting data based on the shooting data acquired by the shooting data acquisition unit 120 and the trend information acquired by the trend information acquisition unit 127. Specifically, the detection unit 131 detects highlights based on location information associated with the shooting data and location information of each spot included in the trend information. The detection unit 131 detects highlights included in the shooting data based on the determination result by the determination unit 130. A highlight means a scene that is valuable for the user to record as a memory of the trip. When the determination unit 130 detects a change in the surrounding information, the detection unit 131 detects the change in the surrounding information as a highlight.

[0046] For example, when the determination unit 130 determines that the interior of the vehicle is excited, the detection unit 131 detects the captured image data of the interior of the vehicle that shows the excited state as a highlight. The detection unit 131 may detect the excitement in the vehicle interior as a highlight from the captured image data only when the determination unit 130 determines that the interior of the vehicle is excited by a predetermined amount or more.

[0047] The detection condition adjustment unit 132 adjusts the detection conditions under which the trend information acquisition unit 127 acquires trend information. The detection condition adjustment unit 132 adjusts the detection conditions under which trend information is acquired based on the user's evaluation input via the communication unit 16. For example, when the user's evaluation of the highlight data detected based on the trend information is high, the detection condition adjustment unit 132 maintains the subject included in the trend information so that it continues to be detected as trend information. For example, when the user's evaluation of the highlight data detected based on the trend information is low, the detection condition adjustment unit 132 adjusts the subject included in the trend information so that it is less likely to be detected as trend information.

[0048] [Server device] The configuration of the server device according to the embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram showing an example of the configuration of the server device according to the embodiment.

[0049] As shown in FIG. 3, the server device 20 includes a communication unit 21, a recording unit 22, and a control unit .

[0050] The communication unit 21 is a communication circuit that inputs and outputs signals to and from an external device. The communication unit 21 transmits and receives information between the recording device 10 and the user terminal 30.

[0051] The recording unit 22 records various types of data. The recording unit 22 records highlight data received from the recording device 10. That is, the recording unit 22 records highlight data including trend information. The recording unit 22 can be realized by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or a solid state drive.

[0052] The control unit 23 is realized, for example, by a CPU, an MPU, or the like executing a program recorded in a storage unit (not shown) using a RAM or the like as a working area. The control unit 23 is also a controller, and may be realized, for example, by an integrated circuit such as an ASIC or an FPGA. The control unit 23 may be realized by a combination of software and hardware.

[0053] The control unit 23 includes a communication control unit 231 , a recording control unit 232 , and an evaluation receiving unit 233 .

[0054] The communication control unit 231 controls communication via the communication unit 21. The communication control unit 231 communicates with the recording device 10 and the user terminal 30 via the communication unit 21.

[0055] The recording control unit 232 records various types of data in the recording unit 22. The recording control unit 232 records the highlight data received from the recording device 10 in the recording unit 22.

[0056] The evaluation receiving unit 233 receives evaluations of the highlight data from the user terminal 30 via the communication unit 21. The evaluation receiving unit 233 receives high and low evaluations of the highlight data from the user terminal. The evaluation receiving unit 233 outputs the evaluations of the highlight data to the recording device 10.

[0057] [User device] The configuration of a user terminal according to an embodiment will be described with reference to Fig. 4. Fig. 4 is a block diagram showing an example of the configuration of a user terminal.

[0058] As shown in FIG. 4, the user terminal 30 includes a communication unit 31, an operation unit 32, an output unit 33, a display unit , a recording unit 35, and a control unit .

[0059] The communication unit 31 is a communication circuit that inputs and outputs signals to and from an external device. The communication unit 31 transmits and receives information to and from the server device 20. The communication unit 31 transmits and receives information to and from an SNS.

[0060] The operation unit 32 accepts various operations on the user terminal 30. For example, the operation unit 32 accepts an operation for playing highlight data stored in the server device 20. The operation unit 32 accepts an operation for inputting an evaluation of the played highlight data. The operation unit 32 accepts an operation for sending the highlight data received from the server device 20 to an SNS. For example, a user operates the operation unit 32 to accept a high evaluation, a low evaluation, or the like for the played highlight data. The operation unit 32 outputs an operation signal according to the accepted operation to the operation control unit 362. The operation unit 32 can be realized by, for example, a physical switch or a touch panel provided on the display unit 34.

[0061] The output unit 33 outputs various types of information. The output unit 33 outputs sound. For example, the output unit 33 outputs the sound of the reproduced highlight data. The output unit 33 can be realized by, for example, a speaker.

[0062] The display unit 34 displays various images. For example, the display unit 34 displays the played back highlight data. The display unit 34 is, for example, a display including a liquid crystal display or an organic EL display.

[0063] The recording unit 35 records various types of data. For example, the recording unit 35 may record highlight data received from the server device 20. The recording unit 35 can be realized by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or a solid state drive.

[0064] The control unit 36 ​​is realized, for example, by a CPU, an MPU, or the like executing a program recorded in a storage unit (not shown) using a RAM or the like as a working area. The control unit 23 is a controller, and may be realized, for example, by an integrated circuit such as an ASIC or an FPGA. The control unit 23 may be realized by a combination of software and hardware.

[0065] The control unit 36 ​​includes a communication control unit 361 , an operation control unit 362 , an output control unit 363 , a display control unit 364 , and a recording control unit 365 .

[0066] The communication control unit 361 controls communication via the communication unit 31. The communication control unit 361 communicates with the server device 20 via the communication unit 31.

[0067] The operation control unit 362 receives an operation signal related to an operation received from a user or the like via the operation unit 32. For example, the operation control unit 362 receives an operation signal for playing back highlight data from the operation unit 32. For example, the operation control unit 362 receives an operation signal inputting an evaluation of the highlight data from the operation unit 32.

[0068] The output control unit 363 controls the output of audio to the output unit 33. The output control unit 363 outputs a control signal to the output unit 33, for example, to cause the output unit 33 to output the audio of the highlight data.

[0069] The display control unit 364 controls the display of the display unit 34. The display control unit 364 causes the display unit 34 to display an image of the highlight data, for example, by outputting a control signal to the display unit 34.

[0070] The recording control unit 365 records various types of data in the recording unit 35. The recording control unit 365 records highlight data received from the server device 20 in the recording unit 35, for example.

[0071] [Processing of Recording Control Device] The process of the recording control device according to the embodiment will be described with reference to Fig. 5. Fig. 5 is a flow chart showing an example of the process flow of the recording control device according to the embodiment.

[0072] The control unit 100 accepts operation information (step S100). Specifically, the operation control unit 123 accepts an operation signal for designating a predetermined period for detecting highlights. For example, the operation control unit 123 accepts an operation signal for detecting highlights for the current day, or an operation signal for detecting highlights for a predetermined period during which the user was traveling. Then, the process proceeds to step S110.

[0073] The control unit 100 acquires the temporarily recorded data (step S110). Specifically, the recording control unit 121 acquires the photographic data temporarily recorded for the period designated in step S100 from the recording unit 12. Then, the process proceeds to step S120.

[0074] Fig. 6 is a diagram for explaining the photographic data temporarily recorded in the recording unit 12. As shown in Fig. 6, the photographic data is filed such as data file D1, data file D2, data file D3, data file D4, data file D5, . . . and is loop-recorded as temporarily recorded data in the recording unit 12. The recording control unit 121 obtains data files for a period corresponding to a specified period from the recording unit 12.

[0075] The control unit 100 acquires trend information (step S120). Specifically, the trend information acquisition unit 127 acquires trend information corresponding to the vehicle's travel route (location information) from an SNS or the like. Then, the process proceeds to step S130.

[0076] The control unit 100 detects highlights from the temporary record data (step S130). Specifically, the detection unit 131 detects highlight data including highlights from the temporary record data based on trend information. The highlight data based on trend information is made up of image data taken of the outside of the vehicle, but if the detection unit 131 detects excitement inside the vehicle cabin, the highlight data may include image data taken inside the vehicle cabin that shows the excitement. Then, the process proceeds to step S140.

[0077] FIG. 7 is a diagram for explaining a method for detecting highlight data from temporarily recorded photographing data. As shown in FIG. 7, the photographing data includes a data file D11, a data file D12, a data file D13, a data file D14, a data file D15, and so on. For example, the trend information acquisition unit 127 acquires "Restaurant A" and "Point B" that is a hot topic on SNS as trend information on the vehicle's travel route. In this case, the trend information acquisition unit 127 acquires location information of "Restaurant A" and "Point B". In this case, the detection unit 131 detects, as highlight data, the data file D11 associated with the location information of "Restaurant A" and the data file D13 associated with the location information of "Point B". As a result, the data file including "Restaurant A" and "Point B" can be detected as highlight data.

[0078] The control unit 100 records the detected highlights (step S140). Specifically, the recording control unit 121 records the highlight data detected by the detection unit 131 in the recording unit 12 or the recording unit 22 of the server device 20. Then, the process proceeds to step S150.

[0079] The control unit 100 determines whether or not to end the process (step S150). The control unit 100 determines to end the process when, for example, an operation to end the process is received or when the power is turned off. If it is determined to end the process (step S150; Yes), the process of FIG. 5 ends. If it is determined not to end the process (step S150; No), the process proceeds to step S100.

[0080] As described above, in this embodiment, trend information can be acquired and shooting data including a subject included in the trend information can be recorded as highlight data. This allows the embodiment to automatically detect images of restaurants and other things that are popular in the world. As a result, the user can come into contact with things that are popular in the world and trends that the user is not aware of.

[0081] [Variations] Next, a modified example of the embodiment will be described. In the above-mentioned embodiment, highlights are detected based on the location information of the shooting data and the location information of the trend information, but the present invention is not limited to this. For example, in the case where highlight data associated with location information is recorded, the present invention may detect and record, as a highlight, the shooting data when a vehicle passes that location.

[0082] The process of the recording control device according to the modified embodiment will be described with reference to Fig. 8. Fig. 8 is a flow chart showing an example of the process flow of the recording control device according to the modified embodiment.

[0083] The processes in steps S200 and S210 are the same as those in steps S100 and S110 shown in FIG. 5, respectively, and therefore will not be described.

[0084] The control unit 100 determines whether or not highlight data associated with the position information has been recorded (step S220). Specifically, the detection unit 131 determines whether or not highlight data associated with the position information has been recorded in the recording unit 22 of the server device 20. If it is determined that highlight data associated with the position information has been recorded (step S220; Yes), the process proceeds to step S230. If it is determined that highlight data associated with the position information has not been recorded (step S220; No), the process proceeds to step S250.

[0085] If the result of the determination in step S220 is Yes, the control unit 100 detects a highlight based on the position information (step S230). Specifically, if the highlight data and the position information are associated and recorded in the recording unit 22 of the server device 20, the detection unit 131 detects, as a highlight, the photographed data when the vehicle passes the point where the highlight was detected. Then, the process proceeds to step S240.

[0086] FIG. 9 is a diagram for explaining a method in which the detection unit 131 detects highlights based on position information. As shown in FIG. 9, the shooting data includes a data file D21, a data file D22, a data file D23, a data file D24, a data file D25, . . . For example, it is assumed that highlight data of "point C" is associated with position information and recorded in the recording unit 22 of the server device 20. In this case, the detection unit 131 detects the data file D21, the data file D23, and the data file D25 associated with the position information of "point C" as highlight data. This makes it possible to detect data files including the highlighted and detected points as highlight data going back in time.

[0087] The processes in steps S240 and S300 are the same as those in steps S140 and S150 shown in FIG. 5, respectively, and therefore will not be described.

[0088] If the result of the determination in step S220 is No, the control unit 100 recognizes the scenery outside the vehicle from the acquired photographic data (step S250). Specifically, the scenery recognition unit 129 executes a recognition process of the scenery outside the vehicle. Then, the process proceeds to step S260.

[0089] The control unit 100 determines whether or not there is a change in the surrounding information (step S260). Specifically, the determination unit 130 determines whether or not there is a change in the surrounding information based on the recognition result of the scenery recognition unit 129. If it is determined that there is a change in the surrounding information (step S260; Yes), the process proceeds to step S270. If it is determined that there is no change in the surrounding information (step S260; No), the process proceeds to step S300.

[0090] If the result of the determination in step S260 is Yes, the control unit 100 detects a highlight including a point where the surrounding information has changed (step S270). Specifically, the detection unit 131 detects a data file including a highlight as highlight data. Then, the process proceeds to step S280.

[0091] The control unit 100 acquires position information (step S280). Specifically, the position information acquisition unit 124 acquires position information on the position of the highlight data by the detection unit 131. Then, the process proceeds to step S290.

[0092] The control unit 100 associates the highlight with the position information and records them (step S290). Specifically, the recording control unit 121 associates the highlight data detected by the detection unit 131 with the position information acquired by the position information acquisition unit 124 and records them in the recording unit 22 of the server device 20. Then, the process proceeds to step S300.

[0093] In a modified example of this embodiment, a time lapse image may be generated from temporary recording data recorded in the recording unit 12. Specifically, the detection unit 131 detects shooting data captured at different times at the same location as highlight data. Then, the recording control unit 121 generates a time lapse image by connecting the highlight data detected by the detection unit 131 in chronological order.

[0094] FIG. 10 is a diagram for explaining a method for generating a time lapse image. FIG. 10 shows data file 30A, data file 30B, data file 30C, data file 30D, data file 30E, . . . , which are highlight data at point D. Data file 30A to data file 30E are data files captured at the same point at different times. In this case, the recording control unit 121 joins data file 30A, data file 30B, data file 30C, data file 30D, data file 30E, . . . in chronological order to generate one time lapse image. By viewing the time lapse image, the user can grasp the change over time of the points detected as highlights.

[0095] [Detection condition adjustment process] Next, a method for adjusting the detection conditions of the subject included in the trend information will be described. In this embodiment, the user can view highlight data including highlights detected based on the trend information. Since each user has different tastes, it is assumed that the saved highlight data may not be to the user's liking. In this embodiment, the detection conditions of the subject included in the trend information can be adjusted to suit the user's tastes.

[0096] The process of adjusting the conditions for detecting highlights according to the embodiment will be described with reference to Fig. 11. Fig. 11 is a flowchart showing the flow of the process of adjusting the conditions for detecting highlights according to the embodiment.

[0097] The control unit 100 determines whether or not an evaluation of the highlight has been received (step S400). Specifically, the detection condition adjustment unit 132 determines whether or not an evaluation of the highlight data has been received from the server device 20 via the communication unit 16. If it is determined that an evaluation has been received (step S400; Yes), the process proceeds to step S410. If it is determined that an evaluation has not been received (step S400; No), the process proceeds to step S440.

[0098] If it is determined as Yes in step S400, the control unit 100 determines whether or not the evaluation is high (step S410). Specifically, the detection condition adjustment unit 132 determines whether or not the evaluation of the highlight data is high. If it is determined as high (step S410; Yes), the process proceeds to step S420. If it is determined as low (step S410; No), the process proceeds to step S430.

[0099] If the result of the determination in step S410 is Yes, the control unit 100 lowers the threshold of the detection condition of the subject (step S420). Specifically, the detection condition adjustment unit 132 lowers the threshold of the detection condition of the subject included in the trend information, thereby making it easier to detect, as a highlight, shooting data including a subject similar to the highly rated highlight data. More specifically, for example, if highlight data including a restaurant is highly rated, it becomes easier to detect, as a highlight, shooting data including a restaurant as a subject.

[0100] If the result of the determination in step S410 is No, the control unit 100 increases the threshold value of the detection condition of the subject (step S430). Specifically, the detection condition adjustment unit 132 increases the threshold value of the detection condition of the subject included in the trend information, making it more difficult to detect, as a highlight, the captured data including a subject similar to the highlight data that has received a low rating.

[0101] The control unit 100 determines whether or not to end the process (step S440). The control unit 100 determines to end the process when, for example, an operation to end the process is received or when the power is turned off. If it is determined to end the process (step S440; Yes), the process of FIG. 11 ends. If it is determined not to end the process (step S440; No), the process proceeds to step S400.

[0102] Although the embodiments of the present invention have been described above, the present invention is not limited by the contents of these embodiments. The above-mentioned components include those that a person skilled in the art can easily imagine, those that are substantially the same, and those that are within the so-called equivalent range. Furthermore, the above-mentioned components can be appropriately combined. Furthermore, various omissions, substitutions, or changes of the components can be made without departing from the gist of the above-mentioned embodiments, and those in which the expression of the present invention is replaced between a method, an apparatus, a system, a recording medium, a computer program, etc. are also valid as aspects of the present invention. [Explanation of symbols]

[0103] 1. System of Record 10 Recording Device 11 Camera 12,22,35 Recording section 13,34 Display section 14,32 Operation section 15 GNSS signal receiver 16,21,31 Communications Department 17 CAN interface section 20 Server device 30 User terminal 23,36 Control section 100 Control unit (recording control device) 110 Bus 120 Shooting data acquisition unit 121,232 Recording control section 122,364 Display control section 123,362 Operation control section 124 Location information acquisition unit 125,231,361 Communication control section 126 Vehicle Information Acquisition Unit 127 Trend information acquisition department 128 Person recognition section 129 Scenery recognition section 130 Judgment section 131 Detection unit 132 Detection condition adjustment unit 233 Evaluation Reception Department 363 Output Control Section 365 Recording control section

Claims

1. an image data acquisition unit that acquires image data of at least the outside of the vehicle captured by a camera installed in the vehicle; a recording unit that records the photographed data as temporary recorded data; A trend information acquisition unit that acquires trend information along a travel route of the vehicle; a detection unit that detects a highlight including a subject from the temporary record data based on location information associated with the temporary record data and location information of the subject included in the trend information; a recording control unit that associates the temporarily recorded data in which the highlight is detected with position information as highlight data related to the highlight, and records the temporarily recorded data separately from the temporarily recorded data in at least one of the recording unit and another recording unit; A recording control device comprising:

2. When the highlight data and location information are recorded in the recording unit in association with each other, the detection unit detects, as a highlight, the photographed data when the vehicle passes a point where the highlight is detected. The recording control device according to claim 1 .

3. a scenery recognition unit that recognizes scenery outside the vehicle from the photographed data; a determination unit that determines a change in surrounding information outside the vehicle based on a recognition result of the scenery recognition unit; a person recognition unit that recognizes a person present in the vehicle cabin from the photographed data, The determination unit determines a change in surrounding information in the passenger compartment of the vehicle based on a recognition result of the person recognition unit, the detection unit detects highlights from the photographed data based on at least one of a case where the determination unit determines that there is a predetermined or greater change in surrounding information outside the vehicle and a case where the determination unit determines that there is a predetermined or greater change in surrounding information inside the vehicle.

3. A recording control device according to claim 1 or 2.

4. a detection condition adjustment unit that generates adjustment information for adjusting a condition for the detection unit to detect the subject based on a user's evaluation of the highlight data; The recording control device according to claim 1 .

5. A recording control device according to any one of claims 1 to 4, A terminal device that accepts evaluations from users regarding the highlight data. System of record.

Images