Traveling video data recording and processing system and traveling video data recording and processing method

Abstract

To provide a system that records video data while a vehicle is traveling, edits the data, and provides it to a user without using an in-vehicle device with complex functions.SOLUTION: A traveling video data recording and processing system includes a first server 5 that communicates with a device 2 equipped with a camera and mounted on a vehicle 1, and a second server 6 that communicates with an information terminal 4 held by a user 3. The first server 5 receives video data from the device 2 mounted on the vehicle 1 and temporarily stores the data. The second server 6 receives and stores the video data from the first server 5, edits the stored video data, and transmits it to the information terminal 4.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology disclosed herein relates to a system that records video data while a vehicle is traveling, edits the video data, and provides the video data to a user. [Background technology]

[0002] Patent Document 1 discloses a vehicle-based memory sharing system that allows people who traveled together to share memory information more easily. In this system, an in-vehicle device works in conjunction with a user's mobile device brought into the vehicle. The in-vehicle device and the mobile device acquire information representing memories made during the trip, and the integrated memory information is sent from the in-vehicle device to a server. The integrated memory information is then distributed to the mobile device. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-230519 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the vehicle memory sharing system disclosed in Patent Document 1, the vehicle-mounted device must have a function for linking with the user's mobile device brought into the vehicle, a function for integrating memory information acquired by the vehicle-mounted device and the mobile device, etc. Therefore, in order to realize the system, an in-vehicle device with complex functions must be installed in the vehicle. Furthermore, since the functions can only be realized by a specific in-vehicle device compatible with this system, it is difficult to change the in-vehicle device.

[0005] The technology disclosed here has been developed in consideration of these points, and realizes a system that records video data while a vehicle is traveling, edits it, and provides it to the user without using an onboard device with complex functions. [Means for solving the problem]

[0006] In order to solve the above problem, the technology disclosed herein is a system that records video data while a vehicle is traveling, edits the data, and provides it to a user, and includes a first server that communicates with a device equipped with a camera mounted on the vehicle, and a second server that communicates with an information terminal carried by the user, and the first server and second server are capable of communicating with each other, and the first server receives the video data captured by the camera from the device and temporarily stores it, and the second server receives the video data captured by the camera from the first server and stores it, edits the stored video data, and sends it to the information terminal.

[0007] According to this configuration, the driving video data recording and processing system includes a first server that communicates with a device equipped with a camera mounted on a vehicle, and a second server that communicates with an information terminal carried by a user. The first server receives video data captured by the camera from the device mounted on the vehicle and temporarily stores it. The second server receives and stores the video data from the first server, edits the stored video data, and transmits it to the information terminal. This allows the user to view the edited driving video data via the information terminal.

[0008] In this way, with this driving video data recording and processing system, video data acquired by the equipment installed in the vehicle is transmitted to the second server via the first server, and the data is consolidated and managed by the second server. Therefore, there is no need to install equipment with complex functions in the vehicle. Furthermore, even if the equipment installed in the vehicle is changed, the second server can maintain the functionality of the driving video data recording and processing system by communicating with the first server for the new equipment.

[0009] The device may be a drive recorder or a device for ADAS (Advanced driver-assistance systems).

[0010] This makes it possible to realize a driving video data recording and processing system using a drive recorder or ADAS equipment, eliminating the need to install new equipment in the vehicle.

[0011] The device may also transmit location information of the vehicle together with the video data captured by the camera.

[0012] This allows the vehicle position information to be used in the editing process of the video data in the second server, making the editing process easier.

[0013] The technology disclosed herein also provides a driving video data recording and processing method for recording video data while the vehicle is driving, editing the data, and providing it to the user, in a system that includes a first server that communicates with a device equipped with a camera mounted on a vehicle, and a second server that communicates with an information terminal carried by a user, the first server and the second server being capable of communicating with each other, the method including the steps of the first server receiving the video data captured by the camera from the device and temporarily saving it, and the second server receiving the video data captured by the camera from the first server, saving it, editing the saved video data, and sending it to the information terminal. [Effects of the Invention]

[0014] As described above, the technology disclosed herein makes it possible to realize a system that records video data while a vehicle is traveling, edits the video data, and provides it to the user, without using an onboard device with complex functions. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a conceptual diagram illustrating an example of the overall configuration of a driving video data recording and processing system according to an embodiment. [Figure 2] 10 is a screen example provided to an information terminal in the driving video data recording and processing system according to the embodiment. [Figure 3]Configuration example of driving video data recording and processing system according to the embodiment [Figure 4] A flowchart showing an example of the operation of the configuration example of FIG. 3. [Figure 5] A flowchart showing an example of the operation of the configuration example of FIG. 3. [Figure 6] A flowchart showing an example of the operation of the configuration example of FIG. 3. DETAILED DESCRIPTION OF THE INVENTION

[0016] Exemplary embodiments will now be described in detail with reference to the drawings.

[0017] FIG. 1 is a conceptual diagram showing an example of the overall configuration of a driving video data recording and processing system according to an embodiment. In FIG. 1, a drive recorder 6 is installed in a vehicle Q. The drive recorder 6 captures and records video data of the vehicle's surroundings while the vehicle Q is traveling using a camera. The video data is recorded, for example, periodically or when an event such as an impact occurs. The recorded video data includes videos and still images. The drive recorder 6 is a communication-type drive recorder that communicates with a cloud server 5 for the drive recorder. The drive recorder 6 transmits the recorded video data to the cloud server 5, for example, periodically or at any timing.

[0018] A user U driving a vehicle Q or a user U riding in the vehicle Q records video data using an information terminal 8. An app for using this driving video data recording and processing system has been downloaded to the information terminal 8, and a cloud server 1 for the app communicates with the information terminal 8. The information terminal 8 transmits the recorded video data to the cloud server 1. The cloud server 5 and the cloud server 1 can communicate with each other. The cloud server 5 corresponds to the first server in the present disclosure, and the cloud server 1 corresponds to the second server in the present disclosure.

[0019] Cloud server 5 receives the video data captured by the camera of drive recorder 6 from drive recorder 6 and temporarily stores it. Cloud server 1 receives the video data captured by the camera of drive recorder 6 from cloud server 5 and stores it. In accordance with instructions from user U via information terminal 8, cloud server 1 edits the stored video data, creates it as, for example, driving memory information, and transmits it to information terminal 8. Note that cloud server 1 may also perform editing processing using the video data transmitted by user U from information terminal 8.

[0020] Fig. 2 is an example of a screen provided to the information terminal 8. In the example screen of Fig. 2, the date and time of the drive, the route traveled by the vehicle, and the captured video data are displayed as memory information.

[0021] As an initial setting, the user U registers in advance, via the information terminal 8, information (manufacturer, model, ID, etc.) that identifies the drive recorder 6 installed in his / her vehicle Q, in the cloud server 1. The cloud server 1 identifies the cloud server 5 for the drive recorder of the relevant model of the relevant manufacturer, and communicates with this cloud server 5. The cloud server 1 receives from the cloud server 5 the video data recorded by the drive recorder with the ID registered by the user U. The cloud server 1 then edits the received video data, creating, for example, driving memory information, and transmits it to the information terminal 8 of the user U.

[0022] Furthermore, when the user U changes the drive recorder 6 of his / her vehicle Q, the user U registers information identifying the new drive recorder 6 (manufacturer, model, ID, etc.) in the cloud server 1 via the information terminal 8. Then, the cloud server 1 identifies the cloud server 5 for the new drive recorder and communicates with this cloud server 5. The cloud server 1 receives from the cloud server 5 the video data recorded by the drive recorder with the ID registered by the user U. The cloud server 1 then edits the received video data, creating, for example, driving memory information, and transmits it to the information terminal 8 of the user U.

[0023] Furthermore, the cloud server 1 may automatically execute a process such as displaying images captured on the travel path as thumbnails, in addition to following instructions from the user.

[0024] 1 includes a cloud server 5 that communicates with a drive recorder 6 installed in a vehicle Q, and a cloud server 1 that communicates with an information terminal 8 carried by a user. The cloud server 5 receives and temporarily stores video data from the drive recorder 6. The cloud server 1 receives and stores the video data from the cloud server 5, and edits the stored video data, for example, in accordance with an instruction from a user U via the information terminal 8, or automatically, and transmits the edited video data to the information terminal 8. This allows the user U to view the edited driving video data via the information terminal 8.

[0025] As described above, according to the driving video data recording and processing system of this embodiment, video data acquired by the drive recorder 6 is transmitted to the cloud server 1 via the cloud server 5, and is then consolidated and managed in the cloud server 1. Therefore, there is no need to install equipment with complex functions in the vehicle. Furthermore, even if the drive recorder 6 is replaced, the cloud server 1 can maintain the functionality of the driving video data recording and processing system by communicating with the cloud server for the new drive recorder.

[0026] Furthermore, since a driving video data recording and processing system can be realized using the drive recorder 6, there is no need to install new equipment in the vehicle. Note that the drive recorder 6 is an example of equipment installed in a vehicle and equipped with a camera in the present disclosure. In addition to the drive recorder, for example, equipment for ADAS (Advanced Driver-Assistance Systems) equipped with a camera may be used. Alternatively, a camera-equipped information terminal equipped with an app for a drive record function may be used.

[0027] Furthermore, for example, if the driver recorder 6 has a GPS function, it may transmit GPS information together with the video data. That is, a device mounted on a vehicle and having a camera may transmit location information of the vehicle Q together with the video data captured by the camera to the cloud server 5. This allows the location information of the vehicle Q to be used in the editing process of the video data in the cloud server 1, making the editing process easier.

[0028] <Configuration example> 3 shows an example of the configuration of a driving video data recording and processing system according to an embodiment. In FIG. 3, a content management server 1 corresponds to the cloud server 1 in FIG. 1, and a device management server 5 corresponds to the cloud server 5 in FIG.

[0029] 3 includes a content management server 1 that records video data of a vehicle Q (e.g., an automobile) while it is traveling. Note that the functions of the content management server 1 described below may be implemented in a processor, integrated circuit, conventional circuit, circuitry, or processing circuitry that is stored in a memory or the like and executes computer-readable instructions (e.g., a program) to perform predetermined processing steps to perform a specific function.

[0030] 3, the content management server 1 includes a device gateway 2, a content manager 3, a backend for frontend (BFF) 4, a messaging basis 11, and a function group 14. The content management server 1 is a cloud-based server that receives and manages event notifications or content data Dc from vehicles Q (Q1, Q2, Q3, ...).

[0031] The device gateway 2 is connected to a network such as the Internet, and has a communication interface for communicating with other computers, servers, information terminals, etc. The device gateway 2 communicates with the device management server 5.

[0032] The BFF 4 functions as a data connection window with a mobile application implemented in the information terminal 8. Specifically, the BFF 4 has a communication interface that receives and processes commands from a mobile application implemented in the information terminal 8 of the user U.

[0033] The content management unit 3 integrates the video data Dv1 transmitted from the device management server 5 and the video data Dv2 transmitted from the information terminal 8, stores them in storage, and centrally manages them. In this centralized management, the content management unit 3 manages "content files" and "content information" separately. "Management" here refers to enabling each application module 15 in the functional group 14 and the information terminal 8 to search, refer to, and retrieve "content files" and "content information" based on predetermined conditions. The "content files" and "content information" are related to each other, for example, via a content ID. This configuration enables centralized management of the video data Dv1 and Dv2. For example, when a mobile application (hereinafter referred to as "mobile app") on the information terminal 8 accesses the content management unit 3 to view videos or still images captured while driving, the search becomes easier, improving search speed and accuracy.

[0034] The messaging infrastructure 11 includes an event hub 12 and a message router 13. The messaging infrastructure 11 is provided in correspondence with the functions provided as a function group 14, and is used by application modules 15 for realizing each function. The messaging infrastructure 11 is a messaging system used for exchanging messages between application modules 15 included in the function group 14.

[0035] The function group 14 includes an application module 15 for realizing various functions and services (hereinafter collectively referred to as "functions"). The application module 15 includes a memory for storing application programs for realizing the various functions and services, a processor for executing the application programs, etc., and various functions are realized by executing the application programs. Examples of functions realized by the function group 14 include, for example, a "user information management function," a "drive log management function," a "video data management function," and a "memory movie (equivalent to a short movie) creation and editing function."

[0036] The content management server 1 may be configured to automatically link predetermined data using, for example, technology such as API linkage or SFTP linkage. Although a detailed description is omitted because it is not the subject of the disclosed technology, the content management server 1 is configured with hardware such as a processor, memory, and storage, and software such as control programs and data implemented in the memory. Each function of the content management server 1 is realized by a processor such as a CPU or GPU executing a program stored in the memory. The content management server 1 is not limited to a cloud server, and an on-premise server may also be used.

[0037] The device management server 5 acquires content data Dc (including positioning data Dp and video data Dv1) acquired by a managed drive recorder 6 through data linkage. Furthermore, for a drive recorder 6 that is a linkage target with the content management server 1, the device management server 5 performs data linkage between the content management server 1 and the content data Dc acquired from the drive recorder 6.

[0038] The content management server 1 is data-linked with the device management server 5. Specifically, when content data Dc is transmitted from a specific vehicle Q to the device management server 5, the device management server 5 transmits the content data Dc (including positioning data Dp and video data Dv1) of that vehicle Q to the content management server 1 via a communication interface. The content management server 1 receives the positioning data Dp and the video data Dv1 separately as separate files.

[0039] <Example of operation> A series of operational flows from the start of driving to the end of driving in the driving video data recording system M will be described with reference to the flowcharts of Figures 4 to 6. Figures 4 to 6 describe the operations and processes in the content management server 1.

[0040] When the power of the vehicle Q is turned on, power is supplied to the drive recorder 6, and the drive recorder 6 starts operating. Specifically, the drive recorder 6 starts capturing images with a camera and acquiring positioning data Dp with a positioning system. The drive recorder 6 shares content data Dc, including video data Dv1 captured by the camera and positioning data Dp acquired by the positioning system, with the device management server 5.

[0041] When the device management server 5 receives content data Dc (including video data Dv1 and positioning data Dp) from a drive recorder 6 that is the subject of data exchange with the content management server 1, it sends an event notification to the content management server 1.

[0042] In step S1 of Fig. 4, when the device gateway 2 receives an event notification from the device management server 5, it determines whether the event notification indicates the acquisition of video data Dv1 (S2). The device gateway 2 also executes an additional event determination process and a message transmission process (S4). Note that, in the example of Fig. 4, an example is described in which the process of step S2 and the process of step S4 are executed in parallel, but step S2 and step S4 may also be executed in series. For example, step S4 may be executed after the process of step S2, or step S2 may be executed after the process of step S4.

[0043] [Video data acquisition and storage processing] In step S2, if the event notification is a notification regarding acquisition of the video data Dv1 (YES in S2), the content management server 1 executes acquisition processing and storage processing of the video data Dv1 in step S3. In the flow diagram, the process proceeds to step S12 in FIG.

[0044] FIG. 5 is a flowchart showing an example of the process of acquiring and saving video data.

[0045] In step S12, the device gateway 2 requests the device management server 5 to acquire the video data Dv1. The device management server 5 then transmits the video data Dv1 to the device gateway 2, and the device gateway 2 receives the video data Dv1 (S13). At this time, the positioning data Dp and the video data Dv1 associated with the positioning data Dp are received separately. The associated positioning data Dp and the video data Dv1 do not need to be received at the same time; they may be received at different times. The video data Dv1 received by the device gateway 2 is transmitted to the content management unit 3, and the positioning data Dp is transmitted as topic information Dm to the event hub 12.

[0046] In step S14, the content management unit 3 determines whether the received video data Dv1 is normal driving video (e.g., time lapse video) taken during normal vehicle driving, or event video taken based on a specified input at a timing different from the normal driving video.

[0047] -For normal driving video data- In step S14, if the video data Dv1 is normal driving video data, the flow proceeds to step S15.

[0048] In step S15, the content management unit 3 assigns a content ID to the video data Dv1 and transmits the content ID to the device gateway 2. The content ID of the video data Dv1 received by the device gateway 2 is transmitted to the event hub 12 as topic information Dm. The content management unit 3 also performs a virus check on the video data Dv1 to confirm the safety of the data. Once the safety of the data is confirmed, the video data Dv1 is saved in storage. The content management unit 3 also executes a process to acquire content information. The acquired content information is associated with the content ID created earlier and saved in storage. The flow then proceeds to step S4 in Figure 4.

[0049] -For event video data- In step S14 of FIG. 5, if the video data Dv1 is event video data, the flow proceeds to step S16. In step S16, the same process as in step S15 is executed. In the next step S17, the content management unit 3 creates additional content based on the video data Dv1. The additional content is not particularly limited, but may be, for example, a thumbnail of the video data Dv1. The path of the additional content and the video data are saved in storage. Note that the processes of step S16 and step S17 may be executed in parallel. Thereafter, the flow proceeds to step S5 of FIG. 4.

[0050] In step S5 of Fig. 4, the device gateway 2 determines whether the state in which no event notification is received (hereinafter referred to as "no reception state") has exceeded a predetermined drive end determination period for determining the end of the vehicle drive. If the duration of the state in which no event notification is received is equal to or shorter than the drive end determination period (NO in S5), the processes in steps S1 to S5 described above are repeated. On the other hand, if the duration of the state in which no event notification is received exceeds the drive end determination period (NO in S5), it is determined that the drive has ended, and in step S6, processing associated with the drive end determination is performed.

[0051] [Additional event determination processing and message transmission processing] Next, the process of determining whether an additional event has occurred and the process of sending a message in step S4 of Fig. 4 will be described. As described above, when an event notification is received (S1), the device gateway 2 executes the process of determining whether an additional event has occurred and the process of sending a message (S4). Fig. 6 is a flowchart showing the process of determining whether an additional event has occurred and the process of sending a message.

[0052] 6, the device gateway 2 determines whether or not there is an additional event. Here, an additional event refers to an additional event that is different from the event notification related to the normal acquisition of a driving log received from the device gateway 2. For example, when the device gateway 2 receives the first event notification after the start of driving the vehicle Q, the device gateway 2 determines that there is an additional event of "start driving" described above.

[0053] If it is determined that there is an additional event, the flow proceeds from step S43 to step S44, and the device gateway 2 transmits a notification of the normal driving event and a notification of the additional event to the event hub 12 as topic information Dm. That is, the device gateway 2 transmits a notification of the event information to the event hub 12. The topic information Dm of the normal driving event may indicate, for example, "that photos and videos have been received from the drive record as normal driving log data," "that a specific content ID has been assigned to the content manager and saved," or "that positioning information has been received." Furthermore, the topic information Dm of the additional event may indicate, for example, "that the normal driving log data (photos and videos) is data indicating the start of a drive." The same applies when it is determined in step S6 of FIG. 4, which will be described later, that there is an additional event of "end of drive." For example, "that the drive has ended" is notified from the device gateway 2 to the event hub 12 as the topic information Dm of the additional event.

[0054] Returning to FIG. 6, in the next step S46, the topic information Dm received by the event hub 12 is passed to the message router 13 via the topic for the input message. The message router 13 then determines to which function each event should be sent, i.e., the application module 15 to which each event information (here, the normal driving event and the additional event) should be sent. Based on the determination of the message router 13, the event hub 12 sends each event information to the application module 15 to which it should be sent. For example, the event hub 12 notifies the drive log management function and the memory movie editing function of the additional events "normal driving event," "start of drive," and "end of drive" when they are received, or at a predetermined timing. The flow then proceeds to step S5 in FIG. 4.

[0055] On the other hand, if it is determined in step S42 that there is no additional event, the flow proceeds from step S43 to step S45, where the device gateway 2 transmits topic information Dm of a normal driving event to the event hub 12. For example, the normal driving log data from when the "drive starts" is determined to when the "drive ends" is determined generally corresponds to a state of "no additional event." In this case, the device gateway 2 transmits topic information Dm indicating a "normal driving event" to the event hub 12. A specific example of the topic information Dm of the normal driving event is the same as in step S44 described above. Thereafter, as in the case of "there is an additional event," in the next step S46, the topic information Dm received by the event hub 12 is passed to the message router 13, which determines the application module 15 to which the information is to be sent, and then passes it back to the event hub 12. Then, based on the determination of the message router 13, the event hub 12 transmits the event information to the application module 15 to which the information is to be sent. For example, the normal driving event is notified to the drive log management function. The flow then proceeds to step S5 of FIG. 4.

[0056] In the above-described embodiment, the system is applied to an automobile. However, the traveling video data recording and processing system according to the present disclosure can also be applied to vehicles other than automobiles, such as bicycles and motorbikes. Furthermore, the traveling video data recording and processing system according to the present disclosure can be applied not only to vehicles but also to various other moving bodies such as trains and airplanes.

[0057] The above embodiments may be combined as appropriate. The above embodiments are essentially preferred examples and are not intended to limit the scope of the technology disclosed herein, its applications, or its uses. [Industrial Applicability]

[0058] The technology disclosed herein can record video data while a vehicle is traveling, edit it, and provide it to users without using an onboard device with complex functions, making it useful, for example, for realizing a service that creates driving memory information. [Explanation of symbols]

[0059] 1. Cloud server for apps, content management server (second server) 5 Cloud server for drive recorder, device management server (first server) 6 Drive recorder (device) 8 Information terminals U User Q Vehicle

Claims

1. A system that records video data while a vehicle is running, edits the video data, and provides the video data to a user, a first server that communicates with a device equipped with a camera and that is mounted on a vehicle; a second server that communicates with an information terminal carried by a user; the first server and the second server are capable of communicating with each other; The first server receives the video data captured by the camera from the device and temporarily stores it; The second server receives and stores the video data captured by the camera from the first server, edits the stored video data, and transmits the edited video data to the information terminal. Driving video data recording and processing system.

2. 2. The driving video data recording and processing system according to claim 1, The device is a drive recorder or a device for ADAS (Advanced Driver Assistance Systems). Driving video data recording and processing system.

3. 2. The driving video data recording and processing system according to claim 1, The device transmits the position information of the vehicle together with the video data captured by the camera. Driving video data recording and processing system.

4. A method for recording video data of a vehicle while it is traveling, editing the video data, and providing the video data to a user in a system that includes a first server that communicates with a device equipped with a camera and that is mounted on a vehicle, and a second server that communicates with an information terminal carried by a user, the first server and the second server being capable of communicating with each other, the method comprising: the first server receiving the video data captured by the camera from the device and temporarily storing the video data; and a step in which the second server receives and stores the video data captured by the camera from the first server, edits the stored video data, and transmits the edited video data to the information terminal. Driving video data recording and processing method.

Images