Video processing system, video processing method, display method

Abstract

It is easy to recognize where the displayed image was taken while driving. [Solution] The control unit 3 generates data for displaying the integrated image P5, which includes an image P10 related to the vehicle 10 shown in video data obtained by the camera 151 while the vehicle 10 is in motion, and a map image P20 shown in map data. The integrated image P5 includes an image P10 related to the vehicle 10 shown in video data obtained by the camera 151 while the vehicle 10 is in motion, and a line L1 indicating the driving trajectory of the vehicle 10, which is generated based on a plurality of position information obtained in chronological order by the positioning unit 152 while the vehicle 10 is in motion, is placed on the map image P20, and a point Q1 indicating the position of the vehicle 10 on the map image P20 moves along the line L1 in synchronization with the temporal changes in the image P10 related to the vehicle.

Description

Technical Field

[0001] The technology disclosed herein relates to a video processing system, a video processing method, and a display method.

Background Art

[0002] Patent Document 1 discloses a shooting information browsing system including a server, a vehicle, and a terminal device. In this system, the server enables the terminal device to browse information collected from the vehicle. The above information includes shooting location information capable of specifying the shooting location, shooting image data obtained by shooting with an out-vehicle shooting means, and the like.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

[0007] In the above configuration, by referring to a point that moves along a line indicating the driving trajectory placed on the "map image" included in the integrated image (a point indicating the vehicle's position on the map image), it is easy to recognize where the "vehicle-related image" included in the integrated image was taken while the vehicle was traveling.

[0008] In the above video processing system, The control unit may generate data for displaying the integrated image such that a figure enclosing the point is placed on the map image included in the integrated image, and the figure moves in accordance with the movement of the point.

[0009] In the above configuration, the position of a point in a map image can be easily recognized by referring to the shapes (shapes that enclose points) placed on the map image.

[0010] In the above video processing system, The aforementioned camera may include an external camera that photographs the area outside the vehicle. The images related to the vehicle may include images of the vehicle's exterior.

[0011] With the above configuration, it is easy to recognize where the vehicle was traveling when the "exterior vehicle image" included in the integrated image was taken.

[0012] The technology disclosed herein relates to a video processing method that uses a computer to process video data obtained by a camera while a vehicle having a camera and a positioning unit for acquiring location information is in motion, and this video processing method is: Input step, The process includes a generation step, The image shown in the video data obtained by the camera is an image related to the vehicle and is an image that changes over time. The position information obtained by the positioning unit indicates the position of the vehicle. In the input step, the video data obtained by the camera while the vehicle is in motion, a plurality of location information obtained in chronological order by the positioning unit while the vehicle is in motion, and map data showing a map including the vehicle's driving trajectory are input. In the generation step, data for displaying the integrated image is generated such that, in an integrated image including an image related to the vehicle shown in the video data and an image of the map shown in the map data, a line indicating the vehicle's driving trajectory, generated based on the plurality of location information, is placed on the map image, and a point indicating the vehicle's position in the map image moves along the line in synchronization with the temporal changes in the image related to the vehicle.

[0013] The method described above allows for easy recognition of the location of a vehicle in a vehicle-related image by referring to a point (a point indicating the vehicle's position in the map image) that moves along a line representing the vehicle's trajectory, which is placed on the map image included in the integrated image.

[0014] The technology disclosed herein relates to a display method that causes a computer to display an integrated image on a display device, which includes an image shown in video data obtained by a camera while a vehicle having a camera and a positioning unit for acquiring location information is in motion, and an image of a map including the vehicle's driving trajectory, wherein the display method is A receiving step of receiving data for displaying the integrated image, The system includes a display step of displaying the integrated image on the display device based on data for displaying the integrated image, The image shown in the video data obtained by the camera is an image related to the vehicle and is an image that changes over time. The position information obtained by the positioning unit indicates the position of the vehicle. The data for displaying the integrated image includes the video data obtained by the camera while the vehicle is in motion, and map data showing a map that includes the vehicle's driving trajectory, generated based on a plurality of location information obtained in chronological order by the positioning unit while the vehicle is in motion. In the display step, the integrated image, which includes an image related to the vehicle shown in the video data and an image of the map shown in the map data, is displayed on the display device such that a line indicating the vehicle's driving trajectory, generated based on the plurality of location information, is placed on the map image, and a point indicating the vehicle's position in the map image moves along the line in synchronization with the temporal changes in the image related to the vehicle.

[0015] The method described above allows for easy recognition of the location of a vehicle in a vehicle-related image by referring to a point (a point indicating the vehicle's position in the map image) that moves along a line representing the vehicle's trajectory, which is placed on the map image included in the integrated image. [Effects of the Invention]

[0016] According to the technology disclosed herein, it is possible to easily recognize where the image (image related to the vehicle) included in the integrated image was taken while the vehicle was traveling.

Brief Description of the Drawings

[0017] [Figure 1] It is a schematic diagram illustrating the configuration of the system according to the embodiment. [Figure 2] It is a flowchart illustrating the flow of the recording process. [Figure 3] It is a flowchart illustrating the flow of the video recording process. [Figure 4] It is a flowchart illustrating the flow of the list display process. [Figure 5] It is a schematic diagram illustrating an out-of-vehicle list image showing a list of out-of-vehicle images. [Figure 6] It is a schematic diagram illustrating an in-vehicle list image showing a list of in-vehicle images. [Figure 7] It is a flowchart illustrating the flow of the integrated display process. [Figure 8] It is a schematic diagram illustrating an integrated image including an out-of-vehicle image. [Figure 9] It is a schematic diagram illustrating an integrated image including an in-vehicle image. [Figure 10] It is a schematic diagram illustrating an integrated image including a time information area and a position information area. [Figure 11] It is a block diagram illustrating the configuration of the second cloud server.

Modes for Carrying Out the Invention

[0018] Hereinafter, embodiments will be described in detail with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals and their description will not be repeated.

[0019] (System) FIG. 1 illustrates the configuration of the system according to the embodiment. This system includes a vehicle 10, a first cloud server 20, a second cloud server 30, and an information terminal 40.

[0020] 〔vehicle〕 Vehicle 10 is ridden by a user. Examples of vehicle 10 include four-wheeled vehicles and motorcycles. Examples of users include the driver who operates vehicle 10 and passengers who do not drive vehicle 10 but ride in it.

[0021] In this example, vehicle 10 has a drive recorder 15. The drive recorder 15 includes a camera 151, a positioning unit 152, a communication unit 153, a storage unit 154, and a control unit 155.

[0022] Camera 151 is positioned on the vehicle 10 and generates video data showing images related to the vehicle 10 by capturing images. Camera 151 generates video data showing images within a predetermined shooting range by capturing images within that range. The images shown in the video data obtained by camera 151 include at least one of video and still images. Video includes time lapses. Hereinafter, images related to the vehicle 10 will be referred to as "vehicle-related images". In addition, time information indicating the time when the video data was obtained is associated with the video data.

[0023] In this example, camera 151 includes an external camera 156 and an internal camera 157.

[0024] The external camera 156 generates video data showing the outside of the vehicle 10 by capturing images of the area outside the vehicle. For example, the external image shows the scenery in front of the vehicle 10. Hereafter, the video data showing the outside of the vehicle 10 will be referred to as "external video data". The external image is an example of a vehicle-related image.

[0025] The in-vehicle camera 157 generates video data showing images of the interior of vehicle 10 by capturing images of the interior. For example, the in-vehicle image shows the view of the interior of vehicle 10 from the front. Hereafter, the video data showing images of the interior of vehicle 10 will be referred to as "in-vehicle video data". The in-vehicle image is an example of a vehicle-related image.

[0026] The positioning unit 152 acquires location information indicating the position (specifically, latitude and longitude) of the vehicle 10 on which the drive recorder 15 is installed. For example, the positioning unit 152 acquires location information using GPS (Global Positioning System). In addition, time information indicating the time when the location information was obtained is associated with the location information.

[0027] The communication unit 153 transmits and receives various types of information and data. The communication unit 153 is a communication interface provided for communication between the drive recorder 15 and an external device (e.g., the first cloud server 20) and the control unit 155. For example, the communication unit 153 is composed of a communication device having modulation and / or demodulation functions.

[0028] The memory unit 154 stores various types of information and data. Specifically, the memory unit 154 stores information and data used for control or information processing in the drive recorder 15 (e.g., setting values ​​such as thresholds), and information and data obtained by various parts of the drive recorder 15 (e.g., video data and location information). For example, the memory unit 154 is composed of non-volatile memory.

[0029] The control unit 155 is connected to each part of the drive recorder 15 (in this example, the camera 151, positioning unit 152, communication unit 153, and storage unit 154) by signal lines and controls each part of the drive recorder 15. The control unit 155 also sends and receives various information and data via the communication unit 153. The control unit 155 then performs various processes based on instructions input to the drive recorder 15, various information and data stored in the storage unit 154, etc. In this example, the control unit 155 controls the camera 151, transmits video data, transmits location information, etc.

[0030] For example, the control unit 155 is composed of a computer having a processor and a memory for storing various programs executed by the processor. The various processes (functions) of the control unit 155 are realized by the processor executing various programs. The control unit 155 may also include a microcontroller that performs predetermined control (calculations). Furthermore, the control unit 155 may be realized solely by hardware such as dedicated circuits, or by a combination of hardware and software such as programs.

[0031] [First Cloud Server] The first cloud server 20 communicates with the drive recorder 15. The first cloud server 20 is composed of one or more information processing devices. The first cloud server 20 has a storage unit 21 and a control unit 22.

[0032] The memory unit 21 stores various types of information and data. Specifically, the memory unit 21 stores information and data used for control or information processing in the first cloud server 20 (e.g., setting values ​​such as thresholds), information and data input to the first cloud server 20 (e.g., video data and location information), and information and data obtained in the first cloud server 20. For example, the memory unit 21 is composed of non-volatile memory.

[0033] The control unit 22 performs various processes based on instructions input to the first cloud server 20, various information and data stored in the storage unit 21, etc. In this example, the control unit 22 receives and stores video data and location information, and transmits video data and location information.

[0034] For example, the control unit 22 is composed of a computer having a processor and a memory for storing various programs executed by the processor. The various processes (functions) of the control unit 22 are realized by the processor executing various programs. The control unit 22 may also include a microcontroller that performs predetermined control (calculations). Furthermore, the control unit 22 may be realized solely by hardware such as dedicated circuits, or by a combination of hardware and software such as programs.

[0035] [Second Cloud Server] The second cloud server 30 communicates with the information terminal 40. The second cloud server 30 also communicates with the first cloud server 20. The second cloud server 30 is composed of one or more information processing devices. The second cloud server 30 has a storage unit 31 and a control unit 32.

[0036] The storage unit 31 stores various types of information and data. Specifically, the storage unit 31 stores information and data used for control or information processing in the second cloud server 30 (e.g., setting values ​​such as thresholds), information and data input to the second cloud server 30 (e.g., video data and location information), and information and data obtained in the second cloud server 30. For example, the configuration of the storage unit 31 is the same as that of the storage unit 21.

[0037] The control unit 32 performs various processes based on instructions input to the second cloud server 30, various information and data stored in the storage unit 31, etc. In this example, the control unit 22 receives and stores video data and location information, edits video data, and transmits video data. For example, the configuration of the control unit 32 is the same as that of the control unit 22.

[0038] [Information terminal] The information terminal 40 is used by the user. The information terminal 40 has application software (hereinafter referred to as "app") installed for using the video processing system 1, which will be described later. This enables the information terminal 40 to communicate with the second cloud server 30.

[0039] Examples of information terminals 40 include portable information terminals such as smartphones, tablets, and notebook personal computers, and stationary information terminals such as desktop personal computers. In this example, the information terminal 40 has an operation unit 41, a display unit 42, a communication unit 43, a storage unit 44, and a control unit 45.

[0040] The operation unit 41 is operated by the user of the information terminal 40. The operation input to the operation unit 41 is transmitted to the control unit 45. Specifically, the operation unit 41 transmits an electrical signal to the control unit 45 corresponding to the operation input to the operation unit 41. As a result, information corresponding to the operation input to the operation unit 41 is input to the control unit 45. Examples of the operation unit 41 include a keyboard, mouse, and touchpad.

[0041] The display unit 42 displays video for the user of the information terminal 40. Specifically, the display unit 42 displays video by playing video based on video data supplied from the control unit 45. An example of the display unit 42 is a display device.

[0042] The communication unit 43 transmits and receives various types of information and data. The communication unit 153 is a communication interface provided for communication between the information terminal 40 and external equipment (e.g., the second cloud server 30) and the control unit 45. For example, the communication unit 43 is composed of a communication device having modulation and / or demodulation functions.

[0043] The storage unit 44 stores various types of information and data. Specifically, the storage unit 44 stores information and data used for control or information processing in the information terminal 40 (e.g., setting values ​​such as thresholds), information and data input from outside the information terminal 40 (e.g., video data), and information and data obtained by various parts of the information terminal 40. For example, the storage unit 44 is composed of non-volatile memory.

[0044] The control unit 45 is connected to each part of the information terminal 40 (in this example, the operation unit 41, the display unit 42, the communication unit 43, and the storage unit 44) by signal lines and controls each part of the information terminal 40. The control unit 45 also sends and receives various information and data via the communication unit 43. The control unit 45 then performs various processes based on instructions input to the information terminal 40, various information and data stored in the storage unit 44, etc. In this example, the control unit 45 receives and plays back video data, etc.

[0045] For example, the control unit 45 is composed of a computer having a processor and a memory for storing various programs executed by the processor. The various processes (functions) of the control unit 45 are realized by the processor executing various programs. The control unit 45 may also include a microcontroller that performs predetermined control (calculations). Furthermore, the control unit 45 may be realized solely by hardware such as dedicated circuits, or by a combination of hardware and software such as programs.

[0046] [Image Processing System] The system shown in Figure 1 includes a video processing system 1. The video processing system 1 processes video data transmitted from the vehicle 10. The processing performed by the video processing system 1 is an example of a video processing method. The video data processed by the video processing system 1 is displayed on the display unit (display device) of other devices such as the information terminal 40. In other words, the video processing system 1 can be described as a video processing system for display.

[0047] The video processing system 1 comprises a storage unit 2 and a control unit 3. In this example, the video processing system 1 is configured on a second cloud server 30. The storage unit 2 and the control unit 3 are composed of the storage unit 31 and the control unit 32 of the second cloud server 30. The second cloud server 30 is an example of a server located outside the vehicle 10 and configured to communicate with an information terminal 40.

[0048] <Storage Unit and Control Unit> In this example, the control unit 3 determines whether the video data transmitted from the vehicle 10 is "external video data showing an external image of the vehicle 10 obtained by the external camera 156" or "in-vehicle video data showing an internal image of the vehicle 10 obtained by the internal camera 157". In this example, "video data transmitted from the vehicle 10" is the video data transmitted from the vehicle 10 to the second cloud server 30 via the first cloud server 20.

[0049] The control unit 3 then stores the video data transmitted from the vehicle 10 in the storage unit 2 so that it can distinguish between external video data and internal video data based on the results of the above determination. Specifically, the control unit 3 generates discrimination information indicating whether the video data is external video data or internal video data based on the results of the above determination. The control unit 3 then stores the video data transmitted from the vehicle 10 and the discrimination information in the storage unit 2. The video data and discrimination information are associated with each other. For example, the control unit 3 associates the video data and discrimination information with each other by associating a common identifier (such as an ID) with both the video data and the discrimination information.

[0050] In this example, the memory unit 2 stores video data obtained by the camera 151 while the vehicle 10 is in motion, multiple location information obtained in chronological order by the positioning unit 152 while the vehicle 10 is in motion, and map data showing a map including the vehicle 10's driving trajectory.

[0051] The control unit 3 then generates data for displaying the integrated image, which includes an integrated image containing "an image related to the vehicle shown in the video data (vehicle-related image)" and "an image of the map shown in the map data (map image)." The data is generated such that a line indicating the driving trajectory of the vehicle 10, generated based on multiple location information, is placed on the map image, and a point indicating the position of the vehicle 10 in the map image moves along the line (the line indicating the driving trajectory of the vehicle 10) in synchronization with the temporal changes in the vehicle-related image.

[0052] The processing performed in the system according to this embodiment (the system including the video processing system 1) will be explained in detail later.

[0053] [Association between user, camera, and information terminal] In the system according to this embodiment, a user is associated with a camera 151 (specifically, a drive recorder 15) and an information terminal 40. This association information (which camera 151 and which information terminal 40 are associated with which user) is stored in the storage unit 2 (storage unit 31 in this example) of the video processing system 1. By referring to this association information, various processes can be performed for each user.

[0054] For example, as an initial setup, the user operates the information terminal 40 to send user information (such as an ID) to identify the user and drive recorder information (such as the manufacturer, model, ID, and name of the vehicle 10) to identify the drive recorder 15 installed in the vehicle 10 to the video processing system 1 (in this example, the second cloud server 30). The control unit 3 (in this example, the control unit 32) of the video processing system 1 generates the above association information based on the information transmitted from the information terminal 40 and stores the generated association information in the storage unit 2. The control unit 3 performs various processes for each user by referring to the association information.

[0055] In this example, the control unit 3 identifies a first cloud server 20 that can communicate with the drive recorder 15 based on the drive recorder information transmitted from the information terminal 40, and sets the identified first cloud server 20 as the "first cloud server 20 for obtaining video data related to the user." The control unit 3 then registers information indicating this setting (which cloud server is the "first cloud server 20 for obtaining video data related to the user") in the association information.

[0056] [Processing performed within the system] Next, we will describe the processes performed in the system according to the embodiment (the system including the video processing system 1). In this example, recording processing, list display processing, and integrated display processing are performed. For the sake of simplicity, the following explanation will use the processing performed on "one user" among the users of the system according to the embodiment as an example.

[0057] [Recording process] Next, the recording process will be explained with reference to Figure 2. The recording process is the process of storing the video data and location information obtained in the vehicle 10 in the storage unit 2. When the vehicle 10 starts moving, the following process is repeated. The point at which the vehicle 10 starts moving is, for example, the point at which the vehicle 10 changes from a non-movable state (e.g., ignition off) to a movable state (e.g., ignition on).

[0058] <Step S11> The camera 151 of the drive recorder 15 installed in the vehicle 10 takes pictures. The camera 151 takes pictures continuously (uninterruptedly in the case of video, and periodically at predetermined intervals in the case of still images). This obtains video data of the vehicle 10 while it is in motion. The control unit 155 of the drive recorder 15 associates "time information indicating the time when the video data was obtained" with the video data obtained by the camera 151, and stores the video data associated with that time information in the storage unit 154.

[0059] Furthermore, the positioning unit 152 of the drive recorder 15 located in the vehicle 10 acquires location information indicating the position of the vehicle 10. The control unit 155 of the drive recorder 15 associates "time information indicating the time when the video data was obtained" with the location information obtained by the positioning unit 152, and stores the location information associated with that time information in the storage unit 154.

[0060] <Step S12> Next, the control unit 155 of the drive recorder 15 transmits the "video data associated with time information" and "location information associated with time information" stored in the storage unit 154 in step S11 to the first cloud server 20. The video data and location information transmitted to the first cloud server 20 are then erased from the storage unit 154. The storage unit 154 functions as a buffer for temporarily storing the video data and location information. The transmission of video data and location information is performed at predetermined intervals (for example, every second).

[0061] The control unit 22 of the first cloud server 20 stores the "video data associated with time information" and "location information associated with time information" transmitted from the drive recorder 15 in the storage unit 21 of the first cloud server 20.

[0062] <Step S13> Next, the control unit 22 of the first cloud server 20 transmits the "video data associated with time information" and "location information associated with time information" stored in the storage unit 21 in step S12 to the second cloud server 30.

[0063] <Step S14> Next, the control unit 32 of the second cloud server 30 (control unit 3 of the video processing system 1) stores the "video data associated with time information" and "location information associated with time information" transmitted from the first cloud server 20 in step S13 in the storage unit 31 of the second cloud server 30 (storage unit 2 of the video processing system 1).

[0064] In this example, video recording is performed in step S14. The video recording process is a process to store the above video data (video data transmitted from vehicle 10) in the storage unit 2 so that external video data and internal video data can be distinguished. The video recording process will be explained in detail later.

[0065] Furthermore, the above processes are performed continuously from the start of vehicle 10's journey until the end of its journey. For example, the start of vehicle 10's journey is the point at which vehicle 10 changes from a non-operational state (e.g., ignition off) to an operationable state (e.g., ignition on), and the end of vehicle 10's journey is the point at which vehicle 10 changes from an operationable state to a non-operational state. In addition, in order to maintain real-time performance, the transmission and reception of the above data and information are performed immediately (as quickly as possible).

[0066] As a result of the above processing, "video data obtained by the camera 151 placed on the vehicle 10" The "location information indicating the position of vehicle 10" can be stored at any time in the storage unit 31 of the second cloud server 30 (storage unit 2 of the video processing system 1).

[0067] In this example, the series of video data obtained from the start to the end of the vehicle 10's journey is managed as a single drive data. Specifically, the control unit 32 of the second cloud server 30 (control unit 3 of the video processing system 1) stores the series of video data obtained from the start to the end of the vehicle 10's journey as a single drive data in the storage unit 31 of the second cloud server 30 (storage unit 2 of the video processing system 1).

[0068] [Video recording processing] Next, with reference to Figure 3, the video recording process (details of step S14 of the recording process) will be described. In step S14 of the recording process, the control unit 32 of the second cloud server 30 (control unit 3 of the video processing system 1) performs the following processing.

[0069] <Step S15 (Judgment Step)> First, the control unit 32 determines whether the video data transmitted from the first cloud server 20 (video data transmitted from the vehicle 10 via the first cloud server 20) corresponds to "external video data obtained by the vehicle 10's external camera 156" or "in-vehicle video data obtained by the vehicle 10's internal camera 157". If the video data corresponds to "external video data", the process in step S16 is performed, and if the video data corresponds to "in-vehicle video data", the process in step S18 is performed.

[0070] In this example, the video data is associated with an identifier that identifies the camera that generated the video data. The association of the identifier with the video data may be performed at camera 151 or at the control unit 155 that controls camera 151. The video data obtained by the external camera 156 is associated with an identifier indicating "external camera 156," and the video data obtained by the internal camera 157 is associated with an identifier indicating "internal camera 157."

[0071] The control unit 32 makes the above determination based on the identifier associated with the video data transmitted from the first cloud server 20. Specifically, the control unit 32 determines that the video data is "external video data" if the identifier associated with the video data indicates "external camera 156", and determines that the video data is "in-vehicle video data" if the identifier associated with the video data indicates "in-vehicle camera 157".

[0072] <Step S16> If the video data transmitted from the vehicle 10 via the first cloud server 20 corresponds to "external vehicle video data," the control unit 32 generates discrimination information indicating that the video data corresponds to "external vehicle video data."

[0073] <Step S17 (Memory Step)> Next, the control unit 32 stores the above video data and the discrimination information generated in step S16 (discrimination information indicating that the above video data corresponds to "external vehicle video data") in the storage unit 31 of the second cloud server 30 (storage unit 2 of the video processing system 1). By referring to this discrimination information, the control unit 32 can determine that the above video data corresponds to "external vehicle video data".

[0074] <Step S18> On the other hand, if the video data transmitted from the vehicle 10 via the first cloud server 20 corresponds to "in-vehicle video data", the control unit 32 generates discrimination information indicating that the video data corresponds to "in-vehicle video data".

[0075] <Step S19 (Memory Step)> Next, the control unit 32 stores the above video data and the discrimination information generated in step S18 (discrimination information indicating that the above video data corresponds to "in-vehicle video data") in the storage unit 31 of the second cloud server 30 (storage unit 2 of the video processing system 1). By referring to this discrimination information, the control unit 32 can determine that the above video data corresponds to "in-vehicle video data".

[0076] Through the above processing, the video data transmitted from the vehicle 10 via the first cloud server 20 can be stored in the storage unit 31 of the second cloud server 30 (storage unit 2 of the video processing system 1) in such a way that external video data and internal video data can be distinguished.

[0077] [List display processing] Next, the list display process will be explained with reference to Figure 4. In this example, when an operation to send a viewing request instruction to request viewing of a list of vehicle-related images (images related to vehicle 10) is input to the operation unit 41 of the information terminal 40, the control unit 45 of the information terminal 40 sends the viewing request instruction to the second cloud server 30 (image processing system 1) via the communication unit 43. When the above viewing request instruction is sent to the second cloud server 30, the following processing is performed.

[0078] <Step S21> The control unit 32 of the second cloud server 30 (control unit 3 of the video processing system 1) extracts video data corresponding to the viewing request instruction from the storage unit 31 of the second cloud server 30 (storage unit 2 of the video processing system 1). Specifically, if the viewing request instruction is an instruction to "request viewing a list of exterior images," the control unit 32 extracts exterior video data representing the exterior images from the storage unit 31. On the other hand, if the viewing request instruction is an instruction to "request viewing a list of interior images," the control unit 32 extracts interior video data representing the interior images from the storage unit 31.

[0079] <Step S22> Next, the control unit 32 generates data for displaying a list of images (vehicle-related images) shown in the video data extracted in step S21. Specifically, if the viewing request instruction is an instruction to "request viewing a list of exterior images," the control unit 32 generates data for displaying the list of exterior images. On the other hand, if the viewing request instruction is an instruction to "request viewing a list of interior images," the control unit 32 generates data for displaying the list of interior images.

[0080] <Step S23> Next, the control unit 32 transmits the "data for displaying the list of images" generated in step S22 to the information terminal 40.

[0081] <Step S24> Next, the control unit 45 of the information terminal 40 receives the "data for displaying a list of images" transmitted from the control unit 32 of the second cloud server 30 in step S23, and based on the "data for displaying a list of images," it causes the display unit 42 of the information terminal 40 to display a list image showing the list of images. Specifically, if the viewing request instruction is an "instruction to request viewing of a list of exterior images," the display unit 42 displays an exterior list image showing a list of exterior images. On the other hand, if the viewing request instruction is an "instruction to request viewing of a list of interior images," the display unit 42 displays an interior list image showing a list of interior images.

[0082] [Exterior view images] Figure 5 illustrates an example of an exterior view list image P1. As shown in Figure 5, the exterior view list image P1 contains a list of exterior view images P11. In this example, the exterior view list image P1 contains a list of thumbnails (reduced versions of exterior view images P11) of the exterior view images P11.

[0083] In this example, the exterior view image P1 includes the exterior button B1 and the interior button B2. When an operation to select the "interior button B2" included in the exterior view image P1 displayed on the display unit 42 of the information terminal 40 (for example, the user touching the interior button B2 with their finger) is input to the operation unit 41 of the information terminal 40, the control unit 45 of the information terminal 40 sends a viewing request instruction to the second cloud server 30 (video processing system 1) requesting to view a list of interior images. As a result, the list image displayed on the display unit 42 of the information terminal 40 switches from the "exterior view image P1" to the "interior view image P2 (see Figure 6)".

[0084] Furthermore, when an operation to select one exterior image P11 from among multiple exterior images P11 included in the exterior list image P1 (for example, when the user touches one exterior image P11 with their finger) is input to the operation unit 41 of the information terminal 40, the control unit 45 of the information terminal 40 sends a viewing request instruction to the second cloud server 30 (video processing system 1) requesting to view the integrated image (see Figure 8) that includes the selected exterior image P11. This initiates the integrated display process, which will be described later.

[0085] [Images of the interior of the train] Figure 6 illustrates an example of an interior viewer image P2. As shown in Figure 6, the interior viewer image P2 contains a list of interior viewer images P12. In this example, the interior viewer image P2 contains a list of thumbnails (reduced versions of interior viewer images P12).

[0086] In this example, the in-vehicle list image P2 includes both the exterior button B1 and the in-vehicle button B2. When an operation to select the "exterior button B1" included in the in-vehicle list image P2 displayed on the display unit 42 of the information terminal 40 (for example, the user touching the exterior button B1 with their finger) is input to the operation unit 41 of the information terminal 40, the control unit 45 of the information terminal 40 sends a viewing request instruction to the second cloud server 30 (video processing system 1) requesting to view a list of exterior images. As a result, the list image displayed on the display unit 42 of the information terminal 40 switches from the "in-vehicle list image P2" to the "exterior list image P1 (see Figure 5)".

[0087] Furthermore, when an operation to select one interior image P12 from among multiple interior images P12 included in the interior list image P2 (for example, when the user touches one interior image P12 with their finger) is input to the operation unit 41 of the information terminal 40, the control unit 45 of the information terminal 40 sends a viewing request instruction to the second cloud server 30 (video processing system 1) requesting to view the integrated image (see Figure 9) that includes the selected interior image P12. This initiates the integrated display process described later.

[0088] [Integrated display processing] Next, the integrated display process will be described with reference to Figure 7. In this example, when an operation to send a viewing request instruction to request viewing of a list of integrated images (see Figures 8 and 9) is input to the operation unit 41 of the information terminal 40, the control unit 45 of the information terminal 40 sends the viewing request instruction to the second cloud server 30 (video processing system 1) via the communication unit 43. The above viewing request instruction includes selection information indicating the image (vehicle-related image) selected by the user. When the above viewing request instruction is sent to the second cloud server 30, the following processing is performed.

[0089] <Step S31> The control unit 32 of the second cloud server 30 (control unit 3 of the video processing system 1) extracts a series of video data from the storage unit 31 of the second cloud server 30 (storage unit 2 of the video processing system 1), including video data showing the image indicated in the selection information included in the viewing request instruction (vehicle-related image selected by the user). This series of video data is a series of videos obtained from the start to the end of the vehicle 10's journey, and is a collection of video data managed as a single drive data. The images shown in this video data (series of video data) are vehicle-related images associated with the vehicle 10, and are images (moving images) that change over time.

[0090] <Step S32> Furthermore, the control unit 32 extracts from the storage unit 31 "multiple location information obtained in chronological order" corresponding to the video data (series of video data) extracted in step S31. The time at which these multiple location pieces were obtained corresponds to the time at which the above-mentioned video data (series of video data) was obtained. For example, the time at which the kth (k is an integer) of the multiple location pieces arranged in chronological order was obtained is the same as the time at which the kth video data was obtained among the multiple video data arranged in chronological order. In addition, the multiple location pieces arranged in chronological order indicate the driving trajectory of the vehicle 10.

[0091] <Step S33> Furthermore, the control unit 32 acquires map data that includes the driving trajectory of the vehicle 10, as indicated in the "multiple location information obtained in chronological order" extracted in step S32, and stores the map data in the storage unit 31. For example, the map data is map data distributed from an external system (not shown) that manages map data. The control unit 32 requests the distribution of map data from the above system and receives the map data distributed from the above system.

[0092] <Step S34> Next, the control unit 32 generates data to display a line indicating the vehicle 10's trajectory (see line L1 in Figures 8 and 9) on the map image shown in the map data obtained in step S33, based on the multiple location information obtained in step S32 in chronological order. For example, the line indicating the vehicle 10's trajectory is generated by connecting points corresponding to the vehicle 10's position shown in each of the multiple location information obtained in chronological order (specifically, points identified by longitude and latitude) with lines.

[0093] <Step S35> Next, the control unit 32 generates data for displaying points (see point Q1 in Figures 8 and 9) that are placed on the "line indicating the driving trajectory of the vehicle 10" shown in the data generated in step S34. The points placed on the "line indicating the driving trajectory of the vehicle 10" indicate the position of the vehicle 10 in the map image shown in the map data obtained in step S33.

[0094] Specifically, the control unit 32 generates the above data (data for displaying points placed on the line indicating the vehicle 10's travel trajectory) so that the points shown in the above data (points indicating the position of the vehicle 10 in the map image) move along the "line indicating the vehicle 10's travel trajectory" shown in the data generated in step S34, in synchronization with the temporal changes of the image shown in the video data (series of video data) extracted in step S31.

[0095] <Step S36> Next, the control unit 32 generates data to display a figure (see figure R1 in Figures 8 and 9) that encloses the "point" shown in the data generated in step S34. In this example, the figure is a circle centered on the point.

[0096] Specifically, the control unit 32 generates the above data (data for displaying the above-mentioned figure surrounding the points) so that the figure shown in the above data (the figure surrounding the points) moves in accordance with the movement of the points shown in the data generated in step S35 (points placed on the line indicating the travel trajectory of the vehicle 10).

[0097] <Step S37> Next, the control unit 32 generates data for displaying an integrated image (see Figures 8 and 9) based on the video data (series of video data) extracted in step S31, the map data obtained in step S33, and the data generated in steps S34 to S36. In this example, the control unit 32 generates data for displaying an integrated image by integrating this data.

[0098] <Step S38> Next, the control unit 3 transmits the "data for displaying the integrated image" generated in step S37 to the information terminal 40.

[0099] <Step S39 (Receiving step and display step)> Next, the control unit 45 of the information terminal 40 receives the "data for displaying the integrated image" transmitted from the control unit 32 in step S38, and displays the integrated image on the display unit 42 of the information terminal 40 based on the "data for displaying the integrated image". In this example, the integrated image includes an integrated image that includes an image of the outside of the vehicle and an integrated image that includes an image of the inside of the vehicle.

[0100] Specifically, the control unit 45 of the information terminal 40 displays the integrated image on the display unit 42 of the information terminal 40 such that, in an integrated image including "vehicle-related images" shown in video data (a series of video data) and "map images" shown in map data, a "line L1 indicating the driving trajectory of vehicle 10" generated based on multiple location information is placed on the map image, and a point Q1 indicating the position of vehicle 10 in the map image moves along line L1 in synchronization with the temporal changes in the vehicle-related images.

[0101] The process in step S39 is an example of a display method that displays an integrated image on the display unit 42 (display device) of an information terminal, which includes an image shown in video data obtained by the camera 151 while the vehicle 10, which has a camera 151 and a positioning unit 152 that acquires position information, is in motion, and an image of a map including the driving trajectory of the vehicle 10. The process in which the control unit 45 of the information terminal 40 receives "data for displaying the integrated image" is an example of a receiving step for receiving data for displaying the integrated image, and the process in which the control unit 45 of the information terminal 40 displays the integrated image on the display unit 42 of the information terminal 40 is an example of a display step for displaying the integrated image on the display device based on the data for displaying the integrated image.

[0102] [Integrated image including exterior view] Figure 8 illustrates an integrated image P5 that includes an external image P11. This integrated image P5 includes the external image P11 and a map image P20. In the example in Figure 8, the external image P11 and the map image P20 are adjacent to each other vertically within the integrated image P5.

[0103] Map image P20 contains a line L1 that shows the driving trajectory of vehicle 10. Point Q1 is placed on line L1, indicating the position of vehicle 10 in map image P20. Point Q1 moves along line L1 in synchronization with the temporal changes of the external image P11. The position of point Q1 corresponds to the position of vehicle 10 shown in the position information obtained at the same time as the external image P11 displayed in the integrated image P5 was obtained. In addition, a figure R1 enclosing point Q1 is placed on map image P20. Figure R1 moves in accordance with the movement of point Q1.

[0104] In this example, the integrated image P5, which includes the exterior image P11, also includes the exterior button B1 and the interior button B2. When an operation to select the "interior button B2" included in the "integrated image P5 including the exterior image P11" displayed on the display unit 42 of the information terminal 40 is input to the operation unit 41 of the information terminal 40, the control unit 45 of the information terminal 40 sends a viewing request instruction to the second cloud server 30 (video processing system 1) requesting to view the integrated image P5 including the interior image P12 (see Figure 9). As a result, the integrated image P5 displayed on the display unit 42 of the information terminal 40 switches from the "integrated image P5 including the exterior image P11" to the "integrated image P5 including the interior image P12 (see Figure 9)". Specifically, the vehicle-related image P10 included in the integrated image P5 switches from the "exterior image P11" to the "interior image P12".

[0105] [Integrated image including interior photos] Figure 9 illustrates an integrated image P5 that includes the in-vehicle image P12. This integrated image P5 includes the in-vehicle image P12 and the map image P20. The map image P20 is the same as the map image P20 shown in Figure 8.

[0106] In this example, the integrated image P5, which includes the interior image P12, also includes the exterior button B1 and the interior button B2. When an operation to select the "exterior button B1" included in the "integrated image P5 including the interior image P12" displayed on the display unit 42 of the information terminal 40 is input to the operation unit 41 of the information terminal 40, the control unit 45 of the information terminal 40 sends a viewing request instruction to the second cloud server 30 (video processing system 1) requesting to view the integrated image P5 including the exterior image P11 (see Figure 8). As a result, the integrated image P5 displayed on the display unit 42 of the information terminal 40 switches from the "integrated image P5 including the interior image P12" to the "integrated image P5 including the exterior image P11 (see Figure 8)". Specifically, the vehicle-related image P10 included in the integrated image P5 switches from the "interior image P12" to the "exterior image P11".

[0107] [Effects of the Embodiment] As described above, in the system according to the embodiment, the control unit 3 determines whether the video data transmitted from the vehicle 10 corresponds to "external video data showing an external image P11 of the vehicle 10 obtained by the external camera 156" or "in-vehicle video data showing an internal image P12 of the vehicle 10 obtained by the internal camera 157". Based on the result of the above determination, the control unit 3 stores the video data transmitted from the vehicle 10 in the storage unit 2 so that external video data and internal video data can be distinguished.

[0108] In the above configuration, by storing the video data transmitted from the vehicle 10 in the storage unit 2 in such a way that external video data and internal video data can be distinguished, it becomes easy to distinguish and extract external video data and internal video data.

[0109] Furthermore, in the system according to this embodiment, the control unit 3 generates discrimination information indicating whether the video data is external video data or internal video data, based on the result of the above determination. The control unit 3 then stores the video data transmitted from the vehicle 10 and the discrimination information in the storage unit 2.

[0110] In the above configuration, by referring to the discrimination information, it is possible to determine whether the video data stored in the memory unit 2 corresponds to external video data or internal video data. This makes it easy to distinguish and extract external video data and internal video data.

[0111] Furthermore, in the system according to this embodiment, a viewing request instruction is transmitted from an information terminal 40 operated by the user. When the viewing request instruction is an instruction to "request viewing a list of exterior images P11", the control unit 3 transmits data to the information terminal 40 for displaying a list of exterior images P11 shown in the exterior video data stored in the storage unit 2. Also, when the viewing request instruction is an instruction to "request viewing a list of interior images P12", the control unit 3 transmits data to the information terminal 40 for displaying a list of interior images P12 shown in the interior video data stored in the storage unit 2.

[0112] With the above configuration, it is easy to distinguish and extract external and internal video data, so it is easy to generate data for displaying a list of external images P11 (or a list of internal images P12). This makes it easy to display a list of external images P11 (or a list of internal images P12).

[0113] Furthermore, in the system according to this embodiment, the images shown in the video data obtained by the camera 151 are images related to the vehicle 10 (vehicle-related images P10) and are images that change over time. The position information obtained by the positioning unit 152 indicates the position of the vehicle 10. The storage unit 2 stores video data obtained by the camera 151 while the vehicle 10 is in motion, multiple pieces of position information obtained in chronological order by the positioning unit 152 while the vehicle 10 is in motion, and map data showing a map including the driving trajectory of the vehicle 10.

[0114] In the system according to this embodiment, the control unit 3 generates data for displaying the integrated image P5, which includes an integrated image P5 that is an image related to a vehicle shown in the video data, and a map image P20 that is an image of a map shown in the map data. The integrated image P5 includes a vehicle-related image P10, which is an image related to a vehicle shown in the video data, and a line L1 indicating the driving trajectory of the vehicle 10, generated based on a plurality of location pieces of information, is placed on the map image P20, and a point Q1 indicating the position of the vehicle 10 on the map image P20 moves along the line L1 in synchronization with the temporal changes of the vehicle-related image P10.

[0115] In the above configuration, by referring to point Q1 (a point indicating the position of vehicle 10 in map image P20) that moves along line L1 indicating the driving trajectory placed in map image P20 included in integrated image P5, it is possible to easily recognize where the "vehicle-related image P10" included in integrated image P5 was taken when the vehicle was traveling.

[0116] Furthermore, in the system according to this embodiment, the control unit 3 generates data for displaying the integrated image P5 such that a figure R1 surrounding point Q1 is placed on the map image P20 included in the integrated image P5, and the figure R1 moves in accordance with the movement of point Q1.

[0117] In the above configuration, the position of point Q1 on map image P20 can be easily recognized by referring to shape R1 (the shape enclosing point Q1) placed on map image P20.

[0118] Furthermore, in the system according to this embodiment, the camera 151 includes an external camera 156 that takes pictures of the area outside the vehicle, and the vehicle-related image P10 includes an external image P11 of the vehicle 10.

[0119] With the above configuration, it is easy to recognize where the vehicle was traveling when the external image P11 included in the integrated image P5 was taken.

[0120] [Example of integrated image] As shown in Figure 10, the integrated image P5 may include a time information area R11 and a location information area R12 in addition to the vehicle-related image P10 and map image P20 shown in Figures 8 and 9. In the example in Figure 10, the vehicle-related image P10, map image P20, time information area R11, and location information area R12 are arranged in order from top to bottom of the integrated image P5. Note that the order of these images and information is not limited to the order shown in Figure 10.

[0121] The time information area R11 is provided to display time information corresponding to the vehicle-related image P10 and the map image P20. The time displayed in the time information area R11 corresponds to the time when the video data showing the vehicle-related image P10 was obtained, and also corresponds to the time when the position information showing point Q1 shown in the map image P20 was obtained. The time displayed in the time information area R11 changes in accordance with the temporal changes of the vehicle-related image P10 and the map image P20 (especially the position of point Q1). The time displayed in the time information area R11 is generated based on multiple time information corresponding to the video data (a series of video data) showing the vehicle-related image P10.

[0122] Location information area R12 is provided to display location information corresponding to the vehicle-related image P10 and the map image P20. The location (latitude and longitude) displayed in location information area R12 corresponds to the location when the video data showing the vehicle-related image P10 was obtained, and also corresponds to the location of point Q1 shown in the map image P20. The location displayed in location information area R12 changes in accordance with the temporal changes of the vehicle-related image P10 and the map image P20 (especially the location of point Q1). The location displayed in location information area R12 is generated based on multiple location pieces of information corresponding to the video data (a series of video data) showing the vehicle-related image P10.

[0123] Furthermore, in step S37 described above, the control unit 32 of the second cloud server 30 (the control unit 3 of the video processing system 1) may generate data for displaying the integrated image P5 shown in Figure 10 based on the video data (series of video data) extracted in step S31, the map data obtained in step S33, the data generated in steps S34 to S36, and multiple time and location information corresponding to the video data (series of video data).

[0124] Furthermore, in step S39 described above, the control unit 45 of the information terminal 40 displays the integrated image on the display unit 42 of the information terminal 40 such that, in the integrated image P5, in synchronization with the temporal changes of the vehicle-related image P10, point Q indicating the position of the vehicle 10 in the map image P20 moves along line L1 indicating the driving trajectory of the vehicle 10, the time displayed in the time information area R11 changes, and the time displayed in the position information area R12 changes.

[0125] As described above, by providing a time information area R11 and a location information area R12 in the integrated image P5, the user can easily recognize the time and location at which the "vehicle-related image P10" included in the integrated image P5 was obtained. This makes it easier for the user to recall the scenery when the vehicle 10 was driving based on the integrated image P5. For example, the user can easily recognize what time the sunset on the highway shown in the exterior image P11, which is the vehicle-related image P10, took place, making the recollection of memories even more enjoyable.

[0126] [Details of the second cloud server] Next, the configuration of the second cloud server 30 will be described in detail with reference to Figure 11. As shown in Figure 11, the second cloud server 30 comprises a device gateway 51, a Contents Manager 52, a Backend for frontend (BFF) 53, a messaging basis 11, and a set of functions 64. The second cloud server 30 is a cloud-type server (content management server) that receives and manages content data Dc. The messaging basis 61 includes an event hub 62 and a message router 63.

[0127] <Device Gateway> The device gateway 51 is connected to a network such as the Internet and has a communication interface for communicating with other devices (other computers, servers, information terminals, etc.). In this example, the device gateway 51 communicates with the first cloud server 20.

[0128] When content data Dc is transmitted from vehicle 10 to the first cloud server 20, the first cloud server 20 transmits the content data Dc of vehicle 10 to the second cloud server 30. The content data Dc includes positioning data Dp and first video data Dv1 obtained by the drive recorder 15 of vehicle 10. The second cloud server 30 receives the content data Dc via the device gateway 51. The device gateway 51 receives the positioning data Dp and the first video data Dv1 separately.

[0129] Positioning data Dp is positioning information of vehicle 10 at a predetermined time. For example, positioning data Dp includes some or all of the following: the date the positioning information was acquired (data acquisition date), the time the positioning information was acquired (data acquisition time), the location information of vehicle 10 (e.g., latitude and longitude), the direction of travel of vehicle 10, the bearing and orientation information of vehicle 10, the speed of vehicle 10, and the identification ID of vehicle 10. The file format of positioning data Dp is not particularly limited, but for example, it is a text file conforming to the NMEA format.

[0130] For example, the first video data Dv1 may include video data obtained by the camera 151 while the vehicle is in motion, video data obtained by the camera 151 in response to direct or indirect instructions from the driver or passenger, or video data automatically obtained by the camera 151 when a specific location or condition is observed (detected) by a sensor or the like. Examples of the above "specific condition" include a state where the atmosphere inside the car is lively, or a state where scenery or landmark buildings in a tourist area are included within the imaging range of the camera 151.

[0131] The positioning data Dp and the first video data Dv1 may be linked to each other. For example, the time at which the positioning data Dp was obtained may be the same as the time at which the first video data Dv1 was obtained, and common time information may be associated with the positioning data Dp and the first video data Dv1.

[0132] Furthermore, the device gateway 51 transmits topic information Dm to the event hub 62 of the messaging infrastructure 61. Topic information Dm is used by the message router 63 of the messaging infrastructure 61 to determine which application module 65 (which application module 65 corresponds to which function / service) to which message should be sent. For example, the device gateway 51 detects (determines) an event occurring in the vehicle 10 based on content data Dc and generates topic information Dm based on that event. Topic information Dm includes positioning data Dp, content ID, event information, etc.

[0133] <Content Management Department> The content management unit 52 integrates the first video data Dv1 transmitted from the first cloud server 20 and the second video data Dv2 transmitted from the information terminal 40, and stores them in storage (storage unit 31) for centralized management. For example, the second video data Dv2 is video data obtained by a camera (not shown) also received by the information terminal 40.

[0134] The content management unit 52 manages "content files" and "content information" separately in the centralized management described above. Management here refers to managing so that the application module 65 of the function group 64 and the information terminal 40 can search, refer to, and retrieve "content files" and "content information" based on predetermined conditions. For example, "content files" and "content information" that have a predetermined relationship are assigned a common content ID and linked together. This configuration allows for centralized management of the first video data Dv1 and the second video data Dv2. For example, when the information terminal 40 accesses the content management unit 52 to view images of the vehicle 10 in motion, image searching can be made easier, thereby improving search speed and / or search accuracy.

[0135] Furthermore, the content information may include the "determination information indicating whether the video data is external video data or internal video data" as described above.

[0136] <BFF> The BFF53 functions as a data exchange interface with the information terminal 40 (for example, a mobile application implemented on the information terminal 40). Specifically, the BFF53 has a communication interface that receives and processes commands (instructions) from the information terminal 40.

[0137] The BFF53 executes processing in response to commands from the information terminal 40. For example, the BFF53 receives various commands from the information terminal 40, such as "request to view vehicle-related images," and executes processing in response to these commands. Specifically, when the BFF53 receives a "request to view vehicle-related images" from the information terminal 40, it notifies the application module 65, which has a "request to view vehicle-related images function," receives the data created by the application module 65 (data for displaying vehicle-related images), and sends that data to the information terminal 40 (mobile app).

[0138] <Messaging infrastructure> The messaging infrastructure 61 is a messaging system used for exchanging messages between application modules 65 included in the function group 64. The messaging infrastructure 61 is provided in accordance with the functions available in the function group 64 and is used by the application modules 65 that realize each of those functions.

[0139] Specifically, the topic information Dm obtained by the event hub 62 is passed to the message router 63. The message router 63 determines which application module 65 to send the topic information Dm to and passes that topic information Dm to the event hub 62. The event hub 62 sends the topic information Dm to the application module 65 according to the result of the decision made by the message router 63.

[0140] <Functional Groups> Function group 64 includes an application module 65 for realizing various functions and services (hereinafter collectively referred to as "functions"). The application module 65 includes memory for storing application programs for realizing various functions, a processor for executing those application programs, and so on, and the execution of the application programs realizes the various functions. Examples of functions realized by function group 64 include functions for managing various types of information such as user information, functions for managing various types of data such as video data, and functions for executing various processes such as list display processing and integrated display processing.

[0141] As described above, each application module 65 obtains topic information Dm necessary for realizing its respective function from the messaging infrastructure 61. The messaging infrastructure 61 may also send requests for various processes based on messages (such as list display processing and integrated display processing) to the target application module 65.

[0142] Furthermore, the second cloud server 30 may be configured to automatically link predetermined data using technologies such as API integration or SFTP integration. Specifically, the second cloud server 30 may consist of hardware such as a processor, memory, and storage, and software such as control programs and data implemented in memory. Various functions of the second cloud server 30 may be realized by a processor such as a CPU or GPU executing programs stored in memory. Note that the second cloud server 30 is not limited to a cloud server, and an on-premises server may also be used.

[0143] (Other embodiments) In the above explanation, the following configuration or processing may be used.

[0144] Instead of the first cloud server 20 and the second cloud server 30, a single cloud server capable of communicating with both the vehicle 10 and the information terminal 40 may be provided.

[0145] Part or all of the video processing system 1 (specifically, the memory unit 2 and control unit 3 of the video processing system 1) may be included in other information processing devices other than the second cloud server 30. For example, part or all of the video processing system 1 may be included in the vehicle 10, or in the one cloud server provided in place of the first cloud server 20 and the second cloud server 30.

[0146] The camera installed in the vehicle 10 is not limited to the camera 151 included in the drive recorder 15. For example, the camera installed to obtain video data showing the exterior image P11 may be the exterior camera 156 included in the drive recorder 15, a camera included in equipment for ADAS (Advanced Driver-Assistance Systems) (not shown), or a camera included in a mobile information terminal on which a drive recorder function application is installed (not shown). Similarly, the camera installed to obtain video data showing the interior image P12 may be the interior camera 157 included in the drive recorder 15, or a camera installed in the vehicle 10 to photograph the interior (not shown).

[0147] The positioning unit provided in the vehicle 10 is not limited to the positioning unit 152 included in the drive recorder 15. For example, the positioning unit provided to obtain location information of the vehicle 10 may be the positioning unit of a car navigation system (not shown) provided in the vehicle 10.

[0148] Vehicle 10 is not limited to four-wheeled vehicles, but may also be, for example, a bicycle, a motorbike, a railway vehicle, an electric scooter, a rickshaw, etc.

[0149] If the vehicle 10 is one of the following: "bicycle," "motorcycle," "railway vehicle," or "electric kick scooter," the positioning unit that acquires the location information of the vehicle 10 may be the positioning unit 152 included in the drive recorder 15. In this case, the association between the information terminal 40 and the drive recorder 15 may be the same as those associations in the above-described embodiment (when the vehicle 10 is a four-wheeled vehicle). If the vehicle 10 is a "railway vehicle," the approval of user registration in the association between the information terminal 40 and the drive recorder 15 may be performed by the railway company.

[0150] If vehicle 10 is a "rickshaw," the camera 151 placed on vehicle 10 may be attached to the rickshaw. Also, if vehicle 10 is a "rickshaw," the positioning unit that acquires the location information of vehicle 10 may be a positioning unit included in a device attached to the rickshaw (such as a drive recorder 15), or a positioning unit included in an information terminal held by the rickshaw driver. In the latter case, the positioning data (location information) acquired by the positioning unit of the information terminal held by the rickshaw driver may be used as the "positioning data (location information) of vehicle 10." Furthermore, the positioning unit included in an information terminal held by a passenger riding in the rickshaw may also be used as the "positioning unit that acquires the location information of vehicle 10."

[0151] Furthermore, as a reference example, the technology disclosed herein can also be applied to aircraft. When applied to aircraft, the positioning unit that acquires location information may be a positioning unit included in the flight recorder. In linking the information terminal 40 with the flight recorder, user registration approval may be performed by the airline.

[0152] Furthermore, as another possible application, the technology disclosed herein can also be applied to privately operated vessels such as pleasure boats. In this case, the system of the aforementioned embodiment (the system when vehicle 10 is a four-wheeled vehicle) can be applied to a vessel by replacing "map" with "nautical chart" and "driving trajectory" with "navigation trajectory".

[0153] Furthermore, the above embodiments may be combined as appropriate. The above embodiments are essentially preferred examples and are not intended to limit the scope of the technologies, applications, or uses disclosed herein. [Industrial applicability]

[0154] As explained above, the technology disclosed herein is useful as an image processing technology. [Explanation of Symbols]

[0155] 1. Video Processing System 2 Storage section 3. Control Unit 10 vehicles 15. Dashcam 151 Camera 152 Positioning Unit 153 Communications Department 154 Storage section 155 Control Unit 156 Exterior car camera 157 In-car camera 20. First Cloud Server 21 Memory section 22 Control Unit 30 Second Cloud Server 31 Storage section 32 Control Unit 40 Information terminals 41 Operation section 42 Display section 43 Communications Department 44 Storage section 45 Control Unit P1 Exterior View Images P2 Interior view images P5 Integrated Image P10 Vehicle-related images P11 Exterior view P12 Interior image P20 Map Image L1 Line indicating the travel trajectory Q1 point R1 Shape B1 Exterior Button B2 In-car button

Claims

1. A video processing system that processes video data obtained by a camera while a vehicle is in motion, the vehicle having a camera and a positioning unit that acquires location information, Memory unit and, It includes a control unit, The image shown in the video data obtained by the camera is an image related to the vehicle and is an image that changes over time. The position information obtained by the positioning unit indicates the position of the vehicle. The storage unit stores the video data obtained by the camera while the vehicle is in motion, a plurality of location information obtained in chronological order by the positioning unit while the vehicle is in motion, and map data showing a map including the vehicle's driving trajectory. The control unit generates data for displaying the integrated image, which includes an image related to the vehicle shown in the video data and an image of the map shown in the map data, such that a line indicating the vehicle's travel trajectory, generated based on the plurality of location information, is placed on the map image, and a point indicating the vehicle's position in the map image moves along the line in synchronization with the temporal changes in the image related to the vehicle. Video processing system.

2. In the video processing system of claim 1, The control unit generates data for displaying the integrated image such that a figure surrounding the point is placed on the map image included in the integrated image, and the figure moves in accordance with the movement of the point. Video processing system.

3. In the video processing system according to claim 1 or 2, The aforementioned camera includes an external camera that photographs the area outside the vehicle. The images related to the vehicle include images of the exterior of the vehicle. Video processing system.

4. A video processing method that uses a computer to process video data obtained by a camera while a vehicle is in motion, the vehicle having a camera and a positioning unit that acquires location information, Input step, The process includes a generation step, The image shown in the video data obtained by the camera is an image related to the vehicle and is an image that changes over time. The position information obtained by the positioning unit indicates the position of the vehicle. In the input step, the video data obtained by the camera while the vehicle is in motion, a plurality of location information obtained in chronological order by the positioning unit while the vehicle is in motion, and map data showing a map including the vehicle's driving trajectory are input. In the generation step, data for displaying the integrated image is generated such that, in an integrated image including an image related to the vehicle shown in the video data and an image of the map shown in the map data, a line indicating the vehicle's driving trajectory, generated based on the plurality of location information, is placed on the map image, and a point indicating the vehicle's position in the map image moves along the line in synchronization with the temporal changes in the image related to the vehicle. Image processing methods.

5. A display method for displaying an integrated image on a display device, which includes an image shown in video data obtained by a camera while a vehicle having a camera and a positioning unit that acquires location information is in motion, and an image of a map including the vehicle's driving trajectory, using a computer, A receiving step of receiving data for displaying the integrated image, The system includes a display step of displaying the integrated image on the display device based on data for displaying the integrated image, The image shown in the video data obtained by the camera is an image related to the vehicle and is an image that changes over time. The position information obtained by the positioning unit indicates the position of the vehicle. The data for displaying the integrated image includes the video data obtained by the camera while the vehicle is in motion, and map data showing a map that includes the vehicle's driving trajectory, generated based on a plurality of location information obtained in chronological order by the positioning unit while the vehicle is in motion. In the display step, the integrated image, which includes an image related to the vehicle shown in the video data and an image of the map shown in the map data, is displayed on the display device such that a line indicating the vehicle's travel trajectory, generated based on the plurality of location information, is placed on the map image, and a point indicating the vehicle's position in the map image moves along the line in synchronization with the temporal changes in the image related to the vehicle. Display method.

Images