Information processing device, information processing method, and information processing program

Abstract

To provide a user with a video of a traveling vehicle as intended.SOLUTION: An information processing apparatus 100 includes: an acquisition unit 135 which acquires, on the basis of information on a target vehicle, which is a vehicle for which a video is generated, a captured image including the target vehicle out of captured images captured by imaging means mounted on another vehicle which is different from the target vehicle; and a generation unit 136 which generates a video of the traveling target vehicle on the basis of the captured image.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] Conventionally, a system has been proposed for recording the conditions of a vehicle before and after a vehicle accident occurs. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-157554 Summary of the Invention [Problem to be solved by the invention]

[0004] For example, there are needs such as wanting to see footage of the vehicle you are driving from a third-person perspective, or wanting to objectively evaluate your own driving skills from external footage of the vehicle.

[0005] However, the above-mentioned conventional technologies cannot necessarily meet these needs. Specifically, the above-mentioned conventional technologies cannot necessarily provide users with the desired video of a vehicle traveling.

[0006] For example, the above-mentioned prior art discloses a camera that records video around the time of detection when a vehicle accident has occurred or when a risk of a vehicle accident has been detected. With such a camera, for example, even if it is possible to capture the situation outside the vehicle when a vehicle accident has occurred or when a risk of a vehicle accident has been detected, it is not possible to capture the situation of the vehicle in which the camera is installed traveling.

[0007] Therefore, the above-mentioned conventional technology does not necessarily provide the user with the desired video of the vehicle in motion.

[0008] The present invention has been made in view of the above, and has an object to realize an information processing device, an information processing method, and an information processing program that can provide a user with a desired video of a vehicle traveling. [Means for solving the problem]

[0009] The information processing device described in claim 1 is characterized by having an acquisition unit that acquires, based on information about a target vehicle, which is a vehicle for which an image is to be generated, an image of the target vehicle from among images captured by an imaging means mounted on a vehicle other than the target vehicle, and a generation unit that generates a driving image of the target vehicle driving based on the captured image.

[0010] The information processing method described in claim 18 is an information processing method executed by an information processing device, and is characterized by including an acquisition step of acquiring, based on information about a target vehicle, which is a vehicle for which an image is to be generated, an image of the target vehicle from among images captured by an imaging means mounted on a vehicle other than the target vehicle, and a generation step of generating a driving image of the target vehicle driving based on the captured image.

[0011] The information processing program described in claim 19 is an information processing program for causing an information processing device to execute an acquisition procedure for acquiring an image of the target vehicle from among images captured by an imaging means mounted on a vehicle other than the target vehicle, based on information about the target vehicle for which an image is to be generated, and a generation procedure for generating a driving image of the target vehicle driving based on the captured image. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram illustrating an example of an information processing system according to an embodiment. [Figure 2] FIG. 2 is an explanatory diagram illustrating an overall image of information processing according to the embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of a process for generating a driving video. [Figure 4] FIG. 4 is a diagram illustrating an example of the configuration of the information processing device according to the embodiment. [Figure 5] FIG. 5 is a diagram illustrating an example of a captured image database according to the embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of a sensor information database according to the embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of a vehicle information database according to the embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of a video database according to the embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of a registration form according to the embodiment. [Figure 10] FIG. 10 is a flowchart showing the procedure of the detection process according to the embodiment. [Figure 11] FIG. 11 is a flowchart showing the procedure of the generation process according to the embodiment. [Figure 12] FIG. 12 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0013] An example of a form for implementing an information processing device, an information processing method, and an information processing program (hereinafter referred to as an "embodiment") will be described in detail below with reference to the drawings. Note that the information processing device, the information processing method, and the information processing program are not limited to this embodiment. Furthermore, the same components in the following embodiments will be assigned the same reference numerals, and duplicated descriptions will be omitted.

[0014] [1. System Configuration] First, the configuration of an information processing system according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of an information processing system according to an embodiment. Fig. 1 shows an information processing system 1 as an example of an information processing system according to an embodiment.

[0015] 1, the information processing system 1 may include an in-vehicle device 10, a user device 60, and an information processing device 100. The in-vehicle device 10, the user device 60, and the information processing device 100 are communicably connected via a network N, either wired or wirelessly. The information processing system 1 shown in FIG. 1 may include any number of in-vehicle devices 10, any number of user devices 60, and any number of information processing devices 100.

[0016] The in-vehicle device 10 may be a dedicated navigation device built into or externally attached to the vehicle VEx, or may be a recording device (drive recorder) installed in the vehicle VEx for crime prevention or to prevent aggressive driving.

[0017] The in-vehicle device 10 may also be configured with a navigation device and a recording device. As an example, the in-vehicle device 10 may be a composite device in which a navigation device and a recording device that are independent from each other are connected to each other so that they can communicate with each other. As another example, the in-vehicle device 10 may be a single device that has both a navigation function and a recording function.

[0018] Furthermore, a user can install a predetermined application into a portable terminal device (for example, a smartphone, tablet terminal, notebook PC, desktop PC, PDA, etc.) that they use on a daily basis, and use the device as the in-vehicle device 10. For example, a portable terminal device on which a predetermined navigation application or a predetermined recording application is installed can be understood as the in-vehicle device 10 referred to here. When the portable terminal device is used as the in-vehicle device 10, it is installed, for example, on the dashboard of the vehicle VEx while driving.

[0019] The in-vehicle device 10 may also include various sensors. For example, the in-vehicle device 10 may include various sensors such as a camera, an acceleration sensor, a gyro sensor, a GPS sensor, and an air pressure sensor. The information processing device 100, which will be described later, may perform various information processing based on sensor information detected by these sensors. The information processing device 100 may also use sensor information detected by sensors provided in the vehicle VEx itself as a safe driving system, in addition to the sensors provided in the in-vehicle device 10.

[0020] In the following embodiment, the in-vehicle device 10 will be described as being a drive recorder in particular.

[0021] The user device 60 is a portable terminal device owned by a user. As described above, the portable terminal device may be a smartphone, a tablet terminal, a notebook PC, a desktop PC, a PDA, or the like. In this embodiment, an application AP (hereinafter abbreviated as "application AP") that enables transmission and reception of information between the user device 60 and the information processing device 100 is installed in the user device 60. For example, the user can register information to the information processing device 100 via the application AP.

[0022] The information processing device 100 is a device that performs information processing according to an embodiment. For example, the information processing device 100 acquires captured images of the target vehicle, which is a vehicle for which a video is to be generated, from captured images captured by an imaging unit mounted on a vehicle other than the target vehicle, based on information about the target vehicle. Then, the information processing device 100 generates a driving video of the target vehicle traveling based on the acquired captured images, and provides the generated driving video to, for example, the owner of the target vehicle.

[0023] Such information processing realizes a vehicle video providing service (hereinafter, sometimes referred to as "Service SA") that provides users with driving video captured from a third-person perspective by a vehicle designated by the user. Service SA allows users to obtain driving video captured from a third-person perspective of the vehicle they are driving, so that by viewing this driving video, users can, for example, objectively evaluate their own driving skills or recall what it was like when they were driving.

[0024] Furthermore, to enable the user to use the service SA, an application corresponding to the service SA (hereinafter, sometimes referred to as an "application AP") may be installed in the user device 60. For example, the application AP may be a general-purpose application such as a browser, or may be implemented as an application dedicated to the service SA.

[0025] Here, if the in-vehicle device 10 is an edge computer that performs edge processing near the user, the information processing device 100 may be, for example, a cloud computer that performs processing on the cloud side. In other words, the information processing device 100 may be a server device.

[0026] Furthermore, in the following embodiment, an example is shown in which information processing according to the embodiment is realized in the information processing system 1 by transmitting and receiving information between the in-vehicle device 10 and the information processing device 100. However, the information processing according to the embodiment may be realized only on the edge side, i.e., by the in-vehicle device 10. In this case, the in-vehicle device 10 may be configured to behave like the information processing device 100 by, for example, an information processing program according to the embodiment.

[0027] [2. Overview of information processing] From here, an overview of information processing according to the embodiment will be described with reference to Fig. 2. Fig. 2 is an explanatory diagram illustrating an overview of information processing according to the embodiment.

[0028] Figure 2 shows a scene in which, in response to a request from user U1 using service SA, a moving video of a target vehicle specified by user U1 (specifically, a vehicle owned by user U1) is generated using images captured by multiple other vehicles that were capturing images of the target vehicle.

[0029] Here, Figure 2(a) shows the state of vehicle VEx traveling through the intersection area in the morning of October 10, 2021. According to this example, at a specific time in the morning of October 10, 2021, a total of eight vehicles, vehicle VE1, vehicle VE2, vehicle VE3, vehicle VE4, vehicle VE5, vehicle VE6, vehicle VE7, and vehicle VE8, were traveling in the intersection area in the positional relationship shown in Figure 2(a).

[0030] Each of vehicle VE1, vehicle VE2, vehicle VE3, vehicle VE4, vehicle VE5, vehicle VE6, vehicle VE7, and vehicle VE8 is an example of vehicle VEx, and when there is no need to distinguish between them, they will be collectively referred to as vehicle VEx.

[0031] Moreover, the driving behavior of each vehicle VEx, such as the driving position, traveling direction, and driving speed, as well as the driving route, change as time passes. However, the information processing device 100 can track information related to the driving behavior and driving route of the vehicle VEx, for example, based on sensor information transmitted in real time from the in-vehicle device 10 of each vehicle VEx.

[0032] In the example of Figure 2(a), each time the information processing device 100 receives sensor information transmitted in real time from the on-board device 10 of each vehicle VEx, it detects the driving behavior of each vehicle VEx and the status of the driving route corresponding to the driving behavior based on the received sensor information, and acquires the detection results as vehicle information (step S1).

[0033] The sensor information may also include captured images captured by the in-vehicle device 10, and the information processing device 100 may store the captured images included in the sensor information in a captured image database 121 (FIG. 5). The information processing device 100 may store sensor information other than the captured images in a sensor information database 122 (FIG. 6). The information processing device 100 may record vehicle information acquired based on the sensor information in a vehicle information database 123 (FIG. 7).

[0034] Here, among the above eight vehicles VEx, the vehicle VEx owned by user U1 is assumed to be vehicle VE1. Furthermore, after finishing driving, user U1 launches application AP from his / her user device 60 to use service SA, and performs an operation to request generation of a driving video of vehicle VE1. In this case, user device 60 transmits a generation request to information processing device 100 in response to user U1's operation (step S2). In the example of FIG. 2, user device 60 transmits a generation request with content instructing generation of a driving video of vehicle VE1 when it was driving (turning left) through intersection area AR. In this example, vehicle VEx, the target vehicle for which video is to be generated, is vehicle VE1. Therefore, hereinafter, vehicle VE1 may be referred to as target vehicle VE1.

[0035] When the information processing device 100 receives the generation request, it extracts vehicle information corresponding to this generation request from the vehicle information that has been detected so far for the target vehicle VE1 (step S3). For example, the vehicle information may include information indicating the travel date and time when the target vehicle VE1 was traveling, the position of the target vehicle at the travel date and time, the traveling direction of the target vehicle VE1 at this position, and the condition of the route (road) on which the target vehicle VE1 was traveling at the travel date and time, and the information processing device 100 may extract vehicle information that was detected when the target vehicle VE1 was located in the intersection area AR based on this information.

[0036] Next, based on the vehicle information (information about the target vehicle) extracted in step S3, the information processing device 100 estimates the vehicle VEx that captured the image of the target vehicle VE1 via the in-vehicle device 10 among the vehicles VEx other than the target vehicle VE1, and identifies the estimated vehicle VEx as the image source vehicle (step S4).

[0037] For example, the information processing device 100 may extract vehicle information detected when the other vehicle VEx was located in the intersection area AR from among the vehicle information detected so far of the other vehicle VEx, and may estimate the vehicle VEx that captured the image of the target vehicle VE1 by comparing the vehicle information corresponding to the target vehicle VE1 with the vehicle information corresponding to the other vehicle VEx. More specifically, the information processing device 100 may estimate the vehicle VEx that captured the image of the target vehicle VE1 based on the positional relationship between the position of the target vehicle VE1 and the position of the other vehicle VEx, the directional relationship between the traveling direction of the target vehicle VE1 and the traveling direction of the other vehicle VEx, or the orientation of the other vehicle VEx relative to the target vehicle VE1.

[0038] Here, the information processing device 100 estimates that, of the other vehicles VEx, vehicles VE2, VE3, VE4, VE5, VE6, VE7, and VE8, vehicles VE2, VE3, and VE4 captured the target vehicle VE1. In this case, the information processing device 100 identifies the three other vehicles VEx, vehicles VE2, VE3, and VE4, as the image-capturing vehicles. Figure 2(b) shows a scene in which vehicles VE2, VE3, and VE4 captured the target vehicle VE1 when the target vehicle VE1 was traveling through the intersection area AR (the morning of October 10, 2021).

[0039] 2(b) shows an example in which the information processing device 100 has identified multiple image capture source vehicles, but there are also cases in which only one image capture source vehicle can be identified, or in which no image capture source vehicles can be identified. If the information processing device 100 cannot identify any image capture source vehicles, it may output to the user device 60 a message that it is impossible to generate a driving video. Furthermore, if it is impossible to generate a driving video, the information processing device 100 may search for an easy route for the vehicle VEx equipped with the in-vehicle device 10 from the driving history of the vehicle VEx, and recommend to the user U1 that the vehicle VEx travel along the searched route.

[0040] Next, the information processing device 100 acquires the captured image in which the target vehicle VE1 is shown from among the captured images captured by the image capturing source vehicles VE2, VE3, and VE4 (image capturing source vehicles VE2 to VE4) (step S5). For example, the information processing device 100 may acquire the captured image in which the target vehicle VE1 is shown by comparing the captured images captured by the image capturing source vehicles VE2 to VE4 with the appearance, model, vehicle number, color, optional equipment status, etc. of the target vehicle VE1.

[0041] Then, the information processing device 100 generates a traveling video of the target vehicle VE1 traveling based on the acquired captured images (step S6). For example, the information processing device 100 may generate a traveling video of the target vehicle VE1 traveling by performing a synthesis process in which the captured images are joined together in chronological order according to the dates and times at which the captured images were captured.

[0042] Furthermore, by identifying a plurality of original image vehicles that are in the positional relationship shown in FIG. 2(b), the information processing device 100 can acquire captured images of the target vehicle VE1 captured from different directions. In such a case, the information processing device 100 may generate a driving video in which the scene of the target vehicle VE1 traveling changes depending on the direction. More specifically, the information processing device 100 may generate a driving video in which the scene of the target vehicle VE1 turning left at the intersection area AR changes depending on the imaging direction.

[0043] A specific example of steps S5 and S6 will now be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of a process for generating a running video.

[0044] In the example of Fig. 3, the information processing device 100 acquires image IMG21 as an image of the target vehicle VE1 among the images captured by the image capture source vehicle VE2. Image IMG21 is an image captured from behind of the target vehicle VE1 attempting to enter the intersection area AR. According to the example of Fig. 3, the image capture date and time when image IMG21 was captured is date and time TM1.

[0045] 3, the information processing device 100 acquires image IMG22 as an image of the target vehicle VE1 from among the images captured by the image capture source vehicle VE2. Image IMG22 is an image captured from behind the target vehicle VE1 as it begins to turn left at the intersection area AR. According to the example of FIG. 3, the image capture date and time when image IMG22 was captured is date and time TM2.

[0046] 3, the information processing device 100 acquires image IMG31 as an image of the target vehicle VE1 among the images captured by the image source vehicle VE3. Image IMG31 is an image captured from the right front of the target vehicle VE1 as it begins to turn left at the intersection area AR. According to the example of FIG. 3, the image IMG31 was captured on date and time TM3.

[0047] 3, the information processing device 100 acquires image IMG32 as an image of the target vehicle VE1 from among the images captured by the image source vehicle VE3. Image IMG32 is an image captured from the right front of the target vehicle VE1 while it is turning left at the intersection area AR. According to the example of FIG. 3, the image IMG32 was captured on date and time TM4.

[0048] 3, the information processing device 100 acquires image IMG41 as a captured image of the target vehicle VE1 from among the captured images captured by the image capture source vehicle VE4. Image IMG41 is a captured image captured from the left diagonally forward of the target vehicle VE1 while it is turning left at the intersection area AR. According to the example of FIG. 3, the image IMG41 was captured on date and time TM5.

[0049] 3, the information processing device 100 acquires image IMG42 as an image of the target vehicle VE1 from among the images captured by the image capture source vehicle VE4. Image IMG42 is an image captured from the left diagonally forward direction of the target vehicle VE1 as it finishes turning left at the intersection area AR. According to the example of FIG. 3, the image capture date and time when image IMG42 was captured is date and time TM6.

[0050] In this state, the information processing device 100 splices together images IMG21, IMG22, IMG31, IMG32, IMG41, and IMG42 in the following chronological order: date and time TM1 > date and time TM2 > date and time TM3 > date and time TM4 > date and time TM5 > date and time TM6. As a result, the information processing device 100 generates a driving video PC24 in which the scene of the target vehicle VE1 turning left at the intersection area AR changes depending on the imaging direction. In this way, the driving video PC24 is a video of the target vehicle VE1 as seen from a third party.

[0051] 2, the information processing device 100 provides the user U1 with the driving video PC24 generated in steps S5 and S6 (step S7). For example, the information processing device 100 controls the output of the user device 60 via the application AP so that the driving video PC24 is displayed.

[0052] In this way, by using service SA, user U1 can receive driving footage PC24 of his or her own vehicle VE1 captured from a third-person perspective, and by viewing the driving footage PC24, he or she can objectively evaluate his or her own driving skills and recall what the situation was like when he or she was driving.

[0053] Note that FIG. 2 shows an example in which the information processing device 100 generates a driving video of a scene in response to a generation request received from the user U1.

[0054] However, the information processing device 100 may generate in advance a driving video of a scene in which the vehicle VE1 is driving through each characteristic area (for example, an intersection area, a curve area, or a straight area) included in the driving route of the vehicle VE1. In such a case, the information processing device 100 may inquire of the user U1 as to which characteristic area the user U1 wishes to be provided with a driving video of.

[0055] As another example, the information processing device 100 may determine, based on a captured image showing the vehicle VE1, what kind of scene (for example, a scene passing through an intersection area, or a curve scene, etc.) it is possible to generate a driving video of, and may accept a scene specification by presenting the determination result to the user U1, and may generate a driving video of the accepted scene.

[0056] 3. Configuration of Information Processing Device From here, the information processing device 100 according to the embodiment will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of the configuration of the information processing device 100 according to the embodiment. As shown in Fig. 4, the information processing device 100 has a communication unit 110, a storage unit 120, and a control unit 130.

[0057] (Regarding the communication unit 110) The communication unit 110 is realized by, for example, a network interface card (NIC), etc. The communication unit 110 is connected to the network N by wire or wirelessly, and transmits and receives information to and from the in-vehicle device 10, for example.

[0058] (Regarding the storage unit 120) The storage unit 120 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 120 has a captured image database 121, a sensor information database 122, a vehicle information database 123, and a video database 124.

[0059] (Regarding the captured image database 121) The captured image database 121 stores information related to captured images captured by the in-vehicle device 10 (imaging means) among the sensor information. An example of the captured image database 121 according to the embodiment is shown in Fig. 5. In the example of Fig. 5, the captured image database 121 has items such as "vehicle ID," "capture date and time," "capture location," and "captured image."

[0060] "Vehicle ID" indicates identification information that identifies the vehicle VEx (or the in-vehicle device 10) that transmitted the "captured image." "Captured date and time" is information regarding the date and time when the "captured image" was captured. "Captured location" is information regarding the location of the vehicle VEx when the "captured image" was captured. The "captured image" is captured image data captured by the in-vehicle device 10 possessed by the vehicle VEx indicated by the "vehicle ID."

[0061] 5 shows an example in which a vehicle ID "VE1", an image capture date and time "TM11", an image capture position "PT11", and a captured image "IMG11" are associated with each other. This example shows an example in which the in-vehicle device 10 of the vehicle VE1 acquires the captured image IMG11 by capturing an image at the date and time TM11. This example also shows an example in which the vehicle VE1 was traveling at the position PT11 when the captured image IMG11 was acquired.

[0062] (About the sensor information database 122) The sensor information database 122 stores sensor information detected by a sensor included in the in-vehicle device 10 or a sensor included in the vehicle VEx itself. An example of the sensor information database 122 according to the embodiment is shown in Fig. 6. In the example of Fig. 6, the sensor information database 122 has items such as "vehicle ID," "detection date and time," and "sensor information."

[0063] "Vehicle ID" indicates identification information that identifies the source vehicle VEx (or the source vehicle-mounted device 10) that transmitted the "sensor information." "Detection date and time" indicates information regarding the date and time when the "sensor information" was detected. "Sensor information" indicates information detected by a sensor possessed by the vehicle VEx indicated by the "vehicle ID."

[0064] 6 shows an example in which a vehicle ID "VE1", a detection date and time "TM11", and sensor information "SC11" are associated with each other. This example shows an example in which a sensor possessed by the vehicle VE1 (for example, a sensor possessed by the in-vehicle device 10 provided in the vehicle VE1, or a sensor possessed by the vehicle VE1 itself) detected the sensor information SC11 at the date and time TM11.

[0065] (About Vehicle Information Database 123) The vehicle information database 123 stores information about the vehicle VEx detected from the sensor information. Here, FIG. 7 shows an example of the vehicle information database 123 according to the embodiment. In the example of FIG. 7, the vehicle information database 123 has items such as "vehicle ID," "driving behavior," and "route status." As shown in FIG. 7, information indicating "driving behavior" and information indicating "route status" are "vehicle information" related to the vehicle VEx.

[0066] As shown in FIG. 7, examples of "driving behavior" include "driving position," "traveling direction," and "driving speed" at "driving date and time." As shown in FIG. 7, examples of "route status" include "route type" and "route characteristics." The "driving behavior" and "route status" may be detected based on the "sensor information" in FIG. 6.

[0067] "Vehicle ID" is identification information for identifying the vehicle VEx for which "vehicle information" was detected, and corresponds to "Vehicle ID" in FIG.

[0068] "Driving date and time" indicates information regarding the date and time when the vehicle VEx indicated by the "Vehicle ID" was traveling. "Driving date and time" also indicates information regarding the date and time when "Vehicle information" was detected based on the "Sensor information" (Figure 6) corresponding to the vehicle VEx indicated by the "Vehicle ID". "Driving position" is information indicating the position of the vehicle VEx at the "Driving date and time". "Traveling direction" is information indicating the traveling direction of the vehicle VEx at the "Driving date and time". "Traveling speed" is information indicating the speed of the vehicle VEx at the "Traveling date and time".

[0069] 7 shows an example in which a vehicle ID "VE1", a travel date and time "TM11", a travel position "PT11", a traveling direction "DR11", and a travel speed "SP11" are associated with each other. This example shows that at the date and time TM11, the vehicle VE1 is traveling at the position PT11, facing in the direction DR11, and traveling at the speed SP11.

[0070] "Route type" is information indicating the type of route (road) that the vehicle VEx was traveling on at the "travel date and time," and may be any information that can identify the route. For example, as shown in Figure 7, "route type" may be the name of a road included in the route, such as National Route XX.

[0071] "Route characteristics" is information that characterizes the state of the route (road) at the position where the vehicle VEx was traveling at the "travel date and time." For example, if the vehicle VEx was curved at the "travel date and time," the route characteristic "curve" may be associated, and if the vehicle VEx was passing through an intersection area, the route characteristic "intersection" may be associated.

[0072] (About Video Database 124) The video database 124 stores information related to the generation of driving footage. An example of the video database 124 according to the embodiment is shown in Fig. 8. In the example of Fig. 8, the video database 124 has items such as "user ID," "target vehicle ID," "generation conditions," and "driving footage."

[0073] "User ID" indicates identification information that identifies the user who has requested the generation of a driving video among users of the service SA.

[0074] The "target vehicle ID" indicates identification information that identifies the target vehicle VEx, which is the target vehicle for which driving footage is to be generated. The target vehicle is a vehicle owned by the user indicated by the "user ID," and may be controlled so that only this user (or someone related to this user) can specify it. The target vehicle indicated by the "target vehicle ID" can also be said to be a designated vehicle specified by the user indicated by the "user ID."

[0075] As will be described later, a user can use a registration form to request the generation of driving footage of a scene that meets certain conditions. Therefore, the "generation conditions" referred to here correspond to information that sets conditions for the type of driving footage to be generated.

[0076] The "generation condition" may be set using a geographical range to determine the section in which the target vehicle (designated vehicle) indicated by the "target vehicle ID" is traveling when generating a driving video. The geographical range here may be the type of characteristic area that a road has, such as an intersection area, a curve area, or an expressway, or it may be the name of the characteristic area (for example, intersection A) or specific coordinates.

[0077] The "generation condition" may be set using a time range for determining the period during which the target vehicle (designated vehicle) indicated by the "target vehicle ID" was traveling when generating the driving video. The time range may be, for example, a date and a time period.

[0078] The "generation conditions" may be set using the driving mode of the target vehicle (designated vehicle) indicated by the "target vehicle ID" to generate a driving video. The driving mode may be straight driving, curve driving, or speed.

[0079] The "driving footage" is a driving footage generated based on an image captured by the imaging means of a vehicle VEx other than the target vehicle indicated by the "target vehicle ID," which captures the target vehicle, and is a driving footage of a scene that satisfies the "generation conditions."

[0080] (Regarding the control unit 130) 4, the control unit 130 is realized by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like executing various programs (for example, the information processing program according to the embodiment) stored in a storage device inside the information processing device 100 using a RAM as a work area. The control unit 130 is also realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0081] As shown in Fig. 3, the control unit 130 has a receiving unit 131, a detecting unit 132, a request accepting unit 133, an identifying unit 134, an acquiring unit 135, a generating unit 136, a providing unit 137, and a route control unit 138, and realizes or executes the functions and actions of the information processing described below. Note that the internal configuration of the control unit 130 is not limited to the configuration shown in Fig. 4, and may have other configurations as long as they perform the information processing described below. Furthermore, the connection relationship between the processing units included in the control unit 130 is not limited to the connection relationship shown in Fig. 4, and may have other connection relationships.

[0082] (Regarding the receiving unit 131) The receiving unit 131 acquires sensor information detected by a sensor possessed by the vehicle VEx (for example, a sensor possessed by an in-vehicle device 10 provided in the vehicle VEx, or a sensor possessed by the vehicle VEx itself). Then, the receiving unit 131 stores the acquired sensor information in the sensor information database 122. Furthermore, if the acquired sensor information includes a captured image captured by the in-vehicle device 10, the receiving unit 131 may store the captured image in the captured image database 121.

[0083] (Regarding the detection unit 132) The detection unit 132 detects a situation related to the vehicle VEx. For example, the detection unit 132 may detect a driving behavior of the vehicle VEx and a situation of a travel route corresponding to the driving behavior, based on the sensor information acquired by the reception unit 131.

[0084] For example, the detection unit 132 may detect, as the driving behavior of the vehicle VEx, the position of the vehicle VEx at the date and time when the sensor information is detected, the direction of travel of the vehicle VEx at the date and time when the sensor information is detected, and the speed of the vehicle VEx at the date and time when the sensor information is detected.

[0085] Furthermore, the detection unit 132 may detect, as the status of the travel route, the type of route (road) on which the vehicle VEx was traveling at the date and time when the sensor information was detected. Furthermore, the detection unit 132 may also detect, as the status of the travel route, information that characterizes the state of the route at the position on which the vehicle VEx was traveling at the date and time when the sensor information was detected.

[0086] Furthermore, the detection unit 132 may acquire the detection result as vehicle information and store the acquired vehicle information in the vehicle information database 123. Note that the information that the detection unit 132 can detect is not limited to the above example.

[0087] (Regarding the request receiving unit 133) The request receiving unit 133 receives a generation request from a user requesting that a driving video be generated.

[0088] For example, the request receiving unit 133 may receive a generation request in which a geographical range is specified as a generation condition for generating driving footage of a designated vehicle VEx (an example of a target vehicle VEx for which footage is to be generated), which is a vehicle designated by a user, traveling through a certain section.

[0089] The request receiving unit 133 may also receive a generation request in which a time range for generating a driving video of the designated vehicle VEx during a certain period of time is specified as a generation condition.

[0090] The request receiving unit 133 may also receive a generation request in which a driving mode of the designated vehicle VEx is specified as a generation condition for generating a driving video image when the designated vehicle VEx is driving.

[0091] Furthermore, the request receiving unit 133 may store the generation information received as a generation request in the video database 124. As a result, the video database 124 shown in FIG.

[0092] The request receiving unit 133 may receive registration of various information such as generation conditions via a registration form C1 as shown in Fig. 9. Fig. 9 is a diagram showing an example of the registration form C1 according to the embodiment. For example, the request receiving unit 133 may provide the registration form C1 as shown in Fig. 9 within the application AP.

[0093] In the example of Figure 9, the registration form C1 has an input field IF1 for entering a last name, an input field IF2 for entering a first name, an input field IF3 for entering an email address, and an input field IF4 for specifying the target vehicle VEx for which an image will be generated.

[0094] Furthermore, in the example of FIG. 9, the registration form C1 has input fields IF5, IF6, and IF7 for specifying the generation conditions.

[0095] Input field IF5 is an input field for specifying a geographical range as a generation condition for generating driving footage of the specified vehicle VEx entered in input field IF4 when traveling through a certain section. The geographical range here may be the type of characteristic area of ​​a road, such as an intersection area, a curve area, or an expressway, or it may be the name of the characteristic area (for example, intersection A) or specific coordinates.

[0096] The input field IF6 is an input field for specifying a time range as a generation condition for generating a driving video of the specified vehicle VEx entered in the input field IF4 during what period of time the vehicle VEx was traveling. The time range here may be, for example, a date and a time period.

[0097] The input field IF7 is an input field for specifying the driving mode as a generation condition for generating a driving video of the designated vehicle VEx entered in the input field IF4. The driving mode may be various driving modes such as straight driving, curve driving, speed, etc.

[0098] The configuration of the registration form C1 is not limited to the example shown in Fig. 9. Also, although this embodiment shows an example in which a generation request is accepted using the registration form C1 shown in Fig. 9, the method for accepting a generation request from a user is not limited.

[0099] (Regarding the identification unit 134) Based on information about the target vehicle VEx, the identification unit 134 estimates the vehicle VEx among other vehicles VEx that captured the image of the target vehicle VEx via the in-vehicle device 10, and identifies the estimated vehicle VEx as the vehicle that captured the image.

[0100] For example, the identification unit 134 may identify a vehicle VEx that is estimated to have captured an image of the target vehicle VEx when the target vehicle VEx passes through a predetermined characteristic area on a road as the image-captured vehicle. Fig. 2(b) shows an example in which the identification unit 134 estimates vehicles VE2, VE3, and VE4 as other vehicles VEx that captured images of the target vehicle VE1 when the target vehicle VE1 passes through an intersection area (an example of a characteristic area), and identifies these other vehicles VEx as the image-captured vehicles.

[0101] Furthermore, the identification unit 134 can identify the image-captured vehicle based on any of the information about the target vehicle VEx, such as the travel date and time when the target vehicle VEx was traveling, the position of the target vehicle VEx at the travel date and time, the traveling direction of the target vehicle VEx at this position, or information about the traveling route of the target vehicle VEx. For example, the identification unit 134 can identify the image-captured vehicle based on the positional relationship between the position of the target vehicle VEx and the positions of other vehicles VEx, the relationship between the traveling direction of the target vehicle VEx and the traveling directions of the other vehicles VEx, or the orientation of the other vehicles VEx relative to the target vehicle VEx.

[0102] (Regarding the acquisition unit 135) The acquisition unit 135 acquires, based on information about the target vehicle VEx from which the video is generated, captured images in which the target vehicle VEx is shown from among captured images taken by the in-vehicle device 10 mounted on a vehicle VEx other than the target vehicle VEx. For example, the acquisition unit 135 acquires, from among captured images taken by the in-vehicle device 10 of the image-taking source vehicle identified by the identification unit 134, captured images in which the target vehicle VEx is shown.

[0103] For example, suppose that the identification unit 134 identifies a vehicle VEx that is estimated to have captured an image of the target vehicle VEx when the target vehicle VEx passes through a predetermined characteristic area on a road as the image-capturing vehicle. In this case, the acquisition unit 135 acquires captured images in which the target vehicle VEx is captured in the characteristic area from among the captured images captured by the in-vehicle device 10 of the image-capturing vehicle.

[0104] When multiple image capture source vehicles that are estimated to have captured images of the target vehicle VEx are identified, the acquisition unit 135 may acquire captured images in which the target vehicle is captured from different directions from among the captured images captured by the in-vehicle devices 10 of the multiple image capture source vehicles. FIG. 2(b) shows an example in which the acquisition unit 135 acquires captured images in which the target vehicle VE1 is captured from among the captured images captured by the in-vehicle devices 10 of the image capture source vehicles VE2, VE3, and VE4. In this case, the acquisition unit 135 may acquire captured images in which the target vehicle VE1 is captured by, for example, comparing the captured images captured by the image capture source vehicles VE2 to VE4 with the appearance, model, vehicle number, color, optional equipment status, etc. of the target vehicle VE1.

[0105] Furthermore, the acquisition unit 135 acquires, as information about the target vehicle VEx, captured images in which the designated vehicle VEx is captured by the in-vehicle device 10 of a vehicle VEx other than the designated vehicle VEx, based on information about the designated vehicle VEx specified in the generation request. For example, when a geographical range is specified as the generation condition, the acquisition unit 135 acquires captured images in which the designated vehicle VEx is captured based on information corresponding to this geographical range among the information about the designated vehicle VEx. Furthermore, when a temporal range is specified as the generation condition, the acquisition unit 135 acquires captured images in which the designated vehicle VEx is captured based on information corresponding to this temporal range among the information about the designated vehicle VEx. Furthermore, when a driving mode is specified as the generation condition, the acquisition unit 135 acquires captured images in which the designated vehicle VEx is captured based on information corresponding to this driving mode among the information about the designated vehicle VEx.

[0106] (Regarding the generation unit 136) The generation unit 136 generates a traveling video of the target vehicle VEx traveling based on the captured image acquired by the acquisition unit 135.

[0107] For example, the generation unit 136 generates a driving video of a scene in which the target vehicle VEx passes through a characteristic area based on the captured images. In this case, the generation unit 136 may generate a driving video of a scene from when the target vehicle VEx starts passing through the characteristic area to when it finishes passing through the characteristic area. Furthermore, the generation unit 136 may generate a driving video in which the scene in which the target vehicle VEx is traveling changes depending on the direction based on the captured images. Furthermore, the generation unit 136 may generate a driving video in which the target vehicle VEx is traveling by performing a synthesis process in which the captured images are joined together in chronological order depending on the date and time at which the captured images were captured.

[0108] In addition, when a geographical range is specified as a generation condition, the generation unit 136 generates a driving video of a scene in which the designated vehicle VEx is driving through the area indicated by this geographical range, based on an image of the designated vehicle VEx obtained based on information corresponding to this geographical range.

[0109] In addition, when a time range is specified as a generation condition, the generation unit 136 generates a driving video of a scene in which the designated vehicle VEx is driving during the time period indicated by this time range based on an image of the designated vehicle VEx obtained based on information corresponding to this time range.

[0110] In addition, when a driving mode is specified as a generation condition, the generation unit 136 generates a driving video of a scene in which the designated vehicle VEx is driving in this driving mode based on an image of the designated vehicle VEx obtained based on information corresponding to this driving mode.

[0111] 3, the generation unit 136 combines images IMG21, IMG22, IMG31, IMG32, IMG41, and IMG42 in the following chronological order: date and time TM1 > date and time TM2 > date and time TM3 > date and time TM4 > date and time TM5 > date and time TM6. As a result, the generation unit 136 generates a traveling video PC24 in which the scene of the target vehicle VE1 turning left at the intersection area AR changes depending on the imaging direction.

[0112] (About the provider 137) The providing unit 137 provides the user with the driving video generated by the generating unit 136. For example, when a generation request for generating driving video is accepted, the providing unit 137 provides the user who transmitted the generation request with the driving video generated based on the generation conditions specified by the user who transmitted the generation request.

[0113] (Regarding the route control unit 138) The route control unit 138 identifies a route along which the user's own vehicle VEx is likely to be photographed by the vehicle-mounted device 10 mounted on another vehicle VEx, based on the tendency of the route traveled by the vehicle VEx on which the vehicle-mounted device 10 is mounted, and recommends the identified route to the user.

[0114] For example, the route control unit 138 identifies a route among routes to the user's destination that allows the user's vehicle VEx to be more likely to be photographed by an in-vehicle device 10 mounted on another vehicle VEx, and recommends the identified route to the user as a candidate guide route. At this time, the route control unit 138 may display information indicating that the identified route is more likely to be photographed by another vehicle VEx, together with the identified route.

[0115] [4. Processing Procedure] Next, the procedure of information processing realized by the information processing device 100 will be described with reference to Fig. 10 and Fig. 11. The procedure of information processing can be roughly divided into two steps: a detection step for detecting various information based on sensor information, and a generation step for generating a driving video based on the detection results. Therefore, Fig. 10 will describe the procedure of the detection step, and Fig. 11 will describe the procedure of the generation step.

[0116] [4-1. Processing Procedure (1)] First, the procedure of information processing performed in the detection step will be described with reference to Fig. 10. Fig. 10 is a flowchart showing the procedure of the detection process according to the embodiment.

[0117] 10, first, the receiving unit 131 determines whether or not driving has started for each vehicle VEx equipped with the in-vehicle device 10, which is an example of an imaging means (step S1001). While the receiving unit 131 determines that driving of the vehicle VEx has not started (step S1001; No), the receiving unit 131 waits until it can determine that driving of the vehicle VEx has started.

[0118] On the other hand, when it is determined that driving of the vehicle VEx has started (step S1001; Yes), the receiving unit 131 receives sensor information sent in real time from the in-vehicle device 10 in accordance with the driving of the vehicle VEx (step S1002). Furthermore, the receiving unit 131 stores the received sensor information in the sensor information database 122. Furthermore, when a captured image captured by the in-vehicle device 10 is included in the acquired sensor information, the receiving unit 131 may store the captured image in the captured image database 121.

[0119] Next, the detection unit 132 detects the driving behavior of the vehicle VEx based on the sensor information received by the reception unit 131 (step S1003).

[0120] Furthermore, the detection unit 132 detects the state of the travel route according to the driving behavior of the vehicle VEx based on the sensor information received by the reception unit 131 (step S1004).

[0121] Furthermore, the detection unit 132 acquires the detection results detected in steps S1003 and S1004 as vehicle information, and registers the acquired vehicle information in the vehicle information database 123 (step S1005).

[0122] Next, the detection unit 132 determines whether or not the driving of the vehicle VEx has ended (step S1006). For example, the detection unit 132 may determine whether or not the driving of the vehicle VEx has ended based on whether or not the sensor information has been received from the reception unit 131. For example, the detection unit 132 determines that the driving of the vehicle VEx is continuing while the sensor information can be received from the reception unit 131 (step S1006; No), and executes the processing from step S1002 again. On the other hand, when the detection unit 132 is no longer able to receive sensor information from the reception unit 131, the detection unit 132 determines that the driving of the vehicle VEx has ended (step S1006; Yes), and ends the processing.

[0123] [4-2. Processing Procedure (2)] Next, the procedure of information processing performed in the generation step will be described with reference to Fig. 11. Fig. 11 is a flowchart showing the procedure of the generation process according to the embodiment.

[0124] 11, the request receiving unit 133 determines whether or not a generation request for generating a driving video has been received (step S1101). For example, the request receiving unit 133 may provide a registration form C1 in response to access from a user, and determine whether or not a generation request in which a target vehicle VEx and a generation condition have been specified has been received via the registration form C1.

[0125] While it is determined that a generation request has not been received (step S1101; No), the request receiving unit 133 waits until it can determine that a generation request has been received.

[0126] In the following, an example will be described in which a user U1, who is the owner of the vehicle VE1, sends a generation request via the registration form C1. In this case, the request receiving unit 133 determines that the generation request has been received (step S1101; Yes), and outputs information specified by the user U1 (for example, information indicating the target vehicle VE1 and the generation conditions) to the identification unit 134.

[0127] If the identification unit 134 determines that the generation request has been received (step S1101; Yes), it extracts vehicle information corresponding to the generation conditions (e.g., geographical range, temporal range, or driving mode) from the vehicle information corresponding to the target vehicle VE1 stored in the vehicle information database 123 (step S1102).

[0128] Next, based on the vehicle information extracted in step S1102, the identification unit 134 estimates the vehicle VEx that captured the image of the target vehicle VE1 via the in-vehicle device 10 from among the vehicles VEx other than the target vehicle VE1, and identifies the estimated vehicle VEx as the image source vehicle (step S1103).

[0129] Next, the acquisition unit 135 acquires, from among the captured images captured by the image capture source vehicle, captured images that show the target vehicle VE1 and that satisfy the generation condition (step S1104).

[0130] Next, the generation unit 136 generates a traveling video of a scene in which the target vehicle VE1 is traveling based on the acquired captured images (step S1105). For example, the generation unit 136 generates a traveling video of a scene in which the target vehicle VE1 is traveling in a manner that satisfies a generation condition by performing a synthesis process in which the captured images are arranged in chronological order and synthesized according to the date and time at which the captured images were captured.

[0131] Finally, the providing unit 137 provides the generated driving video to the user U1 (step S1106). For example, the providing unit 137 may control output to the user device 60 of the user U1 via the application AP so that the driving video is displayed.

[0132] [5. Hardware Configuration] The information processing device 100 according to the embodiment described above is realized, for example, by a computer 1000 configured as shown in Fig. 12. Fig. 12 is a hardware configuration diagram showing an example of a computer that realizes the functions of the information processing device 100. The computer 1000 has a CPU 1100, a RAM 1200, a ROM 1300, an HDD 1400, a communication interface (I / F) 1500, an input / output interface (I / F) 1600, and a media interface (I / F) 1700.

[0133] The CPU 1100 operates and controls each unit based on programs stored in the ROM 1300 or the HDD 1400. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 starts up, programs that depend on the hardware of the computer 1000, and the like.

[0134] The HDD 1400 stores programs executed by the CPU 1100, data used by such programs, etc. The communication interface 1500 receives data from other devices via a predetermined communication network and sends it to the CPU 1100, and transmits data generated by the CPU 1100 to other devices via the predetermined communication network.

[0135] The CPU 1100 controls output devices such as a display and a printer, and input devices such as a keyboard and a mouse, via the input / output interface 1600. The CPU 1100 acquires data from the input devices via the input / output interface 1600. The CPU 1100 also outputs generated data to the output devices via the input / output interface 1600.

[0136] Media interface 1700 reads a program or data stored in recording medium 1800 and provides it to CPU 1100 via RAM 1200. CPU 1100 loads the program or data from recording medium 1800 onto RAM 1200 via media interface 1700 and executes the loaded program. Recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase Change Rewritable Disc), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0137] For example, when the computer 1000 functions as the information processing device 100 according to the embodiment, the CPU 1100 of the computer 1000 executes programs loaded onto the RAM 1200 to realize the functions of the control unit 130. The CPU 1100 of the computer 1000 reads and executes these programs from the recording medium 1800, but as another example, the CPU 1100 may obtain these programs from another device via a predetermined communication network.

[0138] [6. Other] Furthermore, among the processes described in each of the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using known methods. In addition, the information including the processing procedures, specific names, various data, and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified. For example, the various information shown in each drawing is not limited to the information shown in the drawings.

[0139] Furthermore, the components of each device shown in the figure are conceptual functional components and do not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

[0140] Furthermore, the above-described embodiments can be combined as appropriate within the scope of not causing any contradiction in the processing content.

[0141] Although some of the embodiments of the present application have been described in detail above with reference to the drawings, these are merely examples, and the present invention can be implemented in other forms that include the embodiments described in the Disclosure of the Invention section and that have undergone various modifications and improvements based on the knowledge of those skilled in the art. [Explanation of symbols]

[0142] 1. Information Processing Systems 10 Onboard equipment 60 User equipment 100 Information processing device 120 Storage section 121 Image Database 122 Sensor Information Database 123 Vehicle Information Database 124 Video Database 130 Control Unit 131 Receiving unit 132 Detector 133 Request Reception Unit 134 Specific part 135 Acquisition Department 136 Generation part 137 Provision Department 138 Route control section

Claims

1. A request receiving unit that receives a generation request from a user, in which a driving mode of a target vehicle that is a target vehicle for which a driving video is to be generated is specified as a generation condition; an acquisition unit that acquires, in response to the generation request, an image captured by an imaging unit mounted on a vehicle other than the target vehicle, when the target vehicle is traveling in the traveling mode; a generating unit that generates the driving video of the scene in which the target vehicle is driving in the driving manner based on the captured image acquired by the acquiring unit; An information processing device comprising:

2. The request receiving unit receives the generation request in which one of straight driving, curve driving, or an arbitrary speed is specified as the generation condition.

2. The information processing apparatus according to claim 1, wherein:

3. The system further includes a route control unit that identifies a route on which the user's vehicle is likely to be photographed by the imaging means mounted on other vehicles based on the tendency of routes traveled by the vehicle equipped with the imaging means, and recommends the identified route to the user.

2. The information processing apparatus according to claim 1, wherein:

4. The route control unit identifies a route among routes to the user's destination that allows the user's vehicle to be more easily photographed by the imaging means mounted on the other vehicle, and recommends the identified route to the user as a candidate guide route.

4. The information processing apparatus according to claim 3,

5. The route control unit displays information indicating that the specified route is a route that is likely to be photographed by other vehicles, together with the specified route.

5. The information processing apparatus according to claim 3, wherein the information processing apparatus is a computer.

6. An information processing method executed by an information processing device, a request receiving step of receiving, from a user, a generation request in which a driving mode of a target vehicle, which is a target vehicle for which a driving video is to be generated, is specified as a generation condition; an acquisition step of acquiring, in response to the generation request, an image captured by an imaging means mounted on a vehicle other than the target vehicle, when the target vehicle is traveling in the traveling mode; a generating step of generating the driving video of the scene in which the target vehicle is driving in the driving manner based on the captured image acquired in the acquiring step; An information processing method comprising:

7. A request receiving step of receiving a generation request from a user, in which a driving mode of a target vehicle, which is a target vehicle for which a driving video is to be generated, is specified as a generation condition; an acquisition step of acquiring, in response to the generation request, an image captured by an imaging means mounted on a vehicle other than the target vehicle, when the target vehicle is traveling in the traveling mode; a generating step of generating the driving video of the scene in which the target vehicle is driving in the driving manner based on the captured image acquired by the acquiring step; An information processing program for causing an information processing device to execute the above.

Images