Information processing device, information processing method, and program

Abstract

According to the present invention, on the basis of the travel history of one or more vehicles, a plurality of icons indicating the position of each vehicle at a plurality of times are displayed on a map. In response to receiving an instruction from a user, a video request is transmitted to each of two or more vehicles or on-vehicle devices corresponding to two or more icons included in a first area set on the map, requesting transmission of a video captured at a corresponding time.

Description

Information processing device, information processing method, and program

[0001] The present disclosure relates to vehicle technology.

[0002] There are technologies for monitoring using mobile objects equipped with cameras. For example, Patent Document 1 discloses a system that uses a server device to manage the history of vehicle locations and identifies vehicles equipped with drive recorders that are located near a caller.

[0003] JP 2023-019792 A

[0004] The present disclosure aims to improve the convenience of a vehicle-based monitoring system.

[0005] One aspect of an embodiment of the present disclosure is an information processing device having a control unit that performs the following operations: displaying multiple icons on a map indicating the positions of one or more vehicles at multiple times based on the driving history of the one or more vehicles; and, in response to receiving instructions from a user, transmitting a video request to each of two or more vehicles or vehicle-mounted devices corresponding to two or more icons included in a first area set on the map, requesting the transmission of video captured at the corresponding time.

[0006] One aspect of an embodiment of the present disclosure is an information processing method executed by a computer, the information processing method including: a first step of displaying on a map a plurality of icons indicating the positions of one or more vehicles at a plurality of times based on the driving history of the one or more vehicles; and a second step of, in response to receiving an instruction from a user, sending a video request to each of two or more vehicles or on-board devices corresponding to two or more icons included in a first area set on the map, requesting the transmission of video captured at the corresponding time.

[0007] Another aspect of the present invention is a program for causing a computer to execute the above-described method, or a computer-readable storage medium that non-transitoryly stores the program.

[0008] According to the present disclosure, it is possible to improve the convenience of a monitoring system using a vehicle.

[0009] 1 is a schematic diagram of a monitoring system according to a first embodiment. A hardware configuration diagram of an in-vehicle device according to an embodiment. A hardware configuration diagram of a server device according to an embodiment. A hardware configuration diagram of a terminal device according to an embodiment. A software configuration diagram of an in-vehicle device according to an embodiment. A software configuration diagram of the server device according to an embodiment. A software configuration diagram of a terminal device according to an embodiment. A configuration diagram of video data acquired by an in-vehicle device. Examples of vehicle data, a vehicle data table, and a vehicle list. An example of a screen displayed on a terminal device in the first embodiment. An example of a screen displayed on a terminal device in the first embodiment. An example of a screen displayed on a terminal device in the first embodiment. An example of a playback screen for in-vehicle video. An example of a screen for collectively specifying icons included in a specified range. A diagram for explaining icons included in a specified range. A diagram for explaining the order in which requests are issued in a collective designation method. A diagram for explaining the order in which requests are issued in a collective designation method. A sequence diagram of a process in which a server device collects vehicle data. A sequence diagram of a process in which a server device provides a vehicle list. A sequence diagram of a process in which in-vehicle video is played back. A flowchart of a process in which a terminal device issues a playback request.

[0010] In recent years, automobiles have become increasingly connected, with an increasing number of vehicles equipped with communication functions and in-vehicle cameras.

[0011] In this regard, images captured by a camera (such as a dashcam) mounted on a vehicle can be accessed by a designated organization, which can be useful for initial responses to incidents and accidents. For example, when a call is made to 110 or 119, the relevant organization can quickly confirm the situation at the scene by obtaining dashcam footage from a vehicle that was traveling near the relevant location around the time the incident occurred. One known system for this purpose is one that maps icons representing the driving positions of multiple vehicles at a designated time in the past on a map and presents the maps to an operator.

[0012] For example, the system acquires location information of vehicles traveling in a specified area around a time specified by the operator (e.g., five minutes before the time of the report) and maps this information on a map. The operator selects a vehicle on the map that was traveling near a desired location (e.g., the location where the report was made) at any time close to the desired time (e.g., the time of the report). The system also requests that vehicle to transmit dashcam footage captured at the corresponding time. This allows the operator to view dashcam footage that may capture the scene of the report.

[0013] In such a system, it is expected that the operator will click multiple adjacent icons in sequence until the desired video is obtained. However, since there is a time lag between requesting the vehicle to acquire drive recorder video and the video actually becoming available for viewing, in such cases, the operator may experience intermittent waiting times, which may prevent the operator from smoothly reviewing the video. The information processing device disclosed herein solves such a problem.

[0014] An information processing device according to one aspect of the present disclosure has a control unit that performs the following operations: displays multiple icons on a map indicating the positions of one or more vehicles at multiple times based on the driving history of the vehicles; and, in response to receiving instructions from a user, sends a video request to each of two or more vehicles or on-board devices corresponding to two or more icons included in a first area set on the map, requesting the transmission of video captured at the corresponding time.

[0015] The control unit, for example, displays a map on which the locations of vehicles that have traveled within a predetermined area within a predetermined time span are mapped with icons based on the vehicle's travel history. The map display may be performed by a terminal connected to the information processing device. This makes it possible, for example, to request the provision of video from a vehicle corresponding to a specified icon or an in-vehicle device installed in the vehicle. For example, when one of a plurality of icons is selected, the control unit may transmit the video request to the corresponding vehicle or the in-vehicle device installed in the vehicle.

[0016] In addition, in order to address the above-mentioned problem, the information processing device disclosed herein automatically sends a video request to each of one or more vehicles or in-vehicle devices corresponding to multiple icons included in the first area in response to receiving instructions from a user.

[0017] The first area is an area having a predetermined size set on the map. For example, the first area may be automatically positioned near the center of the map, or may be positioned based on an operator's operation. The instruction may be performed by the user selecting the inside of the first area. For example, when the operator performs an operation to select the inside of the first area, a video request is issued to one or more vehicles or in-vehicle devices corresponding to icons within the area. The control unit may issue video requests to multiple vehicles or in-vehicle devices in parallel.

[0018] In this case, the timing of issuing the video request may be controlled so that two or more video requests are not sent to the same vehicle at the same time, because issuing multiple requests to one vehicle at the same time may result in unexpected behavior, such as the cancellation of an earlier request or the refusal of subsequent requests.

[0019] Furthermore, when the control unit receives the instruction, the control unit may preferentially transmit the video request to a vehicle or an in-vehicle device corresponding to an icon of a time that is closer to the predetermined time.

[0020] The predetermined time may be, for example, a time specified by an operator. If there are multiple icons in the first area, the video request can be transmitted in order, starting with the icon closest to the predetermined time. With this configuration, the video request can be transmitted preferentially to a vehicle that is more likely to capture the object desired by the operator.

[0021] In addition, when a first area is set on the map by the instruction, the control unit may preferentially send the video request to a vehicle or in-vehicle device corresponding to an icon that is closer to a specified position within the first area.

[0022] When there are multiple icons in the first area, the video request may be transmitted in order, for example, starting with the icon closest to the center of the first area. With this configuration, the video request can be transmitted preferentially to a vehicle that is more likely to capture the target object desired by the operator.

[0023] The control unit may also change the appearance of an icon corresponding to the vehicle or in-vehicle device to which the video request was transmitted. The control unit may also change the appearance of the icon corresponding to the vehicle or in-vehicle device when the user becomes able to view the video after the video request is transmitted. The control unit may also change the appearance of the icon corresponding to the vehicle or in-vehicle device when the vehicle or in-vehicle device to which the video request was transmitted notifies the user that preparations for video transmission are complete.

[0024] With this configuration, the operator can distinguish between vehicles to which a video request has been sent and those to which it has not been sent, or between vehicles to which video viewing is possible and those to which it has not been made possible.

[0025] The control unit may also output time information corresponding to one or more of the plurality of icons. With this configuration, the operator can know the time when the corresponding vehicle passed through the point on the map. That is, the operator can select a more preferable vehicle as the vehicle to request the provision of video.

[0026] The control unit may also output a list of time information corresponding to each of the plurality of icons included in the first area. Icons tend to be densely packed on the map in places where vehicles tend to gather, such as near stop lines before intersections. Therefore, the control unit may output a list of time information corresponding to the icons included in the first area. This allows the operator to determine which icon has the most recent time.

[0027] Specific embodiments of the present disclosure will be described below with reference to the accompanying drawings. Unless otherwise specified, the hardware configuration, module configuration, functional configuration, etc. described in each embodiment are not intended to limit the technical scope of the disclosure to those configurations.

[0028] (First embodiment) [System overview] An overview of a monitoring system according to a first embodiment will be described with reference to Fig. 1. The monitoring system according to this embodiment includes a plurality of vehicles 10, a server device 2, and a terminal device 3. The vehicles 10 are connected vehicles that can access a wireless communication network. The vehicles 10 can communicate with the server device 2 via the wireless communication network (e.g., a cellular communication network).

[0029] The vehicle 10 is equipped with an in-vehicle device 1. The in-vehicle device 1 functions as a drive recorder and a wireless communication device. The in-vehicle device 1 has a function of acquiring and storing video images (hereinafter referred to as "video" or "in-vehicle video") of the area in front of the vehicle using an in-vehicle camera, and a function of providing the stored video to an external device (e.g., a server device 2) in response to a request from the external device. The vehicle 10 also has a function of periodically transmitting location information to the server device 2.

[0030] In this embodiment, the in-vehicle device 1 has a wireless communication function, but the vehicle 10 may have a data communication module (hereinafter referred to as DCM) for connecting the components of the vehicle to a network. In other words, the in-vehicle device and the wireless communication device may be separate.

[0031] The server device 2 is a device configured to be able to communicate with multiple vehicles 10 via a network. The server device 2 periodically acquires location information from the multiple vehicles 10 under its management and stores the information in a database. By referring to the database managed by the server device 2, it is possible to identify locations and times where the multiple vehicles 10 under the management of the system have traveled in the past.

[0032] Based on the information acquired from the server device 2, the terminal device 3 identifies a vehicle 10 that is presumed to have the desired vehicle-mounted video, and acquires the vehicle-mounted video from the vehicle 10. Specifically, the terminal device 3 queries the server device 2 and acquires a list of vehicles that were traveling within a specified area at a specified time. The terminal device 3 also maps the relevant vehicles on a map and accepts vehicle selection from an operator. The terminal device 3 also requests the selected vehicle 10 to transmit the vehicle-mounted video. This allows the operator of the terminal device 3 to check the video captured by the target vehicle.

[0033] The terminal device 3 is installed in an institution that uses the video captured by the vehicle 10. Examples of such institutions include a police agency and a fire department. For example, when a fire department uses the terminal device 3, a call to 119 serves as a trigger for an operator to identify a vehicle 10 that was traveling near the scene of the call around the time of the call, and acquire on-board video from the vehicle 10.

[0034] [Hardware Configuration] Next, the hardware configuration of each device constituting the system will be described. First, the components of the vehicle 10 will be described. Fig. 2A is a diagram schematically illustrating an example of the hardware configuration of the in-vehicle device 1 mounted on the vehicle 10.

[0035] The in-vehicle device 1 can be configured as a computer having a processor (CPU, GPU, etc.), a main memory device (RAM, ROM, etc.), and an auxiliary memory device (EPROM, hard disk drive, removable media, etc.). The auxiliary memory device stores an operating system (OS), various programs, various tables, etc., and by executing the programs stored therein, various functions (software modules) that match specific purposes, as described below, can be realized. However, some or all of the functions may be realized as hardware modules using hardware circuits such as ASICs and FPGAs.

[0036] The in-vehicle device 1 includes a control unit 11 , a storage unit 12 , a communication unit 13 , a position information acquisition unit 14 , and a camera 15 .

[0037] The control unit 11 is a computing unit that executes predetermined programs to realize various functions of the in-vehicle device 1. The control unit 11 can be realized by, for example, a hardware processor such as a CPU. The control unit 11 may also be configured to include a RAM, a ROM (Read Only Memory), a cache memory, etc.

[0038] The storage unit 12 is a means for storing information, and is configured with storage media such as RAM, a magnetic disk, a flash memory, etc. The storage unit 12 stores programs executed by the control unit 11, data used by the programs, etc.

[0039] The communication unit 13 is a wireless communication interface for connecting the in-vehicle device 1 to an external network. The communication unit 13 is configured to be able to communicate with the server device 2 via, for example, a wireless LAN or a cellular communication network such as 3G, 4G, or 5G.

[0040] The position information acquisition unit 14 acquires position information of the vehicle 10. The position information acquisition unit 14 includes a GPS antenna and a positioning module for determining the position information. The GPS antenna is an antenna that receives positioning signals transmitted from positioning satellites (also referred to as GNSS satellites). The positioning module is a module that calculates position information based on the signals received by the GPS antenna. The position information acquisition unit 14 may determine the traveling direction of the vehicle 10 based on the transition of the position information. The camera 15 is an optical unit that includes an image sensor for acquiring images. The camera 15 is mounted, for example, facing forward of the vehicle 10.

[0041] Next, the server device 2 will be described. Fig. 2B is a diagram schematically illustrating an example of the hardware configuration of the server device 2. Like the in-vehicle device 1, the server device 2 can be configured as a computer having a processor (CPU, GPU, etc.), a main storage device (RAM, ROM, etc.), and an auxiliary storage device (EPROM, hard disk drive, removable media, etc.).

[0042] The server device 2 includes a control unit 21 , a storage unit 22 , and a communication unit 23 .

[0043] The control unit 21 is a computing unit that executes predetermined programs to realize various functions of the server device 2. The control unit 21 can be realized by, for example, a hardware processor such as a CPU. The control unit 21 may also be configured to include RAM, ROM (Read Only Memory), cache memory, etc.

[0044] The storage unit 22 is a means for storing information, and is configured with storage media such as RAM, a magnetic disk, a flash memory, etc. The storage unit 22 stores programs executed by the control unit 21, data used by the programs, etc.

[0045] The communication unit 23 is a communication interface for connecting the server device 2 to a network. The communication unit 23 is configured to be able to communicate with the network via, for example, Ethernet (registered trademark), a wireless LAN, a cellular communication network, or the like.

[0046] Next, the terminal device 3 will be described. Fig. 2C is a diagram schematically illustrating an example of the hardware configuration of the terminal device 3. Like the in-vehicle device 1 and the server device 2, the terminal device 3 can be configured as a computer having a processor (CPU, GPU, etc.), a main memory device (RAM, ROM, etc.), and an auxiliary memory device (EPROM, hard disk drive, removable media, etc.).

[0047] The terminal device 3 includes a control unit 31, a storage unit 32, a communication unit 33, and an input / output unit .

[0048] The control unit 31 is a computing unit that executes predetermined programs to realize various functions of the terminal device 3. The control unit 31 can be realized by a hardware processor such as a CPU, for example. The control unit 31 may also be configured to include RAM, ROM (Read Only Memory), cache memory, etc.

[0049] The storage unit 32 is a means for storing information, and is configured with storage media such as RAM, a magnetic disk, a flash memory, etc. The storage unit 32 stores programs executed by the control unit 31, data used by the programs, etc.

[0050] The communication unit 33 is a wireless communication interface for connecting the terminal device 3 to an external network. The communication unit 33 is configured to be able to communicate with the server device 2 via, for example, a wireless LAN or a cellular communication network such as 3G, 4G, or 5G.

[0051] The input / output unit 34 is a unit that receives input from an operator of the device and presents information to the operator. Specifically, the input / output unit 34 is composed of a touch panel and its control means, and a liquid crystal display and its control means. In this embodiment, the touch panel and the liquid crystal display are composed of a single touch panel display.

[0052] [Software Configuration] Next, the software configuration of each device constituting the system will be described. Fig. 3A is a diagram showing a schematic software configuration of the in-vehicle device 1 according to this embodiment.

[0053] In this embodiment, the control unit 11 of the in-car device 1 is configured to have three software modules: an image capturing unit 111, a notification unit 112, and a provision unit 113. Each software module may be realized by the control unit 11 (e.g., a CPU) executing a program stored in the storage unit 12. Note that the information processing executed by the software modules is synonymous with the information processing executed by the control unit 11 (e.g., a CPU).

[0054] While the device is in operation, the imaging unit 111 uses the camera 15 to capture video (in-vehicle video) and stores the obtained data in the storage unit 12. The imaging unit 111 receives power supply from the vehicle and continuously captures video, and records the obtained video data in the storage unit 12.

[0055] FIG. 4 is a diagram illustrating the structure of video data generated by the image capture unit 111 and stored in the storage unit 12. The video data is composed of multiple video files. The length of a video corresponding to one file is limited (e.g., one minute), and if the limit is exceeded, a new video file is generated. If the storage capacity is insufficient, the image capture unit 111 deletes the oldest video file to free up space and then continues recording. In this embodiment, each video file is associated with an identifier (date and time ID) that identifies the date and time the video file was generated (i.e., the date and time when recording began). The date and time ID may uniquely identify the year, month, day, hour, minute, and second. Operation of the image capture unit 111 causes the in-vehicle device 1 to function as a drive recorder.

[0056] The notification unit 112 periodically generates data related to the traveling of the vehicle (hereinafter referred to as vehicle data) and transmits it to the server device 2. In this embodiment, the vehicle data includes the position information of the vehicle 10, the traveling direction of the vehicle 10, and a date and time ID indicating the date and time when the position information was acquired. FIG. 5A shows an example of the vehicle data. In this embodiment, the vehicle data includes an identifier (vehicle ID) that uniquely identifies the vehicle 10, the date and time when the vehicle data was generated, an identifier (date and time ID) that uniquely identifies the date and time, the position information, and information indicating the traveling direction. The position information and the traveling direction can be acquired from the position information acquisition unit 14, which will be described later.

[0057] The vehicle data is generated and transmitted at a predetermined cycle. The predetermined cycle may be measured independently or may be linked to the image capturing unit 111. For example, the notification unit 112 may generate vehicle data at the timing when the image capturing unit 111 generates a new video file (for example, at one-minute intervals). In this case, the date and time indicated by the vehicle data will be the same as the date and time when the new video file was generated.

[0058] The providing unit 113 provides the terminal device 3 with a video file that was shot at a specified time, based on a request (video file provision request) sent from the terminal device 3. For example, when the providing unit 113 receives a provision request including a date and time ID from the terminal device 3, the providing unit 113 obtains the video file corresponding to the date and time ID from the storage unit 22 and transmits the video file to the terminal device 3.

[0059] Next, a description will be given of the software configuration of the server device 2. Fig. 3B is a diagram schematically showing the software configuration of the server device 2 according to this embodiment.

[0060] In this embodiment, the control unit 21 of the server device 2 is configured to have two software modules: a data update unit 211 and an information providing unit 212. Each software module may be realized by the control unit 11 (e.g., a CPU) executing a program stored in the storage unit 22. Note that the information processing executed by the software module is synonymous with the information processing executed by the control unit 21 (e.g., a CPU).

[0061] The data update unit 211 receives vehicle data from a plurality of vehicles 10 (on-board devices 1) and stores the vehicle data in the storage unit 22. For example, the data update unit 211 may store the vehicle data received from a plurality of vehicles 10 in a table (referred to as a vehicle data table) such as that shown in FIG.

[0062] The information providing unit 212 provides information to the terminal device 3 based on the information stored in the storage unit 22. Specifically, the information providing unit 212 performs the following two types of processing.

[0063] (1) Providing Information on Vehicle Travel Position The vehicle data table stores the location information of the vehicle 10 equipped with the in-vehicle device 1 having the function of a drive recorder, associated with date and time information. By referring to this, it is possible to extract vehicles that have traveled within a specified area within a specified time range. When a request is received from the terminal device 3, the information providing unit 212 extracts vehicles that have traveled within a specified area within a specified time range and transmits pairs of location information and date and time information to the terminal device 3. This allows mapping on a map on the terminal device 3 side. The location information may also include information indicating the direction of travel.

[0064] (2) Relay of In-Vehicle Video When the terminal device 3 selects a target vehicle for which in-vehicle video is requested, the information providing unit 212 receives a video playback request from the terminal device 3. The video playback request includes information identifying the target vehicle and the date and time of the playback target. Based on this information, the information providing unit 212 transfers the video playback request to the target vehicle. In addition, the information providing unit 212 transfers the video file (or video stream) of the in-vehicle video transmitted from the target vehicle to the terminal device 3.

[0065] The storage unit 22 stores a table (vehicle data table) that stores data received from the in-vehicle device 1 .

[0066] Next, a description will be given of the software configuration of the terminal device 3. Fig. 3C is a diagram schematically illustrating the software configuration of the terminal device 3 according to this embodiment.

[0067] In this embodiment, the control unit 31 of the terminal device 3 is configured to have two software modules: a display control unit 311 and a video acquisition unit 312. Each software module may be realized by the control unit 11 (CPU, etc.) executing a program stored in the storage unit 32. Note that the information processing executed by the software module is synonymous with the information processing executed by the control unit 31 (CPU, etc.).

[0068] The display control unit 311 generates and outputs a user interface (GUI) for requesting acquisition of in-vehicle video. In this embodiment, the display control unit 311 generates and outputs a user interface including a road map image based on stored or previously acquired road map data. The user interface includes a user interface component for specifying a target period.

[0069] Furthermore, the display control unit 311 requests the server device 2 for a "list of vehicles that have traveled within the display range of the map during a target period" based on an operation performed by the operator, and maps icons representing the positions of the vehicles on the map based on the response. For example, if the operator specifies a period of "from 17:00 to 17:03 today," the display control unit 311 requests the server device 2 for a "list of vehicles that have traveled within the display range of the map from 17:00 to 17:03 today." Furthermore, the display control unit 311 maps icons based on the received list of vehicles.

[0070] The video acquisition unit 312 can issue a request to acquire in-vehicle video to the vehicle 10 using two methods. The first method is a method in which, when the operator designates one of the multiple mapped icons, a request to play in-vehicle video is sent to the vehicle 10 corresponding to that icon. Hereinafter, this method will be referred to as the individual designation method.

[0071] The second method is a method in which, when the operator selects a predetermined range set on a map, all icons within that range are designated at once. In this case, a request to play back in-vehicle video is sequentially sent to multiple vehicles 10 corresponding to the icons within that range. Hereinafter, this method will be referred to as the collective designation method. The detailed method will be described later.

[0072] In either method, the video acquisition unit 312 receives and plays back the video file or video stream transmitted from the vehicle 10. If video files or video streams are transmitted from multiple vehicles 10 at the same time, the video acquisition unit 312 can receive the multiple video files or video streams in parallel and selectively play them back based on the operator's operation. In the following description, it is assumed that the playback request is transmitted to the vehicle 10 via the server device 2. It is also assumed that the video file or video stream is received via the server device 2.

[0073] 2A to 2C and 3A to 3C are merely examples, and all or part of the illustrated functions may be performed using specially designed circuits. Furthermore, programs may be stored or executed using a combination of a main memory device and an auxiliary memory device other than those illustrated.

[0074] [Details of Processing] Next, specific examples of operations performed by the operator via the terminal device 3 and the processing executed as a result will be described.

[0075] 6 shows an example of an initial screen provided to an operator of the terminal device 3. The display control unit 311 of the terminal device 3 generates a user interface including a road map centered on a predetermined point, and outputs the generated user interface to the input / output unit 34. The user interface includes a component for specifying a time (reference numeral 601), a component for specifying a time range (reference numeral 602), and a button for executing a search (reference numeral 603).

[0076] Reference numeral 601 denotes a user interface component for specifying a time. The operator uses this user interface component to set, for example, the time when a 119 emergency call was made. Reference numeral 602 denotes a user interface component for specifying a target period for vehicle search. For example, if "3 minutes ago" is specified here, the search period will be "from 3 minutes before the set time until the set time."

[0077] When the operator presses the search button (reference numeral 603), data requesting a list of vehicles that have traveled within the display range of the map during a set period (for example, from 16:57 to 17:00) is transmitted from the terminal device 3 to the server device 2. The request includes, for example, information for specifying the display range of the map, information indicating the target period, etc.

[0078] In response to the request, the server device 2 (information provider 212) generates a list of applicable vehicles (vehicle list) and returns it to the terminal device 3. FIG. 5(C) shows an example of the returned vehicle list. Note that although it is referred to as a "vehicle list" here, the records included in the list may include the same vehicle. For example, the same vehicle may transmit vehicle data multiple times within a set period within the display range of the map. In this case, multiple records with the same vehicle ID will be included in the vehicle list.

[0079] The terminal device 3 (display control unit 311) that receives the vehicle list maps icons representing vehicles at corresponding positions on a map. FIG. 7 shows an example of a map on which icons are mapped. The icons may be positioned taking into consideration the vehicle's direction of travel. An icon is generated for each record in the vehicle list. In other words, the same vehicle may be displayed by two or more icons. However, since the differences between vehicles are not important to the operator, in this embodiment, information for identifying the vehicle is not displayed on the screen.

[0080] The terminal device 3 can select an icon by either the individual designation method or the collective designation method described above, and request the vehicle 10 to transmit the in-vehicle video.

[0081] First, we will explain the operation in the case of the individual designation method. When an operator selects (e.g., clicks) one of the icons mapped on the map, an operation to acquire and play back in-vehicle video from the vehicle corresponding to that icon is initiated.

[0082] 8 is an example of a screen that is displayed when the operator selects an icon on the map in the example shown in FIG. 7. When an icon is selected, the date and time (reference numeral 801) corresponding to the selected icon is displayed on the screen. The date and time corresponding to the selected icon can be obtained from the vehicle list received from the server device 2. This allows the operator to understand the time sequence even when there are multiple icons near a desired location (e.g., a reporting location).

[0083] After checking the date and time, the operator determines whether to acquire in-vehicle video from the vehicle corresponding to the icon. If acquiring in-vehicle video from the vehicle corresponding to the icon, for example, the operator presses the corresponding icon again.

[0084] When the operator performs the above-mentioned operation, a request to play back the in-vehicle video (hereinafter referred to as a playback request) is sent from the terminal device 3 to the server device 2. The playback request includes a vehicle ID for specifying the vehicle and a date and time ID for specifying the date and time. Upon receiving the playback request, the server device 2 identifies the target vehicle and transfers the playback request to the corresponding in-vehicle device 1.

[0085] Upon receiving the playback request, the in-car device 1 extracts a video file corresponding to the specified date and time ID and transmits it to the server device 2. The transmission target may be a video file of in-car video or a video stream. The in-car device 1 may extract a video file that matches the specified date and time ID, or, if there is no video file that matches the specified date and time ID, may extract a video file captured at the closest time. The transmitted video file (or video stream) is relayed by the server device 2 to the terminal device 3, where the video is played. FIG. 9 shows an example of a playback screen on the terminal device 3.

[0086] In this example, since video files are generated in one-minute increments, playback is also performed in one-minute increments. To play back the previous or next video file, the operator presses a button (reference numeral 901) that instructs moving forward or backward. When the button is pressed, a playback request for acquiring the corresponding video file is generated and transmitted to the target in-vehicle device 1 through the same process. This allows the operator of the terminal device 3 to play back the in-vehicle video for the specified date and time.

[0087] Next, the operation in the case of the collective designation method will be described. When the terminal device 3 is operating in the collective designation method, an area for collective designation of icons (hereinafter referred to as the designated area) is drawn as a graphic in a predetermined area on the map. The designated area is an example of a "first area." FIG. 10 shows an example of a screen displayed when the terminal device 3 is operating in the collective designation method. In this embodiment, a graphic 1001 representing the designated area is drawn in the center of the map. Note that in this example, the designated area is automatically set in the center of the map, but the designated area may also be set on the map based on an operation by the operator. When the display range or scale of the map is changed, the position and size of the designated area may also be reset at the same time.

[0088] When the operator selects (e.g., clicks on) an icon within the specified area or a point within the specified area, a video playback request is automatically issued to the vehicle corresponding to each icon within the specified area. In this example, the issuance of a playback request is triggered by the operator selecting (e.g., clicking on) the interior of the specified area, but the issuance of a playback request may also be triggered by some other operation. For example, the issuance of a playback request may be triggered by clicking on an interface (e.g., a button) such as that indicated by reference numeral 1002, or the issuance of a playback request may be instructed without using a screen, such as by voice input.

[0089] Next, the order in which playback requests are issued when there are multiple icons within the specified area will be described. In this embodiment, a method using time and a method using distance will be exemplified as methods for determining the order in which playback requests are issued.

[0090] (1) Method Using Time This method transmits playback requests in order, starting with the icon whose associated time is closest to the time specified by the operator. FIG. 11 shows an example of a list of vehicles corresponding to icons in a specified area. In this example, three vehicles form five icons. Furthermore, the time specified by the operator is 12:00, and vehicle data from five minutes before (11:55) to the specified time is extracted.

[0091] In this example, when the records are sorted in order from closest to the specified time, the order is icons A, B, E, C, and D. Therefore, when the specified area is selected, the terminal device 3 issues playback requests in this order. FIG. 12 is a diagram explaining the order in which the terminal device 3 issues playback requests in this example. As shown in the figure, the terminal device 3 issues playback requests to the corresponding vehicles in order of closest to the time specified by the operator. In addition, the terminal device 3 sequentially receives the video files transmitted in response to the playback requests.

[0092] (2) Method Using Distance In this method, playback requests are sent in order from the position mapped on the map that is closest to the center of the specified area.

[0093] In the example of FIG. 11 , when the records are sorted in descending order from the center point of the designated area, the order is icons A, D, B, C, and E. Therefore, when the designated area is selected, the terminal device 3 issues playback requests in this order. FIG. 13 is a diagram explaining the order in which the terminal device 3 issues playback requests in this example. As shown in the figure, the terminal device 3 issues playback requests to corresponding vehicles in descending order from the center point of the designated area. The terminal device 3 also sequentially receives video files transmitted in response to the playback requests. The distance from the center point of the designated area can be determined based on the location information included in the vehicle list. Note that, although the distance from the center point of the designated area is used in this example, the distance from any other point within the designated area can also be used.

[0094] Note that icons A and B, and icons D and E, represent the same vehicle. In this case, the terminal device 3 issues the next playback request only after receiving a video file in response to the playback request. This prevents duplicate playback requests from being sent to the same vehicle.

[0095] When the reception of the video file is completed, the corresponding icon enables the playback (viewing) of the in-vehicle video. The terminal device 3 may change the appearance (color, etc.) of the icon for which the reception of the video file is completed. Furthermore, the terminal device 3 may start the playback of the in-vehicle video in response to the operator clicking on the icon whose appearance has changed.

[0096] To provide information to the operator, the terminal device 3 may output the corresponding date and time for each of the multiple icons included in the designated area, as in the example shown in Fig. 8. That is, the terminal device 3 may output a list of multiple dates and times. This allows the operator to understand the time sequence even when there are multiple icons in the designated area.

[0097] [Processing Sequence] Next, the steps executed by each device and the flow of data exchanged between devices will be described.

[0098] 14 is a sequence diagram of a process for transmitting vehicle data from the in-vehicle device 1 to the server device 2. The illustrated process is executed at a predetermined cycle. The predetermined cycle may be determined by the notification unit 112, or may be determined by the notification unit 112 in response to a notification from the image capture unit 111. For example, the image capture unit 111 may transmit a notification to the notification unit 112 every time a video file is switched (e.g., every minute), and in response to this, the notification unit 112 may start the illustrated process.

[0099] First, in step S11, the in-vehicle device 1 (notification unit 112) generates vehicle data. In this embodiment, the notification unit 112 acquires the position information and traveling direction of the vehicle from the position information acquisition unit 14, and generates vehicle data by combining the vehicle identifier and date and time information. The generated vehicle data is transmitted to the server device 2 (step S12).

[0100] In step S13 , the server device 2 (data update unit 211 ) receives the vehicle data and stores the vehicle data in the vehicle data table in the storage unit 22 .

[0101] By performing the above-described process for multiple vehicles 10, the server device 2 can collect and manage past location information for multiple vehicles under its management. Furthermore, by referencing the vehicle data table stored in the server device 2, it becomes possible to search for vehicles that were traveling at a specific location on a specific date and time. Because each vehicle records on-board video while traveling, it becomes possible to search for vehicles that are likely to have captured video of a specific event (e.g., a traffic accident or other event related to a report).

[0102] 15 is a sequence diagram of a process in which the terminal device 3 outputs a map onto which vehicle position information is mapped. The illustrated process is started, for example, when a map is displayed on the terminal device 3 (display control unit 311).

[0103] First, in step S21, the terminal device 3 (display control unit 311) determines whether redrawing is necessary. For example, if a map is drawn for the first time or if an operation to change the display range of the map has been performed, this step will result in a positive determination. It is assumed here that an operator of the terminal device 3 has displayed a predetermined range of the map and specified a target period. If the determination in this step is positive, the terminal device 3 generates a vehicle list generation request and transmits it to the server device 2. The vehicle list generation request includes information indicating the display range of the map and information indicating the target period specified by the operator.

[0104] In step S22, the server device 2 (information provider 212) generates a vehicle list based on the received request. Specifically, records of vehicles that have traveled within a specified range during a specified period are extracted from the vehicle data table stored in the storage unit 22, and a vehicle list consisting of the extracted records is generated. The generated vehicle list is transmitted to the terminal device 3.

[0105] In step S23, the terminal device 3 (display control unit 311) draws a map based on the acquired vehicle list and maps icons to corresponding points on the map.

[0106] When the process of step S23 is completed, a plurality of icons are mapped on the map on the terminal device 3 as shown in Fig. 7. The icons represent the positions of vehicles that have been traveling within the display range of the map during the specified period. When the collective designation method is adopted, the terminal device 3 may also draw a figure representing the specified area on the map at the same time as drawing the icons, as shown in Fig. 10.

[0107] 16 is a sequence diagram of a process in which the in-vehicle device 1 provides an image to the terminal device 3 based on a request from the terminal device 3. The process shown in FIG. 16 is started by the terminal device 3 (display control unit 311) after a vehicle icon is mapped on a map.

[0108] First, the process for specifying a vehicle using the individual specification method will be described. When the operator of the terminal device 3 performs an operation to select one of the displayed icons, the display control unit 311 accepts this. In response to the selection operation, the display control unit 311 may refer to the vehicle list, obtain date and time information corresponding to the selected icon, and output it (for example, superimpose it on the screen).

[0109] Next, the video acquisition unit 312 generates a playback request (step S31). The playback request is data for requesting the in-vehicle device 1 to provide the in-vehicle video, and includes an identifier (vehicle ID) of the in-vehicle device 1 (vehicle 10) and an identifier (date and time ID) of the date and time when the in-vehicle video was captured. The vehicle ID and date and time ID are included in the vehicle list received by the terminal device 3 from the server device 2. When the operator selects an icon on the map, the terminal device 3 can identify the vehicle ID and date and time ID of the corresponding vehicle.

[0110] The playback request is transmitted from the terminal device 3 to the server device 2, and is relayed from the server device 2 to the target in-vehicle device 1. At this time, the server device 2 may read the vehicle ID included in the playback request and identify the in-vehicle device 1 to which the playback request is to be relayed. For this reason, it is preferable that the server device 2 stores data associating the vehicle ID with the connection destination (e.g., network address) of the corresponding in-vehicle device 1. The playback request relayed by the server device 2 is received by the corresponding in-vehicle device 1.

[0111] Upon receiving the playback request, the in-car device 1 (providing unit 113) extracts a video file corresponding to the date and time ID from the stored video data (step S32). For example, in the example of FIG. 4, if the date and time ID T002 is specified, the in-car device 1 (providing unit 113) extracts a video file that started recording at 12:01. The extracted video file is transmitted to the terminal device 3 via the server device 2. The terminal device 3 generates a playback screen such as that shown in FIG. 9 and starts playback of the received video file (step S33). Note that if an operation to specify another video file is performed (e.g., if the button indicated by reference numeral 901 is pressed), the process shown in FIG. 16 is executed again, and a new playback request is generated. Note that, although a video file is the target of transmission in this example, a video stream or the like may also be the target of transmission.

[0112] Next, a process for specifying vehicles using the collective designation method will be described. FIG. 16 shows a sequence for when an operator specifies one vehicle and issues a playback request. On the other hand, when the collective designation method is adopted, the terminal device 3 executes a process for generating and issuing multiple playback requests for multiple icons included in a designated area. FIG. 17 is a flowchart of this process. The process shown in FIG. 17 is started by the terminal device 3 (display control unit 311) when a designated area on a map is selected by the operator. The designated area may be set automatically by the system or may be set by the operator.

[0113] First, in step S41, information about the icons included in the specified area is obtained. In this step, information included in the vehicle list is obtained for each icon included in the specified area. Next, in step S42, the order in which playback requests are issued is determined. As described above, the order in which playback requests are issued may be determined based on time (the difference from the specified time) or distance (the distance from the center of the specified area).

[0114] The processes of steps S43 to S45 are executed for each of the vehicles corresponding to the icons included in the specified area in the order in which the playback requests were issued. The vehicle corresponding to a specific icon can be determined based on the vehicle list.

[0115] In step S43, it is determined whether a regeneration request can be issued to the vehicle being processed. For example, if a regeneration request has been sent to a vehicle and a response has not yet been received from that vehicle, another regeneration request should not be sent. This is because sending a duplicate regeneration request in such a case could cause an error. In such a case, it is determined that a regeneration request cannot be issued to the target vehicle.

[0116] If a playback request can be issued to the target vehicle, the process proceeds to step S44, where a playback request is issued to the vehicle. As a result, the process shown in FIG. 16 is executed, and the video file is received. If a playback request cannot be issued to the target vehicle, the process proceeds to step S46, where the process waits until a playback request can be issued. For example, if a response is received to a previously issued playback request during the wait, it is determined that a new playback request can be issued.

[0117] In step S45, it is determined whether all playback requests have been issued, i.e., whether processing has been completed for all icons included in the specified area. If the determination in this step is negative, the process returns to step S43. If the determination in this step is positive, the process ends.

[0118] In the batch designation method, multiple video files may be transmitted to the terminal device 3 at the same time. When the terminal device 3 receives a video file, it may mark up the corresponding icon, i.e., change the appearance of the icon. Markup includes, for example, highlighting the icon, changing the color, or changing the design. When an icon is marked up, it means that the video file has been received and is ready to be played. When a marked-up icon is selected, the terminal device 3 may start playing the corresponding video file.

[0119] As described above, in the monitoring system according to this embodiment, the terminal device 3 maps the vehicle's driving history on a map based on the vehicle data stored in the server device 2. The operator of the terminal device 3 can specify a vehicle on the displayed map and obtain a video file from the vehicle. This allows the operator of the terminal device 3 to check on-board video captured by the vehicle traveling at a desired location at a desired time.

[0120] Furthermore, the monitoring system according to this embodiment can automatically issue a request to play back in-vehicle video for multiple icons included in a specified area set on a map. This allows the operator to automatically acquire in-vehicle video for multiple vehicles traveling within a desired area without having to select each icon one by one. Furthermore, the terminal device 3 controls the system so that multiple playback requests are not issued to the same vehicle. This prevents errors and enables in-vehicle video to be acquired in a short time.

[0121] (Modifications) The above-described embodiment is merely an example, and the present disclosure may be modified as appropriate within the scope of the gist thereof. For example, the processes and means described in the present disclosure may be freely combined and implemented as long as no technical contradiction occurs.

[0122] Furthermore, in the embodiment, a vehicle equipped with an on-board device is exemplified, but any moving object other than a vehicle may be used as long as it is capable of capturing video.

[0123] In addition, in the embodiment, an example is given in which the appearance of the corresponding icon is changed when the terminal device 3 has completed receiving a video file, but when in-car video is being streamed, the terminal device 3 may receive a notification from the in-car device 1 that preparations for streaming have been completed, and in response to this, the terminal device 3 may change the appearance of the corresponding icon.

[0124] In the embodiment, an example has been given in which the appearance of the corresponding icon is changed when the terminal device 3 has completed receiving a video file, but the terminal device 3 may change the appearance of the icon at the timing of transmitting a playback request. Furthermore, the appearance of the icon may be changed in multiple steps, such as "transmission of playback request completed" and "reception of video file completed." In this case, the terminal device 3 may, for example, perform control to gradually change the color of the icon.

[0125] In the embodiment, the designated area is drawn on the map (reference numeral 1001), but the designated area does not necessarily have to be drawn on the map. Even in this embodiment, an invisible designated area is set on the map, and a reproduction request is issued to each vehicle by the above-mentioned process.

[0126] In addition, in the embodiment, the terminal device 3 generates and outputs a user interface including a road map based on information received from the server device 2. However, the road map may be generated by the server device 2. In this case, instead of the server device 2 generating a vehicle list and transmitting it to the terminal device 3, the server device 2 may draw the road map and map icons. In this case, a user interface screen including the road map generated by the server device 2 may be transmitted to the terminal device 3, and the terminal device 3 may output it. In other words, the functions of the terminal device 3 (display control unit 311) described in the embodiment may be provided in the server device 2 (information providing unit 212). The information processing device according to the present disclosure may be realized as the server device 2 or as the terminal device 3.

[0127] Furthermore, a process described as being performed by one device may be shared and executed by multiple devices. Alternatively, a process described as being performed by different devices may be executed by a single device. In a computer system, the hardware configuration (server configuration) by which each function is realized can be flexibly changed.

[0128] The present disclosure can also be realized by supplying a computer program that implements the functions described in the above embodiments to a computer, and having one or more processors in the computer read and execute the program. Such a computer program may be provided to the computer on a non-transitory computer-readable storage medium connectable to the computer's system bus or via a network. Non-transitory computer-readable storage media include, for example, any type of disk, such as a magnetic disk (e.g., a floppy disk, a hard disk drive (HDD), etc.), an optical disk (e.g., a CD-ROM, a DVD disk, a Blu-ray disk), a read-only memory (ROM), a random access memory (RAM), an EPROM, an EEPROM, a magnetic card, a flash memory, an optical card, or any type of medium suitable for storing electronic instructions.

[0129] DESCRIPTION OF SYMBOLS 1... In-vehicle device 2... Server device 3... Terminal device 11, 21, 31... Control unit 12, 22, 32... Storage unit 13, 23, 33... Communication unit 14... Position information acquisition unit 15... Camera 34... Input / output unit

Claims

1. An information processing device having a control unit that performs the following operations: displaying multiple icons on a map indicating the positions of one or more vehicles at multiple times based on the driving history of the vehicles; and, in response to receiving instructions from a user, sending a video request to each of two or more vehicles or in-vehicle devices corresponding to two or more icons included in a first area set on the map, requesting the transmission of video captured at the corresponding time.

2. The information processing device according to claim 1, wherein when one of the plurality of icons is selected, the control unit transmits the video request to a corresponding vehicle or an in-vehicle device installed in the vehicle.

3. The information processing device according to claim 1, wherein the instruction is given by the user selecting an area inside the first area.

4. The information processing device according to claim 1, wherein, when the control unit receives the instruction, the control unit transmits the video request with higher priority to the vehicle or in-vehicle device corresponding to the icon of the time closest to the predetermined time.

5. The information processing device according to claim 1, wherein when a first area is set on the map by the instruction, the video request is sent preferentially to a vehicle or in-vehicle device corresponding to an icon that is closer to a predetermined position within the first area.

6. The information processing device according to claim 1, wherein the control unit changes the appearance of an icon corresponding to the vehicle or in-vehicle device to which the video request was sent.

7. The information processing device according to claim 1, wherein the control unit changes the appearance of an icon corresponding to the vehicle or in-vehicle device when the user becomes able to view the video after the video request is transmitted.

8. The information processing device according to claim 1, wherein the control unit changes the appearance of an icon corresponding to the vehicle or in-vehicle device when the vehicle or in-vehicle device to which the video request was sent notifies the control unit that preparations for video transmission are complete.

9. An information processing method executed by a computer, comprising: a first step of displaying on a map a plurality of icons indicating the positions of one or more vehicles at a plurality of times based on the driving history of the vehicles; and a second step of, in response to receiving an instruction from a user, sending a video request to each of two or more vehicles or vehicle-mounted devices corresponding to two or more icons included in a first area set on the map, requesting the transmission of video captured at the corresponding times.

10. The information processing method according to claim 9, further comprising a third step of transmitting the video request to a corresponding vehicle or an in-vehicle device mounted on the vehicle when any of the plurality of icons is selected.

11. The information processing method according to claim 9, wherein the instruction is given by the user selecting the inside of the first area.

12. The information processing method according to claim 9, wherein in the second step, when the instruction is received, the video request is sent with higher priority to the vehicle or in-vehicle device corresponding to the icon of the time closest to the specified time.

13. An information processing method as described in claim 9, wherein in the second step, when a first area is set on the map by the instruction, the video request is sent with higher priority to a vehicle or in-vehicle device corresponding to an icon that is closer to a specified position within the first area.

14. The information processing method according to claim 9, further comprising a fourth step of changing the appearance of an icon corresponding to the vehicle or in-vehicle device to which the video request was sent.

15. The information processing method according to claim 9, further comprising a fourth step of changing the appearance of an icon corresponding to the vehicle or in-vehicle device when the user is able to view the video after the video request has been sent.

16. The information processing method according to claim 9, further comprising a fourth step of changing the appearance of an icon corresponding to the vehicle or in-vehicle device when a notification is received from the vehicle or in-vehicle device to which the video request was sent that preparations for video transmission have been completed.

17. A program for causing a computer to execute the information processing method according to any one of claims 9 to 16.

Images