Information processing apparatus, information processing method, and storage medium

Abstract

The invention relates to an information processing apparatus, an information processing method, and a storage medium. The present disclosure improves convenience in a monitoring system using a vehicle. On the basis of a travel history of one or more vehicles, position information for each vehicle for generating a map in which a plurality of icons indicating the position of each vehicle at a plurality of times are disposed is output, and time information corresponding to one or more icons among the plurality of icons is output. A video request for requesting transmission of a video captured at the corresponding time is transmitted to a vehicle or an in-vehicle device corresponding to a first icon specified by the first user operation.

Description

Technical Field

[0001] The present disclosure relates to vehicle technology. Background Art

[0002] There are technologies for monitoring mobile objects equipped with cameras. For example, Patent Document 1 discloses a system that manages the driving location history of vehicles using a server device and identifies vehicles equipped with drive recorders that are located near a reporter.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Laid-Open No. 2023-019792 Summary of the Invention

[0006] An object of the present disclosure is to improve the convenience of a monitoring system using a vehicle.

[0007] One embodiment of the present disclosure is an information processing device having a control unit that performs the following actions: outputting position information of each vehicle for generating a map based on the driving history of one or more vehicles, wherein the map is a map configured with multiple icons representing the positions of each vehicle at multiple times; outputting time information corresponding to one or more of the multiple icons; and sending an image request to the vehicle or vehicle-mounted device corresponding to the first icon specified by a first user operation to request the transmission of an image captured at the corresponding time.

[0008] One embodiment of the present disclosure is an information processing method, wherein an information processing device performs the following actions: outputting position information of each vehicle for generating a map based on the driving history of one or more vehicles, wherein the map is a map configured with multiple icons representing the positions of the vehicles at multiple moments; outputting time information corresponding to one or more of the multiple icons; and sending an image request to a vehicle or a vehicle-mounted device corresponding to a first icon specified by a first user operation, requesting the transmission of an image captured at a corresponding moment.

[0009] Furthermore, as another embodiment, there may be a program for causing a computer to execute the above-described method, or a computer-readable storage medium that non-transitorily stores the program.

[0010] Effects of the Invention

[0011] According to the present disclosure, the convenience of a monitoring system using a vehicle can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 This is a schematic diagram of the monitoring system according to the first embodiment.

[0013] Figure 2A This is a hardware configuration diagram of the vehicle-mounted device according to the embodiment.

[0014] Figure 2B This is a hardware configuration diagram of a server device according to an embodiment.

[0015] Figure 2C This is a hardware configuration diagram of a terminal device according to an embodiment.

[0016] Figure 3A This is a software configuration diagram of the vehicle-mounted device according to the embodiment.

[0017] Figure 3B It is a software configuration diagram of the server device according to the embodiment.

[0018] Figure 3C It is a software configuration diagram of the terminal device according to the embodiment.

[0019] Figure 4 This is a diagram showing the structure of moving image data acquired by the vehicle-mounted device.

[0020] Figure 5 is an example of vehicle data, vehicle data table, and vehicle list.

[0021] Figure 6 This is an example of a screen displayed on the terminal device in the first embodiment.

[0022] Figure 7 This is an example of a screen displayed on the terminal device in the first embodiment.

[0023] Figure 8 This is an example of a screen displayed on the terminal device in the first embodiment.

[0024] Figure 9 This is an example of a reproduced image of an in-vehicle image.

[0025] Figure 10 This is a sequence diagram of the process of collecting vehicle data by the server device.

[0026] Figure 11 This is a sequence diagram of the process of providing a vehicle list by the server device.

[0027] Figure 12 This is a flowchart of the processing executed by the terminal device in the first embodiment.

[0028] Figure 13 This is a sequence diagram of the process of reproducing the in-vehicle image.

[0029] Figure 14 This is a diagram for explaining the technical problem to be solved by the second embodiment.

[0030] Figure 15 This is a diagram for explaining the technical problem to be solved by the second embodiment.

[0031] Figure 16 This is a flowchart of the processing executed by the terminal device in the second embodiment.

[0032] Figure 17 This is an example of a map displayed on a terminal device in the second embodiment.

[0033] Description of reference numerals:

[0034] 1: In-vehicle device; 2: Server device; 3: Terminal device; 11, 21, 31: Control unit; 12, 22, 32: Storage unit; 13, 23, 33: Communication unit; 14: Position information acquisition unit; 15: Camera; 34: Input / output unit. DETAILED DESCRIPTION

[0035] In recent years, the interconnectedness of automobiles has been advancing, and more and more vehicles are equipped with communication functions. In addition, more and more vehicles are equipped with onboard cameras.

[0036] In connection with this, images captured by a camera (such as a drive recorder) mounted on a vehicle can be inquired from a predetermined organization, thereby contributing to emergency response to incidents and accidents.

[0037] For example, when a call to 110 (police) or 119 (fire alarm) is made, the relevant departments can quickly confirm the situation at the scene by obtaining dashcam footage from vehicles traveling near the corresponding location before and after the incident.

[0038] One known system for this purpose maps icons representing the driving positions of multiple vehicles at specified past times onto a map and presents them to the operator. The operator selects a vehicle on the map that was traveling near a desired location (e.g., the location of the notification) at a desired time (e.g., the time the notification was generated). This allows the operator to request dashcam footage from that vehicle. Furthermore, the operator can confirm dashcam footage that likely captured the notification scene.

[0039] However, when the notification is generated, there may not be a vehicle equipped with a drive recorder near the scene of the incident. Therefore, it is considered to set a certain time width (for example, 5 minutes before the notification) and map the vehicle's position information within this time width.

[0040] However, when a time width is set, while icons (vehicle candidates) appear on the map, it is difficult to know when the vehicle was located at the location indicated by the icon. This is because only icons are displayed on the map, so it is impossible to tell from the appearance what time and minute the location information of the displayed icon is.

[0041] The information processing device in the present disclosure solves this problem.

[0042] An information processing device of one embodiment of the present invention has a control unit that performs the following actions: outputting position information of each vehicle for generating a map based on the driving history of one or more vehicles, wherein the map is a map configured with multiple icons representing the positions of the vehicles at multiple moments; outputting time information corresponding to one or more of the multiple icons; and sending an image request to the vehicle or vehicle-mounted device corresponding to the first icon specified by a first user operation to request the transmission of an image captured at the corresponding moment.

[0043] For example, based on the vehicle's driving history, the control unit outputs location information that maps the location of vehicles that have driven within a specified area within a specified time period onto a map using icons. The device operator can identify the vehicle by specifying the icon on the map mapped based on this location information. The mapping of location information can also be performed on the terminal device operated by the operator. Furthermore, the operator can request the provision of images from a specific vehicle or its onboard device based on the operator's operation.

[0044] However, simply mapping the location information onto a map allows you to know the vehicle's location, but not the time it was at that location. For example, if multiple icons are displayed on a map, it is impossible to determine which icon is closest to the reported time.

[0045] Therefore, in the present disclosure, the control unit is configured to output time information corresponding to one or more icons. Thus, by clicking an icon, for example, the operator can find out the time when the corresponding vehicle passed a location on a map. In other words, the operator can select a more preferred vehicle as the target vehicle for requesting image provision.

[0046] Furthermore, in response to the first user operation, the control unit requests the vehicle or vehicle-mounted device corresponding to the icon to transmit the image captured at the corresponding time. The first user operation may be, for example, an operation such as clicking a predetermined interface component within a screen after the time information corresponding to the icon is output to the screen. The predetermined interface component may also be the icon itself.

[0047] The time information may be output automatically or triggered by a user operation. For example, if a second user operation designating an icon is performed before a first user operation, the time information corresponding to the icon designated by the second user operation may be output.

[0048] In addition, when the first area includes a specified number of icons or more, the control unit outputs a list of time information corresponding to the multiple icons included in the first area, wherein the first area is an area of ​​a specified size that includes the icons specified by the second user operation.

[0049] In places where vehicles tend to gather, such as near stop lines near intersections, icons tend to be densely packed on the map. In such cases, operators may need to click on multiple icons in sequence to obtain time information. Therefore, if a first area on the map contains more than a specified number of icons, the time information corresponding to the icons in that area may be tabulated and output. This allows the operator to determine, for example, which icon represents the most recent time within a specified timeframe.

[0050] In this case, when an operation is performed as the first user operation to select any of the time information included in the list, the control unit may transmit an image request to the corresponding vehicle or vehicle-mounted device, requesting the transmission of an image captured at the corresponding time. Since each time information is associated with an icon, the operation of selecting a time information can be regarded as an operation of specifying an icon.

[0051] Furthermore, the control unit may highlight and display the latest time information among the time information included in the list.

[0052] For example, when many icons of vehicles waiting at a traffic light are displayed, if the latest icon is known, the probability of identifying the vehicle whose image was captured at the moment of starting travel increases.

[0053] Specific embodiments of the present disclosure are described below with reference to the accompanying drawings. Unless otherwise specified, the hardware configuration, module configuration, functional configuration, etc. described in each embodiment are not intended to limit the scope of the disclosed technology to these.

[0054] (First embodiment)

[0055] [System Overview]

[0056] Reference Figure 1The monitoring system of the first embodiment will be described in general terms. The monitoring system of this embodiment is configured to include multiple vehicles 10, a server device 2, and a terminal device 3. Vehicles 10 are networked vehicles capable of accessing a wireless communication network. Vehicles 10 can communicate with server device 2 via a wireless communication network (e.g., a cellular communication network).

[0057] Vehicle 10 is equipped with an onboard device 1. The onboard device 1 functions as a drive recorder and a wireless communication device. The onboard device 1 has the function of capturing and storing moving images (hereinafter referred to as "images" or "onboard images") in front of the vehicle using an onboard camera, and the function of providing the stored images to an external device (e.g., server device 2) in response to a request from the external device. Furthermore, the vehicle 10 has the function of periodically transmitting location information to the server device 2.

[0058] It should be noted that in this embodiment, the in-vehicle device 1 is configured as a device having wireless communication capabilities. However, the vehicle 10 may also include a data communication module (DCM) for connecting components of the vehicle to a network. In other words, the in-vehicle device and the wireless communication device may be separate.

[0059] Server device 2 is a device capable of communicating with multiple vehicles 10 via a network. Server device 2 periodically acquires location information from the multiple vehicles 10 under its management and stores it in a database. By referencing the database managed by server device 2, it is possible to determine the locations and times at which the multiple vehicles 10 under system management have traveled in the past.

[0060] The terminal device 3 identifies the vehicle 10 estimated to have the desired in-vehicle image based on the information acquired from the server device 2 , and acquires the in-vehicle image from the vehicle 10 .

[0061] Specifically, the terminal device 3 queries the server device 2 to obtain a list of vehicles traveling within a specified range at a specified time. Furthermore, the terminal device 3 maps the corresponding vehicles on a map and accepts a vehicle selection from the operator. Furthermore, the terminal device 3 requests the selected vehicle 10 to transmit onboard images. This allows the operator of the terminal device 3 to confirm images captured by the passing vehicle.

[0062] Terminal device 3 is configured for an organization that utilizes images captured by vehicle 10. Examples of such organizations include police and fire departments. For example, when a fire department utilizes terminal device 3, upon receiving a 119 notification, an operator identifies a vehicle 10 traveling near the notification site around the time of the notification and acquires onboard images from that vehicle 10.

[0063] [Hardware composition]

[0064] Next, the hardware configuration of each device constituting the system will be described.

[0065] First, components of the vehicle 10 will be described. Figure 2A 1 is a diagram schematically showing an example of the hardware configuration of the vehicle-mounted device 1 mounted on the vehicle 10 .

[0066] The vehicle-mounted device 1 can be configured as a computer having a processor (CPU (Central Processing Unit), GPU (Graphic Processing Unit), etc.), a main storage device (RAM (Random Access Memory), ROM (Read-Only Memory), etc.), and an auxiliary storage device (EPROM (Erasable Programmable Read-Only Memory), hard disk drive, removable media, etc.). The auxiliary storage device stores an operating system (OS), various programs, various tables, etc., and by executing the programs stored therein, various functions (software modules) that meet the prescribed purposes as described below can be realized. However, some or all of the functions can be realized as hardware modules using hardware circuits such as ASICs (Application Specific Integrated Circuits) and FPGAs (Field Programmable Gate Arrays).

[0067] The vehicle-mounted device 1 is configured to include a control section 11 , a storage section 12 , a communication section 13 , a position information acquisition section 14 , and a camera 15 .

[0068] The control unit 11 is a computing component that implements the various functions of the vehicle-mounted device 1 by executing predetermined programs. The control unit 11 can be implemented, for example, by a hardware processor such as a CPU. Alternatively, the control unit 11 may be configured to include RAM, ROM (Read Only Memory), cache memory, and the like.

[0069] The storage unit 12 is a means for storing information and is composed of a storage medium such as RAM, a magnetic disk, and a flash memory. The storage unit 12 stores programs executed by the control unit 11 and data used by the programs.

[0070] The communication unit 13 is a wireless communication interface for connecting the in-vehicle device 1 to an external network. The communication unit 13 is configured to communicate with the server device 2 via, for example, a wireless LAN (Local Area Network), a cellular communication network such as 3G (3rd Generation), 4G (4th Generation), or 5G (5th Generation).

[0071] The position information acquisition unit 14 acquires the position information of the vehicle 10. The position information acquisition unit 14 includes a GPS (Global Positioning System) antenna and a positioning module for determining the position information. The GPS antenna receives positioning signals transmitted from positioning satellites (also known as GNSS (Global Navigation Satellite System) satellites). The positioning module calculates the position information based on the signals received by the GPS antenna. It should be noted that the position information acquisition unit 14 can also determine the direction of travel of the vehicle 10 based on the changes in the position information.

[0072] The camera 15 is an optical component including an image sensor for acquiring an image and is installed to face the front of the vehicle 10, for example.

[0073] Next, the server device 2 will be described. Figure 2B This is a diagram schematically showing an example of the hardware configuration of the server device 2 .

[0074] Like the in-vehicle device 1 , the server device 2 may be configured as a computer having a processor (CPU, GPU, etc.), a main storage device (RAM, ROM, etc.), and an auxiliary storage device (EPROM, hard disk drive, removable media, etc.).

[0075] The server device 2 is configured to include a control section 21 , a storage section 22 , and a communication section 23 .

[0076] The control unit 21 is a computing component that realizes various functions of the server device 2 by executing a predetermined program. The control unit 21 can be realized by a hardware processor such as a CPU. Alternatively, the control unit 21 can be configured to include RAM, ROM, cache memory, etc.

[0077] The storage unit 22 is a means for storing information and is composed of a storage medium such as RAM, a magnetic disk, and a flash memory. The storage unit 22 stores programs executed by the control unit 21 and data used by the programs.

[0078] The communication unit 23 is a communication interface for connecting the server device 2 to a network. The communication unit 23 can be configured to be able to communicate with the network via, for example, Ethernet (registered trademark), wireless LAN, a cellular communication network, or the like.

[0079] Next, the terminal device 3 will be described. Figure 2C This is a diagram schematically showing an example of the hardware configuration of the terminal device 3 .

[0080] The terminal device 3, like the vehicle-mounted device 1 and the server device 2, can be configured as a computer having a processor (CPU, GPU, etc.), a main storage device (RAM, ROM, etc.), and an auxiliary storage device (EPROM, hard disk drive, removable media, etc.).

[0081] The terminal device 3 is configured to include a control unit 31 , a storage unit 32 , a communication unit 33 , and an input / output unit 34 .

[0082] The control unit 31 is a computing component that realizes various functions of the terminal device 3 by executing a predetermined program. The control unit 31 can be realized by a hardware processor such as a CPU. In addition, the control unit 31 can be configured to include RAM, ROM, cache memory, etc.

[0083] The storage unit 32 is a means for storing information and is composed of a storage medium such as RAM, a magnetic disk, and a flash memory. The storage unit 32 stores programs executed by the control unit 31 and data used by the programs.

[0084] The communication unit 33 is a wireless communication interface for connecting the terminal device 3 to an external network. The communication unit 33 is configured to be able to communicate with the server device 2 via a wireless LAN, a cellular communication network such as 3G, 4G, or 5G.

[0085] The input / output unit 34 receives input from the operator of the device and presents information to the operator. Specifically, the input / output unit 34 comprises a touch panel and its control unit, and a liquid crystal display and its control unit. In this embodiment, the touch panel and liquid crystal display consist of a single touch panel display.

[0086] [Software composition]

[0087] Next, the software configuration of each device constituting the system will be described. Figure 3A It is a diagram schematically showing the software configuration of the vehicle-mounted device 1 according to the present embodiment.

[0088] In this embodiment, the control unit 11 of the vehicle-mounted device 1 is configured to include three software modules: an imaging unit 111, a notification unit 112, and a providing unit 113. Each software module can also be implemented by the control unit 11 (CPU, etc.) executing a program stored in the storage unit 12. It should be noted that the information processing performed by the software modules has the same meaning as the information processing performed by the control unit 11 (CPU, etc.).

[0089] During operation of the device, the imaging unit 111 captures moving images (vehicle-mounted images) using the camera 15 and stores the obtained data in the storage unit 12. The imaging unit 111 receives power from the vehicle to constantly capture images and stores the obtained moving image data in the storage unit 12.

[0090] Figure 4 This is a diagram illustrating the structure of motion image data generated by the shooting unit 111 and stored in the storage unit 12. The motion image data is composed of a plurality of motion image files. There is an upper limit (for example, 1 minute) to the length of the motion image corresponding to one file. If the upper limit is exceeded, a new motion image file is generated. It should be noted that when the storage capacity is insufficient, the shooting unit 111 will delete the oldest motion image file and continue shooting on the basis of ensuring free capacity. In the present embodiment, each motion image file is associated with an identifier (date and time ID) for identifying the date and time when the motion image file is generated (i.e., the date and time when shooting starts). The date and time ID can also uniquely identify the year, month, day, hour, minute, and second.

[0091] When the imaging unit 111 operates, the vehicle-mounted device 1 functions as a drive recorder.

[0092] The notification unit 112 periodically generates data related to the travel of the vehicle (hereinafter referred to as vehicle data) and transmits it to the server device 2. In this embodiment, the vehicle data includes the location information of the vehicle 10, the direction of travel of the vehicle 10, and a date and time ID indicating the date and time when the location information was acquired. Figure 5(A) shows an example of vehicle data.

[0093] The vehicle data in this embodiment is configured to include an identifier (vehicle ID) that uniquely identifies the vehicle 10, the date and time the vehicle data was generated, an identifier (date and time ID) that uniquely identifies the date and time, location information, and information indicating the direction of travel. The location information and direction of travel can be acquired from the location information acquisition unit 14 described later.

[0094] Vehicle data is generated and transmitted at a predetermined period. This predetermined period can be measured independently, or it can be measured in conjunction with the image capture unit 111. For example, the notification unit 112 may generate vehicle data when the image capture unit 111 generates a new moving image file (e.g., at a one-minute interval). In this case, the date and time indicated in the vehicle data is the same as the date and time the new moving image file was generated.

[0095] The providing unit 113 provides the moving image file captured at a specified time to the terminal device 3 based on a request (moving image file provision request) transmitted from the terminal device 3. For example, when the providing unit 113 receives a provision request including a date and time ID from the terminal device 3, it retrieves the moving image file corresponding to the date and time ID from the storage unit 12 and transmits the file to the terminal device 3.

[0096] Next, the software configuration of the server device 2 will be described. Figure 3B It is a diagram schematically showing the software configuration of the server device 2 according to the present embodiment.

[0097] In this embodiment, the control unit 21 of the server device 2 is configured to include two software modules: a data update unit 211 and an information provision unit 212. Each software module can also be implemented by the control unit 21 (CPU, etc.) executing a program stored in the storage unit 22. It should be noted that the information processing performed by the software module has the same meaning as the information processing performed by the control unit 21 (CPU, etc.).

[0098] The data updating unit 211 receives vehicle data from a plurality of vehicles 10 (vehicle-mounted devices 1) and stores the data in the storage unit 22. For example, the data updating unit 211 may store the vehicle data received from a plurality of vehicles 10 in a storage unit such as Figure 5 In the table shown in (B) (referred to as the vehicle data table).

[0099] The information providing unit 212 provides information to the terminal device 3 based on the information stored in the storage unit 22. Specifically, the information providing unit 212 performs the following two processes.

[0100] (1) Provide information related to the vehicle's driving position

[0101] The vehicle data table stores the location information of vehicles 10 equipped with the vehicle-mounted device 1 functioning as a drive recorder, associated with date and time information. By referring to this location information, it is possible to extract vehicles that have traveled within a specified area within a specified time period.

[0102] When a request is received from the terminal device 3, the information providing unit 212 extracts vehicles that have traveled within a designated area within a designated time period and transmits a combination of location information and date and time information to the terminal device 3. This enables mapping to a map on the terminal device 3. The location information may also include information indicating the direction of travel.

[0103] (2) Relay of vehicle-mounted images

[0104] When a target vehicle for which in-vehicle imagery is requested is selected via terminal device 3, information providing unit 212 receives a request to reproduce the imagery from terminal device 3. The request to reproduce the imagery includes information identifying the target vehicle and the date and time of the reproduction. Based on this information, information providing unit 212 transmits the request to reproduce the imagery to the target vehicle. Furthermore, information providing unit 212 transmits the moving image file (or image stream) of the in-vehicle imagery transmitted from the target vehicle to terminal device 3.

[0105] The storage unit 22 stores a table (vehicle data table) for storing data received from the vehicle-mounted device 1 .

[0106] Next, the software configuration of the terminal device 3 will be described. Figure 3C It is a diagram schematically showing the software configuration of the terminal device 3 according to the present embodiment.

[0107] In this embodiment, the control unit 31 of the terminal device 3 is configured to include two software modules: a display control unit 311 and an image acquisition unit 312. Each software module can also be implemented by the control unit 31 (CPU, etc.) executing a program stored in the storage unit 32. It should be noted that the information processing performed by the software module has the same meaning as the information processing performed by the control unit 31 (CPU, etc.).

[0108] The display control unit 311 generates and outputs a graphical user interface (GUI) for requesting the acquisition of in-vehicle images. In this embodiment, the display control unit 311 generates and outputs a user interface including a road map image based on stored or previously acquired road map data. This user interface includes a user interface component for specifying a target period.

[0109] Furthermore, the display control unit 311 requests the server device 2 for “a list of vehicles traveling within the display range of the map during the target period” based on an operation performed by the operator, and maps icons indicating vehicle positions on the map based on the response.

[0110] For example, if the operator specifies a time period such as "from 5:00 PM today to 5:03 PM," the display control unit 311 requests the server device 2 for a list of vehicles that traveled within the map display area between 5:00 PM and 5:03 PM today. The display control unit 311 then maps icons based on the received vehicle list.

[0111] When the operator selects any of the mapped icons, the image acquisition unit 312 transmits a request to reproduce the onboard image to the corresponding vehicle 10. Furthermore, the image acquisition unit 312 reproduces the moving image file or image stream transmitted from the vehicle 10. It should be noted that in the following description, the reproduction request is transmitted to the vehicle 10 via the server device 2. Furthermore, the moving image file or image stream is received via the server device 2.

[0112] It should be noted that Figures 2A to 2C as well as Figure 3A to Figure 3C The structure shown is an example, and all or part of the functions shown can also be performed using specially designed circuits. In addition, the program can be stored or executed by a combination of a main storage device and an auxiliary storage device other than those shown.

[0113] [Details of the processing]

[0114] Next, operations performed by an operator via the terminal device 3 and processing executed thereby will be described using specific examples.

[0115] Figure 6 This is an example of an initial screen provided to the operator of the terminal device 3.

[0116] The display control unit 311 of the terminal device 3 generates a user interface including a road map centered at a predetermined point and outputs the generated user interface to the input / output unit 34. The user interface includes a component for specifying a time (reference numeral 601), a component for specifying a time range (reference numeral 602), and a button for executing a search (reference numeral 603).

[0117] Reference numeral 601 is a user interface component for specifying a time. The operator uses this user interface component to set, for example, the time at which the 119 notification is generated.

[0118] Reference numeral 602 is a user interface component for specifying a target period for searching for vehicles. For example, if "3 minutes ago" is specified here, then "from 3 minutes before the set time to the set time" becomes the search target period.

[0119] When the operator presses the search button (reference numeral 603), data requesting a list of vehicles that have traveled within the map display range within a set period (e.g., from 16:57 to 17:00) is transmitted from the terminal device 3 to the server device 2. This request includes, for example, information for specifying the map display range and information indicating the target period.

[0120] In response to the request, the server device 2 (information providing unit 212 ) generates a list of corresponding vehicles (vehicle list) and sends the list back to the terminal device 3 . Figure 5 (C) is an example of a returned vehicle list. It should be noted that although this is referred to as a "vehicle list," the records included in the list can also include the same vehicle. For example, the same vehicle may send vehicle data multiple times within a set period within the map display range. In this case, the vehicle list will contain multiple records with the same vehicle ID.

[0121] The terminal device 3 (display control unit 311 ) that has received the vehicle list maps icons representing the vehicles to corresponding positions on the map. Figure 7 This is an example of a map with icons mapped on it. The icons can also be arranged in consideration of the vehicle's travel direction.

[0122] An icon is generated for each record in the vehicle list. That is, the same vehicle may sometimes be displayed using two or more icons. However, since the differences between vehicles are not important to the operator, in this embodiment, information for identifying the vehicle is not displayed on the screen.

[0123] When the operator selects (for example, clicks) any of the icons, an operation for acquiring and reproducing an in-vehicle image from the vehicle corresponding to the icon is started.

[0124] It should be noted that it is possible to immediately start acquiring images from the target vehicle when an icon is specified, but in this embodiment, the following structure is adopted: the date and time corresponding to the selected icon (i.e., the date and time when the corresponding vehicle travels at the target location) is displayed, and the operator who confirms the date and time separately instructs the acquisition of images.

[0125] Figure 8 is Figure 7The example shown is an example of a screen displayed when an operator selects an icon on a map. When an icon is selected, the date and time corresponding to that icon are displayed on the screen (reference numeral 801). The date and time corresponding to the selected icon can be obtained from the vehicle list received from server device 2. This allows the operator to understand the temporal relationship even if multiple icons are located near a desired location (e.g., a notification location).

[0126] The date and time corresponding to an icon is an example of “time information” in the present disclosure. Time information may also include information related to the date.

[0127] The operator who has confirmed the date and time determines whether to obtain the in-vehicle image from the vehicle corresponding to the icon. In the case of obtaining the in-vehicle image from the vehicle corresponding to the icon, for example, the operator presses the button indicating the start of image reproduction (reference numeral 802). It should be noted that in this example, a button indicating the start of image reproduction is provided, but it is also possible to start image reproduction by clicking the corresponding icon again. As long as it is an operation performed by the operator (the "first user operation" in this disclosure), the reproduction of the image can be triggered by any operation. The first user operation includes not only the operation of directly clicking on the icon, but also the operation of indirectly specifying the icon via a label or button associated with the icon.

[0128] When the operator performs the above-described operation, a request to reproduce the in-vehicle image (hereinafter referred to as a reproduction request) is transmitted from the terminal device 3 to the server device 2. The reproduction request includes a vehicle ID for specifying the vehicle and a date and time ID for specifying the date and time. Upon receiving the reproduction request, the server device 2 identifies the target vehicle and transmits the reproduction request to the corresponding in-vehicle device 1.

[0129] Upon receiving the playback request, the onboard device 1 extracts the moving image file corresponding to the specified date and time ID and transmits it to the server 2. It should be noted that the transmitted object can be either a moving image file or a video stream from the onboard video. The transmitted moving image file (or video stream) is relayed to the terminal device 3 via the server 2, where the moving image is reproduced. Figure 9 This is an example of a playback screen in the terminal device 3.

[0130] It should be noted that in this example, motion image files are generated in one-minute increments, and therefore, playback is also performed in one-minute increments. To play back the previous and next motion image files, the operator presses the button (reference numeral 901) indicating forward and backward movement. Pressing this button generates a playback request for the corresponding motion image file and, through the same process, transmits it to the target vehicle-mounted device 1. This allows the operator of the terminal device 3 to play back the vehicle-mounted imagery for the specified date and time.

[0131] [Processing sequence]

[0132] Next, the steps executed by each device and the flow of data sent and received between the devices are described.

[0133] Figure 10 This is a sequence diagram of the process of transmitting vehicle data from the vehicle-mounted device 1 to the server device 2. The illustrated process is executed at a predetermined period. The predetermined period may be determined by the notification unit 112 or by the notification unit 112 in response to a notification from the imaging unit 111. For example, the imaging unit 111 may send a notification to the notification unit 112 every time a moving image file is switched (e.g., every minute), and the notification unit 112 may initiate the illustrated process in response to the notification.

[0134] First, in step S11, the in-vehicle device 1 (notification unit 112) generates vehicle data. In this embodiment, the notification unit 112 obtains the vehicle's location information and travel direction from the location information acquisition unit 14 and combines the vehicle's identifier with date and time information to generate the vehicle data. The generated vehicle data is then transmitted to the server device 2 (step S12).

[0135] In step S13 , the server device 2 (data updating unit 211 ) receives the vehicle data and stores the vehicle data in the vehicle data table in the storage unit 22 .

[0136] By executing the above-described process for multiple vehicles 10, server device 2 can collect and manage the past location information of the multiple vehicles under its management. Furthermore, by referencing the vehicle data table stored in server device 2, it can search for vehicles traveling at a specified location at a specified date and time. By capturing onboard video while driving, it is possible to identify vehicles with a high probability of capturing specific events (e.g., events relevant to reporting, such as traffic accidents) through video.

[0137] Figure 11 This is a sequence diagram of a process for outputting a map on which the position information of the vehicle is mapped by the terminal device 3. The illustrated process is started when a map is displayed on the terminal device 3 (display control unit 311), for example.

[0138] First, in step S21, the terminal device 3 (display control unit 311) determines whether redrawing is necessary. For example, this step results in a positive determination when the map is being drawn for the first time or when an operation is performed to change the map display range. It should be noted that, here, it is assumed that the operator of the terminal device 3 displays the map within a specified range and specifies the target period.

[0139] If a positive determination is made in this step, the terminal device 3 generates a vehicle list generation request and transmits it to the server device 2. The vehicle list generation request includes information indicating the display range of the map and information indicating the target period specified by the operator.

[0140] In step S22, the server device 2 (information providing unit 212) generates a vehicle list based on the received request. Specifically, it extracts records of vehicles that have traveled within a specified range during a specified period from the vehicle data table stored in the storage unit 22, and generates a vehicle list consisting of these extracted records. The generated vehicle list is transmitted to the terminal device 3.

[0141] In step S23 , the terminal device 3 (display control unit 311 ) draws a map based on the acquired vehicle list, and maps icons to corresponding locations on the map.

[0142] When the processing of step S23 is completed, Figure 7 As shown in FIG. 1 , a plurality of icons are mapped on the map in the terminal device 3. The icons indicate the positions of vehicles traveling within the display range of the map within a specified period.

[0143] Figure 12 This is a flowchart of the processing performed by the terminal device 3 (display control unit 311) after the vehicle icon is mapped on the map.

[0144] When the operator of the terminal device 3 performs an operation to select any of the displayed icons, the display control unit 311 accepts the operation (step S31). The operation of selecting an icon performed here is an example of a "second user operation" in the present disclosure.

[0145] In response to the selection operation, the display control unit 311 refers to the vehicle list, obtains the date and time information corresponding to the selected icon, and outputs it (step S32). Figure 8 As shown, date and time information corresponding to the icon and a component for requesting playback of a video are drawn on the screen.

[0146] When the operator of the terminal device 3 performs an operation to confirm the output date and time information, the display control unit 311 accepts the operation. The operation to confirm the date and time information may also be an operation to interact (for example, click) with a specified user interface component after the date and time information is output. Examples of the specified user interface component include icons mapped on a map, labels for date and time information, buttons for requesting the reproduction of an image, and the like. For example, in Figure 8 In the example of FIG, if the areas indicated by reference numerals 801 and 802 are clicked, the date and time information can be considered to have been confirmed. The operation of confirming the date and time information is an example of a "first user operation" in this disclosure. In other words, the operation of confirming the date and time information corresponding to a certain icon has the same meaning as the operation of specifying the icon.

[0147] If the date and time information is confirmed (step S33: Yes), the image acquisition unit 312 generates a playback request (described later). If the date and time information is not confirmed (step S33: No), the process returns to step S31.

[0148] Figure 13 This is a sequence diagram of the process of the vehicle-mounted device 1 providing an image to the terminal device 3 based on a request from the terminal device 3. Figure 12 Step S33 starts.

[0149] First, the terminal device 3 (image acquisition unit 312) generates a playback request (step S41). This playback request is data used to request the vehicle-mounted device 1 to provide the vehicle-mounted image. It contains the identifier (vehicle ID) of the vehicle-mounted device 1 (vehicle 10) and the identifier (date and time ID) of the date and time the vehicle-mounted image was captured. The vehicle ID and date and time ID are included in the vehicle list received by the terminal device 3 from the server device 2. When the operator selects an icon on the map, the terminal device 3 can determine the vehicle ID and date and time ID of the corresponding vehicle.

[0150] The playback request is sent from the terminal device 3 to the server device 2, which then relays it to the target vehicle-mounted device 1. At this point, the server device 2 can also read the vehicle ID included in the playback request to identify the target vehicle-mounted device 1 to which the playback request is to be relayed. Therefore, it is preferable that the server device 2 stores data associating the vehicle ID with the connection destination (e.g., network address) of the corresponding vehicle-mounted device 1. The playback request relayed by the server device 2 is received by the target vehicle-mounted device 1.

[0151] The vehicle-mounted device 1 (providing unit 113) that has received the reproduction request extracts the moving image file specified by the date and time ID from the stored moving image data (step S42). Figure 4 In the example, when the date and time ID T002 is specified, the vehicle-mounted device 1 (providing unit 113) extracts the moving image file recorded starting at 12:01.

[0152] The extracted moving image file is sent to the terminal device 3 via the server device 2. The terminal device 3 generates Figure 9 The playback screen shown in FIG. 1 starts the playback of the received motion picture file. It should be noted that, when an operation of designating another motion picture file is performed (for example, when the button shown by reference numeral 901 is pressed), the playback operation is performed again. Figure 13 The processing shown generates a new reproduction request.

[0153] It should be noted that, in this example, a moving image file is used as the object to be sent, but the object to be sent may also be a video stream or the like.

[0154] As described above, in the monitoring system of this embodiment, the terminal device 3 maps the vehicle's driving performance onto a map based on the vehicle data stored in the server device 2. The operator of the terminal device 3 can specify a vehicle on the displayed map and retrieve a moving image file from that vehicle. This allows the operator of the terminal device 3 to view in-vehicle images captured by a vehicle traveling at a desired location at a desired time.

[0155] If the vehicles extracted based on the specified time width are mapped directly to the map, the operator cannot be informed of the actual time when the vehicle-mounted image was captured. However, the terminal device 3 of this embodiment has a function of outputting date and time information for each icon, allowing the operator to understand the time when the vehicle-mounted image was captured.

[0156] It should be noted that in the first embodiment, when there is an operation of selecting an icon (second user operation), the date and time information is output, and when there is an operation of confirming the date and time information (first user operation), a reproduction request is sent. However, the output of the date and time information can also be performed without being triggered by the operator's operation. For example, it is also possible to overlap and display the date and time information near the icon when the icon is drawn. In this case, the second user operation can be omitted. In this case, a reproduction request can be sent by performing an operation of selecting an icon once (first user operation).

[0157] (Second embodiment)

[0158] In the first embodiment, when the operator clicks a specific icon on the map, the terminal device 3 outputs the date and time information corresponding to the icon. However, with this configuration, the time when the vehicle-mounted image was captured may not be known efficiently.

[0159] Reference Figure 14 Explain this.

[0160] Because the location information managed by the server device 2 is periodically transmitted from multiple vehicles 10, there are sometimes areas where icons tend to be densely packed and areas where they are not. For example, if there is a traffic light on a road, vehicles are more likely to stop in front of the light due to a red light. Therefore, icons tend to be densely packed in the area just before the stop line (reference numeral 1401). Conversely, in the area after the traffic light (reference numeral 1402), the density of icons tends to become sparse.

[0161] Assume that in such a situation, the area (referred to as the area of ​​interest) that the operator wants to acquire an image of is located in the area indicated by reference numeral 1403. In this case, one of the vehicles located in the area 1401 may have an image of the vehicle passing through the area of ​​interest 1403 at a certain time.

[0162] However, even if an icon is selected at random, the obtained vehicle-mounted image may not capture the moment of passing through the focus area 1403. For example, it is also possible that a one-minute image is entirely spent waiting for a traffic light.

[0163] Figure 15 This diagram illustrates the movement of four vehicles corresponding to the icons included in area 1401 in a time series. Here, it is assumed that a traffic light remains red from 12:00 to 12:01 and then turns green at 12:01. If the operator specifies "11:59 to 12:00" as the target period, the four vehicles, Vehicles A through D, that transmitted vehicle data during that period are mapped on the map.

[0164] However, in the first embodiment, it is impossible to determine which of the four vehicles, vehicles A through D, has an image that passes through area of ​​interest 1403. For example, vehicle A remains stationary for one minute. Therefore, if the operator selects vehicle A, area of ​​interest 1403 cannot be confirmed with a single playback request. For example, if the operator selects vehicle A, they must press the "Back One Minute" button and issue another playback request.

[0165] On the other hand, vehicle D departs immediately after transmitting the vehicle data, and therefore the operator who has selected vehicle D can smoothly check the situation of the area of ​​interest 1403 .

[0166] In order to solve this problem, it is sufficient to determine which vehicle among the densely packed icons on the map has the latest transmission time of its vehicle data. Figure 15 In the case of , the vehicle that sends vehicle data the latest is vehicle D.

[0167] In the second embodiment, when a predetermined number of icons or more are included in an area of ​​a predetermined size (e.g., area 1401) that includes icons designated by an operator, a list of time information corresponding to the plurality of icons included in the area is output. With this configuration, the operator, after checking the list, can identify the icon with the latest associated time.

[0168] Figure 16 : is a flowchart of the processing performed by the terminal device 3 in the second embodiment. It should be noted that the dashed line represents the flowchart ( Figure 12 ) The same steps are repeated and detailed descriptions are omitted.

[0169] In the second embodiment, when an icon is selected by the operator, the display control unit 311 determines whether a predetermined number or more of icons are included in a predetermined range (eg, a circular area with a predetermined radius) centered on the icon (step S31A).

[0170] The icon selection operation performed here is an example of a “second user operation” in the present disclosure.

[0171] The predetermined range may be, for example, a range where vehicles are crowded due to waiting at a traffic light, etc. (e.g., a radius of 50 meters). If the determination in step S31A is positive, the process proceeds to step S32A. If the determination in step S31A is negative, the process proceeds to step S32.

[0172] In step S32A, the display control unit 311 generates and outputs a list of date and time information corresponding to icons included in a predetermined range by referring to the vehicle list. Figure 17 This is an example of a screen displaying a list of date and time information. The illustrated example shows a situation where three icons exist within a predetermined range centered on the selected icon. In this case, the date and time information corresponding to each of the three icons is output. If the image playback start button is pressed after selecting any of the three date and time information items in the list, processing proceeds to step S33, where image playback begins using the same method as in the first embodiment.

[0173] The operation of selecting any one of the date and time information included in the list is an example of a “first user operation” in the present disclosure. The first user operation also includes an operation of indirectly specifying a corresponding icon via the listed date and time information.

[0174] It should be noted that, in this step, the content corresponding to the latest time in the content included in the list can also be highlighted. In addition, in this example, an example of a list containing only date and time information is shown, but information for identifying the vehicle can also be added to each element included in the list.

[0175] As described above, in the second embodiment, when a predetermined number or more of icons are clustered together in a predetermined area, a list of date and time information corresponding to these icons is output. This allows the operator to recognize which icon is the latest without selecting each icon one by one, thus enabling smooth image verification.

[0176] (Variation)

[0177] The above-described embodiment is merely an example, and the present disclosure can be implemented with appropriate modifications without departing from the spirit and scope of the present disclosure.

[0178] For example, the processes and solutions described in the present disclosure can be freely combined and implemented as long as no technical contradiction occurs.

[0179] Furthermore, in the embodiment, a vehicle equipped with an in-vehicle device is exemplified, but any moving object other than a vehicle may be used as long as it can capture an image.

[0180] In addition, in the embodiment, the terminal device 3 generates and outputs a user interface including a road map based on the information received from the server device 2, but the road map can also be generated by the server device 2. In this case, the server device 2 can also depict the road map and map the icons instead of the server device 2 generating the vehicle list and sending it to the terminal device 3. In this case, the user interface screen including the road map generated by the server device 2 can also be sent to the terminal device 3 and output by the terminal device 3. That is, the server device 2 (information providing unit 212) can also have the functions of the terminal device 3 (display control unit 311) described in the embodiment. The information processing device of the present disclosure can be implemented as a server device 2 or as a terminal device 3. When the server device 2 depicts the user interface screen, the server device 2 outputs the location information for configuring the icon on the map and the time information corresponding to the icon. It should be noted that the output is not limited to the output to the external device, but also includes the exchange of information between the elements possessed by the server device 2.

[0181] Furthermore, a process described as being performed by a single device may be shared and executed by multiple devices. Alternatively, a process described as being performed by different devices may be performed by a single device. In a computer system, the hardware configuration (server configuration) used to implement each function can be flexibly changed.

[0182] The present disclosure can also be implemented by providing a computer program equipped with the functions described in the above-mentioned embodiments to a computer, and having one or more processors of the computer read out and execute the program. Such a computer program can be provided to the computer via a non-temporary computer-readable storage medium that can be connected to the system bus of the computer, or it can be provided to the computer via a network. Non-temporary computer-readable storage media include, for example, any type of disk such as a magnetic disk (floppy disk (registered trademark), hard disk drive (HDD: Hard Disk Driver)), optical disk (CD-ROM, DVD disk / Blu-ray disk, etc.), read-in dedicated memory (ROM), random access memory (RAM), EPROM (Erasable Programmable read only memory), EEPROM (Electrically Erasable Programmable read only memory), magnetic card, flash memory, optical card, and any type of medium suitable for storing electronic instructions.

Claims

1. An information processing device comprising a control unit, The control unit performs the following actions: Outputting the position information of each vehicle for generating a map based on the travel history of one or more vehicles, The map is a map on which a plurality of icons representing positions of the vehicles at a plurality of times are arranged; outputting time information corresponding to one or more of the plurality of icons; as well as An image request is transmitted to the vehicle or the vehicle-mounted device corresponding to the first icon designated by the first user operation, requesting transmission of an image captured at the corresponding time.

2. The information processing device according to claim 1, wherein The control unit acquires a second user operation of designating an icon on the map, which was performed before the first user operation. The control unit outputs the time information corresponding to the icon designated by the second user operation.

3. The information processing apparatus according to claim 1, wherein The control unit acquires a second user operation of designating an icon on the map, which was performed before the first user operation. When a first area includes a predetermined number or more of icons, the control unit outputs a list of time information corresponding to each of the plurality of icons included in the first area, wherein the first area is an area of ​​a predetermined size including the icon specified by the second user operation and having a predetermined size.

4. The information processing device according to claim 3, wherein: When the first user operation is performed to select any of the time information included in the list, the control unit transmits an image request requesting transmission of an image captured at the corresponding time to the corresponding vehicle or vehicle-mounted device. The information processing apparatus according to claim 3 , wherein: The control unit highlights and displays the latest time information among the time information included in the list.

6. An information processing method, wherein: The information processing device performs the following actions: outputting position information of each vehicle for generating a map based on the travel history of one or more vehicles, wherein the map is a map on which a plurality of icons representing the positions of the vehicles at a plurality of times are arranged; outputting time information corresponding to one or more of the plurality of icons; and An image request is transmitted to the vehicle or the vehicle-mounted device corresponding to the first icon designated by the first user operation, requesting transmission of an image captured at the corresponding time.

7. The information processing method according to claim 6, wherein: The information processing device further performs the following actions: Obtaining a second user operation performed before the first user operation, which specifies an icon on the map; and The time information corresponding to the icon designated by the second user operation is output.

8. The information processing method according to claim 6, wherein: The information processing device further performs the following actions: Obtaining a second user operation performed before the first user operation, which specifies an icon on the map; and When a first area includes a predetermined number or more of icons, a list of time information corresponding to each of the plurality of icons included in the first area is output, wherein the first area is an area of ​​a predetermined size including the icon specified by the second user operation and having a predetermined size.

9. The information processing method according to claim 8, wherein: The information processing device further performs the following actions: When the first user operation is performed to select any of the time information included in the list, an image request is sent to the corresponding vehicle or vehicle-mounted device to request transmission of an image captured at the corresponding time.

10. The information processing method according to claim 8, wherein: The information processing device further performs the following actions: The latest time information among the time information included in the list is highlighted and displayed. 11 . A non-transitory storage medium having recorded thereon a program for causing an information processing device to execute the information processing method according to claim 6 .

Images