Information processing device, information processing method and program
By displaying location icons on a map based on vehicle history and automatically sending image requests in the vehicle monitoring system, the problem of unsmooth image acquisition in the vehicle monitoring system is solved, and the image acquisition efficiency is improved.
Patent Information
- Application Number
- CN202580028214.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2024-04-16
- Filing Date
- 2025-04-09
- Publication Date
- 2026-11-17
AI Technical Summary
In existing vehicle monitoring systems, operators experience excessively long waiting times when acquiring vehicle dashcam footage, leading to difficulties in image confirmation.
The information processing device displays vehicle location icons at multiple times on the map based on the vehicle's driving history, and responds to user instructions by automatically sending image requests in parallel to multiple vehicles or in-vehicle devices within the area set on the map, prioritizing the sending of requests to vehicles that are closer to the specified time or location.
It improves the efficiency of image acquisition, reduces waiting time, and ensures that operators can more smoothly confirm the required images.
Smart Images

Figure CN123125632A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to vehicle technology. Background Technology
[0002] There is a technology for surveillance using a mobile device equipped with a camera. Relatedly, for example, Patent Document 1 discloses a system that manages the history of a vehicle's driving location via a server device to identify vehicles equipped with dashcams located near the notifier.
[0003] Existing technical documents Patent documents Patent Document 1: Japanese Patent Application Publication No. 2023-019792 Summary of the Invention
[0004] The problem that the invention aims to solve The purpose of this disclosure is to improve the convenience of monitoring systems that use vehicles.
[0005] Solution for solving the problem One embodiment of this disclosure is an information processing apparatus having a control unit that performs the following processing: displaying multiple icons representing the positions of each vehicle at multiple times on a map based on the driving history of one or more vehicles; and, in response to receiving an instruction from a user, sending image requests to two or more vehicles or vehicle-mounted devices corresponding to two or more icons set in a first area on the map, requesting the transmission of images captured at corresponding times.
[0006] One embodiment of this disclosure is an information processing method executed by a computer, wherein the information processing method includes: a first step of displaying multiple icons representing the positions of each vehicle at multiple times on a map based on the driving history of one or more vehicles; and a second step of sending image requests, in response to an instruction received from a user, to two or more vehicles or vehicle-mounted devices corresponding to two or more icons set in a first area on the map, requesting the transmission of images captured at corresponding times.
[0007] In addition, as another option, a program for causing a computer to perform the above-described methods or a computer-readable storage medium that non-temporarily stores the program is provided.
[0008] Invention Effects According to this disclosure, convenience can be improved in monitoring systems that use vehicles. Attached Figure Description
[0009] Figure 1This is a schematic diagram of the monitoring system according to the first embodiment.
[0010] Figure 2A This is a hardware configuration diagram of the vehicle-mounted device according to the implementation method.
[0011] Figure 2B This is a hardware configuration diagram of the server device in the implementation method.
[0012] Figure 2C This is a hardware configuration diagram of the terminal device in the implementation method.
[0013] Figure 3A This is a software configuration diagram of the vehicle-mounted device according to the implementation method.
[0014] Figure 3B This is a software configuration diagram of the server device in the implementation method.
[0015] Figure 3C This is a software configuration diagram of the terminal device in the implementation method.
[0016] Figure 4 This is a diagram of motion image data acquired through an onboard device.
[0017] Figure 5 This is an example of vehicle data, a vehicle data table, or a vehicle list.
[0018] Figure 6 This is an example of a screen displayed on a terminal device in the first embodiment.
[0019] Figure 7 This is an example of a screen displayed on a terminal device in the first embodiment.
[0020] Figure 8 This is an example of a screen displayed on a terminal device in the first embodiment.
[0021] Figure 9 This is an example of a video reproduced from an in-vehicle camera.
[0022] Figure 10 This is an example of a screen that specifies all icons included within a given range.
[0023] Figure 11 It is a diagram used to illustrate icons contained within a specified range.
[0024] Figure 12 This diagram illustrates the order in which requests are issued in a combined specified manner.
[0025] Figure 13 This diagram illustrates the order in which requests are issued in a combined specified manner.
[0026] Figure 14 This is a sequence diagram of the processing of vehicle data collected by the server device.
[0027] Figure 15 This is a sequence diagram of the processing of the vehicle list provided by the server device.
[0028] Figure 16 This is a sequence diagram of the processing for reproducing in-vehicle images.
[0029] Figure 17 This is a flowchart of the process for processing a playback request issued by a terminal device. Detailed Implementation
[0030] In recent years, the connectivity of automobiles has been continuously advancing, and the number of vehicles with communication capabilities is constantly increasing. In addition, the number of vehicles equipped with in-vehicle cameras is also increasing.
[0031] Relatedly, by enabling access to images captured by cameras (such as dashcams) mounted on vehicles from designated agencies, it can aid in the initial handling of incidents and accidents.
[0032] For example, in the event of a report from 110 or 119, by obtaining dashcam footage from vehicles that were driving near the relevant location before and after the incident, the relevant authorities can confirm the situation at the scene as early as possible.
[0033] As a system for this purpose, there is a known system that maps icons representing the driving positions of multiple vehicles at a specified time in the past onto a map and presents them to an operator.
[0034] The system, for example, acquires the location information of vehicles that traveled within a specified area before or after a time specified by the operator (e.g., five minutes before the notification time) and maps this location information onto a map. The operator selects vehicles on the map that traveled near a desired location (e.g., the notification location) at any time close to the desired time (e.g., the notification time). Furthermore, the system requests the vehicle to transmit dashcam footage captured at the corresponding time. Thus, the operator can view and listen to dashcam footage that may have captured the notification scene.
[0035] In such a system, it is assumed that the operator clicks on multiple adjacent icons sequentially until the desired image is obtained. However, there is a time lag between requesting dashcam footage from the vehicle and actually being able to view the footage. Therefore, in such a situation, the following problem may occur: intermittent waiting time may be generated, making it impossible for the operator to smoothly review the footage.
[0036] The information processing apparatus described in this disclosure will solve such problems.
[0037] One aspect of the information processing apparatus disclosed herein includes a control unit that performs the following processing: displaying multiple icons representing the positions of each vehicle at multiple times on a map based on the driving history of one or more vehicles; and, in response to an instruction received from a user, sending image requests to two or more vehicles or vehicle-mounted devices corresponding to two or more icons contained in a first area on the map, requesting the transmission of images captured at the corresponding times.
[0038] The control unit displays a map, for example, based on the vehicle's driving history. This map maps the locations of vehicles that have driven within a defined area over a specified time span, using icons. The map can be displayed via a terminal connected to an information processing device. Thus, for example, it is possible to request imagery from the vehicle corresponding to the designated icon and the onboard device installed in that vehicle.
[0039] Alternatively, for example, if any one of multiple icons is selected, the control unit sends the image request to the corresponding vehicle or the on-board unit mounted on the vehicle.
[0040] In addition, to address the aforementioned issues, in the information processing apparatus of this disclosure, in response to receiving an instruction from a user, image requests are automatically sent to one or more vehicles or in-vehicle devices corresponding to the multiple icons contained in the first area.
[0041] The first region refers to an area of a predetermined size defined on a map. The first region may be automatically positioned near the center of the map, or it may be configured based on operator input. Alternatively, the aforementioned instruction may be given by the user selecting an area within the first region. For example, when an operator selects an area within the first region, an image request is issued to one or more vehicles or in-vehicle devices corresponding to the icons located within that region. The control unit may also issue image requests to multiple vehicles or in-vehicle devices in parallel.
[0042] Furthermore, the timing of image request issuance can be controlled to prevent sending more than two image requests to the same vehicle at once. This is because sending multiple requests to a single vehicle simultaneously could lead to unexpected actions such as the cancellation of the first request or the rejection of subsequent requests.
[0043] Alternatively, upon receiving the instruction, the control unit may send the image request more preferentially to the vehicle or on-board device corresponding to the icon of the time closer to the specified time.
[0044] The specified time can be, for example, a time specified by the operator.
[0045] If there are multiple icons in the first area, image requests can be sent sequentially starting from the icon closest to the specified time.
[0046] Based on this configuration, image requests can be prioritized and sent to vehicles that are more likely to capture the object desired by the operator.
[0047] Alternatively, if a first region is defined on the map by the instruction, the control unit may send the image request more preferentially to the vehicle or vehicle-mounted device corresponding to the icon that is closer to a specified location within the first region.
[0048] If there are multiple icons in the first area, image requests can be sent sequentially starting from the icon closest to the center of the first area.
[0049] Based on this configuration, image requests can be prioritized and sent to vehicles that are more likely to capture the object desired by the operator.
[0050] Alternatively, the control unit may change the appearance of the icon corresponding to the vehicle or in-vehicle device that received the image request.
[0051] Alternatively, if the user is able to view or listen to the image after the image request is sent, the control unit may change the appearance of the icon corresponding to the vehicle or in-vehicle device.
[0052] Alternatively, the control unit may change the appearance of the icon corresponding to the vehicle or vehicle-mounted device when there is a notification from the vehicle or vehicle-mounted device that the image transmission preparation has been completed.
[0053] According to this configuration, the operator can identify vehicles that have received image requests and vehicles that have not received image requests, or vehicles that can perform image viewing and vehicles that cannot perform image viewing and audiovisual activities.
[0054] Alternatively, the control unit may output time information corresponding to one or more of the plurality of icons.
[0055] Based on this configuration, the operator can know the time when the corresponding vehicle passed through the location on the map. That is, a more preferred vehicle can be selected as the destination vehicle for the requested image.
[0056] Alternatively, the control unit may output a list of time information corresponding to the plurality of icons contained in the first region.
[0057] In areas where vehicles tend to congregate, such as near stop lines at intersections, icons tend to be densely packed on the map. Therefore, the time information corresponding to the icons in the first area can be listed and output. This allows the operator to determine which icon represents the most up-to-date time.
[0058] Hereinafter, specific embodiments of the present disclosure will be described based on the accompanying drawings. Unless otherwise specified, the hardware configuration, module configuration, functional configuration, etc., described in each embodiment are not intended to limit the scope of the disclosed technology.
[0059] (First Implementation) [System Overview] Reference Figure 1 An overview of the monitoring system according to the first embodiment will be described. The monitoring system of this embodiment is configured to include multiple vehicles 10, a server device 2, and a terminal device 3. The vehicles 10 are networked vehicles that can access wireless communication networks. The vehicles 10 can communicate with the server device 2 via a wireless communication network (e.g., a cellular communication network).
[0060] Vehicle 10 is equipped with an on-board unit 1. The on-board unit 1 functions as a driving recorder and a wireless communication device. The on-board unit 1 has the following functions: acquiring and storing dynamic images (hereinafter referred to as "images" or "on-board images") of the area in front of the vehicle via an on-board camera; and providing the stored images to an external device (e.g., server device 2) in response to a request. Furthermore, vehicle 10 has the function of periodically sending location information to server device 2.
[0061] It should be noted that in this embodiment, the vehicle-mounted device 1 is assumed to have wireless communication functionality, but the vehicle 10 may also be configured to have a data communication module (DCM) for connecting the vehicle's components to a network. That is, the vehicle-mounted device and the wireless communication device can also be separate.
[0062] Server device 2 is configured to communicate with multiple vehicles 10 via a network. Server device 2 periodically obtains location information from the multiple vehicles 10 under its management and stores the location information in a database. By referring to the database managed by server device 2, the locations and times in which the multiple vehicles 10 under the system's management have traveled in the past can be determined.
[0063] The terminal device 3 determines the vehicle 10 that is presumed to have the desired in-vehicle image based on the information obtained from the server device 2, and obtains the in-vehicle image from the vehicle 10.
[0064] Specifically, terminal device 3 queries server device 2 to obtain a list of vehicles that have traveled within a specified range at a specified time. Furthermore, terminal device 3 maps the corresponding vehicles onto a map, allowing the operator to select a vehicle. Additionally, terminal device 3 requests the transmission of in-vehicle images from the selected vehicle 10. Thus, the operator of terminal device 3 can confirm the images captured by the passing vehicle.
[0065] Terminal device 3 is configured in an organization that utilizes images captured by vehicle 10. Examples of such organizations include police agencies and fire departments. For instance, in the case of a fire department using terminal device 3, triggered by a 119 emergency call, the operator identifies vehicle 10 that was traveling near the scene of the emergency call before or after the call time and acquires in-vehicle images from that vehicle 10.
[0066] [Hardware Configuration] Next, the hardware configuration of each device constituting the system will be explained.
[0067] First, the constituent elements of vehicle 10 will be explained. Figure 2A This is a diagram schematically illustrating an example of the hardware configuration of an on-board device 1 mounted on a vehicle 10.
[0068] The vehicle-mounted device 1 can be configured as a computer with a processor (CPU (Central Processing Unit), GPU (Graphics Processing Unit), etc.), main storage (RAM (Random Access Memory), ROM (Read Only Memory), etc.), and auxiliary storage (EPROM (Erasable Programmable Read Only Memory), hard disk drive, removable media, etc.). The auxiliary storage device stores the operating system (OS), various programs, various tables, etc. By executing the programs stored in the auxiliary storage device, the various functions (software modules) that meet the specified purpose, as described later, can be achieved. However, some or all of the functions can also be implemented as hardware modules using hardware circuits such as ASIC (Application Specific Integrated Circuit) and FPGA (Field Programmable Gate Array).
[0069] The vehicle-mounted device 1 is configured to include a control unit 11, a storage unit 12, a communication unit 13, a location information acquisition unit 14, and a camera 15.
[0070] The control unit 11 is a computing unit that executes a prescribed program to realize various functions of the vehicle-mounted device 1. The control unit 11 can be implemented, for example, by a hardware processor such as a CPU. In addition, the control unit 11 may also be configured to include RAM, ROM (read-only memory), cache memory, etc.
[0071] The storage unit 12 is a unit for storing information and is composed of storage media such as RAM, disk, and flash memory. The storage unit 12 stores programs executed by the control unit 11 and the data used by those programs.
[0072] The communication unit 13 is a wireless communication interface for connecting the vehicle-mounted device 1 to an external network. The communication unit 13 is configured, for example, to communicate with the server device 2 via a cellular communication network such as a wireless LAN (Local Area Network), 3G (3rd Generation), 4G (4th Generation), or 5G (5th Generation).
[0073] The location information acquisition unit 14 acquires the location information of the vehicle 10. The location information acquisition unit 14 includes a GPS (Global Positioning System) antenna and a positioning module for measuring the location information. The GPS antenna is an antenna that receives positioning signals transmitted from positioning satellites (also known as GNSS (Global Navigation Satellite System) satellites). The positioning module is a module that calculates the location information based on the signals received through the GPS antenna. It should be noted that the location information acquisition unit 14 can also determine the direction of travel of the vehicle 10 based on the shift of the location information.
[0074] Camera 15 is an optical unit that includes an image sensor for acquiring images. Camera 15 is mounted, for example, facing forward of vehicle 10.
[0075] Next, server device 2 will be described. Figure 2B This is a diagram that schematically illustrates an example of the hardware configuration of server device 2.
[0076] Server device 2, like vehicle-mounted device 1, can be configured as a computer with a processor (CPU, GPU, etc.), main storage device (RAM, ROM, etc.), and auxiliary storage device (EPROM, hard disk drive, removable media, etc.).
[0077] The server device 2 is configured to include a control unit 21, a storage unit 22, and a communication unit 23.
[0078] The control unit 21 is a computing unit that executes prescribed programs to implement various functions of the server device 2. The control unit 21 can be implemented, for example, by a hardware processor such as a CPU. Furthermore, the control unit 21 may also be configured to include RAM, ROM (read-only memory), cache memory, etc.
[0079] The storage unit 22 is a unit for storing information and is composed of storage media such as RAM, disk, and flash memory. The storage unit 22 stores programs executed by the control unit 21 and the data used by those programs.
[0080] The communication unit 23 is a communication interface for connecting the server device 2 to a network. The communication unit 23 is configured, for example, to communicate with the network via Ethernet (registered trademark), wireless LAN, cellular communication network, etc.
[0081] Next, terminal device 3 will be described. Figure 2C This is a diagram that schematically illustrates an example of the hardware configuration of terminal device 3.
[0082] Terminal device 3, like vehicle-mounted device 1 and server device 2, can be configured as a computer with a processor (CPU, GPU, etc.), main storage device (RAM, ROM, etc.), and auxiliary storage device (EPROM, hard disk drive, removable media, etc.).
[0083] The terminal device 3 is configured to include a control unit 31, a storage unit 32, a communication unit 33, and an input / output unit 34.
[0084] The control unit 31 is a computing unit that executes a predetermined program to implement various functions of the terminal device 3. The control unit 31 can be implemented, for example, by a hardware processor such as a CPU. Furthermore, the control unit 31 may also be configured to include RAM, ROM (read-only memory), cache memory, etc.
[0085] The storage unit 32 is a unit for storing information and is composed of storage media such as RAM, disk, and flash memory. The storage unit 32 stores programs executed by the control unit 31 and data used by those programs.
[0086] The communication unit 33 is a wireless communication interface for connecting the terminal device 3 to an external network. The communication unit 33 is configured, for example, to communicate with the server device 2 via a cellular communication network such as a wireless LAN, 3G, 4G, or 5G.
[0087] The input / output unit 34 is a unit that receives input from the operator of the device and presents information to the operator. Specifically, the input / output unit 34 consists of a touch panel and its control unit, and a liquid crystal display and its control unit. In this embodiment, the touch panel and the liquid crystal display are combined into a single touch panel display.
[0088] [Software Composition] Next, the software configuration of each device constituting the system will be explained. Figure 3A This diagram schematically illustrates the software configuration of the vehicle-mounted device 1 according to this embodiment.
[0089] In this embodiment, the control unit 11 of the vehicle-mounted device 1 is configured to have three software modules: a camera unit 111, a notification unit 112, and a provision unit 113. Each software module can be implemented by the control unit 11 (CPU, etc.) executing a program stored in the storage unit 12. It should be noted that the information processing performed by the software modules has the same meaning as the information processing performed by the control unit 11 (CPU, etc.).
[0090] The imaging unit 111 uses the camera 15 to capture moving images (vehicle-mounted images) during the operation of the device and saves the acquired data in the storage unit 12. The imaging unit 111 receives power from the vehicle to continuously capture images and records the acquired moving image data in the storage unit 12.
[0091] Figure 4 This diagram illustrates the structure of motion picture data generated by the imaging unit 111 and stored in the storage unit 12. The motion picture data consists of multiple motion picture files. There is an upper limit to the length of the motion picture corresponding to a file (e.g., 1 minute); when this limit is exceeded, a new motion picture file is generated. It should be noted that in the event of insufficient storage capacity, the imaging unit 111 continues imaging while deleting the oldest motion picture file to ensure available space. In this embodiment, each motion picture file is associated with an identifier (date and time ID) used to identify the date and time of motion picture file generation (i.e., the date and time when imaging began). The date and time ID can be a unique identifier for year, month, day, hour, minute, and second.
[0092] The camera unit 111 operates, and the vehicle-mounted device 1 functions as a dashcam.
[0093] The notification unit 112 periodically generates data related to the driving of the vehicle (hereinafter referred to as vehicle data) and sends the data to the server device 2. In this embodiment, the vehicle data includes the location information of the vehicle 10, the driving direction of the vehicle 10, and a date and time ID indicating the date and time when the location information was obtained. Figure 5An example of vehicle data is shown in (A).
[0094] In this embodiment, the vehicle data is configured to include a unique identifier (vehicle ID) representing vehicle 10, the date and time the vehicle data was generated, a unique identifier (date and time ID) representing that date and time, location information, and information indicating the direction of travel. The location information and direction of travel can be obtained from the location information acquisition unit 14, which will be described later.
[0095] Vehicle data is generated and transmitted at a predetermined period. This predetermined period can be measured independently, or it can be linked to the imaging unit 111. For example, it can be set to notify the unit 112 to generate vehicle data at a timer (e.g., a 1-minute period) when the imaging unit 111 generates a new moving image file. In this case, the date and time represented by the vehicle data are the same as the date and time of the newly generated moving image file.
[0096] The providing unit 113 provides a motion image file captured at a specified time to the terminal device 3 based on a request (motion image file provision request) sent from the terminal device 3. For example, if a provision request containing a date and time ID is received from the terminal device 3, the providing unit 113 retrieves the motion image file corresponding to the date and time ID from the storage unit 22 and sends the motion image file to the terminal device 3.
[0097] Next, the software configuration of server device 2 will be described. Figure 3B This diagram schematically illustrates the software configuration of the server device 2 in this embodiment.
[0098] In this embodiment, the control unit 21 of the server device 2 is configured to have two software modules: a data update unit 211 and an information provision unit 212. Each software module can be implemented by the control unit 11 (CPU, etc.) executing a program stored in the storage unit 22. It should be noted that the information processing performed by the software modules has the same meaning as the information processing performed by the control unit 21 (CPU, etc.).
[0099] The data update unit 211 receives vehicle data from multiple vehicles 10 (vehicle-mounted devices 1) and stores the vehicle data in the storage unit 22. For example, the data update unit 211 can store the vehicle data received from the multiple vehicles 10 in a storage device such as... Figure 5 The table shown in (B) is called the vehicle data table.
[0100] The information providing unit 212 provides information to the terminal device 3 based on the information stored in the storage unit 22. Specifically, the information providing unit 212 performs the following two processes.
[0101] (1) Provision of information related to the vehicle's location In the vehicle data table, the location information of the vehicle 10 equipped with an on-board device 1 that functions as a dashcam is stored in a state that is associated with date and time information. By referring to this vehicle data table, vehicles that have driven within a specified area over a specified time span can be extracted.
[0102] Upon a request from terminal device 3, information providing unit 212 retrieves vehicles that have traveled within a specified area over a specified time span and sends the location information along with a pair of date and time information to terminal device 3. This enables mapping to a map on terminal device 3. The location information may also include information indicating the direction of travel.
[0103] (2) Relay of vehicle-mounted images When a vehicle for which an in-vehicle image is requested is selected via terminal device 3, information providing unit 212 receives an image playback request from terminal device 3. The image playback request includes information identifying the target vehicle and the date and time of playback. Based on this information, information providing unit 212 transmits the image playback request to the target vehicle. Furthermore, information providing unit 212 transmits the motion image file (or video stream) of the in-vehicle image sent from the target vehicle to terminal device 3.
[0104] The storage unit 22 stores a table (vehicle data table) that stores data received from the vehicle-mounted device 1.
[0105] Next, the software configuration of terminal device 3 will be explained. Figure 3C This diagram schematically illustrates the software configuration of the terminal device 3 in this embodiment.
[0106] In this embodiment, the control unit 31 of the terminal device 3 is configured to have two software modules: a display control unit 311 and an image acquisition unit 312. Each software module can be implemented by the control unit 11 (CPU, etc.) executing a program stored in the storage unit 32. It should be noted that the information processing performed by the software modules has the same meaning as the information processing performed by the control unit 31 (CPU, etc.).
[0107] The display control unit 311 generates and outputs a user interface (GUI) for requesting the acquisition of vehicle-mounted images. In this embodiment, the display control unit 311 generates and outputs a user interface containing road map images based on stored or previously acquired road map data. This user interface includes user interface components for specifying objects.
[0108] Furthermore, the display control unit 311 requests a "list of vehicles that have traveled within the display range of the map during the target period" from the server device 2 based on the operation performed by the operator, and maps icons representing the locations of the vehicles onto the map based on the response.
[0109] For example, if the operator specifies a period such as "from 17:00 to 17:03 today", the display control unit 311 requests the server device 2 "a list of vehicles that traveled within the map's display area from 17:00 to 17:03 today". Furthermore, the display control unit 311 maps icons based on the received list of vehicles.
[0110] The image acquisition unit 312 can send a request to the vehicle 10 to acquire in-vehicle images in two ways.
[0111] The first method is as follows: when the operator specifies one of the multiple mapped icons, a request to reproduce the in-vehicle image is sent to the vehicle 10 corresponding to that icon. This method will henceforth be referred to as the individual specification method.
[0112] The second method is as follows: when the operator selects a specified range on the map, icons located within that range are also specified. In this case, playback requests for the vehicle-mounted images are sequentially sent to multiple vehicles 10 corresponding to the icons within that range. Hereafter, this method will be referred to as the simultaneous specification method. A detailed description of the method will follow.
[0113] In either case, the image acquisition unit 312 receives and reproduces motion image files or video streams sent from the vehicle 10. It should be noted that when motion image files or video streams are sent simultaneously from multiple vehicles 10, the image acquisition unit 312 can receive multiple motion image files or video streams in parallel and selectively reproduce these multiple motion image files or video streams based on the operator's operation.
[0114] It should be noted that, in the following description, the reproduction request is assumed to be sent to the vehicle 10 via server device 2. Furthermore, the motion picture file or video stream is assumed to be received via server device 2.
[0115] It should be noted that, Figures 2A to 2C as well as Figures 3A-3C The configuration shown is an example; all or part of the functions illustrated can also be executed using specially designed circuitry. Furthermore, programs can be stored or executed using a combination of main and auxiliary storage devices, other than those shown in the diagram.
[0116] [Processing Details] Next, specific examples are given to illustrate the operations performed by the operator via terminal device 3 and the processes executed therefrom.
[0117] Figure 6 This is an example of the initial screen provided to the operator of terminal device 3.
[0118] The display control unit 311 of the terminal device 3 generates a user interface containing a road map centered on a specified location, and outputs the user interface to the input / output unit 34. The user interface includes a component for specifying a time (reference numeral 601), a component for specifying a time range (reference numeral 602), and a button for performing a search (reference numeral 603).
[0119] Reference numeral 601 in the attached figure is a user interface component used to specify a time. The operator uses this user interface component to, for example, set the time for a notification of 119.
[0120] Reference numeral 602 is a user interface component used to specify the period for searching for vehicles. For example, when "3 minutes ago" is specified here, "from 3 minutes ago to the set time" becomes the period for searching.
[0121] When the operator presses the search button (reference numeral 603), the terminal device 3 sends a request to the server device 2 for a list of vehicles that have traveled within the map's display area during a set period (e.g., from 16:57 to 17:00). This request may include, for example, information for determining the map's display area and information indicating the period in question.
[0122] In response to the request, server device 2 (information providing unit 212) generates a corresponding list of vehicles (vehicle list) and replies the list of vehicles to terminal device 3. Figure 5 (C) is an example of a list of vehicles that has been responded to. It should be noted that while referred to here as a "vehicle list," the records contained in the list may also contain the same vehicles. For example, sometimes the same vehicle sends vehicle data multiple times within a set period and within the map's display area. In this case, multiple records with the same vehicle ID are included in the vehicle list.
[0123] The terminal device 3 (display control unit 311) that receives the vehicle list maps the vehicle icons to the corresponding locations on the map. Figure 7 This is an example of a map with icons. The icons can be configured to take into account the direction of vehicle travel.
[0124] The icons are generated for each record in the vehicle list. That is, the same vehicle may sometimes be displayed with more than one icon. However, the differences between the vehicles are not important to the operator, so in this embodiment, the information used for vehicle identification is not displayed on the screen.
[0125] Terminal device 3 can select an icon and request the transmission of in-vehicle images from vehicle 10 by either the aforementioned individual or combined designation method.
[0126] First, the actions in cases where a specific method is specified will be explained.
[0127] When the operator selects (e.g., clicks) any of the icons mapped on the map, the action of acquiring and reproducing the in-vehicle image from the vehicle corresponding to that icon begins.
[0128] Figure 8 Is Figure 7 The example shown is one of the screens displayed when the operator selects an icon on the map. When an icon is selected, the date and time corresponding to that icon (reference numeral 801) are output to the screen. The date and time corresponding to the selected icon can be obtained from the vehicle list received from server device 2. Thus, even if there are multiple icons near the desired location (e.g., the reporting location), the operator can grasp the temporal relationship.
[0129] The operator, having confirmed the date and time, determines whether to acquire the in-vehicle image from the vehicle corresponding to the icon. If the operator wishes to acquire the in-vehicle image from the vehicle corresponding to the icon, they will press the corresponding icon again.
[0130] When the operator performs the aforementioned operation, a request for reproducing the vehicle-mounted image (hereinafter referred to as the reproduction request) is sent from the terminal device 3 to the server device 2. The reproduction request includes a vehicle ID for specifying the vehicle and a date and time ID for specifying the date and time. Upon receiving the reproduction request, the server device 2 determines the target vehicle and transmits the reproduction request to the corresponding vehicle-mounted device 1.
[0131] Upon receiving a playback request, the vehicle-mounted device 1 extracts a motion image file corresponding to the specified date and time ID and sends the motion image file to the server device 2. It should be noted that the object sent can be a motion image file from the vehicle-mounted video feed or a video stream. The vehicle-mounted device 1 can extract a motion image file matching the specified date and time ID; if no motion image file matching the specified date and time ID is available, it can also extract a motion image file captured at the most recent moment.
[0132] The sent motion picture file (or video stream) is relayed to the terminal device 3 via the server device 2, where the motion picture is reproduced. Figure 9 This is an example of a reproduced image in terminal device 3.
[0133] It should be noted that in this example, since the motion picture file is generated in 1-minute increments, the playback is also in 1-minute increments. To playback the motion picture file before and after the specified time, the operator presses the button indicating forward and backward movement (reference numeral 901). When this button is pressed, a playback request for the corresponding motion picture file is generated and sent to the vehicle-mounted device 1 through the same processing. Thus, the operator of the terminal device 3 can playback the vehicle-mounted video for the specified date and time.
[0134] Next, the operation in the case of simultaneous specification will be explained. When terminal device 3 is operating in simultaneous specification mode, a region for simultaneous specification icons (hereinafter referred to as the specification region) is graphically drawn in a designated area on the map. The specification region is an example of a "first region".
[0135] Figure 10 This is an example of a screen displayed when terminal device 3 is operating in a specified manner. In this embodiment, a graphic 1001 representing a specified area is drawn at the center of the map. It should be noted that in this example, the specified area is set automatically at the center of the map, but the specified area may also be set on the map based on operator input. Alternatively, if the map's display range or scale is changed, the position and size of the specified area may also be reset simultaneously.
[0136] When an operator selects (for example, clicks) an icon located within a specified area or any location within a specified area, a request to reproduce the image is automatically sent to the vehicle corresponding to the icon located within that specified area.
[0137] It should be noted that in this example, the playback request is triggered by the operator selecting (e.g., clicking) the interior of a designated area, but the playback request can also be triggered by other actions. For example, the playback request can also be triggered by clicking an interface (button, etc.) as shown by reference numeral 1002 in the attached figure, or the playback request can be issued by voice input or other means without going through the screen.
[0138] Next, the order in which reproduction requests are issued when there are multiple icons within a specified area will be explained. In this embodiment, methods using time and methods using distance are illustrated as examples of methods for determining the order in which reproduction requests are issued.
[0139] (1) Method of using time This method involves sending reproduction requests sequentially, starting with the icon whose association time is closer to the time specified by the operator.
[0140] Figure 11 This is an example of a list of vehicles corresponding to icons located within a specified area. In this example, three vehicles form five icons. Furthermore, the operator-specified time is set to 12:00, and vehicle data from 5 minutes ago (11:55) up to the specified time is retrieved.
[0141] In this example, when the records are sorted in order from closest to furthest from the specified time, the sequence becomes icon A, icon B, icon E, icon C, and icon D. Therefore, when the specified area is selected, the terminal device 3 issues a playback request in this order.
[0142] Figure 12 This diagram illustrates the sequence in which terminal device 3 issues playback requests in this example. As shown, terminal device 3 issues playback requests to the corresponding vehicles in order of proximity to the time specified by the operator, from closest to furthest. Furthermore, terminal device 3 sequentially receives motion picture files sent in response to the playback requests.
[0143] (2) Using distance method This is a method that sends reproduction requests sequentially starting from the icon whose location on the map is closer to the center of the specified area.
[0144] exist Figure 11 In the example case, when the records are sorted in order from closest to furthest from the center point of the designated area, the order becomes icon A, icon D, icon B, icon C, icon E. Therefore, when the designated area is selected, the terminal device 3 issues a reproduction request in this order.
[0145] Figure 13 This diagram illustrates the sequence in which terminal device 3 issues playback requests in this example. As shown, terminal device 3 issues playback requests to the corresponding vehicles in order of distance from the center point of the designated area, from closest to furthest. Furthermore, terminal device 3 sequentially receives motion picture files sent in response to the playback requests. The distance from the center point of the designated area can be determined based on the location information contained in the vehicle list.
[0146] It should be noted that in this example, the distance from the center point of the specified area is used, but as long as it is within the specified area, the distance from any other point can also be used.
[0147] It should be noted that icons A and B, and icons D and E, refer to the same vehicle. In this case, terminal device 3 issues a next playback request based on the receipt of a motion image file in response to the playback request. This prevents playback requests from being sent repeatedly to the same vehicle.
[0148] When the reception of the motion image file is complete, the corresponding icon can be used to reproduce the in-vehicle image (audiovisual). Alternatively, the terminal device 3 can be configured to change the appearance (color, etc.) of the icon indicating that the reception of the motion image file has been completed. Furthermore, the terminal device 3 can also respond to the operator clicking on the icon whose appearance has changed to start reproducing the in-vehicle image.
[0149] It should be noted that, in order to provide information to the operator, terminal device 3 can also target each of the multiple icons contained in a designated area, such as... Figure 8 The terminal device 3 can output the corresponding date and time as shown in the example. That is, it can also output a list of multiple dates and times. Therefore, even when there are multiple icons within a specified area, the operator can grasp the temporal relationship between them.
[0150] [Processing Sequence] Next, the steps performed by each device and the flow of data transmitted and received between the devices will be explained.
[0151] Figure 14 This is a sequence diagram of the process of sending vehicle data from the vehicle-mounted device 1 to the server device 2. The illustrated process is executed at a predetermined period. The predetermined period can be determined by the notification unit 112, or it can be determined by the notification unit 112 in response to a notification from the imaging unit 111. For example, it is also possible that whenever a switching of a moving image file occurs (e.g., every minute), a notification is sent from the imaging unit 111 to the notification unit 112, and in response to this, the notification unit 112 starts the illustrated process.
[0152] First, in step S11, the vehicle-mounted device 1 (notification unit 112) generates vehicle data. In this embodiment, the notification unit 112 obtains the vehicle's location information and direction of travel from the location information acquisition unit 14, and combines the vehicle's identifier with date and time information to generate vehicle data. The generated vehicle data is then sent to the server device 2 (step S12).
[0153] In step S13, the server device 2 (data update unit 211) receives the vehicle data and stores the vehicle data in the vehicle data table in the storage unit 22.
[0154] By performing the processing described above on multiple vehicles 10, server device 2 can collect and manage the past location information of multiple vehicles under its management. Furthermore, by referring to the vehicle data table stored in server device 2, vehicles that traveled at a specified location on a specified date and time can be retrieved. Since each vehicle is capturing onboard images while driving, it is possible to identify vehicles with a high probability of capturing specific phenomena (e.g., traffic accidents or other phenomena related to reporting) through the images.
[0155] Figure 15 This is a sequence diagram of the process of the terminal device 3 outputting a map mapped with the vehicle's location information. The process illustrated begins, for example, when the map is displayed in the terminal device 3 (display control unit 311).
[0156] First, in step S21, the terminal device 3 (display control unit 311) determines whether a redrawing is necessary. For example, if the map is being drawn for the first time or if an operation to change the map's display area has been performed, this step becomes an affirmative determination. It should be noted that, here, it is assumed that the operator of the terminal device 3 displays the specified area of the map and specifies the target period.
[0157] If this step is deemed affirmative, terminal device 3 generates a vehicle list generation request and sends the request to server device 2. The vehicle list generation request includes information indicating the map's display area and information indicating the period of the objects specified by the operator.
[0158] In step S22, the server device 2 (information providing unit 212) generates a vehicle list based on the received request. Specifically, it extracts records of vehicles that have driven within a specified range during a specified period from the vehicle data table stored in the storage unit 22, and generates a vehicle list consisting of the extracted records. The generated vehicle list is sent to the terminal device 3.
[0159] In step S23, the terminal device 3 (display control unit 311) draws a map based on the acquired vehicle list and maps icons to the corresponding locations on the map.
[0160] When the processing in step S23 is completed, as follows: Figure 7 As shown, in terminal device 3, this is represented by multiple icons mapped onto the map. Each icon indicates the location of a vehicle that has traveled within the map's display area during a specified period.
[0161] It should be noted that, when using the method of specifying together, terminal device 3 can, at the same time as depicting the icon, such as... Figure 10 As shown, the graphic representing the specified area is depicted on the map.
[0162] Figure 16 This is a sequence diagram of the processing of images provided by the vehicle-mounted device 1 to the terminal device 3 based on a request from the terminal device 3. Figure 16 The processing shown is initiated by terminal device 3 (display control unit 311) after the vehicle icon is mapped onto the map.
[0163] First, the handling of cases where vehicles are designated through a separate designation method will be explained.
[0164] When the operator of terminal device 3 selects any of the displayed icons, display control unit 311 accepts the operation. It should be noted that display control unit 311 may also respond to the selection operation by referring to the vehicle list to obtain the date and time information corresponding to the selected icon, and output the date and time information (for example, by overlaying the date and time information on the screen).
[0165] Next, the image acquisition unit 312 generates a playback request (step S31). The playback request is data used to request the provision of vehicle images from the vehicle-mounted device 1, including the identifier of the vehicle-mounted device 1 (vehicle 10) (vehicle ID) and the identifier of the date and time when the vehicle images were captured (date and time ID). The vehicle ID and the date and time ID are included in the vehicle list received by the terminal device 3 from the server device 2. When the operator selects an icon on the map, the terminal device 3 can determine the corresponding vehicle ID and date and time ID.
[0166] A playback request is sent from terminal device 3 to server device 2, and then relayed from server device 2 to the target vehicle-mounted device 1. At this time, server device 2 can read the vehicle ID contained in the playback request and identify the vehicle-mounted device 1 as the destination of the playback request. Therefore, it is preferable that server device 2 stores data that associates the vehicle ID with the corresponding connection destination (e.g., network address) of the vehicle-mounted device 1. The playback request relayed by server device 2 is received by the corresponding vehicle-mounted device 1.
[0167] Upon receiving the reproduction request, the vehicle-mounted device 1 (providing unit 113) extracts the motion image file corresponding to the date and time ID from the stored motion image data (step S32). For example, in Figure 4 In the example, when a date and time ID such as T002 is specified, the vehicle-mounted device 1 (providing unit 113) extracts the motion image file that started recording at 12:01.
[0168] The extracted motion image file is sent to the terminal device 3 via server device 2. Terminal device 3 generates a file as follows: Figure 9The reproduced image shown begins the reproduction of the received motion picture file (step S33).
[0169] It should be noted that if an operation to specify a different moving image file has been performed (for example, if the button indicated by reference numeral 901 has been pressed), then executing the command again will not work. Figure 16 The process shown generates a new reproduction request.
[0170] It should be noted that in this example, a moving image file is sent, but the object can also be a video stream, etc.
[0171] Next, the handling of cases where vehicles are specified in a combined manner will be explained.
[0172] Figure 16 This is the sequence in which the operator designates a single vehicle to issue a playback request. On the other hand, when using the simultaneous designation method, the terminal device 3 performs the process of generating and issuing multiple playback requests for multiple icons contained in the designated area. Figure 17 This is the flowchart for the process. Figure 17 The process shown is initiated by terminal device 3 (display control unit 311) when the operator selects a designated area on the map. The designated area can be set automatically by the system or by the operator.
[0173] First, in step S41, information related to the icons contained in the specified area is obtained. In this step, for each icon contained in the specified area, information contained in the vehicle list is obtained.
[0174] Next, in step S42, the order in which the reproduction requests are issued is determined. As mentioned above, the order in which the reproduction requests are issued can be determined based on time (the difference from a specified time) or on distance (the distance from the center of a specified area).
[0175] Steps S43 to S45 are performed for each of the multiple vehicles corresponding to the multiple icons contained in the specified area, in the order in which the aforementioned reproduction requests were issued. The vehicle corresponding to a specific icon can be determined based on a vehicle list.
[0176] In step S43, it is determined whether a reproduction request can be issued to the vehicle being processed. For example, if a reproduction request has been sent to a vehicle and a response has not yet been received from that vehicle, another reproduction request should not be sent. This is because repeatedly sending a reproduction request in such a situation could potentially cause errors. In such cases, it is determined that a reproduction request cannot be issued to the target vehicle.
[0177] If a reproduction request can be issued to the target vehicle, the process transfers to step S44 to issue a reproduction request to that vehicle. As a result, execution... Figure 16 The process shown receives motion image files.
[0178] If a reproduction request cannot be issued to the target vehicle, the process proceeds to step S46, waiting until a reproduction request can be issued. For example, if a response to a previously issued reproduction request is received while waiting, it is determined that a new reproduction request can be issued.
[0179] In step S45, it is determined whether the issuance of all reproduction requests has been completed, that is, whether the processing for all icons contained in the specified area has been completed. If the determination is negative in this step, the process returns to step S43. If the determination is positive in this step, the process ends.
[0180] It should be noted that in the simultaneous specification method, multiple moving image files may sometimes be sent to the terminal device 3 at the same time. Upon receiving a moving image file, the terminal device 3 can mark the corresponding icon, i.e., change the icon's appearance. Marking may include, for example, emphasizing the icon, changing its color, or altering its design. Marking an icon signifies that the reception of the moving image file is complete and it is ready for playback. When a marked icon is selected, the terminal device 3 can begin playing back the corresponding moving image file.
[0181] As explained above, in the monitoring system of this embodiment, the terminal device 3 maps the vehicle's driving performance onto a map based on vehicle data stored in the server device 2. The operator of the terminal device 3 can specify a vehicle on the displayed map and obtain motion image files from that vehicle. Thus, the operator of the terminal device 3 can confirm the onboard images captured by the vehicle that has driven at the desired location at the desired time.
[0182] Furthermore, in the monitoring system of this embodiment, a request to reproduce vehicle images can be automatically issued for multiple icons contained in a designated area on the map. Thus, even without selecting each icon individually, the operator can automatically acquire vehicle images for multiple vehicles traveling within the desired area.
[0183] Furthermore, terminal device 3 controls the process to prevent multiple playback requests from being sent to the same vehicle. This prevents errors and allows for the acquisition of in-vehicle images in a short time.
[0184] (Modified example) The above-described implementation is merely an example, and this disclosure may be implemented with appropriate modifications without departing from its spirit.
[0185] For example, as long as there is no technical contradiction, the treatments and methods described in this disclosure can be freely combined and implemented.
[0186] Furthermore, in the implementation, a vehicle equipped with an in-vehicle device is shown as an example, but any mobile body other than a vehicle can be used as long as it can capture images.
[0187] Furthermore, in the implementation, an example is given where the appearance of the corresponding icon changes after the terminal device 3 has completed receiving the motion image file. However, in the case of streaming the in-vehicle video, it can also be set up so that the terminal device 3 receives a notification from the in-vehicle device 1 that preparation for receiving streaming media has been completed, and in response to this, the terminal device 3 changes the appearance of the corresponding icon.
[0188] Furthermore, while the embodiment provides an example of the terminal device 3 changing the appearance of the corresponding icon after receiving the motion picture file, the terminal device 3 can also change the icon's appearance at the time when a playback request is sent. Moreover, the icon's appearance can change at multiple steps, such as "playback request sent successfully" or "motion picture file received successfully." In this case, the terminal device 3 can, for example, control the icon's color or other aspects to change in stages.
[0189] Furthermore, in this implementation, the designated area is depicted on a map (reference numeral 1001), but the designated area may not necessarily be depicted on a map. In this method, the invisible designated area is also set on the map, and a reproduction request is issued to each vehicle through the aforementioned processing.
[0190] Furthermore, in this embodiment, the terminal device 3 generates and outputs a user interface containing a road map based on information received from the server device 2, but the road map can also be generated by the server device 2. In this case, the server device 2 can perform road map drawing and icon mapping instead of generating a vehicle list and sending the vehicle list to the terminal device 3. In this case, a user interface screen containing the road map generated by the server device 2 can be sent to the terminal device 3, and the terminal device 3 can output the user interface screen. That is, the server device 2 (information providing unit 212) can also have the functions of the terminal device 3 (display control unit 311) described in this embodiment. The information processing apparatus of this disclosure can be implemented as a server device 2 or as a terminal device 3.
[0191] Furthermore, a process described as being performed by one device can also be executed by multiple devices. Alternatively, a process described as being performed by different devices can also be executed by one device. In a computer system, the hardware configuration (server configuration) used to implement each function can be flexibly changed.
[0192] This disclosure can also be implemented by supplying a computer program that implements the functions described in the above embodiments to a computer, which then reads and executes the program using one or more processors. Such a computer program can be provided to the computer via a non-transitory computer-readable storage medium that can be connected to the computer's system bus, or via a network. Non-transitory computer-readable storage media include, for example, any type of disk such as a hard disk (floppy disk, hard disk drive, etc.), an optical disk (CD-ROM, DVD, Blu-ray disc, etc.), a read-only memory (ROM), a random access memory (RAM), an EPROM, an EEPROM (Electrically Erasable Programmable Read Only Memory), a magnetic card, flash memory, an optical card, or any type of medium suitable for storing electronic instructions.
[0193] Explanation of reference numerals in the attached figures 1: Vehicle-mounted device; 2: Server device; 3: Terminal device; 11, 21, 31: Control unit; 12, 22, 32: Storage unit; 13, 23, 33: Communication unit; 14: Location information acquisition unit; 15: Camera; 34: Input / output unit.
Claims
1. An information processing apparatus comprising a control unit, the control unit performing the following processing: Based on the driving history of more than one vehicle, multiple icons representing the positions of each vehicle at multiple times are displayed on the map; and In response to an instruction received from a user, an image request is sent to two or more vehicles or on-board devices corresponding to two or more icons set in a first area on the map, requesting the transmission of images captured at the corresponding time.
2. The information processing apparatus according to claim 1, wherein, When any one of the multiple icons is selected, the control unit sends the image request to the corresponding vehicle or the on-board device mounted on the vehicle.
3. The information processing apparatus according to claim 1, wherein, The instruction is given by the user selecting the interior of the first area.
4. The information processing apparatus according to claim 1, wherein, Upon receiving the instruction, the control unit prioritizes sending the image request to the vehicle or on-board device corresponding to the icon that is closer to the specified time.
5. The information processing apparatus according to claim 1, wherein, When a first region is defined on the map by the instruction, the image request is sent with higher priority to the vehicle or vehicle-mounted device corresponding to the icon that is closer to the specified location within the first region.
6. The information processing apparatus according to claim 1, wherein, The control unit changes the appearance of the icon corresponding to the vehicle or in-vehicle device that received the image request.
7. The information processing apparatus according to claim 1, wherein, When the user is able to view or listen to the image after the image request is sent, the control unit changes the appearance of the icon corresponding to the vehicle or in-vehicle device.
8. The information processing apparatus according to claim 1, wherein, When a notification is received from the vehicle or vehicle-mounted device that the image request was sent, indicating that preparation for image transmission has been completed, the control unit changes the appearance of the icon corresponding to the vehicle or vehicle-mounted device.
9. An information processing method, executed by a computer, wherein, The information processing method includes: The first step is to display multiple icons representing the locations of various vehicles at multiple times on a map, based on the driving history of one or more vehicles; and The second step involves, in response to the user accepting the instruction, sending image requests to two or more vehicles or in-vehicle devices corresponding to two or more icons set in the first area on the map, requesting the transmission of images captured at the corresponding times.
10. The information processing method according to claim 9, further comprising: The third step involves sending the image request to the corresponding vehicle or the in-vehicle device mounted on the vehicle if any one of the multiple icons is selected.
11. The information processing method according to claim 9, wherein, The instruction is given by the user selecting the interior of the first area.
12. The information processing method according to claim 9, wherein, In the second step, if the instruction is accepted, the image request is sent with higher priority to the vehicle or on-board device corresponding to the icon of the time closer to the specified time.
13. The information processing method according to claim 9, wherein, In the second step, if a first region is defined on the map by the instruction, the image request is sent with higher priority to the vehicle or vehicle-mounted device corresponding to the icon that is closer to the specified location within the first region.
14. The information processing method according to claim 9, further comprising: The fourth step is to change the appearance of the icon corresponding to the vehicle or in-vehicle device that received the image request.
15. The information processing method according to claim 9, further comprising: The fourth step involves changing the appearance of the icon corresponding to the vehicle or in-vehicle device after the image request has been sent, provided that the user is able to view or listen to the image.
16. The information processing method according to claim 9, further comprising: The fourth step involves changing the appearance of the icon corresponding to the vehicle or vehicle-mounted device if a notification is received from the vehicle or vehicle-mounted device that the image transmission preparation has been completed.
17. A program for causing a computer to perform the information processing method as described in any one of claims 9 to 16.
Citation Information
Patent Citations
Image acquisition system and image acquisition method
JP2023019792A
