Image Generation System

Abstract

To provide a technique for facilitating the status monitoring of a plurality of vehicles.SOLUTION: An on-vehicle device 12 includes: an acquisition unit that acquires status information indicating an operating state of an on-vehicle function; a packet generation unit that generates packet data including the acquired status information; and a transmission control unit that transmits the generated packet data to a server device every predetermined unit time. The server device includes: a holding unit that accumulates the transmitted packet data; an extraction unit that extracts packet data in the same time zone transmitted within a range of the unit time from the accumulated packet data; and an image generation unit that generates a status image based on the status information included in the packet data in the same time zone. The extraction unit extracts the packet data in the same time zone from the packet data accumulated over a time longer than the unit time. The image generation unit generates the status image by arranging the operating state of the on-vehicle function indicated in the status information in a distinguishable manner on the image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a technique for imaging status information of multiple vehicles. [Background technology]

[0002] Patent Document 1 discloses an information providing method in which a server device provides information for selecting a channel to a vehicle capable of communicating using multiple channels. This server device acquires report data including location information from the vehicle and generates a network information database that shows the wireless communication status for each geographical area. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-163935 Summary of the Invention [Problem to be solved by the invention]

[0004] When the number of vehicles to be monitored increases, the volume of reported data from the vehicles increases, making it difficult for the administrator to grasp the fault condition from the reported data.

[0005] An object of the present invention is to provide a technique that makes it easy to monitor the status of multiple vehicles. [Means for solving the problem]

[0006] In order to solve the above problems, an image generation system according to one aspect of the present invention includes an in-vehicle device mounted on each of a plurality of vehicles, and a server device capable of communicating with the plurality of in-vehicle devices. The in-vehicle device includes an acquisition unit that acquires status information indicating the operating status of an in-vehicle function, a packet generation unit that generates packet data including the acquired status information, and a transmission control unit that transmits the generated packet data to the server device every predetermined unit time. The server device includes a storage unit that accumulates the transmitted packet data, an extraction unit that extracts packet data transmitted within the same time period from the accumulated packet data, and an image generation unit that generates a status image based on the status information included in the packet data transmitted within the same time period. a display unit that displays the generated status image; The multiple status images generated by the image generation unit are displayed in chronological order. On the display The extraction unit extracts packet data for the same time period from packet data accumulated over a period longer than the unit time, and the image generation unit generates a status image by arranging the operational states of the in-vehicle functions indicated in the status information in a distinguishable manner on the image. At the current image generation timing, the image generation unit generates a status image based on the status information from the previous image generation timing, and also generates surplus status images based on the status information before the previous image generation timing. On the display If there is a processing delay, the output control unit displays the surplus status images. [Effects of the Invention]

[0007] According to the present invention, a technique can be provided that makes it easy to monitor the status of multiple vehicles. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating a configuration of an image generation system according to an embodiment. [Figure 2] FIG. 1 is a diagram illustrating a functional configuration of an image generation system according to an embodiment. [Figure 3] FIG. 10 is a diagram illustrating packet extraction timing. [Figure 4] FIG. 10 is a diagram showing a status image generated by an image generation unit. [Figure 5] 10 is a flowchart of a process for imaging the status of a vehicle. DETAILED DESCRIPTION OF THE INVENTION

[0009] FIG. 1 is a diagram showing the configuration of an image generation system 1 according to an embodiment. The image generation system 1 includes a distributed server device 10a, a central server device 10b, and an in-vehicle device 12. The in-vehicle device 12 is provided in each of a plurality of vehicles and is capable of wireless communication with the distributed server device 10a via a wireless station. The plurality of in-vehicle devices 12 each transmits vehicle status information to the distributed server device 10a. In FIG. 1, two in-vehicle devices 12 each transmit data to the distributed server device 10a, but the actual number of vehicles is expected to be in the tens of thousands.

[0010] The distributed server devices 10a are provided in various locations and connected to the central server device 10b via a network. The distributed server device 10a converts status information received from the multiple in-vehicle devices 12 into an image and transmits it to the central server device 10b. The central server device 10b combines the images of the status information received from the multiple distributed server devices 10a and displays them to an administrator. When there is no need to distinguish between the distributed server device 10a and the central server device 10b, they are simply referred to as server device 10.

[0011] 2 shows the functional configuration of the image generation system 1 according to the embodiment. The various functions of the image generation system 1 can be configured in terms of hardware using circuit blocks, memory, and other LSIs, and can be realized in terms of software using programs loaded into memory. Therefore, it will be understood by those skilled in the art that the various functions of the image generation system 1 can be realized in various forms using only hardware, only software, or a combination thereof, and are not limited to any of these.

[0012] The distributed server device 10a includes a communication unit 20, a storage unit 22, an extraction unit 24, an image generation unit 26, and an output control unit 28. The central server device 10b includes a communication unit 30, an image generation unit 32, an output control unit 34, an estimation unit 36, and a display unit 38. The in-vehicle device 12 includes a GPS receiving unit 40, a detection unit 42, an acquisition unit 44, a packet generation unit 46, and a transmission control unit 48.

[0013] The GPS receiver 40 of the in-vehicle device 12 periodically acquires the current position information of the vehicle by using the GPS (Global Positioning System) and attaching a timestamp to the information. The GPS receiver 40 functions as a position information acquisition unit that acquires the position information of the vehicle. Note that the location information may be acquired using another GNSS (Global Navigation Satellite System) instead of the GPS.

[0014] The detector 42 is provided in each in-vehicle device and detects the operating state of an in-vehicle function. For example, the detector 42 detects the operating state of a motor device, a steering device, a brake device, an air conditioner device, a headlight device, a short-range wireless communication device, an audio device, a driving support function such as an auto-cruise, etc. The detector 42 may be a driving state detection sensor that detects the driving state of the vehicle.

[0015] The acquisition unit 44 acquires status information indicating the operating state of the in-vehicle functions as the detection result by the detection unit 42. The acquisition unit 44 also acquires current location information of the vehicle from the GPS receiving unit 40.

[0016] The packet generator 46 generates a packet including the acquired status information. The packet generated by the packet generator 46 is a UDP heartbeat packet, and is sent to the distribution server device 10a every unit time. The unit time is, for example, one second, but is not limited to one second and may be several seconds. In the following description, one second is used as the unit time.

[0017] The packet generator 46 generates a packet by adding one or more pieces of status information to the packet. The status information includes a status ID indicating the type of status and information representing the status. The information representing the status can be, for example, "0" or "1," where "0" indicates normal and "1" indicates abnormal. The information representing the status is not limited to two levels, but may be three or more levels. In addition, a timestamp is stamped on the packet as the time on the sending side.

[0018] The packet generator 46 may add status information and vehicle location information to the packet. By including the vehicle location information in the packet, it becomes possible to grasp a fault that has occurred in a region and its impact.

[0019] The transmission control unit 48 transmits packet data generated by the packet generation unit 46 to the distribution server device 10a every unit time. The transmission control unit 48 simply sends the packets without checking whether they have arrived at the distribution server device 10a. However, because packets are transmitted every second, even if one packet does not arrive, the next packet is transmitted immediately. The packet data transmitted from the transmission control unit 48 includes the vehicle ID.

[0020] The communication unit 20 of the distribution server device 10a receives packet data from the multiple on-board devices 12 that it manages. The on-board devices 12 that it manages are assigned to each region. The distribution server device 10a collects vehicle information from the on-board devices 12 in the management region. The storage unit 22 stores and accumulates the received packet data. The packet data stored in the storage unit 22 is stamped with a timestamp as the receiving side time. The storage unit 22 may discard packet data whose receiving side time is older than 10 seconds.

[0021] The extraction unit 24 extracts packet data from the held packet data that was transmitted within a unit time range and that occurred in the same time period. Based on the transmitting side time included in the packet data, the extraction unit 24 extracts packet data transmitted between a first time and a second time as data from the same time period. The difference between the first time and the second time is the unit time, which is one second.

[0022] The extraction unit 24 extracts packets from the same time period from packets accumulated for a predetermined extraction target time longer than the unit time. In other words, the packets to be extracted are those accumulated for a predetermined extraction target time longer than the unit time. For example, the extraction unit 24 extracts packets whose sending side time falls within one second from packets accumulated for 10 seconds by the holding unit 22. This makes it possible to extract packet data in which a transmission delay has occurred as a packet from the same time period.

[0023] Fig. 3 is a diagram for explaining packet extraction timing. The horizontal axis in Fig. 3(a) and Fig. 3(b) is the same time axis, and Fig. 3(b) shows the extraction process in the cycle next to the extraction process shown in Fig. 3(a).

[0024] 3(a), the extraction unit 24 extracts packets from the packets accumulated for 10 seconds on the receiving side time, which are packets from the same time period on the transmitting side time. That is, the extraction unit 24 extracts the packets accumulated for 10 seconds by separating them into 1-second intervals on the transmitting side time. The status information contained in the extracted packets for each 1-second interval is output as an image generated by the image generation unit 26, which will be described later.

[0025] In Figure 3(b), the extraction process is executed 5 seconds after the extraction process shown in Figure 3(a). In this way, packets are extracted from the extraction target for 10 seconds, which is longer than the unit time. The status for 5 seconds is duplicated, but this is to absorb the processing delay.

[0026] Returning to Figure 2, the image generation unit 26 generates a status image based on the status information contained in the packet data for the same time period. The image generation unit 26 generates a status image by arranging the operating states of the in-vehicle functions indicated in the status information so that they can be distinguished on the image. Making them distinguishable on the image is achieved by changing the symbols or colors corresponding to the operating states. By visualizing them, the administrator can grasp at a glance the occurrence of an abnormality and its spread, making it easier to identify the cause of the abnormality. By extracting packets from the same time period from the accumulated packets, packets with delayed transmission can be included in the analysis. A status image is generated for each type of status.

[0027] The image generation unit 26 generates 10 status images based on the status information extracted every second. The image generation timing by the image generation unit 26 is every 5 seconds, the same as the extraction unit 24. The image generation unit 26 generates the status images by arranging the status information according to the vehicle position information.

[0028] Fig. 4 shows a status image generated by the image generation unit 26. In Fig. 4, a square indicates a normal status, and a square indicates an abnormal status, with the status plotted for one second. The status image is generated by plotting the status of each vehicle on a map based on the vehicle's position information.

[0029] The number of vehicles that can connect to the server device 10 is expected to reach several million, and monitoring such a large number of vehicles is not easy. For example, if a disaster occurs and the server device 10 receives a large number of notifications from each vehicle reporting an abnormality, it is difficult for an administrator to understand what kind of problem has occurred even when viewing the large number of notifications. In the image generation system 1 of the embodiment, a status image is displayed, allowing the administrator to grasp at a glance the area where an abnormality is occurring and to accurately identify the disaster that has occurred in the area. Note that the status image may not only be plotted on a map, but may also be generated by plotting the status images in chronological order of transmission.

[0030] Returning to Figure 2, the image generation unit 26 generates 10 status images every 5 seconds, so extra images are generated to compensate for processing delays when the status images are finally output as a video.

[0031] The output control unit 28 causes the generated status image to be transmitted to the central server device 10b together with the area ID. The output control unit 28 may display the generated status images on the display in order.

[0032] The communication unit 30 of the central server device 10b receives status images from the distributed server device 10a. The image generation unit 32 combines status images for the same time period received from multiple distributed server devices 10a based on their area IDs. This generates a status image for all areas. By distributing the processing, the processing load can be reduced.

[0033] The output control unit 34 displays the generated status images in chronological order on the display unit 38. If there is no processing delay, the output control unit 34 discards and displays five seconds of status images, but if there is a processing delay, it displays the excess status images. The display unit 38 is a display device. The manager can view the status images as video, allowing him to understand how the problem has spread. This allows the manager to determine whether the problem only occurs while the vehicle is in a specific area, whether it occurs because the vehicle has passed through a specific area, or whether it is a problem that occurred individually for the vehicle.

[0034] The estimation unit 36 analyzes the status image generated by the image generation unit 32 and estimates the type of failure based on the status image. That is, the estimation unit 36 executes image diagnosis processing and, when the status image is input, outputs information indicating the type of failure. The estimation unit 36 may estimate the type of failure using an image diagnosis model trained using, for example, a neural network technique. By imaging the status information, the type of failure can be estimated by image diagnosis. The type of failure may be, for example, various natural disasters, such as earthquakes, flooding, or frozen roads.

[0035] 5 is a flowchart of the process of imaging the vehicle status. The detection unit 42 of the in-vehicle device 12 detects the operating state of the in-vehicle functions and sends the status information to the acquisition unit 44 (S10). The GPS receiving unit 40 acquires the current location information of the vehicle and sends it to the packet generation unit 46.

[0036] Packet generation unit 46 adds status information and vehicle position information to the heartbeat packet to generate packet data (S12). Transmission control unit 48 transmits the generated packet data to distribution server device 10a every unit time (S14). The packet data transmitted from transmission control unit 48 is stamped with the transmission side time.

[0037] The storage unit 22 of the distributed server device 10a stamps the received packet data with the receiving side time, stores the data, and accumulates it (S16). The extraction unit 24 extracts packet data for the same time period based on the sending side time from the accumulated packet data (S18). The image generation unit 26 plots the status information contained in the extracted packet data for the same time period to generate a status image (S20). The generated status image is assigned information indicating that the time period is the same, and a regional ID. The communication unit 20 transmits the generated status image to the central server device 10b (S22).

[0038] The image generation unit 32 of the central server device 10b combines the status images for the same time period received from the multiple distributed server devices 10a based on the area ID (S24). The output control unit 34 displays the combined status image on the display unit 38 (S26). By having the output control unit 34 display the status images in chronological order on the display unit 38, the administrator can see the state of the failure in video and understand the occurrence of the abnormality and its impact.

[0039] It should be understood by those skilled in the art that the embodiments are merely illustrative and that various modifications are possible in the combination of the components, and that such modifications are also within the scope of the present invention.

[0040] In the embodiment, the server device 10 is divided into the distributed server device 10a and the central server device 10b to perform distributed processing, but this is not limiting. For example, the server device 10 may process packet data from the in-vehicle device 12 in a lump. [Explanation of symbols]

[0041] 1 Image generation system, 10a Distributed server device, 10b Central server device, 12 In-vehicle device, 20 Communication unit, 22 Storage unit, 24 Extraction unit, 26 Image generation unit, 28 Output control unit, 30 Communication unit, 32 Image generation unit, 34 Output control unit, 36 Estimation unit, 38 Display unit, 40 GPS receiving unit, 42 Detection unit, 44 Acquisition unit, 46 Packet generation unit, 48 Transmission control unit.

Claims

1. an on-board device mounted in each of a plurality of vehicles; a server device capable of communicating with a plurality of in-vehicle devices; The in-vehicle device an acquisition unit that acquires status information indicating an operating state of an in-vehicle function; a packet generation unit that generates packet data including the acquired status information; a transmission control unit that transmits the generated packet data to a server device at predetermined unit time intervals; The server device a storage unit for storing transmitted packet data; an extracting unit that extracts packet data transmitted within a unit time period from the accumulated packet data; an image generating unit that generates a status image based on status information included in packet data in the same time period; a display unit that displays the generated status image; an output control unit that causes the display unit to display the plurality of status images generated by the image generation unit in chronological order; the extraction unit extracts packet data for the same time period from packet data accumulated for a time longer than the unit time; the image generation unit generates a status image by arranging the operational states of the in-vehicle functions indicated in the status information in a distinguishable manner on the image; the image generation unit generates a status image at the current image generation timing based on status information from the previous image generation timing, and generates surplus status images based on status information before the previous image generation timing; The image generating system is characterized in that the output control unit causes the display unit to display surplus status images when there is a processing delay.

2. The server device a plurality of distributed server devices; a central server device that receives status images transmitted from the plurality of distributed server devices; the distributed server device transmits a status image generated by plotting the operational status of the in-vehicle function on a map and an area ID to the central server device; 2. The image generation system according to claim 1, wherein the central server combines status images received from the plurality of distributed server devices for the same time period based on the area ID.

Images