Information processing device

Abstract

To acquire target images more efficiently from images captured by the vehicle. [Solution] The information processing device includes a communication unit that communicates with an in-vehicle device, and a control unit that, when the vehicle's driving history indicates that the vehicle is passing near a target object, acquires an image captured near the target object from the vehicle's in-vehicle device, sends information indicating the state of the target object to a terminal device, and causes the terminal device to output the information.

Description

Technical Field

[0007]

[0001] This disclosure relates to an information processing apparatus.

Background Art

[0002] In order to verify the surrounding situation after an accident or the like occurs in a vehicle, imaging means around the vehicle such as a drive recorder is known. For example, Patent Document 1 discloses a technique for carefully selecting and collecting highly accurate images from a large number of similar images captured by a plurality of vehicles.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When acquiring a target image from a large number of captured images, the load of image processing becomes extremely large. Therefore, there is room for improvement in the efficiency of acquiring a target captured image from the captured images by a vehicle.

[0005] Hereinafter, an information processing apparatus and the like that enable more efficiently acquiring a target captured image from the captured images by a vehicle will be disclosed.

Means for Solving the Problems

[0006] The information processing apparatus in this disclosure includes a communication unit that communicates with a vehicle, and a control unit that, when the driving history of the vehicle indicates that the vehicle has passed near a target object, acquires a captured image captured near the target object from the vehicle and outputs information indicating the state of the target object.

Effects of the Invention

[0007] According to the information processing device etc. described in this disclosure, it becomes possible to acquire a target image from images captured by a vehicle more efficiently. [Brief explanation of the drawing]

[0008] [Figure 1] This is a diagram showing an example of the configuration of an information provision system. [Figure 2] This is a sequence diagram showing an example of the operating procedure of the information provision system. [Modes for carrying out the invention]

[0009] The embodiments will be described below with reference to the drawings.

[0010] Figure 1 shows an example of the configuration of a vehicle control system in one embodiment. The vehicle control system 1 has one or more server devices 10, one or more in-vehicle devices 12 mounted on each vehicle 13, and one or more terminal devices 15, all connected to each other via a network 11 to enable information communication. The server devices 10 correspond to the "information processing device" in this embodiment and are, for example, one or more server computers belonging to a cloud computing system or other computing system, which function as servers that implement various functions. The vehicles 13 are passenger cars, commercial vehicles, etc., and include internal combustion engine vehicles, hybrid vehicles (HEV; Hybrid Electric Vehicle), plug-in hybrid vehicles (PHEV; Plug-in Hybrid Electric Vehicle), etc. The in-vehicle devices 12 are computers having communication functions and information processing functions, and control the operation of the vehicle 13 and the operation of the imaging device 14 mounted on the vehicle 13. The imaging device 14 is a so-called drive recorder device and has a digital camera and its control circuit installed in a position that can capture images of the inside and outside of the vehicle 13. The terminal device 15 is an information processing device equipped with communication functions, and includes, for example, a personal computer, a smartphone, etc. The network 11 includes, for example, a mobile communication network, the Internet, an ad hoc network, a LAN (Local Area Network), a MAN (Metropolitan Area Network), or other networks, or any combination thereof.

[0011] The server device 10 includes a communication unit 101 and a control unit 103. The communication unit 101 communicates with the vehicle 13. When the vehicle 13's driving history indicates that the vehicle 13 is passing near a target object, the control unit 103 acquires an image captured near the target object from the vehicle 13 and outputs information indicating the state of the target object. The target object is, for example, a building that requires time-series monitoring of its degree of deterioration, and includes buildings such as office buildings and roadside installations such as signs. In the vehicle 13, the onboard device 12 communicates with the server device 10 and controls the imaging device 14 to acquire an image. The server device 10 then sends the image acquired from the vehicle 13 to the terminal device 15. By continuously performing these operations, the operator who acquires the image via the terminal device 15 can monitor the appearance of the target object included in the image over time. The server device 10 selects a vehicle 13 that is highly likely to be traveling near the target object based on the driving history of multiple vehicles 13, and has that vehicle 13 send images taken near the target object. This eliminates the need to extract the target object from images collected from many other vehicles 13, and allows the operator to be provided with images that are highly likely to contain the target object. Therefore, it becomes possible to acquire the target image from images taken by vehicles more efficiently.

[0012] An example configuration of the server device 10 will be described. The server device 10 includes a communication unit 101, a control unit 103, and a storage unit 102. The server device 10 may be a single computer, or it may consist of two or more computers that are connected and operate in coordination with each other. When the server device 10 consists of two or more computers, the configuration shown in Figure 1 is appropriately arranged on the two or more computers.

[0013] The communication unit 101 includes one or more communication interfaces. The communication interface is, for example, a LAN interface. The communication unit 101 receives information used in the operation of the control unit 103 and transmits information obtained through the operation of the control unit 103. The server device 10 is connected to the network 11 by the communication unit 101 and communicates information with the in-vehicle device 12, terminal device 15, etc. via the network 11.

[0014] The storage unit 102 includes, for example, one or more semiconductor memories, one or more magnetic memories, one or more optical memories, or a combination of at least two of these, which function as main memory, auxiliary memory, or cache memory. The semiconductor memory is, for example, RAM (Random Access Memory) or ROM (Read Only Memory). The RAM is, for example, SRAM (Static RAM) or DRAM (Dynamic RAM). The ROM is, for example, EEPROM (Electrically Erasable Programmable ROM). The storage unit 102 stores information used in the operation of the control unit 103 and information obtained by the operation of the control unit 103.

[0015] The control unit 103 includes one or more processors, one or more dedicated circuits, or a combination thereof. The processors are, for example, general-purpose processors such as CPUs (Central Processing Units) or dedicated processors such as GPUs (Graphics Processing Units) specialized for specific processing. The dedicated circuits are, for example, FPGAs (Field-Programmable Gate Arrays) or ASICs (Application Specific Integrated Circuits). The control unit 103 controls each part of the server device 10 and performs information processing related to the operation of the server device 10.

[0016] The functions of the server device 10 are realized by executing a control program on a processor included in the control unit 103. The control program is a program that causes a computer to execute the processing steps included in the operation of the server device 10, thereby realizing the functions corresponding to the processing of those steps. In other words, the control program is a program that causes a computer to function as the server device 10. Furthermore, some or all of the functions of the server device 10 may be realized by a dedicated circuit included in the control unit 103. In addition, the control program may be stored in a non-transient recording / storage medium readable by the server device 10, and the server device 10 may read it from the medium.

[0017] The configuration of the in-vehicle device 12 will now be described. The in-vehicle device 12 consists of one or more control devices and a communication device and other devices. The control device includes a microcomputer such as an ECU (Electronic Control Unit). The control device has a user interface and can receive information input by user operation and output information obtained through information processing. The communication device includes a DCM (Data Communication Module) and is connected to the network 11 by mobile communication and configured to communicate information with the server device 10 via the network 11. In addition, the in-vehicle device 12 and each part of the vehicle 13 are connected to communicate information via an in-vehicle network compliant with standards such as CAN (Controller Area Network). The in-vehicle device 12 controls the operation of the vehicle 13 by acquiring detection results from sensors such as vehicle speed and acceleration of the vehicle 13 and controlling various actuators that perform the operation of the vehicle 13, and also controls the operation of the imaging device 14 to acquire captured images from the imaging device 14. Furthermore, the in-vehicle device 12 has one or more GNSS (Global Navigation Satellite System) receivers and acquires the position information of the vehicle 13. The in-vehicle device 12 may also include information processing devices such as a navigation system, smartphone, or tablet terminal.

[0018] FIG. 2 is a sequence diagram for explaining the operation procedure of the vehicle control system 1 in the present embodiment. FIG. 2 shows the procedure related to the cooperative operation of the server device 10, the in-vehicle device 12 of the vehicle 13, and the terminal device 15. The steps related to various information processes of the server device 10 in FIG. 2 are executed by the control unit 103. Also, the steps related to the transmission and reception of various information of the server device 10 are executed by the control unit 103 transmitting and receiving information via the communication unit 101. In the server device, the control unit 103 appropriately stores the information processed, referred to, or transmitted and received in the storage unit 102. Further, the operation of the conductor 13 in the present embodiment is realized by the operation of the in-vehicle device 12 in FIG. 2.

[0019] The procedure in FIG. 2 is executed daily or at an arbitrary frequency several times a day. The procedure in FIG. 2 may be performed before the start of travel of the vehicle 13 or during travel.

[0020] In S201, the server device 10 requests the in-vehicle device 12 for a travel history. The server device 10, for example, stores in advance the identification information for each in-vehicle device 12 necessary for transmitting and receiving information to and from one or more in-vehicle devices 12 to be linked. Then, the server device 10 sends information requesting the travel history to each in-vehicle device 12 based on the identification information.

[0021] In S202, the in-vehicle device 12 sends travel history information to the server device 10 in response to the request from the server device 10. The travel history includes the past travel route of the vehicle 13, the travel speed for each arbitrary section, the time when located at an arbitrary point on the travel route, and the like. The in-vehicle device 12 stores in an internal memory or the like a travel history based on the history of the position information of the vehicle 13, the history of the travel speed, etc., and sends the travel history information to the server device 10 in response to the request from the server device 10.

[0022] In S203, the server device 10 analyzes the driving history of each vehicle 13. The server device 10 determines whether there is a driving history of passing near one or more target objects based on the map information. In the server device 10, information indicating the positions of the target objects is stored in advance. The vicinity of the target object is within an arbitrary reference distance range from the target object and includes points where the target object can be imaged from the road. The reference distance is arbitrarily set in advance based on the imaging performance of the imaging device 14 of the vehicle 13. Even within the reference distance, an area of a blind spot where imaging of the target object is blocked by obstacles such as high-rise buildings is excluded from the vicinity of the target object. Such an area of the blind spot is derived by an arbitrary algorithm based on the height of the target object, the distance between the target object and the surrounding buildings, the height of the surrounding buildings, etc. Information such as the target object and the imaging performance of the imaging device 14 is appropriately set, for example, by an operator operating the terminal device 15 and stored in the server device 10. When a plurality of target objects are set, even if the vehicles 13 having a driving history of passing near each target object are different, it is acceptable.

[0023] In S205, the server device 10 selects the vehicle 13 that acquires the imaging image of the target object. The server device 10 selects the vehicle 13 whose driving route passes near the target object. When two or more vehicles 13 can be selected for one target object, two or more vehicles 13 may be selected, or any one of the vehicles 13 may be selected. When selecting any one of the vehicles 13, for example, it is possible to select the vehicle 13 with a lower speed passing near the target object or the vehicle 13 that has passed near the target object more times. The lower the speed passing near the target object or the more times passing near the target object, the greater the possibility of obtaining a higher-quality image of the target object.

[0024] In S206, the server device 10 sends a request for an imaging image to the selected vehicle 13. The request for the imaging image includes position information corresponding to the vicinity of the target object and an instruction to cause the in-vehicle device 12 to send the imaging image in the vicinity of the target object. Further, the request for the imaging image may include an instruction to cause the vehicle 13 to travel at an arbitrary reference speed near the target object. The reference speed is an arbitrary speed that is sufficiently low so that a clearer imaging image can be obtained as long as there is no obstacle to safe traffic.

[0025] In S207, the on-board device 12 starts the vehicle 13 moving as planned or in response to user input. The on-board device 12 periodically acquires the current position of the vehicle 13 and causes the imaging device 14 to start imaging at an arbitrary frame rate.

[0026] In S208, the on-board device 12 sends the captured image taken near the target object to the server device 10. When the vehicle 13's current position is included near the target object, the on-board device 12 sends the image taken by the imaging device 14 to the server device 10. Alternatively, the on-board device 12 may store the image taken when the vehicle 13 is located near the target object in a way that allows it to be identified from other captured images, and send it to the server device 10 at any time.

[0027] In S209, the server device 10 analyzes the target object included in the captured image. Specifically, the server device 10 performs image processing on the captured image using an arbitrary algorithm to detect the target object and determine the degree of deterioration of the target object. The degree of deterioration of the target object is determined based on the degree of color distribution that differs from other parts, such as the deterioration of a wall surface, and the presence or absence of images showing cracks, etc. The server device 10 may also pre-store a judgment model that has been generated by machine learning using images of buildings with associated degrees of deterioration as training data, and use that judgment model to determine the degree of deterioration of the target object included in the captured image.

[0028] In S210, the server device 10 sends the captured image acquired from the in-vehicle device 12 and the analysis results thereof, that is, the results of the determination of the degree of deterioration of the target object, to the terminal device 15.

[0029] In S211, the terminal device 15 displays the captured image and analysis results acquired from the server device 10 to the operator. This allows the operator to visually confirm the image of the target object and recognize the degree of deterioration of the target object.

[0030] According to the operation of the vehicle control system 1 as described above, the selected vehicle 13 sends images taken near the target object, so that the operator can be provided with images that have a high probability of containing the target object without the need to extract the target object from images collected from many other vehicles 13. Therefore, it becomes possible to acquire the target image from images taken by vehicles more efficiently.

[0031] As described above, embodiments have been explained based on various drawings and examples, but it should be noted that those skilled in the art will find it easy to make various modifications and alterations based on this disclosure. Therefore, it should be noted that these modifications and alterations are within the scope of this disclosure. For example, the functions, etc., included in each means, each step, etc., can be rearranged in a logically consistent manner, and multiple means, steps, etc., can be combined into one or divided. [Explanation of Symbols]

[0032] 1. Information provision system, 10. Server equipment, 11. Network, 12. In-vehicle equipment, 13 Vehicle, 14 Imaging device, 15 Terminal device, 101 Communication unit, 102 Storage unit, 103 Control unit

Claims

1. A communication unit that communicates with the in-vehicle device, When the vehicle's driving history determines that the vehicle is passing near a target object, the control unit acquires an image captured near the target object from the vehicle's onboard device, sends information indicating the state of the target object to a terminal device, and causes the terminal device to output the information. An information processing device having

2. A communication unit that communicates with the vehicle, When the vehicle's driving history indicates that the vehicle is passing near a target object, a control unit acquires an image of the target object from the vehicle and outputs information indicating the state of the target object. An information processing device having

3. The control unit determines which of the multiple vehicles will acquire the captured image based on the driving history of the multiple vehicles. The information processing apparatus according to claim 2.

4. The control unit determines which of the plurality of vehicles will acquire the captured image, on the condition that it is traveling at a predetermined speed. The information processing apparatus according to claim 3.

5. The control unit sends an instruction to the vehicle to limit the vehicle's speed. The information processing apparatus according to claim 2.

Images