Information processing equipment and in-vehicle equipment

Abstract

To provide an information processing device and an on-vehicle device that can accurately notify an occupant on a vehicle about a specific anomaly in an abnormal portion.SOLUTION: A cloud server 12 comprises: a reception unit 50 that receives position information for an abnormal portion from a vehicle 14 that detected the abnormal portion during traveling and image information for the abnormal portion; an image processing unit 52 that patterns (e.g., a road caving, a tire as a fallen object, a wood material as a fallen object, or the like) the image information received by the reception unit 50; and a transmission unit 54 that transmits an abnormal pattern patterned by the image processing unit 52 or pattern information representing the abnormal pattern to a vehicle behind the vehicle that detected the abnormal portion corresponding to the abnormal pattern.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information processing device and an in-vehicle device.

Background Art

[0002] Patent Document 1 proposes a driving support system that detects a driver's gaze position, gaze circle, and visual field area from a captured image captured by a driver camera, detects a feature ahead of the vehicle in the traveling direction from a captured image captured by a front camera and feature image information, specifies the driver's visual recognition mode for the feature ahead of the vehicle in the traveling direction based on each detection result, and provides guidance regarding the feature based on the specified visual recognition mode.

Prior Art Literature

Patent Literature

[0003]

Patent Document 1

Summary of the Invention

Problem to be Solved by the Invention

[0004] There is a technology in which the vehicle system determines what kind of obstacles actually exist on a road from detection results and depicts the obstacles on a display unit. In this case, it is difficult for the vehicle to distinguish abnormal locations such as obstacles, and the patterns of depiction icons possessed by the vehicle are limited. Therefore, even in the best case, the depiction modes on the display are often limited to only a few types.

[0005] It is also desired that the vehicle accurately determine the abnormal content of an abnormal location (for example, the type of obstacle, the type of road abnormality such as road depression, etc.) and notify the vehicle occupants of the content.

[0006] The present invention has been made in consideration of the above facts, and an object of the present invention is to provide an information processing device and an in-vehicle device that can accurately notify vehicle occupants of the abnormal content of an abnormal location.

Means for Solving the Problem

[0007] The information processing device according to the first embodiment includes a receiving unit that receives location information of an abnormal location and image information of the abnormal location from a vehicle that has detected an abnormal location while driving, and a transmitting unit that patterns the image information received by the receiving unit and transmits the patterned abnormal pattern or pattern information representing the abnormal pattern, along with the location information, to a vehicle following the vehicle that detected the abnormal location.

[0008] According to the first embodiment, the receiving unit receives location information and image information of the abnormal area from a vehicle that has detected an abnormal area while driving. The transmitting unit then patterns the image information received by the receiving unit and transmits the patterned abnormal pattern or pattern information representing the abnormal pattern, along with the location information, to a vehicle following the vehicle that detected the abnormal area. This makes it possible to accurately inform the occupants of the vehicle of the nature of the abnormal area.

[0009] The information processing device according to the second embodiment prohibits the transmission of the abnormal pattern to the following vehicle in subsequent instances if the abnormal location is eliminated and the following vehicle does not detect the abnormal location in the information processing device according to the first embodiment.

[0010] According to the second embodiment, it is possible to prevent the abnormal pattern from being incorrectly displayed in a following vehicle.

[0011] The information processing device according to the third embodiment is an information processing device according to the first or second embodiment in which the transmission unit transmits an icon image, which is a pattern of the image information, to the following vehicle as the abnormal pattern.

[0012] According to the third embodiment, it becomes possible to increase the number of depiction patterns for abnormal areas in the following vehicle.

[0013] The in-vehicle device according to the fourth embodiment includes: an information transmission unit that, when a detection unit for detecting abnormal locations in front of a vehicle detects an abnormal location, transmits to an external device image information representing an image of the abnormal location and location information representing the location of the abnormal location; a pattern receiving unit that receives from the external device an abnormal pattern obtained by patterning the image information or pattern information representing the abnormal pattern, along with the location information; and a display processing unit that, when the pattern receiving unit receives the abnormal pattern or pattern information from the external device, or when the detection unit detects an abnormal location, causes the display unit to display the abnormal pattern represented by the abnormal pattern or pattern information received by the pattern receiving unit at a position corresponding to the location of the abnormal location detected by the detection unit.

[0014] According to the fourth embodiment, the nature of the abnormality at the abnormal location can be accurately communicated to the vehicle occupants. Furthermore, the relative positional relationship between the abnormal location and the vehicle can be displayed with high precision.

[0015] In the in-vehicle device according to the fifth embodiment, the display processing unit displays the abnormal pattern on the display unit based on the result of comparing the location information of the abnormal location detected by the detection unit with the location information received by the pattern receiving unit.

[0016] According to the fifth embodiment, it is possible to confirm whether the detected abnormal location is the received abnormal location, and the accuracy of displaying the abnormal location can be improved.

[0017] Furthermore, the information processing system may include an information processing device relating to any one of the first to third embodiments and an in-vehicle device relating to the fourth or fifth embodiment.

[0018] Furthermore, the program may be used to cause a computer to function as a component of an information processing device according to any one of the first to third embodiments, or as a program to cause a computer to function as a component of an in-vehicle device according to the fourth or fifth embodiment. [Effects of the Invention]

[0019] As described above, according to the present invention, an information processing apparatus and an in-vehicle apparatus capable of accurately notifying vehicle occupants of the abnormality details of an abnormal location can be provided. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] [Figure 1] FIG. 1 is a block diagram showing a schematic configuration of the information processing system according to the present embodiment. [Figure 2] FIG. 2 is a block diagram showing a configuration of a control unit of an on-board device and a central processing unit of a cloud server. [Figure 3] FIG. 3 is a diagram for explaining the operation of each part of the information processing system according to the present embodiment. [Figure 4] FIG. 4 is a diagram showing a display example on a display unit. [Figure 5] FIG. 5 is a diagram for explaining the operation of each unit when an abnormality is eliminated in the information processing system according to the present embodiment. [Figure 6] FIG. 6 is a flowchart showing an example of a processing flow when transmitting abnormal data (image information and position information) to a cloud server when an abnormality is detected in the on-board device of the information processing system according to the present embodiment. [Figure 7] FIG. 7 is a flowchart showing an example of a processing flow performed by a cloud server when abnormal data (image information and position information) is transmitted from an on-board device in the information processing system according to the present embodiment. [Figure 8] FIG. 8 is a flowchart showing an example of a processing flow performed by an on-board device when a patterned result is transmitted from a cloud server in the information processing system according to the present embodiment. MODE FOR CARRYING OUT THE INVENTION

[0021] Hereinafter, an example of an embodiment of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing a schematic configuration of the information processing system according to the present embodiment.

[0022] In this embodiment, the information processing system 10 consists of an in-vehicle device 16, which is an example of an in-vehicle device mounted on a vehicle 14, and a cloud server 12, which is an example of an information processing device, connected via a communication network 18. Figure 1 shows two vehicles 14, but the in-vehicle devices 16 of three or more vehicles 14 are connected to the cloud server 12 via the communication network 18.

[0023] In the information processing system 10 according to this embodiment, the cloud server 12 collects and stores image information representing captured images taken by multiple vehicles 14 and vehicle information representing the status of each vehicle 14. The cloud server 12 also identifies the details of abnormal locations on the road, such as obstacles and sinkholes, based on the stored information, and performs processing such as transmitting abnormal patterns representing the details of the identified abnormal locations, or pattern information representing the abnormal patterns, to the vehicles in motion.

[0024] The onboard unit 16 mounted on the vehicle 14 detects abnormal areas from the forward-facing image, and if an abnormal area is detected, it transmits the location of the abnormal area and the captured image to the cloud server 12. The onboard unit 16 also receives abnormal patterns or pattern information transmitted from the cloud server 12, and if an abnormal area is detected forward, it determines whether the location received from the cloud server 12 corresponds to the detected location. If both locations correspond, it performs processing such as displaying the abnormal pattern received from the cloud server 12, or the abnormal pattern corresponding to the pattern information received from the cloud server 12.

[0025] In detail, the in-vehicle unit 16 comprises a control unit 20, a vehicle information detection unit 22, an imaging unit 24, a communication unit 26, and a display unit 28.

[0026] The vehicle information detection unit 22 detects vehicle information related to the vehicle 14. Examples of vehicle information include the vehicle's position, speed, acceleration, steering angle, accelerator opening, distance to obstacles around the vehicle, and route. Specifically, the vehicle information detection unit 22 can apply multiple types of sensors and devices to acquire information representing the surrounding environment of the vehicle 14. Examples of sensors and devices include sensors mounted on the vehicle 14, such as a vehicle speed sensor and an acceleration sensor, as well as a GNSS (Global Navigation Satellite System) device, an in-vehicle communication device, a navigation system, an ultrasonic sensor, and a radar device. The GNSS device determines the position of the vehicle 14 by receiving GNSS signals from multiple GNSS satellites. The accuracy of positioning improves as the number of receivable GNSS signals increases. The in-vehicle communication device is a communication device that performs at least one of vehicle-to-vehicle communication with other vehicles 14 and vehicle-to-infrastructure communication with roadside devices via the communication unit 26. The navigation system includes a map information storage unit that stores map information, and performs processing to display the position of the vehicle 14 on a map and guide the vehicle to a destination based on the position information obtained from the GNSS device and the map information stored in the map information storage unit. The radar device includes multiple radars with different detection ranges, and detects objects such as pedestrians and other vehicles 14 that are present around the vehicle 14, and acquires the relative position and relative speed of the detected objects and the vehicle 14. The radar device also has a built-in processing unit that processes the detection results of surrounding objects. Based on the changes in relative position and relative speed of individual objects included in the most recent multiple detection results, this processing unit excludes noise and roadside objects such as guardrails from the monitoring targets, and tracks and monitors pedestrians and other vehicles 14 as monitored objects. The radar device then outputs information such as the relative position and relative speed of each monitored object. Note that the vehicle information detection unit 22 does not need to detect all of this vehicle information, and may detect only some of the vehicle information.

[0027] In this embodiment, the imaging unit 24 includes an outward-facing camera for imaging the outside of the vehicle and an inward-facing camera for imaging the inside of the vehicle, but it may also consist of only an outward-facing camera. The imaging unit 24 uses the outward-facing camera to image the area around the vehicle, such as the front, and generates outward-facing video data as image information, representing the captured video. The imaging unit 24 also uses the inward-facing camera to image the driver inside the vehicle and generates inward-facing video data as image information, representing the captured video. For example, a drive recorder can be used as the outward-facing camera, and a driver monitor camera can be used as the inward-facing camera. The outward-facing camera may also further image the area around at least one of the sides and rear of the vehicle 14.

[0028] The communication unit 26 establishes communication with the cloud server 12 via the communication network 18 and transmits and receives information such as image information obtained by the imaging unit 24 and vehicle information detected by the vehicle information detection unit 22 as CAN (Controller Area Network) data.

[0029] The display unit 28 provides various information to the occupants by displaying information. In this embodiment, for example, it displays information provided by the cloud server 12. The display unit 28 may be, for example, a combination meter or a display provided on the instrument panel.

[0030] As shown in Figure 2, the control unit 20 is composed of a general-purpose microcomputer including a CPU (Central Processing Unit) 20A, ROM (Read Only Memory) 20B, RAM (Random Access Memory) 20C, storage 20D, interface (I / F) 20E, and bus 20F.

[0031] The control unit 20 controls the uploading of image information of a moving image captured by the shooting unit 24, and vehicle information detected by the vehicle information detection unit 22 at the time of image capture, to the cloud server 12. When uploading the image information and vehicle information, identification information that identifies at least one of the vehicle and the driver is added and transmitted. The information that identifies the driver may be, for example, an image of the driver, identification information of a smart key carried by the driver, or other information that can identify the driver.

[0032] Furthermore, the control unit 20 functions as a detection unit 40, an information transmission unit 42, a pattern reception unit 44, and a display processing unit 46 by having the CPU 20A load the program stored in the ROM 20B into the RAM 20C and execute it.

[0033] The detection unit 40 detects abnormal locations in front of the vehicle 14 (for example, road collapses, various obstacles, etc.) based on at least one of the images captured by the imaging unit 24 and the detection results of the vehicle information detection unit 22. For example, abnormal locations may be detected from the captured images by simple image recognition, or they may be detected by ultrasonic sensors or radar devices.

[0034] When the detection unit 40 detects an abnormality, the information transmission unit 42 transmits image information representing the image captured by the shooting unit 24 of the area in front of the vehicle 14, and location information representing the location of the abnormality, to the cloud server 12, which is an external device. The location information is obtained by correcting the location information representing the vehicle's position, which is detected by the vehicle information detection unit 22, to the location of the abnormality detected by the detection unit 40, and then transmitting it to the cloud server 12. For example, the direction and distance to the abnormality are determined from the vehicle's position based on a radar device, ultrasonic sensor, captured image, etc., and the location information of the vehicle's position is corrected to derive the location information of the abnormality.

[0035] The pattern receiving unit 44 receives from the cloud server 12 an abnormal pattern, which is an image information patterned in the cloud server 12, or pattern information representing the abnormal pattern.

[0036] When the pattern receiving unit 44 receives an abnormal pattern or pattern information from the cloud server 12, or when the detection unit 40 detects an abnormal location, the display processing unit 46 performs the process of displaying the abnormal pattern represented by the abnormal pattern or pattern information received by the pattern receiving unit 44 on the display unit 28 at a position corresponding to the location of the abnormal location detected by the detection unit 40. The following describes the case in which the abnormal pattern is displayed on the display unit 28 when the pattern receiving unit 44 receives an abnormal pattern or pattern information via push transmission from the cloud server 12. In more detail, the display processing unit 46 displays the abnormal pattern on the display unit 28 based on the comparison result between the location information of the abnormal location detected by the detection unit 40 and the location information received by the pattern receiving unit 44. When the pattern receiving unit 44 receives pattern information, the display processing unit 46 may prepare multiple types of icon images representing abnormal patterns in advance and display the icon image corresponding to the pattern information on the display unit 28.

[0037] On the other hand, the cloud server 12 includes a central processing unit 30, a central communication unit 36, and a DB (database) 38.

[0038] As shown in Figure 3, the central processing unit 30 is composed of a general-purpose microcomputer including a CPU 30A, ROM 30B, RAM 30C, storage 30D, interface (I / F) 30E, and bus 30F.

[0039] The central processing unit 30 functions as a receiving unit 50, an image processing unit 52, and a transmitting unit 54 by having the CPU 30A load the program stored in the ROM 30B into the RAM 30C and execute it.

[0040] The receiving unit 50 receives location information and image information of the abnormal area from the vehicle 14 that detected the abnormal area while driving, and stores them in the database (DB) 38.

[0041] The image processing unit 52 patterns the image information received by the receiving unit 50 (for example, road collapse, fallen tire, fallen wood, etc.). For example, the image processing unit 52 uses a well-known image recognition model to identify abnormal areas and types of abnormal areas in the captured image from the captured image received by the receiving unit 50. Then, it selects a corresponding abnormal pattern from a predetermined set of abnormal patterns according to the content of the abnormal area. For example, icon images such as road collapse icons, fallen tire icons, and fallen wood icons may be prepared in advance as abnormal patterns, and the icon corresponding to the content of the identified abnormal area may be selected as the abnormal pattern. Alternatively, an icon image may be generated from the image of the identified abnormal area and used as the abnormal pattern.

[0042] The transmitting unit 54 transmits the patterned abnormal pattern or pattern information representing the abnormal pattern, generated by the image processing unit 52, to a vehicle following the vehicle that has detected the abnormal location corresponding to the abnormal pattern. Furthermore, by transmitting an icon image representing the abnormal pattern as the abnormal pattern, the transmitting unit 54 can increase the number of depiction patterns for the abnormal location in the following vehicle.

[0043] The central communications unit 36 ​​establishes communication with the in-vehicle device 16 via the communication network 18 to send and receive information such as image information and vehicle information.

[0044] The database 38 stores vehicle information such as image information and location information received by the receiving unit 50 from multiple vehicles 14.

[0045] Next, the operation of each part of the information processing system 10 according to this embodiment will be explained. Figure 3 is a diagram illustrating the operation of each part of the information processing system 10 according to this embodiment.

[0046] When the detection unit 40 in each vehicle 14 detects an abnormality, it sends image information representing the captured image of the abnormality and location information of the abnormality as data during driving to the cloud server 12. As a result, the cloud server 12 stores the image information and location information of the abnormality detected by the detection unit 40 of each vehicle 14 in the database 38.

[0047] The cloud server 12 analyzes the image information collected from the vehicle 14 and processes it to create patterns for abnormal locations. For example, as shown in Figure 3, the abnormal locations are patterned as abnormal pattern A, abnormal pattern B, etc. Here, we show an example where an icon image corresponding to the abnormal location is selected from a predetermined set of icons. The cloud server 12 then stores the patterned abnormal patterns along with location information in the database 38. For example, as shown in Figure 3, abnormal information 1 (coordinates (x1, y1, z1), abnormal pattern A), abnormal information 2 (coordinates (x2, y2, z2), abnormal pattern A), abnormal information 3 (x3, y3, z3), abnormal pattern B) are stored in the database 38.

[0048] As a result, when a following vehicle 14 travels to the location of an anomaly stored in the database 38 and the detection unit 40 of the following vehicle 14 detects an anomaly, it compares the image information of the anomaly at the time of detection with the anomaly pattern to determine the appropriate anomaly pattern. Specifically, the cloud server 12 compares the location information of the image information at the time of detection with the location coordinates of the anomaly pattern stored in the database 38. If the locations of both correspond, the cloud server 12 transmits the anomaly pattern or pattern information representing the anomaly pattern to the vehicle 14 that detected the anomaly. As a result, the anomaly pattern is displayed on the display unit 28 of the vehicle 14, and the occupants can recognize what kind of anomaly it is based on the anomaly pattern. Therefore, the details of the anomaly at the anomaly can be accurately communicated to the occupants of the vehicle 14.

[0049] When displaying an abnormal pattern on the display unit 28, the relative position of the abnormal location to the vehicle 14 is also displayed as a graphic on the screen that displays the surrounding environment in real time. For example, as shown in Figure 4, a vehicle icon 60 representing the current vehicle is displayed, as well as a preceding vehicle icon 62 representing a preceding vehicle ahead, and an abnormal pattern icon 64 representing the abnormal pattern is displayed as an icon image. Figure 4 is a diagram showing an example of display on the display unit 28.

[0050] Furthermore, if an anomaly is eliminated, as shown in Figure 5, the driving data (image information and location information) detected by the vehicle 14 that passed the location registered as an anomaly is compared with the anomaly information (image information and location information) stored in the cloud server 12. If it is determined that the anomaly has been eliminated, the anomaly pattern icon 64 on the display unit 28 is not displayed and is hidden, as shown in the in-vehicle display in Figure 5. Figure 5 is a diagram illustrating the operation of each part of the information processing system 10 according to this embodiment when an anomaly has been eliminated.

[0051] Next, we will describe the specific processes performed in each part of the information processing system 10 according to this embodiment.

[0052] First, we will explain the process of transmitting abnormal data (image information and location information) to the cloud server 12 when an abnormality is detected in the in-vehicle unit 16 of the vehicle 14. Figure 6 is a flowchart showing an example of the processing flow when an abnormality is detected in the in-vehicle unit 16 of the information processing system 10 according to this embodiment and abnormal data (image information and location information) is transmitted to the cloud server 12. Note that the processing in Figure 6 starts, for example, when an ignition switch (not shown) of the vehicle 14 is turned on.

[0053] In step 100, the CPU 20A acquires vehicle information and captured images and proceeds to step 102. Specifically, it acquires vehicle information detected by the vehicle information detection unit 22 and image information representing the captured images taken by the shooting unit 24.

[0054] In step 102, the CPU 20A determines whether or not it has detected an abnormal location. This determination is made, for example, by the detection unit 40, which determines whether or not it has detected an abnormal location in front of the vehicle 14 (for example, a road collapse or various obstacles) based on at least one of the images captured by the imaging unit 24 and the detection results of the vehicle information detection unit 22. If the determination is affirmative, the process proceeds to step 104; otherwise, the process proceeds to step 106.

[0055] In step 104, the CPU 20A sends location information and image information representing the captured image at the time of anomaly detection to the cloud server 12 and proceeds to step 106. The location information sent to the cloud server 12 is the location information of the anomaly, which is obtained by correcting the direction and distance of the obstacle detected by the detection unit 40 to the vehicle's own location information.

[0056] In step 106, the CPU 20A determines whether or not to terminate the process. This determination is made, for example, by determining whether an ignition switch (not shown) has been turned off. If the determination is negative, the process returns to step 100 and the above process is repeated until the determination is positive, at which point the series of processes is terminated.

[0057] Next, we will explain the processing performed by the cloud server 12 when abnormal data (image information and location information) is transmitted from the in-vehicle device 16. Figure 7 is a flowchart showing an example of the processing flow performed by the cloud server 12 when abnormal data (image information and location information) is transmitted from the in-vehicle device 16 in the information processing system 10 according to this embodiment. Note that Figure 7 starts, for example, when abnormal data is transmitted from the in-vehicle device 16.

[0058] In step 200, the CPU 30A receives image information representing the captured image of the abnormal area and location information of the abnormal area from the in-vehicle device 16, and proceeds to step 202.

[0059] In step 202, the CPU 30A determines whether or not it has received a captured image without any abnormalities. This determination determines whether or not it has received a captured image without any abnormalities and location information. If there are abnormalities in the captured image and the determination is denied, the process proceeds to step 204. If there are no abnormalities in the captured image and the determination is affirmed, the process proceeds to step 210.

[0060] In step 204, the CPU 30A stores the received data in the database 38 and proceeds to step 206. Specifically, it stores the image information and location information of the abnormal area received from the in-vehicle device 16 in the database 38.

[0061] In step 206, the CPU 30A performs anomaly patterning and proceeds to step 208. Anomaly patterning is performed by the image processing unit 52, which patterns the image information received by the receiving unit 50 (for example, road collapse, fallen tire, fallen wood, etc.). For example, the image processing unit 52 uses a well-known image recognition model to identify anomalies in the captured image and the type of anomaly in the captured image received by the receiving unit 50. Then, it selects a corresponding anomaly pattern from a predetermined set of anomaly patterns according to the content of the anomaly.

[0062] In step 208, the CPU 30A sends the patterned result, including location information, to the following vehicle 14 traveling over the abnormal location, thereby ending the series of processes. In this embodiment, the patterned result is sent to the vehicle 14 by push notification from the cloud server 12, but this is not the only option, and the vehicle 14 may also receive it via pull notification. That is, when the vehicle 14 detects an abnormal location, it may send a transmission request to the cloud server 12, and the requesting vehicle 14 may receive the patterned result from the cloud server 12.

[0063] Meanwhile, in step 210, the CPU 30A updates the abnormal data registered in the database 38, assuming that the abnormality has been eliminated, and terminates the series of processes. For example, it prohibits the transmission of the abnormal pattern to subsequent vehicles 14 in the future, and deletes the abnormal pattern and location information of the abnormal location registered in the database 38 without transmitting the abnormal pattern to the vehicle 14.

[0064] Next, we will describe the processing performed in the in-vehicle device 16 when the patterned results are transmitted from the cloud server 12. Figure 8 is a flowchart showing an example of the processing flow performed in the in-vehicle device 16 when the patterned results are transmitted from the cloud server 12 in the information processing system 10 according to this embodiment. Note that the processing in Figure 8 starts when the patterned results are transmitted from the cloud server 12.

[0065] In step 300, CPU 20A receives patterned results containing abnormal information from cloud server 12 and proceeds to step 302.

[0066] In step 302, the CPU 20A acquires vehicle information and captured images and proceeds to step 304. Specifically, it acquires vehicle information detected by the vehicle information detection unit 22 and image information representing the captured images taken by the shooting unit 24.

[0067] In step 304, the CPU 20A determines whether or not it has detected an abnormal location. This determination is made, for example, by the detection unit 40, which determines whether or not it has detected an abnormal location in front of the vehicle 14 (for example, a road collapse or various obstacles) based on at least one of the images captured by the imaging unit 24 and the detection results of the vehicle information detection unit 22. If the determination is negative, the process proceeds to step 306; if it is positive, the process proceeds to step 310.

[0068] In step 306, the CPU 20A determines whether or not it has passed the location indicated by the location information. This determination determines whether or not it has passed the location of the anomaly notified by the cloud server 12 without detecting an anomaly. If the determination is affirmative, the process proceeds to step 308; otherwise, it returns to step 302 and repeats the above process.

[0069] In step 308, the CPU 20A sends the image of the location indicated by the location information to the cloud server 12 along with the location information as the image of the abnormal location, and the series of processes ends. As a result, the cloud server 12 confirms the judgment in step 202 and updates the database 38 in step 210, indicating that the abnormality has been eliminated.

[0070] Meanwhile, in step 310, the CPU 20A compares the location of the anomaly and proceeds to step 312. That is, it compares the location information sent from the cloud server 12 with the location information of the anomaly where the anomaly was detected.

[0071] In step 312, the CPU 20A determines whether the location of the anomaly matches. This determination determines whether an anomaly was detected at the location corresponding to the location information transmitted from the cloud server 12. If the determination is negative, the process proceeds to step 314; if it is positive, the process proceeds to step 316.

[0072] In step 314, the CPU 20A sends the location information and captured images at the time of anomaly detection to the cloud server 12, and the series of processes ends.

[0073] Meanwhile, in step 316, the CPU 20A displays the abnormal location on the display unit 28 based on the patterned result and proceeds to step 318. That is, the display processing unit 46 displays the abnormal pattern or the abnormal pattern represented by the pattern information received by the pattern receiving unit 44 on the display unit 28. This allows the occupant to recognize what kind of abnormal location it is based on the abnormal pattern, and the nature of the abnormality at the location can be accurately communicated to the occupant of the vehicle 14.

[0074] In step 318, the CPU 20A determines whether or not it has passed the abnormal location. It waits until this determination is confirmed and then proceeds to step 320.

[0075] In step 320, the CPU 20A finishes displaying the abnormal location on the display unit 28 and terminates the series of processes.

[0076] In this embodiment, the information processing system 10 uses a cloud server to pattern the abnormalities in the abnormal locations from the images of the abnormal locations collected from the in-vehicle device 16 and transmits this pattern to the in-vehicle device 16. This makes it possible to accurately identify the nature of the abnormalities in the abnormal locations and notify the occupants.

[0077] In the above embodiment, the processing of the information processing system 10 was described as an example, but it is not limited to the above, and some of the processing of the cloud server 12 may be performed by the in-vehicle device 16, or some of the processing of the in-vehicle device 16 may be performed by the cloud server 12. For example, in the above embodiment, when an abnormality is detected on the in-vehicle device 16 side, the location of the detected abnormality is compared with the location of the abnormality received from the cloud server 12, but when an abnormality is detected, the location of the abnormality may be sent to the cloud server 12, and the cloud server 12 side may perform the comparison of the locations of the abnormalities.

[0078] Furthermore, although the processing performed by the information processing system 10 in each of the above embodiments has been described as software processing performed by executing a program, it is not limited to this. For example, it may be processing performed by hardware such as a GPU (Graphics Processing Unit), ASIC (Application Specific Integrated Circuit), and FPGA (Field-Programmable Gate Array). Alternatively, it may be processing that combines both software and hardware. In the case of software processing, the program may be stored on various storage media and distributed.

[0079] Furthermore, the present invention is not limited to the above, and it is of course possible to implement it in various modified forms without departing from its spirit. [Explanation of symbols]

[0080] 10 Information Processing Systems 12 Cloud Servers 14 vehicles 16 Onboard equipment 20 Control Unit 22 Vehicle Information Detection Unit 24 Photography Department 28 Display section 40 Detection unit 42 Information Transmission Section 44 Pattern receiving section 46 Display Processing Unit 50 Receiver 52 Image Processing Unit 54 Transmitter 60 Vehicle Images 62 Image of the vehicle ahead 64 Abnormal Patterns

Claims

1. A receiving unit that receives location information and image information of an abnormal area from a vehicle that has detected an abnormal area while driving, A receiving unit patterns the image information it receives, stores it in a storage unit in association with the location information, and transmits the patterned abnormality pattern or pattern information representing the abnormality pattern, along with the location information, to a vehicle following the vehicle that detected the abnormality. Equipped with, When the following vehicle detects an abnormal location, the receiving unit receives location information and image information of the abnormal location detected by the following vehicle from the following vehicle. The transmitting unit compares the location information received from the following vehicle with the location information stored in the storage unit, and if the locations of both correspond, transmits the abnormal pattern or pattern information stored in the storage unit to the following vehicle. If the abnormal location is eliminated and the following vehicle does not detect the abnormal location, the transmitting unit prohibits the transmission of the abnormal pattern or pattern information to the following vehicle in subsequent instances.

2. The information processing apparatus according to Claim 1, wherein the transmitting unit transmits the abnormal pattern or the pattern information to the following vehicle in response to a transmission request transmitted from the following vehicle when the following vehicle detects an abnormal location.

3. The information processing apparatus according to claim 1, wherein the transmitting unit transmits an icon image, which is a pattern of the image information, to the following vehicle as the abnormal pattern.

4. When a detection unit that detects abnormal areas in front of the vehicle detects the abnormal area, an information transmission unit transmits image information representing a captured image of the abnormal area and location information representing the location of the abnormal area to an external device. A pattern receiving unit that receives an abnormal pattern obtained by patterning the image information or pattern information representing the abnormal pattern together with the position information from the external device, A display processing unit, which receives the abnormal pattern or pattern information from the external device by the pattern receiving unit and detects the abnormal location by the detection unit, and which, based on the comparison result between the location information of the abnormal location detected by the detection unit and the location information received by the pattern receiving unit, displays the abnormal pattern represented by the abnormal pattern or pattern information received by the pattern receiving unit on the display unit at a position corresponding to the location of the abnormal location detected by the detection unit; Equipped with, The information transmission unit is an in-vehicle device that transmits image information representing a captured image of a location and location information to the external device when the vehicle passes over a location represented by the location information received by the pattern receiving unit without the detection unit detecting the abnormal location.

5. The in-vehicle device according to claim 4, wherein the information transmission unit transmits the location information of the abnormal location detected by the detection unit and image information representing the captured image to the external device when the detection unit detects an abnormal location and the location information of the abnormal location detected by the detection unit does not correspond to the location information received by the pattern receiving unit.

Images