Information processing device and on-vehicle device

Abstract

To provide an information processing device and an on-vehicle device that can accurately notify an occupant on a vehicle about a specific anomaly in an abnormal portion.SOLUTION: A cloud server 12 comprises: a reception unit 50 that receives position information for an abnormal portion from a vehicle 14 that detected the abnormal portion during traveling and image information for the abnormal portion; an image processing unit 52 that patterns (e.g., a road caving, a tire as a fallen object, a wood material as a fallen object, or the like) the image information received by the reception unit 50; and a transmission unit 54 that transmits an abnormal pattern patterned by the image processing unit 52 or pattern information representing the abnormal pattern to a vehicle behind the vehicle that detected the abnormal portion corresponding to the abnormal pattern.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information processing apparatus and an in-vehicle apparatus.

Background Art

[0002] Patent Document 1 proposes a driving support system that detects a driver's line-of-sight position, line-of-sight circle, and field-of-view area from an imaging image captured by a driver camera, detects a feature in front of the vehicle's traveling direction from an imaging image captured by a front camera or feature object image information, identifies the driver's visual recognition mode with respect to the feature in front of the vehicle's traveling direction based on each detection result, and provides guidance regarding the feature based on the identified visual recognition mode.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] There is a technology in which a vehicle system determines from detection results what actual obstacles are on the road and depicts them on a display unit. In this case, it is difficult for the vehicle to discriminate abnormal locations such as obstacles, and the pattern of the depiction icons the vehicle has is limited. Therefore, the modes of depiction on the display are often limited to only a few types at most.

[0005] Also, it is desirable for the vehicle to accurately determine the abnormal content of an abnormal location (for example, the type of obstacle, the type of road abnormality such as a depression, etc.) and notify the occupant.

[0006] The present invention has been made in consideration of the above facts, and an object thereof is to provide an information processing apparatus and an in-vehicle apparatus capable of accurately notifying a vehicle occupant of the abnormal content of an abnormal location.

Means for Solving the Problems

[0007] The information processing apparatus according to the first aspect includes a receiving unit that receives position information of the abnormal location and image information of the abnormal location from a vehicle that has detected the abnormal location during traveling, and a transmitting unit that patterns the image information received by the receiving unit and transmits the patterned abnormal pattern or pattern information representing the abnormal pattern to a following vehicle of the vehicle that has detected the abnormal location together with the position information.

[0008] According to the first aspect, in the receiving unit, position information of the abnormal location and image information of the abnormal location are received from a vehicle that has detected the abnormal location during traveling. Then, in the transmitting unit, the image information received by the receiving unit is patterned, and the patterned abnormal pattern or pattern information representing the abnormal pattern is transmitted to a following vehicle of the vehicle that has detected the abnormal location together with the position information. Thereby, it becomes possible to accurately notify the vehicle occupants of the abnormal content of the abnormal location.

[0009] The information processing apparatus according to the second aspect is the information processing apparatus according to the first aspect, and when the abnormal location is eliminated and the following vehicle does not detect the abnormal location, transmission of the abnormal pattern to the following vehicle after the next time is prohibited.

[0010] According to the second aspect, it is possible to prevent the abnormal pattern from being misdisplayed in the following vehicle.

[0011] The information processing apparatus according to the third aspect is the information processing apparatus according to the first aspect or the second aspect, and the transmitting unit transmits an icon image obtained by patterning the image information to the following vehicle as the abnormal pattern.

[0012] According to the third aspect, it becomes possible to increase the description pattern of the abnormal location in the following vehicle.

[0013] When the in-vehicle device according to the fourth aspect detects an abnormal location in front of the vehicle, an information transmission unit that transmits to an external device image information representing a captured image of the abnormal location and position information representing the position of the abnormal location, a pattern reception unit that receives, from the external device, an abnormal pattern obtained by patterning the image information or pattern information representing the abnormal pattern together with the position information, and when the pattern reception unit receives the abnormal pattern or the pattern information from the external device, or when the detection unit detects the abnormal location, a display processing unit that causes the display unit to display the abnormal pattern represented by the abnormal pattern or the pattern information received by the pattern reception unit at a position corresponding to the position of the abnormal location detected by the detection unit.

[0014] According to the fourth aspect, the abnormal content of the abnormal location can be accurately notified to the vehicle occupants. In addition, the relative positional relationship between the abnormal location and the vehicle can be accurately displayed.

[0015] The in-vehicle device according to the fifth aspect is the in-vehicle device according to the fourth aspect, wherein the display processing unit causes the display unit to display the abnormal pattern based on a comparison result between the position information of the abnormal location detected by the detection unit and the position information received by the pattern reception unit.

[0016] According to the fifth aspect, it is possible to confirm whether the detected abnormal location is the received abnormal location, and it is possible to improve the display accuracy of the abnormal location.

[0017] Note that an information processing system including the information processing device according to any one of the first to third aspects and the in-vehicle device according to the fourth or fifth aspect may also be used.

[0018] Further, it may be a program for causing a computer to function as each part of the information processing device according to any one of the first to third aspects, or it may be a program for causing a computer to function as each part of the in-vehicle device according to the fourth or fifth aspect.

Advantages of the Invention

[0019] As described above, according to the present invention, it is possible to provide an information processing apparatus and an in-vehicle apparatus that can accurately notify a vehicle occupant of the abnormal content of an abnormal location.

Brief Description of the Drawings

[0020]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Modes for Carrying Out the Invention

[0021] Hereinafter, an example of an embodiment of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing a schematic configuration of an information processing system according to this embodiment.

[0022] The information processing system 10 according to this embodiment is connected via a communication network 18 with an in-vehicle device 16 as an example of an in-vehicle device mounted on a vehicle 14 and a cloud server 12 as an example of an information processing device. In FIG. 1, two vehicles 14 are shown, but in-vehicle devices 16 of three or more vehicles 14 are connected to the cloud server 12 via the communication network 18.

[0023] In the information processing system 10 according to this embodiment, the cloud server 12 collects and stores image information representing captured images captured by a plurality of vehicles 14 and vehicle information representing the state of each vehicle 14. Further, based on the stored information, the cloud server 12 identifies the content of abnormal locations such as obstacles or depressions on the road, and performs processing such as transmitting an abnormal pattern representing the content of the identified abnormal location, or pattern information representing the abnormal pattern, to the vehicles in motion.

[0024] The in-vehicle device 16 mounted on the vehicle 14 detects an abnormal location from a captured image in front, and when an abnormal location is detected, transmits the position of the abnormal location and the captured image to the cloud server 12. Further, the in-vehicle device 16 receives the abnormal pattern or pattern information transmitted from the cloud server 12, and when an abnormal location is detected in front, determines whether the position received from the cloud server 12 corresponds to the detected position. Then, when both positions correspond, the in-vehicle device 16 performs processing such as displaying the abnormal pattern received from the cloud server 12, or the abnormal pattern corresponding to the pattern information received from the cloud server 12.

[0025] Specifically, the in-vehicle device 16 includes a control unit 20, a vehicle information detection unit 22, a photographing unit 24, a communication unit 26, and a display unit 28.

[0026] The vehicle information detection unit 22 detects vehicle information regarding the vehicle 14. As an example of the vehicle information, for example, vehicle information such as the position information of the vehicle 14, vehicle speed, acceleration, steering angle, accelerator opening, distance to obstacles around the vehicle, and route is detected. Specifically, the vehicle information detection unit 22 can apply a plurality of types of sensors and devices that acquire information indicating the situation of the surrounding environment of the vehicle 14. Examples of the sensors and devices include sensors mounted on the vehicle 14 such as a vehicle speed sensor and an acceleration sensor, a GNSS (Global Navigation Satellite System) device, an in-vehicle communicator, a navigation system, an ultrasonic sensor, and a radar device. The GNSS device receives GNSS signals from a plurality of GNSS satellites and measures the position of the own vehicle 14. The positioning accuracy of the GNSS device improves as the number of receivable GNSS signals increases. The in-vehicle communicator is a communication device that performs at least one of vehicle-to-vehicle communication with other vehicles 14 and road-to-vehicle communication with roadside units via the communication unit 26. The navigation system includes a map information storage unit that stores map information, and based on the position information obtained from the GNSS device and the map information stored in the map information storage unit, it performs processes such as displaying the position of the own vehicle 14 on the map and guiding the route to the destination. Also, the radar device includes a plurality of radars with different detection ranges, detects objects such as pedestrians and other vehicles 14 existing around the own vehicle 14, and acquires the relative position and relative speed between the detected objects and the own vehicle 14. Further, the radar device incorporates a processing device that processes the detection results of surrounding objects. Based on changes in the relative position and relative speed with respect to individual objects included in the most recent multiple detection results, the processing device excludes roadside objects such as noise and guardrails from the monitoring targets and performs follow-up monitoring with pedestrians and other vehicles 14 as monitoring target objects. Then, the radar device outputs information such as the relative position and relative speed with respect to individual monitoring target objects. Note that the vehicle information detection unit 22 does not necessarily need to detect all of these vehicle information, and it may detect some of the vehicle information.

[0027] In this embodiment, the imaging unit 24 includes an outward-facing camera that images the outside of the vehicle and an inward-facing camera that images the interior of the vehicle, but only the outward-facing camera may be used. The imaging unit 24 images the periphery of the vehicle such as the front with the outward-facing camera, and generates outward-facing moving image data representing the captured image of the moving image as image information. Further, the imaging unit 24 images the driver in the vehicle interior with the inward-facing camera, and generates inward-facing moving image data representing the captured image of the moving image as image information. For example, a drive recorder or the like can be applied as the outward-facing camera, and a driver monitor camera or the like can be applied as the inward-facing camera. Note that the outward-facing camera may further image at least one of the side and the rear of the vehicle 14.

[0028] The communication unit 26 establishes communication with the cloud server 12 via the communication network 18, and transmits and receives information such as the image information obtained by the imaging of the imaging unit 24 and the vehicle information detected by the vehicle information detection unit 22 as CAN (Controller Area Network) data.

[0029] The display unit 28 provides various information to the passengers by displaying the information. In this embodiment, for example, the information provided from the cloud server 12 and the like is displayed. The display unit 28 may apply, for example, a combination meter, or a display provided on the instrument panel or the like.

[0030] As shown in FIG. 2, the control unit 20 is composed of a general microcomputer including a CPU (Central Processing Unit) 20A, a ROM (Read Only Memory) 20B, a RAM (Random Access Memory) 20C, a storage 20D, an interface (I / F) 20E, a bus 20F, and the like.

[0031] The control unit 20 performs control such as uploading the image information of the moving image representing the captured image captured by the imaging unit 24 and the vehicle information detected by the vehicle information detection unit 22 at the time of image capture to the cloud server 12. When uploading the image information and the vehicle information, identification information for identifying at least one of the vehicle and the driver is attached and transmitted. The information for identifying the driver may be, for example, a captured image of the driver, the identification information of the smart key carried by the driver, or other information capable of identifying the driver.

[0032] Further, the control unit 20 functions as a detection unit 40, an information transmission unit 42, a pattern reception unit 44, and a display processing unit 46 by expanding and executing the program stored in the ROM 20B by the CPU 20A in the RAM 20C.

[0033] The detection unit 40 detects an abnormal portion (for example, a road depression, various obstacles, etc.) in front of the vehicle 14 based on at least one of the captured image of the imaging unit 24 and the detection result of the vehicle information detection unit 22. For example, the abnormal portion may be detected from the captured image by simple image recognition, or the abnormal portion may be detected by an ultrasonic sensor or a radar device.

[0034] When the detection unit 40 detects an abnormal portion, the information transmission unit 42 transmits the image information representing the captured image of the front of the vehicle 14 captured by the imaging unit 24 and the position information representing the position of the abnormal portion to the cloud server 12 as an external device. The position information is corrected by the position information representing the own vehicle position detected by the vehicle information detection unit 22 to the position of the abnormal portion detected by the detection unit 40 and transmitted to the cloud server 12. For example, based on a radar device, an ultrasonic sensor, a captured image, etc., the direction and distance of the abnormal portion from the own vehicle position are obtained, and the position information of the own vehicle position is corrected to derive the position information of the abnormal portion.

[0035] The pattern reception unit 44 receives from the cloud server 12 an abnormal pattern obtained by patterning the image information in the cloud server 12 or pattern information representing the abnormal pattern.

[0036] When the pattern receiving unit 44 receives an abnormal pattern or pattern information from the cloud server 12, or when the detection unit 40 detects an abnormal location, the display processing unit 46 performs a process of causing the display unit 28 to display the abnormal pattern represented by the abnormal pattern or pattern information received by the pattern receiving unit 44 at a position corresponding to the position of the abnormal location detected by the detection unit 40. Hereinafter, the case where the display unit 28 displays the abnormal pattern when the pattern receiving unit 44 receives the abnormal pattern or pattern information by push transmission from the cloud server 12 will be described. Further, in detail, the display processing unit 46 causes the display unit 28 to display the abnormal pattern based on the comparison result between the position information of the abnormal location detected by the detection unit 40 and the position information received by the pattern receiving unit 44. When the pattern receiving unit 44 receives pattern information, the display processing unit 46 may prepare in advance a plurality of types of icon images representing the abnormal pattern and cause the display unit 28 to display the icon image corresponding to the pattern information.

[0037] On the other hand, the cloud server 12 includes a central processing unit 30, a central communication unit 36, and a DB (database) 38.

[0038] As shown in FIG. 3, the central processing unit 30 is composed of a general microcomputer including a CPU 30A, a ROM 30B, a RAM 30C, a storage 30D, an interface (I / F) 30E, a bus 30F, and the like.

[0039] The central processing unit 30 functions as a receiving unit 50, an image processing unit 52, and a transmitting unit 54 by expanding and executing the program stored in the ROM 30B by the CPU 30A in the RAM 30C.

[0040] The receiving unit 50 receives the position information of the abnormal location and the image information of the abnormal location from the vehicle 14 that has detected the abnormal location during traveling, and stores them in the database (DB) 38.

[0041] The image processing unit 52 patterns the image information received by the receiving unit 50 (for example, road depression, fallen object tire, fallen object wood, etc.). For example, the image processing unit 52 uses a well-known image recognition model to identify abnormal locations and the types of abnormal locations in the captured image from the captured image received by the receiving unit 50. Then, a corresponding abnormal pattern is selected from predetermined abnormal patterns according to the content of the abnormal location. For example, as abnormal patterns, icon images such as a road depression icon, a fallen object tire, and a fallen object wood icon may be prepared in advance, and an icon corresponding to the content of the identified abnormal location may be selected as the abnormal pattern. Alternatively, an icon image may be generated from the image of the identified abnormal location and used as the abnormal pattern.

[0042] The transmitting unit 54 transmits the abnormal pattern patterned by the image processing unit 52 or pattern information representing the abnormal pattern to the following vehicle of the vehicle that detected the abnormal location corresponding to the abnormal pattern. Note that by transmitting an icon image representing the abnormal pattern as the abnormal pattern by the transmitting unit 54, it is possible to increase the description pattern of the abnormal location in the following vehicle.

[0043] The central communication unit 36 establishes communication with the in-vehicle device 16 via the communication network 18 and performs transmission and reception of information such as image information and vehicle information.

[0044] The database 38 stores vehicle information such as image information and position information received by the receiving unit 50 from a plurality of vehicles 14.

[0045] Subsequently, the operations of each part of the information processing system 10 according to the present embodiment will be described. FIG. 3 is a diagram for explaining the operations of each part of the information processing system 10 according to the present embodiment.

[0046] In each vehicle 14, when the detection unit 40 detects an abnormal location, image information representing a captured image of the abnormal location and the location information of the abnormal location are transmitted to the cloud server 12 as data during driving. As a result, in the cloud server 12, the image information and the location information of the abnormal locations detected by the detection unit 40 of each vehicle 14 are stored in the database 38.

[0047] In the cloud server 12, a process of analyzing the image information collected from the vehicle 14 and patternizing the abnormal locations is performed. For example, as shown in FIG. 3, patternization is performed such as abnormal pattern A, abnormal pattern B, etc. Here, an example of selecting an icon image corresponding to the abnormal location from among predetermined icons is shown. Then, the cloud server 12 stores the patternized abnormal patterns together with the location information in the database 38. For example, as shown in FIG. 3, abnormal information 1 (coordinates (x1, y1, z1), abnormal pattern A), abnormal information 2 (coordinates (x2, y2, z2), abnormal pattern A), abnormal information 3 (x3, y3, z3), abnormal pattern B) are stored in the database 38.

[0048] As a result, when a subsequent vehicle 14 travels to the location of the abnormal location stored in the database 38 and the detection unit 40 of the subsequent vehicle 14 detects an abnormality, the image information of the abnormal location and the abnormal pattern when the abnormal location was detected during driving are compared to determine an appropriate abnormal pattern. Specifically, the cloud server 12 compares the location information of the image information when the abnormal location was detected with the location coordinates of the abnormal pattern stored in the database 38. When the locations of both correspond, the abnormal pattern or the pattern information representing the abnormal pattern is transmitted from the cloud server 12 to the vehicle 14 that detected the abnormal location. As a result, in the vehicle 14, the abnormal pattern is displayed on the display unit 28, and the passenger can recognize what kind of abnormal location it is based on the abnormal pattern. Therefore, the abnormal content of the abnormal location can be accurately notified to the passengers of the vehicle 14.

[0049] When displaying the abnormal pattern on the display unit 28, the relative position of the abnormal part with respect to the vehicle 14 is also displayed as a graphic on the screen that displays the surrounding environment in real time. For example, as shown in FIG. 4, a vehicle icon 60 representing the host vehicle is displayed, and a preceding vehicle icon 62 representing the preceding vehicle ahead is displayed, or an abnormal pattern icon 64 representing the abnormal pattern is displayed as an icon image. FIG. 4 is a diagram showing an example of the display on the display unit 28.

[0050] Also, when the abnormality is eliminated, as shown in FIG. 5, the traveling data (image information and position information) detected by the vehicle 14 that has passed through the position registered as the abnormal part is compared with the abnormal information (image information and position information) stored in the cloud server 12. Then, when it is determined that the abnormality has been eliminated, as shown in the in-vehicle display of FIG. 5, the display of the abnormal pattern icon 64 on the display unit 28 is not performed and is made non-display. Note that FIG. 5 is a diagram for explaining the operation of each part when the abnormality is eliminated in the information processing system 10 according to the present embodiment.

[0051] Next, specific processing performed by each part of the information processing system 10 according to the present embodiment will be described.

[0052] First, the processing when transmitting abnormal data (image information and position information) to the cloud server 12 when an abnormality is detected in the in-vehicle device 16 of the vehicle 14 will be described. FIG. 6 is a flowchart showing an example of the processing flow when transmitting abnormal data (image information and position information) to the cloud server 12 when an abnormality is detected in the in-vehicle device 16 of the information processing system 10 according to the present embodiment. Note that the processing in FIG. 6 starts, for example, when an ignition switch (not shown) of the vehicle 14 is turned on.

[0053] In step 100, the CPU 20A acquires vehicle information and a captured image and proceeds to step 102. That is, the vehicle information detected by the vehicle information detection unit 22 and the image information representing the captured image captured by the imaging unit 24 are acquired.

[0054] In step 102, the CPU 20A determines whether an abnormal location has been detected. This determination is made, for example, by the detection unit 40 determining whether an abnormal location (e.g., a road depression, various obstacles, etc.) in front of the vehicle 14 has been detected based on at least one of the captured image of the imaging unit 24 and the detection result of the vehicle information detection unit 22. If this determination is affirmed, the process proceeds to step 104; if it is negated, the process proceeds to step 106.

[0055] In step 104, the CPU 20A transmits the position information at the time of abnormal detection and the image information representing the captured image to the cloud server 12 and proceeds to step 106. Note that the position information transmitted to the cloud server 12 is the position information of the abnormal location obtained by correcting the direction and distance of the obstacle detected by the detection unit 40 with respect to the position information of the host vehicle.

[0056] In step 106, the CPU 20A determines whether to end the process. This determination is made, for example, by determining whether an ignition switch (not shown) has been turned off. If this determination is negated, the process returns to step 100 to repeat the above-described process, and when the determination is affirmed, the series of processes ends.

[0057] Subsequently, the processing performed by the cloud server 12 when abnormal data (image information and position information) is transmitted from the in-vehicle device 16 will be described. FIG. 7 is a flowchart showing an example of the flow of processing performed by the cloud server 12 when abnormal data (image information and position information) is transmitted from the in-vehicle device 16 in the information processing system 10 according to the present embodiment. Note that FIG. 7 starts, for example, when the transmission of abnormal data is performed from the in-vehicle device 16.

[0058] In step 200, the CPU 30A receives the image information representing the captured image of the abnormal location and the position information of the abnormal location from the in-vehicle device 16 and proceeds to step 202.

[0059] In step 202, the CPU 30A determines whether it has received a captured image without any abnormal areas. This determination is to determine whether it has received a captured image and position information without any abnormal areas. If it is determined that there is an abnormal area in the captured image and the determination is negative, the process proceeds to step 204. If it is determined that there is no abnormal area in the captured image and the determination is positive, the process proceeds to step 210.

[0060] In step 204, the CPU 30A accumulates the received data in the database 38 and proceeds to step 206. That is, the image information and position information of the abnormal area received from the vehicle-mounted device 16 are stored in the database 38.

[0061] In step 206, the CPU 30A performs abnormal area patterning and proceeds to step 208. For abnormal area patterning, the image processing unit 52 patterns the image information received by the receiving unit 50 (for example, road depression, fallen object tire, fallen object wood, etc.). For example, the image processing unit 52 identifies the abnormal area and the type of abnormal area in the captured image from the captured image received by the receiving unit 50 using a well-known image recognition model. Then, a corresponding abnormal pattern is selected from the pre-determined abnormal patterns according to the content of the abnormal area.

[0062] In step 208, the CPU 30A transmits the patterned result including the position information to the subsequent vehicle 14 traveling through the abnormal area, and ends the series of processes. In this embodiment, the patterned result is transmitted to the vehicle 14 by push notification from the cloud server 12, but it is not limited to this, and it may be pulled and received from the vehicle 14 side. That is, when the vehicle 14 side detects an abnormal area, the vehicle 14 that is the requester may receive the patterned result from the cloud server 12 by sending a transmission request to the cloud server 12.

[0063] On the other hand, in step 210, the CPU 30A updates the abnormal data registered in the database 38 as if the abnormality has been eliminated, and ends a series of processes. For example, transmission of the abnormal pattern to the subsequent vehicle 14 after the next time is prohibited, and the abnormal pattern and the position information of the abnormal location registered in the database 38 are deleted without transmitting the abnormal pattern to the vehicle 14.

[0064] Subsequently, the processing performed by the in-vehicle device 16 when the patterned result is transmitted from the cloud server 12 will be described. FIG. 8 is a flowchart showing an example of the flow of processing performed by the in-vehicle device 16 when the patterned result is transmitted from the cloud server 12 in the information processing system 10 according to the present embodiment. Note that the processing in FIG. 8 starts when the patterned result is transmitted from the cloud server 12.

[0065] In step 300, the CPU 20A receives the patterned result including the abnormal information from the cloud server 12 and proceeds to step 302.

[0066] In step 302, the CPU 20A acquires the vehicle information and the captured image and proceeds to step 304. That is, the vehicle information detected by the vehicle information detection unit 22 and the image information representing the captured image captured by the imaging unit 24 are acquired.

[0067] In step 304, the CPU 20A determines whether or not an abnormal location has been detected. This determination is made, for example, by the detection unit 40 determining whether or not an abnormal location (for example, a road depression, various obstacles, etc.) in front of the vehicle 14 has been detected based on at least one of the captured image of the imaging unit 24 and the detection result of the vehicle information detection unit 22. If the determination is negative, the process proceeds to step 306, and if the determination is positive, the process proceeds to step 310.

[0068] In step 306, the CPU 20A determines whether it has passed through the location represented by the position information. This determination is to determine whether it has passed through the position of the abnormal location notified by the cloud server 12 without detecting an abnormality. If this determination is affirmed, the process proceeds to step 308; if it is negated, the process returns to step 302 and the above-described processing is repeated.

[0069] In step 308, the CPU 20A transmits the captured image of the location represented by the position information to the cloud server 12 as the captured image of the abnormal location together with the position information, and ends a series of processes. As a result, in the cloud server 12, since the determination in step 202 described above is affirmed, the database 38 is updated in step 210 as if the abnormality has been eliminated.

[0070] On the other hand, in step 310, the CPU 20A compares the positions of the abnormal locations and proceeds to step 312. That is, it compares the position information transmitted from the cloud server 12 with the position information of the abnormal location where the abnormality was detected.

[0071] In step 312, the CPU 20A determines whether the positions of the abnormal locations match. This determination is to determine whether an abnormality was detected at the position corresponding to the position information transmitted from the cloud server 12. If this determination is negated, the process proceeds to step 314; if it is affirmed, the process proceeds to step 316.

[0072] In step 314, the CPU 20A transmits the position information and the captured image at the time of abnormality detection to the cloud server 12 and ends a series of processes.

[0073] On the other hand, in step 316, the CPU 20A displays the abnormal location on the display unit 28 based on the patterned result and proceeds to step 318. That is, the display processing unit 46 displays the abnormal pattern received by the pattern receiving unit 44 or the abnormal pattern represented by the pattern information on the display unit 28. As a result, the occupant can recognize what kind of abnormal location it is from the abnormal pattern, and the abnormal content of the abnormal location can be accurately notified to the occupant of the vehicle 14.

[0074] In step 318, the CPU 20A determines whether it has passed through the abnormal location. It waits until this determination is affirmed and then proceeds to step 320.

[0075] In step 320, the CPU 20A ends the display on the display unit 28 of the abnormal location and ends the series of processes.

[0076] As described above, in the information processing system 10 according to this embodiment, since the cloud server patterns the abnormality of the abnormal location from the captured image of the abnormal location collected from the in-vehicle device 16 and transmits it to the in-vehicle device 16, it is possible to accurately identify the content of the abnormality of the abnormal location and notify the passenger.

[0077] In the above embodiment, the processing of the information processing system 10 has been described as an example, but it is not limited to the above. A part of the processing of the cloud server 12 may be executed by the in-vehicle device 16, or a part of the processing of the in-vehicle device 16 may be executed by the cloud server 12. For example, in the above embodiment, when an abnormal location is detected on the in-vehicle device 16 side, the position of the detected abnormal location is compared with the position of the abnormal location received from the cloud server 12. However, when an abnormal location is detected, the position of the abnormal location may be transmitted to the cloud server 12, and the cloud server 12 may compare the positions of the abnormal locations.

[0078] Also, the processing performed by the information processing system 10 in each of the above embodiments has been described as software processing performed by executing a program, but it is not limited to this. For example, it may be processing performed by hardware such as a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), and an FPGA (Field-Programmable Gate Array). Alternatively, it may be processing that combines both software and hardware. Also, when it is software processing, the program may be stored in various storage media and distributed.

[0079] Furthermore, the present invention is not limited to the above, and it goes without saying that various modifications can be made and implemented within the scope without departing from the gist of the invention other than the above.

Explanation of Reference Numerals

[0080] 10 Information processing system 12 Cloud server 14 Vehicle 16 On-vehicle device 20 Control unit 22 Vehicle information detection unit 24 Photographing unit 28 Display unit 40 Detection unit 42 Information transmission unit 44 Pattern reception unit 46 Display processing unit 50 Reception unit 52 Image processing unit 54 Transmission unit 60 Vehicle image 62 Front vehicle image 64 Abnormal pattern

Claims

1. A receiving unit that receives position information of the abnormal location and image information of the abnormal location from a vehicle that has detected an abnormal location during travel; A transmitting unit that patterns the image information received by the receiving unit and transmits the patterned abnormal pattern or pattern information representing the abnormal pattern to a following vehicle of the vehicle that has detected the abnormal location together with the position information; An information processing apparatus comprising the above.

2. The information processing apparatus according to claim 1, wherein when the abnormal location is eliminated and the following vehicle does not detect the abnormal location, transmission of the abnormal pattern to the following vehicle after the next time is prohibited.

3. The information processing apparatus according to claim 1, wherein the transmitting unit transmits an icon image obtained by patterning the image information to the following vehicle as the abnormal pattern.

4. An information transmitting unit that transmits, to an external device, image information representing a captured image of an abnormal location and position information representing the position of the abnormal location when the abnormal location is detected by a detection unit that detects an abnormal location in front of the vehicle; A pattern receiving unit that receives, from the external device, an abnormal pattern obtained by patterning the image information or pattern information representing the abnormal pattern together with the position information; A display processing unit that causes the display unit to display the abnormal pattern represented by the abnormal pattern or the pattern information received by the pattern receiving unit at a position corresponding to the position of the abnormal location detected by the detection unit when the abnormal pattern or the pattern information is received from the external device by the pattern receiving unit, or when the abnormal location is detected by the detection unit; An in-vehicle device comprising the above.

5. The in-vehicle device according to claim 4, wherein the display processing unit causes the display unit to display the abnormal pattern based on a comparison result between the position information of the abnormal location detected by the detection unit and the position information received by the pattern receiving unit.

Images