Vehicle Service Decision System

Abstract

To provide an abnormality detection system for detecting a vehicle in which an abnormality occurs.SOLUTION: An abnormality detection system 1 includes: an information acquisition section 10 for receiving information which is transmitted from each of a plurality of vehicles and indicates positions of the vehicles and images of the periphery of the vehicles; and an abnormal vehicle detection section 20 for detecting an abnormal vehicle in which an abnormality occurs from the plurality of vehicles based on the acquired respective positions of the vehicles and the images of the periphery of the vehicles. The abnormal vehicle detection section 20 performs: detecting a second vehicle included in the plurality of vehicles from images of the periphery of a first vehicle included in the plurality of vehicles; extracting a difference between a normal image being an image of the second vehicle when no abnormality occurs in the second vehicle and the images of the periphery of the first vehicle; and determining whether or not the second vehicle is the abnormal vehicle based on the difference.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an anomaly detection system and a vehicle service decision system. [Background technology]

[0002] Active sensors including radar or LIDAR that are mounted on moving bodies such as autonomous vehicles and acquire information related to the surrounding environment of the moving body have been known (Patent Document 1). The system described in Patent Document 1 changes the resolution and field of view of the active sensor by changing the power configuration of the active sensor based on the operating status of the moving body. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] WO 19 / 173155 Summary of the Invention [Problem to be solved by the invention]

[0004] However, while active sensors mounted on a moving object can properly recognize the situation around the vehicle, they cannot recognize the state of the vehicle itself. Therefore, the system described in Patent Document 1 is likely to take a long time to detect an abnormality when one occurs in the vehicle.

[0005] The present invention has been made in view of the above-mentioned problems, and has as its object to provide an abnormality detection system that can detect a vehicle in which an abnormality has occurred. [Means for solving the problem]

[0006] An abnormality detection system according to one aspect of the present invention includes an information acquisition unit that receives information indicating the position of each of a plurality of vehicles and an image of the surroundings, transmitted from each of the plurality of vehicles, and an abnormal vehicle detection unit that detects an abnormal vehicle from among the plurality of vehicles in which an abnormality has occurred, based on the acquired image of the position and surroundings of each of the plurality of vehicles. The abnormal vehicle detection unit detects a second vehicle from among the images of the surroundings of a first vehicle, extracts a difference between a normal image, which is an image of the second vehicle when no abnormality has occurred in the second vehicle, and the image of the surroundings of the first vehicle, and determines whether the second vehicle is an abnormal vehicle based on the difference. [Effects of the Invention]

[0007] According to the present invention, a vehicle in which an abnormality has occurred can be detected. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of an anomaly detection system according to the first embodiment. [Figure 2] FIG. 2 is a schematic diagram showing an example of a traffic scene in which the anomaly detection system according to the first embodiment detects a second vehicle from an image of the surroundings of a first vehicle. [Figure 3] FIG. 3 is a schematic diagram showing an example of the type of abnormality identified by the abnormality detection system according to the first embodiment and the content of a notification to an abnormal vehicle determined based on the type of abnormality. [Figure 4] FIG. 4 is a flowchart showing an example of processing performed by the abnormality detection system according to the first embodiment, from identifying an abnormal vehicle to notifying the abnormal vehicle of an abnormality. [Figure 5] FIG. 5 is a block diagram showing an example of the configuration of a vehicle service determination system according to the second embodiment. [Figure 6] FIG. 6 is a diagram showing an example of the service content to be provided to an abnormal vehicle, which is determined by the vehicle service decision system according to the second embodiment based on the type of abnormality. [Figure 7A]FIG. 7A is a flowchart (part 1) showing an example of processing up to when the vehicle service decision system according to the second embodiment outputs a maintenance instruction to an abnormal vehicle. [Figure 7B] FIG. 7B is a flowchart (part 2) showing an example of the processing up to when the vehicle service decision system according to the second embodiment outputs a maintenance instruction to an abnormal vehicle. DETAILED DESCRIPTION OF THE INVENTION

[0009] The embodiments will be described with reference to the drawings. In the description of the drawings, the same parts are designated by the same reference numerals and the description thereof will be omitted.

[0010] (First embodiment) [Configuration of anomaly detection system] The configuration of an anomaly detection system 1 according to a first embodiment will be described with reference to FIG. 1. The anomaly detection system 1 is a system that receives information indicating the position of each of the multiple vehicles and an image of the surroundings of each of the multiple vehicles, transmitted from each of the multiple vehicles, and detects an abnormal vehicle from among the multiple vehicles in which an abnormality has occurred, based on the position of each of the multiple vehicles and the image of the surroundings of each of the multiple vehicles. The anomaly detection system 1 detects an abnormal vehicle from among the multiple vehicles registered in the anomaly detection system 1. The anomaly detection system 1 is used for a service that notifies users of vehicle maintenance, which is one of the connected services provided by automobile manufacturers, or for maintenance management of vehicles used by transportation service providers.

[0011] The abnormality detection system 1 includes an information acquisition unit 10, an abnormal vehicle detection unit 20, a storage unit 30, and an abnormality notification unit 40. The abnormality detection system 1 is installed in an operations center that responds to reports from vehicles of accidents or sudden illnesses of occupants, but the installation location is not limited to this. The abnormality detection system 1 only needs to be installed in a location where it can receive information indicating the vehicle's position and an image of the surrounding area, transmitted from each of multiple vehicles.

[0012] The information acquisition unit 10 is a receiving device that receives information indicating the vehicle's position and an image of the surroundings, transmitted from each of the multiple vehicles. The information indicating the vehicle's position is GPS data (absolute position) received by a GPS receiver mounted on the vehicle. The information indicating the image of the vehicle's surroundings is image data captured by a camera mounted on the vehicle. The information acquisition unit 10 receives the information indicating the vehicle's position and an image of the surroundings, transmitted from each of the multiple vehicles, and transfers the received information to the abnormal vehicle detection unit 20.

[0013] The abnormal vehicle detection unit 20 is a computing device including a general-purpose server or microcomputer equipped with a CPU (Central Processing Unit), memories (storage units) such as RAM and ROM, and input / output units. A computer program for functioning as the abnormality detection system 1 is installed in the computing device. By executing the computer program, the computing device functions as the multiple information processing circuits (21, 22, 23, 24) equipped in the abnormality detection system 1. Note that, although this embodiment shows an example in which the multiple information processing circuits (21, 22, 23, 24) equipped in the abnormality detection system 1 are realized by software, it is also possible to configure the information processing circuits by providing dedicated hardware for executing each information processing. Furthermore, the multiple information processing circuits may be configured as individual hardware.

[0014] The abnormal vehicle detection unit 20 identifies an abnormal vehicle from among the multiple vehicles, based on the acquired information indicating the position of each of the multiple vehicles and the image of the surroundings of each of the multiple vehicles. The abnormal vehicle detection unit 20 also identifies the type of abnormality occurring in the abnormal vehicle, and determines a method of dealing with the abnormality based on the identified type of abnormality.

[0015] The storage unit 30 is an information storage device including an HDD (hard disk drive) and an SSD (solid state drive). The storage unit 30 pre-stores map data and normal images of each of the multiple vehicles when no abnormalities occur. Furthermore, the storage unit 30 stores information indicating the vehicle's position and surrounding images received from each of the multiple vehicles. The normal images of each of the multiple vehicles pre-stored in the storage unit 30 include images of the front, side, rear, and top of the vehicle. Therefore, the normal images of each of the multiple vehicles may be three-dimensional images of the vehicle. While the abnormality detection system 1 includes the storage unit 30 in this embodiment, if the memory capacity of the abnormal vehicle detection unit 20 is sufficient to store this data, the storage unit 30 is not required.

[0016] The abnormality notification unit 40 is a transmitting device that transmits, to the abnormal vehicle identified by the abnormal vehicle detection unit 20, information indicating the type of abnormality occurring in the abnormal vehicle and how to deal with the abnormality. The abnormality notification unit 40 may also transmit the information indicating the type of abnormality and how to deal with the abnormality to a mobile terminal of a user riding in the abnormal vehicle. Specifically, the abnormality notification unit 40 transmits to the abnormal vehicle a signal instructing a display unit mounted on the vehicle to display the type of abnormality and how to deal with the abnormality.

[0017] Next, a specific description will be given of the multiple information processing circuits included in the abnormal vehicle detection unit 20. The abnormal vehicle detection unit 20 includes a vehicle detection unit 21, a difference extraction unit 22, an abnormality type determination unit 23, and an abnormal vehicle identification unit 24.

[0018] The vehicle detection unit 21 acquires image data of the surroundings of each of the multiple vehicles from the information acquisition unit 10, and detects a second vehicle included in the multiple vehicles from the image of the surroundings of a first vehicle included in the multiple vehicles. Note that the first vehicle in this embodiment includes all of the multiple vehicles except for the second vehicle. The vehicle detection unit 21 can detect vehicles from the acquired image of the surroundings of the first vehicle by using known image recognition technology. The vehicle detection unit 21 determines whether the vehicle detected from the image of the surroundings of the first vehicle is the second vehicle included in the multiple vehicles based on the position information of each of the multiple vehicles.

[0019] Specifically, the vehicle detection unit 21 acquires the absolute position of a first vehicle and the relative position of a vehicle detected from an image of the surroundings of the first vehicle relative to the first vehicle, and identifies the absolute position of the detected vehicle based on the absolute position of the first vehicle and the relative position of the detected vehicle relative to the first vehicle. The vehicle detection unit 21 then acquires the absolute position of each of a plurality of vehicles and determines whether a second vehicle is present at the absolute position of the detected vehicle based on the acquired absolute positions of each of the plurality of vehicles. If the vehicle detection unit 21 determines that a second vehicle is present at the position of the detected vehicle, it determines that the detected vehicle is the second vehicle. This allows the vehicle detection unit 21 to detect the second vehicle from the image of the surroundings of the first vehicle. The vehicle detection unit 21 transfers the image of the surroundings of the first vehicle when the second vehicle is detected (hereinafter also referred to as the "image in which the second vehicle is detected") to the difference extraction unit 22.

[0020] The processing method for detecting a second vehicle from an image of the surroundings of a first vehicle is not limited to the above processing method. For example, by storing the license plate number of a vehicle using the anomaly detection system in advance in the storage unit 30, it is possible to determine whether the detected vehicle is the second vehicle. Also, by storing the vehicle identification number of a vehicle using the anomaly detection system in advance in the storage unit 30 and attaching the vehicle identification number to the front, rear, left, and right of the vehicle, it is possible to determine whether the detected vehicle is the second vehicle. It is preferable that the vehicle identification number be written as an encrypted barcode that can only be decoded by the anomaly detection system. This makes it possible to prevent the leakage of information regarding personal information identified from the vehicle identification number.

[0021] Here, referring to FIG. 2, an example of an actual traffic scene in which the vehicle detection unit 21 detects a second vehicle from an image of the surroundings of a first vehicle will be described. FIG. 2 shows an example of a traffic scene in which the vehicle detection unit 21 detects a second vehicle from an image of the surroundings of a first vehicle. In the traffic scene shown in FIG. 2, vehicles V1 and V3 are defined as first vehicles, and vehicle V2 is defined as a second vehicle. The traffic scene will describe a situation in which vehicle V2 (second vehicle) is detected from an image acquired by a camera (hereinafter referred to as a "front camera") mounted on vehicle V1 and vehicle V3 (first vehicle) to capture an image of the area in front of the vehicle. However, in this traffic scene, vehicle V1 and vehicle V3 could also be the second vehicle. Specifically, vehicle V1 may be imaged by a front camera mounted on vehicle V2, and vehicle V3 may be imaged by a camera mounted on the side of vehicle V1 and vehicle V2. Therefore, all vehicles (vehicles V1 to V3) are in a situation where they can capture images of other vehicles other than their own vehicle using the cameras mounted on each vehicle, and all vehicles can be both first and second vehicles. That is, in a scene where vehicles pass each other or where the paths of the vehicles intersect, each vehicle is both the first and second vehicle.

[0022] As shown in FIG. 2, vehicles V1, V2, and V3 are traveling toward their respective fronts (f1, f2, f3). More specifically, vehicles V1 and V2 are traveling in opposing lanes, and vehicle V3 is traveling in a lane intersecting the lane on which vehicles V1 and V2 are traveling. Vehicle V3 is waiting for vehicles V1 and V2 to pass through the intersection, and is decelerating toward the intersection or stopped before the intersection. The dashed lines extending from the windshields of vehicles V1 and V3 (first vehicles) indicate the viewing angles (Va1, Va3) of the front cameras mounted on vehicles V1 and V3. The front cameras mounted on the vehicles continuously capture images of the area ahead of their respective vehicles at a predetermined cycle. Vehicles V1 and V3 continuously transmit the captured images to the anomaly detection system at a predetermined cycle. In this traffic scene, vehicle V2 (a second vehicle) is included in the view angle Va1 of the front camera of vehicle V1 and the view angle Va3 of the front camera of vehicle V3. Therefore, vehicle V2 is captured in the images captured by the front cameras of vehicle V1 and vehicle V3. The vehicle detection unit 21 acquires the absolute position of vehicle V1, the relative position of vehicle V2 captured in the image of the periphery of vehicle V1, the absolute position of vehicle V3, and the relative position of vehicle V2 captured in the image of the periphery of vehicle V3, and identifies the absolute position of vehicle V2 based on this acquired position information. If the vehicle detection unit 21 determines that a second vehicle is present at the identified absolute position of vehicle V2 based on the acquired absolute positions of each of the multiple vehicles, it determines that vehicle V2 is the second vehicle. Therefore, in a traffic scene such as that shown in FIG. 2, the vehicle detection unit 21 can detect the second vehicle (vehicle V2) from the images captured by the first vehicles (vehicle V1 and vehicle V3). In this embodiment, the vehicle detection unit 21 executes the process of detecting the second vehicle from the images received from each of the multiple vehicles (first vehicles) in real time as the images are acquired, but the timing of executing the process of detecting the second vehicle is not limited to this. For example, the received images may be stored in the storage unit 30, and the vehicle detection unit 21 may execute the process of detecting the second vehicle from the stored images at predetermined intervals.

[0023] The difference extraction unit 22 extracts the difference between a normal image, which is an image of the second vehicle when no abnormality occurs in the second vehicle, and an image of the surroundings of the first vehicle. Specifically, the difference extraction unit 22 acquires an image of the surroundings of the first vehicle when the second vehicle is detected from the vehicle detection unit 21, and acquires a normal image of the second vehicle when no abnormality occurs in the second vehicle from the storage unit 30. The difference extraction unit 22 uses a known image processing technique to superimpose and match the second vehicle shown in the image of the surroundings of the first vehicle onto the normal image of the second vehicle. The difference extraction unit 22 can use, for example, affine transformation as a known image processing technique. By using the affine transformation, the difference extraction unit 22 can freely rotate the image of the surroundings of the first vehicle at any coordinate position and superimpose and match the second vehicle shown in the image of the surroundings of the first vehicle onto the normal image of the second vehicle. The difference extraction unit 22 compares the second vehicle shown in the image of the surroundings of the first vehicle with a normal image of the second vehicle in an overlaid state, and extracts the difference between the normal image of the second vehicle and the second vehicle shown in the image of the surroundings of the first vehicle. Note that the difference extraction unit 22 can extract the difference between the images by using, for example, background subtraction as a known image processing technique.

[0024] Furthermore, the difference extraction unit 22 acquires from the storage unit 30 images of the surroundings of the first vehicle captured before and after the second vehicle passes a predetermined point, and extracts the difference between the images of the surroundings of the first vehicle captured before and after the second vehicle passes the predetermined point. The process of extracting the difference is the same as the process of extracting the difference between the normal image of the second vehicle and the second vehicle captured in the image of the surroundings of the first vehicle. The difference extraction unit 22 identifies the pixel position from which the difference was extracted and calculates the difference in brightness at the pixel position from which the difference was extracted. The difference extraction unit 22 transfers the pixel position from which the difference was extracted and the difference in brightness at the pixel position from which the difference was extracted as difference information to the abnormality type determination unit 23.

[0025] The abnormality type determination unit 23 acquires the difference information from the difference extraction unit 22, and identifies the type of abnormality occurring in the second vehicle based on the pixel position where the difference was extracted and the difference in brightness at the pixel position where the difference was extracted. The abnormality type determination unit 23 determines a method of dealing with the abnormality to be notified to the second vehicle. Here, with reference to FIG. 3, the types of abnormalities identified by the abnormality type determination unit 23, the process of identifying the types of abnormalities, and the content of the notification to the second vehicle that the abnormality type determination unit 23 determines based on the types of abnormalities will be described. As shown in FIG. 3, the abnormality type determination unit 23 determines whether the type of abnormality occurring in the second vehicle corresponds to "fluid leakage," "light failure," "wiper failure," or "scratch." Specifically, when the abnormality type determination unit 23 detects a detection item registered for each type of abnormality, it selects the type of abnormality to which the detected detection item corresponds and identifies the type of abnormality.

[0026] (Process to identify fluid leaks) If the abnormality type determination unit 23 detects an "oil leak" or a "coolant leak" from the second vehicle, it determines that the type of abnormality occurring in the second vehicle is a "liquid leak." The abnormality type determination unit 23 detects an "oil leak" or a "coolant leak" from the second vehicle based on the difference between images taken before and after the second vehicle passed a predetermined point. Specifically, the abnormality type determination unit 23 determines whether a difference has been extracted at a pixel position corresponding to the road surface in the images taken before and after the second vehicle passed the predetermined point. If the abnormality type determination unit 23 determines that a difference has been extracted at a pixel position corresponding to the road surface, it determines the shape of the extracted difference image. If the shape of the extracted difference image is determined to be circular, it determines that an "oil leak" or a "coolant leak" has occurred in the second vehicle. If oil or coolant leaks onto the road surface, the oil or coolant appears on the road surface as a circular stain due to surface tension. Therefore, the abnormality type determination unit 23 can determine whether an "oil leak" or a "coolant leak" has occurred on the second vehicle based on the shape of the difference extracted on the road surface. When a difference is extracted at a pixel position corresponding to the road surface, the abnormality type determination unit 23 may determine whether the difference in brightness at the pixel position from which the difference was extracted is equal to or greater than a first predetermined difference, and if it determines that the difference in brightness is equal to or greater than the first predetermined difference, determine whether an "oil leak" or a "coolant leak" has occurred on the second vehicle. Coolant used in vehicles is colored. Furthermore, if oil leaks onto the road surface, it will develop a color on the road surface. Therefore, the abnormality type determination unit 23 can determine whether an "oil leak" or a "coolant leak" has occurred on the second vehicle based on the difference in brightness extracted on the road surface.

[0027] (Process to identify lighting failures) If the abnormality type determination unit 23 detects that the lighting devices of the second vehicle are "off" or "insufficient light," it identifies the type of abnormality occurring in the second vehicle as "faulty lighting." Note that the detection of "off" or "insufficient light" of the lighting devices is performed only when the vehicle's lighting devices are turned on. Examples of when the vehicle's lighting devices are turned on include after sunset for headlights, when the second vehicle is entering a right- or left-turn lane for turn signals, and when deceleration of the second vehicle is detected for brake lights. The abnormality type determination unit 23 detects "off" or "insufficient light" of the second vehicle based on the difference between a normal image of the second vehicle, i.e., an image of the second vehicle when the second vehicle turns on its lighting devices, and an image in which the second vehicle is detected. Specifically, the abnormality type determination unit 23 determines whether the difference is extracted at a pixel position corresponding to the position of the lighting devices of the second vehicle. If the abnormality type determination unit 23 determines that a difference has been extracted at a pixel position corresponding to the position of the lighting device of the second vehicle, it determines whether the luminance of the image in which the second vehicle is detected at the pixel position corresponding to the lighting device is lower by a predetermined lighting luminance or more than the luminance of a normal second vehicle. If the abnormality type determination unit 23 determines that the luminance of the image in which the second vehicle is detected at the pixel position corresponding to the lighting device is lower by a predetermined lighting luminance or more than the luminance of a normal second vehicle, it determines that the lighting device of the second vehicle is "off" or "insufficient light amount."

[0028] (Process to identify wiper malfunctions) When the abnormality type determination unit 23 detects a "reduced wiper wiping force" or a "wiper malfunction" of the wiper of the second vehicle, it identifies the type of abnormality occurring in the second vehicle as a "wiper malfunction." The abnormality type determination unit 23 detects a "reduced wiper wiping force" or a "wiper malfunction" of the wiper of the second vehicle based on the difference between a normal image of the second vehicle and an image in which the second vehicle is detected. Specifically, the abnormality type determination unit 23 determines whether the difference is extracted at a pixel position corresponding to the position of the windshield of the second vehicle. When the abnormality type determination unit 23 determines that the difference is extracted at a pixel position corresponding to the position of the windshield of the second vehicle, it calculates the percentage of pixels in the pixel range corresponding to the position of the windshield where the difference in brightness is equal to or greater than a second predetermined difference. The abnormality type determination unit 23 determines whether the percentage of pixels in the pixel range corresponding to the position of the windshield where the difference in brightness is equal to or greater than the second predetermined difference is equal to or greater than a predetermined percentage. When the abnormality type determination unit 23 determines that the proportion of pixels in the range of pixels corresponding to the windshield position where the difference in brightness is equal to or greater than the second predetermined difference is equal to or greater than a predetermined proportion, the abnormality type determination unit 23 detects a "decreased wiper wiping force" or a "wiper malfunction" of the wipers provided on the second vehicle. This allows the abnormality type determination unit 23 to detect that the windshield of the second vehicle is dirty with rain or dust and that forward visibility is not adequately ensured. In other words, the abnormality type determination unit 23 can detect that the forward visibility of the second vehicle is not adequately ensured due to a wiper malfunction.

[0029] (Process to identify scratches) When the abnormality type determination unit 23 detects a "windshield scratch" or a "body scratch" of the second vehicle, it identifies the type of abnormality occurring in the second vehicle as a "scratch." The abnormality type determination unit 23 detects a "windshield scratch" or a "body scratch" of the second vehicle based on the difference between a normal image of the second vehicle and an image in which the second vehicle is detected. Specifically, the abnormality type determination unit 23 determines whether the difference has been extracted at a pixel position corresponding to the windshield or body of the second vehicle. When the abnormality type determination unit 23 determines that the difference has been extracted at a pixel position corresponding to the windshield or body of the second vehicle, it determines whether the difference occurring in the windshield or body continues for a predetermined length or more. When the abnormality type determination unit 23 determines that the difference occurring in the windshield or body continues for a predetermined length or more, it detects a "windshield scratch" or a "body scratch" of the second vehicle.

[0030] (Determination of notification content) If the type of abnormality is "liquid leakage" or "light failure", the abnormality type determination unit 23 determines the content of the notification to be sent to the second vehicle as "notifying the second vehicle that a liquid leakage or light failure has occurred, and notifying the second vehicle to immediately stop and undergo vehicle maintenance". If the type of abnormality is "wiper malfunction", the abnormality type determination unit 23 determines the content of the notification to be sent to the second vehicle as "1. notifying the second vehicle that a wiper malfunction has occurred, and notifying the second vehicle to immediately stop and undergo vehicle maintenance" and "2. notifying the second vehicle to wait for the weather to improve if the abnormality cannot be resolved". If the type of abnormality is "scratch", the abnormality type determination unit 23 determines the content of the notification to be sent to the second vehicle as "notifying the second vehicle that a scratch has occurred on the vehicle, and notifying the second vehicle to undergo maintenance".

[0031] When the type of abnormality occurring in the second vehicle is identified by the abnormality type determination unit 23, the abnormal vehicle identification unit 24 identifies the second vehicle as an abnormal vehicle. This allows the abnormal vehicle in which an abnormality is occurring to be detected from among multiple vehicles. The abnormal vehicle identification unit 24 transfers the type of abnormality occurring in the abnormal vehicle and the content of the notification to the abnormal vehicle to the abnormality notification unit 40.

[0032] [Anomaly detection method] Next, an example of the processing performed by the anomaly detection system 1 according to the first embodiment, from identifying an abnormal vehicle to notifying the abnormal vehicle of an abnormality, will be described with reference to Fig. 4. The processing performed by the anomaly detection system 1 shown in the flowchart of Fig. 4 is performed as long as the anomaly detection system 1 receives images of the vehicle's position and surroundings from each of a plurality of vehicles.

[0033] In step S10, vehicle detection unit 21 acquires image data of the position and surroundings of each of the multiple vehicles from information acquisition unit 10. In step S20, vehicle detection unit 21 searches for a second vehicle included in the multiple vehicles from an image of the surroundings of a first vehicle included in the multiple vehicles. Specifically, vehicle detection unit 21 determines whether a vehicle detected from the image of the surroundings of the first vehicle is the second vehicle based on the position information of each of the multiple vehicles, and searches for the second vehicle from the image of the surroundings of the first vehicle.

[0034] In step S30, if the vehicle detection unit 21 can detect the second vehicle from the image of the periphery of the first vehicle (YES in step S30), the process proceeds to step S40. In step S30, if the vehicle detection unit 21 cannot detect the second vehicle from the image of the periphery of the first vehicle (NO in step S30), the process ends. In step S40, the difference extraction unit 22 extracts the difference between a normal image, which is an image of the second vehicle when no abnormality occurs in the second vehicle, and the image of the periphery of the first vehicle. Specifically, the difference extraction unit 22 uses a known image processing technique to superimpose the second vehicle captured in the image of the periphery of the first vehicle on the normal image of the second vehicle, and extracts the difference between the normal image of the second vehicle and the second vehicle captured in the image of the periphery of the first vehicle. Furthermore, the difference extraction unit 22 extracts the difference between the images of the periphery of the first vehicle captured before and after the second vehicle passes a predetermined point.

[0035] If a difference is extracted in step S50 (YES in step S50), the process proceeds to step S60. If a difference is not extracted (NO in step S50), the process ends. In step S60, the abnormality type determination unit 23 identifies the type of abnormality occurring in the second vehicle based on the pixel position from which the difference was extracted and the difference in luminance at the pixel position from which the difference was extracted. Specifically, the abnormality type determination unit 23 detects a detection item registered for each abnormality type based on the pixel position from which the difference was extracted and the difference in luminance at the pixel position from which the difference was extracted. When the abnormality type determination unit 23 detects a detection item registered for each abnormality type, it determines whether the detected detection item corresponds to the abnormality type: "fluid leakage," "faulty lighting," "wiper malfunction," or "scratch." When the abnormality type determination unit 23 identifies the abnormality type as "fluid leakage," "faulty lighting," "wiper malfunction," or "scratch," it determines a countermeasure for the abnormality to be notified to the second vehicle based on the abnormality type. The process proceeds to step S70, and if the type of abnormality is identified (YES in step S70), the process proceeds to step S80, and if the type of abnormality is not identified (NO in step S70), the process ends.

[0036] In step S80, if the type of abnormality occurring in the second vehicle is identified by the abnormality type determination unit 23, the abnormal vehicle identification unit 24 identifies the second vehicle as an abnormal vehicle. The abnormal vehicle identification unit 24 transfers the type of abnormality occurring in the abnormal vehicle and the content of the notification to the abnormal vehicle to the abnormality notification unit 40. In step S90, the abnormality notification unit 40 transmits information indicating the type of abnormality occurring and a method of dealing with the abnormality to the abnormal vehicle identified by the abnormal vehicle identification unit 24.

[0037] [Action and effect] As described above, the first embodiment provides the following advantageous effects.

[0038] The abnormality detection system 1 includes an information acquisition unit 10 that receives information indicating the position of each of the multiple vehicles and an image of the surroundings of each of the multiple vehicles, transmitted from each of the multiple vehicles, and an abnormal vehicle detection unit 20 that detects an abnormal vehicle from among the multiple vehicles that is experiencing an abnormality based on the acquired images of the position and surroundings of each of the multiple vehicles. This allows the abnormality detection system 1 to acquire the position of each of the multiple vehicles and an image of the surroundings of each of the multiple vehicles, and to detect an abnormal vehicle from among the multiple vehicles that is experiencing an abnormality based on the acquired images of the position of each of the multiple vehicles and the surroundings of the multiple vehicles. Furthermore, the abnormality detection system 1 can quickly detect an abnormal vehicle by detecting an abnormal vehicle from among the multiple vehicles based on the acquired images of the position and surroundings of each of the multiple vehicles. In other words, the abnormality detection system 1 can detect an abnormal vehicle before the effects of an abnormal vehicle spread to multiple vehicle systems.

[0039] The abnormality detection system 1 detects a second vehicle included in the plurality of vehicles from an image of the surroundings of a first vehicle included in the plurality of vehicles, and extracts the difference between a normal image, which is an image of the second vehicle when no abnormality has occurred in the second vehicle, and an image of the surroundings of the first vehicle. In this way, the abnormality detection system 1 can extract the difference between the normal image when no abnormality has occurred in the second vehicle and an image of the second vehicle included in the image of the surroundings of the first vehicle, and can determine whether the second vehicle is an abnormal vehicle.

[0040] (Second embodiment) [Configuration of vehicle service decision system] An example of the configuration of a vehicle service determination system 2 according to the second embodiment will be described with reference to FIG. 5 . The vehicle service determination system 2 is a system that determines the content of services to be provided using vehicles based on information about abnormal vehicles. The vehicle service determination system 2 is used to determine the service content for transportation services or ride-hailing services provided by unmanned autonomous vehicles. The vehicle service determination system 2 according to the second embodiment differs from the first embodiment in that it includes a vehicle service determination unit 60 instead of the abnormal vehicle detection unit 20 and a service content instruction unit 50 instead of the abnormality notification unit 40, but the other configurations are the same. The vehicle service determination unit 60 differs from the abnormal vehicle detection unit 20 in that it further includes a service content determination unit 61. Therefore, only the differences will be described, and a description of other common parts will be omitted. Note that, although the present embodiment illustrates a configuration in which the vehicle service determination system 2 is incorporated into the anomaly detection system 1, the configuration of the vehicle service determination system 2 is not limited to this. For example, the vehicle service determination system 2 and the anomaly detection system 1 may be configured in different housings.

[0041] The service content determination unit 61 acquires the type of abnormality occurring in the abnormal vehicle from the abnormality detection system 1 and determines the content of the service to be provided by the abnormal vehicle based on the acquired type of abnormality. Specifically, the service content determination unit 61 determines whether the abnormal vehicle can continue driving based on the type of abnormality occurring in the abnormal vehicle. If the service content determination unit 61 determines that the abnormal vehicle cannot continue driving, it determines whether the abnormal vehicle is currently providing service. If it determines that the abnormal vehicle is currently providing service, it selects a service to arrange a replacement car as the service to be provided by the abnormal vehicle. Note that whether the abnormal vehicle is currently providing service can be determined based on information indicating the service status transmitted from the abnormal vehicle. If a replacement car cannot be arranged, the service content determination unit 61 searches for a route to the destination by public transportation and selects a service to provide a route to the destination by public transportation to a service recipient who is currently receiving a service provided by the abnormal vehicle. Furthermore, if the service content determination unit 61 changes the content of the service to be provided by the abnormal vehicle, it transfers information indicating the reason for changing the content of the service to be notified to the service recipient and the selected content of the changed service to the service content instruction unit. This notifies the service recipient of the reason for changing the content of the service provided to the abnormal vehicle and the content of the changed service.

[0042] The service content instruction unit 50 is a transmitting device that transmits information indicating the content of the service to be provided to the abnormal vehicle, which is determined by the service content determination unit 61, to the abnormal vehicle.

[0043] Here, with reference to FIG. 6, an example of the content of the service provided by the abnormal vehicle determined by the service content determination unit 61 will be described. Note that FIG. 6 shows the content of the service selected when the abnormal vehicle is currently providing service. As shown in FIG. 6, if the type of abnormality occurring in the abnormal vehicle is "liquid leakage" or "faulty lighting" (at the time of abnormality detection), the service content determination unit 61 determines that the abnormal vehicle cannot continue driving and selects the service of "1. Immediately stop the vehicle and arrange for a replacement vehicle." At the same time, the service content determination unit 61 selects the service of "2. Arrange a monitoring vehicle." A monitoring vehicle is a vehicle for monitoring abnormal vehicles, and is a vehicle dedicated to monitoring vehicles not currently providing service or abnormal vehicles. After detecting an abnormality, the service content determination unit 61 performs the following: "3. Continuously monitor the status of the abnormal vehicle using the monitoring vehicle until a replacement vehicle arrives." This allows the safety of the service recipient to be confirmed. Furthermore, if a replacement vehicle cannot be arranged, the service content determination unit 61 selects the service of "4. Provide a route to the destination by public transportation." If the type of abnormality occurring in the abnormal vehicle is "liquid leakage" or "failure of lighting", the service content determination unit 61 notifies the service recipient that the vehicle has stopped due to the abnormality and that an alternative service will be provided.

[0044] If the type of abnormality occurring in the abnormal vehicle is "wiper malfunction" (when the abnormality is detected), the service content determination unit 61 determines that the abnormal vehicle cannot continue driving and selects a service of "1. Immediately stop the vehicle and ask the service recipient whether or not to wait for the weather to improve." The service content determination unit 61 obtains information about the weather at the location of the abnormal vehicle from a weather distribution service and identifies the estimated time when the weather will improve. The service content determination unit 61 transmits the estimated time when the weather will improve to a screen installed in the vehicle or the mobile terminal of the service recipient. The service content determination unit 61 selects a service of "2. If the service recipient cannot wait for the weather to improve, arrange for a replacement car." The service content determination unit 61 also selects a service of "3. Arrange for a monitoring vehicle." After detecting the abnormality, the service content determination unit 61 performs "4. Continuously monitor the status of the abnormal vehicle using a monitoring vehicle until the weather improves or a replacement car arrives." If a replacement car cannot be arranged, the service content determination unit 61 selects a service of "5. Provide a route to the destination by public transportation." The service content determination unit 61 selects the service "6. If the weather improves, start traveling to the destination."

[0045] If the type of abnormality occurring in the abnormal vehicle is "damage," the service content determination unit 61 continues the service currently being provided to the abnormal vehicle. The service content determination unit 61 selects a command to instruct maintenance when the service currently being provided to the service recipient is completed.

[0046] [Vehicle service determination method] Next, an example of the processing up to when the vehicle service decision system 2 according to the second embodiment outputs a maintenance instruction to an abnormal vehicle will be described with reference to Fig. 7. The processing of the vehicle service decision system 2 shown in Fig. 7 shows the processing from step S80 onwards of the abnormality detection system 1 shown in Fig. 4. Therefore, the processing up to step S100 of the vehicle service decision system 2 is the same as that of the abnormality detection system 1, except for the processing of step S90.

[0047] In step S100, the service content determination unit 61 acquires the type of abnormality occurring in the abnormal vehicle from the abnormal vehicle identification unit 24. In step S110, if the type of abnormality is "liquid leakage" or "faulty lighting device" (YES in step S110), the process proceeds to step S120. In step S110, if the type of abnormality is not "liquid leakage" or "faulty lighting device" (NO in step S110), the process proceeds to step S190. In step S120, the service content determination unit 61 selects an instruction to immediately stop the abnormal vehicle and transfers the selected instruction to the service content instruction unit 50. In step S130, the service content determination unit 61 determines whether the abnormal vehicle is currently providing service based on the information indicating the service status transmitted from the abnormal vehicle. In step S130, if the service content determination unit 61 determines that an abnormal vehicle is currently providing service (YES in step S130), the processing proceeds to step S140, and if the service content determination unit 61 determines that an abnormal vehicle is not currently providing service (NO in step S130), the processing proceeds to step S180.

[0048] In step S140, the service content determination unit 61 checks whether a loaner vehicle can be arranged. If a loaner vehicle can be arranged (YES in step S140), the process proceeds to step S150, where the service content determination unit 61 arranges a loaner vehicle. If a loaner vehicle cannot be arranged (NO in step S140), the process proceeds to step S160, where the service content determination unit 61 searches for a route to the destination by public transportation and transmits the route to the destination by public transportation to a display unit provided in the abnormal vehicle or a mobile terminal of the service recipient. In step S170, the service content determination unit 61 determines whether a loaner vehicle has arrived based on an image of the surroundings of the abnormal vehicle acquired by the abnormal vehicle or GPS data transmitted from the loaner vehicle. In step S170, if the service content determination unit 61 determines that a loaner vehicle has not arrived (NO in step S170), the process remains in step S170 until a loaner vehicle arrives. In step S170, if the service content determination unit 61 determines that a replacement vehicle has arrived (YES in step S170), it notifies the transportation service provider to perform maintenance on the abnormal vehicle. Specifically, the service content determination unit 61 notifies the transportation service provider of the type of abnormality occurring in the abnormal vehicle and the location of the abnormal vehicle.

[0049] In step S190, if the type of abnormality is "wiper malfunction" (YES in step S190), the process proceeds to step S200. In step S190, if the type of abnormality is not "wiper malfunction" (NO in step S190), the process proceeds to step S260. In step S200, the service content determination unit 61 selects an instruction to immediately stop the abnormal vehicle and transfers the selected instruction to the service content instruction unit 50. In step S210, the service content determination unit 61 determines whether the abnormal vehicle is currently providing service. If the service content determination unit 61 determines that the abnormal vehicle is currently providing service (YES in step S210), the process proceeds to step S220. If the service content determination unit 61 determines that the abnormal vehicle is not currently providing service (NO in step S210), the process proceeds to step S180. In step S220, the service content determination unit 61 transmits the estimated time when the weather will improve to the display on the vehicle or the mobile terminal of the service recipient that receives the service content, and confirms with the service recipient whether or not they will wait for the weather to improve. If the service recipient will wait for the weather to improve (YES in step S220), the process proceeds to step S230. If the service recipient cannot wait for the weather to improve (NO in step S220), the process proceeds to step S140.

[0050] In step S230, the service content determination unit 61 acquires an image of the area around the abnormal vehicle or the current weather at the location of the abnormal vehicle from a weather distribution service, and determines whether the weather has improved. If the service content determination unit 61 determines that the weather has improved (YES in step S230), the process proceeds to step S240. If the service content determination unit 61 determines that the weather has not improved even though the scheduled time for weather improvement notified to the service recipient has arrived (NO in step S230), the process proceeds to step S140. In step S240, the service content determination unit 61 selects a command to instruct the abnormal vehicle to travel to the destination, and transfers the selected command to the service content instruction unit 50. In step S250, if the service content determination unit 61 determines that the abnormal vehicle has arrived at the destination (YES in step S250), the process proceeds to step S180.

[0051] In step S260, if the type of abnormality is "damage" (YES in step S260), the process proceeds to step S270. If the type of abnormality is not "damage" (NO in step S260), the process ends. In step S270, the service content determination unit 61 determines whether or not an abnormal vehicle is currently providing service. In step S270, if the service content determination unit 61 determines that an abnormal vehicle is currently providing service (YES in step S270), the process proceeds to step S280. If the service content determination unit 61 determines that an abnormal vehicle is not currently providing service (NO in step S270), the process proceeds to step S180. In step S280, the service content determination unit 61 selects a command to instruct the vehicle to continue traveling to the destination and transfers the selected command to the service content instruction unit 50. In step S290, if the service content determination unit 61 determines that the abnormal vehicle has arrived at the destination (YES in step S270), the process proceeds to step S180.

[0052] [Action and effect] As described above, the second embodiment provides the following advantageous effects.

[0053] By acquiring the type of abnormality that has occurred in the abnormal vehicle, the vehicle service determination system 2 can determine the content of the service to be provided to the abnormal vehicle based on the type of abnormality that has occurred in the abnormal vehicle. In other words, the vehicle service determination system 2 can provide an appropriate service in accordance with the type of abnormality that has occurred in the vehicle.

[0054] The vehicle service decision system 2 determines whether the abnormal vehicle can continue to drive based on the type of abnormality, and if it determines that the abnormal vehicle cannot continue to drive, it determines whether the abnormal vehicle is currently providing service, and if it determines that the abnormal vehicle is currently providing service, it arranges for a replacement vehicle. In this way, the vehicle service decision system 2 can provide an alternative service to the service recipient even if an abnormality occurs in a vehicle currently providing service that prevents it from continuing to drive.

[0055] If a replacement vehicle cannot be arranged, the vehicle service decision system 2 searches for a route to the destination by public transportation and provides the route to the destination by public transportation to the service recipient who is receiving the abnormal vehicle service, thereby enabling the service recipient to travel to the destination by public transportation.

[0056] When changing the content of the service for an abnormal vehicle, the vehicle service decision system 2 notifies the service recipient of the reason for the change in the content of the service and the content of the changed service. This allows the service recipient to understand the reason for the change in the content of the service and to agree to the changed content of the service. [Explanation of symbols]

[0057] 1. Anomaly detection system 2 Vehicle service decision system 10 Information acquisition department 20 Abnormal vehicle detection unit 60 Vehicle Service Decision Department 61 Service Content Decision Department

Claims

1. 1. An abnormality detection system that detects a vehicle in which an abnormality is occurring from among a plurality of vehicles based on a position of each of the plurality of vehicles and an image of a surrounding area of ​​each of the plurality of vehicles, an information acquisition unit that receives information indicating the position and an image of the surroundings transmitted from each of the plurality of vehicles; an abnormal vehicle detection unit that detects an abnormal vehicle in which an abnormality is occurring from among the plurality of vehicles based on the acquired information indicating the positions of each of the plurality of vehicles and the surrounding image; a storage unit in which normal images of the plurality of vehicles are stored in advance, the normal images being images of the vehicles when no abnormality occurs; Equipped with The abnormal vehicle detection unit Detecting a second vehicle included in the plurality of vehicles from an image of a periphery of a first vehicle included in the plurality of vehicles; extracting a difference between the normal image of the second vehicle and an image of the surroundings of the first vehicle from the normal images stored in the storage unit; Based on the difference, it is determined whether the second vehicle is the abnormal vehicle. A vehicle service determination system that determines the content of a service to be provided using the vehicle based on information about the abnormal vehicle detected by an abnormality detection system, and a service content determination unit that acquires the type of abnormality occurring in the abnormal vehicle from the abnormality detection system and determines the content of a service to be provided to the abnormal vehicle based on the type of abnormality. Vehicle service decision system.

2. The service content determination unit determining whether the abnormal vehicle can continue to travel based on the type of abnormality; If it is determined that the abnormal vehicle cannot continue traveling, it is determined whether the abnormal vehicle is currently providing service; If it is determined that the abnormal vehicle is currently in service, a replacement vehicle will be arranged. The vehicle service decision system of claim 1 .

3. The service content determination unit If a replacement car cannot be arranged, we will search for a route to your destination by public transportation. Providing the route to a service recipient who is receiving service for the abnormal vehicle The vehicle service decision system of claim 2 .

4. When changing the content of the service for the abnormal vehicle, the service content decision unit notifies the service recipient of the reason for changing the content of the service and the content of the service after the change. The vehicle service decision system of claim 3 .

Images