Anomaly detection device, anomaly detection method, and computer program for anomaly detection

Abstract

To provide an anomaly detection device capable of detecting an anomaly that would prevent normal travel of a vehicle.SOLUTION: An anomaly detection device includes: an identifying unit 31 configured to identify a path region in an image representing surroundings of a vehicle 10, the path region representing a lane on which the vehicle 10 is traveling; an extraction unit 32 configured to extract a feature quantity indicating the condition of a road surface regarding the path region from the image by inputting the path region in the image into a feature extractor that has been trained to extract the feature quantity; and a detection unit 33 configured to detect an abnormal condition in which the vehicle 10 cannot travel normally on the lane, when the feature quantity is outside a normal range that represents a tolerable range in which the vehicle 10 can travel normally.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to an abnormality detection device, an abnormality detection method, and a computer program for abnormality detection that detect abnormalities based on an image representing the surroundings of a vehicle.

Background Art

[0002] A technique has been proposed that determines a driving-suitable area suitable for the running of a vehicle based on an image obtained by an imaging device mounted on the vehicle, and uses the determination result for automatic driving control or driving support of the vehicle (see Patent Document 1).

[0003] The driving area determination device described in Patent Document 1 determines a specific area in an image obtained by imaging the surroundings of the vehicle from an imaging device using a machine learning model that has machine-learned a plurality of images in which the subject is known to be a road surface. Then, this driving area determination device determines whether or not the specific area is a driving-suitable area suitable for the running of the vehicle. Thereby, for example, an area where there may be an obstacle is determined as a driving-inappropriate area inappropriate for the running of the vehicle. Further, this driving area determination device determines a specific area for an area within the driving lane of the vehicle in the image.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Obstacles such as fallen objects may be present in the vehicle's path. Such obstacles may be objects whose color, shape, and size cannot be determined in advance. Therefore, even using a classifier designed to detect specific objects from images, it may not be possible to accurately detect such obstacles. As a result, it may be difficult to accurately determine whether the road surface in the vehicle's path is passable or not.

[0006] Therefore, the present invention aims to provide an abnormality detection device capable of detecting abnormalities that interfere with the normal operation of a vehicle. [Means for solving the problem]

[0007] According to one embodiment, an anomaly detection device is provided. This anomaly detection device includes: an identification unit that identifies a road area representing the lane in which the vehicle is traveling in an image representing the area around the vehicle; an extraction unit that inputs the road area of ​​the image into a feature extractor that has been pre-trained to extract feature quantities representing the road surface conditions from the image, thereby extracting the feature quantities of the road area; and a detection unit that detects an abnormal situation in which the vehicle cannot normally travel in its lane if the feature quantities are not included in the normal range, which represents the permissible range in which the vehicle can normally travel.

[0008] In this anomaly detection device, it is preferable that the extraction unit inputs the range below the lower end of the preceding vehicle area, which represents the preceding vehicle traveling directly in front of the vehicle, into the feature extractor to extract feature quantities.

[0009] Another embodiment provides an anomaly detection method. This anomaly detection method includes identifying a road region representing the vehicle's own lane in an image representing the area around the vehicle, inputting the road region of the image into a feature extractor that has been pre-trained to extract feature quantities representing the road surface conditions from the image, extracting the feature quantities for the road region, and detecting an abnormal situation in which the vehicle cannot normally travel in its own lane if the feature quantities are not included in the normal range representing the permissible range in which the vehicle can normally travel.

[0010] In yet another embodiment, a computer program for anomaly detection is provided. This computer program for anomaly detection includes instructions to cause a processor installed in the vehicle to perform the following actions: identify a road region in an image representing the area around the vehicle that represents the lane in which the vehicle is traveling; input the road region of the image into a feature extractor that has been pre-trained to extract feature quantities representing the road surface conditions from the image; extract the feature quantities for the road region; and if the feature quantities are not included in the normal range, which represents the permissible range in which the vehicle can normally travel, the program detects an abnormal situation in which the vehicle cannot normally travel in its lane. [Effects of the Invention]

[0011] The abnormality detection device described herein has the effect of being able to detect abnormalities that interfere with the normal operation of a vehicle. [Brief explanation of the drawing]

[0012] [Figure 1] This is a schematic diagram of a vehicle control system in which an anomaly detection device is implemented. [Figure 2] This is a hardware configuration diagram of an electronic control unit, which is one embodiment of an anomaly detection device. [Figure 3] This is a functional block diagram of the processor of an electronic control unit related to vehicle control processing, including anomaly detection processing. [Figure 4] This figure illustrates an example of anomaly detection processing according to this embodiment. [Figure 5] This is an operation flowchart of vehicle control processing, including anomaly detection processing. [Modes for carrying out the invention]

[0013] The following describes the anomaly detection device, the anomaly detection method implemented by the anomaly detection device, and the computer program for anomaly detection, with reference to the diagram. This anomaly detection device identifies the road area representing the lane in which the vehicle is traveling (hereinafter sometimes referred to as the vehicle's own lane) in an image representing the area around the vehicle generated by an imaging unit installed on the vehicle. Furthermore, this anomaly detection device inputs the road area of ​​the image into a feature extractor to extract feature quantities that represent the condition of the road surface. This anomaly detection device then determines whether the extracted feature quantities fall within the normal range, which indicates that the vehicle can normally travel. If the feature quantities fall outside the normal range, it detects an abnormal situation in which the vehicle cannot normally travel in its own lane.

[0014] Furthermore, for a vehicle to be able to travel normally in its own lane means that the vehicle can travel without having to decelerate beyond a predetermined deceleration rate or steer beyond a predetermined amount to avoid contact with any obstacle. Obstacles include, for example, any three-dimensional structure that should not be on the road surface, such as a fallen object, or road surface defects that involve steps, such as potholes formed in the road surface.

[0015] The following section describes an example of applying an anomaly detection device to a vehicle control system.

[0016] Figure 1 is a schematic diagram of a vehicle control system in which an anomaly detection device is implemented. The vehicle control system 1 is mounted on and controls the vehicle 10. To this end, the vehicle control system 1 includes a camera 2 and an electronic control unit (ECU) 3, which is an example of an anomaly detection device. The camera 2 and the ECU 3 are connected to each other so as to be able to communicate via an in-vehicle network compliant with a communication standard such as a controller area network. The vehicle control system 1 may also have a distance measuring sensor (not shown), such as LiDAR or radar, for measuring the distance from the vehicle 10 to objects in the vicinity of the vehicle 10. Furthermore, the vehicle control system 1 may have a positioning device (not shown), such as a GPS receiver, for determining the position of the vehicle 10 based on signals from satellites. Furthermore, the vehicle control system 1 may have a navigation device (not shown) for searching for a planned route to a destination. Furthermore, the vehicle control system 1 may have a storage device (not shown) for storing map information referenced in the automatic driving control of the vehicle 10.

[0017] Camera 2 is an example of an imaging unit that generates an image representing the surroundings of the vehicle 10. Camera 2 has a two-dimensional detector composed of an array of photoelectric conversion elements sensitive to visible light, such as a CCD or C-MOS, and an imaging optical system that forms an image of the area to be photographed on the two-dimensional detector. Camera 2 is mounted, for example, inside the vehicle 10 so as to face forward of the vehicle 10. Camera 2 then photographs the area in front of the vehicle 10 at predetermined shooting cycles (e.g., 1 / 30 second to 1 / 10 second) and generates an image of that area. The image obtained by Camera 2 may be a color image or a grayscale image. Note that the vehicle 10 may be equipped with two or more cameras with different shooting directions or focal lengths.

[0018] Camera 2 outputs the generated image to ECU 3 via the in-vehicle network each time it generates an image.

[0019] The ECU 3 is configured to automatically control the vehicle 10 under a predetermined situation.

[0020] FIG. 2 is a hardware configuration diagram of the ECU 3 which is an example of an abnormality detection device. As shown in FIG. 2, the ECU 3 includes a communication interface 21, a memory 22, and a processor 23. The communication interface 21, the memory 22, and the processor 23 may each be configured as separate circuits, or may be integrally configured as one integrated circuit.

[0021] The communication interface 21 has an interface circuit for connecting the ECU 3 and the camera 2. And each time the communication interface 21 receives an image from the camera 2, it transfers the received image to the processor 23.

[0022] The memory 22 is an example of a storage unit and includes, for example, a volatile semiconductor memory and a non-volatile semiconductor memory. And the memory 22 stores various data used in vehicle control processing including abnormality detection processing executed by the processor 23 of the ECU 3. For example, the memory 22 stores parameters for identifying an identifier used for detecting a travel region, and parameters for identifying a feature extractor used for extracting feature amounts representing the state of the road surface. Further, the memory 22 stores a normal range representing an allowable range within which the vehicle 10 can normally travel with respect to the feature amounts. Further, the memory 22 temporarily stores images received from the camera 2. Furthermore, the memory 22 temporarily stores various data generated during the vehicle control processing.

[0023] The processor 23 includes one or more CPUs (Central Processing Units) and their peripheral circuits. The processor 23 may further include other arithmetic circuits such as a logical arithmetic unit, a numerical arithmetic unit, or a graphic processing unit. And the processor 23 executes vehicle control processing for the vehicle 10.

[0024] Figure 3 is a functional block diagram of the processor 23 relating to vehicle control processing, including abnormality detection processing. The processor 23 has a specific unit 31, an extraction unit 32, a detection unit 33, and a vehicle control unit 34. Each of these units of the processor 23 is, for example, a functional module realized by a computer program running on the processor 23. Alternatively, each of these units of the processor 23 may be a dedicated arithmetic circuit provided on the processor 23. Of these units, the specific unit 31, the extraction unit 32, and the detection unit 33 are related to abnormality detection processing.

[0025] The identification unit 31 identifies the road area in which the current lane is represented in the latest image received by the ECU 3 from the camera 2 at predetermined intervals.

[0026] For example, the identification unit 31 identifies the track area by inputting the image acquired from the camera 2 into a classifier that has been pre-trained to identify the track area. The identification unit 31 can use a deep neural network (DNN) with a convolutional neural network (CNN) architecture as such a classifier. More specifically, a semantic segmentation DNN, such as a Fully Convolutional Network (FCN) or U-net, which identifies the object represented in each pixel, can be used as the classifier. Alternatively, the identification unit 31 may use a semantic segmentation classifier based on a machine learning method other than a neural network, such as a random forest. The classifier is pre-trained using a large number of training images representing the track area, according to a predetermined learning method such as backpropagation.

[0027] The identification unit 31 defines the set of pixels output by the classifier, which are assumed to represent the current lane, as the road area.

[0028] Alternatively, instead of using the classifier described above, the identification unit 31 may detect two lane markings that demarcate the current lane from the image by image analysis, and define the area between the two detected lane markings as the road area. Generally, lane markings have a brighter color (white or yellow) than the surrounding road surface. Therefore, in the image, the brightness value of a pixel representing a lane marking will be higher than the brightness value of a pixel representing the surrounding road surface. Thus, the identification unit 31 extracts pixels in the area where the road surface is assumed to be represented on the image whose brightness value is above a predetermined value. Alternatively, the identification unit 31 may extract the pixel with the higher brightness value when the brightness difference between two adjacent pixels in the horizontal direction is above a predetermined threshold. The identification unit 31 then approximates the set of extracted pixels with a straight line to detect individual lane markings, and on both the left and right sides of the image, the lane marking closest to the center of the image is defined as the lane marking that demarcates the current lane.

[0029] Furthermore, the identification unit 31 may identify an area in the image that shows a preceding vehicle traveling directly in front of vehicle 10 in its own lane (hereinafter sometimes referred to as the preceding vehicle area). As described above, when the road area is identified using a semantic segmentation classifier, it is sufficient that the classifier is pre-trained to also identify areas that show other vehicles. The identification unit 31 then designates the area that overlaps with the road area and is located at the bottom of the image as the preceding vehicle area. In this way, the identification unit 31 can identify not only the road area but also the preceding vehicle area by inputting the image into the classifier. Also, when the road area is identified by image analysis, the identification unit 31 may detect other vehicles traveling around vehicle 10 shown in the image by inputting the image into a vehicle detector that has been pre-trained to detect vehicles from the image. The identification unit 31 then designates the area that shows the other vehicle that overlaps with the road area and is located at the bottom of the image as the preceding vehicle area. The specific unit 31 can use a DNN for object detection, such as a Single Shot MultiBox Detector or Faster R-CNN, as such a vehicle detector. Such a vehicle detector is pre-trained using a number of training images representing the vehicles to be detected, according to a predetermined learning method such as backpropagation.

[0030] Each time the identification unit 31 detects a road area from the image, it notifies the extraction unit 32 of the information representing that road area. Furthermore, if the identification unit 31 detects a preceding vehicle area from the image, it also notifies the extraction unit 32 of the information representing that preceding vehicle area. The information representing the road area can, for example, be an image with the same size as the original image, where pixels within the road area and pixels outside the road area have different values. The same applies to the information representing the preceding vehicle area.

[0031] The extraction unit 32 extracts features representing the road surface conditions (hereinafter, these features may be referred to as road surface condition features) from the road area in the image. To do this, the extraction unit 32 inputs the road area of ​​the image into a feature extractor that has been pre-trained to extract road surface condition features. The feature extractor is composed of a DNN having a CNN-type architecture with multiple convolutional layers, such as VGG16 or VGG19. The feature extractor may also have one or more fully connected layers on the output side of the multiple convolutional layers. However, the feature map output from the furthest output convolutional layer (or the furthest output convolutional layer and any of the convolutional layers on the input side of that layer) is used as the road surface condition features output by the feature extractor. The road surface condition features output by the feature extractor are represented, for example, as a feature vector having one or more element values.

[0032] The feature extractor is pre-trained using a predetermined learning method, such as backpropagation, to enable it to classify various types of objects, not just road surfaces, using a large number of images representing those objects. As a result, the road surface feature vectors become a condensed representation of the various information shown in the images. Therefore, the feature extractor can output road surface feature vectors that represent the condition of the road surface as it appears in the road area.

[0033] Alternatively, a DNN pre-trained by so-called unsupervised learning, such as an Auto-Encoder or Stacked What-Where Auto-Encoders, may be used as the feature extractor. In this case, the feature extractor has, in order from the input side, an encoder that outputs feature quantities with reduced dimensionality compared to the input data (in this embodiment, the road area), and a decoder that receives the feature quantities output from the encoder. The feature extractor is pre-trained using a large number of images as described above so that the data input to the encoder and the data output from the decoder are the same. Then, by inputting the road area into the trained feature extractor, the feature quantities output by the encoder are obtained as road surface condition features. Furthermore, a DNN trained by a self-supervised learning method, such as Self-Supervised Learning, may be used as the feature extractor. By using such a feature extractor, the extraction unit 32 can obtain road surface condition features that appropriately represent the road surface conditions of the lane, even when obstacles of undefined color, shape, and size exist on the lane.

[0034] Furthermore, the extraction unit 32 masks areas outside the track region in the image by replacing the values ​​of each pixel outside the track region with predetermined values, so that the feature extractor can determine road surface condition features based only on the track region of the image. The extraction unit 32 then inputs the image with the areas outside the track region masked into the feature extractor. Alternatively, the extraction unit 32 may cut out the track region from the image and input the cut-out track region into the feature extractor. In this case, the extraction unit 32 may perform preprocessing such as upsampling, downsampling, or padding on the cut-out track region so that it becomes a region with a predetermined shape and size. The extraction unit 32 may then input the preprocessed track region into the feature extractor. This makes it possible to simplify the feature extractor because the shape and size of the track region input into the feature extractor are constant.

[0035] Furthermore, if a preceding vehicle traveling in the same lane is detected, the extraction unit 32 may input the area of ​​the road region corresponding to the section between vehicle 10 and the preceding vehicle to the feature extractor. That is, the extraction unit 32 inputs the area of ​​the road region below the lower end of the preceding vehicle region to the feature extractor. This prevents the preceding vehicle from affecting the calculation of road surface condition features.

[0036] The extraction unit 32 passes the extracted road surface condition features to the detection unit 33.

[0037] The detection unit 33 determines whether the road surface condition features received from the extraction unit 32 fall within the normal range read from the memory 22. The normal range, as described above, represents the permissible range within which the vehicle 10 can normally drive, relative to the road surface condition features. The normal range is pre-set to include road surface condition features extracted from each of the many images representing the road area when the vehicle 10 can normally drive, and not to include road surface condition features extracted from each of the many images representing the road area when the vehicle 10 cannot normally drive. The detection unit 33 detects an abnormal situation in which the vehicle 10 cannot normally drive in its own lane if the road surface condition features extracted by the extraction unit 32 are not included in the normal range. On the other hand, if the road surface condition features are included in the normal range, the detection unit 33 does not detect such an abnormal situation.

[0038] The detection unit 33 notifies the vehicle control unit 34 of the result of its determination of whether or not it has detected an abnormal situation.

[0039] When the vehicle control unit 34 is notified by the detection unit 33 that an abnormal situation has been detected, it controls various parts of the vehicle 10 to prevent the vehicle 10 from becoming endangered by that abnormal situation. For example, when the vehicle control unit 34 is notified that an abnormal situation has been detected, it decelerates the vehicle 10 at a predetermined deceleration rate.

[0040] The vehicle control unit 34 sets the accelerator opening or brake amount to achieve a set deceleration. The vehicle control unit 34 then determines the fuel injection amount according to the set accelerator opening and outputs a control signal corresponding to that fuel injection amount to the fuel injection device of the vehicle 10's engine. Alternatively, the vehicle control unit 34 controls the power supply device to the motor that drives the vehicle 10 so that it supplies power corresponding to the set accelerator opening to the motor. Or, the vehicle control unit 34 outputs a control signal corresponding to the set brake amount to the brakes of the vehicle 10.

[0041] Furthermore, the vehicle control unit 34 may notify the driver of a warning indicating that an abnormal situation has been detected via notification devices installed in the passenger compartment of the vehicle 10. For example, if a display device is provided as an example of a notification device, the vehicle control unit 34 may cause the display device to display a warning message or icon indicating that an abnormal situation has been detected. If a speaker is provided as another example of a notification device, the vehicle control unit 34 may cause the speaker to output a warning sound indicating that an abnormal situation has been detected. Furthermore, if a vibrator is provided on the driver's seat or steering wheel as yet another example of a notification device, the vehicle control unit 34 may vibrate the vibrator. Furthermore, if one or more light sources are provided as yet another example of a notification device, the vehicle control unit 34 may illuminate or flash the light source corresponding to the detected abnormal situation. Also, if multiple notification devices are provided in the passenger compartment, the vehicle control unit 34 may notify the driver of a warning indicating that an abnormal situation has been detected via two or more of these multiple notification devices.

[0042] Alternatively, the vehicle control unit 34 may reduce the level of automated driving control applied to the vehicle 10. For example, if so-called Level 3 automated driving control, as defined by the Society of Automotive Engineers (SAE), is applied to the vehicle 10, the vehicle control unit 34 will reduce the level of automated driving control applied to the vehicle 10 to one of levels 0 to 2. Also, if the level of automated driving control applied to the vehicle 10 is Level 2 driving control, the vehicle control unit 34 will reduce the level of automated driving control applied to the vehicle 10 to Level 0 or Level 1. Alternatively, the vehicle control unit 34 may request the driver of the vehicle 10 to hold the steering wheel. In this case as well, the vehicle control unit 34 will notify the driver of the change in the level of automated driving control via a notification device provided in the vehicle 10's cabin. That is, if a display device is provided as an example of a notification device, the vehicle control unit 34 will cause the display device to display a notification message or icon indicating that the level of automated driving control applied has been changed and the changed level. Furthermore, if a speaker is provided as another example of a notification device, the vehicle control unit 34 will cause the speaker to output a notification sound indicating that the level of the applied automated driving control has been changed and that the level has been changed. Furthermore, if a vibrator is provided on the driver's seat or steering wheel as another example of a notification device, the vehicle control unit 34 will vibrate the vibrator in a manner corresponding to the change in the level of the applied automated driving control. Furthermore, if one or more light sources are provided as another example of a notification device, the vehicle control unit 34 will illuminate or flash the light source corresponding to the changed level of automated driving control. Also, if multiple notification devices are provided in the vehicle cabin, the vehicle control unit 34 may notify the vehicle of the change in the level of the applied automated driving control through two or more of these multiple notification devices.

[0043] Alternatively, the vehicle control unit 34 may not perform either deceleration of the vehicle 10 or change the level of applied automatic driving control, but may simply notify the driver via a notification device installed in the vehicle cabin that an abnormal situation has been detected.

[0044] Figure 4 illustrates an example of anomaly detection processing according to this embodiment. In the image 400 shown in Figure 4, the road area 401 representing the current lane is identified. In this example, since there is a preceding vehicle traveling in the current lane, the preceding vehicle area 402 representing the preceding vehicle is also identified. The lower end of the preceding vehicle area 402 is assumed to represent the position of the rear end of the preceding vehicle or the position of the rear wheels of the preceding vehicle. Therefore, the area 401a below the lower end of the preceding vehicle area 402 within the road area 401 is input to the feature extractor 410, and the road surface condition feature quantity F is extracted from the feature extractor 410. If the extracted road surface condition feature quantity F is included in the normal range NR, no anomaly is detected. On the other hand, if the road surface condition feature quantity F is outside the normal range NR, an anomaly is detected.

[0045] Figure 5 is an operation flowchart of the vehicle control process, including anomaly detection processing, executed by the processor 23. The processor 23 executes the vehicle control process according to the following operation flowchart at predetermined intervals.

[0046] The identification unit 31 of the processor 23 identifies the road area representing the current lane in the latest image obtained by the camera 2 (step S101). The identification unit 31 also identifies the preceding vehicle area representing the preceding vehicle in that image (step S102).

[0047] The extraction unit 32 of the processor 23 extracts road surface condition features from the road area on the image, excluding the area of ​​the preceding vehicle (step S103). The detection unit 33 of the processor 23 then determines whether the road surface condition features fall within the normal range (step S104). If the road surface condition features do not fall within the normal range (step S104-No), the detection unit 33 detects an abnormal situation in which the vehicle 10 cannot normally drive in its own lane (step S105). The vehicle control unit 34 of the processor 23 then decelerates the vehicle 10 or reduces the level of automatic driving control applied to the vehicle 10 so that the detected abnormal situation does not endanger the vehicle 10 (step S106).

[0048] On the other hand, in step S104, if the road surface condition characteristics fall within the normal range (step S104-Yes), the detection unit 33 does not detect an abnormal situation in which the vehicle 10 cannot normally drive in its own lane. The vehicle control unit 34 then continues the control of the vehicle 10 that is currently being applied to the vehicle 10 (step S107).

[0049] After step S106 or S107, the processor 23 terminates the vehicle control process.

[0050] As explained above, this anomaly detection device identifies the road area representing the vehicle's lane in an image of the vehicle's surroundings generated by an imaging unit installed on the vehicle. Furthermore, this anomaly detection device inputs the road area from the image into a feature extractor to extract road surface condition features. The device then determines whether the extracted road surface condition features fall within the normal range, which indicates that the vehicle can normally travel. If the features fall outside the normal range, it detects an abnormal situation where the vehicle cannot normally travel in its lane. As a result, this anomaly detection device can accurately detect abnormal situations even when there are obstacles of varying color, shape, and size in the vehicle's lane, such as fallen objects or road defects.

[0051] In a modified example, the vehicle control unit 34 may change the control applied to the vehicle 10 depending on the duration for which an abnormal situation is detected. For example, if the duration for which an abnormal situation is detected is less than a first duration threshold, the vehicle control unit 34 warns the driver that an abnormal situation has been detected via a notification device. Furthermore, if the duration for which an abnormal situation is detected is greater than or equal to the first duration threshold but less than a second duration threshold (provided that the second duration threshold > the first duration threshold), the vehicle control unit 34 reduces the level of automatic driving control applied to the vehicle 10. In addition, if the duration for which an abnormal situation is detected is greater than or equal to the second duration threshold, the vehicle control unit 34 decelerates the vehicle 10. In this way, by changing the control applied to the vehicle 10 depending on the duration for which an abnormal situation is detected, the vehicle control unit 34 can control the vehicle 10 more appropriately to prevent danger to the vehicle 10.

[0052] In another modification, if the road surface condition feature is outside the normal range, the vehicle control unit 34 may change the control applied to the vehicle 10 according to the distance from the road surface condition feature to the normal range. For example, if the distance from the road surface condition feature to the normal range is less than a first distance threshold, the vehicle control unit 34 warns the driver via a notification device that an abnormal situation has been detected. Furthermore, if the distance from the road surface condition feature to the normal range is greater than or equal to the first distance threshold and less than a second distance threshold (provided that the second distance threshold > the first distance threshold), the vehicle control unit 34 reduces the level of automatic driving control applied to the vehicle 10. In addition, if the distance from the road surface condition feature to the normal range is greater than or equal to the second distance threshold, the vehicle control unit 34 decelerates the vehicle 10. In this way, by changing the control applied to the vehicle 10 according to the distance from the road surface condition feature to the normal range, the vehicle control unit 34 can control the vehicle 10 more appropriately to prevent danger to the vehicle 10.

[0053] The computer program that realizes the functions of the processor 23 of the ECU3 according to the above embodiment or modification may be provided in the form of being recorded on a computer-readable portable recording medium such as semiconductor memory, magnetic recording medium, or optical recording medium.

[0054] As described above, those skilled in the art can make various modifications within the scope of the present invention to suit the implemented form. [Explanation of Symbols]

[0055] 1. Vehicle control system 10 vehicles 2 cameras 3. Electronic control unit (ECU, anomaly detection device) 21 Communication Interface 22 memory 23 processors 31 Specific section 32 Extraction part 33 Detection unit 34 Vehicle Control Unit

Claims

1. A identifying unit in an image representing the area around a vehicle that identifies the road area representing the vehicle's own lane, An extraction unit that extracts the feature quantities representing the road surface conditions from the aforementioned image by inputting the road area into a feature extractor that has been pre-trained to extract the feature quantities representing the road surface conditions from the aforementioned image, A detection unit that detects an abnormal situation in which the vehicle cannot normally travel in its own lane if the feature quantity is not included in the normal range which represents the permissible range in which the vehicle can normally travel, A vehicle control unit that, when the aforementioned feature quantity falls outside the normal range, modifies the control applied to the vehicle according to the distance from the feature quantity to the normal range, It has, The vehicle control unit, If the aforementioned distance is less than the first distance threshold, the driver of the vehicle is warned via a notification device installed in the vehicle that an abnormal situation has been detected. If the distance is greater than or equal to the first distance threshold and less than a second distance threshold greater than the first distance threshold, the level of automatic driving control applied to the vehicle is reduced. If the distance is greater than or equal to the second distance threshold, the vehicle is decelerated. Anomaly detection device.

2. The anomaly detection device according to claim 1, wherein the extraction unit extracts the feature quantity by inputting the range below the lower end of the preceding vehicle area, which represents the preceding vehicle traveling directly in front of the vehicle, into the feature extractor.

3. In an image representing the area surrounding the vehicle, the road region representing the vehicle's own lane is identified. By inputting the road area into a feature extractor that has been pre-trained to extract features representing the road surface conditions from the aforementioned image, the features for the road area are extracted. If the feature quantity is not included in the normal range, which represents the permissible range within which the vehicle can normally drive, an abnormal situation is detected in which the vehicle cannot normally drive in its own lane. If the feature value falls outside the normal range, the control applied to the vehicle is changed according to the distance from the feature value to the normal range. This includes, Changing the control applied to the aforementioned vehicle means If the aforementioned distance is less than the first distance threshold, the driver of the vehicle is warned via a notification device installed in the vehicle that an abnormal situation has been detected. If the distance is greater than or equal to the first distance threshold and less than a second distance threshold greater than the first distance threshold, the level of automatic driving control applied to the vehicle is reduced. If the distance is greater than or equal to the second distance threshold, the vehicle is decelerated. An anomaly detection method that includes the following.

4. In an image representing the area surrounding the vehicle, the road region representing the vehicle's own lane is identified. By inputting the road area into a feature extractor that has been pre-trained to extract features representing the road surface conditions from the aforementioned image, the features for the road area are extracted. If the feature quantity is not included in the normal range, which represents the permissible range within which the vehicle can normally drive, an abnormal situation is detected in which the vehicle cannot normally drive in its own lane. If the feature value falls outside the normal range, the control applied to the vehicle is changed according to the distance from the feature value to the normal range. An anomaly detection computer program for causing a processor mounted on the vehicle to perform the following: Changing the control applied to the aforementioned vehicle means If the aforementioned distance is less than the first distance threshold, the driver of the vehicle is warned via a notification device installed in the vehicle that an abnormal situation has been detected. If the distance is greater than or equal to the first distance threshold and less than a second distance threshold greater than the first distance threshold, the level of automatic driving control applied to the vehicle is reduced. If the distance is greater than or equal to the second distance threshold, the vehicle is decelerated. A computer program for anomaly detection, including the following.

Images