Abnormal detection device

Abstract

To suitably determine presence or absence of abnormality of a periphery recognition model of an automatic operating system.SOLUTION: An abnormality determination device which determines presence or absence of abnormality of a periphery recognition model which is a machine learning model used for periphery recognition of an automatic operating system of one's own vehicle includes: a prediction position estimation unit which, on the basis of a detection result of an external sensor of the own vehicle, uses the periphery recognition model to estimate a prediction position of a moving body around the own vehicle at a predetermined point in time; and an abnormality determination unit which, on the basis of a result of comparison between the prediction positions of the plurality of moving bodies at the predetermined point in time and detection positions of the plurality of moving bodies at the predetermined point in time, determines the presence or absence of the abnormality of the periphery recognition model.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to an abnormality determination device.

Background Art

[0002] Conventionally, as a technical document related to an abnormality determination device, Japanese Unexamined Patent Application Publication No. 2022-032109 is known. In this publication, in a collision prevention system that determines the reliability of the movement characteristics of an object to be determined for collision using detection results obtained from a plurality of different types of detectors, it is shown that, based on a predetermined index (the deviation between the posture of the target object and the velocity vector), it is determined that the error of the calculated posture or velocity vector of the target object is large and the reliability is low.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, it has been considered to use a machine learning model for the peripheral recognition of the host vehicle by an automatic driving system. In this case, when an abnormality occurs in the peripheral recognition of the automatic driving system, it is required to appropriately determine whether the abnormality is due to a sensor or the like or an abnormality in the machine learning model.

Means for Solving the Problems

[0005] One aspect of the present invention is an anomaly detection device for determining whether or not there is an anomaly in a surrounding recognition model, which is a machine learning model used for surrounding recognition in an autonomous driving system of a vehicle, comprising: a prediction position estimation unit that estimates the predicted position of moving objects around the vehicle at a predetermined time based on the detection results of external sensors of the vehicle, using the surrounding recognition model; and an anomaly detection unit that determines whether or not there is an anomaly in the surrounding recognition model based on the comparison result between the predicted positions of multiple moving objects at a predetermined time and the detected positions of multiple moving objects at a predetermined time. The anomaly detection unit determines that no anomaly has occurred in the surrounding recognition model if there are no more than a certain number of moving objects whose difference between the predicted position and the detected position is greater than or equal to the detection threshold. If there are more than a certain number of moving objects whose difference between the predicted position and the detected position is greater than or equal to the detection threshold, the unit determines whether the state of the predicted position and the detected position satisfies the pre-set anomaly occurrence conditions. The anomaly occurrence conditions include conditions other than having a difference greater than or equal to the detection threshold, and if it is determined that the anomaly occurrence conditions have been met, the unit determines that an anomaly has occurred in the surrounding recognition model. .

[0006] According to one aspect of the present invention, when an abnormality occurs in the surrounding recognition model of an automated driving system, it is considered that a characteristic effect will occur in the comparison result between the predicted position of multiple moving objects at a predetermined time and the detected position of multiple moving objects at a predetermined time. Based on this comparison result, it is possible to appropriately determine whether or not there is an abnormality in the surrounding recognition model.

[0007] In an abnormality detection device according to one aspect of the present invention, The conditions for an abnormality to occur include at least one of the following: 1) The detection positions of a predetermined number of moving objects are outside the sensor field of view of the external sensor; 2) The detection positions of a predetermined number of moving objects are concentrated within a predetermined range within the sensor field of view of the external sensor as viewed from the vehicle itself; 3) A certain number or more of moving objects exist for a certain period of time in which the difference between the predicted position and the detected position is greater than or equal to a judgment threshold; 4) A certain number or more of moving objects exist, which is greater than a certain number, in which the difference between the predicted position and the detected position is greater than or equal to a judgment threshold; and 5) The increase or decrease in the number of detected positions of moving objects at a predetermined time relative to the number of predicted positions of moving objects at a predetermined time is greater than or equal to an error threshold. That's good too. [Effects of the Invention]

[0008] According to one aspect of the present invention, it is possible to appropriately determine whether or not there is an abnormality in the surrounding recognition model of an autonomous driving system. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram showing an automated driving system (abnormality detection device) according to one embodiment. [Figure 2] This is a flowchart showing an example of the predictive position estimation process. [Figure 3] This flowchart shows an example of an anomaly detection process. [Modes for carrying out the invention]

[0010] Embodiments of the present invention will be described below with reference to the drawings.

[0011] Figure 1 is a block diagram showing an automated driving system (anomaly detection device) 100 according to one embodiment. The automated driving system 100 shown in Figure 1 is a system that performs automated driving of its own vehicle 1. Automated driving is a vehicle control system that allows the vehicle 1 to automatically drive along a preset route or along the road it is currently traveling on, without the driver performing any driving operations. Automated driving corresponds to, for example, automated driving level 2 or higher certified by the Society of Automotive Engineers (SAE). Automated driving may be limited to automated driving level 3 or higher.

[0012] The automated driving system 100 is configured to include an abnormality detection device 50. The abnormality detection device 50 is a device that determines whether or not there is an abnormality in the surrounding recognition performance of the automated driving system 100. Surrounding recognition performance refers to the performance of the vehicle 1 in terms of surrounding recognition (recognition of the external environment). Note that the abnormality detection device 50 does not necessarily have to be part of the automated driving system 100, and may be provided independently of the automated driving system 100.

[0013] The autonomous driving system 100 is equipped with an autonomous driving ECU 30 [Electronic Control Unit]. The autonomous driving ECU 30 is an electronic control unit having a CPU [Central Processing Unit] and a memory unit. The memory unit consists of, for example, ROM [Read Only Memory], RAM [Random Access Memory], EEPROM [Electrically Erasable Programmable Read-Only Memory], etc. The autonomous driving ECU 30 realizes various functions by executing programs stored in the memory unit with the CPU. The autonomous driving ECU 30 may be composed of multiple electronic units.

[0014] As shown in Figure 1, the autonomous driving ECU 30 is connected to external sensors, an external camera 10 and a radar sensor 11. The external camera 10 is an imaging device (external sensor) that captures images of the external situation of the vehicle 1. The external camera 10 is installed, for example, behind the windshield of the vehicle 1 and captures images of the area in front of the vehicle 1. The external camera 10 may also be installed on the side or rear of the vehicle 1 and configured to capture images of the area around the vehicle 1. The external camera 10 transmits the captured images of the area outside the vehicle 1 to the autonomous driving ECU 30.

[0015] The radar sensor 11 is a detection device (external sensor) that uses radio waves (e.g., millimeter waves) or light to detect objects around the vehicle 1. The radar sensor 11 includes, for example, millimeter-wave radar or LiDAR (Light Detection and Ranging) installed in multiple directions around the vehicle 1. The radar sensor 11 detects objects by transmitting radio waves or light around the vehicle and receiving the radio waves or light reflected by the objects. The radar sensor 11 transmits information about the detected objects to the autonomous driving ECU 30. Objects include fixed obstacles such as guardrails and buildings, as well as moving obstacles such as pedestrians, bicycles, and other vehicles.

[0016] Next, the functional configuration of the autonomous driving ECU 30 will be described. As shown in Figure 1, the autonomous driving ECU 30 has a surrounding area recognition unit 31, an abnormality determination unit 32, and a vehicle control unit 33. In this embodiment, the surrounding area recognition unit 31 and the abnormality determination unit 32 constitute the abnormality determination device 50 described above. Note that the abnormality determination device 50 does not necessarily have to include all of the surrounding area recognition unit 31, but may include the predictive position estimation unit 31c, which will be described later.

[0017] The surrounding recognition unit 31 performs surrounding recognition (recognition of the external environment) of the vehicle 1 in order to perform autonomous driving of the vehicle 1. The surrounding recognition unit 31 performs surrounding recognition of the vehicle 1 based on at least one of the images captured by the external camera 10 and the detection results of the radar sensor 11. The surrounding recognition unit 31 may also perform surrounding recognition of the vehicle 1 by so-called sensor fusion.

[0018] The surrounding recognition unit 31 has a surrounding recognition model 31a, a detection position recognition unit 31b, and a predicted position estimation unit 31c. The surrounding recognition model 31a is a machine learning model used for recognizing the surroundings of the host vehicle 1.

[0019] The surrounding recognition model 31a is a machine learning model trained by deep learning so as to output a surrounding recognition result from at least one of, for example, the captured image of the external camera 10 and the detection result (information on the detected object) of the radar sensor 11. The surrounding recognition result includes the predicted positions of moving objects around the host vehicle 1. A moving object is a movable object such as another vehicle, a pedestrian, or a bicycle. The surrounding recognition result may include the detection positions of moving objects around the host vehicle 1. The detection positions and the predicted positions will be described later.

[0020] The surrounding recognition model 31a is configured using a neural network [NeuralNetwork]. As the neural network, a convolutional neural network [CNN: Convolutional Neural Network] including a plurality of layers including a plurality of convolutional layers and pooling layers can be used. The surrounding recognition model 31a may be configured as a recurrent neural network [RNN: Recurrent neural network]. The configuration of the surrounding recognition model 31a is not particularly limited as long as it can output a surrounding recognition result as a machine learning model. The surrounding recognition model 31a may be composed of a plurality of machine learning models.

[0021] The detection position recognition unit 31b recognizes the detection position of a moving object around the host vehicle 1 based on at least one of the captured image of the external camera 10 and the detection result of the radar sensor 11. The moving object around the host vehicle 1 is a moving object included in the imaging range of the external camera 10 or the detection range of the radar sensor 11 of the host vehicle 1. Hereinafter, the imaging range and the detection range are referred to as the sensor field of view. The sensor field of view is determined in advance according to, for example, the specifications of each device and the settings of the user. The user is an occupant or a driver of the host vehicle 1. The detection position of the moving object is the position of the moving object detected by the external camera 10 or the radar sensor 11. The detection position is recognized in, for example, a vehicle coordinate system based on the host vehicle 1.

[0022] The detection position recognition unit 31b may recognize the detection position of the moving object using the surrounding recognition model 31a. The detection position recognition unit 31b may input at least one of the captured image of the external camera 10 and the detection result of the radar sensor 11 to cause the surrounding recognition model 31a to output the detection position of the moving object around the host vehicle 1. In addition to the detection position, the detection position recognition unit 31b may recognize the traveling direction and the moving speed of the moving object.

[0023] The predicted position estimation unit 31c estimates the predicted position of the moving object around the host vehicle 1 using the surrounding recognition model 31a. The predicted position estimation unit 31c estimates the predicted position of the moving object at a predetermined time in the future. The predetermined time may be 1 second ahead, 2 seconds ahead, 5 seconds ahead, or 10 seconds ahead. The predetermined time is not particularly limited. The predicted position estimation unit 31c may estimate the predicted positions for a plurality of predetermined times.

[0024] The predicted position estimation unit 31c outputs the predicted position of the moving object from the surrounding recognition model 31a with the detection position of the moving object by the detection position recognition unit 31b as an input. The predicted position estimation unit 31c may use, as an input, the detection positions of the moving object for a plurality of times included within a certain past time. The predicted position estimation unit 31c may use, as an input, the predicted position of the moving object estimated one time before. The predicted position estimation unit 31c may use, as an input, the traveling direction and the moving speed of the moving object recognized by the detection position recognition unit 31b.

[0025] The anomaly detection unit 32 determines whether or not there is an anomaly in the surrounding recognition model 31a. The anomaly detection unit 32 determines whether or not there is an anomaly in the surrounding recognition model 31a based on the comparison result between the predicted position of multiple moving objects at a predetermined time and the detected position of multiple moving objects at a predetermined time. If the surrounding recognition model 31a includes multiple machine learning models, it is assumed that the same machine learning model is used to estimate the predicted position.

[0026] First, we will explain the filtering of moving objects used for anomaly detection in the surrounding recognition model 31a (exclusion of moving objects exhibiting abnormal behavior). The anomaly detection unit 32 performs anomaly detection by excluding moving objects exhibiting abnormal behavior from among the moving objects in the vicinity of the vehicle 1.

[0027] The abnormality determination unit 32 excludes a moving object as an abnormally behaving object if, for example, the average difference between the detected position and the predicted position of other moving objects around the vehicle 1 is small, but there is a moving object whose difference between the detected position and the predicted position is exceptionally large. In other words, the abnormality determination unit 32 excludes a single moving object as an abnormally behaving object if, when the average difference between the detected position and the predicted position of other moving objects is less than the exclusion start determination threshold, the difference between the difference between the detected position and the predicted position of a single moving object and the above average value is greater than or equal to the abnormal behavior threshold. The exclusion start determination threshold and the abnormal behavior threshold can be set to any values ​​used to determine abnormally behaving objects.

[0028] Furthermore, if the difference between the detected position and the predicted position of a single moving object described above is not a distance that can actually be traveled, it may be related to an anomaly in the surrounding recognition model 31a, and therefore the single moving object will not be excluded as an abnormally behaving object. Whether or not it is a distance that can actually be traveled can be determined, for example, based on the attributes of the moving object (attributes such as four-wheeled vehicle, two-wheeled vehicle, bicycle, pedestrian, etc.) and the time difference between the time of prediction and the time at the predicted position. An upper limit value for the expected speed change is set for each attribute of the moving object. In addition, the anomaly determination unit 32 may exclude a moving object in an overtaking scene as an abnormally behaving object. An overtaking scene is a scene in which the target moving object is a vehicle, and there is a slow-moving vehicle or obstacle in front of the target moving object and the adjacent lane is clear.

[0029] If the abnormal behavior detection unit 32 excludes an abnormal behavior object, it may notify the user of the presence of an abnormal behavior object. The abnormal behavior detection unit 32 notifies the user of the location of the abnormal behavior object, etc., by displaying an image on the vehicle's display and outputting audio from the speaker, at least one of the above.

[0030] Next, the abnormality determination of the surrounding recognition model 31a will be explained. First, the abnormality determination unit 32 determines whether there is a certain number of moving objects whose difference between the predicted position and the detected position at a predetermined time is greater than or equal to a determination threshold. The certain number may be 2, 5, or 10. The certain number is not particularly limited.

[0031] The anomaly detection unit 32 calculates the difference (the distance between the predicted position and the detected position) between the predicted position of a moving object, for example, at 14:00, and the detected position of the moving object when it actually reaches 14:00, and counts the number of moving objects for which this difference is greater than or equal to the detection threshold. The detection threshold is a threshold value appropriately set for anomaly detection by the surrounding recognition model 31a. Hereafter, moving objects for which the difference between the predicted position and the detected position is greater than or equal to the detection threshold will be referred to as target moving objects.

[0032] The abnormality determination unit 32 determines that no abnormality has occurred in the surrounding recognition model 31a if there are no more than a certain number of moving objects whose difference between the predicted position and the detected position at a predetermined time is greater than or equal to a determination threshold.

[0033] The abnormality determination unit 32 determines whether a preset abnormality condition has been met if a certain number or more target moving objects exist. For example, the abnormality determination unit 32 determines that an abnormality condition has been met if the detection positions of a predetermined number of target moving objects are outside the sensor's field of view. The predetermined number may be a different value from or the same as the certain number described above. The predetermined number can be one or more.

[0034] The abnormality determination unit 32 may determine that the abnormality conditions have been met if the detection positions of a predetermined number of target moving objects are concentrated within a predetermined range within the sensor's field of view as viewed from the vehicle 1. Concentration within a predetermined range means, for example, that although the predicted positions of the target moving objects were estimated to be located in front of, behind, to the left and right of the vehicle 1 at a predetermined time, all of the target moving objects or a predetermined number of target moving objects are detected clustered together in a range diagonally to the right as viewed from the vehicle 1.

[0035] The anomaly determination unit 32 may determine that an anomaly has occurred if the direction of the deviation vectors between the detected position and predicted position of a predetermined number of target moving objects at a first predetermined time is different from the direction of the deviation vectors between the detected position and predicted position of the same target moving objects at a second predetermined time, and the magnitude of each deviation vector is greater than or equal to a vector threshold. A deviation vector is, for example, a vector calculated to connect the detected position and the predicted position in a plan view. Different directions of deviation vectors mean that the angle formed by the deviation vectors at multiple predetermined times for the same moving object is greater than or equal to a certain angle. The magnitude of the deviation vector corresponds to the distance between the detected position and the predicted position.

[0036] The abnormality detection unit 32 may determine that the abnormality occurrence condition has been met if the state in which a certain number or more target moving objects exist continues for a certain period of time. The certain period of time is not particularly limited and can be any period of time.

[0037] The abnormality determination unit 32 may determine that the abnormality condition has been met if the increase or decrease in the number of detected positions of the target moving object at a predetermined time relative to the number of predicted positions of the target moving object at that predetermined time is equal to or greater than an error threshold. The error threshold may be 3, 5, or 8. The error threshold is not particularly limited. The abnormality determination unit 32 may also perform the above determination on the premise that the surrounding environment of the vehicle 1 is unlikely to change. A case in which the surrounding environment of the vehicle 1 is unlikely to change is, for example, when the vehicle speed of the vehicle 1 is below a certain speed and visibility of the surroundings is good.

[0038] The abnormality determination unit 32 may determine that the abnormality condition has been met if there are a predetermined number or more target moving objects whose difference between the detected position and the predicted position is greater than or equal to the abnormality threshold. The abnormality threshold is not particularly limited and only needs to be a value greater than the determination threshold used to extract the target moving objects. In addition, the abnormality determination unit 32 may not perform the above determination in scenes where the moving object needs to suddenly change its driving state, such as in a construction zone.

[0039] The abnormality determination unit 32 may set different abnormality thresholds depending on the attributes of the target moving object (such as four-wheeled vehicles, two-wheeled vehicles, bicycles, and pedestrians). The abnormality determination unit 32 may determine that an abnormality has occurred in the surrounding recognition model 31a if there are a predetermined number or more target moving objects for which the difference between the detected position and the predicted position is greater than or equal to the abnormality threshold, using the abnormality threshold according to the attributes of the target moving object.

[0040] The abnormality determination unit 32 may determine that the abnormality condition has been met if the difference (absolute value) between the change in predicted position and the change in detected position between multiple predetermined time periods for a predetermined number of target moving objects is greater than or equal to an acceptable threshold. The acceptable threshold is not particularly limited, and different values ​​may be set depending on the attributes of the target moving objects.

[0041] The abnormality determination unit 32 may determine that an abnormality has occurred if the detection positions of a predetermined number of target moving objects overlap with other moving objects or structures such as walls. The abnormality determination unit 32 may determine the overlap using, for example, the size of the target moving objects predetermined according to the attributes of the target moving objects. The abnormality determination unit 32 may also recognize the actual size of each target moving object based on the captured image from the external camera 10 or the detection result from the radar sensor 11, and determine the overlap using the actual size of the target moving objects.

[0042] The anomaly detection unit 32 may determine that an anomaly has occurred if the comparison result of the detected position and predicted position of the target moving object is similar to a pre-set model anomaly occurrence pattern. A model anomaly occurrence pattern is a pattern of comparison results between the predicted position and the detected position that is pre-stored for anomaly detection of the surrounding recognition model 31a. Model anomaly occurrence patterns include, for example, a pattern in which the detected position or predicted position of the target moving object is biased to a specific range of the sensor field of view of the vehicle 1, and a pattern in which a blind spot area occurs in which the object is no longer detected once the predicted position enters a specific range of the sensor field of view of the vehicle 1. Model anomaly occurrence patterns are pre-set based on simulation data of the surrounding recognition model 31a.

[0043] If the abnormality detection unit 32 determines that the conditions for an abnormality have been met, it determines that an abnormality has occurred in the surrounding recognition model 31a. The abnormality detection unit 32 warns the user of the abnormality and transmits an abnormality signal to the vehicle control unit 33.

[0044] The vehicle control unit 33 performs automatic driving of the vehicle 1 based on the surrounding recognition results (including the detected position and predicted position of moving objects) from the surrounding recognition unit 31. The vehicle control unit 33 performs automatic driving by controlling the movement of the vehicle 1, for example, by transmitting control signals to engine actuators, brake actuators, and steering actuators (not shown).

[0045] If the vehicle control unit 33 detects an abnormality in the surrounding recognition model 31a of the automatic driving system 100 by the abnormality detection device 50 (abnormality detection unit 32) during the automatic driving of its own vehicle 1, it will warn the user of the abnormality. If the vehicle control unit 33 cannot continue automatic driving, it will notify the user that automatic driving cannot be continued and then terminate the automatic driving.

[0046] Next, the control method of the abnormality detection device 50 according to this embodiment will be described with reference to the drawings. Figure 2 is a flowchart showing an example of the predicted position estimation process. The predicted position estimation process is performed as part of the automatic driving process when the vehicle 1 is in automatic driving mode.

[0047] As shown in Figure 2, the abnormality detection device 50 (autonomous driving system 100) recognizes the detected position of moving objects around the vehicle 1 using the detection position recognition unit 31b in S1. The detection position recognition unit 31b recognizes the detected position of moving objects around the vehicle 1 based on at least one of the images captured by the external camera 10 and the detection results of the radar sensor 11. After that, the abnormality detection device 50 proceeds to S2.

[0048] As S2, the abnormality detection device 50 estimates the predicted position of moving objects around its own vehicle 1 using the predicted position estimation unit 31c. The predicted position estimation unit 31c takes the detected position of the moving object by the detected position recognition unit 31b as input and outputs the predicted position of the moving object from the surrounding recognition model 31a.

[0049] Figure 3 is a flowchart showing an example of an anomaly detection process. The anomaly detection process is executed each time, for example, the predicted position estimation process shown in Figure 2 is performed.

[0050] As shown in Figure 3, in S10, the abnormality detection device 50 uses the abnormality detection unit 32 to compare the predicted position and the detected position of the moving object at predetermined time intervals. After that, the abnormality detection device 50 proceeds to S11.

[0051] In S11, the abnormality determination device 50 excludes abnormal behavior objects from among the moving objects around the vehicle 1 that exhibit abnormal behavior, using the abnormality determination unit 32. For example, if the average difference between the detected position and the predicted position of other moving objects around the vehicle 1 is small, the abnormality determination unit 32 excludes the moving object as an abnormal behavior object if there is a moving object whose difference between the detected position and the predicted position is exceptionally large.

[0052] In S12, the abnormality detection device 50 determines whether there is a certain number of moving objects whose difference between the predicted position and the detected position is greater than or equal to the determination threshold, as determined by the abnormality detection unit 32. If the abnormality detection device 50 determines that there is a certain number of moving objects whose difference between the predicted position and the detected position is greater than or equal to the determination threshold (S12: YES), it proceeds to S13. If the abnormality detection device 50 does not determine that there is a certain number of moving objects whose difference between the predicted position and the detected position is greater than or equal to the determination threshold (S12: NO), it terminates the current abnormality detection process.

[0053] In S13, the abnormality detection device 50 determines whether the abnormality occurrence conditions have been met by the abnormality detection unit 32. If the abnormality detection device 50 determines that the abnormality occurrence conditions have been met (S13: YES), it proceeds to S14. If the abnormality detection device 50 determines that the abnormality occurrence conditions have not been met (S13: NO), it terminates the current abnormality detection process.

[0054] In S14, the abnormality detection device 50 determines, via the abnormality detection unit 32, that an abnormality has occurred in the surrounding recognition model 31a. Subsequently, in S15, the abnormality detection device 50 notifies the user of the abnormality in the surrounding recognition model 31a by transmitting an abnormality occurrence signal to the vehicle control unit 33 via the abnormality detection unit 32. The vehicle control unit 33 may, if necessary, warn the user and terminate the automated driving.

[0055] According to the abnormality determination device 50 of this embodiment described above, if an abnormality occurs in the surrounding recognition model of the automatic driving system 100, it is thought that a characteristic effect will occur in the comparison result between the predicted position of multiple moving objects at a predetermined time and the detected position of multiple moving objects at a predetermined time. Based on this comparison result, it is possible to appropriately determine whether or not there is an abnormality in the surrounding recognition model 31a.

[0056] Although embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above. The present invention can be implemented in various forms, starting with the embodiments described above, by making various changes and improvements based on the knowledge of those skilled in the art.

[0057] The anomaly detection unit 32 does not necessarily need to exclude abnormally behaving objects. The anomaly detection unit 32 may use all detected moving objects for anomaly detection in the surrounding recognition model 31a. Furthermore, the anomaly detection unit 32 does not necessarily need to extract target moving objects. The anomaly detection unit 32 may determine that an anomaly has occurred in the surrounding recognition model 31a if the anomaly occurrence conditions are met based on the detected position and predicted position of the moving object. [Explanation of symbols]

[0058] 1...Vehicle, 30...Automated driving ECU, 31...Surroundings recognition unit, 31a...Surroundings recognition model, 31b...Detected position recognition unit, 31c...Predicted position estimation unit, 32...Anomaly detection unit, 33...Vehicle control unit, 50...Anomaly detection device, 100...Automated driving system.

Claims

1. An anomaly detection device that determines whether or not there is an anomaly in the surrounding recognition model, which is a machine learning model used for surrounding recognition in the autonomous driving system of the vehicle, Based on the detection results of the vehicle's external sensors, the predictive position estimation unit estimates the predicted position of moving objects around the vehicle at a predetermined time using the surrounding recognition model. An anomaly determination unit determines whether or not there is an anomaly in the surrounding recognition model based on the comparison result between the predicted position of a plurality of moving objects at a predetermined time and the detected position of a plurality of moving objects at a predetermined time, Equipped with, The abnormality determination unit, If there are no more than a certain number of moving objects whose difference between the predicted position and the detected position is greater than or equal to the judgment threshold, It is determined that no abnormality has occurred in the aforementioned surrounding recognition model. If there are a certain number or more of the moving bodies whose difference between the predicted position and the detected position is greater than or equal to the determination threshold, It is determined whether the state of the predicted position and the detected position satisfies the pre-set abnormality occurrence conditions. The aforementioned abnormal occurrence conditions include conditions other than having a difference greater than or equal to the judgment threshold, An anomaly detection device that determines that an anomaly has occurred in the surrounding recognition model when it is determined that the aforementioned anomaly occurrence conditions have been met.

2. The abnormal occurrence conditions are: The first predetermined number of the moving bodies are outside the sensor field of view of the external sensor, Second, the detection positions of a predetermined number of the moving bodies are unevenly distributed within a predetermined range within the sensor field of view of the external sensor as viewed from the vehicle itself. The state in which a certain number or more of the moving bodies exist, where the difference between the predicted position and the detected position is greater than or equal to the judgment threshold, continues for a certain period of time. There are three predetermined numbers of moving bodies, more than the certain number, where the difference between the predicted position and the detected position is greater than or equal to an abnormal threshold which is greater than the judgment threshold. The increase or decrease in the number of detected positions of the moving body at the predetermined time relative to the number of predicted positions of the moving body at the predetermined time is equal to or greater than the error threshold. An abnormality determination device according to claim 1, comprising at least one of the following.

3. The abnormality determination device according to claim 1 or 2, wherein the abnormality determination unit determines whether or not there is an abnormality in the surrounding recognition model by excluding abnormal behavior objects from the moving objects that are exhibiting a preset abnormal behavior.

4. The abnormality determination device according to claim 3, wherein the abnormality determination unit notifies the user of the vehicle of the presence of the abnormal behavioral body.

Images