Anomaly detection system and anomaly detection method

Abstract

To properly detect abnormality of an automatic operation system of a vehicle.SOLUTION: An abnormality detection system is applied to an automatic operation system of a target vehicle. A travel plan information indicates a travel plan of the target vehicle in a first section which is generated by an automatic operation system. Reference travel information indicates travel achievement of a reference vehicle different from the target vehicle in the first section. The abnormality detection system calculates, based on the travel plan information and the reference travel information, deviation per determination position within the first section between the travel plan of the target vehicle and the travel achievement of the reference vehicle. The abnormality detection system extracts the determination position where the deviation exceeds the threshold value as an abnormal position related to abnormality of the automatic operation system.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present disclosure relates to an anomaly detection technology applied to an autonomous driving system for a vehicle. [Background technology]

[0002] Patent Document 1 discloses an automated driving device. The automated driving device recognizes the driving environment based on the output signals of a perimeter monitoring sensor, and executes a control plan based on the recognition results. The automated driving device also monitors the stability of the recognition results from the perimeter monitoring sensor over time to determine whether there is an abnormality in the recognition system. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-024741 Summary of the Invention [Problem to be solved by the invention]

[0004] According to the technology disclosed in the above-mentioned Patent Document 1, the presence or absence of an abnormality is determined by monitoring the stability of the recognition results by the perimeter monitoring sensor over time. However, if an object that should be recognized is not recognized in the first place, it is not possible to determine the presence or absence of an abnormality. Furthermore, if erroneous recognition (false detection) of an object continues, it is not possible to determine the presence or absence of an abnormality. In other words, whether or not an abnormality can be accurately detected depends on the recognition performance of the recognition system including the perimeter monitoring sensor.

[0005] One object of the present disclosure is to provide a technology that can appropriately detect an abnormality in a vehicle's autonomous driving system. [Means for solving the problem]

[0006] The first aspect relates to an anomaly detection system applied to the autonomous driving system of a target vehicle. The anomaly detection system includes one or more processors and one or more storage devices. The one or more storage devices store the travel plan information and the reference travel information. The driving plan information indicates a driving plan for the target vehicle in the first section generated by the autonomous driving system. The reference driving information indicates the driving performance of a reference vehicle different from the target vehicle in the first section. The one or more processors calculate, for each determination position within the first section, a deviation between the travel plan of the target vehicle and the travel performance of the reference vehicle, based on the travel plan information and the reference travel information. The one or more processors extract the determined position where the deviation exceeds a threshold as an abnormal position related to an abnormality in the autonomous driving system.

[0007] The second aspect relates to an anomaly detection method applied to an autonomous driving system of a target vehicle. The anomaly detection method is performed by a computer. The anomaly detection method is Obtaining driving plan information indicating a driving plan for a target vehicle in a first section generated by an autonomous driving system; acquiring reference travel information indicating a travel performance of a reference vehicle different from the target vehicle in a first section; Calculating a deviation between the travel plan of the target vehicle and the travel performance of the reference vehicle for each determination position within the first section based on the travel plan information and the reference travel information; The judgment position where the deviation exceeds the threshold is extracted as an abnormal position related to an abnormality in the autonomous driving system. Includes. [Effects of the Invention]

[0008] According to the present disclosure, driving plan information indicating a driving plan for a target vehicle is compared with reference driving information indicating the driving history of a reference vehicle. Then, a location where the deviation between the driving plan for the target vehicle and the driving history of the reference vehicle exceeds a threshold is extracted as an abnormality location related to an abnormality in the autonomous driving system. Because this method uses the driving history of the reference vehicle as reference information, it does not depend on the recognition performance of the autonomous driving system. In other words, it is possible to appropriately detect an abnormality in the autonomous driving system 10 without relying on the recognition performance of the autonomous driving system. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a block diagram illustrating a configuration example of an autonomous driving system according to an embodiment. [Figure 2] 3 is a block diagram showing an example of driving environment information according to an embodiment; [Figure 3] FIG. 1 is a conceptual diagram for explaining an example of an abnormality related to automatic driving control. [Figure 4] FIG. 1 is a conceptual diagram for explaining an example of an abnormality related to automatic driving control. [Figure 5] 1 is a conceptual diagram for explaining an overview of an anomaly detection system according to an embodiment. [Figure 6] FIG. 1 is a conceptual diagram for explaining processing by an anomaly detection system according to an embodiment. [Figure 7] 1 is a block diagram illustrating a configuration example of an anomaly detection system according to an embodiment. [Figure 8] 1 is a flowchart illustrating an outline of processing performed by an anomaly detection system according to an embodiment. [Figure 9] FIG. 2 is a conceptual diagram for explaining a first application example of the anomaly detection system according to the embodiment. [Figure 10] FIG. 10 is a conceptual diagram for explaining a second application example of the anomaly detection system according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0011] 1. Autonomous Driving System 1 is a conceptual diagram for explaining an overview of an automatic driving system 10 according to this embodiment. The automatic driving system 10 controls the automatic driving of a vehicle 1. Typically, the automatic driving system 10 is mounted on the vehicle 1.

[0012] The autonomous driving system 10 includes a recognition sensor 20, a vehicle state sensor 30, a position sensor 40, a driving device 50, a communication device 60, and a control device 70. At least the recognition sensor 20, the vehicle state sensor 30, the position sensor 40, the driving device 50, and the communication device 60 are mounted on a vehicle 1.

[0013] The recognition sensor 20 recognizes (detects) the situation around the vehicle 1. Examples of the recognition sensor 20 include a camera, a LIDAR (Laser Imaging Detection and Ranging), and a radar. The vehicle state sensor 30 detects the state of the vehicle 1. For example, the vehicle state sensor 30 includes a speed sensor, an acceleration sensor, a yaw rate sensor, a steering angle sensor, and the like. The position sensor 40 detects the position and orientation of the vehicle 1. For example, the position sensor 40 includes a GNSS (Global Navigation Satellite System).

[0014] The traveling device 50 includes a steering device, a drive device, and a braking device. The steering device steers the wheels. For example, the steering device includes an electric power steering (EPS) device. The drive device is a power source that generates driving force. Examples of the drive device include an engine, an electric motor, and an in-wheel motor. The braking device generates braking force.

[0015] The communication device 60 communicates with the outside via a communication network. Examples of communication methods include mobile communication such as 5G and wireless LAN.

[0016] The control device 70 is a computer that controls the vehicle 1. Typically, the control device 70 is mounted on the vehicle 1. However, a part of the control device 70 may be disposed in an external device and control the vehicle 1 remotely. The control device 70 includes one or more processors 71 (hereinafter simply referred to as processor 71) and one or more storage devices 72 (hereinafter simply referred to as storage devices 72). The processor 71 executes various processes. For example, the processor 71 includes a CPU (Central Processing Unit). The storage device 72 stores various information. Examples of the storage device 72 include a volatile memory, a non-volatile memory, an HDD (Hard Disk Drive), an SSD (Solid State Drive), etc.

[0017] The control program 80 is a computer program for controlling the vehicle 1. The functions of the control device 70 are realized by cooperation between the processor 71 that executes the control program 80 and the storage device 72. The control program 80 is stored in the storage device 72. Alternatively, the control program 80 may be recorded on a computer-readable recording medium.

[0018] The control device 70 acquires driving environment information 90 that indicates the driving environment of the vehicle 1. The driving environment information 90 is stored in the storage device 72.

[0019] 2 is a block diagram showing an example of the driving environment information 90. The driving environment information 90 includes map information 91, surrounding situation information 92, vehicle state information 93, and vehicle position information 94.

[0020] The map information 91 includes a general navigation map. The map information 91 may indicate lane layouts and road shapes. The map information 91 may also include location information for structures, traffic lights, signs, and the like. The control device 70 acquires the map information 91 for a required area from a map database. The map database may be stored in the storage device 72, or may be stored in a map management device external to the vehicle 1. In the latter case, the control device 70 communicates with the map management device via the communication device 60 to acquire the required map information 91.

[0021] The surrounding situation information 92 is information obtained based on the recognition result by the recognition sensor 20, and indicates the situation around the vehicle 1. The control device 70 recognizes the situation around the vehicle 1 using the recognition sensor 20 and acquires the surrounding situation information 92. For example, the surrounding situation information 92 includes an image IMG captured by a camera. As another example, the surrounding situation information 92 includes point cloud information obtained by LIDAR.

[0022] The surrounding situation information 92 further includes object information OBJ related to objects (targets) around the vehicle 1. Examples of objects include pedestrians, bicycles, motorcycles, other vehicles (preceding vehicles, parked vehicles, etc.), white lines, traffic lights, structures (e.g., utility poles, pedestrian bridges), signs, and obstacles. The object information OBJ indicates the relative position and relative speed of the object with respect to the vehicle 1. For example, by analyzing images IMG acquired by a camera, it is possible to identify the object and calculate the relative position of the object. It is also possible to identify the object and obtain the relative position and relative speed of the object based on point cloud information acquired by LIDAR. The control device 70 may track the recognized object. In this case, the object information OBJ also includes trajectory information of the recognized object.

[0023] The vehicle state information 93 is information detected by the vehicle state sensor 30 and indicates the state of the vehicle 1. The state of the vehicle 1 includes the vehicle speed, acceleration, yaw rate, steering angle, etc. The control device 70 acquires the vehicle state information 93 from the vehicle state sensor 30. The vehicle state information 93 may indicate the driving state (automatic driving / manual driving) of the vehicle 1.

[0024] The vehicle position information 94 is information indicating the current position of the vehicle 1. The control device 70 acquires the vehicle position information 94 from the detection result by the position sensor 40. The control device 70 may also acquire highly accurate vehicle position information 94 by a well-known self-position estimation process (Localization) that uses the object information OBJ and the map information 91.

[0025] The control device 70 also performs vehicle driving control to control the driving of the vehicle 1. The vehicle driving control includes steering control, acceleration control, and deceleration control. The control device 70 performs vehicle driving control by controlling the driving devices 50 (steering device, drive device, braking device).

[0026] Furthermore, the control device 70 performs automatic driving control to control automatic driving of the vehicle 1. Here, automatic driving means that at least a portion of the steering, acceleration, and deceleration of the vehicle 1 is performed automatically, independent of the driver's operation. As an example, automatic driving of level 3 or higher may be performed. The control device 70 generates a driving plan for the vehicle 1 based on the driving environment information 90. Examples of the driving plan include maintaining the current driving lane, changing lanes, making right or left turns, and avoiding collisions with objects. More specifically, the driving plan includes a route plan and a speed plan for the vehicle 1. The route plan is a set of target positions for the vehicle 1. The speed plan is a set of target speeds for each target position. The combination of the route plan and the speed plan is also called a "target trajectory." In other words, the target trajectory includes the target position and target speed of the vehicle 1. The control device 70 performs vehicle driving control so that the vehicle 1 follows the target trajectory TR.

[0027] 2. Abnormalities related to automatic driving control 3 and 4 are conceptual diagrams for explaining various examples of abnormalities related to autonomous driving control. The X direction is the forward direction of the vehicle 1, or the direction in which the road extends. The Y direction is the lateral direction perpendicular to the X direction. For convenience, the vehicle 1 that is the target of autonomous driving control by the autonomous driving system 10 will be referred to as the "target vehicle 1" hereinafter. A leading vehicle 5 is present ahead of the target vehicle 1.

[0028] In the example shown in FIG. 3, there is a significant discrepancy between the route plan of the target vehicle 1 and the movement trajectory of the preceding vehicle 5. Specifically, in the example shown in (A) of FIG. 3, the preceding vehicle 5 is steering as if to avoid something. On the other hand, the driving plan of the target vehicle 1 does not include steering around the position where the preceding vehicle 5 has steered. In this case, there is a possibility that the autonomous driving system 10 has not recognized an object that should have been recognized. In other words, there is a possibility that an object has been missed (missed detection).

[0029] In the example shown in (B) of Figure 3, the driving plan of the target vehicle 1 includes steering in a direction away from the object. However, the leading vehicle 5 is traveling straight without steering. In this case, there is a possibility that the autonomous driving system 10 is misrecognizing (misdetecting) the object.

[0030] In the example shown in FIG. 4, there is a significant discrepancy between the speed plan of the target vehicle 1 and the speed profile of the preceding vehicle 5. Specifically, in the example shown in (A) of FIG. 4, the preceding vehicle 5 is decelerating for some reason. On the other hand, the driving plan of the target vehicle 1 does not include deceleration around the position where the preceding vehicle 5 decelerated. In this case, there is a possibility that the autonomous driving system 10 has not recognized an object that should have been recognized. In other words, there is a possibility that an object has been missed (missed detection).

[0031] In the example shown in (B) of Figure 4, the driving plan of the target vehicle 1 includes deceleration to reduce the risk of collision with an object. However, the preceding vehicle 5 does not decelerate at all. In this case, there is a possibility that the autonomous driving system 10 has erroneously recognized (misdetected) the object.

[0032] As described above, if there is a significant discrepancy between the driving plan of the target vehicle 1 and the driving record of the preceding vehicle 5, the discrepancy may be due to an abnormality in the autonomous driving system 10. Conversely, if such a discrepancy can be found, it is believed that an abnormality in the autonomous driving system 10 can be detected. This method uses the driving record of the preceding vehicle 5 as reference information, and therefore does not depend on the recognition performance of the autonomous driving system 10. In other words, it is possible to appropriately detect an abnormality in the autonomous driving system 10 without relying on the recognition performance of the autonomous driving system 10.

[0033] The "anomaly detection system" based on the above viewpoint will be described in detail below.

[0034] 3. Anomaly detection system Overview 5 is a conceptual diagram for explaining an overview of an anomaly detection system 100 according to this embodiment. The anomaly detection system 100 is applied to an autonomous driving system 10 of a target vehicle 1, and detects an anomaly in the autonomous driving system 10.

[0035] The anomaly detection system 100 may be mounted on the target vehicle 1, or may be included in a management device (management server) external to the target vehicle 1. In either case, the anomaly detection system 100 is configured to communicate with the autonomous driving system 10 of the target vehicle 1 and acquire necessary information from the autonomous driving system 10. The anomaly detection system 100 may be part of the autonomous driving system 10 of the target vehicle 1.

[0036] The anomaly detection system 100 acquires vehicle information VCL from the automatic driving system 10 of the target vehicle 1. The vehicle information VCL includes at least a driving plan for the target vehicle 1 generated by the automatic driving system 10. In particular, the vehicle information VCL includes a driving plan for the target vehicle 1 for a "first section SA" ahead of the target vehicle 1. The first section SA is, for example, a section of a predetermined distance along the road on which the target vehicle 1 is traveling. The driving plan for the target vehicle 1 includes a route plan and a speed plan for the target vehicle 1, i.e., a target trajectory for the target vehicle 1.

[0037] The driving plan information 101 is information indicating a driving plan for the target vehicle 1 in the first section SA. The abnormality detection system 100 acquires the driving plan information 101 based on vehicle information VCL obtained from the automatic driving system 10 of the target vehicle 1.

[0038] The reference travel information 102 is information that indicates the travel record of the reference vehicle 2 in the same first section SA. The reference vehicle 2 is a vehicle different from the target vehicle 1, but travels in the first section SA in the same way as the target vehicle 1. For example, the reference vehicle 2 is a preceding vehicle 5 (see Figures 3 and 4) that travels in the first section SA before the target vehicle 1. The preceding vehicle 5 here does not necessarily have to be the vehicle immediately preceding the target vehicle 1. The preceding vehicle 5 only needs to have passed through the first section SA within a certain period of time before the target vehicle 1 passes through the first section SA. As another example, the reference vehicle 2 may be a following vehicle that travels in the first section SA after the target vehicle 1.

[0039] The driving history of the reference vehicle 2 includes the route history and speed history of the reference vehicle 2. The route history of the reference vehicle 2 is a set of positions of the reference vehicle 2. The speed history of the reference vehicle 2 is a set of speeds for each position of the reference vehicle 2. The speed history may further include acceleration and jerk. There are various examples of methods for acquiring such reference driving information 102.

[0040] For example, the vehicle information VCL obtained from the autonomous driving system 10 of the target vehicle 1 further includes object information OBJ, vehicle state information 93, and vehicle position information 94. As described above, the object information OBJ includes information on the relative positions and relative speeds of surrounding vehicles (e.g., the preceding vehicle 5, the following vehicle) recognized by the recognition sensor 20. The object information OBJ may also include trajectory information of the surrounding vehicles recognized by the recognition sensor 20. The vehicle position information 94 and the vehicle state information 93 indicate the absolute position and absolute speed of the target vehicle 1, respectively. Based on this information, the anomaly detection system 100 can acquire information on the absolute positions and absolute speeds of surrounding vehicles around the target vehicle 1. That is, the anomaly detection system 100 can acquire reference driving information 102 that indicates the driving performance of a reference vehicle 2 (surrounding vehicle) around the target vehicle 1.

[0041] As another example, an infrastructure sensor 200 installed in the first section SA may be used. For example, the infrastructure sensor 200 includes an infrastructure camera. The infrastructure sensor 200 may also include a LIDAR. A reference vehicle 2 traveling in the first section SA is recognized (detected) by the infrastructure sensor 200. The anomaly detection system 100 communicates with the infrastructure sensor 200 and acquires information related to the recognition result by the infrastructure sensor 200. The position (trajectory) of the reference vehicle 2 is calculated based on the recognition result by the infrastructure sensor 200. The speed of the reference vehicle 2 is the change in the position of the reference vehicle 2 over time. Therefore, the anomaly detection system 100 can acquire reference travel information 102 indicating the travel performance of the reference vehicle 2 traveling in the first section SA.

[0042] As yet another example, a recognition sensor mounted on a third vehicle that is neither the target vehicle 1 nor the reference vehicle 2 may be used. The reference vehicle 2 traveling in the first section SA is recognized by the recognition sensor mounted on the third vehicle. The anomaly detection system 100 communicates with the third vehicle and acquires information regarding the recognition results by the recognition sensor. Based on this information, the anomaly detection system 100 can acquire reference travel information 102 that indicates the travel history of the reference vehicle 2 traveling in the first section SA.

[0043] In this way, the anomaly detection system 100 acquires driving plan information 101 that indicates the driving plan of the target vehicle 1 and reference driving information 102 that indicates the driving record of the reference vehicle 2. Based on the driving plan information 101 and the reference driving information 102, the anomaly detection system 100 compares the driving plan of the target vehicle 1 with the driving record of the reference vehicle 2 in the first section SA. More specifically, the anomaly detection system 100 compares the driving plan of the target vehicle 1 with the driving record of the reference vehicle 2 for each determination position within the first section SA.

[0044] 6 is a conceptual diagram for explaining the determination position. For example, a plurality of determination positions are set discretely within the first section SA ahead of the vehicle 1. The plurality of determination positions are spaced apart in the X direction. For example, the positions of waypoints on the target trajectory TR of the target vehicle 1 may be used as the determination positions.

[0045] The anomaly detection system 100 compares the driving plan of the target vehicle 1 with the driving record of the reference vehicle 2 for each determination position within the first section SA. Through this comparison, the anomaly detection system 100 calculates the deviation between the driving plan of the target vehicle 1 and the driving record of the reference vehicle 2 for each determination position within the first section SA. The anomaly detection system 100 then extracts determination positions where the deviation exceeds a threshold as abnormal positions related to an abnormality in the autonomous driving system 10. Other determination positions are determined to be normal positions.

[0046] For example, the anomaly detection system 100 compares the route plan of the target vehicle 1 with the route performance of the reference vehicle 2 for each determination position within the first section SA. In other words, the anomaly detection system 100 compares the Y-direction target position of the target vehicle 1 with the Y-direction position of the reference vehicle 2 for each determination position. Through this comparison, the anomaly detection system 100 calculates the positional deviation (Y-direction distance) between the route plan of the target vehicle 1 and the route performance of the reference vehicle 2 for each determination position. Then, the anomaly detection system 100 extracts a determination position where the positional deviation exceeds a first threshold as an anomaly position related to an anomaly in the autonomous driving system 10.

[0047] For example, the anomaly detection system 100 compares the speed plan of the target vehicle 1 with the actual speed of the reference vehicle 2 for each determination position within the first section SA. Through this comparison, the anomaly detection system 100 calculates the speed deviation between the speed plan of the target vehicle 1 and the actual speed of the reference vehicle 2 for each determination position. Then, the anomaly detection system 100 extracts the determination position where the speed deviation exceeds a second threshold as an anomaly position related to an anomaly in the autonomous driving system 10.

[0048] Extraction of the abnormality position means that there is an abnormality in the autonomous driving system 10 of the target vehicle 1. In this way, the abnormality detection system 100 can detect an abnormality in the autonomous driving system 10 of the target vehicle 1.

[0049] The anomaly detection process by the anomaly detection system 100 may be performed in real time or offline. For example, the anomaly detection system 100 may acquire the vehicle information VCL from the automatic driving system 10 in real time and determine in real time whether or not an anomaly exists in the automatic driving system 10. As another example, the anomaly detection system 100 may temporarily store the vehicle information VCL acquired from the automatic driving system 10 and verify whether or not an anomaly exists in the automatic driving system 10 at any timing.

[0050] 3-2.Configuration example 7 is a block diagram showing an example configuration of an anomaly detection system 100. The anomaly detection system 100 includes a communication device 110, one or more processors 120 (hereinafter simply referred to as processors 120), and one or more storage devices 130 (hereinafter simply referred to as storage devices 130).

[0051] The communication device 110 communicates with the target vehicle 1 (automated driving system 10), the infrastructure sensor 200, a third vehicle, etc. Examples of communication methods include mobile communication such as 5G and wireless LAN.

[0052] The processor 120 executes various processes. For example, the processor 120 includes a CPU. The storage device 130 stores various information. Examples of the storage device 130 include a volatile memory, a non-volatile memory, an HDD, and an SSD.

[0053] The anomaly detection program 140 is a computer program executed by the processor 120. The functions of the anomaly detection system 100 are realized by cooperation between the processor 120, which executes the anomaly detection program 140, and the storage device 130. The anomaly detection program 140 is stored in the storage device 130. Alternatively, the anomaly detection program 140 may be recorded on a computer-readable recording medium.

[0054] The processor 120 acquires vehicle information VCL from the autonomous driving system 10 of the target vehicle 1 via the communication device 110. The processor 120 may acquire information about the recognition result from the infrastructure sensor 200 via the communication device 110. The processor 120 may acquire information about the recognition result from a third vehicle via the communication device 110. The processor 120 stores the acquired information in the storage device 130. Furthermore, the processor 120 acquires driving plan information 101 and reference driving information 102 based on the acquired information. The processor 120 stores the driving plan information 101 and the reference driving information 102 in the storage device 130. Then, the processor 120 executes the above-described anomaly detection process based on the driving plan information 101 and the reference driving information 102.

[0055] FIG. 8 is a flowchart showing an outline of the abnormality detection process.

[0056] In step S101, the processor 120 acquires the travel plan information 101 indicating the travel plan of the target vehicle 1 in the first section SA.

[0057] In step S102, the processor 120 acquires the reference driving information 102 indicating the driving performance of the reference vehicle 2 in the first section SA.

[0058] In step S103, the processor 120 compares the travel plan information 101 with the reference travel information 102. Through this comparison, the processor 120 calculates the deviation between the travel plan of the target vehicle 1 and the travel record of the reference vehicle 2. The deviation is calculated for each determination position within the first section SA. Then, the process proceeds to step S104.

[0059] In step S104, processor 120 determines whether the deviation exceeds the threshold. If the deviation exceeds the threshold (step S104; Yes), the process proceeds to step S105. On the other hand, if the deviation is equal to or smaller than the threshold (step S104; No), the process proceeds to step S106.

[0060] In step S105, the processor 120 extracts the current determination position as an abnormality position related to an abnormality in the autonomous driving system 10. Then, the process proceeds to step S106.

[0061] In step S106, processor 120 determines whether the next determination position remains within first section SA. If the next determination position remains (step S106; No), the process returns to step S103, and the next determination position is selected. When the determination process is completed for all determination positions (step S106; Yes), the process ends.

[0062] Effects As described above, according to this embodiment, the driving plan information 101 indicating the driving plan of the target vehicle 1 is compared with the reference driving information 102 indicating the driving record of the reference vehicle 2. Then, a position where the deviation between the driving plan of the target vehicle 1 and the driving record of the reference vehicle 2 exceeds a threshold is extracted as an abnormality position related to an abnormality in the autonomous driving system 10. This method does not depend on the recognition performance of the autonomous driving system 10 because it uses the driving record of the reference vehicle 2 as reference information. In other words, it is possible to appropriately detect an abnormality in the autonomous driving system 10 without relying on the recognition performance of the autonomous driving system 10.

[0063] 4. Application Examples 4-1. First application example Fig. 9 is a conceptual diagram for explaining a first application example of the anomaly detection system 100. In the example shown in Fig. 9, the anomaly detection system 100 acquires vehicle information VCL from the autonomous driving system 10 in real time. The vehicle information VCL includes at least a driving plan for the target vehicle 1. Then, the anomaly detection system 100 determines whether or not an anomaly exists in the autonomous driving system 10 in real time.

[0064] Consider a case where an abnormality position is extracted in the first section SA. The extraction of the abnormality position means that an abnormality has been detected in the autonomous driving system 10 of the target vehicle 1. In this case, the abnormality detection system 100 provides feedback to the autonomous driving system 10 in real time to the abnormality detection. More specifically, the abnormality detection system 100 transmits notification information INF indicating the abnormality detection to the autonomous driving system 10. Upon receiving the notification information INF, the autonomous driving system 10 performs, for example, a fail operation. For example, the fail operation includes safely decelerating and stopping the target vehicle 1. As another example, the fail operation may include evacuating the target vehicle 1 to a predetermined safe position, such as the shoulder of the road.

[0065] The anomaly detection system 100 may explicitly instruct the automated driving system 10 to perform a fail operation. More specifically, the anomaly detection system 100 transmits notification information INF instructing the automated driving system 10 to perform a fail operation. In response to the notification information INF, the automated driving system 10 performs a fail operation.

[0066] In this way, the safety of the target vehicle 1 is ensured by performing a fail operation in response to the detection of an abnormality.

[0067] 4-2. Second example 10 is a conceptual diagram for explaining a second application example of the anomaly detection system 100. As described above, the autonomous driving system 10 performs autonomous driving control based on sensor detection information detected by various sensors (20, 30, 40) mounted on the target vehicle 1. The vehicle information VCL that the anomaly detection system 100 acquires from the autonomous driving system 10 includes not only the driving plan of the target vehicle 1, but also log data LOG related to the autonomous driving control.

[0068] For example, the log data LOG includes sensor detection information (e.g., image IMG, object information OBJ, vehicle state information 93, and vehicle position information 94) used for autonomous driving control. As another example, the log data LOG may include the control amount of the target vehicle 1 determined by the autonomous driving system 10. As yet another example, the log data LOG may include intermediate data used when calculating the control amount of the target vehicle 1 from the sensor detection information.

[0069] Consider the case where an abnormality position is extracted in the first section SA. The extraction of the abnormality position means that an abnormality has been detected in the autonomous driving system 10 of the target vehicle 1. In this case, the abnormality detection system 100 stores the log data LOG obtained in the storage target section in the storage device 130. The storage target section includes at least the extracted abnormality position. For example, the storage target section is a section corresponding to several seconds before and after the abnormality position.

[0070] The log data LOG stored in the storage device 130 is used, for example, to verify the autonomous driving system 10. The verification system 300 acquires the log data LOG for the target storage section. Then, the verification system 300 verifies the autonomous driving system 10 for the target storage section based on the log data LOG for the target storage section.

[0071] The log data LOG stored in the storage device 130 may be used for learning an autonomous driving AI (machine learning model). The learning system 400 acquires the log data LOG for the section to be saved as learning data. If the log data LOG includes an image IMG captured by a camera, annotation processing may be performed on the image IMG. In other words, the learning system 400 may acquire the log data LOG for the section to be saved as annotation target data. Useful learning data can be obtained by annotating the image IMG around the abnormality position. [Explanation of symbols]

[0072] 1...vehicle, 2...reference vehicle, 10...autonomous driving system, 20...recognition sensor, 70...control device, 90...driving environment information, 100...anomaly detection system, 101...travel plan information, 102...reference travel information, OBJ...object information, SA...first section, VCL...vehicle information

Claims

1. An anomaly detection system applied to an autonomous driving system of a target vehicle, one or more processors; one or more storage devices; Equipped with The one or more storage devices Travel plan information indicating a travel plan for the target vehicle in a first section generated by the autonomous driving system; Reference travel information indicating a travel record of a reference vehicle different from the target vehicle in the first section; Store the one or more processors: calculating a deviation between the travel plan of the target vehicle and the travel record of the reference vehicle for each determination position within the first section based on the travel plan information and the reference travel information; The determined position where the deviation exceeds a threshold is extracted as an abnormal position related to an abnormality of the autonomous driving system. Anomaly detection system.

2. The anomaly detection system according to claim 1, the travel plan of the target vehicle in the first section includes a route plan and a speed plan of the target vehicle in the first section; The driving performance of the reference vehicle in the first section includes a route performance and a speed performance of the reference vehicle in the first section. Anomaly detection system.

3. The anomaly detection system according to claim 1, the driving plan for the target vehicle in the first section includes a route plan for the target vehicle in the first section; the travel history of the reference vehicle in the first section includes a route history of the reference vehicle in the first section; the one or more processors: calculating a positional deviation between the route plan of the target vehicle and the route performance of the reference vehicle for each of the determination positions within the first section based on the travel plan information and the reference travel information; The determined position where the position deviation exceeds a first threshold is extracted as the abnormal position. Anomaly detection system.

4. The anomaly detection system according to claim 1, the travel plan for the target vehicle in the first section includes a speed plan for the target vehicle in the first section; the travel history of the reference vehicle in the first section includes a speed history of the reference vehicle in the first section; the one or more processors: calculating a speed deviation between the speed plan of the target vehicle and the actual speed of the reference vehicle for each of the determination positions within the first section based on the travel plan information and the reference travel information; The determined position where the speed deviation exceeds a second threshold is extracted as the abnormal position. Anomaly detection system.

5. The anomaly detection system according to claim 1, the reference vehicle is a surrounding vehicle recognized by a recognition sensor mounted on the target vehicle, The reference travel information is obtained from the recognition result by the recognition sensor. Anomaly detection system.

6. The anomaly detection system according to claim 5, The reference vehicle is a preceding vehicle traveling ahead of the target vehicle. Anomaly detection system.

7. The anomaly detection system according to claim 1, the reference vehicle is a vehicle recognized by an infrastructure sensor; The reference travel information is generated based on the recognition results of the infrastructure sensors. Anomaly detection system.

8. The anomaly detection system according to any one of claims 1 to 7, The one or more processors acquire the driving plan of the target vehicle from the autonomous driving system in real time; When the abnormality position is extracted in the first section, the one or more processors feed back an abnormality detection to the autonomous driving system. Anomaly detection system.

9. The anomaly detection system according to any one of claims 1 to 7, The one or more processors acquire the driving plan of the target vehicle from the autonomous driving system in real time; When the abnormal position is extracted in the first section, the one or more processors instruct the autonomous driving system to execute a fail operation. Anomaly detection system.

10. The anomaly detection system according to any one of claims 1 to 7, the autonomous driving system executes autonomous driving control to control autonomous driving of the target vehicle based on sensor detection information detected by a sensor mounted on the target vehicle; The one or more processors store log data related to the automatic driving control in the storage target section including the abnormal position in the one or more storage devices. Anomaly detection system.

11. An anomaly detection method applied to an autonomous driving system of a target vehicle, The anomaly detection method is executed by a computer, Obtaining driving plan information indicating a driving plan for the target vehicle in a first section generated by the autonomous driving system; acquiring reference travel information indicating a travel record of a reference vehicle different from the target vehicle in the first section; Calculating a deviation between the travel plan of the target vehicle and the travel record of the reference vehicle for each determination position within the first section based on the travel plan information and the reference travel information; extracting the determined position where the deviation exceeds a threshold as an abnormality position related to an abnormality of the autonomous driving system; Contains Anomaly detection methods.

Images