Anomaly diagnosis device, anomaly diagnosis method, and computer program

Abstract

To improve the accuracy of anomaly detection in vehicle self-position estimation. [Solution] The anomaly diagnosis device comprises a self-position estimation unit 14 that calculates the amount of change in the vehicle's self-estimated position based on the output of a GNSS receiver 2 installed on the vehicle 1; a first movement amount calculation unit 15 that calculates the amount of movement of the vehicle based on the output of a LiDAR 3 installed on the vehicle; a second movement amount calculation unit 16 that calculates the amount of movement of the vehicle based on the output of a camera 4 installed on the vehicle; and an anomaly detection unit 17 that detects an anomaly in the vehicle's self-position estimation. The first movement amount calculation unit calculates the amount of lateral movement of the vehicle, and the second movement amount calculation unit calculates the amount of vertical movement of the vehicle. The anomaly detection unit detects an anomaly in the vehicle's self-position estimation by comparing the amount of change in the self-estimated position with the estimated amount of movement of the vehicle obtained from the lateral and vertical movement amounts.

Description

Technical Field

[0001] The present invention relates to an abnormality diagnosis device, an abnormality diagnosis method, and a computer program.

Background Art

[0002] Patent Document 1 describes that a vehicle such as a dump truck is provided with a front LiDAR and a rear LiDAR, and outputs of the front LiDAR and the rear LiDAR are used to detect whether there is any abnormality in the self-position of the vehicle calculated by the self-position estimation device based on GPS positioning data.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the above technique, abnormality diagnosis of self-position estimation is performed based on whether road shoulder feature points detected based on the output of the rear LiDAR are located on the trajectory of road shoulder feature points detected based on the output of the front LiDAR. In this case, even if an abnormality in the lateral self-position estimation of the vehicle can be detected according to the amount of deviation from the trajectory, it is difficult to detect an abnormality in the longitudinal (traveling direction) self-position estimation of the vehicle.

[0005] Accordingly, in view of the above problems, an object of the present invention is to improve the accuracy of abnormality diagnosis of self-position estimation of a vehicle.

Means for Solving the Problems

[0006] The gist of the present disclosure is as follows.

[0007] (1) An anomaly diagnosis device comprising: a self-position estimation unit that calculates the amount of change of the self-estimated position of the vehicle based on the output of a GNSS receiver provided on the vehicle; a first movement amount calculation unit that calculates the amount of movement of the vehicle based on the output of a LiDAR provided on the vehicle; a second movement amount calculation unit that calculates the amount of movement of the vehicle based on the output of a camera provided on the vehicle; and an anomaly detection unit that detects an anomaly in the self-position estimation of the vehicle, wherein the first movement amount calculation unit calculates the amount of lateral movement of the vehicle, the second movement amount calculation unit calculates the amount of vertical movement of the vehicle, and the anomaly detection unit detects an anomaly in the self-position estimation of the vehicle by comparing the amount of change of the self-estimated position with the estimated amount of movement of the vehicle obtained from the amount of lateral movement and the amount of vertical movement.

[0008] (2) The abnormality diagnosis device according to (1) above, wherein the first movement amount calculation unit calculates the amount of movement of the vehicle based on the output of the LiDAR and the output of an IMU provided on the vehicle, and the second movement amount calculation unit calculates the amount of movement of the vehicle based on the output of the camera and the output of the IMU.

[0009] (3) The abnormality diagnosis device according to (1) or (2) above, wherein the first movement amount calculation unit calculates the amount of movement in the lateral direction and the amount of movement in the vertical direction, and the abnormality detection unit detects the abnormality by comparing the amount of change of the self-estimated position with the estimated amount of movement of the vehicle obtained from the amount of movement in the lateral direction calculated by the first movement amount calculation unit and the amount of movement in the vertical direction calculated by the second movement amount calculation unit when the amount of movement in the vertical direction calculated by the first movement amount calculation unit is less than a predetermined value, and detects the abnormality by comparing the amount of change of the self-estimated position with the estimated amount of movement of the vehicle obtained from the amount of movement in the lateral direction and the amount of movement in the vertical direction calculated by the first movement amount calculation unit.

[0010] (4) An anomaly diagnosis method performed by a computer, comprising: calculating a change in the self-estimated position of the vehicle based on the output of a GNSS receiver installed on the vehicle; calculating a lateral movement of the vehicle based on the output of a LiDAR installed on the vehicle; calculating a longitudinal movement of the vehicle based on the output of a camera installed on the vehicle; and detecting an anomaly in the self-position estimation of the vehicle by comparing the change in the self-estimated position with the estimated movement of the vehicle obtained from the lateral movement and the longitudinal movement.

[0011] (5) A computer program that causes a computer to perform the following actions: calculate the amount of change in the vehicle's self-estimated position based on the output of a GNSS receiver installed on the vehicle; calculate the amount of lateral movement of the vehicle based on the output of a LiDAR installed on the vehicle; calculate the amount of vertical movement of the vehicle based on the output of a camera installed on the vehicle; and detect an anomaly in the vehicle's self-position estimation by comparing the amount of change in the self-estimated position with the estimated amount of movement of the vehicle obtained from the amount of lateral movement and the amount of vertical movement. [Effects of the Invention]

[0012] According to the present invention, the accuracy of anomaly diagnosis in vehicle self-position estimation can be improved. [Brief explanation of the drawing]

[0013] [Figure 1] This is a schematic diagram of an abnormality diagnosis system including an abnormality diagnosis device according to the first embodiment of the present invention. [Figure 2] This is a functional block diagram of the ECU processor. [Figure 3] This diagram schematically illustrates a situation where there is a discrepancy in the vehicle's self-position estimation results. [Figure 4] This is a flowchart showing the control routine for abnormality diagnosis processing in the first embodiment. [Figure 5]This is a flowchart showing the control routine for abnormality diagnosis processing in the second embodiment. [Figure 6] This flowchart shows the control routine for abnormality diagnosis processing in the third embodiment. [Modes for carrying out the invention]

[0014] Embodiments of the present invention will be described in detail below with reference to the drawings. In the following description, similar components will be given the same reference numerals.

[0015] <First Embodiment> The first embodiment of the present invention will be described below with reference to Figures 1 to 4. Figure 1 is a schematic configuration diagram of an anomaly diagnosis system 100 including an anomaly diagnosis device according to the first embodiment of the present invention. The anomaly diagnosis system 100 is mounted on a vehicle 1 and performs anomaly diagnosis of the vehicle 1's self-position estimation. In this embodiment, the vehicle 1 is a four-wheeled automobile.

[0016] As shown in Figure 1, the anomaly diagnosis system 100 comprises a GNSS receiver 2, a LiDAR (Laser Imaging Detection And Ranging) 3, a camera 4, an inertial measurement unit (IMU) 5, a vehicle speed sensor 6, an actuator 7, a map database 8, an output device 9, and an electronic control unit (ECU) 10. Specifically, the GNSS receiver 2, LiDAR 3, camera 4, IMU 5, vehicle speed sensor 6, actuator 7, map database 8, output device 9, and ECU 10 are installed in the vehicle 1. The GNSS receiver 2, LiDAR 3, camera 4, IMU 5, vehicle speed sensor 6, actuator 7, map database 8, and output device 9 are electrically connected to the ECU 10 via an in-vehicle network compliant with standards such as CAN (Controller Area Network) or Ethernet.

[0017] The GNSS receiver 2 measures the self-position of the vehicle 1 and generates self-position data of the vehicle 1. Specifically, the GNSS receiver 2 detects the current position of the vehicle 1 (for example, the latitude, longitude, and altitude of the vehicle 1) based on positioning information obtained from a plurality of (for example, three or more) positioning satellites. A specific example of the GNSS receiver includes a GPS (Global Positioning System) receiver. The output of the GNSS receiver 2, that is, the current position of the vehicle 1 detected by the GNSS receiver 2, is transmitted to the ECU 10.

[0018] The LiDAR 3 irradiates laser light around the vehicle 1 and receives reflected light of the laser light. Then, the LiDAR 3 generates point cloud data representing features around the vehicle 1 based on the reflected light of the laser light. The point cloud data includes information such as the distance, reflection intensity, and irradiation angle of each reflection point. For example, the LiDAR 3 is disposed at a front portion of the vehicle 1 (for example, a front bumper of the vehicle 1), and detects features in front of and on the front side of the vehicle 1. The output of the LiDAR 3, that is, the point cloud data generated by the LiDAR 3, is transmitted to the ECU 10.

[0019] The camera 4 captures an image of the surroundings of the vehicle 1 to generate a surrounding image of the vehicle 1. For example, the camera 4 is disposed at a front portion of the vehicle 1 (for example, a front bumper of the vehicle 1, a back surface of a rearview mirror in a vehicle interior, etc.), and generates images in front of and on the front side of the vehicle 1. The camera 4 may be a monocular camera or a stereo camera. The output of the camera 4, that is, the surrounding image generated by the camera 4, is transmitted to the ECU 10.

[0020] The IMU 5 includes a 3-axis gyro sensor that detects angular velocities in three axis directions and a 3-axis acceleration sensor that detects accelerations in three axis directions, so as to detect dynamic behavior of the vehicle 1. With this configuration, the IMU 5 can sample the posture change, turning operation, and acceleration / deceleration operation of the vehicle 1 at a high frequency. For example, the IMU 5 is disposed near the center of gravity of the vehicle 1. The output of the IMU 5, that is, the angular velocities and accelerations in three axis directions detected by the IMU 5, is transmitted to the ECU 10.

[0021] The vehicle speed sensor 6 detects the speed of vehicle 1. For example, the vehicle speed sensor 6 detects the speed of vehicle 1 by detecting the rotational speed of the vehicle's wheels. The output of the vehicle speed sensor 6, i.e., the speed of vehicle 1 detected by the vehicle speed sensor 6, is transmitted to the ECU 10.

[0022] The actuator 7 operates the vehicle 1 in response to instructions from the ECU 10. The actuator 7 comprises a drive actuator, a brake actuator, and a steering actuator. The drive actuator controls the acceleration of the vehicle 1 via the vehicle's drive system (e.g., an internal combustion engine, an electric motor, or a hybrid configuration of both). The brake actuator controls the braking of the vehicle 1 via the vehicle's brake system. The steering actuator controls the steering of the vehicle 1 via the vehicle's steering system. In this embodiment, the ECU 10 uses the actuator 7 to control the behavior of the vehicle 1 (e.g., the acceleration, braking, and steering of the vehicle 1). That is, the vehicle 1 is an autonomous vehicle in which the acceleration, braking, and steering of the vehicle 1 are automatically controlled.

[0023] Map information is stored in the map database 8. This map information includes, for example, road location information, road shape information (e.g., type of curve and straight section, curve curvature, road gradient, etc.), road type, speed limit, and other road information. Alternatively, the map database 8 may be located outside the vehicle 1 (e.g., on a server), and the ECU 10 may acquire map information from outside the vehicle 1.

[0024] Output device 9 provides notifications to the occupants of vehicle 1 (e.g., the driver). Output device 9 includes at least one of a display, a warning light, a speaker, a buzzer, and a vibration unit. Output device 9 notifies the occupants of vehicle 1 of an output corresponding to a signal transmitted from ECU 10.

[0025] The ECU 10 performs various controls on the vehicle 1. As shown in Figure 1, the ECU 10 includes a communication interface 11, a memory 12, and a processor 13. The communication interface 11 and the memory 12 are connected to the processor 13 via signal lines. In this embodiment, one ECU 10 is provided, but multiple ECUs may be provided for each function. Furthermore, the communication interface 11, the memory 12, and the processor 13 may be configured as a single integrated circuit, or they may be configured as separate circuits.

[0026] The communication interface 11 has an interface circuit for connecting the ECU 10 to the in-vehicle network. The ECU 10 is connected to other in-vehicle equipment via the communication interface 11. The communication interface 11 transmits signals received from the GNSS receiver 2, LiDAR 3, camera 4, IMU 5, vehicle speed sensor 6, and map database 8 to the processor 13. The communication interface 11 also transmits signals output from the processor 13 to the actuator 7 and output device 9.

[0027] Memory 12 includes, for example, volatile semiconductor memory (e.g., DRAM (Dynamic Random Access Memory), SRAM (Static Random Access Memory), etc.) and non-volatile semiconductor memory (e.g., ROM (Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), flash memory, etc.). Memory 12 stores temporary data, computer programs used for various processes by the processor 13 (control programs for the ECU 10), ECU 10 setting data, log data, vehicle information, etc.

[0028] The processor 13 has one or more CPUs (Central Processing Units) and their peripheral circuits. The processor 13 executes computer programs stored in the memory 12. The processor 13 may also have other arithmetic circuits such as a logical operation unit, a numerical operation unit, or a graphics processing unit.

[0029] In this embodiment, the ECU 10 functions as an anomaly diagnosis device that diagnoses anomalies in the self-position estimation of the vehicle 1. Note that the ECU 10 is just one example of an anomaly diagnosis device.

[0030] Figure 2 is a functional block diagram of the processor 13 of the ECU 10. As shown in Figure 2, the processor 13 includes a self-position estimation unit 14, a first movement amount calculation unit 15, a second movement amount calculation unit 16, and an anomaly detection unit 17. The self-position estimation unit 14, the first movement amount calculation unit 15, the second movement amount calculation unit 16, and the anomaly detection unit 17 are functional modules that are realized by the execution of a computer program stored in the memory 12 of the ECU 10 by the processor 13 of the ECU 10. Note that each of these functional modules may be realized by a dedicated arithmetic circuit provided in the processor 13.

[0031] The self-position estimation unit 14 estimates the self-position of the vehicle 1. In this embodiment, the self-position estimation unit 14 obtains the self-estimated position of the vehicle 1 based on the output of the GNSS receiver 2. In this embodiment, the self-estimated position of the vehicle 1 is used for control of the vehicle 1 for autonomous driving. For example, the ECU 10 generates a driving path for the vehicle 1 based on the self-estimated position of the vehicle 1.

[0032] Figure 3 schematically illustrates a situation where there is a discrepancy in the estimated position of vehicle 1. In Figure 3, the actual position of vehicle 1 at time t and time t+1 is shown by a solid line, and the self-estimated position of vehicle 1 at time t+1 is shown by a dashed line. In the example in Figure 3, there is a discrepancy between the actual position of vehicle 1 and the self-estimated position of vehicle 1 at time t+1. This discrepancy is caused by the effects of multipath, clock errors, radio interference, etc.

[0033] If a discrepancy occurs between the actual self-position and the self-estimated position, it becomes difficult for vehicle 1 to continue autonomous driving. Therefore, it is desirable for vehicle 1 to be able to diagnose abnormalities in its self-position estimation. For example, it is conceivable to diagnose abnormalities in self-position estimation by comparing the amount of change in vehicle 1's self-estimated position calculated based on GNSS receiver 2 with the amount of movement of vehicle 1 calculated based on LiDAR 3.

[0034] However, in point cloud data generated by LiDAR3, the vertical resolution of vehicle 1 is usually lower than the horizontal resolution of vehicle 1. Also, unlike camera 4, LiDAR3 cannot acquire detailed texture information. For this reason, even if the above method can detect the deviation of vehicle 1's self-estimated position in the horizontal direction, it is difficult to detect the deviation of vehicle 1's self-estimated position in the vertical direction. This is particularly noticeable when only features with little shape variation in the vertical direction of vehicle 1 (e.g., road shoulder (see Figure 3), embankment, etc.) exist around vehicle 1. In this specification, the vertical direction of vehicle 1 means the direction of vehicle 1's movement (up and down direction in Figure 3), and the horizontal direction of vehicle 1 means the direction perpendicular to the direction of vehicle 1's movement (left and right direction in Figure 3).

[0035] In view of the above, in this embodiment, the amount of movement of the vehicle 1 is calculated using not only the LiDAR 3 but also the camera 4 to perform anomaly diagnosis of the vehicle 1's self-position estimation. This improves the accuracy of anomaly diagnosis of the vehicle 1's self-position estimation. The specific control is described below.

[0036] The self-position estimation unit 14 calculates the amount of change in the self-estimated position of the vehicle 1 (hereinafter simply referred to as "amount of change in self-estimated position") based on the output of the GNSS receiver 2. The self-position estimation unit 14 calculates the amount of change in the self-estimated position at predetermined time intervals. For example, the self-position estimation unit 14 obtains the self-estimated position at time t and time t=1 based on the output of the GNSS receiver 2, and calculates the difference between these self-estimated positions as the amount of change in self-estimated position.

[0037] Meanwhile, the first movement calculation unit 15 calculates the movement of the vehicle 1 based on the output of the LiDAR 3, and the second movement calculation unit 16 calculates the movement of the vehicle 1 based on the output of the camera 4. In this case, the first movement calculation unit 15 is also referred to as the LiDAR odometry calculation unit, and the second movement calculation unit 16 is also referred to as the camera odometry calculation unit.

[0038] In particular, in this embodiment, the first movement calculation unit 15 calculates the lateral movement of the vehicle 1 based on the output of the LiDAR 3, and the second movement calculation unit 16 calculates the longitudinal movement of the vehicle 1 based on the output of the camera 4. By using the LiDAR 3 and the camera 4 together, the shortcomings of each can be compensated for, and consequently, the longitudinal and lateral movement of the vehicle 1 can be calculated with high accuracy.

[0039] The first movement amount calculation unit 15 calculates the lateral movement of the vehicle 1 at predetermined time intervals. For example, the first movement amount calculation unit 15 calculates the lateral movement of the vehicle 1 between time t and time t+1. The second movement amount calculation unit 16 calculates the longitudinal movement of the vehicle 1 at predetermined time intervals. For example, the second movement amount calculation unit 16 calculates the longitudinal movement of the vehicle 1 between time t and time t+1.

[0040] The anomaly detection unit 17 detects an anomaly in the self-position estimation of vehicle 1 by comparing the amount of change in the self-estimated position calculated by the self-position estimation unit 14 with the estimated amount of movement of vehicle 1 obtained from the lateral movement of vehicle 1 calculated by the first movement amount calculation unit 15 and the vertical movement of vehicle 1 calculated by the second movement amount calculation unit 16. For example, the anomaly detection unit 17 determines that the self-position estimation of vehicle 1 is abnormal when the difference between the amount of change in the self-estimated position and the estimated amount of movement of vehicle 1 is greater than or equal to a predetermined threshold, and determines that the self-position estimation of vehicle 1 is normal when this difference is less than the threshold.

[0041] The following describes the process flow for executing the control described above, with reference to the flowchart in Figure 4. Figure 4 is a flowchart of the control routine related to the abnormality diagnosis process in the first embodiment. This control routine is repeatedly executed at predetermined execution intervals by the processor 13 of the ECU 10, for example, according to a computer program stored in the memory 12 of the ECU 10.

[0042] First, in step S101, the self-position estimation unit 14 of the processor 13 estimates the self-position of the vehicle 1 using the GNSS receiver 2. Specifically, the self-position estimation unit 14 obtains the self-estimated position of the vehicle 1 based on the output of the GNSS receiver 2.

[0043] Next, in step S102, the self-position estimation unit 14 calculates the amount of change in the self-estimated position of the vehicle 1 over a predetermined time interval. For example, the predetermined time interval is the execution interval of this control routine, and the self-position estimation unit 14 calculates the difference between the self-estimated position obtained in step S101 of the current control routine and the self-estimated position obtained in step S101 of the previous control routine as the amount of change in the self-estimated position.

[0044] Next, in step S103, the first movement amount calculation unit 15 of the processor 13 calculates the lateral movement amount of the vehicle 1 in the predetermined time interval based on the output of the LiDAR 3. If the predetermined time interval is the time interval between time t and time t+1, the first movement amount calculation unit 15 calculates the lateral movement amount of the vehicle 1 by comparing the point cloud data generated by the LiDAR 3 at time t with the point cloud data generated by the LiDAR 3 at time t+1. The point cloud data at time t is the point cloud data from the previous frame, and the point cloud data at time t+1 is the point cloud data from the current frame.

[0045] For example, the first movement calculation unit 15 calculates the lateral movement of vehicle 1 using a LiDAR odometry method. Specifically, the first movement calculation unit 15 calculates the lateral movement of vehicle 1 by matching feature points of point cloud data between preceding and succeeding frames and calculating coordinate transformation parameters (e.g., a coordinate transformation matrix) of the two point cloud data. When calculating the coordinate transformation parameters, the first movement calculation unit 15 can use known algorithms such as ICP (Iterative Closest Point) or NDT (Normal Distributions Transform). In addition, the first movement calculation unit 15 can use known methods such as RANSAC (Random Sample Consensus) or Statistical Outlier Removal (SOR) to remove outliers or noise from the point cloud data.

[0046] Next, in step S104, the second movement amount calculation unit 16 of the processor 13 calculates the amount of vertical movement of the vehicle 1 in the predetermined time interval based on the output of the camera 4. If the predetermined time interval is the time interval between time t and time t+1, the second movement amount calculation unit 16 calculates the amount of vertical movement of the vehicle 1 by comparing the image generated by the camera 4 at time t with the image generated by the camera 4 at time t+1. The image at time t is the image from the previous frame, and the image at time t+1 is the image from the current frame.

[0047] For example, the second movement calculation unit 16 calculates the vertical movement of vehicle 1 using the Visual Odometry method. Specifically, the second movement calculation unit 16 calculates the vertical movement of vehicle 1 by matching feature points of images between preceding and succeeding frames and estimating the ego-motion of camera 4. When matching feature points of two images, the second movement calculation unit 16 can use known methods such as nearest neighbor search or k-nearest neighbor search. Furthermore, when estimating the ego-motion of camera 4, the second movement calculation unit 16 can use known methods such as essential matrix estimation or homography matrix estimation.

[0048] Next, in step S105, the anomaly detection unit 17 of the processor 13 determines whether the difference between the change in the self-estimated position and the estimated movement of the vehicle 1 obtained from the lateral and vertical movement of the vehicle 1 is greater than or equal to a predetermined threshold. The threshold is predetermined considering the desired accuracy of the self-position estimation of the vehicle 1, etc. For example, the anomaly detection unit 17 determines that the difference between the change in the self-estimated position and the estimated movement of the vehicle 1 is greater than or equal to a threshold when at least one of the difference between the lateral movement of the self-estimated position and the lateral movement of the vehicle 1, and the difference between the vertical movement of the self-estimated position and the vertical movement of the vehicle 1 is greater than or equal to a threshold. Alternatively, the anomaly detection unit 17 may determine that the difference between the change in the self-estimated position and the estimated movement of the vehicle 1 is greater than or equal to a threshold when both of the above two differences are greater than or equal to a threshold. Alternatively, the anomaly detection unit 17 may calculate the change in the self-estimated position and the estimated movement of the vehicle 1 as vectors, and by comparing these vectors, determine whether the difference between the change in the self-estimated position and the estimated movement of the vehicle 1 is greater than or equal to a threshold.

[0049] If, in step S105, it is determined that the difference between the change in the self-estimated position and the estimated movement of vehicle 1 is greater than or equal to a threshold, the control routine proceeds to step S106. In step S106, the anomaly detection unit 17 determines that the self-position estimation of vehicle 1 is abnormal. That is, the anomaly detection unit 17 detects an abnormality in the self-position estimation of vehicle 1. In this case, for example, the anomaly detection unit 17 notifies the occupant of vehicle 1 of the anomaly via the output device 9 and uses the actuator 7 to stop vehicle 1 in a safe place such as the shoulder of the road. The anomaly detection unit 17 may also request the occupant of vehicle 1 to switch from automatic driving to manual driving (a so-called handover) via the output device 9. After step S106, the control routine terminates.

[0050] On the other hand, if it is determined in step S105 that the difference between the change in the self-estimated position and the estimated movement of vehicle 1 is less than a threshold, the control routine proceeds to step S107. In step S107, the anomaly detection unit 17 determines that the self-position estimation of vehicle 1 is normal. After step S107, the control routine terminates.

[0051] In step S101, the self-position estimation unit 14 may estimate the vehicle's position using at least one of the IMU 5 and the vehicle speed sensor 6, in addition to the GNSS receiver 2.

[0052] <Second Embodiment> The configuration and control of the anomaly diagnosis device according to the second embodiment are basically the same as those of the anomaly diagnosis device according to the first embodiment, except for the points described below. Therefore, the second embodiment of the present invention will be described below, focusing on the parts that differ from the first embodiment.

[0053] In the second embodiment, the first movement calculation unit 15 calculates the movement of the vehicle 1 based on the output of the LiDAR 3 and the output of the IMU 5, and the second movement calculation unit 16 calculates the movement of the vehicle 1 based on the output of the camera 4 and the output of the IMU 5. This makes it possible to calculate the movement of the vehicle 1 with greater accuracy, and consequently improves the accuracy of anomaly diagnosis in the self-position estimation of the vehicle 1. In the second embodiment, the first movement calculation unit 15 is also referred to as the LiDAR inertial odometry calculation unit, and the second movement calculation unit 16 is also referred to as the camera inertial odometry calculation unit.

[0054] Figure 5 is a flowchart showing the control routine for abnormality diagnosis processing in the second embodiment. This control routine is repeatedly executed at predetermined execution intervals by the processor 13 of the ECU 10, for example, according to a computer program stored in the memory 12 of the ECU 10.

[0055] Steps S201 and S202 are performed in the same manner as steps S101 and S102 in Figure 4.

[0056] After step S202, in step S203, the first displacement calculation unit 15 of the processor 13 calculates the amount of lateral movement of the vehicle 1 in a predetermined time interval (for example, the time interval between time t and time t+1) based on the output of the LiDAR 3 and the output of the IMU 5. For example, the first displacement calculation unit 15 calculates the amount of lateral movement of the vehicle 1 using the LiDAR inertial odometry method. That is, the first displacement calculation unit 15 calculates the amount of lateral movement of the vehicle 1 by integrating the output of the LiDAR 3 and the output of the IMU 5.

[0057] For example, the first displacement calculation unit 15 integrates the output of LiDAR3 and the output of IMU5 using a loosely coupled method. In this case, the first displacement calculation unit 15 independently calculates an estimated value based on the output of LiDAR3 and an estimated value based on the output of IMU5, and then integrates these in a subsequent filter (e.g., a Kalman filter) to calculate the final estimated value. Alternatively, the first displacement calculation unit 15 may integrate the output of LiDAR3 and the output of IMU5 using a tightly coupled method. In this case, the first displacement calculation unit 15 incorporates the output of LiDAR3 and the output of IMU5 into the same optimization problem or state estimation framework (e.g., nonlinear optimization, factor graph, extended Kalman filter, particle filter, etc.) to calculate the estimated value.

[0058] Next, in step S204, the second movement calculation unit 16 of the processor 13 calculates the amount of longitudinal movement of the vehicle 1 in a predetermined time interval (for example, the time interval between time t and time t+1) based on the output of the camera 4 and the output of the IMU 5. For example, the second movement calculation unit 16 calculates the amount of longitudinal movement of the vehicle 1 using the Visual Inertia Odometry method. That is, the second movement calculation unit 16 calculates the amount of longitudinal movement of the vehicle 1 by integrating the output of the camera 4 and the output of the IMU 5. For example, the second movement calculation unit 16 integrates the output of the camera 4 and the output of the IMU 5 using an extended Kalman filter or nonlinear optimization.

[0059] After step S204, steps S205 to S207 are executed in the same manner as steps S105 to S107 in Figure 4.

[0060] <Third Embodiment> The configuration and control of the anomaly diagnosis device according to the third embodiment are basically the same as those of the anomaly diagnosis device according to the first embodiment, except for the points described below. Therefore, the third embodiment of the present invention will be described below focusing on the differences from the first embodiment.

[0061] In the third embodiment, the first movement calculation unit 15 calculates the lateral and vertical movement amounts of the vehicle 1 based on the output of the LiDAR 3. When the vertical movement amount calculated by the first movement calculation unit 15 is less than a predetermined value, the anomaly detection unit 17 detects an anomaly in the self-estimation of the vehicle 1 by comparing the change in the self-estimated position calculated by the self-position estimation unit 14 with the estimated movement amount of the vehicle 1 obtained from the lateral movement amount calculated by the first movement calculation unit 15 and the vertical movement amount calculated by the second movement calculation unit 16. On the other hand, when the vertical movement amount calculated by the first movement calculation unit 15 is greater than or equal to a predetermined value, the anomaly detection unit 17 detects an anomaly in the self-estimation of the vehicle 1 by comparing the change in the self-estimated position calculated by the self-position estimation unit 14 with the estimated movement amount of the vehicle 1 obtained from the lateral and vertical movement amounts calculated by the first movement calculation unit 15. This allows for the effective combined use of LiDAR3 and camera4 to accurately diagnose anomalies in the vehicle's self-position estimation.

[0062] Figure 6 is a flowchart showing the control routine for abnormality diagnosis processing in the third embodiment. This control routine is repeatedly executed at predetermined execution intervals by the processor 13 of the ECU 10, for example, according to a computer program stored in the memory 12 of the ECU 10.

[0063] Steps S301 and S302 are performed in the same manner as steps S101 and S102 in Figure 4.

[0064] Following step S302, in step S303, the first displacement calculation unit 15 of the processor 13 calculates the lateral and longitudinal displacement of the vehicle 1 at a predetermined time interval (for example, the time interval between time t and time t+1) based on the output of the LiDAR 3. The method for calculating the longitudinal displacement is the same as the method for calculating the lateral displacement described above with respect to step S103 in Figure 4.

[0065] Next, in step S304, the abnormality detection unit 17 of the processor 13 determines whether the longitudinal movement of the vehicle 1 calculated by the first movement amount calculation unit 15 is greater than or equal to a predetermined value. If it is determined that the longitudinal movement of the vehicle 1 is less than the predetermined value, the control routine proceeds to step S305. Steps S305 to S308 are executed in the same manner as steps S104 to S107 in Figure 4.

[0066] On the other hand, if it is determined in step S304 that the longitudinal movement of vehicle 1 is greater than or equal to a predetermined value, the control routine proceeds to step S306. In step S306, the abnormality detection unit 17 determines whether the difference between the change in the self-estimated position calculated by the self-position estimation unit 14 and the estimated movement of vehicle 1 obtained from the lateral and longitudinal movement amounts calculated by the first movement amount calculation unit 15 is greater than or equal to a predetermined threshold. The determination method is the same as the method described above for step S105 in Figure 4.

[0067] If, in step S306, it is determined that the difference between the change in the self-estimated position and the estimated movement of vehicle 1 is greater than or equal to a threshold, the control routine proceeds to step S307. In step S307, similar to step S106 in Figure 4, the anomaly detection unit 17 determines that the self-position estimation of vehicle 1 is abnormal. After step S307, the control routine terminates.

[0068] On the other hand, if in step S306 it is determined that the difference between the change in the self-estimated position and the estimated movement of vehicle 1 is less than a threshold, the control routine proceeds to step S308. In step S308, the anomaly detection unit 17 determines that the self-position estimation of vehicle 1 is normal. After step S308, the control routine terminates.

[0069] <Other Embodiments> Although preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes can be made within the scope of the claims.

[0070] For example, a server located outside of vehicle 1 and capable of communicating with vehicle 1 may function as an abnormality diagnosis device. In this case, the information necessary to perform the control described above is transmitted from vehicle 1 to the server.

[0071] Furthermore, the second and third embodiments can be implemented in combination. In this case, in step S303 of the control routine for the abnormality diagnosis process in Figure 6, the first movement amount calculation unit 15 calculates the lateral and longitudinal movement amounts of the vehicle 1 based on the output of the LiDAR 3 and the output of the IMU 5. Also, in step S305, the second movement amount calculation unit 16 calculates the longitudinal movement amount of the vehicle 1 based on the output of the camera 4 and the output of the IMU 5.

[0072] Furthermore, a computer program that enables a computer to implement the functions of each part of the processor 13 of the ECU 10 or the server's processor may be provided in the form of a recording medium readable by a computer, or as part of a computer program product. The recording medium readable by a computer may be, for example, a magnetic recording medium, an optical recording medium, or a semiconductor memory. [Explanation of Symbols]

[0073] 1 vehicle 2 GNSS receivers 3 LiDAR 4 cameras 10. Electronic Control Unit (ECU) 13 processors 14 Self-position estimation part 15 First movement amount calculation section 16 Second movement amount calculation section 17 Anomaly detection unit

Claims

1. A self-position estimation unit calculates the amount of change in the vehicle's self-estimated position based on the output of a GNSS receiver installed in the vehicle, A first movement amount calculation unit calculates the amount of movement of the vehicle based on the output of a LiDAR installed in the vehicle, A second movement amount calculation unit calculates the amount of movement of the vehicle based on the output of a camera installed on the vehicle, An anomaly detection unit that detects an anomaly in the vehicle's self-position estimation, Equipped with, The first movement calculation unit calculates the lateral movement of the vehicle, and the second movement calculation unit calculates the longitudinal movement of the vehicle. The abnormality detection unit is an abnormality diagnostic device that detects an abnormality in the self-position estimation of the vehicle by comparing the amount of change in the self-estimated position with the estimated amount of movement of the vehicle obtained from the amount of movement in the lateral direction and the amount of movement in the vertical direction.

2. The first displacement calculation unit calculates the displacement of the vehicle based on the output of the LiDAR and the output of the IMU provided on the vehicle. The abnormality diagnosis device according to claim 1, wherein the second movement amount calculation unit calculates the amount of movement of the vehicle based on the output of the camera and the output of the IMU.

3. The first movement amount calculation unit calculates the lateral movement amount and the vertical movement amount, An anomaly detection device according to claim 1 or 2, wherein the anomaly detection unit detects an anomaly by comparing the amount of change of the self-estimated position with the estimated amount of movement of the vehicle obtained from the amount of movement in the horizontal direction calculated by the first amount of movement calculation unit and the amount of movement in the vertical direction calculated by the second amount of movement calculation unit when the amount of movement in the vertical direction calculated by the first amount of movement calculation unit is less than a predetermined value, and detects an anomaly by comparing the amount of change of the self-estimated position with the estimated amount of movement of the vehicle obtained from the amount of movement in the horizontal direction calculated by the first amount of movement calculation unit and the amount of movement in the vertical direction when the amount of movement in the vertical direction calculated by the first amount of movement calculation unit is greater than or equal to the predetermined value.

4. A computer-based method for diagnosing anomalies, The amount of change in the vehicle's self-estimated position is calculated based on the output of the GNSS receiver installed in the vehicle, The amount of lateral movement of the vehicle is calculated based on the output of the LiDAR installed in the vehicle, The amount of vertical movement of the vehicle is calculated based on the output of a camera installed on the vehicle. An anomaly in the vehicle's self-position estimation is detected by comparing the amount of change in the self-estimated position with the estimated movement of the vehicle obtained from the amount of movement in the lateral direction and the amount of movement in the vertical direction. Methods for diagnosing abnormalities, including those mentioned above.

5. The amount of change in the vehicle's self-estimated position is calculated based on the output of the GNSS receiver installed in the vehicle, The amount of lateral movement of the vehicle is calculated based on the output of the LiDAR installed in the vehicle, The amount of vertical movement of the vehicle is calculated based on the output of a camera installed on the vehicle. An anomaly in the vehicle's self-position estimation is detected by comparing the amount of change in the self-estimated position with the estimated movement of the vehicle obtained from the amount of movement in the lateral direction and the amount of movement in the vertical direction. A computer program that causes a computer to execute something.

Images