Sensor fault diagnosis method, electronic equipment, storage medium and program product

Abstract

The invention provides a sensor fault diagnosis method, electronic equipment, a storage medium and a program product, and relates to the technical field of automatic driving, and the method comprises the steps: obtaining a first pose of a front vehicle and external parameter calibration information of the front vehicle sent by the front vehicle, obtaining a second pose of an own vehicle, physical characteristic parameters of the sensor and sensor data, collected by the sensor, of the front vehicle are obtained; calculating an expected sensing result of the sensor according to the first pose, the external parameter calibration information, the second pose and the physical characteristic parameters; generating an actual sensing result of the sensor according to the sensor data of the front vehicle; and calculating the similarity between the expected sensing result and the actual sensing result, and determining a sensor fault diagnosis result based on the similarity and a preset similarity threshold. According to the invention, fault diagnosis is carried out through independent redundant information provided by combined positioning of the front vehicle and the rear vehicle, additional sensors and computing power requirements do not need to be introduced, the fault diagnosis result is stable and reliable, and the real-time performance is guaranteed.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a sensor fault diagnosis method, electronic equipment, storage medium and program product. Background Art

[0002] In the autonomous driving system, the perception module is one of the core components, and its main function is to perceive and understand the environment around the vehicle in real time. The perception module generates a detailed description of the surrounding environment by processing data from various sensors, providing a basis for path planning and control decisions. The perception module is crucial in the autonomous driving system, and its accuracy and reliability directly affect the safety and performance of the vehicle. If there is a problem with the sensor, it may lead to a series of serious consequences such as environmental perception errors of the perception module, lane and road recognition errors, traffic sign and signal recognition errors, dynamic object prediction errors, positioning and navigation errors, etc. At present, no better solution has been proposed for how to optimize the fault diagnosis process of sensors. Summary of the invention

[0003] The embodiments of the present application provide a sensor fault diagnosis method, an electronic device, a storage medium, and a program product to solve one or more of the above-mentioned technical problems.

[0004] In a first aspect, an embodiment of the present application provides a sensor fault diagnosis method, wherein the sensor is arranged in a following vehicle traveling in a platoon, and the method comprises: obtaining a first posture of a preceding vehicle and external parameter calibration information of the preceding vehicle sent by the preceding vehicle, obtaining a second posture of the own vehicle, and obtaining physical characteristic parameters of the sensor and sensor data of the preceding vehicle collected by the sensor; wherein the first posture is obtained by correcting a third posture; the third posture is calculated by the following vehicle based on the second posture and the perception information of the preceding vehicle; the expected perception result of the sensor is calculated according to the first posture, the external parameter calibration information, the second posture and the physical characteristic parameters; the actual perception result of the sensor is generated according to the sensor data of the preceding vehicle; the similarity between the expected perception result and the actual perception result is calculated, and the sensor fault diagnosis result is determined based on the similarity and a preset similarity threshold.

[0005] In a second aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor implements any of the methods described above when executing the computer program.

[0006] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any of the methods described above is implemented.

[0007] In a fourth aspect, an embodiment of the present application provides a computer program product, wherein the computer program product includes a computer program, and when the computer program is executed by a processor, it implements any of the methods described above.

[0008] Compared with the related art, this application has the following advantages:

[0009] According to the embodiment of the present application, the first posture of the preceding vehicle and the external parameter calibration information of the preceding vehicle sent by the preceding vehicle are obtained, the second posture of the self-vehicle is obtained, and the physical characteristic parameters of the sensor and the sensor data of the preceding vehicle collected by the sensor are obtained; wherein, the first posture is obtained by correcting the third posture; the third posture is calculated by the following vehicle based on the second posture and the perception information of the preceding vehicle; the expected perception result of the sensor is calculated according to the first posture, the external parameter calibration information, the second posture and the physical characteristic parameters; the actual perception result of the sensor is generated according to the sensor data of the preceding vehicle; the similarity between the expected perception result and the actual perception result is calculated, and the sensor fault diagnosis result is determined based on the similarity and the preset similarity threshold. The embodiment of the present application can combine the first posture of the joint positioning result of the preceding vehicle and the following vehicle, the second posture of the positioning result of the following vehicle, the external parameter calibration information and the physical characteristic parameters to calculate the expected perception result of the sensor, calculate the expected perception result of the sensor, and then calculate the similarity between the expected perception result and the actual perception result, and the sensor fault diagnosis result can be determined based on the similarity and the preset similarity threshold. The present application performs fault diagnosis through independent redundant information provided by the joint positioning of the leading vehicle and the trailing vehicle, without the need to introduce additional sensors and computing power requirements. The fault diagnosis results are stable, reliable, and real-time performance is guaranteed.

[0010] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments according to the present application and should not be regarded as limiting the scope of the present application.

[0012] Figure 1 A flow chart of a sensor fault diagnosis method provided in an embodiment of the present application is shown;

[0013] Figure 2A flowchart of sensor, perception module, and calibration parameter fault diagnosis by joint positioning in the formation mode provided in an embodiment of the present application is shown;

[0014] Figure 3 A schematic diagram showing the effect of fault type judgment provided in an embodiment of the present application is shown;

[0015] Figure 4 A schematic diagram of extracting the preceding vehicle information features provided in an embodiment of the present application is shown;

[0016] Figure 5 A schematic diagram of calculating the expected perception result based on the sensor hardware characteristics provided in an embodiment of the present application is shown;

[0017] Figure 6 A schematic diagram of fault diagnosis based on probability statistics provided in an embodiment of the present application is shown;

[0018] Figure 7 A block diagram of an electronic device used to implement an embodiment of the present application is shown. DETAILED DESCRIPTION

[0019] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the concept or scope of the present application. Therefore, the drawings and descriptions are considered to be exemplary in nature and not restrictive.

[0020] To facilitate understanding of the technical solutions of the embodiments of the present application, the following describes the related technologies of the embodiments of the present application. The following related technologies can be combined with the technical solutions of the embodiments of the present application as optional solutions, and they all belong to the protection scope of the embodiments of the present application.

[0021] First, the terms involved are explained.

[0022] Pose: Contains spatial position information, expressed as (x, y, z), and attitude information, expressed as (roll angle, pitch angle and yaw angle).

[0023] Traffic participants: motor vehicles, non-motor vehicles, pedestrians and other objects on the road.

[0024] Planning intention: The autonomous vehicle’s intention to drive to the target location next.

[0025] Joint positioning: In the vehicle platooning autonomous driving mode, the positioning of the front and rear vehicles and their sensor information are comprehensively utilized to jointly solve the position and posture of the front and rear vehicles.

[0026] Vehicle platoon autonomous driving: two or more vehicles travel in a platoon, with the front and rear vehicles maintaining a short distance (for example, within 10 meters). The front vehicle in the platoon can be manually driven, while the rear vehicle in the platoon is autonomously driven.

[0027] V2V: vehicle to vehicle, information transmission between vehicles.

[0028] In order to diagnose sensor failures, the existing solution 1 needs to ensure that the system has sufficient redundant information, which will inevitably bring additional production costs and computing power requirements, and has high requirements for module delay, sensor time alignment and spatial calibration, which reduces the reliability of online fault diagnosis. In addition, since the accuracy of redundant sensors is generally not higher than that of currently used sensors, this seriously limits the accuracy of online fault diagnosis. On the other hand, this solution generally does not work in extreme scenarios such as dirt, rain, snow, and dust, or it will generate a lot of misdiagnosis information, which ultimately makes it difficult to actually use it on a large scale.

[0029] The second existing solution is to provide more information through multi-vehicle communication to share sensor information or vehicle-road collaboration roadside sensor perception results. Since there must be multiple autonomous vehicles in the area or dense sensors installed on the roadside to ensure overall safety, it will be difficult to popularize it to all road scenarios in terms of cost and maintenance. It is currently only used in demonstration experimental roads for vehicle-road collaboration. In addition, due to the limitations of vehicle-side and cloud computing capabilities, it is difficult for this solution to provide separate computing and services for each autonomous vehicle on the road, and if it is necessary to provide customized information on the area of ​​interest for each autonomous vehicle, the bandwidth pressure for transmitting data will be even greater. Finally, since they are independent systems, the positioning between autonomous vehicles and the positioning consistency between autonomous vehicles and roadside sensors is not guaranteed, and the transmitted perception information is mixed with positioning errors, making it difficult to use in the end.

[0030] In the scenario of autonomous driving, the sensor may be covered with mud, affected by extreme weather such as rain, snow and dust, and there may also be problems with the calibration of sensor external parameters. In the scenario of an autonomous driving fleet, the relative position of the front vehicle to the rear vehicle can be accurately obtained through the positioning of the front and rear vehicles in the queue and the communication between the vehicles in the vehicle workshop. Therefore, the expected perception result of the sensor under normal conditions can be determined by confirming the theoretical position and posture of the front vehicle in the queue and combining the hardware characteristics of the sensor. Once this perception result does not match the expected perception result, the problem of the sensor and algorithm node can be confirmed through statistical calculation and an alarm can be issued to improve the overall stability of the system.

[0031] Based on this, the present application provides a sensor fault diagnosis method, electronic device, storage medium and program product. The method is based on a joint positioning framework. During driving, the positioning information, planning intention, perception result information and perception capability information of the front and rear vehicles are transmitted to calculate the accurate and reliable visual information of the front vehicle relative to the rear vehicle sensor in real time. Since the information transmitted by the front vehicle and the sensor of the rear vehicle are independent and redundant, this type of additional information can be regarded as a stable and accurate marker within the field of view of the rear vehicle sensor, and online real-time fault diagnosis can be achieved for the rear vehicle sensor, perception module and calibration parameters based on this. The method obtains real-time and accurate reference features based on the positioning and geometric information of the front vehicle without relying on additional sensors, and the positioning consistency based on joint positioning can improve the reliability and accuracy of fault diagnosis, thereby ensuring the overall stability of the system.

[0032] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The several specific embodiments listed can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0033] The present application provides a sensor fault diagnosis method, which can be applied to the vehicle formation automatic driving scenario (the rear vehicle automatically follows the front vehicle). In the automatic driving platoon mode, the lead vehicle is controlled by the driver, while the following vehicle follows the front vehicle through automatic driving technology. Figure 1 The flowchart of a sensor fault diagnosis method according to an embodiment of the present application is shown. The method may include:

[0034] Step S101, obtaining the first posture of the preceding vehicle and the external parameter calibration information of the preceding vehicle sent by the preceding vehicle, obtaining the second posture of the self-vehicle, and obtaining the physical characteristic parameters of the sensor and the sensor data of the preceding vehicle collected by the sensor. The first posture is obtained by correcting the third posture; the third posture is calculated by the following vehicle based on the second posture and the perception information of the preceding vehicle.

[0035] In the embodiment of the present application, the sensor for fault diagnosis is set in the following vehicle of the platoon driving automatically. Therefore, the self-vehicle refers to the following vehicle on which the sensor for fault diagnosis is set, and the front vehicle is the vehicle in the platoon that is adjacent to the self-vehicle and travels in front of the self-vehicle. The front vehicle can be a pilot vehicle or a following vehicle. The vehicles in the platoon can be trucks or other types of motor vehicles.

[0036] Obtain the first posture of the front vehicle and the external parameter calibration information of the front vehicle sent by the front vehicle. It should be noted that the front vehicle and the rear vehicle can communicate in a V2V manner. After the rear vehicle obtains the second posture (which can be obtained through the positioning system), the third posture is calculated based on the second posture and the perception information of the front vehicle by the vehicle itself, that is, the third posture is calculated by the following vehicle based on the second posture and the perception information of the front vehicle. In this step, the perception information is used to determine the relative position relationship between the front vehicle and the rear vehicle, which can be calculated by the perception module of the rear vehicle. Afterwards, the rear vehicle sends the third posture to the front vehicle, and the front vehicle calculates based on the received third posture and the positioning information obtained by the front vehicle (which can be obtained through the positioning system) to obtain the first posture, that is, the first posture is corrected by the third posture.

[0037] The external parameter calibration information includes the vehicle's geometric information. The vehicle's geometric information describes the vehicle's size, shape, and the geometric relationship between the installation positions of each sensor. The vehicle's geometric information includes, but is not limited to: vehicle length, vehicle width, and vehicle height, vehicle outline, sensor position, sensor posture, vehicle size, wheel position, wheelbase, etc.

[0038] The physical characteristic parameters of the sensor may include, but are not limited to, the following information: sensor type, sensing technology, resolution and accuracy, field of view, measurement range, physical size and weight.

[0039] In this step, both the front and rear vehicles in the queue can obtain their own positions through the positioning system. Combining the perception information and positioning information, they can correct each other's positioning to ensure the relative consistency of the positioning of the front and rear vehicles.

[0040] Step S102, calculating an expected perception result of the sensor according to the first posture, the external parameter calibration information, the second posture and the physical characteristic parameters.

[0041] In the embodiment of the present application, based on the external parameter calibration information, the geometric calibration information of the front vehicle is known, and after coordinate transformation and feature extraction, the rear vehicle can obtain the precise geometric and feature information of the front vehicle. Therefore, the expected perception result of the sensor for the front vehicle under normal circumstances can be determined by combining the hardware characteristics of the sensor, that is, the physical characteristic parameters, the first pose and the second pose.

[0042] In the implementation of this application, the perception result refers to the data obtained by the sensor from calculating the original feature data extracted by the preceding vehicle, such as point cloud, feature vector, etc. The expected perception result is used to characterize the perception result obtained based on the sensor under normal conditions.

[0043] In this step, the expected perception effect is calculated based on the joint positioning information and the hardware characteristics of the sensor, so as to achieve fault diagnosis by comparing the actual perception result with the expected perception result.

[0044] Step S103: generating an actual perception result of the sensor according to the sensor data of the preceding vehicle.

[0045] In the embodiment of the present application, the type of sensor can be determined according to actual needs, for example, it can be a camera, a laser radar (LiDAR) or a millimeter wave radar, etc. The sensor data of the preceding vehicle collected by the sensor is subjected to steps including but not limited to: preprocessing, feature extraction, data fusion, target tracking and high-level understanding to obtain an actual perception result. The actual perception result is used to characterize the perception result obtained based on the data collected by the sensor in real time.

[0046] Step S104: calculating the similarity between the expected perception result and the actual perception result, and determining the sensor fault diagnosis result based on the similarity and a preset similarity threshold.

[0047] In an embodiment of the present application, the expected perception result and the actual perception result are independent and redundant information. Therefore, the expected perception result can be regarded as a stable and accurate marker within the field of view of the rear vehicle sensor. Based on the expected perception result, online real-time fault diagnosis of the rear vehicle sensor can be performed. When the sensor is normal, the similarity between the expected perception result and the actual perception result is high or consistent. The sensor fault diagnosis result is determined based on the similarity and a preset similarity threshold. For example, if the similarity is not lower than the preset similarity threshold, the sensor is determined to be fault-free, otherwise the sensor is determined to be faulty. Among them, the similarity threshold can be set according to actual needs or experience, and the embodiment of the present application does not make specific limitations on this.

[0048] In this step, fault diagnosis is performed through the independent redundant information provided by the joint positioning of the pilot vehicle and the following vehicle. There is no need to introduce additional sensors and computing power requirements. The fault diagnosis results are stable, reliable, and real-time performance is guaranteed.

[0049] According to the embodiment of the present application, the first posture of the preceding vehicle and the external parameter calibration information of the preceding vehicle sent by the preceding vehicle are obtained, the second posture of the self-vehicle is obtained, and the physical characteristic parameters of the sensor and the sensor data of the preceding vehicle collected by the sensor are obtained; wherein, the first posture is obtained by correcting the third posture; the third posture is calculated by the following vehicle based on the second posture and the perception information of the preceding vehicle; the expected perception result of the sensor is calculated according to the first posture, the external parameter calibration information, the second posture and the physical characteristic parameters; the actual perception result of the sensor is generated according to the sensor data of the preceding vehicle; the similarity between the expected perception result and the actual perception result is calculated, and the sensor fault diagnosis result is determined based on the similarity and the preset similarity threshold. The embodiment of the present application can combine the first posture of the joint positioning result of the preceding vehicle and the following vehicle, the second posture of the positioning result of the following vehicle, the external parameter calibration information and the physical characteristic parameters to calculate the expected perception result of the sensor, calculate the expected perception result of the sensor, and then calculate the similarity between the expected perception result and the actual perception result, and the sensor fault diagnosis result can be determined based on the similarity and the preset similarity threshold. The present application performs fault diagnosis through independent redundant information provided by the joint positioning of the leading vehicle and the trailing vehicle, without the need to introduce additional sensors and computing power requirements. The fault diagnosis results are stable, reliable, and real-time performance is guaranteed.

[0050] In a possible implementation, obtaining sensor data of a leading vehicle collected by the sensor includes: determining the relative position between the following vehicle and the leading vehicle based on the first posture and the second posture; determining theoretical corner points of the leading vehicle observable by the sensor based on the relative position and the external parameter calibration information; determining that the sensor is in a visible state if the current information collected by the sensor includes a target corner point in the theoretical corner point; and obtaining sensor data of the leading vehicle collected by the sensor in the visible state.

[0051] In this possible implementation, the first pose can be converted into data in the rear vehicle coordinate system, and the relative position can be used to describe the spatial position relationship between the front vehicle and the rear vehicle in the rear vehicle coordinate system, that is, the relative position between the following vehicle and the front vehicle is determined based on the first pose and the second pose. After the relative position is determined, the theoretical corner points of the front vehicle observable by the sensor are determined in combination with the external parameter calibration information. It should be noted that the corner points are feature points in the observation information of the sensor, and the corner points can be in areas where the brightness changes dramatically in the image, such as the edges, corners or texture-rich areas of objects.

[0052] The current information collected by the sensor is the observation information currently collected by the sensor. If the current information collected by the sensor includes the target corner point in the theoretical corner point, it means that the sensor can observe the preceding vehicle, and it is determined that the sensor is in a visible state. After determining that the sensor is in a visible state, the sensor data of the preceding vehicle collected by the sensor is obtained.

[0053] In this possible implementation, the target corner points are multiple theoretical corner points pre-determined in the theoretical corner points according to actual needs. Since the observation information currently collected by the sensor may include information of the preceding vehicle and other vehicles, by determining that the current information collected by the sensor includes the target corner points in the theoretical corner points, it can be determined that the sensor can observe the preceding vehicle information.

[0054] In one possible implementation, the actual perception result of the sensor is generated based on the sensor data of the leading vehicle, including: determining the area of ​​interest of the sensor for the leading vehicle based on the physical characteristic parameters, the first posture, the second posture and the external parameter calibration information; extracting feature information of the sensor data of the leading vehicle falling in the area of ​​interest; and generating the actual perception result of the sensor based on the feature information.

[0055] In this possible implementation, the size of the region of interest can be determined according to the physical characteristic parameters, and the position of the region of interest can be determined according to the first pose, the second pose and the external parameter calibration information. After the region of interest is determined, the feature information of the sensor data of the preceding vehicle falling in the region of interest is extracted. For different types of sensor data, the type of feature information is also different. For example, when the sensor data is an image, the feature information can be a feature vector. When the sensor data is data collected by a laser radar, the feature information can be point cloud data. The feature information is understood to obtain the actual perception result.

[0056] In a possible implementation, the physical characteristic parameters include a resolution and angle of a horizontal viewing angle, and a resolution and angle of a vertical viewing angle of the sensor; the expected perception result of the sensor is calculated according to the first posture, the second posture, the external parameter calibration information and the physical characteristic parameters, including: determining an area of ​​interest of the sensor for the horizontal viewing angle of the preceding vehicle according to the angle of the horizontal viewing angle, the first posture, the second posture and the external parameter calibration information, and determining an area of ​​interest of the sensor for the vertical viewing angle of the preceding vehicle according to the angle of the vertical viewing angle, the first posture, the second posture and the external parameter calibration information; in the A plurality of horizontal perspective target points are determined in a horizontal perspective region of interest, and a horizontal perspective target distance between the horizontal perspective target point and the sensor is calculated; a plurality of vertical perspective target points are determined in a vertical perspective region of interest, and a vertical perspective target distance between the vertical perspective target point and the sensor is calculated; the number of horizontal perspective point clouds of the sensor is calculated based on the horizontal perspective target distance and the resolution of the horizontal perspective; the number of vertical perspective point clouds of the sensor is calculated based on the vertical perspective target distance and the resolution of the vertical perspective; and the product of the number of horizontal perspective point clouds and the number of vertical perspective point clouds is used as the expected perception result of the sensor.

[0057] In this possible implementation, Figure 5 This paper demonstrates a possible way to calculate the expected perception results based on the sensor hardware characteristics. Figure 5 This step is described in detail. Figure 5 , Figure 5 The left side of the middle shows the calculation process of the number of point clouds in the case of a top-down perspective, that is, the horizontal perspective. Figure 5 The right side of the figure shows the calculation process of the number of vertical view point clouds in the front view case. According to the horizontal view angle, the first posture, the second posture and the external parameter calibration information, the horizontal view area of ​​interest of the sensor to the front vehicle is determined. Figure 5 The small dotted box on the left shows the horizontal viewing angle region of interest, where the length of feature extraction (Length) is determined based on the horizontal viewing angle, and the width of the front vehicle (Width) is determined based on the external parameter calibration information. Based on the vertical viewing angle, the first posture, the second posture and the external parameter calibration information, the vertical viewing angle region of interest of the sensor for the front vehicle is determined. Figure 5 The small dotted box on the right side shows the region of interest from a vertical perspective, where the height of the front vehicle is determined based on the external parameter calibration information.

[0058] Determine a plurality of horizontal viewing angle target points in the horizontal viewing angle region of interest, Figure 5The three circles on the left side of the figure represent three horizontal viewing angle target points. The horizontal viewing angle target distance between the horizontal viewing angle target point and the sensor is calculated. Figure 5 The left distance (LeftDistance), the nearest distance (NearestDistance) and the right distance (RightDistance) on the left side of the middle represent the horizontal viewing angle target distance.

[0059] Determine a plurality of vertical viewing angle target points in the vertical viewing angle region of interest, Figure 5 The two circles on the right side of the figure represent two vertical viewing angle target points. The vertical viewing angle target distance between the vertical viewing angle target point and the sensor is calculated. Figure 5 The high distance (HigherDistance) and low distance (LowerDistance) on the right side of the middle represent the vertical viewing target distance.

[0060] The number of horizontal viewing angle point clouds of the sensor is calculated based on the horizontal viewing angle target distance and the resolution of the horizontal viewing angle, which can be calculated according to Figure 5 The number of vertical viewing angle point clouds of the sensor is calculated based on the vertical viewing angle target distance and the vertical viewing angle resolution, which can be calculated according to Figure 5 The calculation is performed using the formula shown on the right.

[0061] In a possible implementation, the sensor is a lidar or a millimeter-wave radar, and calculating the similarity between the expected perception result and the actual perception result includes: calculating the ratio of the number of point clouds in the actual perception result to the number of point clouds in the expected perception result; and using the ratio as the similarity between the expected perception result and the actual perception result.

[0062] In this possible implementation, when the sensor is a laser radar or a millimeter wave radar, the information collected by the sensor is in the form of point cloud data. Therefore, the similarity between the expected perception result and the actual perception result can be calculated by calculating the ratio of the number of point clouds in the actual perception result to the number of point clouds in the expected perception result.

[0063] In one possible implementation, a sensor fault diagnosis result is determined based on the similarity and a preset similarity threshold, including: if the ratio is not less than the similarity threshold, it is determined that the sensor is not faulty; if the ratio is less than the similarity threshold, it is determined that the sensor is faulty, and the sensor fault type is determined to be point cloud missing.

[0064] In this possible implementation, when the sensor is a laser radar or a millimeter wave radar, if the ratio of the number of point clouds in the actual perception result to the number of point clouds in the expected perception result is not less than the similarity threshold, it is determined that the sensor is fault-free, otherwise, it is determined that the sensor is faulty, and the sensor fault type is determined to be point cloud missing, wherein point cloud missing means that the current number of point clouds has not reached the expected number. The reason for the point cloud missing may be data generation or transmission delay. When the sensor fault type is point cloud missing, you can wait for a while. If the problem of point cloud missing can be overcome, there is no need to deal with the sensor fault.

[0065] In a possible implementation, the sensor is a camera, and calculating the similarity between the expected perception result and the actual perception result includes: generating a first feature vector of the image in the actual perception result, and a second feature vector of the image in the expected perception result; calculating the similarity between the first feature vector and the second feature vector to obtain the similarity between the expected perception result and the actual perception result.

[0066] In this possible implementation, when the sensor is a camera, the information collected by the sensor is in the form of an image. Therefore, feature vectors may be extracted from the image, and then based on the extracted feature vectors, the similarity between the expected perception result and the actual perception result may be calculated.

[0067] In a possible implementation, after determining the sensor fault diagnosis result based on the similarity and a preset similarity threshold, it also includes: determining the area of ​​interest of the sensor for the leading vehicle based on the physical characteristic parameters, the first posture, the second posture and the external parameter calibration information; obtaining the number of first point clouds within the area of ​​interest, and the number of second point clouds outside the area of ​​interest; calculating the ratio of the number of first point clouds to the number of second point clouds, and if the ratio satisfies the set ratio range, determining that the sensor fault type is sensor occlusion.

[0068] In this possible implementation, the size of the region of interest can be determined based on the physical characteristic parameters, and the position of the region of interest can be determined based on the first pose, the second pose and the external parameter calibration information. When the sensor is blocked, the number of first point clouds falling within the region of interest is less than the number of first point clouds falling within the region of interest when there is no blockage. In order to determine whether the sensor is blocked, the number of second point clouds falling outside the region of interest can also be obtained, and the ratio of the first number of point clouds to the second number of point clouds is calculated. If the ratio satisfies the set ratio range, the sensor fault type is determined to be sensor occlusion. Among them, the set ratio range can be set according to experience, and the implementation of this application does not make specific limitations on this.

[0069] In a possible implementation, it also includes: within a set time period, if the sensor fault type is determined to be sensor occlusion for multiple consecutive times, then the occlusion type is determined to be dirty occlusion, and sensor cleaning reminder information is generated based on the dirty occlusion; otherwise, the occlusion type is determined to be dust occlusion.

[0070] In this possible implementation, dirty occlusion is a stubborn occlusion on the sensor that will not be automatically cleared, while dust occlusion is a temporary occlusion on the sensor that may be automatically cleared due to the external environment, such as wind. Within the set time, if the sensor fault type is determined to be sensor occlusion for multiple consecutive times, the occlusion type is determined to be dirty occlusion, and sensor cleaning reminder information is generated based on the dirty occlusion. Otherwise, the occlusion type is determined to be dust occlusion, thereby distinguishing the occlusion types so that the user can take relevant occlusion processing measures. Based on the above steps, the coverage scenarios of fault diagnosis are expanded.

[0071] In a possible implementation, it also includes: generating an actual comprehensive perception result of the perception module based on multiple actual perception results, and generating an expected comprehensive perception result of the perception module based on multiple expected perception results; calculating the Mahalanobis distance between the actual comprehensive perception result and the expected comprehensive perception result, and generating a fault diagnosis result of the perception module based on the Mahalanobis distance and a preset Mahalanobis distance threshold.

[0072] In this possible implementation, the perception module may include multiple sensors. The actual perception results of the multiple sensors are fused to obtain the actual comprehensive perception result of the perception module. Similarly, the expected perception results of the multiple sensors are fused to obtain the expected comprehensive perception result of the perception module. It should be noted that the perception results of multiple sensors of the same type can be fused, for example, the perception results of multiple laser radars can be fused, and the perception results of multiple different types of sensors can also be fused, for example, the perception results of multiple laser radars and cameras can be fused.

[0073] Mahalanobis distance is a measurement method for measuring the distance between multidimensional data points. In the case of perception module failure, this application uses Mahalanobis distance to measure the uncertainty score of the perception module based on joint positioning information, generates fault diagnosis results, and thus realizes fault diagnosis. This step can reduce the demand for computing power and can stably ensure real-time performance.

[0074] In a possible implementation, it also includes: calculating a test statistic result based on the expected perception result, the actual perception result and the SPRT algorithm; when the test statistic result is greater than a preset first test threshold, determining that the calibration parameters of the sensor are incorrect; when the test statistic result is less than a preset second test threshold, determining that the calibration parameters of the sensor are correct.

[0075] In this possible implementation, the expected perception result and the actual perception result are used as input information of the SPRT algorithm, and the algorithm outputs a test statistic result. The test statistic result is compared with the first test threshold and the second test threshold. When the test statistic result is greater than the preset first test threshold, it is determined that the calibration parameters of the sensor are incorrect; when the test statistic result is less than the preset second test threshold, it is determined that the calibration parameters of the sensor are correct.

[0076] The preset first inspection threshold and the preset second inspection threshold can be determined according to actual needs and experience, and the embodiments of the present application do not specifically limit this.

[0077] In the case of calibration parameter failure, in this step, calibration fault diagnosis is achieved based on the joint positioning information through the SPRT (Sequential Probability Ratio Test) algorithm, which can improve the calculation efficiency and the real-time performance of the test.

[0078] Considering that traditional fault diagnosis systems have many false alarms and large computing power requirements, and in extreme scenarios, redundant perception information often has insufficient accuracy or fails together with the main sensor, resulting in the failure of the fault diagnosis system to function properly. Therefore, in a possible implementation, it also includes: receiving the positioning state sent by the leading vehicle; wherein the positioning state is used to characterize whether the first posture is a credible state; obtaining the formation state of the following vehicle; wherein the formation state is used to characterize whether the following vehicle is in a formation driving state; checking the data obtained by the sensor to obtain a test result; wherein the test result is used to characterize whether the sensor can obtain the specified features of the leading vehicle; when the positioning state is a credible state, the formation state is in a formation driving state, and the test result is that the sensor can obtain the specified features of the leading vehicle, the fault diagnosis result is determined to be valid, otherwise the fault diagnosis result is determined to be invalid.

[0079] In this possible implementation, the positioning state is used to characterize whether the first posture of the front vehicle is a credible state. For example, when the reliability and accuracy of the first posture are high, the credible state is credible. When the reliability or accuracy of the first posture is poor, the credible state is determined to be untrustworthy. The positioning state can be sent by the front vehicle, and the positioning state is obtained by communicating with the front vehicle. The formation state can be obtained from the vehicle. The data obtained by the sensor may include the front vehicle of the vehicle in the formation, and may also include other traffic participants, such as vehicles that are driving in front of the vehicle and are not in the formation. By checking the data obtained by the sensor, it is identified whether the sensor of the vehicle can observe the characteristics of the front vehicle. The characteristics can be part of all the characteristics of the front vehicle, that is, the specified characteristics. The specified characteristics can be set in advance. When the positioning state is a credible state, the formation state is in a formation driving state, and the inspection result is that the sensor can obtain the specified characteristics of the front vehicle, it is determined that the fault diagnosis result is valid, otherwise it is determined that the fault diagnosis result is invalid, thereby improving the reliability of the fault diagnosis result.

[0080] In this step, a highly effective and reliable fault diagnosis system is implemented through joint positioning, which greatly reduces the occurrence of false alarms and missed alarms. It has high accuracy and recall rates, and can ensure the effectiveness of fault diagnosis even in extreme scenarios due to reliable independent and redundant information.

[0081] See also Figure 2 The flowchart of sensor, perception module and calibration parameter fault diagnosis through joint positioning in the formation mode shown in the figure is combined with Figure 2 The implementation process of this method is described.

[0082] like Figure 2 As shown in the figure, in the formation autonomous driving queue, both the front and rear vehicles can use the positioning system to obtain the position of the vehicle, and can correct each other's positioning by combining the perception information and positioning information to ensure the relative consistency of the positioning of the front and rear vehicles. Since the geometric calibration information of the front vehicle is also known, after coordinate transformation and feature extraction, the rear vehicle can obtain the precise geometric and feature information of the front vehicle. Therefore, the hardware characteristics of the sensor can be combined to determine the expected perception results of the sensor for the front vehicle under normal circumstances. The hardware characteristics, Mahalanobis distance measurement and probability statistics methods can be used to confirm the problems of sensors, algorithm nodes and calibration, and issue alarms to improve the overall stability of the system. The overall technical route is as follows: Figure 2 As shown, the specific steps are as follows:

[0083] 1. Initialization: After the front and rear vehicles are powered on, they integrate the global positioning system (such as satellite navigation system, or positioning system based on high-precision map matching) and perception information to obtain a corrected and unified positioning result Tp. f and Tp1; where T pf =(x pf,y pf , z pf ,roll pf , pith pf , yaw pf ), represents the position of the global positioning system of the rear vehicle in the queue (roll: pitch angle, pitch: roll angle, yaw: heading angle), T pl =(x pl ,y pl , z pl , roll pl , pith pl , yaw pl ), represents the position of the global positioning system of the front vehicle in the queue.

[0084] 2. Information transmission of the leading vehicle: The leading vehicle sends its position Tp1, calibration information, planning intention (such as left or right lane change, left or right turn, emergency stop, etc.) and positioning status to the trailing vehicle through V2X and other communication methods. Among them, the calibration information may include the geometric calibration information of the leading vehicle in the queue, including the length and width of the vehicle head, the length and width of the vehicle trailer, the distance from the vehicle trailer articulation point to the center of the vehicle head, and the vehicle height.

[0085] 3. Update the visibility of the leading vehicle: The following vehicle updates the position of the leading vehicle based on its position Tp1 and the positioning result Tp of the self-vehicle. f and multi-sensor observation information on the characteristics of the preceding vehicle[O f ] x , to determine the sensor's visibility of the front vehicle's features; where [O f ] x =[(x f ,y f , z f , roll f , pith f , yaw f )] x , represents the global position of the front vehicle feature observed by the xth sensor of the rear vehicle in the queue, such as the corner point information of the bounding box. It should be noted that [o f ] x It is the expected perception result obtained after processing the sensor data of the preceding vehicle directly collected by the sensor.

[0086] 4. Feature extraction and coordinate transformation: Select the sensor where the features of the front vehicle are visible and extract the feature information in the area of ​​interest, and then convert it to the coordinate system of the rear vehicle for unified processing;

[0087] 5. Fault diagnosis based on sensor hardware characteristics: Determine the expected perception results of the sensor under normal conditions based on the physical characteristics of the sensor. Take a certain laser radar as an example. The horizontal field of view angle of the radar is 120°, the resolution is 0.1°, the vertical field of view angle is 25.4°, and the resolution is 0.2°. The point cloud obtained after extracting features based on the information of the preceding vehicle is as follows: Figure 4 As shown in the figure, the expected perception result can be calculated based on the visual angle and angular resolution of the sensor according to the information of the preceding vehicle. The calculation method is as follows: Figure 5 As shown; reliable fault diagnosis results are output based on the comparison between the actual perception results and the expected perception results.

[0088] 6. Fault diagnosis of perception module based on Mahalanobis distance measurement: This application assumes that the multi-modal observation noise obeys Gaussian distribution. Then, the joint positioning information of the preceding vehicle and the observation information of each modal sensor of the self-vehicle on the preceding vehicle can be used to diagnose the fault of the perception module based on the Mahalanobis distance measurement. f ] x and its variance to calculate the Mahalanobis distance score of the entire perception module; and output the fault diagnosis result of the perception module by comparing the Mahalanobis distance score with the threshold score of the acceptable uncertainty degree of the perception module. It should be noted that in this step, based on the [O f ] x , the global pose of the front vehicle features tracked by the rear vehicle in the queue can be obtained.

[0089] 7. Calibration fault diagnosis based on probability statistics: This application assumes that the observation noise follows a Gaussian distribution, and the calibration error is a fixed deviation affected by the heading angle (yaw angle) and translation; the calibration fault diagnosis process based on probability statistics is as follows Figure 6 As shown, the stable sensor observation information of the front vehicle is obtained [O f ] x Afterwards, the Sequential Probability Ratio Test (SPRT) is used to output the calibration fault diagnosis results.

[0090] 8. Reasonableness verification: In order to reduce the frequency of false alarms, it is necessary to verify whether the fault diagnosis scenario meets the following assumptions: the joint positioning state of the front and rear vehicles is normal; the front and rear vehicles maintain the automatic driving mode of the formation; the rear vehicle sensor can extract the characteristic information of the front vehicle. If the scenario assumptions are not met, the results of this fault diagnosis are abandoned to reduce the interference of abnormal factors; if the scenario assumptions are met, the fault diagnosis results are output for downstream use.

[0091] The effects achieved by this application are as follows Figure 3 As shown, Figure 3The actual perception results and expected perception results in the cases of missing point clouds, dust occlusion, and dirty sensors are shown respectively. This application significantly improves the reliability and accuracy of identifying sensor and perception module failures and calibration parameter failures through joint positioning, and the system stability is enhanced. This means that the system in the formation autonomous driving mode is more intelligent and safer, and can actively identify situations such as sensor failures that exceed the autonomous driving operational design domain (ODD) and issue alarms to remind the navigator to make corresponding decisions.

[0092] The present application provides a sensor fault diagnosis method, electronic device, storage medium and program product. The method does not require the introduction of additional sensors, and the expected perception results based on hardware characteristics, Mahalanobis distance measurement and fault diagnosis methods based on probability statistics have low demand for computing power and can stably guarantee real-time performance. In traditional sensor and algorithm node fault diagnosis, it is often necessary to introduce redundant sensor information or consume high communication bandwidth and computing power support, making it difficult to generate actual benefits. The present application provides stable and reliable prior information for sensor fault diagnosis through joint positioning of front and rear vehicles, greatly improving the accuracy and effectiveness of system fault diagnosis.

[0093] In addition, this application can also improve the coverage of fault diagnosis scenarios. In extreme scenarios such as dirty sensors, rain, snow, and dust, traditional fault diagnosis algorithms will not work or will generate a lot of misdiagnosis information because redundant sensors are affected by extreme conditions. The independent redundant information brought by joint positioning in this application shows extremely high reliability in such scenarios, and can still maintain stable and accurate fault diagnosis output.

[0094] In summary, the present application has broad application prospects and can be used in fields such as platoon autonomous driving systems and queue vehicle sensor management.

[0095] Corresponding to the application scenario and method of the method provided in the embodiment of the present application, the embodiment of the present application also provides a sensor fault diagnosis device. The device may include:

[0096] An acquisition module is used to acquire the first posture of the preceding vehicle and the external parameter calibration information of the preceding vehicle sent by the preceding vehicle, acquire the second posture of the own vehicle, and acquire the physical characteristic parameters of the sensor and the sensor data of the preceding vehicle collected by the sensor; wherein, the first posture is obtained by correcting the third posture; the third posture is calculated by the following vehicle based on the second posture and the perception information of the preceding vehicle; an expectation module is used to calculate the expected perception result of the sensor according to the first posture, the external parameter calibration information, the second posture and the physical characteristic parameters; an actual module is used to generate the actual perception result of the sensor according to the sensor data of the preceding vehicle; a judgment module is used to calculate the similarity between the expected perception result and the actual perception result, and determine the sensor fault diagnosis result based on the similarity and a preset similarity threshold.

[0097] In a possible implementation, obtaining sensor data of a leading vehicle collected by the sensor includes: determining the relative position between the following vehicle and the leading vehicle based on the first posture and the second posture; determining theoretical corner points of the leading vehicle observable by the sensor based on the relative position and the external parameter calibration information; determining that the sensor is in a visible state if the current information collected by the sensor includes a target corner point in the theoretical corner point; and obtaining sensor data of the leading vehicle collected by the sensor in the visible state.

[0098] In one possible implementation, the actual perception result of the sensor is generated based on the sensor data of the leading vehicle, including: determining the area of ​​interest of the sensor for the leading vehicle based on the physical characteristic parameters, the first posture, the second posture and the external parameter calibration information; extracting feature information of the sensor data of the leading vehicle falling in the area of ​​interest; and generating the actual perception result of the sensor based on the feature information.

[0099] In a possible implementation, the physical characteristic parameters include a resolution and angle of a horizontal viewing angle, and a resolution and angle of a vertical viewing angle of the sensor; the expected perception result of the sensor is calculated according to the first posture, the second posture, the external parameter calibration information and the physical characteristic parameters, including: determining an area of ​​interest of the sensor for the horizontal viewing angle of the preceding vehicle according to the angle of the horizontal viewing angle, the first posture, the second posture and the external parameter calibration information, and determining an area of ​​interest of the sensor for the vertical viewing angle of the preceding vehicle according to the angle of the vertical viewing angle, the first posture, the second posture and the external parameter calibration information; in the A plurality of horizontal perspective target points are determined in a horizontal perspective region of interest, and a horizontal perspective target distance between the horizontal perspective target point and the sensor is calculated; a plurality of vertical perspective target points are determined in a vertical perspective region of interest, and a vertical perspective target distance between the vertical perspective target point and the sensor is calculated; the number of horizontal perspective point clouds of the sensor is calculated based on the horizontal perspective target distance and the resolution of the horizontal perspective; the number of vertical perspective point clouds of the sensor is calculated based on the vertical perspective target distance and the resolution of the vertical perspective; and the product of the number of horizontal perspective point clouds and the number of vertical perspective point clouds is used as the expected perception result of the sensor.

[0100] In a possible implementation, the sensor is a lidar or a millimeter-wave radar, and calculating the similarity between the expected perception result and the actual perception result includes: calculating the ratio of the number of point clouds in the actual perception result to the number of point clouds in the expected perception result; and using the ratio as the similarity between the expected perception result and the actual perception result.

[0101] In a possible implementation, the sensor is a camera, and calculating the similarity between the expected perception result and the actual perception result includes: generating a first feature vector of the image in the actual perception result, and a second feature vector of the image in the expected perception result; calculating the similarity between the first feature vector and the second feature vector to obtain the similarity between the expected perception result and the actual perception result.

[0102] In one possible implementation, a sensor fault diagnosis result is determined based on the similarity and a preset similarity threshold, including: if the ratio is not less than the similarity threshold, it is determined that the sensor is not faulty; if the ratio is less than the similarity threshold, it is determined that the sensor is faulty, and the sensor fault type is determined to be point cloud missing.

[0103] In a possible implementation, after determining the sensor fault diagnosis result based on the similarity and a preset similarity threshold, it also includes: determining the area of ​​interest of the sensor for the leading vehicle based on the physical characteristic parameters, the first posture, the second posture and the external parameter calibration information; obtaining the number of first point clouds within the area of ​​interest, and the number of second point clouds outside the area of ​​interest; calculating the ratio of the number of first point clouds to the number of second point clouds, and if the ratio satisfies the set ratio range, determining that the sensor fault type is sensor occlusion.

[0104] In a possible implementation, it also includes: within a set time period, if the sensor fault type is determined to be sensor occlusion for multiple consecutive times, then the occlusion type is determined to be dirty occlusion, and sensor cleaning reminder information is generated based on the dirty occlusion; otherwise, the occlusion type is determined to be dust occlusion.

[0105] In a possible implementation, it also includes: generating an actual comprehensive perception result of the perception module based on multiple actual perception results, and generating an expected comprehensive perception result of the perception module based on multiple expected perception results; calculating the Mahalanobis distance between the actual comprehensive perception result and the expected comprehensive perception result, and generating a fault diagnosis result of the perception module based on the Mahalanobis distance and a preset Mahalanobis distance threshold.

[0106] In a possible implementation, it also includes: calculating a test statistic result based on the expected perception result, the actual perception result and the SPRT algorithm; when the test statistic result is greater than a preset first test threshold, determining that the calibration parameters of the sensor are incorrect; when the test statistic result is less than a preset second test threshold, determining that the calibration parameters of the sensor are correct.

[0107] In a possible implementation, it also includes: receiving the positioning status sent by the leading vehicle; wherein the positioning status is used to characterize whether the first posture is a credible state; obtaining the formation status of the following vehicle; wherein the formation status is used to characterize whether the following vehicle is in a formation driving state; verifying the data obtained by the sensor to obtain a verification result; wherein the verification result is used to characterize whether the sensor can obtain the specified characteristics of the leading vehicle; when the positioning status is a credible state, the formation status is in a formation driving state, and the verification result is that the sensor can obtain the specified characteristics of the leading vehicle, it is determined that the fault diagnosis result is valid, otherwise it is determined that the fault diagnosis result is invalid.

[0108] The functions of each module in each device in the embodiments of the present application can be found in the corresponding description in the above method, and have corresponding beneficial effects, which will not be repeated here.

[0109] Figure 7FIG. 1 is a block diagram of an electronic device used to implement an embodiment of the present application. Figure 7 As shown, the electronic device includes: a memory 701 and a processor 702. The memory 701 stores a computer program that can be run on the processor 702. When the processor 702 executes the computer program, the method in the above embodiment is implemented. The number of the memory 701 and the processor 702 can be one or more.

[0110] The electronic device also includes:

[0111] The communication interface 703 is used to communicate with external devices and perform data exchange transmission.

[0112] If the memory 701, the processor 702 and the communication interface 703 are implemented independently, the memory 701, the processor 702 and the communication interface 703 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0113] Optionally, in a specific implementation, if the memory 701, the processor 702 and the communication interface 703 are integrated on a chip, the memory 701, the processor 702 and the communication interface 703 can communicate with each other through an internal interface.

[0114] An embodiment of the present application provides a computer-readable storage medium storing a computer program, which implements the method provided in the embodiment of the present application when the program is executed by a processor.

[0115] An embodiment of the present application provides a computer program product, wherein the computer program product includes a computer program, and when the computer program is executed by a processor, the method provided in the embodiment of the present application is implemented.

[0116] An embodiment of the present application also provides a chip, which includes a processor for calling and executing instructions stored in the memory from the memory, so that a communication device equipped with the chip executes the method provided by the embodiment of the present application.

[0117] An embodiment of the present application also provides a chip, including: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected via an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the method provided in the embodiment of the application.

[0118] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor supporting the Advanced RISC Machines (ARM) architecture.

[0119] Further, optionally, the above-mentioned memory may include a read-only memory and a random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. Among them, the non-volatile memory may include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may include a random access memory (RAM), which is used as an external cache. By way of exemplary but not limiting description, many forms of RAM are available. For example, static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DR RAM).

[0120] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0121] It should be noted that the information (including but not limited to device information, personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0122] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0123] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0124] Any process or method described in the flow chart or otherwise described herein can be understood as a module, fragment or portion of a code representing one or more executable instructions for implementing the steps of a specific logical function or process. And the scope of the preferred embodiment of the present application includes other implementations, in which the functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved.

[0125] The logic and / or steps described in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, which can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or used in combination with these instruction execution systems, devices or apparatuses.

[0126] It should be understood that the various parts of the present application can be implemented with hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented with software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above embodiment method can be completed by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium, and when the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0127] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.

[0128] The above is only an exemplary embodiment of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope recorded in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A sensor fault diagnosis method, wherein the sensor is arranged in a following vehicle traveling in a convoy, the method comprising: Acquire the first posture of the preceding vehicle and the external parameter calibration information of the preceding vehicle sent by the preceding vehicle, acquire the second posture of the self-vehicle, and acquire the physical characteristic parameters of the sensor and the sensor data of the preceding vehicle collected by the sensor; wherein the first posture is obtained by correcting the third posture; and the third posture is calculated by the following vehicle based on the second posture and the perception information of the preceding vehicle; Calculating an expected perception result of the sensor according to the first posture, the external parameter calibration information, the second posture and the physical characteristic parameter; Generating an actual perception result of the sensor according to the sensor data of the preceding vehicle; The similarity between the expected perception result and the actual perception result is calculated, and the sensor fault diagnosis result is determined based on the similarity and a preset similarity threshold.

2. The method according to claim 1, wherein: Acquiring sensor data of the preceding vehicle collected by the sensor, including: Determine a relative position between the following vehicle and the leading vehicle based on the first posture and the second posture; Determining a theoretical corner point of the preceding vehicle observable by the sensor based on the relative position and the external parameter calibration information; If the current information collected by the sensor includes the target corner point in the theoretical corner point, it is determined that the sensor is in a visible state; The sensor data of the preceding vehicle collected by the visual state sensor is obtained.

3. The method according to claim 1, wherein: Generating an actual perception result of the sensor according to the sensor data of the preceding vehicle includes: Determining a region of interest of the sensor for the preceding vehicle according to the physical characteristic parameter, the first posture, the second posture, and the external parameter calibration information; Extracting feature information of sensor data of a preceding vehicle falling within the region of interest; The actual sensing result of the sensor is generated according to the feature information.

4. The method according to claim 1, wherein: The physical characteristic parameters include the resolution and angle of the horizontal viewing angle, and the resolution and angle of the vertical viewing angle of the sensor; Calculating an expected perception result of the sensor according to the first posture, the second posture, the external parameter calibration information, and the physical characteristic parameter, including: Determine the horizontal viewing angle region of interest of the sensor to the preceding vehicle according to the angle of the horizontal viewing angle, the first posture, the second posture and the external parameter calibration information, and determine the vertical viewing angle region of interest of the sensor to the preceding vehicle according to the angle of the vertical viewing angle, the first posture, the second posture and the external parameter calibration information; Determine a plurality of horizontal viewing angle target points in the horizontal viewing angle region of interest, calculate the horizontal viewing angle target distance between the horizontal viewing angle target point and the sensor, determine a plurality of vertical viewing angle target points in the vertical viewing angle region of interest, calculate the vertical viewing angle target distance between the vertical viewing angle target point and the sensor; Calculating the number of horizontal viewing angle point clouds of the sensor based on the horizontal viewing angle target distance and the resolution of the horizontal viewing angle; calculating the number of vertical viewing angle point clouds of the sensor based on the vertical viewing angle target distance and the resolution of the vertical viewing angle; The product of the number of horizontal viewing point clouds and the number of vertical viewing point clouds is taken as the expected perception result of the sensor.

5. The method according to claim 1, wherein: The sensor is a laser radar or a millimeter wave radar, and calculating the similarity between the expected perception result and the actual perception result includes: Calculating a ratio of the number of point clouds in the actual perception result to the number of point clouds in the expected perception result; The ratio is used as the similarity between the expected perception result and the actual perception result.

6. The method according to claim 1, wherein: The sensor is a camera, and calculating the similarity between the expected perception result and the actual perception result includes: generating a first feature vector of the image in the actual perception result and a second feature vector of the image in the expected perception result; The similarity between the first feature vector and the second feature vector is calculated to obtain the similarity between the expected perception result and the actual perception result.

7. The method according to claim 5, wherein: Determining a sensor fault diagnosis result based on the similarity and a preset similarity threshold includes: If the ratio is not less than the similarity threshold, it is determined that the sensor is not faulty; If the ratio is less than the similarity threshold, a sensor failure is determined, and the sensor failure type is determined to be point cloud missing.

8. The method according to claim 1, wherein: After determining the sensor fault diagnosis result based on the similarity and the preset similarity threshold, the method further includes: Determining a region of interest of the sensor for the preceding vehicle according to the physical characteristic parameter, the first posture, the second posture, and the external parameter calibration information; Acquire the number of first point clouds within the region of interest and the number of second point clouds outside the region of interest; A ratio of the first point cloud quantity to the second point cloud quantity is calculated, and if the ratio satisfies a set ratio range, it is determined that the sensor fault type is sensor occlusion.

9. The method according to claim 8, wherein: Also includes: Within the set time, if the sensor fault type is determined to be sensor occlusion for multiple consecutive times, the occlusion type is determined to be dirty occlusion, and sensor cleaning reminder information is generated based on the dirty occlusion; otherwise, the occlusion type is determined to be dust occlusion.

10. The method according to any one of claims 1 to 9, wherein: Also includes: Based on the plurality of actual perception results, an actual comprehensive perception result of the perception module is generated; based on the plurality of expected perception results, an expected comprehensive perception result of the perception module is generated; A Mahalanobis distance between the actual integrated perception result and the expected integrated perception result is calculated, and a fault diagnosis result of the perception module is generated based on the Mahalanobis distance and a preset Mahalanobis distance threshold.

11. The method according to claim 10, wherein: Also includes: Calculate a test statistic result based on the expected perception result, the actual perception result and the SPRT algorithm; When the inspection statistical result is greater than a preset first inspection threshold, determining that the calibration parameters of the sensor are incorrect; When the inspection statistic result is less than a preset second inspection threshold, it is determined that the calibration parameters of the sensor are correct.

12. The method according to claim 11, wherein: Also includes: Receiving the positioning status sent by the preceding vehicle; wherein the positioning status is used to indicate whether the first posture is a credible state; Acquire the formation state of the following vehicle; wherein the formation state is used to indicate whether the following vehicle is in a formation driving state; Checking the data acquired by the sensor to obtain a test result; wherein the test result is used to indicate whether the sensor can acquire the specified features of the preceding vehicle; When the positioning state is a credible state, the formation state is in a formation driving state, and the inspection result is that the sensor can obtain the specified characteristics of the front vehicle, the fault diagnosis result is determined to be valid; otherwise, the fault diagnosis result is determined to be invalid.

13. An electronic device comprising a memory, a processor and a computer program stored in the memory, wherein the processor implements the method according to any one of claims 1 to 12 when executing the computer program.

14. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

15. A computer program product, wherein: The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

Images