Sensor anomaly detection method and device, vehicle and storage medium
By comparing sensor frame data and detecting target objects, sensor anomalies and failure areas are identified, solving the problem of low sensor detection efficiency and poor reliability, and improving the safety and reliability of autonomous vehicles.
Patent Information
- Application Number
- CN202410288754.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-13
- Publication Date
- 2025-09-16
AI Technical Summary
In existing technologies, sensor anomaly detection has low efficiency and poor reliability, making it difficult to effectively identify perception anomalies and failure areas.
By acquiring frame data of the sensing area from multiple sensors, dividing the target sub-area, and comparing it with the detection information of the target object, the sensor anomalies, including dirty or damaged areas, can be identified to improve the accuracy and reliability of detection.
It achieves rapid identification of sensor anomalies and accurate identification of failure areas, improves detection reliability and efficiency, and ensures the safety of autonomous vehicles.
Smart Images
Figure CN120645989A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of autonomous driving technology, and in particular to a sensor anomaly detection method, device, vehicle, and storage medium. Background Art
[0002] With the continuous development of technology, autonomous vehicles are becoming an integral part of future intelligent transportation. Autonomous vehicles primarily utilize multiple onboard sensors to perceive their surroundings and vehicle information, obtaining information about road conditions and obstacles. These sensors then intelligently and autonomously control the vehicle's steering, speed, and braking systems, enabling safe and reliable navigation.
[0003] Therefore, sensors are important components for realizing autonomous driving. In related technologies, the detection of each sensor is achieved based on the internal verification of each sensor, which is inefficient and has poor reliability. Summary of the Invention
[0004] The present application aims to solve one of the technical problems in the related art at least to a certain extent.
[0005] To this end, the present application proposes a sensor abnormality detection method, device, vehicle and storage medium. Based on the frame data collected by each sensor on the same perception area, it can be determined whether the perception of the target sub-area in the perception area by the target sensor among multiple sensors is normal, so as to identify the sensor with perception abnormality and identify the failure area, thereby improving the reliability of detection.
[0006] In one embodiment of the present application, a sensor anomaly detection method is provided, including:
[0007] Acquire frame data of data collected by multiple sensors of the vehicle in a sensing area; wherein the sensing area includes a target sub-area to be detected;
[0008] An abnormality in data collection for the target sub-region by a target sensor among the multiple sensors is determined based on detection information of the target sub-region in the frame data collected by the multiple sensors.
[0009] Another embodiment of the present application provides a sensor abnormality detection device, comprising:
[0010] An acquisition module, configured to acquire frame data of a sensing area acquired by a plurality of sensors of the vehicle; wherein the sensing area includes a target sub-area to be detected;
[0011] The determining module is configured to determine an abnormality in data collection performed on the target sub-region by a target sensor among the multiple sensors based on detection information of the target sub-region in the frame data collected by the multiple sensors.
[0012] Another embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the method described in the above aspect is implemented.
[0013] Another aspect of the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method described in the aforementioned aspect is implemented.
[0014] Another embodiment of the present application provides a computer program product having a computer program stored thereon, which implements the method described in the above aspect when the program is executed by a processor.
[0015] The sensor abnormality detection method, device, vehicle and storage medium proposed in the present application obtain frame data of data collected by multiple sensors of a vehicle on a perception area, wherein the perception area includes a target sub-area to be detected. Based on the detection information of the target sub-area in the frame data collected by the multiple sensors, the abnormality of the data collection of the target sub-area by the target sensor among the multiple sensors is determined. Based on the frame data collected by each sensor on the same perception area, it can be determined whether the perception of the target sub-area in the perception area by the target sensor among the multiple sensors is normal, so as to realize the identification of sensors with perception abnormalities, and can identify failure areas, thereby improving the reliability of detection.
[0016] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:
[0018] Figure 1 A flow chart of a sensor abnormality detection method provided in an embodiment of the present application;
[0019] Figure 2A A schematic diagram of the division of a sensing area provided in an embodiment of the present application;
[0020] Figure 2B A flowchart of another sensor anomaly detection method provided in an embodiment of the present application;
[0021] Figure 3 A flowchart of another sensor anomaly detection method provided in an embodiment of the present application;
[0022] Figure 4A schematic diagram of a sensor anomaly provided in an embodiment of the present application;
[0023] Figure 5 A schematic diagram of the structure of a sensor abnormality detection device provided in an embodiment of the present application;
[0024] Figure 6 Schematic diagram of the structure of a vehicle 600 shown in an embodiment of the present application. DETAILED DESCRIPTION
[0025] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.
[0026] The following describes the sensor abnormality detection method, device, vehicle, and storage medium according to embodiments of the present application with reference to the accompanying drawings.
[0027] Figure 1 A flow chart of a sensor anomaly detection method provided in an embodiment of the present application.
[0028] The execution subject of the sensor abnormality detection method in the embodiment of the present application is a sensor abnormality detection device, which can be set in a vehicle and is not limited in this embodiment.
[0029] like Figure 1 As shown, the method may include the following steps:
[0030] Step 101: Acquire frame data of a sensing area collected by multiple sensors of a vehicle.
[0031] In the embodiment of the present application, the perception area is the area around the vehicle, usually the area in front of the vehicle in the direction of travel. The vehicle can be an autonomous vehicle, which can sense objects in the perception area to identify obstacles and achieve autonomous driving. Among them, the perception areas of different sensors are the same, and the method of dividing the perception area into multiple sub-areas is the same. As an implementation method, Figure 2A As shown, the perception area is divided into multiple sub-areas, for example, 6 sub-areas, wherein the 6 sub-areas are:
[0032] Mesial: horizontal -6 to 6 m, vertical 0 to 50 m;
[0033] Far mid: horizontal -6~6m, vertical 50~100m;
[0034] Near left: horizontal -18 to -6 m, vertical 0 to 50 m;
[0035] Far left: horizontal -18 to -6 m, vertical 50 to 100 m;
[0036] Near right: 6 to 18 m horizontally, 0 to 50 m vertically;
[0037] Far right: 6 to 18 meters horizontally and 50 to 100 meters vertically.
[0038] The vehicle is equipped with multiple sensors with perception capabilities, including lidar, millimeter-wave radar, cameras, etc., which are not limited in the embodiments of this application. Different sensors collect frame data for the perception area, but the frame data collected by different sensors is different. For example, the radar collects a frame of point cloud data, while the camera collects a frame of image data. The frame data collected by each sensor is one or more frames.
[0039] In an embodiment of the present application, since the various sensors in the vehicle are usually turned on and used synchronously during the process of autonomous driving, one scenario in which the present application is used is during vehicle driving or autonomous driving.
[0040] The sensing area can be divided into multiple sub-areas, wherein the target sub-area is one of the multiple sub-areas and is the area to be detected. It should be understood that each of the multiple sub-areas can be used as the target sub-area to be detected.
[0041] Step 102 : determining an abnormality in data collection of the target sub-region by a target sensor among the multiple sensors based on detection information of the target sub-region in the frame data collected by the multiple sensors.
[0042] Among them, the target sensor is any one of the multiple sensors, and each of the multiple sensors can be used as a target sensor to identify whether there is any abnormality in the data collection of the target sensor. The collection abnormality is, for example, caused by the collection window of the target sensor being blocked or dirty.
[0043] In one implementation of the embodiment of the present application, based on the detection information of the target sub-area in the frame data collected by each sensor, the detection information of the object included in the target sub-area is determined, and the target object closest to the vehicle and / or in motion is determined based on the detection information of the object. The detection information of the target object is used as a comparison benchmark to determine whether the target object is detected in the frame data collected by the target sensor for the target sub-area, so as to determine whether the data collection for the target sub-area is normal, thereby identifying whether there is an abnormality in the target sensor, and at the same time determining which area is detected for the abnormality. Based on the area where the abnormality is detected, it is possible to identify which area of the sensor window may be dirty or damaged, so as to provide an early warning and timely troubleshooting of the abnormality.
[0044] Targeting the object closest to the vehicle and in motion can prevent sensor misdetection due to sensor failure. This ensures that sensors are not obstructed by occlusion or contamination, preventing missed detection of the target object, thus improving detection accuracy. Detection information includes location, motion (speed, acceleration, etc.), and size.
[0045] In the sensor abnormality detection method of the embodiment of the present application, frame data of data collection of a perception area by multiple sensors of a vehicle is obtained, wherein the perception area includes a target sub-area to be detected. Based on the detection information of the target sub-area in the frame data collected by the multiple sensors, the abnormality of the data collection of the target sub-area by the target sensor among the multiple sensors is determined. Based on the frame data collected by each sensor for the same perception area, it can be determined whether the perception of the target sub-area in the perception area by the target sensor among the multiple sensors is normal, so as to identify the sensor with the perception abnormality and identify the failure area, thereby improving the reliability of detection.
[0046] Based on the above embodiments, Figure 2B A flow chart of another sensor abnormality detection method provided in an embodiment of the present application is shown as follows: Figure 2B As shown, the method comprises the following steps:
[0047] Step 201: Acquire frame data of a sensing area collected by multiple sensors of a vehicle.
[0048] As an implementation, each sensor collects at least one frame of data from its sensing area at at least one acquisition moment. The sensing area includes the target sub-area to be detected. At each acquisition moment, each of the multiple sensors collects data from the sensing area, resulting in a frame of data collected by each sensor. For example, if there are three sensors, at acquisition moment 1, each of the three sensors collects one frame of data, resulting in three frames of data collected by the three sensors.
[0049] The explanations in the aforementioned embodiment are also applicable to step 201 , and the principles are the same, which will not be described again here.
[0050] Step 202 : determining first detection information of the target object in the target sub-region at each acquisition moment based on the detection information of the object in the target sub-region in the frame data acquired by the multiple sensors at each acquisition moment.
[0051] In an embodiment of the present application, at each acquisition moment, the frame data collected by each sensor at that acquisition moment is identified to determine whether an object is included in the target sub-area. Based on the position information of the object identified in the frame data collected by each sensor at that acquisition moment, the object closest to the vehicle and / or in motion at that acquisition moment is determined, and the detection information of the object is used as a comparison benchmark for that acquisition moment, that is, the object is used as the target object. Wherein, using the object closest to the vehicle and in motion as the target object can avoid missed detection by the sensor due to its perception ability, that is, ensure that if each sensor does not have anomalies caused by reasons such as occlusion or dirt, it will not cause missed detection of the target object, which can improve the accuracy of detection. Wherein, the detection information includes position information, motion information (speed, acceleration, etc.), and size information.
[0052] Step 203 : determining an abnormality in data collection of the target sub-region by the target sensor based on the first detection information of the target object in the target sub-region corresponding to at least one collection moment and the frame data collected by the target sensor among the multiple sensors at at least one collection moment.
[0053] In an embodiment of the present application, for each acquisition moment, the first detection information of the target object in the target sub-area in the target frame data corresponding to the acquisition moment and the second detection information of the target sub-area in the frame data acquired by the target sensor at the acquisition moment are compared to determine whether the target sensor detects the target object in the target sub-area at the acquisition moment. Based on whether the target object in the target sub-area is detected at the acquisition moment, it is determined whether data acquisition for the target sub-area is normal, thereby identifying whether there is an abnormality in the target sensor, and which area is detected abnormally. Based on the area where the abnormality is detected, it is possible to identify which area of the sensor window may be dirty or damaged, so as to provide an early warning and conduct abnormality troubleshooting in a timely manner.
[0054] In the sensor abnormality detection method of the embodiment of the present application, at least one frame of data collected by multiple sensors of a vehicle on a perception area at at least one collection moment is obtained, wherein the perception area includes a target sub-area to be detected, and first detection information of the target object in the target sub-area in the target frame data at each collection moment is determined based on the detection information of the object in the target sub-area in the frame data collected by the multiple sensors at each collection moment, and an abnormality in the data collection of the target sub-area by the target sensor is determined based on the first detection information of the target object in the target sub-area at at least one collection moment and the frame data collected by the target sensor of the multiple sensors at at least one collection moment, and multiple comparisons are performed on the perception situations of the target sensor on the target sub-area in the perception area based on at least one frame of data collected by each sensor on the same perception area at at least one collection moment, and based on the results of the multiple comparisons, whether the perception situations of the target sensor of the multiple sensors on the target sub-area in the perception area are normal can be determined, so as to identify sensors with perception abnormalities and identify failure areas, thereby improving the reliability of detection.
[0055] Based on the above embodiments, Figure 3 A flow chart of another sensor abnormality detection method provided in an embodiment of the present application is shown as follows: Figure 3 As shown, the method comprises the following steps:
[0056] Step 301: Acquire frame data of a sensing area collected by multiple sensors of a vehicle.
[0057] For each sensor, at least one frame of data is collected from the sensing area at at least one collection moment.
[0058] The relevant explanations in the aforementioned embodiment are also applicable to step 301 , and the principles are the same, which will not be repeated here.
[0059] Step 302 : determining first detection information of the target object in the target sub-region at each acquisition moment based on the detection information of the object in the target sub-region in the frame data acquired by the multiple sensors at each acquisition moment.
[0060] In one implementation of the embodiment of the present application, a target sub-area to be detected in the perception area is determined, and for each acquisition moment, at least one moving object is determined based on the detection information of the object in the target sub-area in the frame data acquired by each sensor at the acquisition moment, and then the positions of the detected moving objects are compared to determine the moving object closest to the vehicle. If there are multiple moving objects closest to the vehicle, one of the multiple moving objects can be selected based on the movement speed, object size, etc., and the frame data to which the selected moving object belongs is used as the target frame data, and the selected moving object is used as the target object in the target sub-area as the comparison benchmark.
[0061] The explanations in the aforementioned embodiment are also applicable to step 302 , and the principles are the same, so they will not be repeated here.
[0062] Step 303 : for each acquisition moment, determine a count value of the target sensor failing to detect the target object in the target sub-region at the acquisition moment based on the first detection information of the target object in the target sub-region corresponding to the acquisition moment and the frame data acquired by the target sensor at the acquisition moment.
[0063] In an embodiment of the present application, for each acquisition moment, based on the first detection information of the target object in the target sub-area corresponding to the acquisition moment, it is determined whether the frame data collected by the target sensor at the acquisition moment includes the target object in the target sub-area. As an implementation method, the first detection information of the target object in the target sub-area determined at the acquisition moment as a comparison reference and the second detection information corresponding to the target sub-area in the frame data collected by the target sensor at the acquisition moment are compared.
[0064] In one scenario, if the second detection information does not include target detection information that matches the first detection information, it is determined that the frame data collected by the target sensor at the collection moment does not include the target object in the target subregion. The detection information may be position information of the object. If the target subregion in the frame data collected by the sensor does not include position information of an object that matches the position information of the target object, where mismatched position information means that the difference between the positions is greater than a set threshold, it is determined that the frame data collected by the target sensor does not include the target object in the target subregion, that is, the target sensor did not collect the target object at the collection moment. The first count value is then used as the count value indicating that the target sensor did not detect the target object in the target subregion at the collection moment, where the first count value indicates that the target sensor did not detect the target object in the target subregion at the collection moment.
[0065] In another scenario, if the second detection information includes target detection information that matches the first detection information, it is determined that the frame data collected by the target sensor at the collection moment includes the target object in the target sub-region. The detection information may be the position information of the object. If the position information of the target object matches the position information of any object included in the target sub-region in the frame data collected by the sensor, where a position information match means that the difference between the positions is less than or equal to a set threshold, it is considered that the frame data collected by the target sensor includes the target object in the target sub-region, that is, the target sensor also collected the target object at the collection moment. As an example, the first count value is 1 and the second count value is 0.
[0066] Step 304 : determining an abnormality in data collection of the target sub-area by the target sensor based on the count value of the target sensor determined at at least one collection moment.
[0067] In one implementation of the present application, the count values of the target sensor determined at at least one acquisition moment are summed to obtain a total number of missed object detections corresponding to the target sensor. Based on the total number of missed object detections corresponding to the target sensor, an abnormality in the target sensor's data acquisition of the target sub-area is determined. In one scenario, if the total number is greater than a set threshold number of times corresponding to the target sub-area, it is determined that an abnormality has occurred in the target sensor's data acquisition of the target sub-area. In another scenario, if the total number is less than or equal to the set threshold number of times corresponding to the target sub-area, it is determined that the target sensor's data acquisition of the target sub-area is normal.
[0068] In another implementation of the embodiment of the present application, the count values of the target sensor determined at at least one acquisition moment are summed to obtain a total value of missed detections of the object corresponding to the target sensor. Based on the total value of missed detections of the object corresponding to the target sensor, the missed detection rate of the object missed detection corresponding to the target sensor is determined. Based on the missed detection rate and the total value, the abnormality of the target sensor in collecting data for the target sub-area is determined. In one scenario, a missed detection threshold and a number threshold corresponding to the set target sub-area are obtained. In response to the missed detection rate being greater than the missed detection threshold and the total value being greater than the number threshold, it is determined that there is an abnormality in the target sensor's data collection for the target sub-area, thereby improving the accuracy of abnormality identification. In another scenario, in response to the missed detection rate being less than or equal to the missed detection threshold, or the total value being less than or equal to the number threshold, it is determined that the target sensor's data collection for the target sub-area is normal.
[0069] As an example, the total count value of missed detections of objects is N, and the number of collection moments used to count the total count value is P. Then the missed detection rate R satisfies the following formula:
[0070] R = N / P;
[0071] Furthermore, in an embodiment of the present application, in order to improve the accuracy of anomaly recognition, corresponding set missed detection thresholds and number thresholds are set for different target sensors and detection scenarios. That is, the set missed detection thresholds and number thresholds corresponding to different sensors in different detection scenarios may be different. Among them, the detection scenario includes one or more of the different sub-areas in the detection time and the perception area. As an example, millimeter-wave radar is not prone to missed detection in the near-middle area, so the missed detection threshold corresponding to the near-middle area can be set to 0.3, and the number threshold can be set to 20. For cameras, it is easy to miss detection in the far-middle area, so the missed detection threshold corresponding to the far-middle area can be set to 0.8, and the number threshold can be set to 100.
[0072] It should be understood that by determining the missed detection rate and the number of missed detections, the level of missed detection can be determined. The levels include short-term missed detection, continuous missed detection, obvious missed detection, and small amount of missed detection. According to the level of missed detection, a failure warning can be issued, which is beneficial for managers to formulate strategies and fix anomalies in a timely manner.
[0073] It should be understood that for each acquisition moment, the corresponding target object is determined based on the frame data collected by each sensor. As long as at least one sensor detects the target object in the target sub-area at the acquisition moment, it means that the target object is real, and other sensors should detect it. If no detection is found, it means that there is an abnormality in the detection of the target sub-area by the corresponding sensor.
[0074] As an example, due to the dirt or occlusion of the sensor surface, some areas of the sensor window cannot accurately perceive obstacles in the scene. Figure 4 As shown in , it will cause glare or blur in the lower left area of the captured image, and will also cause the target to be missed. According to the sensor abnormality detection method of the embodiment of the present application, the perception area can be uniformly divided to achieve consistent division of the perception areas of different sensors for easy comparison. During the driving process of the vehicle, the perception results of different sensors are compared to determine that the target sensor that cannot continuously detect the object in the target sub-area to be detected has a detection abnormality, that is, it cannot detect the target object in the target sub-area. The target sub-area is a failure area that cannot be normally detected by the target sensor, and the detection data of multiple sensors is verified. Compared with the internal verification of a single sensor, the sensor with detection abnormality can be found more quickly, and the failure area can be distinguished, thereby improving the accuracy and efficiency of detection.
[0075] In the sensor abnormality detection method of the embodiment of the present application, at least one frame of data of each sensor among multiple sensors of the vehicle collecting data on the perception area at at least one collection moment is obtained, wherein the perception area includes a target sub-area to be detected, and based on the detection information of the object in the target sub-area in the frame data collected by the multiple sensors at each collection moment, first detection information of the target object in the target sub-area in the target frame data used as a comparison benchmark at each collection moment is determined, and based on the first detection information of the target object in the target sub-area at at least one collection moment and the frame data collected by the target sensor among the multiple sensors at at least one collection moment, the abnormality of the data collection of the target sub-area by the target sensor is determined, and based on multiple frames of data collected by each sensor on the same perception area, it can be determined whether the perception of the target sub-area in the perception area by the target sensor among the multiple sensors is normal, so as to identify the sensor with perception abnormality and identify the failure area, thereby improving the reliability of detection.
[0076] In order to implement the above embodiment, the embodiment of the present application further proposes a sensor abnormality detection device.
[0077] Figure 5 A schematic diagram of the structure of a sensor abnormality detection device provided in an embodiment of the present application.
[0078] like Figure 5 As shown, the device may include:
[0079] An acquisition module 51 is configured to acquire frame data of a sensing area acquired by multiple sensors of the vehicle; wherein the sensing area includes a target sub-area to be detected;
[0080] The determination module 52 is configured to determine an abnormality in data collection performed on the target sub-region by a target sensor among the multiple sensors based on detection information of the target sub-region in the frame data collected by the multiple sensors.
[0081] Furthermore, in one implementation of the embodiment of the present application, the frame data collected by each sensor includes frame data collected at at least one collection moment, and the determination module 52 is configured to execute:
[0082] determining first detection information of the target object in the target sub-region at each of the acquisition moments based on detection information of the object in the target sub-region in the frame data acquired by the multiple sensors at each of the acquisition moments;
[0083] An abnormality in data collection of the target sub-area by the target sensor is determined based on the first detection information of the target object in the target sub-area at the at least one collection moment and the frame data collected by the target sensor among the multiple sensors at the at least one collection moment.
[0084] In one implementation of the embodiment of the present application, the determination module 52 is configured to execute:
[0085] For each acquisition moment, determining a count value of the target sensor not detecting the target object in the target subregion at the acquisition moment based on first detection information of the target object in the target subregion at the acquisition moment and frame data acquired by the target sensor at the acquisition moment;
[0086] An abnormality in data collection by the target sensor on the target sub-area is determined according to the count value of the target sensor determined at the at least one collection moment.
[0087] In one implementation of the embodiment of the present application, the determination module 52 is configured to execute:
[0088] determining, based on first detection information of the target object in the target sub-region corresponding to the acquisition moment, whether the frame data acquired by the target sensor at the acquisition moment includes the target object in the target sub-region;
[0089] In response to the frame data collected by the target sensor at the collection time not including the target object in the target sub-region, the first count value is used as the count value of the target sensor not detecting the target object in the target sub-region at the collection time.
[0090] In one implementation of the embodiment of the present application, the determination module 52 is configured to execute:
[0091] In response to the frame data collected by the target sensor at the collection time including the target object in the target sub-region, the second count value is used as the count value of the target sensor not detecting the target object in the target sub-region at the collection time.
[0092] In one implementation of the embodiment of the present application, the determination module 52 is configured to execute:
[0093] Adding the count values of the target sensor determined at the at least one acquisition moment to obtain a total value of missed detections of objects corresponding to the target sensor;
[0094] Determining a missed detection rate of objects corresponding to the target sensor according to a total value of missed detections of objects corresponding to the target sensor;
[0095] An abnormality in data collection by the target sensor on the target sub-area is determined according to the missed detection rate and the total value.
[0096] In one implementation of the embodiment of the present application, the determination module 52 is configured to execute:
[0097] Obtaining the set missed detection threshold and number threshold corresponding to the target sub-region;
[0098] In response to the missed detection rate being greater than the missed detection threshold and the total value being greater than the number threshold, it is determined that an abnormality exists in data collection of the target sub-area by the target sensor.
[0099] In one implementation of the embodiment of the present application, the second determining module 52 is configured to execute:
[0100] In response to the missed detection rate being less than or equal to the missed detection threshold, or the total value being less than or equal to the number threshold, it is determined that the target sensor is collecting data from the target sub-area normally.
[0101] It should be noted that the above explanation of the method embodiment is also applicable to the device of this embodiment and will not be repeated here.
[0102] In the sensor abnormality detection device of the embodiment of the present application, frame data of data collection of a perception area by multiple sensors of a vehicle is obtained, wherein the perception area includes a target sub-area to be detected. Based on the detection information of the target sub-area in the frame data collected by the multiple sensors, the abnormality of the data collection of the target sub-area by the target sensor among the multiple sensors is determined. Based on the multiple frames of data collected by each sensor for the same perception area, it can be determined whether the perception of the target sub-area in the perception area by the target sensor among the multiple sensors is normal, so as to realize the identification of sensors with perception abnormalities and the identification of failure areas, thereby improving the reliability of detection.
[0103] In order to implement the above embodiments, the present application also proposes a vehicle, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the method described in the above method embodiments is implemented.
[0104] In order to implement the above embodiments, the present application also proposes a non-transitory computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method described in the above method embodiments is implemented.
[0105] In order to implement the above embodiments, the present application further proposes a computer program product on which a computer program is stored. When the computer program is executed by a processor, the method described in the above method embodiments is implemented.
[0106] Figure 6 FIG6 is a schematic diagram of the structure of a vehicle 600 according to an embodiment of the present application. For example, vehicle 600 may be a hybrid vehicle, a non-hybrid vehicle, an electric vehicle, a fuel cell vehicle, or another type of vehicle. Vehicle 600 may be an autonomous vehicle, a semi-autonomous vehicle, or a non-autonomous vehicle.
[0107] Reference Figure 6 Vehicle 600 may include various subsystems, such as an infotainment system 610, a perception system 620, a decision control system 630, a drive system 640, and a computing platform 650. Vehicle 600 may also include more or fewer subsystems, and each subsystem may include multiple components. Furthermore, each subsystem and each component of vehicle 600 may be interconnected via wired or wireless means.
[0108] In some embodiments, the infotainment system 610 may include a communication system, an entertainment system, a navigation system, and the like.
[0109] The perception system 620 may include several sensors for sensing information about the environment surrounding the vehicle 600. For example, the perception system 620 may include a global positioning system (which may be a GPS system, a BeiDou system, or other positioning systems), an inertial measurement unit (IMU), a laser radar, a millimeter-wave radar, an ultrasonic radar, and a camera.
[0110] The decision control system 630 may include a computing system, a vehicle controller, a steering system, a throttle, and a braking system.
[0111] The drive system 640 may include components that provide power to the vehicle 600. In one embodiment, the drive system 640 may include an engine, an energy source, a transmission system, and wheels. The engine may be an internal combustion engine, an electric motor, an air compression engine, or a combination thereof. The engine is capable of converting energy provided by the energy source into mechanical energy.
[0112] Some or all functions of the vehicle 600 are controlled by a computing platform 650. The computing platform 650 may include at least one processor 651 and a memory 652. The processor 651 may execute instructions 653 stored in the memory 652.
[0113] The processor 651 can be any conventional processor, such as a commercially available CPU. The processor can also include a graphics processor (GPU), a field programmable gate array (FPGA), a system on chip (SOC), an application specific integrated circuit (ASIC), or a combination thereof.
[0114] The memory 652 may be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.
[0115] In addition to instructions 653 , memory 652 may also store data, such as road maps, route information, and vehicle location, direction, speed, etc. The data stored in memory 652 may be used by computing platform 650 .
[0116] In the embodiment of the present disclosure, the processor 651 may execute the instruction 653 to complete all or part of the steps of the above method embodiment.
[0117] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.
[0118] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of such features. Throughout the description of this application, "plurality" means at least two, for example, two, three, etc., unless otherwise specifically defined.
[0119] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application belong.
[0120] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner if necessary, and then storing it in a computer memory.
[0121] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0122] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.
[0123] In addition, the functional units in the various embodiments of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into a module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
[0124] The storage medium mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application. Persons skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.
Claims
1. A sensor abnormality detection method, characterized in that: include: Acquire frame data of data collected by multiple sensors of the vehicle in a sensing area; wherein the sensing area includes a target sub-area to be detected; An abnormality in data collection for the target sub-region by a target sensor among the multiple sensors is determined based on detection information of the target sub-region in the frame data collected by the multiple sensors.
2. The method according to claim 1, wherein in, The frame data collected by each sensor includes frame data collected at at least one collection moment, and determining, based on detection information of the target sub-area in the frame data collected by the multiple sensors, an abnormality in data collection by a target sensor among the multiple sensors for the target sub-area, includes: determining first detection information of the target object in the target sub-region at each of the acquisition moments based on detection information of the object in the target sub-region in the frame data acquired by the multiple sensors at each of the acquisition moments; An abnormality in data collection of the target sub-area by the target sensor is determined based on the first detection information of the target object in the target sub-area at the at least one collection moment and the frame data collected by the target sensor among the multiple sensors at the at least one collection moment.
3. The method according to claim 2, wherein The determining, based on the first detection information of the target object in the target sub-region at the at least one acquisition moment and the frame data acquired by the target sensor among the multiple sensors at the at least one acquisition moment, an abnormality in data acquisition by the target sensor on the target sub-region, includes: For each acquisition moment, determining a count value of the target sensor not detecting the target object in the target subregion at the acquisition moment based on first detection information of the target object in the target subregion at the acquisition moment and frame data acquired by the target sensor at the acquisition moment; An abnormality in data collection by the target sensor on the target sub-area is determined according to the count value of the target sensor determined at the at least one collection moment.
4. The method according to claim 3, wherein The determining, based on the first detection information of the target object in the target sub-region at the acquisition moment and the frame data acquired by the target sensor at the acquisition moment, a count value of the target object in the target sub-region not being detected by the target sensor at the acquisition moment, includes: determining, based on first detection information of the target object in the target sub-region corresponding to the acquisition moment, whether the frame data acquired by the target sensor at the acquisition moment includes the target object in the target sub-region; In response to the frame data collected by the target sensor at the collection time not including the target object in the target sub-region, the first count value is used as the count value of the target sensor not detecting the target object in the target sub-region at the collection time.
5. The method according to claim 4, wherein The method further comprises: In response to the frame data collected by the target sensor at the collection time including the target object in the target sub-region, the second count value is used as the count value of the target sensor not detecting the target object in the target sub-region at the collection time.
6. The method according to claim 4, wherein The determining, based on the count value of the target sensor determined at the at least one collection moment, an abnormality in data collection by the target sensor on the target sub-area, includes: Adding the count values of the target sensor determined at the at least one acquisition moment to obtain a total value of missed detections of objects corresponding to the target sensor; Determining a missed detection rate of objects corresponding to the target sensor according to a total value of missed detections of objects corresponding to the target sensor; An abnormality in data collection by the target sensor on the target sub-area is determined according to the missed detection rate and the total value.
7. The method according to claim 6, wherein The determining, based on the missed detection rate and the total value, an abnormality in data collection by the target sensor on the target sub-area includes: Obtaining the set missed detection threshold and number threshold corresponding to the target sub-region; In response to the missed detection rate being greater than the missed detection threshold and the total value being greater than the number threshold, it is determined that an abnormality exists in data collection of the target sub-area by the target sensor.
8. The method according to claim 7, wherein The determining, based on the missed detection rate and the total value, an abnormality in data collection by the target sensor on the target sub-area includes: In response to the missed detection rate being less than or equal to the missed detection threshold, or the total value being less than or equal to the number threshold, it is determined that the target sensor is collecting data from the target sub-area normally.
9. A sensor abnormality detection device, characterized in that: include: An acquisition module, configured to acquire frame data of a sensing area acquired by a plurality of sensors of the vehicle; wherein the sensing area includes a target sub-area to be detected; The determining module is configured to determine an abnormality in data collection performed on the target sub-region by a target sensor among the multiple sensors based on detection information of the target sub-region in the frame data collected by the multiple sensors.
10. A vehicle, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: Implement the steps of the method according to any one of claims 1 to 8.
11. A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of a mobile terminal, enables the mobile terminal to perform the steps of the method according to any one of claims 1 to 8.
12. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 8.