Obstacle sensing abnormity monitoring method and device, vehicle and storage medium

By detecting and analyzing obstacles on the lidar point cloud data, identifying and handling abnormal situations, the problem that obstacle perception algorithms in the prior art cannot detect and deal with abnormalities in a timely manner, and timely monitoring and information on obstacle perception abnormalities is realized Saving ensures vehicle safety and convenience of problem analysis.

CN119064953APending Publication Date: 2024-12-03CHINA FAW CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411277119.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-12
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The existing obstacle perception algorithm cannot detect abnormal situations in time, resulting in the inability to obtain relevant information in time, especially in an unmanned testing environment that cannot handle abnormalities in time.

Method used

By target detection and obstacle segmentation processing of point cloud data obtained by lidar scanning, we determine whether the number of obstacles, center point position, size and orientation are abnormal, and monitor and emergency treatment are carried out based on the abnormal analysis results, including safe parking of vehicles on the side and save environmental information, etc.

Benefits of technology

Timely monitoring and information preservation of obstacle perception abnormalities is realized, vehicle safety is ensured, subsequent analysis is facilitated, and the problem of failure to detect and handle abnormalities in the prior art is solved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119064953A_ABST
    Figure CN119064953A_ABST
Patent Text Reader

Abstract

The invention discloses an obstacle sensing abnormity monitoring method and device, a vehicle and a storage medium, and the method comprises the steps: carrying out the obstacle detection processing of point cloud data obtained through laser radar scanning according to a target detection mode and an obstacle segmentation mode, and obtaining a corresponding first obstacle detection result and a second obstacle detection result; according to one or more of the number of obstacles, the central point position of the obstacle, the size of the obstacle and the orientation of the obstacle in the first obstacle detection result and the second obstacle detection result, the obstacle is detected; respectively determining a first anomaly analysis result corresponding to the first obstacle detection result and a second anomaly analysis result corresponding to the second obstacle detection result; according to the first anomaly analysis result and the second anomaly analysis result, determining an obstacle sensing anomaly monitoring result; according to the invention, timely monitoring of obstacle sensing abnormity is realized, related information during abnormity is timely stored, the safety of the vehicle is ensured, and subsequent analysis of abnormity problems is facilitated.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving, and in particular to an obstacle perception abnormality monitoring method, device, vehicle and storage medium. Background Art

[0002] With the rapid development of intelligent driving technology, the obstacle perception algorithm based on LiDAR has also developed rapidly. In the early days, obstacles could only be clustered through traditional geometric operations. Later, with the development of deep learning, various point cloud target detection algorithms based on anchors and anchor-free have emerged. Nowadays, obstacle perception algorithms are increasingly developing in an integrated and fully closed-loop manner.

[0003] However, the current obstacle perception algorithm is not perfect. When entering some special environments or when the system settings are abnormal, the obstacle perception algorithm will produce some abnormal results. When an abnormality occurs, it is very important to save the test environment information and log information in time. However, the testers on the car are likely to fail to respond in time. Even when conducting unmanned testing, there are no testers on the smart driving car, so it is impossible to detect the abnormality of the obstacle perception algorithm in time, and thus it is impossible to obtain relevant information when the algorithm is abnormal. Summary of the invention

[0004] The present invention provides an obstacle perception anomaly monitoring method, device, vehicle and storage medium to solve the problem that the prior art cannot detect the obstacle perception anomaly in time, and thus cannot obtain relevant information in time when the anomaly occurs.

[0005] In a first aspect, an embodiment of the present invention provides a method for monitoring anomaly of obstacle perception, comprising:

[0006] According to the target detection method and the obstacle segmentation method, obstacle detection processing is performed on the point cloud data obtained by the laser radar scanning to obtain corresponding first obstacle detection results and second obstacle detection results;

[0007] Determine, according to one or more of the number of obstacles, the position of the obstacle center point, the size of the obstacle, and the direction of the obstacle in the first obstacle detection result and the second obstacle detection result, a first abnormality analysis result corresponding to the first obstacle detection result and a second abnormality analysis result corresponding to the second obstacle detection result;

[0008] An obstacle perception abnormality monitoring result is determined based on the first abnormality analysis result and the second abnormality analysis result.

[0009] Optionally, after determining the obstacle perception abnormality monitoring result according to the first abnormality analysis result and the second abnormality analysis result, the method further includes:

[0010] Determine the level corresponding to the obstacle perception abnormality monitoring result by performing grading processing on the obstacle perception abnormality monitoring result;

[0011] According to the level corresponding to the obstacle perception abnormality monitoring result, the corresponding emergency handling operation is performed, wherein the emergency handling operation is one or more of the following: the vehicle is safely parked on the side of the road, and the environmental information, log information and full data within a set time before and after the obstacle perception abnormality are saved.

[0012] Optionally, the step of determining a first abnormality analysis result relative to the first obstacle detection result includes:

[0013] If the first obstacle quantity in the first obstacle detection result does not satisfy a set first quantity change trend, determining that the first obstacle quantity is abnormal;

[0014] If the position of the center point of the first obstacle in the first obstacle detection result does not meet the set first center point position arrangement requirement, determining that the position of the center point of the first obstacle is abnormal;

[0015] If the first obstacle size in the first obstacle detection result does not meet the set first size requirement, determining that the first obstacle size is abnormal;

[0016] If the first obstacle orientation in the first obstacle detection result does not meet the set first orientation requirement, it is determined that the first obstacle orientation is abnormal.

[0017] The determination result of whether the number of the first obstacles, the position of the center point of the first obstacle, the size of the first obstacle, and the direction of the first obstacle are abnormal is determined as the first abnormality analysis result.

[0018] Optionally, if the position of the center point of the first obstacle in the first obstacle detection result does not meet the set first center point position arrangement requirement, determining that the position of the center point of the first obstacle is abnormal includes:

[0019] If the position of the center point of the first obstacle in the first obstacle detection result is not within the first detection range, determining that the position of the center point of the first obstacle is abnormal;

[0020] If a first variance determined based on the first obstacle center point position in the first obstacle detection result is less than a set first variance threshold, determining that the first obstacle center point position is abnormal;

[0021] The first obstacle envelope is determined based on the first obstacle center point position, the first obstacle size, and the first obstacle direction in the first obstacle detection result. If the first obstacle envelope in any frame within the period has an intersection-over-union (IOU) with the vehicle envelope, it is determined that the first obstacle center point position is abnormal.

[0022] Optionally, if the first obstacle size in the first obstacle detection result does not meet a set first size requirement, determining that the first obstacle size is abnormal includes:

[0023] If the first obstacle size in the first obstacle detection result is not within the set first size range, determining that the first obstacle size is abnormal;

[0024] If the first obstacle size in the first obstacle detection result does not meet the set first size ratio rule, it is determined that the first obstacle size is abnormal.

[0025] Optionally, the step of determining a second abnormality analysis result relative to the second obstacle detection result includes:

[0026] If the number of second obstacles in the second obstacle detection result does not satisfy the set second number change trend, determining that the number of the second obstacles is abnormal;

[0027] determining a second obstacle envelope based on the second obstacle center point position, the second obstacle size, and the second obstacle orientation in the second obstacle detection result, and determining that the second obstacle envelope is abnormal if an intersection-over-union (IOU) exists between the second obstacle envelopes in any frame within a period;

[0028] determining a second obstacle area based on the second obstacle envelope, and determining that the second obstacle area is abnormal if the second obstacle area is greater than a set second area threshold and the second obstacle height is less than a set second height threshold;

[0029] A determination result of whether the second obstacle quantity, the second obstacle envelope, and the second obstacle area are abnormal is determined as the second abnormality analysis result.

[0030] Optionally, obstacle detection processing is performed on the point cloud data obtained by laser radar scanning according to the target detection method and the obstacle segmentation method, including:

[0031] According to the target detection method and obstacle segmentation method, obstacle detection processing is performed periodically on the point cloud data obtained by laser radar scanning at set time intervals.

[0032] In a second aspect, an embodiment of the present invention provides an obstacle sensing abnormality monitoring device, comprising:

[0033] A detection processing module, used to perform obstacle detection processing on the point cloud data obtained by laser radar scanning according to the target detection mode and the obstacle segmentation mode, and obtain corresponding first obstacle detection results and second obstacle detection results;

[0034] a first determination module, configured to determine, according to one or more of the number of obstacles, the position of the obstacle center point, the size of the obstacle, and the direction of the obstacle in the first obstacle detection result and the second obstacle detection result, a first abnormality analysis result corresponding to the first obstacle detection result and a second abnormality analysis result corresponding to the second obstacle detection result;

[0035] The second determination module is used to determine the obstacle perception abnormality monitoring result according to the first abnormality analysis result and the second abnormality analysis result.

[0036] In a third aspect, an embodiment of the present invention provides a vehicle, including:

[0037] at least one controller; and

[0038] A memory in communication with the at least one controller; wherein,

[0039] The memory stores a computer program that can be executed by the at least one controller, and the computer program is executed by the at least one controller so that the at least one controller can execute the obstacle perception abnormality monitoring method described in any embodiment of the present invention.

[0040] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a controller to implement the obstacle perception anomaly monitoring method described in any embodiment of the present invention when executed.

[0041] The technical solution of the embodiment of the present invention performs obstacle detection processing on the point cloud data obtained by laser radar scanning according to the target detection mode and the obstacle segmentation mode, respectively, to obtain the corresponding first obstacle detection result and the second obstacle detection result; according to one or more of the number of obstacles, the position of the obstacle center point, the obstacle size and the obstacle orientation in the first obstacle detection result and the second obstacle detection result, respectively determine the first abnormality analysis result corresponding to the first obstacle detection result and the second abnormality analysis result corresponding to the second obstacle detection result; according to the first abnormality analysis result and the second abnormality analysis result, determine the obstacle perception abnormality monitoring result, solve the problem in the prior art that the obstacle perception abnormality cannot be detected in time, and then the relevant information when the abnormality occurs cannot be obtained in time, and the timely monitoring of the obstacle perception abnormality is realized, and then the relevant information when the obstacle perception abnormality is monitored can be saved in time, so as to ensure the safety of the vehicle and facilitate the subsequent analysis of abnormal problems.

[0042] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0044] Figure 1 A flowchart of an obstacle perception anomaly monitoring method provided in Embodiment 1 of the present invention;

[0045] Figure 2 A flowchart of an obstacle perception anomaly monitoring method provided in Embodiment 2 of the present invention;

[0046] Figure 3 A schematic diagram of the number of obstacles provided in the second embodiment of the present invention;

[0047] Figure 4 A schematic diagram of the position of the center point of an obstacle provided in the second embodiment of the present invention;

[0048] Figure 5 A schematic diagram of obstacle point cloud distribution provided in Embodiment 2 of the present invention;

[0049] Figure 6A specific flow chart of an obstacle perception anomaly monitoring method provided in Embodiment 2 of the present invention;

[0050] Figure 7 A schematic diagram of the structure of an obstacle sensing abnormality monitoring device provided in Embodiment 3 of the present invention;

[0051] Figure 8 A schematic diagram of the hardware structure of a vehicle provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0052] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0053] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0054] Embodiment 1

[0055] Figure 1 This is a flowchart of an obstacle perception abnormality monitoring method provided in the first embodiment of the present invention. This embodiment is applicable to the case of monitoring obstacle perception abnormalities. The method can be executed by an obstacle perception abnormality monitoring device. The obstacle perception abnormality monitoring device can be implemented in the form of hardware and / or software. The obstacle perception abnormality monitoring device can be configured in a vehicle. Figure 1 As shown, the method includes:

[0056] S110, performing obstacle detection processing on the point cloud data obtained by laser radar scanning according to the target detection method and the obstacle segmentation method, and obtaining corresponding first obstacle detection results and second obstacle detection results.

[0057] Among them, the target detection method may refer to a target detection algorithm based on deep learning; the obstacle segmentation method may refer to an obstacle segmentation algorithm based on geometric clustering; the laser radar may be a mechanical laser radar, a hybrid semi-solid laser radar, a fully solid-state laser radar, etc.; the point cloud data may refer to the three-dimensional space data obtained by scanning the laser radar, specifically, it may refer to a set of points in the three-dimensional space obtained by scanning the laser radar, each point containing information such as three-dimensional coordinates, color, intensity value, etc.; the obstacle detection processing may refer to the obstacle data obtained by processing the point cloud data through the perception algorithm; the first obstacle detection result may refer to the obstacle data obtained by processing the point cloud data by the target detection algorithm based on deep learning; the second obstacle detection result may refer to the obstacle data obtained by processing the point cloud data by the obstacle segmentation algorithm based on geometric clustering.

[0058] Specifically, the point cloud data obtained by laser radar scanning can be processed by the perception algorithm based on the target detection algorithm of deep learning and the obstacle segmentation algorithm based on geometric clustering, respectively, to obtain the first obstacle detection result corresponding to the target detection algorithm of deep learning and the second obstacle detection result corresponding to the obstacle segmentation algorithm of geometric clustering. In addition, the obstacle detection result includes one or more obstacle data, and the one or more obstacle data exist in the form of a list; illustratively, after the point cloud data is processed by the two perception algorithms, the first obstacle detection result, i.e., the first obstacle data list obj_list_det, and the second obstacle detection result, i.e., the second obstacle data list obj_list_seg, will be output respectively. In addition, although obstacle detection will occupy a small amount of computing resources, in the process of driving the autonomous driving vehicle, it is not too much to have sufficient resources. Therefore, the present invention does not need to perform obstacle detection processing on the obtained point cloud data in real time. Specifically, a certain time interval can be set to perform periodic obstacle detection processing on the point cloud data at the set time interval.

[0059] In this embodiment, two obstacle perception algorithms are used to process the point cloud data obtained by laser radar scanning, and a first obstacle data list obj_list_det corresponding to the deep learning target detection algorithm and a second obstacle data list obj_list_seg corresponding to the geometric clustering obstacle segmentation algorithm can be output through two channels.

[0060] S120: Determine, according to one or more of the number of obstacles, the position of the obstacle center point, the size of the obstacle, and the direction of the obstacle in the first obstacle detection result and the second obstacle detection result, a first abnormality analysis result corresponding to the first obstacle detection result and a second abnormality analysis result corresponding to the second obstacle detection result.

[0061] Among them, the number of obstacles may refer to the number of all obstacles in the current frame; the position of the obstacle center point may refer to the three-dimensional coordinates of the center points of all obstacles in the current frame; the obstacle size may refer to the length, width, and height of all obstacles in the current frame; the obstacle orientation may refer to the yaw angle (yaw) of all obstacles in the current frame; the current frame may refer to each frame in the period of obstacle detection processing, which can be understood as each frame in the period of obstacle detection processing can be used as the current frame. The first abnormal analysis result may refer to the determination result of whether the obstacle data in the first obstacle detection result is abnormal; the second abnormal analysis result may refer to the determination result of whether the obstacle data in the second obstacle detection result is abnormal.

[0062] Specifically, the first judgment condition set in advance can be used to judge whether one or more of the first obstacle quantity, the first obstacle center point position, the first obstacle size, and the first obstacle orientation in the first obstacle detection result is abnormal, and the corresponding first abnormal analysis result is determined according to the abnormal judgment result. The second judgment condition set in advance can be used to judge whether one or more of the second obstacle quantity, the second obstacle envelope, and the second obstacle area in the second obstacle detection result is abnormal, and the corresponding first abnormal analysis result is determined according to the abnormal judgment result; wherein, the second obstacle envelope can be determined by the second obstacle center point position, the second obstacle size, and the second obstacle orientation, and the second obstacle area can be determined by the second obstacle envelope.

[0063] In this embodiment, whether the obstacle data in the first obstacle detection result and the second obstacle detection result are abnormal can be judged according to the pre-set first judgment condition and the second judgment condition, and the first abnormal analysis result corresponding to the first obstacle detection result and the second abnormal analysis result corresponding to the second obstacle detection result can be determined according to the judgment result of whether they are abnormal.

[0064] S130. Determine an obstacle perception abnormality monitoring result according to the first abnormality analysis result and the second abnormality analysis result.

[0065] Among them, the obstacle perception abnormality monitoring result may refer to a warning code, and the warning code may refer to a specific digital code or message returned when an abnormal obstacle data occurs, such as "404-the requested webpage does not exist", "503-the service is unavailable", etc. in the prior art. In this embodiment, a corresponding warning code may be set for each obstacle data abnormality. For example, if the number of obstacles is abnormal, "XXX-the number of obstacles is abnormal" is returned; if the size of the obstacle is abnormal, "YYY-the size of the obstacle is abnormal" is returned. In addition, the warning code may also record the relevant content of the obstacle perception abnormality in detail to help subsequent staff analyze and solve the abnormal problem.

[0066] Specifically, it is possible to determine whether the first obstacle detection result is abnormal based on the first abnormality analysis result, and to determine whether the second obstacle detection result is abnormal based on the second abnormality analysis result. If both the first abnormality analysis result and the second abnormality analysis result are no abnormality, it is determined that the obstacle perception is not abnormal; if both the first abnormality analysis result and the second abnormality analysis result are abnormal, or the first abnormality analysis result is no abnormality and the second abnormality analysis result is abnormal, or the first abnormality analysis result is abnormality and the second abnormality analysis result is no abnormality, it is determined that the obstacle perception is abnormal, and a warning code can be determined and returned to the vehicle control system, the driver, the remote control center, etc.

[0067] In this embodiment, obstacle detection processing is performed on the point cloud data obtained by laser radar scanning according to the target detection mode and the obstacle segmentation mode, so as to obtain the corresponding first obstacle detection result and the second obstacle detection result; a first abnormality analysis result corresponding to the first obstacle detection result and a second abnormality analysis result corresponding to the second obstacle detection result are determined according to one or more of the number of obstacles, the position of the obstacle center point, the size of the obstacle and the direction of the obstacle in the first obstacle detection result and the second obstacle detection result; an obstacle perception abnormality monitoring result is determined according to the first abnormality analysis result and the second abnormality analysis result, so as to solve the problem that the obstacle perception abnormality cannot be detected in time in the prior art, and the relevant information when the abnormality occurs cannot be obtained in time, so as to realize the timely monitoring of the obstacle perception abnormality, and the relevant information when the obstacle perception is abnormal can be saved in time according to the obstacle perception abnormality monitoring result, which not only ensures the safety of the vehicle but also facilitates the subsequent analysis of the obstacle perception abnormality problem by the staff.

[0068] Optionally, obstacle detection processing is performed on the point cloud data obtained by laser radar scanning according to the target detection method and the obstacle segmentation method, including:

[0069] According to the target detection method and obstacle segmentation method, obstacle detection processing is performed periodically on the point cloud data obtained by laser radar scanning at set time intervals.

[0070] Among them, the setting time interval can be adaptively set according to the stability of the current vehicle automatic driving system. For example, if the automatic driving system is relatively stable, then 15 minutes can be set as the setting time interval for periodic obstacle detection processing; if the automatic driving system is in the early stage of development, there are many problems and poor stability, then 3 minutes can be set as the setting time interval for periodic obstacle detection processing. Periodic obstacle detection processing can refer to processing the scanned point cloud data within a fixed time interval for obstacle detection processing; specifically, it is very important to select a suitable detection cycle. If the cycle is too long, it may lead to waste of resources and increase in computing burden; if the cycle is too short, it may affect the accuracy of obstacle perception data detection; therefore, the detection cycle can also be adjusted according to the stability of the current vehicle automatic driving system. For example, if the automatic driving system is relatively stable, then the detection cycle can be set to 5 minutes; if the stability of the automatic driving system is poor, then the detection cycle can be set to 15 minutes.

[0071] In a specific embodiment, if the autonomous driving system is relatively stable, the time interval can be set to 15 minutes and the detection cycle to 5 minutes. The specific detection situation can be that after the 5-minute obstacle detection process is completed, there will be an interval of 15 minutes, and no obstacle detection process will be performed during the 15 minutes. After the 15 minutes are completed, obstacle detection process will be performed again for 5 minutes. After the process is completed, there will be an interval of 15 minutes without detection process again, and this process will be repeated alternately until the vehicle stops.

[0072] In this embodiment, according to the target detection method and the obstacle segmentation method, obstacle detection processing is periodically performed on the point cloud data obtained by the laser radar scanning at set time intervals, which can effectively save resources and ensure the accuracy of obstacle perception abnormality monitoring.

[0073] Embodiment 2

[0074] Figure 2 This is a flow chart of an obstacle sensing abnormality monitoring method provided in the second embodiment of the present invention. The technical solution of this embodiment is further refined on the basis of the above embodiment. Figure 2 As shown, the method includes:

[0075] S210: Perform obstacle detection processing on the point cloud data obtained by the laser radar scanning according to the target detection method and the obstacle segmentation method, and obtain corresponding first obstacle detection results and second obstacle detection results.

[0076] S220: Determine, according to one or more of the number of obstacles, the position of the obstacle center point, the size of the obstacle, and the direction of the obstacle in the first obstacle detection result and the second obstacle detection result, a first abnormality analysis result corresponding to the first obstacle detection result and a second abnormality analysis result corresponding to the second obstacle detection result.

[0077] S230. Determine an obstacle perception abnormality monitoring result according to the first abnormality analysis result and the second abnormality analysis result.

[0078] S240: Determine a level corresponding to the obstacle perception abnormality monitoring result by performing classification processing on the obstacle perception abnormality monitoring result.

[0079] Among them, graded processing can refer to level division processing; for example, it can be divided into high-risk, general and other levels.

[0080] Specifically, since different obstacle perception abnormality monitoring results have different degrees of influence on vehicle safety, the present invention can pre-classify the obstacle perception abnormality monitoring results according to the degrees of influence of different obstacle perception abnormality monitoring results on vehicle safety.

[0081] In this embodiment, by classifying the obstacle perception abnormality monitoring result, ie, the warning code, into levels in advance, after the warning code is determined, the level corresponding to the warning code can be determined.

[0082] S250. Execute corresponding emergency handling operations according to the level corresponding to the obstacle perception abnormality monitoring result, wherein the emergency handling operations are one or more of the following: safely pulling over the vehicle, saving environmental information, log information and full data within a set time before and after the obstacle perception abnormality.

[0083] Among them, environmental information may refer to a detailed description of the external and internal conditions of the vehicle within a set time before and after the moment of obstacle perception abnormality. For example, the vehicle's operating system type and version, hardware configuration, network connection status, installed software and its configuration, etc. Log information may refer to a detailed record of operations, events, and obstacle perception abnormalities generated during the vehicle's driving process within a set time before and after the moment of obstacle perception abnormality; specifically, it may include a timestamp, a description of the obstacle perception abnormality event, and a warning code. Full data may refer to a collection of all data in the entire vehicle system within a set time before and after the moment of obstacle perception abnormality, including the sum of all tables, files, logs, and other data. The specific set time may refer to 5 minutes, 10 minutes, or other times before and after the abnormal moment. This embodiment does not limit the specific set time.

[0084] For example, if the level corresponding to the obstacle perception abnormality monitoring result is a high-risk level, the corresponding emergency handling operation may be to control the vehicle to park safely on the side of the road, and to save the environmental information, log information and full data within a set time before and after the obstacle perception abnormality; if the level corresponding to the obstacle perception abnormality monitoring result is a general level, the corresponding emergency handling operation may be only to save the environmental information, log information and full data within a set time before and after the obstacle perception abnormality for subsequent staff to use for abnormality analysis.

[0085] In this embodiment, obstacle detection processing is performed on the point cloud data obtained by laser radar scanning through a target detection method and an obstacle segmentation method, respectively, to obtain a corresponding first obstacle detection result and a second obstacle detection result; a corresponding first abnormality analysis result and a corresponding second abnormality analysis result are determined according to the first obstacle detection result and the second obstacle detection result; an obstacle perception abnormality monitoring result is determined according to the first abnormality analysis result and the second abnormality analysis result; and a vehicle is controlled to park safely by the level corresponding to the obstacle perception abnormality monitoring result, and environmental information, log information and full data within a set time before and after the obstacle perception abnormality are saved; the problem that the existing technology cannot detect the abnormality of obstacle perception in time is solved. The problem of being unable to obtain relevant information in time when the abnormality occurs is solved, which realizes timely monitoring of obstacle perception abnormalities. The vehicle can also be controlled to park safely on the side of the road in time according to the obstacle perception abnormality monitoring results, and the environmental information, log information and full data within a set time before and after the obstacle perception abnormality are saved. This not only ensures the safety of the vehicle, but also facilitates subsequent staff to analyze the obstacle perception abnormality problem according to the environmental information, log information and full data within a set time before and after the obstacle perception abnormality, saving human resources and reducing the pressure on staff. In addition, only by discovering the problem can the problem be better solved. Therefore, the technical solution of the present invention is also of great significance to radar perception and even the entire field of intelligent driving.

[0086] Optionally, the step of determining a first abnormality analysis result relative to the first obstacle detection result includes:

[0087] Specifically, the obstacle detection algorithm based on deep learning performs obstacle detection processing on the point cloud data obtained by laser radar scanning to obtain the corresponding first obstacle detection result, that is, the first obstacle data list obj_list_det. The obstacle detection algorithm based on deep learning usually detects conventional targets such as pedestrians and motor vehicles, and the output target is a rectangular frame. For example, the number of first obstacles in the first obstacle data list of the current frame can be set to N, and the position of the center point of the first obstacle can be set to (cx n ,cy n , cz n), the size of the first obstacle is (w n ,h n , l n ), the first obstacle is yaw n , where n ranges from 0 to (N-1).

[0088] If the first obstacle quantity in the first obstacle detection result does not satisfy a set first quantity change trend, determining that the first obstacle quantity is abnormal;

[0089] Among them, the set first quantity change trend can refer to a pre-set slow quantity change trend; for example, in a continuous scene, the change in the number of obstacles will not change too much, and the obstacles are from far to near, so the number should be a slowly changing trend; if there is a sudden increase, decrease, or even disappearance of obstacles, it may be that the vehicle has driven into an abnormal perception area such as outside the map, or there may be an abnormality in the point cloud data obtained by lidar scanning. Figure 3 A schematic diagram of the number of obstacles provided in the second embodiment of the present invention is shown in FIG. Figure 3 As shown, the normal number of obstacles should be Figure 3 The gentle change shown in the middle left figure, if there is Figure 3 As shown in the middle right figure, it is determined that the number of the first obstacles is abnormal.

[0090] Specifically, the present embodiment does not impose any specific limitation on the method for determining whether the number of first obstacles satisfies the set first number change trend. For example, a straight line fitting may be performed on the number of all obstacles within the obstacle detection processing cycle. If the absolute value of the slope of the fitted straight line is greater than the set slope threshold K_thr, it is determined that the number of first obstacles is abnormal, and a corresponding warning code is reported. If the obstacle data of the previous frame is at a normal level and suddenly changes to 0 in the next frame, it is determined that the number of first obstacles is abnormal, and a corresponding warning code is reported.

[0091] In this embodiment, it can be determined whether the first obstacle quantity in the first obstacle detection result satisfies the slow change trend. If so, it is determined that the first obstacle quantity is normal; if not, it is determined that the first obstacle quantity is abnormal and a corresponding warning code is reported.

[0092] If the position of the center point of the first obstacle in the first obstacle detection result does not meet the set first center point position arrangement requirement, determining that the position of the center point of the first obstacle is abnormal;

[0093] Among them, the set first center point position arrangement requirement may refer to the pre-set distribution requirement of the obstacle center point position coordinates in space; for example, the first obstacle center point position must be within the range of the laser radar scan; the first obstacle center point position cannot overlap with the position of the vehicle, etc.

[0094] In this embodiment, it can be determined whether the pre-set first center point position arrangement requirement is met based on the first obstacle center point position in the first obstacle detection result. If the pre-set first center point position arrangement requirement is met, the first obstacle center point position is determined to be normal; if the pre-set first center point position arrangement requirement is not met, the first obstacle center point position is determined to be abnormal, and a corresponding warning code is reported.

[0095] If the first obstacle size in the first obstacle detection result does not meet the set first size requirement, determining that the first obstacle size is abnormal;

[0096] The set first size requirement may refer to the pre-set length, width and height requirements of the obstacle; for example, the first obstacle size should be within the upper and lower limits of the obstacle size, etc. For example, if the first obstacle size is (w n ,h n , l n ), the first size requirement can be set to w min ≤w n ≤w max ,h min ≤h n ≤h max ,l min ≤l n ≤l max ; where w min is the minimum width value, w max is the maximum width value, h min is the minimum height value, h max is the maximum height value, l min is the minimum length value, l ma The maximum length value.

[0097] In this embodiment, it can be determined whether the first obstacle size in the first obstacle detection result meets the preset first size requirement. If the preset first size requirement is met, it is determined that the first obstacle size is normal; if the preset first size requirement is not met, it is determined that the first obstacle size is abnormal, and a corresponding warning code is reported.

[0098] If the first obstacle orientation in the first obstacle detection result does not meet the set first orientation requirement, it is determined that the first obstacle orientation is abnormal.

[0099] Among them, the first orientation requirement may refer to the first orientation threshold; specifically, under normal circumstances, the orientation of general obstacles in the vehicle driving scene is relatively stable, especially in non-intersection areas. For motor vehicles in the current lane, their driving directions are fixed. The variance of the orientations of all motor vehicle obstacles in the current frame can be counted. If it is greater than the first orientation threshold Y_thr in a normal scenario, a warning code is reported.

[0100] In this embodiment, it can be determined whether the first obstacle orientation in the first obstacle detection result meets the pre-set first orientation requirement. If the pre-set first orientation requirement is met, the first obstacle orientation is determined to be normal; if the pre-set first orientation requirement is not met, the first obstacle orientation is determined to be abnormal, and a corresponding warning code is reported.

[0101] The determination result of whether the number of the first obstacles, the position of the center point of the first obstacle, the size of the first obstacle, and the direction of the first obstacle are abnormal is determined as the first abnormality analysis result.

[0102] In this embodiment, through a pre-set obstacle perception abnormality judgment scheme, one or more of the first obstacle quantity, the first obstacle center point position, the first obstacle size and the first obstacle direction in the first obstacle detection result are judged to determine the first abnormality analysis result. If any one or more of the first obstacle quantity, the first obstacle center point position, the first obstacle size and the first obstacle direction has an abnormal judgment result within a period, a corresponding warning code is reported. According to the level corresponding to the warning code, the vehicle is controlled to park safely on the side of the road, and the environmental information, log information and full data within a set time before and after the obstacle perception abnormality are saved; this not only ensures the safety of the vehicle, but also facilitates subsequent staff to analyze the obstacle perception abnormality problem based on information related to the obstacle perception abnormality, saving human resources and reducing the pressure on staff.

[0103] Optionally, if the position of the center point of the first obstacle in the first obstacle detection result does not meet the set first center point position arrangement requirement, determining that the position of the center point of the first obstacle is abnormal includes:

[0104] If the position of the center point of the first obstacle in the first obstacle detection result is not within the first detection range, determining that the position of the center point of the first obstacle is abnormal;

[0105] Among them, the first detection range can refer to the scanning range of the laser radar on the vehicle; specifically, the scanning range corresponding to each laser radar is fixed, and there will be no targets outside the scanning range. If an obstacle outside the radar scanning range appears, it means that the perception algorithm has an abnormality, and a warning code is reported.

[0106] In this embodiment, if the center point position of the first obstacle in the first obstacle detection result is not within the first detection range, it means that an abnormality has occurred in the perception algorithm. It can be determined that the center point position of the first obstacle is abnormal, and a corresponding warning code is reported.

[0107] If a first variance determined based on the first obstacle center point position in the first obstacle detection result is less than a set first variance threshold, determining that the first obstacle center point position is abnormal;

[0108] The first variance may refer to the variance of all first obstacle center point positions calculated in each frame within the period, that is, each frame within the period corresponds to a first variance. The set first variance threshold may refer to the variance of the obstacle center in a traffic jam scenario.

[0109] Specifically, the variance in this embodiment is an indicator to measure the density of obstacle center points. In real scenes, the positions of obstacle center points must be scattered; Figure 4 A schematic diagram of the center point position of an obstacle provided in the second embodiment of the present invention, such as Figure 4 As shown in the traffic jam scene on the left, even if there is a traffic jam, the obstacles are arranged one after another, and the distribution of their center points is also at a certain distance. The variance of the obstacle center in this scene can be calculated as the first variance threshold C_thr. Determine whether there is a situation where all the first variances in the period are less than C_thr. If the first variance is less than C_thr in the period, it means that the distribution of obstacles is very concentrated, and there may be abnormal detection results, such as Figure 4 If the algorithm is abnormal as shown on the right side of the figure, the corresponding warning code will be reported.

[0110] In this embodiment, if the first variance determined based on the position of the first obstacle center point in the first obstacle detection result is less than the set first variance threshold, it means that an abnormality has occurred in the perception algorithm, and it can be determined that the position of the first obstacle center point is abnormal, and a corresponding warning code is reported.

[0111] The first obstacle envelope is determined based on the first obstacle center point position, the first obstacle size, and the first obstacle direction in the first obstacle detection result. If the first obstacle envelope in any frame within the period has an intersection-over-union (IOU) with the vehicle envelope, it is determined that the first obstacle center point position is abnormal.

[0112] Among them, the obstacle envelope can refer to the boundary area of ​​the obstacle, which can be calculated based on the position of the obstacle center point, the obstacle size and the obstacle direction; the self-vehicle envelope can automatically control the boundary area of ​​the vehicle itself. Specifically, in actual situations, it is obviously impossible for obstacles to overlap with the self-vehicle, so when overlap occurs, the corresponding warning code needs to be reported in time. Specifically, the IOU (intersection over union) of the obstacle envelope and the self-vehicle envelope can be calculated to determine whether there is spatial overlap between the two. There are only two cases of IOU greater than 0 and equal to 0. When IOU is greater than 0, it means that there is spatial overlap, and when IOU is only equal to 0, it means that there is no spatial overlap; illustratively, all obstacle BOX (envelope) in each frame within the cycle can be traversed to calculate the IOU of the obstacle envelope and the self-vehicle envelope. If there is IOU, that is, IOU is not equal to 0, it means that there is spatial overlap and abnormality, and the corresponding warning code needs to be reported in time.

[0113] Specifically, the first obstacle envelope can be calculated according to the first obstacle center point position, the first obstacle size and the first obstacle orientation in the first obstacle detection result, and then the intersection-and-union ratio (IOU) between the first obstacle envelope and the ego vehicle envelope is calculated. If the first obstacle envelope and the ego vehicle envelope in any frame within the period have an intersection-and-union ratio (IOU), that is, the calculated IOU is not equal to 0, it is determined that the first obstacle center point position is abnormal, and a corresponding warning code is reported.

[0114] In this embodiment, the position of the first obstacle center point in the first obstacle detection result is judged respectively by the three judgment conditions included in the pre-set first center point position arrangement requirements. If the position of the first obstacle center point does not meet any one or more judgment conditions within the period, the corresponding warning code is reported. According to the level corresponding to the warning code, the vehicle is controlled to park safely on the side of the road, and the environmental information, log information and full data within the set time before and after the obstacle perception abnormality are saved; this not only ensures the safety of the vehicle, but also facilitates the subsequent staff to analyze the obstacle perception abnormality problem according to the relevant information of the obstacle perception abnormality, saving human resources and reducing the pressure on the staff.

[0115] Optionally, if the first obstacle size in the first obstacle detection result does not meet a set first size requirement, determining that the first obstacle size is abnormal includes:

[0116] If the first obstacle size in the first obstacle detection result is not within the set first size range, determining that the first obstacle size is abnormal;

[0117] The set first size range may refer to a preset maximum value and a minimum value of the length, width and height of the obstacle.

[0118] Specifically, the upper and lower limits of the obstacle size are fixed. For example, it is impossible for an obstacle to be 100 meters wide. Specifically, the upper and lower boundaries (i.e., maximum and minimum values) of the obstacle size can be judged to determine whether the first obstacle size is within the set first size range. If an obstacle outside the range appears, a warning code is reported.

[0119] In this embodiment, if the first obstacle size in the first obstacle detection result is not within the set first size range, it means that an obstacle outside the range has appeared, and it is determined that the first obstacle size is abnormal, and a corresponding warning code is reported.

[0120] If the first obstacle size in the first obstacle detection result does not meet the set first size ratio rule, it is determined that the first obstacle size is abnormal.

[0121] The first size ratio rule may refer to a preset ratio rule of length, width and height of an obstacle. Specifically, the ratio of length, width and height of the obstacle must meet the rule. Since the obstacle detection objects based on deep learning are all regular targets, it is impossible for an obstacle with a length of 4m and a width of 0.5m to appear in the detection result. If an obstacle with an abnormal length-width ratio appears, a corresponding warning code may be reported.

[0122] In this embodiment, if the first obstacle size in the first obstacle detection result does not meet the set first size ratio rule, it means that the length, width and height ratio of the first obstacle is abnormal, and the first obstacle size is determined to be abnormal, and a corresponding warning code is reported.

[0123] Optionally, the step of determining a second abnormality analysis result relative to the second obstacle detection result includes:

[0124] Specifically, the obstacle segmentation algorithm based on geometric rules performs obstacle detection processing on the point cloud data obtained by laser radar scanning to obtain the corresponding second obstacle detection result, that is, the second obstacle data list obj_list_seg. The obstacle objects determined by the obstacle segmentation algorithm based on geometric rules are not only conventional targets, but also the output obstacle envelope is generally a convex hull. For example, the number of second obstacles in the second obstacle data list of the current frame can be set to M, and the position of the center point of the second obstacle can be set to (cx m ,cy m , cz m ), the size of the second obstacle is (w m ,h m , l m ), the second obstacle is yaw m , where n ranges from 0 to (M-1).

[0125] If the number of second obstacles in the second obstacle detection result does not satisfy the set second number change trend, determining that the number of the second obstacles is abnormal;

[0126] The set second quantity change trend may refer to a preset slow quantity change trend; specifically, the normal number of obstacles should be Figure 3 The gentle change shown in the middle left figure, if there is Figure 3 As shown in the middle right figure, it is determined that the number of the second obstacles is abnormal.

[0127] Specifically, the present embodiment does not specifically limit the judgment method for determining whether the second obstacle quantity satisfies the set second quantity change trend. For example, a straight line fitting may be performed on the number of all obstacles within the obstacle detection processing cycle. If the absolute value of the slope of the fitted straight line is greater than the set slope threshold K_thr, it is determined that the second obstacle quantity is abnormal, and a corresponding warning code is reported. If the obstacle data of the previous frame is at a normal level and suddenly changes to 0 in the next frame, it is determined that the second obstacle quantity is abnormal, and a corresponding warning code is reported.

[0128] In this embodiment, it can be determined whether the number of second obstacles in the second obstacle detection result satisfies the slow change trend. If so, it is determined that the number of second obstacles is normal; if not, it is determined that the number of second obstacles is abnormal, and a corresponding warning code is reported.

[0129] determining a second obstacle envelope based on the second obstacle center point position, the second obstacle size, and the second obstacle orientation in the second obstacle detection result, and determining that the second obstacle envelope is abnormal if an intersection-over-union (IOU) exists between the second obstacle envelopes in any frame within a period;

[0130] Specifically, in actual situations, obstacles will not contain or be contained. However, since geometric segmentation clustering constructs obstacles through the connectivity between points or grids, when there are special situations such as algorithm anomalies or abnormal adhesion of point cloud data, obstacles and obstacle envelopes may overlap. In each frame of the cycle The IOU (intersection over union) between obstacle envelopes is used to determine whether there is spatial overlap between obstacles. If there is IOU between obstacles in a single frame, that is, IOU is not equal to 0, it means that there is an abnormality in the perception algorithm or point cloud data, and the corresponding warning code needs to be reported in time.

[0131] In this embodiment, the second obstacle envelope can be determined according to the second obstacle center point position, the second obstacle size and the second obstacle orientation in the second obstacle detection result. If there is an intersection-over-union (IOU) between the second obstacle envelopes in any frame within the period, it means that there is an abnormality in the perception algorithm or point cloud data, and a corresponding warning code is reported.

[0132] determining a second obstacle area based on the second obstacle envelope, and determining that the second obstacle area is abnormal if the second obstacle area is greater than a set second area threshold and the second obstacle height is less than a set second height threshold;

[0133] Among them, the area of ​​the second obstacle may refer to the area of ​​the second obstacle on the XY plane; specifically, the traditional geometric segmentation algorithm is heavily dependent on the calibration accuracy of the point cloud. In a point cloud perception system with multiple radars stitched together, if the calibration accuracy between different radars is very different, or the calibration accuracy of some radars is poor, the ground point cloud will be divided into two or even multiple layers, and the result of ground segmentation will be greatly affected; and ground detection generally adopts plane fitting or line fitting, and it is difficult for multi-layer point clouds to handle similar situations, which in turn causes large-scale ground false detections.

[0134] Specifically, Figure 5 A schematic diagram of obstacle point cloud distribution provided in the second embodiment of the present invention, Figure 5 The side view of the ground point cloud when the calibration is abnormal on the left side is the original ground point cloud distribution. Figure 5 The red plane on the right side is the ground plane fitted according to the ground points. The points below are ground points, and the points above are non-ground points, i.e., obstacle points. In this case, there will be large-scale misdetected targets, and they are generally shorter in height, which is obviously different from large obstacles. Based on this, the area of ​​the obstacle envelope in the XY plane can be calculated, and the area threshold is set to S_thr and the height threshold is set to H_thr. If the calculated area of ​​the second obstacle envelope in the XY plane is greater than the set second area threshold S_thr and the height of the second obstacle is less than the set second height threshold H_thr, it is considered that a related misdetection has occurred, and a warning code is reported.

[0135] In this embodiment, the area of ​​the second obstacle on the XY plane can be determined according to the second obstacle envelope. If the area of ​​the second obstacle is greater than the set second area threshold S_thr and the height of the second obstacle is less than the set second height threshold H_thr, it is determined that the area of ​​the second obstacle is abnormal, and a corresponding warning code is reported.

[0136] A determination result of whether the second obstacle quantity, the second obstacle envelope, and the second obstacle area are abnormal is determined as the second abnormality analysis result.

[0137] In this embodiment, through a pre-set obstacle perception abnormality judgment scheme, one or more of the second obstacle number, the second obstacle envelope and the second obstacle area are judged to determine the second abnormality analysis result. If any one or more of the second obstacle number, the second obstacle envelope and the second obstacle area show an abnormal judgment result within the period, the corresponding warning code is reported. According to the level corresponding to the warning code, the vehicle is controlled to park safely on the side of the road, and the environmental information, log information and full data within a set time before and after the obstacle perception abnormality are saved. This not only ensures the safety of the vehicle, but also facilitates subsequent staff to analyze the obstacle perception abnormality problem based on information related to the obstacle perception abnormality, saving human resources and reducing the pressure on staff.

[0138] For example, Figure 6 This is a specific flow chart of an obstacle sensing abnormality monitoring method provided by Embodiment 2 of the present invention. Figure 6 As shown, by performing obstacle detection processing on the point cloud data obtained by laser radar scanning, determining the obstacle perception abnormal monitoring result, and executing the corresponding emergency processing operation process can be:

[0139] 1. Point cloud data obtained by laser radar scanning.

[0140] 2. The target detection algorithm based on deep learning and the obstacle segmentation algorithm based on geometric clustering respectively perform perception algorithm processing on the obtained point cloud data to obtain the first obstacle detection result obj_list_det processed by the target detection algorithm based on deep learning and the second obstacle detection result obj_list_seg processed by the obstacle segmentation algorithm based on geometric clustering; the obstacle detection result contains one or more obstacle information (i.e., obstacle data, including the number of obstacles, the position of the obstacle center point, the obstacle size, and the obstacle direction, etc.), and the one or more obstacle data exist in the form of a list.

[0141] 3. Through the pre-set obstacle data anomaly judgment scheme, obj_list_det and obj_list_seg are monitored separately. If any obstacle data in obj_list_det and obj_list_seg is abnormal, the corresponding warning code (i.e., obstacle perception abnormality monitoring result) can be reported.

[0142] 4. Perform emergency response operations according to the level corresponding to the relevant warning code, such as controlling the vehicle to activate the emergency system and park safely on the side of the road, and save environmental information, log information, and full data within a certain period of time before and after the abnormal moment.

[0143] In this embodiment, the target detection algorithm based on deep learning and the obstacle segmentation algorithm based on geometric clustering respectively perform perception algorithm processing on the point cloud data obtained by laser radar scanning to obtain a first obstacle detection result after being processed by the target detection algorithm based on deep learning and a second obstacle detection result after being processed by the obstacle segmentation algorithm based on geometric clustering; through a pre-set obstacle data abnormality judgment scheme, the first obstacle detection result and the second obstacle detection result are respectively monitored to determine the obstacle perception abnormality monitoring result; through the level corresponding to the obstacle perception abnormality monitoring result, the vehicle is controlled to park safely on the side of the road, and the environmental information, log information and full data within a set time before and after the obstacle perception abnormality are saved; this solves the problems in the prior art. The problem of being unable to detect abnormalities in obstacle perception in time and thus unable to obtain relevant information in time when the abnormality occurs is solved. The timely monitoring of obstacle perception abnormalities is realized, and the vehicle can be controlled to park safely on the side of the road in time according to the obstacle perception abnormality monitoring results, and the environmental information, log information and full data within a set time before and after the obstacle perception abnormality are saved. This not only ensures the safety of the vehicle, but also makes it convenient for subsequent staff to analyze the obstacle perception abnormality problem according to the environmental information, log information and full data within a set time before and after the obstacle perception abnormality, saving human resources and reducing the pressure on the staff. In addition, only by discovering the problem can the problem be better solved. Therefore, the technical solution of the present invention is also of great significance to radar perception and even the entire field of intelligent driving.

[0144] Embodiment 3

[0145] Figure 7 This is a schematic diagram of the structure of an obstacle sensing abnormality monitoring device provided in the third embodiment of the present invention. Figure 7 As shown, the device comprises:

[0146] The detection processing module 310 is used to perform obstacle detection processing on the point cloud data obtained by the laser radar scanning according to the target detection mode and the obstacle segmentation mode, and obtain the corresponding first obstacle detection result and second obstacle detection result;

[0147] A first determination module 320, configured to determine a first abnormality analysis result corresponding to the first obstacle detection result and a second abnormality analysis result corresponding to the second obstacle detection result, respectively, according to one or more of the number of obstacles, the position of the obstacle center point, the size of the obstacle, and the direction of the obstacle in the first obstacle detection result and the second obstacle detection result;

[0148] The second determination module 330 is used to determine the obstacle perception abnormality monitoring result according to the first abnormality analysis result and the second abnormality analysis result.

[0149] Optionally, the obstacle sensing abnormality monitoring device further includes:

[0150] A grading module, used to determine the level corresponding to the obstacle perception abnormality monitoring result by grading the obstacle perception abnormality monitoring result;

[0151] The execution module is used to perform corresponding emergency processing operations according to the level corresponding to the obstacle perception abnormality monitoring result, wherein the emergency processing operation is one or more of the following: safe parking of the vehicle by the side of the road, saving environmental information, log information and full data within a set time before and after the obstacle perception abnormality.

[0152] Optionally, the first determining module 320 includes:

[0153] a first obstacle quantity determination unit, configured to determine that the first obstacle quantity is abnormal if the first obstacle quantity in the first obstacle detection result does not satisfy a set first quantity change trend;

[0154] a first obstacle center point position determining unit, configured to determine that the first obstacle center point position is abnormal if the first obstacle center point position in the first obstacle detection result does not meet a set first center point position arrangement requirement;

[0155] a first obstacle size determining unit, configured to determine that the first obstacle size is abnormal if the first obstacle size in the first obstacle detection result does not meet a set first size requirement;

[0156] The first obstacle orientation determining unit is configured to determine that the first obstacle orientation is abnormal if the first obstacle orientation in the first obstacle detection result does not meet a set first orientation requirement.

[0157] The first abnormal analysis result determining unit is used to determine the result of determining whether the number of the first obstacles, the position of the center point of the first obstacle, the size of the first obstacle, and the direction of the first obstacle are abnormal as the first abnormal analysis result.

[0158] Optionally, the first obstacle center point position determination unit is specifically configured to:

[0159] If the position of the center point of the first obstacle in the first obstacle detection result is not within the first detection range, determining that the position of the center point of the first obstacle is abnormal;

[0160] If a first variance determined based on the first obstacle center point position in the first obstacle detection result is less than a set first variance threshold, determining that the first obstacle center point position is abnormal;

[0161] The first obstacle envelope is determined based on the first obstacle center point position, the first obstacle size, and the first obstacle direction in the first obstacle detection result. If the first obstacle envelope in any frame within the period has an intersection-over-union (IOU) with the vehicle envelope, it is determined that the first obstacle center point position is abnormal.

[0162] Optionally, the first obstacle size determining unit is specifically configured to:

[0163] If the first obstacle size in the first obstacle detection result is not within the set first size range, determining that the first obstacle size is abnormal;

[0164] If the first obstacle size in the first obstacle detection result does not meet the set first size ratio rule, it is determined that the first obstacle size is abnormal.

[0165] Optionally, the first determining module 320 further includes:

[0166] a second obstacle quantity determination unit, configured to determine that the second obstacle quantity is abnormal if the second obstacle quantity in the second obstacle detection result does not satisfy a set second quantity change trend;

[0167] a second obstacle envelope determining unit, configured to determine a second obstacle envelope based on a second obstacle center point position, a second obstacle size, and a second obstacle orientation in the second obstacle detection result, and determine that the second obstacle envelope is abnormal if an intersection-over-union ratio (IOU) exists between the second obstacle envelopes in any frame within a period;

[0168] a second obstacle area determining unit, configured to determine a second obstacle area based on the second obstacle envelope, and determine that the second obstacle area is abnormal if the second obstacle area is greater than a set second area threshold and the second obstacle height is less than a set second height threshold;

[0169] The second abnormal analysis result determining unit is used to determine the determination result of whether the second obstacle quantity, the second obstacle envelope and the second obstacle area are abnormal as the second abnormal analysis result.

[0170] Optionally, the detection processing module 310 is specifically configured to:

[0171] According to the target detection method and obstacle segmentation method, obstacle detection processing is periodically performed on the point cloud data scanned by the laser radar at set time intervals.

[0172] The obstacle perception anomaly monitoring device provided in the embodiment of the present invention can execute the obstacle perception anomaly monitoring method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0173] Embodiment 4

[0174] Figure 8 This is a schematic diagram of the hardware structure of a vehicle provided in the fourth embodiment of the present invention. The vehicle can be a vehicle in the testing phase or a vehicle in other phases. In addition, the vehicle can be a manned vehicle or an unmanned vehicle. Figure 8 As shown, the vehicle provided in the present application includes a memory 820, a controller 810, and a computer program stored in the memory and executable on the controller. When the controller 810 executes the program, the obstacle perception abnormality monitoring method described in any of the above embodiments is implemented.

[0175] The vehicle may further include a memory 820; the controller 810 in the evaluation device may be one or more, Figure 8 A controller 810 is taken as an example; the memory 820 is used to store one or more programs; the one or more programs are executed by the one or more controllers 810, so that the one or more controllers 810 implement the obstacle perception abnormality monitoring method as described in any of the above embodiments.

[0176] The vehicle further includes an input device 830 and an output device 840 .

[0177] The controller 810, the memory 820, the input device 830 and the output device 840 in the vehicle may be connected via a bus or other means. Figure 8 The example of connecting through bus is taken in the following.

[0178] The input device 830 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the vehicle. The output device 840 may include a display device such as a display screen.

[0179] The memory 820, as a computer-readable storage medium, can be configured to store software programs, computer executable programs and modules, such as the program instructions / modules corresponding to the obstacle perception abnormality monitoring method described in the embodiment of the present application (for example, the detection processing module 310, the first determination module 320, and the first determination module 330 in the obstacle perception abnormality monitoring device). The memory 820 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application required for at least one function; the storage data area may store data created according to the use of the vehicle, etc. In addition, the memory 820 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some examples, the memory 820 may further include a memory remotely arranged relative to the controller 810, and these remote memories may be connected to the vehicle via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0180] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable controller, which can be a dedicated or general programmable controller that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0181] On the basis of the above embodiments, this embodiment further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a controller, the obstacle perception abnormality monitoring method in any of the above embodiments of the present invention is implemented.

[0182] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0183] In the context of the present invention, a computer-readable storage medium stores computer instructions, which are used to implement the obstacle perception anomaly monitoring method provided by the present invention when the controller is executed. A computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the above. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0184] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a cathode ray tube (CRT) or a liquid crystal display (LCD monitor)) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball), through which the user may provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0185] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by digital data communication (e.g., a communication network) in any form or medium. Examples of communication networks include: Local Area Network (LAN), Wide Area Network (WAN), blockchain network, and the Internet.

[0186] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and virtual private servers (VPS) services.

[0187] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0188] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for monitoring anomaly in obstacle perception, characterized in that: include: According to the target detection method and the obstacle segmentation method, obstacle detection processing is performed on the point cloud data obtained by the laser radar scanning to obtain corresponding first obstacle detection results and second obstacle detection results; Determine, according to one or more of the number of obstacles, the position of the obstacle center point, the size of the obstacle, and the direction of the obstacle in the first obstacle detection result and the second obstacle detection result, a first abnormality analysis result corresponding to the first obstacle detection result and a second abnormality analysis result corresponding to the second obstacle detection result; An obstacle perception abnormality monitoring result is determined based on the first abnormality analysis result and the second abnormality analysis result.

2. The method according to claim 1, characterized in that After determining the obstacle perception abnormality monitoring result according to the first abnormality analysis result and the second abnormality analysis result, the method further includes: Determine the level corresponding to the obstacle perception abnormality monitoring result by performing grading processing on the obstacle perception abnormality monitoring result; According to the level corresponding to the obstacle perception abnormality monitoring result, the corresponding emergency handling operation is performed, wherein the emergency handling operation is one or more of the following: the vehicle is safely parked on the side of the road, and the environmental information, log information and full data within a set time before and after the obstacle perception abnormality are saved.

3. The method according to claim 1, characterized in that The step of determining a first abnormality analysis result relative to the first obstacle detection result comprises: If the first obstacle quantity in the first obstacle detection result does not satisfy a set first quantity change trend, determining that the first obstacle quantity is abnormal; If the position of the center point of the first obstacle in the first obstacle detection result does not meet the set first center point position arrangement requirement, determining that the position of the center point of the first obstacle is abnormal; If the first obstacle size in the first obstacle detection result does not meet the set first size requirement, determining that the first obstacle size is abnormal; If the first obstacle orientation in the first obstacle detection result does not meet the set first orientation requirement, it is determined that the first obstacle orientation is abnormal. The determination result of whether the number of the first obstacles, the position of the center point of the first obstacle, the size of the first obstacle, and the direction of the first obstacle are abnormal is determined as the first abnormality analysis result.

4. The method according to claim 3, characterized in that If the position of the center point of the first obstacle in the first obstacle detection result does not meet the set first center point position arrangement requirement, determining that the position of the center point of the first obstacle is abnormal includes: If the position of the center point of the first obstacle in the first obstacle detection result is not within the first detection range, determining that the position of the center point of the first obstacle is abnormal; If a first variance determined based on the first obstacle center point position in the first obstacle detection result is less than a set first variance threshold, determining that the first obstacle center point position is abnormal; The first obstacle envelope is determined based on the first obstacle center point position, the first obstacle size, and the first obstacle direction in the first obstacle detection result. If the first obstacle envelope in any frame within the period has an intersection-over-union (IOU) with the vehicle envelope, it is determined that the first obstacle center point position is abnormal.

5. The method according to claim 3, characterized in that: If the first obstacle size in the first obstacle detection result does not meet the set first size requirement, determining that the first obstacle size is abnormal includes: If the first obstacle size in the first obstacle detection result is not within the set first size range, determining that the first obstacle size is abnormal; If the first obstacle size in the first obstacle detection result does not meet the set first size ratio rule, it is determined that the first obstacle size is abnormal.

6. The method according to claim 1, characterized in that The step of determining a second abnormality analysis result relative to the second obstacle detection result comprises: If the number of second obstacles in the second obstacle detection result does not satisfy the set second number change trend, determining that the number of the second obstacles is abnormal; determining a second obstacle envelope based on the second obstacle center point position, the second obstacle size, and the second obstacle orientation in the second obstacle detection result, and determining that the second obstacle envelope is abnormal if an intersection-over-union (IOU) exists between the second obstacle envelopes in any frame within a period; determining a second obstacle area based on the second obstacle envelope, and determining that the second obstacle area is abnormal if the second obstacle area is greater than a set second area threshold and the second obstacle height is less than a set second height threshold; A determination result of whether the second obstacle quantity, the second obstacle envelope, and the second obstacle area are abnormal is determined as the second abnormality analysis result.

7. The method according to any one of claims 1 to 6, characterized in that: According to the target detection method and obstacle segmentation method, obstacle detection processing is performed on the point cloud data obtained by laser radar scanning, including: According to the target detection method and obstacle segmentation method, obstacle detection processing is performed periodically on the point cloud data obtained by laser radar scanning at set time intervals.

8. An obstacle sensing abnormality monitoring device, characterized in that: include: A detection processing module, used to perform obstacle detection processing on the point cloud data obtained by laser radar scanning according to the target detection mode and the obstacle segmentation mode, and obtain corresponding first obstacle detection results and second obstacle detection results; a first determination module, configured to determine, according to one or more of the number of obstacles, the position of the obstacle center point, the size of the obstacle, and the direction of the obstacle in the first obstacle detection result and the second obstacle detection result, a first abnormality analysis result corresponding to the first obstacle detection result and a second abnormality analysis result corresponding to the second obstacle detection result; The second determination module is used to determine the obstacle perception abnormality monitoring result according to the first abnormality analysis result and the second abnormality analysis result.

9. A vehicle, characterized in that: include: at least one controller; as well as A memory in communication with the at least one controller; wherein, The memory stores a computer program executable by the at least one controller, and the computer program is executed by the at least one controller so that the at least one controller can execute the obstacle perception abnormality monitoring method described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a controller to implement the obstacle perception abnormality monitoring method described in any one of claims 1 to 7 when executed.