Scene diagnosis method and device based on automatic driving, vehicle and storage medium

Abstract

The invention relates to a scene diagnosis method and device based on automatic driving, a vehicle and a storage medium, and the method comprises the steps: obtaining N detection results corresponding to N sensors, N being an integer greater than 1; carrying out association fusion on the N detection results to obtain a fusion result; and performing scene diagnosis according to the N detection results and the fusion result to obtain a scene diagnosis result. In this way, scene diagnosis can be carried out by comprehensively utilizing the N detection results corresponding to the N sensors and the fusion result obtained after correlation fusion of the N detection results, so that whether data of each sensor is transmitted or not can be diagnosed; and the detection result abnormity corresponding to each sensor and the association fusion abnormity among the plurality of sensors can be diagnosed, so that the precision of the sensing result is greatly improved.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and specifically to a scene diagnosis method, device, vehicle and storage medium based on autonomous driving. Background Art

[0002] With the continuous development of autonomous driving technology, autonomous vehicles are becoming more and more popular in people's daily lives. As the eyes of autonomous vehicles, perception technology needs to accurately detect targets and passable areas around the vehicle to ensure the safe driving of the vehicle. Therefore, it is particularly important to perform scene diagnosis on the transmission data of each sensor to ensure that the transmission data of each sensor meets the system requirements.

[0003] However, the existing method of scene diagnosis of the transmission data of each sensor usually only diagnoses whether the data of each sensor is transmitted, and does not diagnose the abnormality of each sensor data and the abnormality of multiple sensor matching. Due to the different advantages and disadvantages of each sensor, if only the diagnosis of whether the data of each sensor is transmitted is performed, the accuracy of the final perception result cannot be guaranteed, which greatly reduces the accuracy of the perception result. Therefore, how to improve the accuracy of the perception result by performing scene diagnosis on the transmission data of each sensor has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] One of the purposes of this application is to provide a scene diagnosis method based on autonomous driving, so as to solve the problem that the method of performing scene diagnosis on the transmission data of each sensor in the prior art cannot guarantee the accuracy of the final perception result, thereby greatly reducing the accuracy of the perception result; the second purpose is to provide a scene diagnosis device based on autonomous driving; the third purpose is to provide a vehicle; and the fourth purpose is to provide a computer-readable storage medium.

[0005] In order to achieve the above purpose, the technical solution adopted in this application is as follows:

[0006] A scene diagnosis method based on autonomous driving, the method comprising:

[0007] Obtain N detection results corresponding to N sensors, where N is an integer greater than 1;

[0008] Associating and fusing the N detection results to obtain a fusion result;

[0009] A scene diagnosis is performed according to the N detection results and the fusion result to obtain a scene diagnosis result.

[0010] According to the above technical means, the N detection results corresponding to the N sensors and the fusion results after the association and fusion of the N detection results can be comprehensively utilized to perform scene diagnosis. Therefore, it is possible to diagnose not only whether the data of each sensor is transmitted, but also the abnormalities of the detection results corresponding to each sensor and the abnormalities of the association and fusion between multiple sensors. This can greatly improve the accuracy of the perception results.

[0011] Further, the N detection results include N target detection results and N passable area detection results corresponding to the N sensors;

[0012] The associating and fusing the N detection results to obtain a fusion result includes:

[0013] Performing time-space synchronization on the N target detection results and the N passable area detection results respectively;

[0014] The N target detection results and the N passable area detection results after time and space synchronization are respectively associated and fused to obtain the fusion result, wherein the fusion result includes the target fusion results and the passable area fusion results corresponding to the N sensors.

[0015] According to the above-mentioned technical means, since the detection results corresponding to each sensor include target detection results and passable area detection results, the target detection results and passable area detection results of each sensor, as well as the fused target fusion results and passable area fusion results can be diagnosed to further improve the accuracy of the perception results.

[0016] Further, performing scene diagnosis according to the N detection results and the fusion result to obtain a scene diagnosis result includes:

[0017] Determine whether the N target detection results, the N passable area detection results, the target fusion result, and the passable area fusion result meet the requirement data corresponding to the target diagnostic scenario, wherein the target diagnostic scenario is any diagnostic scenario among a plurality of pre-set diagnostic scenarios, and the diagnostic scenario corresponds to the requirement data one by one;

[0018] When the N target detection results, the N passable area detection results, the target fusion result, and the passable area fusion result meet the demand data corresponding to the target diagnosis scenario, matching the demand data corresponding to the target diagnosis scenario with the preset conditions corresponding to the target diagnosis scenario;

[0019] In the case where the requirement data corresponding to the target diagnosis scenario successfully matches the preset condition corresponding to the target diagnosis scenario, determining that the target diagnosis scenario has an abnormality;

[0020] In a case where the requirement data corresponding to the target diagnosis scenario fails to match the preset condition corresponding to the target diagnosis scenario, it is determined that there is no abnormality in the target diagnosis scenario.

[0021] According to the above technical means, corresponding demand data and preset conditions can be pre-configured for each diagnostic scenario. When performing scenario diagnosis, judgment is made based on the pre-configured demand data and preset conditions, thereby improving the accuracy and flexibility of the scenario diagnosis results.

[0022] Furthermore, the anomalies existing in the target diagnosis scenario include association matching anomalies, single target detection anomalies, fusion target anomalies and passable area detection anomalies;

[0023] When the requirement data corresponding to the target diagnosis scenario successfully matches the preset condition corresponding to the target diagnosis scenario, determining that the target diagnosis scenario is abnormal includes:

[0024] In the case where the N target detection results and the target fusion result successfully match the preset conditions corresponding to the target diagnosis scene, determining that the target diagnosis scene has the associated matching anomaly, wherein the preset condition is that the target fusion result does not associate and match at least one target detection result among the N target detection results; or

[0025] In the case where the N target detection results successfully match the preset conditions corresponding to the target diagnosis scene, it is determined that the single target detection anomaly exists in the target diagnosis scene, wherein the preset condition is that the target attribute of at least one target detection result among the N target detection results does not meet the preset range, and the target attribute includes at least one of confidence, detection range, speed difference, acceleration difference, and position difference; or,

[0026] In the case where the target fusion result successfully matches the preset condition corresponding to the target diagnosis scene, it is determined that the target diagnosis scene has the fusion target abnormality, wherein the preset condition is that the fusion target attribute in the target fusion result does not meet the preset range, and the fusion target attribute includes at least one of the displacement of the stationary fusion target and the acceleration of the running fusion target; or

[0027] When the N passable area detection results and the passable area fusion result successfully match the preset conditions corresponding to the target diagnostic scene, it is determined that there is a passable area detection anomaly in the target diagnostic scene, wherein the preset condition is that the number of point clouds in the passable area does not meet a preset range.

[0028] According to the above technical means, it is possible to diagnose association matching anomalies, single target detection anomalies, fusion target anomalies and passable area detection anomalies, which expands the diagnosis scope and is conducive to improving the accuracy of perception results.

[0029] Further, the N sensors include visual sensors and radar sensors, and the N target detection results include visual target detection results and radar target detection results;

[0030] Wherein, the association matching anomaly includes at least one of the association matching anomaly of the radar target detection result and the association matching anomaly of the visual target detection result;

[0031] The single target detection anomaly includes at least one of the following: the confidence of the visual target detection result is lower than a first preset threshold, the visual target detection result is empty, the visual sensor fails to detect the visual target within a preset detection range, the radar sensor fails to detect the radar target within a preset detection range, the speed difference between the visual target detection result and the radar target detection result is greater than a second preset threshold, the acceleration difference between the visual target detection result and the radar target detection result is greater than a third preset threshold, and the position difference between the visual target detection result and the radar target detection result is greater than a fourth preset threshold;

[0032] The fusion target abnormality includes at least one of the following: the displacement of the stationary fusion target in the target fusion result is greater than a fifth preset threshold value and the acceleration of the running fusion target in the target fusion result exceeds a first preset range;

[0033] The passable area detection anomaly includes at least one of the number of point clouds of the passable area in the N passable area detection results being less than a sixth preset threshold and the number of point clouds of the passable area in the passable area fusion result being less than the sixth preset threshold.

[0034] According to the above technical means, abnormal diagnosis in different diagnostic scenarios can be achieved for visual sensors and radar sensors, which is conducive to improving the accuracy of perception results.

[0035] Further, after determining that the target diagnostic scenario is abnormal, the method further includes:

[0036] Report the fault code corresponding to the target diagnostic scenario.

[0037] According to the above technical means, the user can be reminded in time to deal with the fault by reporting the fault code, and the vehicle can be reminded in time of the existence of perception abnormalities.

[0038] A scene diagnosis device based on automatic driving, the device comprising:

[0039] An acquisition module, used to acquire N detection results corresponding to N sensors, where N is an integer greater than 1;

[0040] An association fusion module, used for associating and fusing the N detection results to obtain a fusion result;

[0041] The scene diagnosis module is used to perform scene diagnosis according to the N detection results and the fusion result to obtain a scene diagnosis result.

[0042] A vehicle comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0043] Memory, used to store computer programs;

[0044] The processor is used to implement the steps of the above-mentioned scene diagnosis method based on autonomous driving when executing the program stored in the memory.

[0045] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned scene diagnosis method based on autonomous driving.

[0046] Beneficial effects of this application:

[0047] (1) The present application can comprehensively utilize N detection results corresponding to N sensors and the fusion results after the N detection results are associated and fused to perform scene diagnosis. Therefore, it can not only diagnose whether the data of each sensor is transmitted, but also diagnose the abnormalities of the detection results corresponding to each sensor and the abnormalities of the associated fusion between multiple sensors, thereby greatly improving the accuracy of the perception results.

[0048] (2) Since the detection results corresponding to each sensor in the present application include target detection results and passable area detection results, it is possible to diagnose the target detection results and passable area detection results of each sensor, as well as the fused target fusion results and passable area fusion results, thereby further improving the accuracy of the perception results.

[0049] (3) The present application can realize the diagnosis of association matching anomalies, single target detection anomalies, fusion target anomalies and passable area detection anomalies, which expands the diagnosis scope and is conducive to improving the accuracy of perception results. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A schematic diagram of a flow chart of a scene diagnosis method based on autonomous driving provided in an embodiment of the present application;

[0051] Figure 2A schematic diagram of the structure of a multi-sensor scene diagnosis architecture provided in an embodiment of the present application;

[0052] Figure 3 A flowchart of another scene diagnosis method based on autonomous driving provided in an embodiment of the present application;

[0053] Figure 4 A schematic diagram of a processing flow of a scene diagnosis module provided in an embodiment of the present application;

[0054] Figure 5 A schematic diagram of the structure of a scene diagnosis device based on autonomous driving provided in an embodiment of the present application;

[0055] Figure 6 A schematic diagram of the structure of a vehicle provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] The following will describe the implementation methods of the present application with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for illustrating the present application, not for limiting the scope of protection of the present application.

[0057] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application, and thus the drawings only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed at will, and the component layout may also be more complicated.

[0058] See also Figure 1 , Figure 1 The following is a flow chart of a scene diagnosis method based on autonomous driving provided in an embodiment of the present application. Figure 1 As shown, the scene diagnosis method based on autonomous driving may include the following steps:

[0059] Step 101: Obtain N detection results corresponding to N sensors, where N is an integer greater than 1.

[0060] Specifically, the N sensors may include visual sensors (such as forward-looking cameras, circumferential cameras, and surround-view cameras, etc.) and radar sensors (such as forward millimeter-wave radars, forward angle radars, rearward angle radars, and forward laser radars, etc.). The N detection results may include, but are not limited to, N target detection results corresponding to the N sensors and N traversable area detection results.

[0061] Step 102: associate and fuse the N detection results to obtain a fusion result.

[0062] After obtaining N detection results, the N detection results can be synchronized in time and space, and then the time-synchronized and space-synchronized target data can be associated using the Mahalanobis distance and Hungarian matching algorithm, and the time-synchronized and space-synchronized passable area data can be transparently transmitted to achieve data fusion of each sensor.

[0063] Step 103: Perform scene diagnosis based on the N detection results and the fusion results to obtain a scene diagnosis result.

[0064] In this step, the user can pre-set multiple diagnostic scenarios, each of which has a corresponding requirement data, a preset condition and a fault code. If the N detection results and fusion results obtained meet the requirement data of a diagnostic scenario, the diagnostic process of the diagnostic scenario can be triggered, and the requirement data of the diagnostic scenario is matched with the preset conditions to determine whether the diagnostic scenario is abnormal. If the diagnostic scenario is abnormal, the fault code corresponding to the diagnostic scenario is reported.

[0065] In this embodiment, the N detection results corresponding to the N sensors and the fusion results after the N detection results are associated and fused can be comprehensively utilized to perform scene diagnosis. Therefore, it is possible to diagnose not only whether the data of each sensor is transmitted, but also the abnormalities of the detection results corresponding to each sensor and the associated fusion abnormalities between multiple sensors. This can greatly improve the accuracy of the perception results.

[0066] Further, the N detection results include N target detection results corresponding to the N sensors and N passable area detection results;

[0067] In the above step 102, N detection results are associated and fused to obtain a fusion result, including:

[0068] Synchronize N target detection results and N traversable area detection results in time and space respectively;

[0069] The N target detection results and the N passable area detection results after time and space synchronization are associated and fused respectively to obtain a fusion result, wherein the fusion result includes the target fusion results and the passable area fusion results corresponding to the N sensors.

[0070] In one embodiment, the detection results corresponding to each sensor include a target detection result and a passable area detection result. When performing multi-sensor association fusion, the N target detection results and the N passable area detection results can be synchronized in time and space respectively, and then the N target detection results and the N passable area detection results after time and space synchronization can be associated and fused respectively, thereby obtaining a target fusion result and a passable area fusion result.

[0071] In this embodiment, since the detection results corresponding to each sensor include target detection results and passable area detection results, the target detection results and passable area detection results of each sensor, as well as the fused target fusion results and passable area fusion results can be diagnosed to further improve the accuracy of the perception results.

[0072] Furthermore, the above step 103, performing scene diagnosis according to the N detection results and the fusion results to obtain a scene diagnosis result, includes:

[0073] Determine whether the N target detection results, the N passable area detection results, the target fusion results, and the passable area fusion results meet the demand data corresponding to the target diagnosis scenario, wherein the target diagnosis scenario is any diagnosis scenario among a plurality of pre-set diagnosis scenarios, and the diagnosis scenario corresponds to the demand data one by one;

[0074] When the N target detection results, the N traversable area detection results, the target fusion results, and the traversable area fusion results meet the demand data corresponding to the target diagnosis scenario, the demand data corresponding to the target diagnosis scenario is matched with the preset conditions corresponding to the target diagnosis scenario;

[0075] In the case where the demand data corresponding to the target diagnosis scenario successfully matches the preset conditions corresponding to the target diagnosis scenario, determining that an abnormality exists in the target diagnosis scenario;

[0076] In the case where the requirement data corresponding to the target diagnosis scenario fails to match the preset conditions corresponding to the target diagnosis scenario, it is determined that there is no abnormality in the target diagnosis scenario.

[0077] In one embodiment, a user may pre-set multiple diagnostic scenarios, and each diagnostic scenario corresponds to a requirement data and a preset condition. If the N detection results and fusion results obtained meet the requirement data corresponding to the target diagnostic scenario, the diagnostic process of the diagnostic scenario can be triggered, and the requirement data corresponding to the target diagnostic scenario is matched with the preset condition corresponding to the target diagnostic scenario, thereby determining whether the diagnostic scenario has an abnormality. In this way, scenario diagnosis can be performed through pre-configured requirement data and preset conditions, thereby improving the accuracy and flexibility of scenario diagnosis results.

[0078] Furthermore, the anomalies existing in the target diagnosis scenario include association matching anomalies, single target detection anomalies, fusion target anomalies, and traversable area detection anomalies;

[0079] The above steps, when the demand data corresponding to the target diagnosis scenario successfully matches the preset conditions corresponding to the target diagnosis scenario, determine that the target diagnosis scenario is abnormal, include:

[0080] In the case where the N target detection results and the target fusion result successfully match the preset conditions corresponding to the target diagnosis scene, it is determined that there is an associated matching anomaly in the target diagnosis scene, wherein the preset condition is that the target fusion result does not associate and match at least one target detection result among the N target detection results; or

[0081] In the case where N target detection results successfully match the preset conditions corresponding to the target diagnosis scene, it is determined that there is a single target detection anomaly in the target diagnosis scene, wherein the preset condition is that the target attribute of at least one target detection result among the N target detection results does not meet the preset range, and the target attribute includes at least one of the confidence level, detection range, speed difference, acceleration difference, and position difference; or,

[0082] In the case where the target fusion result successfully matches the preset condition corresponding to the target diagnosis scene, it is determined that there is a fusion target abnormality in the target diagnosis scene, wherein the preset condition is that the fusion target attribute in the target fusion result does not meet the preset range, and the fusion target attribute includes at least one of the displacement of the stationary fusion target and the acceleration of the running fusion target; or,

[0083] When the N passable area detection results and the passable area fusion results successfully match the preset conditions corresponding to the target diagnosis scene, it is determined that there is a passable area detection anomaly in the target diagnosis scene, where the preset condition is that the number of point clouds in the passable area does not meet the preset range.

[0084] In one embodiment, the anomalies existing in the above-mentioned target diagnosis scenario may include, but are not limited to, anomaly types such as association matching anomalies, single target detection anomalies, fusion target anomalies, and passable area detection anomalies. When performing scene diagnosis, different demand data and corresponding preset conditions can be compared to determine the type of anomaly that exists. Specifically, the N target detection results and target fusion results can be matched with the preset conditions corresponding to the target diagnosis scenario. If the match is successful, it can be determined that the target diagnosis scenario has an association matching anomaly, wherein the preset condition is that at least one of the N target detection results is not associated with the target fusion result. The N target detection results can be matched with the preset conditions corresponding to the target diagnosis scenario. If the match is successful, it can be determined that the target diagnosis scenario has a single target detection anomaly, wherein the preset condition is that the target attribute of at least one of the N target detection results does not meet the preset range, and the target attributes include confidence, detection range, speed difference, acceleration, etc. At least one of the degree difference and position difference; the target fusion result can be matched with the preset conditions corresponding to the target diagnosis scene. If the match is successful, it can be determined that there is a fusion target anomaly in the target diagnosis scene, wherein the preset condition is that the fusion target attribute in the target fusion result does not meet the preset range, and the fusion target attribute includes at least one of the displacement of the stationary fusion target and the acceleration of the running fusion target; N passable area detection results and the passable area fusion result can be matched with the preset conditions corresponding to the target diagnosis scene. If the match is successful, it can be determined that there is a passable area detection anomaly in the target diagnosis scene, wherein the preset condition is that the number of point clouds in the passable area does not meet the preset range.

[0085] According to the above technical means, it is possible to diagnose association matching anomalies, single target detection anomalies, fusion target anomalies and passable area detection anomalies, which expands the diagnosis scope and is conducive to improving the accuracy of perception results.

[0086] Further, the N sensors include visual sensors and radar sensors, and the N target detection results include visual target detection results and radar target detection results;

[0087] The abnormal correlation matching includes at least one of the abnormal correlation matching of radar target detection results and the abnormal correlation matching of visual target detection results;

[0088] The single target detection anomaly includes at least one of the following: the confidence of the visual target detection result is lower than a first preset threshold, the visual target detection result is empty, the visual sensor fails to detect the visual target within the preset detection range, the radar sensor fails to detect the radar target within the preset detection range, the speed difference between the visual target detection result and the radar target detection result is greater than a second preset threshold, the acceleration difference between the visual target detection result and the radar target detection result is greater than a third preset threshold, and the position difference between the visual target detection result and the radar target detection result is greater than a fourth preset threshold;

[0089] The fusion target abnormality includes at least one of the following: the displacement of the stationary fusion target in the target fusion result is greater than a fifth preset threshold value and the acceleration of the running fusion target in the target fusion result exceeds a first preset range;

[0090] The abnormality of the passable area detection includes at least one of the number of point clouds of the passable area in the N passable area detection results being less than the sixth preset threshold and the number of point clouds of the passable area in the passable area fusion results being less than the sixth preset threshold.

[0091] Specifically, the first preset threshold, the second preset threshold, the third preset threshold, the fourth preset threshold, the fifth preset threshold, the sixth preset threshold and the first preset range can be set according to actual needs and are not specifically limited here.

[0092] In one embodiment, the above-mentioned association matching anomaly includes one or more combinations of radar target detection result association matching anomaly and visual target detection result association matching anomaly, wherein the radar target detection result association matching anomaly means that the target fusion result is only associated and matched with the visual target detection result, but not associated and matched with the radar target detection result; the visual target detection result association matching anomaly means that the target fusion result is only associated and matched with the radar target detection result, but not associated and matched with the visual target detection result.

[0093] The above-mentioned single target detection anomalies include one or more of the following: the confidence level of the visual target detection result is lower than a first preset threshold, the visual target detection result is empty, the visual sensor fails to detect the visual target within the preset detection range, the radar sensor fails to detect the radar target within the preset detection range, the speed difference between the visual target detection result and the radar target detection result is greater than a second preset threshold, the acceleration difference between the visual target detection result and the radar target detection result is greater than a third preset threshold, and the position difference between the visual target detection result and the radar target detection result is greater than a fourth preset threshold. Among them, the confidence of the visual target detection result is lower than the first preset threshold value, which means that the accuracy of the visual target detection result is low; the visual target detection result is empty, which means that the visual sensor has not created a new visual target; the visual sensor fails to detect the visual target within the preset detection range, which means that the visual sensor should have detected the visual target within the preset detection range, but actually did not detect the visual target; the radar sensor fails to detect the radar target within the preset detection range, which means that the radar sensor should have detected the radar target within the preset detection range, but actually did not detect the radar target; the speed difference between the visual target detection result and the radar target detection result is greater than the second preset threshold value, which means that the radar target and the visual target can match, but the speed difference between the fusion target and the visual target is large; the acceleration difference between the visual target detection result and the radar target detection result is greater than the third preset threshold value, which means that the radar target and the visual target can match, but the acceleration difference between the fusion target and the visual target is large; the position difference between the visual target detection result and the radar target detection result is greater than the fourth preset threshold value, which means that the radar target and the visual target can match, but the position difference between the fusion target and the visual target is large.

[0094] The fusion target abnormality includes a combination of one or more of the following: the displacement of the static fusion target in the target fusion result is greater than the fifth preset threshold value and the acceleration of the running fusion target in the target fusion result exceeds the first preset range. Among them, the displacement of the static fusion target in the target fusion result is greater than the fifth preset threshold value, which means that the static fusion target has a large displacement; the acceleration of the running fusion target in the target fusion result exceeds the first preset range, which means that the running fusion target has an extremely high / low acceleration.

[0095] The abnormality of passable area detection includes one or more combinations of the number of point clouds of the passable area in the N passable area detection results being less than the sixth preset threshold and the number of point clouds of the passable area in the passable area fusion result being less than the sixth preset threshold. Among them, the number of point clouds of the passable area in the N passable area detection results being less than the sixth preset threshold means that the passable area detection of a certain sensor is inaccurate; the number of point clouds of the passable area in the passable area fusion result being less than the sixth preset threshold means that the passable area fusion result is inaccurate.

[0096] According to the above technical means, abnormal diagnosis in different diagnostic scenarios can be achieved for visual sensors and radar sensors, which is conducive to improving the accuracy of perception results.

[0097] Furthermore, after the above step of determining that the target diagnosis scenario has an abnormality, the method further includes:

[0098] Report the fault code corresponding to the target diagnostic scenario.

[0099] In one embodiment, after determining that the target diagnostic scenario has an abnormality, a fault code corresponding to the target diagnostic scenario may also be reported. By reporting the fault code, the user may be promptly reminded to handle the fault and the vehicle may be promptly reminded of the perceived abnormality.

[0100] In one embodiment, the Figure 2 The multi-sensor scene diagnosis architecture shown is used to realize the scene diagnosis of multi-sensor data. The multi-sensor scene diagnosis architecture includes multiple sensor processing modules, a multi-sensor fusion module and a scene diagnosis module. Among them, each sensor processing module is used to process the raw data collected by each sensor, and output the target and passable area corresponding to the sensor. The multi-sensor fusion module is used to take the targets and passable areas output by multiple sensor processing modules as input data, perform spatiotemporal synchronization and data association on the targets and passable areas output by multiple sensor processing modules, and output the fusion results of fused targets and passable areas. The scene diagnosis module is used to perform scene diagnosis based on the targets and passable areas output by multiple sensor processing modules, and the fusion results of fused targets and passable areas output by the multi-sensor multi-sensor fusion module, and output the scene diagnosis results.

[0101] See also Figure 3 The scene diagnosis process provided in the embodiment of the present application includes the following steps:

[0102] Step 301: A single sensor processing module receives and processes the latest single frame data, forms the target and passable area of ​​the current frame of each sensor, and outputs it to the multi-sensor fusion module and the scene diagnosis module.

[0103] In this step, the multi-threading mechanism of C++ can be used to run multiple sensor processing modules simultaneously in parallel mode. The sensors currently involved mainly include forward-looking cameras, periscopic cameras, surround-view cameras, forward millimeter-wave radars, forward angle radars, and rearward angle radars.

[0104] Step 302: Use the multi-sensor fusion module to complete the time synchronization and space synchronization of the target and traversable area data processed by each sensor, then associate the aligned target data using the Mahalanobis distance and Hungarian matching algorithm, and transparently transmit the aligned traversable area data to achieve data fusion of each sensor.

[0105] Step 303, use the fusion target and passable area output by the multi-sensor fusion module, and the fusion results of the target and passable area output by each sensor as the input of the scene diagnosis module respectively, and judge the corresponding sensor data according to the scene diagnosis requirements specified by the system. The data that does not meet the scene requirements should return the fault code of the corresponding sensor to facilitate the back-end function and perception module to perform troubleshooting. Specifically, if the corresponding requirement data for each diagnostic scenario is obtained, the corresponding scene diagnosis is activated; otherwise, the corresponding scene diagnosis is not activated. After activating the corresponding scene diagnosis, the configuration of the activated scene diagnosis can be read to complete the conditional judgment of the scene diagnosis. If the conditions are met, the corresponding fault code is reported; otherwise, no operation is performed. The processing flow of the scene diagnosis module is as follows: Figure 4 shown.

[0106] As an optional implementation, the configuration information in the scene diagnosis module is as follows:

[0107]

[0108] Through the above method, it is possible to diagnose data in scenarios such as when the number of point clouds in the traversable area of ​​single sensor and multi-sensor fusion is extremely small, when the acceleration, speed and position of multi-sensor matching exceed the threshold, and when stationary targets have large displacements, so as to detect anomalies in time and improve the accuracy of perception results.

[0109] See also Figure 5 , Figure 5 The following is a schematic diagram of a scene diagnosis device based on autonomous driving provided in an embodiment of the present application. Figure 5 As shown, the scene diagnosis device 500 based on automatic driving includes:

[0110] An acquisition module 501 is used to acquire N detection results corresponding to N sensors, where N is an integer greater than 1;

[0111] The association fusion module 502 is used to associate and fuse the N detection results to obtain a fusion result;

[0112] The scene diagnosis module 503 is used to perform scene diagnosis according to the N detection results and the fusion results to obtain a scene diagnosis result.

[0113] Further, the N detection results include N target detection results corresponding to the N sensors and N passable area detection results;

[0114] The association fusion module 502 includes:

[0115] The spatiotemporal synchronization submodule is used to perform spatiotemporal synchronization on N target detection results and N traversable area detection results respectively;

[0116] The association fusion submodule is used to associate and fuse the N target detection results and the N passable area detection results after time and space synchronization to obtain a fusion result, wherein the fusion result includes the target fusion results and the passable area fusion results corresponding to the N sensors.

[0117] Further, the scene diagnosis module 503 includes:

[0118] A judgment submodule, used to judge whether the N target detection results, the N passable area detection results, the target fusion results and the passable area fusion results meet the demand data corresponding to the target diagnosis scenario, wherein the target diagnosis scenario is any diagnosis scenario among a plurality of pre-set diagnosis scenarios, and the diagnosis scenario corresponds to the demand data one by one;

[0119] A matching submodule, for matching the demand data corresponding to the target diagnosis scenario with the preset conditions corresponding to the target diagnosis scenario when the N target detection results, the N passable area detection results, the target fusion results and the passable area fusion results meet the demand data corresponding to the target diagnosis scenario;

[0120] A first determination submodule is used to determine that an abnormality exists in the target diagnosis scenario when the demand data corresponding to the target diagnosis scenario successfully matches the preset conditions corresponding to the target diagnosis scenario;

[0121] The second determination submodule is used to determine that there is no abnormality in the target diagnosis scenario when the requirement data corresponding to the target diagnosis scenario fails to match the preset conditions corresponding to the target diagnosis scenario.

[0122] Furthermore, the anomalies existing in the target diagnosis scenario include association matching anomalies, single target detection anomalies, fusion target anomalies, and traversable area detection anomalies;

[0123] The first determination submodule includes:

[0124] A first determination unit is configured to determine that there is an associated matching anomaly in the target diagnosis scenario when the N target detection results and the target fusion result successfully match the preset conditions corresponding to the target diagnosis scenario, wherein the preset condition is that the target fusion result does not associate and match at least one target detection result among the N target detection results; or

[0125] A second determination unit is used to determine that a single target detection anomaly exists in the target diagnosis scene when the N target detection results successfully match the preset conditions corresponding to the target diagnosis scene, wherein the preset condition is that a target attribute of at least one target detection result among the N target detection results does not meet a preset range, and the target attribute includes at least one of confidence, detection range, speed difference, acceleration difference, and position difference; or,

[0126] a third determining unit, configured to determine that a fusion target abnormality exists in the target diagnosis scene when the target fusion result successfully matches a preset condition corresponding to the target diagnosis scene, wherein the preset condition is that a fusion target attribute in the target fusion result does not satisfy a preset range, and the fusion target attribute includes at least one of a displacement of a stationary fusion target and an acceleration of a running fusion target; or

[0127] The fourth determination unit is used to determine that there is a traversable area detection anomaly in the target diagnostic scene when the N traversable area detection results and the traversable area fusion results successfully match the preset conditions corresponding to the target diagnostic scene, wherein the preset condition is that the number of point clouds in the traversable area does not meet the preset range.

[0128] Further, the N sensors include visual sensors and radar sensors, and the N target detection results include visual target detection results and radar target detection results;

[0129] The abnormal correlation matching includes at least one of the abnormal correlation matching of radar target detection results and the abnormal correlation matching of visual target detection results;

[0130] The single target detection anomaly includes at least one of the following: the confidence of the visual target detection result is lower than a first preset threshold, the visual target detection result is empty, the visual sensor fails to detect the visual target within the preset detection range, the radar sensor fails to detect the radar target within the preset detection range, the speed difference between the visual target detection result and the radar target detection result is greater than a second preset threshold, the acceleration difference between the visual target detection result and the radar target detection result is greater than a third preset threshold, and the position difference between the visual target detection result and the radar target detection result is greater than a fourth preset threshold;

[0131] The fusion target abnormality includes at least one of the following: the displacement of the stationary fusion target in the target fusion result is greater than a fifth preset threshold value and the acceleration of the running fusion target in the target fusion result exceeds a first preset range;

[0132] The abnormality of the passable area detection includes at least one of the number of point clouds of the passable area in the N passable area detection results being less than the sixth preset threshold and the number of point clouds of the passable area in the passable area fusion results being less than the sixth preset threshold.

[0133] Furthermore, the scene diagnosis module 503 further includes:

[0134] The reporting submodule is used to report the fault code corresponding to the target diagnosis scenario.

[0135] It should be noted that the device 500 can implement the steps of the scene diagnosis method based on autonomous driving provided by any of the aforementioned method embodiments, and can achieve the same technical effects, which will not be described one by one here.

[0136] like Figure 6 As shown, the embodiment of the present application further provides a vehicle, including a processor 611, a communication interface 612, a memory 613 and a communication bus 614, wherein the processor 611, the communication interface 612, and the memory 613 communicate with each other through the communication bus 614.

[0137] Memory 613, used for storing computer programs;

[0138] In one embodiment of the present application, the processor 611 is used to execute the program stored in the memory 613 to implement the scene diagnosis method based on autonomous driving provided by any of the aforementioned method embodiments, including:

[0139] Obtain N detection results corresponding to N sensors, where N is an integer greater than 1;

[0140] Associate and fuse N detection results to obtain a fusion result;

[0141] The scene diagnosis is performed based on the N detection results and the fusion results to obtain the scene diagnosis result.

[0142] An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the scene diagnosis method based on autonomous driving provided in any of the aforementioned method embodiments are implemented.

[0143] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0144] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.

Claims

1. A scene diagnosis method based on autonomous driving, characterized in that: The method comprises: Obtain N detection results corresponding to N sensors, where N is an integer greater than 1; Associating and fusing the N detection results to obtain a fusion result; A scene diagnosis is performed according to the N detection results and the fusion result to obtain a scene diagnosis result.

2. The method according to claim 1, characterized in that The N detection results include N target detection results and N passable area detection results corresponding to the N sensors; The associating and fusing the N detection results to obtain a fusion result includes: Performing time-space synchronization on the N target detection results and the N passable area detection results respectively; The N target detection results and the N passable area detection results after time and space synchronization are respectively associated and fused to obtain the fusion result, wherein the fusion result includes the target fusion results and the passable area fusion results corresponding to the N sensors.

3. The method according to claim 2, characterized in that The performing scene diagnosis according to the N detection results and the fusion result to obtain a scene diagnosis result includes: Determine whether the N target detection results, the N passable area detection results, the target fusion result, and the passable area fusion result meet the requirement data corresponding to the target diagnostic scenario, wherein the target diagnostic scenario is any diagnostic scenario among a plurality of pre-set diagnostic scenarios, and the diagnostic scenario corresponds to the requirement data one by one; When the N target detection results, the N passable area detection results, the target fusion result, and the passable area fusion result meet the demand data corresponding to the target diagnosis scenario, matching the demand data corresponding to the target diagnosis scenario with the preset conditions corresponding to the target diagnosis scenario; In the case where the requirement data corresponding to the target diagnosis scenario successfully matches the preset condition corresponding to the target diagnosis scenario, determining that the target diagnosis scenario has an abnormality; In a case where the requirement data corresponding to the target diagnosis scenario fails to match the preset condition corresponding to the target diagnosis scenario, it is determined that there is no abnormality in the target diagnosis scenario.

4. The method according to claim 3, characterized in that The anomalies existing in the target diagnosis scenario include association matching anomalies, single target detection anomalies, fusion target anomalies and passable area detection anomalies; When the requirement data corresponding to the target diagnosis scenario successfully matches the preset condition corresponding to the target diagnosis scenario, determining that the target diagnosis scenario is abnormal includes: In the case where the N target detection results and the target fusion result successfully match the preset conditions corresponding to the target diagnosis scene, determining that the target diagnosis scene has the associated matching anomaly, wherein the preset condition is that the target fusion result does not associate and match at least one target detection result among the N target detection results; or In the case where the N target detection results successfully match the preset conditions corresponding to the target diagnosis scene, it is determined that the single target detection anomaly exists in the target diagnosis scene, wherein the preset condition is that the target attribute of at least one target detection result among the N target detection results does not meet the preset range, and the target attribute includes at least one of confidence, detection range, speed difference, acceleration difference, and position difference; or, In the case where the target fusion result successfully matches the preset condition corresponding to the target diagnosis scene, it is determined that the target diagnosis scene has the fusion target abnormality, wherein the preset condition is that the fusion target attribute in the target fusion result does not meet the preset range, and the fusion target attribute includes at least one of the displacement of the stationary fusion target and the acceleration of the running fusion target; or When the N passable area detection results and the passable area fusion result successfully match the preset conditions corresponding to the target diagnostic scene, it is determined that there is a passable area detection anomaly in the target diagnostic scene, wherein the preset condition is that the number of point clouds in the passable area does not meet a preset range.

5. The method according to claim 4, characterized in that The N sensors include visual sensors and radar sensors, and the N target detection results include visual target detection results and radar target detection results; Wherein, the association matching anomaly includes at least one of the association matching anomaly of the radar target detection result and the association matching anomaly of the visual target detection result; The single target detection anomaly includes at least one of the following: the confidence of the visual target detection result is lower than a first preset threshold, the visual target detection result is empty, the visual sensor fails to detect the visual target within a preset detection range, the radar sensor fails to detect the radar target within a preset detection range, the speed difference between the visual target detection result and the radar target detection result is greater than a second preset threshold, the acceleration difference between the visual target detection result and the radar target detection result is greater than a third preset threshold, and the position difference between the visual target detection result and the radar target detection result is greater than a fourth preset threshold; The fusion target abnormality includes at least one of the following: the displacement of the stationary fusion target in the target fusion result is greater than a fifth preset threshold value and the acceleration of the running fusion target in the target fusion result exceeds a first preset range; The passable area detection anomaly includes at least one of the number of point clouds of the passable area in the N passable area detection results being less than a sixth preset threshold and the number of point clouds of the passable area in the passable area fusion result being less than the sixth preset threshold.

6. The method according to claim 3, characterized in that After determining that the target diagnostic scenario is abnormal, the method further includes: Report the fault code corresponding to the target diagnostic scenario.

7. A scene diagnosis device based on automatic driving, characterized in that: The device comprises: An acquisition module, used to acquire N detection results corresponding to N sensors, where N is an integer greater than 1; An association fusion module, used for associating and fusing the N detection results to obtain a fusion result; The scene diagnosis module is used to perform scene diagnosis according to the N detection results and the fusion result to obtain a scene diagnosis result.

8. The device according to claim 7, characterized in that The N detection results include N target detection results and N passable area detection results corresponding to the N sensors; The association fusion module includes: A spatiotemporal synchronization submodule, used for performing spatiotemporal synchronization on the N target detection results and the N passable area detection results respectively; The association fusion submodule is used to associate and fuse the N target detection results and the N passable area detection results after time and space synchronization respectively to obtain the fusion result, wherein the fusion result includes the target fusion results and the passable area fusion results corresponding to the N sensors.

9. A vehicle, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor is used to implement the steps of the scene diagnosis method based on autonomous driving as described in any one of claims 1 to 6 when executing the program stored in the memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the scene diagnosis method based on autonomous driving according to any one of claims 1 to 6 are implemented.

Images