Obstacle detection method, system, electronic device and computer readable storage medium

Abstract

The application discloses an obstacle detection method and system, electronic equipment and a computer readable storage medium. The method comprises the following steps: determining a plurality of main sensors and target data corresponding to the plurality of main sensors; determining a predetermined algorithm set corresponding to the target data respectively, and determining a first algorithm corresponding to the target data from the predetermined algorithm set respectively, wherein the predetermined algorithm set comprises at least two algorithms, the at least two algorithms are arranged in a priority order, and the first algorithm is an algorithm with the first priority in the corresponding predetermined algorithm set; processing the corresponding target data by using the first algorithm to obtain corresponding first obstacle information; and determining a target obstacle of a target vehicle based on the first obstacle information corresponding to the plurality of main sensors respectively. The application solves the technical problem of low reliability of the detected obstacle when detecting the target obstacle in the related art.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computers, in particular to an obstacle detection method and system, an electronic device and a computer readable storage medium. BACKGROUND

[0002] At present, environmental perception is one of the core technologies of automatic driving of a vehicle, which mainly collects various types of environmental information by installing various types of vehicle-mounted environmental sensors (such as laser radar, millimeter wave radar, camera and ultrasonic radar, etc.) on the vehicle, aiming to obtain the information of the environment around the vehicle (such as pedestrians, vehicles, traffic signal lights, etc.) to ensure that the automatic driving decision system makes correct and reliable behavior decisions. However, in the related art, in order to improve the accuracy of the environmental perception system, the commonly used method is to continuously optimize and update the perception algorithm software, but the more complex the software algorithm is, the larger the code amount will be, which means that there are more hidden vulnerabilities and BUGs in the software algorithm. Once the hidden vulnerabilities and BUGs are triggered under certain conditions or are exploited by malicious attackers and attacks are launched, the perception ability and system robustness of the automatic driving auxiliary perception system cannot be guaranteed, therefore, how to guarantee the high reliability of the environmental perception system is a difficult problem faced by the automatic driving auxiliary perception system.

[0003] At present, no effective solution has been proposed for the above problems. SUMMARY

[0004] The embodiments of the present application provide an obstacle detection method, system, electronic device and computer readable storage medium to at least solve the technical problem of low reliability of the detected obstacle when detecting a target obstacle in the related art.

[0005] According to an aspect of the embodiments of the present application, an obstacle detection method is provided, comprising: determining a plurality of types of main sensors and target data corresponding to the plurality of types of main sensors respectively, wherein the target data is data obtained by collecting data of a predetermined range to which a target vehicle belongs by using the corresponding main sensor; determining a predetermined algorithm set corresponding to the target data respectively, and determining a first algorithm corresponding to the target data from the predetermined algorithm set respectively, wherein the predetermined algorithm set comprises at least two algorithms, the at least two algorithms are arranged in order of priority, and the first algorithm is an algorithm with the first priority in the corresponding predetermined algorithm set; processing the corresponding target data by using the first algorithm to obtain corresponding first obstacle information; and determining a target obstacle of the target vehicle based on the first obstacle information corresponding to the plurality of types of main sensors respectively.

[0006] Optionally, the determining the target obstacle of the target vehicle based on the first obstacle information corresponding to the plurality of primary sensors respectively comprises: matching the obstacles corresponding to the plurality of primary sensors based on the first obstacle information of the obstacles corresponding to the plurality of primary sensors to obtain a first matching result; and in a case where the first matching result is that the matching of the obstacles corresponding to the plurality of primary sensors is all successful, the target obstacle is determined.

[0007] Optionally, the method further comprises: in a case where the first matching result is that the matching of the obstacles corresponding to the plurality of primary sensors fails, determining first to-be-processed data in the target data, wherein the first to-be-processed data is data corresponding to obstacle information causing the matching failure in the first obstacle information; determining a second algorithm corresponding to the first to-be-processed data from the predetermined algorithm set corresponding to the first to-be-processed data, wherein the second algorithm is an algorithm with a second priority in the predetermined algorithm set corresponding to the first to-be-processed data; processing the first to-be-processed data using the second algorithm to obtain second obstacle information corresponding to the first to-be-processed data; matching the obstacles corresponding to the plurality of primary sensors based on the first obstacle information and the second obstacle information to obtain a second matching result; and in a case where the second matching result is that the matching of the obstacles corresponding to the plurality of primary sensors fails, repeatedly determining next to-be-processed data, a next algorithm corresponding to the next to-be-processed data, next obstacle information corresponding to the next to-be-processed data obtained by processing the next to-be-processed data using the next algorithm, and a next matching result until the matching result is that the matching of the obstacles corresponding to the plurality of primary sensors is all successful, and the target obstacle is determined.

[0008] Optionally, the method further comprises: in a case where there is no matching result that the matching of the obstacles corresponding to the plurality of primary sensors is all successful, determining a first backup sensor, and determining a first faulty sensor and a first available sensor from the plurality of primary sensors to replace the faulty sensor with the backup sensor.

[0009] Optionally, the processing the target data corresponding to the first algorithm to obtain the first obstacle information corresponding to the target data comprises: determining a first confidence corresponding to the first algorithm respectively; in a case that there is a confidence less than a predetermined threshold in the first confidence corresponding to the first algorithm, determining secondary available data in the target data, wherein the secondary available data is data in the target data corresponding to the first confidence corresponding to the first algorithm and less than the predetermined threshold; determining a second algorithm corresponding to the secondary available data from the predetermined algorithm set corresponding to the secondary available data respectively, wherein the second algorithm is an algorithm with a second priority in the predetermined algorithm set corresponding to the target data; and processing the secondary available data corresponding to the second algorithm to obtain the first obstacle information corresponding to the secondary available data.

[0010] Optionally, the determining the multiple types of primary sensors and the target data corresponding to the multiple types of primary sensors respectively comprises: determining the multiple types of primary sensors and a sensor characteristic corresponding to each type of primary sensor in the multiple types of primary sensors; determining a sensor quantity and a sensor position corresponding to each type of primary sensor in the multiple types of primary sensors according to the sensor characteristic; and determining the target data corresponding to the multiple types of primary sensors respectively based on the sensor quantity and the sensor position corresponding to each type of primary sensor in the multiple types of primary sensors.

[0011] Optionally, before the determining the multiple types of primary sensors, the method further comprises: determining multiple types of initial sensors and target data corresponding to the multiple types of initial sensors respectively; in a case that there is abnormal data in the target data corresponding to the multiple types of initial sensors respectively, determining a second backup sensor and a second faulty sensor and a second available sensor from the multiple types of initial sensors; and determining the multiple types of primary sensors according to the second available sensor and the second backup sensor.

[0012] According to an aspect of an embodiment of the present application, there is provided an obstacle detection system, comprising: a first determining module configured to determine a plurality of primary sensors and target data corresponding to the plurality of primary sensors respectively, wherein the target data is data obtained by collecting data of a predetermined range to which a target vehicle belongs by using the corresponding primary sensor; a second determining module configured to determine a predetermined algorithm set corresponding to the target data respectively, and determine a first algorithm corresponding to the target data from the predetermined algorithm set respectively, wherein the predetermined algorithm set comprises at least two algorithms, the at least two algorithms are arranged in a priority order, and the first algorithm is an algorithm with a first priority in the corresponding predetermined algorithm set; an obtaining module configured to process the corresponding target data by using the first algorithm to obtain first obstacle information corresponding to the target data; and a third determining module configured to determine a target obstacle of the target vehicle based on the first obstacle information corresponding to the plurality of primary sensors respectively.

[0013] According to an aspect of an embodiment of the present application, there is provided an electronic device, comprising: a processor; a memory configured to store instructions executable by the processor; and wherein the processor is configured to execute the instructions to implement any of the above obstacle detection methods.

[0014] According to an aspect of an embodiment of the present application, there is provided a computer readable storage medium, when instructions in the computer readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform any of the above obstacle detection methods.

[0015] In the embodiment of the present application, a plurality of types of main sensors are determined, and target data corresponding to the plurality of types of main sensors, wherein the target data is data obtained by collecting data of a predetermined range to which a target vehicle belongs by using the corresponding main sensor. A predetermined algorithm set corresponding to the target data is determined respectively, and a first algorithm corresponding to the target data is determined from the predetermined algorithm set, wherein the predetermined algorithm set includes at least two algorithms, the at least two algorithms are arranged in priority order, and the first algorithm is an algorithm arranged first in priority in the corresponding predetermined algorithm set. The corresponding target data is processed by using the first algorithm to obtain corresponding first obstacle information; and a target obstacle of the target vehicle is determined based on the first obstacle information corresponding to the plurality of types of main sensors respectively. Since the target data is obtained based on the corresponding data collected by the plurality of types of main sensors respectively, it is ensured that a plurality of types of target data are determined. Moreover, the predetermined algorithm set corresponding to the target data is determined respectively, which expands the richness of the algorithm and the diversity of the processed data. Moreover, the first algorithm corresponding to the target data is determined in the predetermined algorithm set, and the corresponding target data is processed by using the first algorithm, and the first algorithm is an algorithm arranged first in priority, so that when the corresponding target data is processed by using the first algorithm, the applicability and reliability of the processing can be ensured. Since the target obstacle is determined based on the first obstacle information corresponding to the plurality of types of main sensors respectively, it is ensured that the target obstacle is finally determined based on the plurality of types of data, and the reliability of the determined target obstacle is ensured, thereby solving the technical problem that the reliability of the detected obstacle is low when the target obstacle is detected in the related art. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings, which are included to provide a further understanding of the present application and are incorporated in and constitute a part of this application, illustrate embodiments of the present application and serve to explain the present application. In the drawings:

[0017] Figure 1 is a flowchart of an obstacle detection method according to an embodiment of the present application;

[0018] Figure 2 is a partial flowchart of an obstacle detection method provided by an optional embodiment of the present application;

[0019] Figure 3 is a schematic diagram in which an image processing module and a millimeter wave radar processing module provided by an optional embodiment of the present application process data by using an algorithm;

[0020] Figure 4 is a schematic diagram in which an image processing module and a millimeter wave radar processing module provided by an optional embodiment of the present application process data by using another algorithm;

[0021] Figure 5 Fig. 5 is a schematic diagram of an image processing module and a millimeter wave radar processing module according to another algorithm for processing data according to an optional embodiment of the present application;

[0022] Figure 6 Fig. 6 is a flowchart of a process for determining faults of sensors according to an optional embodiment of the present application;

[0023] Figure 7 Fig. 7 is a structural block diagram of an obstacle detection device according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the technical personnel in the art better understand the present application, the following will be combined with the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely, obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the scope of protection of the present application.

[0025] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0026] Embodiment 1

[0027] According to an embodiment of the present application, an embodiment of an obstacle detection method is provided. It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in a different order than that shown herein.

[0028] Figure 1 Fig. 8 is a flowchart of an obstacle detection method according to an embodiment of the present application, as shown in Fig. 8, the method comprises the following steps: Figure 1

[0029] Step S102, determine a plurality of main sensors, and target data corresponding to the plurality of main sensors, respectively; ​

[0030] Step S104, respectively determining a predetermined algorithm set corresponding to the target data, and respectively determining a first algorithm corresponding to the target data from the predetermined algorithm set;

[0031] Step S106, processing the corresponding target data by using the first algorithm to obtain corresponding first obstacle information;

[0032] Step S108, determining the target obstacle of the target vehicle based on the first obstacle information respectively corresponding to the plurality of types of main sensors.

[0033] Through the above steps, by determining the plurality of types of main sensors and the target data respectively corresponding to the plurality of types of main sensors, wherein the target data is data obtained by using the corresponding main sensor to collect data of a predetermined range to which the target vehicle belongs, respectively determining a predetermined algorithm set corresponding to the target data, and respectively determining a first algorithm corresponding to the target data from the predetermined algorithm set, wherein the predetermined algorithm set includes at least two algorithms, the at least two algorithms are arranged in order of priority, and the first algorithm is an algorithm with the first priority in the corresponding predetermined algorithm set. Processing the corresponding target data by using the first algorithm to obtain corresponding first obstacle information; determining the target obstacle of the target vehicle based on the first obstacle information respectively corresponding to the plurality of types of main sensors. Since the target data is obtained based on the data respectively collected by the plurality of types of main sensors, it is ensured that a plurality of types of target data are determined. Moreover, the predetermined algorithm set corresponding to the target data is respectively determined, which expands the richness of the algorithm and the diversity of the processed data, and the first algorithm corresponding to the target data is determined in the predetermined algorithm set, and the corresponding target data is processed by using the first algorithm, and the first algorithm is an algorithm with the first priority, therefore, when the corresponding target data is processed by using the first algorithm, the applicability and reliability of the processing can be ensured. Since the target obstacle is determined based on the first obstacle information respectively corresponding to the plurality of types of main sensors, it is ensured that the target obstacle is finally determined based on the plurality of types of data, and the reliability of the determined target obstacle is ensured, thereby solving the technical problem that the reliability of the detected obstacle is low when the target obstacle is detected in the related art.

[0034] As an optional embodiment, a plurality of primary sensors are determined, and target data corresponding to the plurality of primary sensors respectively, wherein the target data is data obtained by collecting data of a predetermined range to which the target vehicle belongs by using the corresponding primary sensor, and the plurality of primary sensors can include multiple categories, such as camera sensors, millimeter wave radar sensors, ultrasonic wave radar sensors, and other sensing sensors, without limitation. The actual application and scene can be set by itself. The target data is data obtained by collecting data of a predetermined range to which the target vehicle belongs by using the corresponding primary sensor. For example, in the case where the plurality of primary sensors include camera sensors, the target data corresponding to the primary sensor is image data; in the case where the plurality of primary sensors include millimeter wave radar sensors, the target data corresponding to the primary sensor is millimeter wave data, and so on. By setting the plurality of primary sensors, not only can the target obstacle be obtained based on multiple types of data, making the determined target obstacle more reliable, but even if a sensor is damaged due to a collision or attack, other normally working sensors can be used to determine the obstacle, enhancing the reliability in the process of determining the target obstacle.

[0035] As an optional embodiment, in the step of determining the plurality of primary sensors and the target data corresponding to the plurality of primary sensors respectively, the following steps can also be included: determining the plurality of primary sensors and sensor characteristics corresponding to each primary sensor in the plurality of primary sensors; determining the number and position of the sensors corresponding to each primary sensor in the plurality of primary sensors according to the sensor characteristics; and determining the target data corresponding to the plurality of primary sensors based on the number and position of the sensors corresponding to each primary sensor in the plurality of primary sensors. Different sensors have different characteristics, for example, different sensors have different ways and ranges of obtaining data in the process of obtaining data. For example, when the plurality of primary sensors include camera sensors, the sensor characteristics of the camera sensors need to be determined. According to the sensor characteristics of the camera sensors, the camera sensors need to be installed on the periphery of the vehicle, and the number of the camera sensors cannot be less than one, so as to clearly obtain image data in the environment around the vehicle and obtain image data in a plurality of angle ranges centered on the vehicle, avoid the shielding phenomenon in obtaining image data, and avoid the phenomenon that the image of the tail part of the vehicle is not collected due to the placement of a camera sensor at a certain place, such as the front of the vehicle, and thus it is difficult to determine the obstacle appearing at the tail part of the vehicle. Therefore, by determining the sensor characteristics corresponding to each primary sensor, the number and position of the sensors corresponding to each primary sensor can be determined, and the appropriate number of different types of sensors can be reasonably arranged at different positions of the vehicle, so that the target data corresponding to the plurality of primary sensors is reasonable and effective.

[0036] As an optional embodiment, a predetermined algorithm set corresponding to the target data is determined respectively, and a first algorithm corresponding to the target data is determined from the predetermined algorithm set respectively, wherein the predetermined algorithm set includes at least two algorithms, the at least two algorithms are arranged in a priority order, the first algorithm is an algorithm arranged first in priority in the corresponding predetermined algorithm set, and the determination of more than two algorithms in the predetermined algorithm set can be determined according to different target data corresponding to different main sensors. For example, when the camera sensor is included in the plurality of main sensors, the target data corresponding to the main sensor is image data. Therefore, algorithms capable of determining obstacles from images according to images can be considered, such as SMOKE (Single-Stage Monocular 3D Object Detection via Keypoint Estimation, one-stage monocular obstacle detection model) target detection algorithm, YOLOv5 (you only look once v5, target detection model only needs to browse once) target detection algorithm, SVM (Support Vector Machine, Support Vector Machine) classification and NMS (non maximum suppression, non maximum suppression) screening algorithm, etc. After the predetermined algorithm set is determined, the predetermined number of algorithms can be arranged in a priority order according to actual application and scene, and the first algorithm is an algorithm arranged first in priority in the corresponding predetermined algorithm set, that is, an algorithm with the highest comprehensive evaluation value of the most suitable current scene, accuracy, efficiency, etc. By determining the predetermined algorithm set, it can be ensured that the target data can be directly processed according to the algorithm in the predetermined algorithm set when processing the target data, and the efficiency of processing the target data can be improved. By determining the first algorithm from the predetermined algorithm set, the first algorithm is an algorithm arranged first in priority, so that the target data is processed more accurately and quickly, and the efficiency of processing the target data is ensured.

[0037] As an optional embodiment, the first algorithm is used to process the corresponding target data to obtain corresponding first obstacle information. Since the first algorithm is an algorithm arranged first in priority in the corresponding predetermined algorithm set, the target data is processed by using the first algorithm, and the corresponding first obstacle information can be better obtained.

[0038] It should be noted that the first obstacle information obtained by processing the corresponding target data by using the first algorithm is different. For example, when the multi-type primary sensor includes a camera sensor, the target data corresponding to the type of primary sensor is image data, and the first obstacle information determined includes the position, size, and type of the obstacle. When the multi-type primary sensor includes a millimeter wave radar sensor, the target data corresponding to the type of primary sensor is millimeter wave data, and the first obstacle information determined includes the angle, distance, and speed of the obstacle and the target vehicle.

[0039] As an optional embodiment, when the corresponding first obstacle information is obtained by processing the corresponding target data by using the first algorithm, the determination of the algorithm confidence can also be included, so as to determine whether the current algorithm is reliable according to the confidence, and whether the current algorithm is still used. The first confidence corresponding to the first algorithm is determined respectively. In the case that there is a confidence less than a predetermined threshold in the first confidence corresponding to the first algorithm, it means that there is an unreliable first algorithm, and therefore, the secondary available data needs to be determined from the target data, wherein the secondary available data is the data corresponding to the first confidence of the first algorithm in the target data, that is, the target data in which the first algorithm is unreliable in the plurality of groups of target data. The second algorithm corresponding to the secondary available data is determined from the predetermined algorithm set corresponding to the secondary available data, wherein the second algorithm is the algorithm with the second priority in the corresponding predetermined algorithm set. The corresponding secondary available data is processed by using the second algorithm to obtain the corresponding first obstacle information. Through the above processing, when the first algorithm is unreliable in the plurality of groups of target data, the second algorithm with the second priority is determined, and the orderliness of the data processing process is ensured.

[0040] As an optional embodiment, the target obstacle of the target vehicle is determined based on the first obstacle information corresponding to the multi-type primary sensor respectively. The target obstacle is determined by using the multi-type primary sensor, so that the target obstacle determined is based on the first obstacle information corresponding to the multi-type primary sensor respectively, which can ensure that the target obstacle of the target vehicle is accurately and reliably determined.

[0041] As an optional embodiment, when the target obstacle of the target vehicle is determined based on the first obstacle information corresponding to the plurality of primary sensors respectively, a plurality of ways can be adopted, for example, the following way can be adopted: based on the first obstacle information of the obstacles corresponding to the plurality of primary sensors respectively, the obstacles corresponding to the plurality of primary sensors respectively are matched to obtain a first matching result, and the target obstacle is determined in the case that the first matching result is that the matching of the obstacles corresponding to the plurality of primary sensors respectively is all successful. For example, when the plurality of primary sensors includes a camera sensor, the target data corresponding to this type of primary sensor is image data, and the first obstacle information determined includes the position, size and type of the obstacle. When the plurality of primary sensors includes a millimeter wave radar sensor, the target data corresponding to this type of primary sensor is millimeter wave data, and the first obstacle information determined includes the angle, distance and speed of the obstacle and the target vehicle. If the first matching result is that the matching of the obstacles corresponding to the plurality of primary sensors respectively is all successful, that is, the obstacle can be matched to the corresponding primary sensor in the primary sensor, it indicates that the corresponding target data collected by the plurality of primary sensors is accurate, which can ensure that the target obstacle determined is determined by a plurality of data cooperatively.

[0042] As an optional embodiment, in the case that the predetermined algorithm set includes at least three algorithms and the first matching result is that the matching of the obstacles corresponding to the plurality of primary sensors respectively fails, the first to-be-processed data is determined in the target data, wherein the first to-be-processed data is the data corresponding to the obstacle information causing the matching failure in the first obstacle information, and the first to-be-processed data is determined in time, and the reason for the obstacle causing the matching failure is determined in time, so that the primary sensor can be reasonably adjusted to more accurately detect the target obstacle next time. The second algorithm corresponding to the first to-be-processed data is determined from the predetermined algorithm set corresponding to the first to-be-processed data, wherein the second algorithm is the second algorithm in the priority arrangement of the corresponding predetermined algorithm set. The corresponding first to-be-processed data is processed by using the second algorithm to obtain the corresponding second obstacle information. Based on the first obstacle information and the second obstacle information, the obstacles corresponding to the plurality of primary sensors respectively are matched to obtain a second matching result. In the case that the second matching result is that the matching of the obstacles corresponding to the plurality of primary sensors respectively fails, the next to-be-processed data, the next algorithm corresponding to the next to-be-processed data, the corresponding next obstacle information obtained by processing the corresponding next to-be-processed data by using the next algorithm, and the next matching result are determined in a loop until the matching result is that the matching of the obstacles corresponding to the plurality of primary sensors respectively is all successful, and the target obstacle is determined. In this way, the target obstacle can be determined in order, and it is ensured that the target obstacle determined is accurate and reliable.

[0043] It should be noted that when there is data corresponding to the obstacle information causing the matching failure, the confidence of the algorithm corresponding thereto can be reduced, that is, the algorithms in the predetermined algorithm set are rearranged according to the confidence, that is, the priority of the algorithms in the predetermined algorithm set can change. In this way, it is ensured that the target obstacle can be efficiently and accurately determined when the obstacle detection is performed next time.

[0044] As an optional embodiment, in the case where there is no matching result in which the matching of the obstacles corresponding to the plurality of primary sensors is all successful, that is, the matching failure can be caused by the failure of the plurality of primary sensors, in this case, the first backup sensor is determined, and the first failure sensor and the first available sensor are determined from the plurality of primary sensors, so as to replace the failure sensor with the backup sensor. In this way, the first failure sensor in the plurality of primary sensors can be determined in time, and the first backup sensor is used to replace the first failure sensor, so that the failure of the plurality of primary sensors can be avoided to affect the obstacle matching, thereby improving the probability of successful obstacle matching.

[0045] As an optional embodiment, before the plurality of primary sensors are determined, the plurality of initial sensors and the target data corresponding to the plurality of initial sensors can also be determined; in the case where there is abnormal data in the target data corresponding to the plurality of initial sensors, the second backup sensor is determined, and the second failure sensor and the second available sensor are determined from the plurality of initial sensors; and the plurality of primary sensors are determined according to the second available sensor and the second backup sensor. In this way, the second failure sensor in the plurality of initial sensors is removed, and the second backup sensor is supplemented, so that it can be ensured that there is no abnormal data in the target data corresponding to the plurality of primary sensors including the second available sensor and the second backup sensor, that is, the plurality of primary sensors have no failure.

[0046] It should be noted that the second backup sensor also includes a predetermined algorithm set, and can perform the operation corresponding to the second available sensor.

[0047] Optionally, in the case where there is abnormal data in the target data corresponding to the plurality of initial sensors, the confidence of the plurality of initial sensors having the abnormal data can be reduced, and the sensor with high confidence can be used to perform the method provided in the present application. The plurality of initial sensors having the abnormal data can also be processed for failure, so as to be repaired in time to ensure the normal use of the sensors.

[0048] Based on the above embodiments and optional embodiments, an optional implementation is provided, which is described in detail as follows.

[0049] An obstacle detection method is provided in the optional embodiment of the present application, which is described by taking a camera sensor and a millimeter wave radar sensor as examples. The following is a detailed introduction to the optional embodiment of the present application.

[0050] Figure 2 A partial flowchart of the obstacle detection method provided by the optional embodiment of the present application is shown in FIG. Figure 2 The method for determining a target obstacle is introduced, and the following is an introduction to the method.

[0051] S1, determining a camera sensor and a millimeter wave radar sensor, image data corresponding to the camera sensor, and millimeter wave radar data corresponding to the millimeter wave radar sensor;

[0052] S2, determining an image algorithm set for the image data and a millimeter wave algorithm set for the millimeter wave radar data;

[0053] The image algorithm set includes three algorithms: SMOKE target detection algorithm, YOLOv5 target detection algorithm, and target SVM classification and NMS screening algorithm; and the millimeter wave algorithm set includes three algorithms: hierarchical clustering algorithm, ROI (region of interest) setting and static noise filtering algorithm. Figure 3 A schematic diagram of the image processing module and the millimeter wave radar processing module provided by the optional embodiment of the present application using an algorithm to process data is shown in FIG. Figure 4 A schematic diagram of the image processing module and the millimeter wave radar processing module provided by the optional embodiment of the present application using another algorithm to process data is shown in FIG. Figure 5 A schematic diagram of the image processing module and the millimeter wave radar processing module provided by the optional embodiment of the present application using another algorithm to process data is shown in FIG. Figure 3 、 4 , 5, the image data is input into the image processing module, and the data is processed according to the algorithms in the image algorithm set; and the millimeter wave radar data is input into the millimeter wave processing module, and the data is processed according to the algorithms in the millimeter wave algorithm set.

[0054] S3, the image data is input into the image processing module, and the data is processed according to the algorithms in the image algorithm set; and the millimeter wave radar data is input into the millimeter wave processing module, and the data is processed according to the algorithms in the millimeter wave algorithm set;

[0055] S4, target detection:

[0056] The first image algorithm is used in the image processing module to process the image data to obtain first obstacle information of the obstacle, and the first millimeter wave algorithm is used in the millimeter wave processing module to process the millimeter wave data to obtain first obstacle information of the obstacle.

[0057] S5, determining the target obstacle according to the first obstacle information of the obstacle obtained by the image processing module and the first obstacle information of the obstacle obtained by the millimeter wave processing module.

[0058] It should be noted that the confidence of the first image algorithm and the first millimeter wave algorithm can also be determined, and the confidence is compared with the confidence threshold corresponding thereto; in the case that the confidence of the first image algorithm and the first millimeter wave algorithm is lower than the corresponding confidence threshold, for example, the confidence of the first millimeter wave algorithm is lower than the corresponding confidence threshold, the first millimeter wave algorithm is replaced by the second millimeter wave algorithm, the second millimeter wave algorithm is used in the millimeter wave processing module to process the millimeter wave data to obtain a group of obstacles and the first obstacle information of the obstacle. The confidence of the second millimeter wave algorithm is determined, and if the confidence is still lower than the confidence threshold, the target algorithm is determined in turn according to the algorithms arranged in the predetermined algorithm set priority, until an algorithm higher than the confidence threshold is determined.

[0059] Figure 6 is a flowchart for fault judgment of the sensor in the optional embodiment of the present application, as shown in Figure 6 if there is no algorithm higher than the confidence threshold, it is proved that the millimeter wave radar sensor has a fault, therefore, the millimeter wave radar sensor is discarded, and the backup sensor, i.e. the laser radar sensor, is enabled.

[0060] Through the above-mentioned optional embodiment, at least the following beneficial effects can be achieved:

[0061] (1) the accuracy and reliability of the target obstacle determined are enhanced;

[0062] (2) the safety in the automatic driving process is improved;

[0063] (3) the faults of the known and unknown environment perception system and the external malicious attacks can be resisted.

[0064] It should be noted that for the above-mentioned method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the action sequence described, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily necessary for the present application.

[0065] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software on a general hardware platform as necessary, and of course can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product in essence or in the part that contributes to the prior art, and the computer software product is stored in a storage medium (such as a ROM / RAM, a magnetic disk, or an optical disk), and includes a plurality of instructions for causing an end device (which can be a mobile phone, a computer, a server, or a network device) to execute the method of each embodiment of the present application.

[0066] Embodiment 2

[0067] According to the embodiments of the present application, a device for implementing the above obstacle detection method is also provided, Figure 7 is a structural block diagram of the obstacle detection device according to the embodiments of the present application, as Figure 7 shown, the device includes a first determination module 702, a second determination module 704, an acquisition module 706, and a third determination module 708, which are described in detail below.

[0068] The first determination module 702 is configured to determine a plurality of primary sensors, and target data corresponding to the plurality of primary sensors, respectively, wherein the target data is data obtained by collecting data of a predetermined range to which a target vehicle belongs by using the corresponding primary sensor; the second determination module 704 is connected to the first determination module 702 and is configured to determine a predetermined algorithm set corresponding to the target data, respectively, and determine a first algorithm corresponding to the target data from the predetermined algorithm set, respectively, wherein the predetermined algorithm set includes at least two algorithms, the at least two algorithms are arranged in order of priority, and the first algorithm is an algorithm with the highest priority in the corresponding predetermined algorithm set; the acquisition module 706 is connected to the second determination module 704 and is configured to process the corresponding target data by using the first algorithm to obtain corresponding first obstacle information; and the third determination module 708 is connected to the acquisition module 706 and is configured to determine a target obstacle of the target vehicle based on the first obstacle information corresponding to the plurality of primary sensors, respectively.

[0069] It should be noted that the first determination module 702, the second determination module 704, the acquisition module 706, and the third determination module 708 correspond to steps S102 to S108 in the method of implementing the obstacle detection method, and the plurality of modules have the same instances and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment 1.

[0070] Embodiment 3

[0071] According to another aspect of the embodiments of the present application, an electronic device is provided, including: a processor; and a memory for storing processor-executable instructions, wherein the processor is configured to execute the instructions to implement any of the above obstacle detection methods.

[0072] Embodiment 4

[0073] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform any of the above obstacle detection methods.

[0074] The above-mentioned embodiment numbers of the present application are only for description, and do not represent the advantages or disadvantages of the embodiments.

[0075] In the above-mentioned embodiments of the present application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0076] In several embodiments provided in the present application, it should be understood that the disclosed technical contents can be implemented by other ways. Among them, the above-mentioned device embodiments are only schematic, for example, the division of the units can be a logical function division, and actual implementation can have another division way, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or modules shown or discussed can be indirect coupling or communication connection through some interfaces, units or modules, which can be electrical or other forms.

[0077] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on multiple units. Part or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment scheme.

[0078] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be realized in the form of hardware or in the form of software functional unit.

[0079] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to make a computer device (which can be a personal computer, a server or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0080] The above is only the preferred embodiment of the present application, it should be pointed out that, for those skilled in the art, without departing from the principles of the present application, can make a number of improvements and refinements, these improvements and refinements should also be considered as the protection scope of the present application.

Claims

1. An obstacle detection method characterized by, The method comprises the following steps: determining a plurality of types of primary sensors and target data corresponding to the plurality of types of primary sensors respectively, wherein the target data is data obtained by collecting data of a predetermined range to which a target vehicle belongs by using the corresponding primary sensor; determining a predetermined algorithm set corresponding to the target data respectively, and determining a first algorithm corresponding to the target data from the predetermined algorithm set respectively, wherein the predetermined algorithm set comprises at least two algorithms, the at least two algorithms are arranged in a priority order, and the first algorithm is an algorithm with the first priority in the corresponding predetermined algorithm set; processing the corresponding target data by using the first algorithm to obtain corresponding first obstacle information; determining a target obstacle of the target vehicle based on the first obstacle information corresponding to the plurality of types of primary sensors respectively; wherein the processing of the corresponding target data by using the first algorithm to obtain the corresponding first obstacle information comprises: determining a first confidence corresponding to the first algorithm respectively; in the case that there is a confidence less than a predetermined threshold in the first confidence corresponding to the first algorithm, determining secondary available data in the target data, wherein the secondary available data is data in the target data corresponding to the first algorithm with the confidence less than the predetermined threshold; determining a second algorithm corresponding to the secondary available data from the predetermined algorithm set corresponding to the secondary available data, wherein the second algorithm is an algorithm with the second priority in the corresponding predetermined algorithm set; and processing the corresponding secondary available data by using the second algorithm to obtain the corresponding first obstacle information.

2. The method of claim 1, wherein, The determination of the target obstacle of the target vehicle based on the first obstacle information corresponding to the plurality of types of primary sensors respectively comprises: matching the obstacles corresponding to the plurality of types of primary sensors based on the first obstacle information of the obstacles corresponding to the plurality of types of primary sensors to obtain a first matching result; in the case that the first matching result is that the matching of the obstacles corresponding to the plurality of types of primary sensors is all successful, determining the target obstacle.

3. The method of claim 2, wherein, Further comprising: in the case that the predetermined algorithm set comprises at least three algorithms, and the first matching result is that the matching of the obstacles corresponding to the plurality of types of primary sensors fails, determining first to-be-processed data in the target data, wherein the first to-be-processed data is data corresponding to obstacle information causing the matching failure in the first obstacle information; determining a second algorithm corresponding to the first to-be-processed data from the predetermined algorithm set corresponding to the first to-be-processed data respectively, wherein the second algorithm is an algorithm with the second priority in the corresponding predetermined algorithm set; processing the corresponding first to-be-processed data by using the second algorithm to obtain corresponding second obstacle information; match the obstacles corresponding to the plurality of primary sensors respectively based on the first obstacle information and the second obstacle information to obtain a second matching result; in a case where the second matching result is that the matching of the obstacles corresponding to the plurality of primary sensors respectively fails, a next to-be-processed data, a next algorithm corresponding to the next to-be-processed data, next obstacle information obtained by processing the next to-be-processed data using the next algorithm, and a next matching result are determined in a loop until it is determined that the matching of the obstacles corresponding to the plurality of primary sensors respectively is successful, and the target obstacle is determined.

4. The method of claim 3, wherein, Further comprising: in a case where there is no matching result in which the matching of the obstacles corresponding to the plurality of primary sensors respectively is successful, a first backup sensor is determined, and a first faulty sensor and a first available sensor are determined from the plurality of primary sensors to replace the faulty sensor with the backup sensor.

5. The method of claim 1, wherein, The determination of the plurality of primary sensors and the target data corresponding to the plurality of primary sensors comprises: determining the plurality of primary sensors and a sensor characteristic corresponding to each primary sensor in the plurality of primary sensors; determining a sensor quantity and a sensor position corresponding to each primary sensor in the plurality of primary sensors according to the sensor characteristic; determining the target data corresponding to the plurality of primary sensors based on the sensor quantity and the sensor position corresponding to each primary sensor in the plurality of primary sensors.

6. The method according to any one of claims 1 to 5, characterized in that, Before determining the plurality of primary sensors, further comprising: determining a plurality of initial sensors and target data corresponding to the plurality of initial sensors; in a case where there is abnormal data in the target data corresponding to the plurality of initial sensors, a second backup sensor is determined, and a second faulty sensor and a second available sensor are determined from the plurality of initial sensors; determining the plurality of primary sensors according to the second available sensor and the second backup sensor.

7. An obstacle detection system characterized by, Comprising: a first determination module configured to determine a plurality of primary sensors and target data corresponding to the plurality of primary sensors, wherein the target data is data obtained by collecting data of a predetermined range to which a target vehicle belongs using a corresponding primary sensor; a second determination module configured to determine a predetermined algorithm set corresponding to the target data respectively, and determine a first algorithm corresponding to the target data from the predetermined algorithm set respectively, wherein the predetermined algorithm set comprises at least two algorithms, the at least two algorithms are arranged in a priority order, and the first algorithm is an algorithm with a first priority in the corresponding predetermined algorithm set; an acquisition module configured to process the corresponding target data using the first algorithm to obtain corresponding first obstacle information; a third determination module configured to determine a target obstacle of the target vehicle based on the first obstacle information corresponding to the plurality of primary sensors respectively. The acquisition module is further configured to determine a first confidence corresponding to the first algorithm respectively; in a case where there is a confidence less than a predetermined threshold in the first confidence corresponding to the first algorithm, determine secondary available data in the target data, wherein the secondary available data is data in the target data corresponding to the first algorithm whose first confidence is less than the predetermined threshold; determine a second algorithm corresponding to the secondary available data from the predetermined algorithm set corresponding to the secondary available data respectively, wherein the second algorithm is an algorithm whose priority ranking is second in the corresponding predetermined algorithm set; and process the corresponding secondary available data by using the second algorithm to obtain the corresponding first obstacle information.

8. An electronic device, comprising: The method comprises: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the obstacle detection method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is enabled to perform the obstacle detection method according to any one of claims 1 to 6.

Images