Obstacle perception method, apparatus, device, vehicle and medium

Abstract

The disclosure provides an obstacle perception method, device, equipment, vehicle and medium, relates to the technical field of computers, specifically to the technical fields of artificial intelligence, computer vision, intelligent vehicles and autonomous driving vehicles. The method comprises the following steps: determining a three-dimensional detection frame and a category of each of a plurality of candidate obstacles around a vehicle; determining a two-dimensional detection frame of the plurality of candidate obstacles projected under the camera angle of the vehicle based on the three-dimensional detection frame of each of the plurality of candidate obstacles; filtering out the same-category occluded candidate obstacles based on the three-dimensional detection frame and the two-dimensional detection frame of each of the plurality of candidate obstacles to obtain a plurality of target obstacles; in response to determining that the number of the plurality of target obstacles is greater than a preset number, determining an importance confidence of each of the plurality of target obstacles; and retaining a preset number of target obstacles with the highest importance confidence in the plurality of target obstacles.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, in particular, to the technical fields of artificial intelligence, computer vision, intelligent vehicle and autonomous vehicle, and particularly relates to an obstacle perception method, an obstacle perception device, an electronic device, a vehicle, a computer readable storage medium and a computer program product. BACKGROUND

[0002] In the intelligent driving scheme, the perceived obstacle signal is very important for the safe driving of the vehicle. To ensure the timeliness of the perceived signal processing, the entire intelligent driving system can be more sensitive.

[0003] The methods described in this section can not have been previously conceived or made. Unless otherwise indicated, it should not be assumed that any of the methods described in this section require that any particular method be practiced in any particular order. Similarly, it should not be assumed that any of the issues discussed in this section have been admitted to be prior art. SUMMARY

[0004] The present disclosure provides an obstacle perception method, an obstacle perception device, an electronic device, a vehicle, a computer readable storage medium and a computer program product.

[0005] According to an aspect of the present disclosure, an obstacle perception method for a vehicle is provided, comprising: determining a three-dimensional detection box and a category of each of a plurality of candidate obstacles around the vehicle; determining a two-dimensional detection box of the plurality of candidate obstacles projected under a camera view of the vehicle based on the three-dimensional detection box of each of the plurality of candidate obstacles; filtering out occluded candidate obstacles of the same category based on the three-dimensional detection box and the two-dimensional detection box of each of the plurality of candidate obstacles to obtain a plurality of target obstacles; in response to determining that the number of the plurality of target obstacles is greater than a preset number, determining an importance confidence of each of the plurality of target obstacles; and retaining a preset number of target obstacles with the highest importance confidence from the plurality of target obstacles.

[0006] According to another aspect of the present disclosure, there is provided an obstacle perception device for a vehicle, comprising: a first determination unit configured to determine a three-dimensional bounding box and a category of each of a plurality of candidate obstacles around the vehicle; a second determination unit configured to determine a two-dimensional bounding box of each of the plurality of candidate obstacles projected under a camera view of the vehicle based on the three-dimensional bounding box of each of the plurality of candidate obstacles; a filtering unit configured to filter out occluded candidate obstacles of the same category based on the three-dimensional bounding box and the two-dimensional bounding box of each of the plurality of candidate obstacles to obtain a plurality of target obstacles; a third determination unit configured to determine an importance confidence of each of the plurality of target obstacles in response to determining that a number of the plurality of target obstacles is greater than a preset number; and a number control unit configured to retain a preset number of target obstacles with the highest importance confidence among the plurality of target obstacles.

[0007] According to another aspect of the present disclosure, there is provided an electronic device, comprising: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the above method.

[0008] According to another aspect of the present disclosure, there is provided a vehicle comprising the above electronic device.

[0009] According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to perform the above method.

[0010] According to another aspect of the present disclosure, there is provided a computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements the above method.

[0011] According to one or more embodiments of the present disclosure, the present disclosure first determines a three-dimensional bounding box of a plurality of candidate obstacles around a vehicle, projects the three-dimensional bounding box to a camera view of the vehicle to obtain a two-dimensional bounding box, filters out occluded candidate obstacles of the same category in combination with the three-dimensional bounding box and the two-dimensional bounding box, and then determines an importance confidence of a plurality of target obstacles according to a number of the plurality of target obstacles to perform further number control. In this way, by comprehensively utilizing information of the candidate obstacles in multiple dimensions, the present disclosure can effectively filter out obstacles with low impact on intelligent driving, while ensuring that key obstacles affecting intelligent driving are retained, thereby achieving effective and reasonable control of the number of perceived obstacles and ensuring the stability of the perception link latency processing.

[0012] It is to be understood that the details set forth herein do not limit the scope of the embodiments of the present disclosure but merely contemplate it to a preferred embodiment. Other features, aspects, and advantages of the disclosure will become apparent from the following description, which refers to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0013] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in

[0014] Figure 1 shows a schematic diagram of an exemplary system in which various methods described herein can be implemented, in accordance with embodiments of the present disclosure;

[0015] Figure 2 shows a flowchart of an obstacle perception method, in accordance with exemplary embodiments of the present disclosure;

[0016] Figure 3 shows a schematic diagram of removing position overlapping obstacles, in accordance with exemplary embodiments of the present disclosure;

[0017] Figure 4 shows a flowchart of filtering out occluded candidate obstacles of the same category, in accordance with exemplary embodiments of the present disclosure;

[0018] Figure 5 shows a flowchart of determining three-dimensional occlusion information of each of a plurality of candidate obstacles, in accordance with exemplary embodiments of the present disclosure;

[0019] Figure 6 shows a schematic diagram of three-dimensional occlusion, in accordance with exemplary embodiments of the present disclosure;

[0020] Figure 7 shows a schematic diagram of a region of interest, in accordance with exemplary embodiments of the present disclosure;

[0021] Figure 8 shows a schematic diagram of a region of non-interest and a cone bucket, in accordance with exemplary embodiments of the present disclosure;

[0022] Figure 9 shows a structural block diagram of an obstacle perception device, in accordance with exemplary embodiments of the present disclosure; and

[0023] Figure 10 shows a structural block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure. DETAILED DESCRIPTION

[0024] Exemplary embodiments of the present disclosure are described herein below with reference to the accompanying drawings, in which various details of embodiments of the present disclosure are set forth in order to provide a thorough understanding of the present disclosure. It will be apparent, however, to those skilled in the art that the embodiments described herein can be practiced without many of the specific details given herein. In other instances, well-known features have not been described in detail in order to avoid obscuring the present disclosure. Thus, the following description is presented to enable any person skilled in the art to practice the embodiments described herein.

[0025] In the present disclosure, the terms "first", "second", and the like are used to describe various elements only for the purpose of distinguishing one element from another, and are not intended to limit the positions, sequence, or importance of the elements. In some examples, a first element and a second element can refer to the same instance of the element, and in some cases, based on the context of the description, they can also refer to different instances.

[0026] The terms used in the description of various described examples in the present disclosure are only for the purpose of describing particular examples and are not intended to be limiting. Unless specifically defined otherwise, an element that is a singular can be plural and vice versa. Furthermore, the term "and / or" used in the present disclosure encompasses any and all possible combinations of one or more of the associated listed items.

[0027] In the related art, the effect of the existing obstacle perception method for vehicles is poor.

[0028] To solve the above problems, the present disclosure first determines a three-dimensional bounding box of a plurality of candidate obstacles around the vehicle, and projects the three-dimensional bounding box to the camera view angle of the vehicle to obtain a two-dimensional bounding box, and then filters out the same category of occluded candidate obstacles in combination with the three-dimensional bounding box and the two-dimensional bounding box, and further determines the importance confidence of a plurality of target obstacles according to the number of the filtered target obstacles to perform further number control. Thus, by comprehensively utilizing the information of the candidate obstacles in multiple dimensions, the obstacles with low impact on intelligent driving can be effectively filtered out, while ensuring that the key obstacles affecting intelligent driving are retained, thereby achieving effective and reasonable control of the number of perceived obstacles and ensuring the stability of the perception link latency processing.

[0029] Embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0030] Figure 1 A schematic diagram of an exemplary system 100 in which various methods and apparatuses described herein can be implemented according to embodiments of the present disclosure is shown. Referring to Figure 1 The system 100 includes a motor vehicle 110, a server 120, and one or more communication networks 130 coupling the motor vehicle 110 to the server 120.

[0031] In embodiments of the present disclosure, the motor vehicle 110 can include a computing device in accordance with embodiments of the present disclosure and / or be configured to perform methods in accordance with embodiments of the present disclosure.

[0032] The server 120 can run one or more services or software applications that implement the obstacle perception method. In certain embodiments, the server 120 can also provide other services or software applications, which can include non-virtual environments and virtual environments. In Figure 1 In the illustrated configuration, the server 120 can include one or more components that implement the functionality performed by the server 120. These components can include software components that are executable by one or more processors, hardware components, or a combination thereof. Users of the motor vehicle 110 can in turn utilize one or more client applications to interact with the server 120 to utilize the services provided by the components. It should be understood that a variety of different system configurations are possible, which can vary from the system 100. Thus, Figure 1 is one example of a system for implementing the various methods described herein and is not intended to be limiting.

[0033] The server 120 can include one or more general purpose computers, special purpose server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other appropriate arrangement and / or combination. The server 120 can include one or more virtual machines running virtual operating systems, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that can be virtualized to maintain virtual storage devices for the server). In various embodiments, the server 120 can run one or more services or software applications that provide the functionality described below.

[0034] The computing units in the server 120 can run one or more operating systems including any of the operating systems described above, as well as any commercially available server operating systems. The server 120 can also run any of a variety of additional server applications and / or middleware applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.

[0035] In some implementations, the server 120 can include one or more applications to analyze and consolidate data feeds and / or event updates received from the motor vehicle 110. The server 120 can also include one or more applications to display the data feeds and / or real-time events via one or more display devices of the motor vehicle 110.

[0036] The networks 130 can be any type of networks familiar to those skilled in the art that can support data communications using any of a variety of available protocols, including without limitation TCP / IP, SNA, IPX, etc. As examples, one or more of the networks 130 can be satellite communications networks, local area networks (LANs), Ethernet-based networks, token ring networks, wide area networks (WANs), the Internet, virtual networks, virtual private networks (VPNs), intranets, extranets, blockchain networks, public switched telephone networks (PSTNs), infrared networks, wireless networks (including, for example, Bluetooth, WiFi), and / or any combination of these and other networks.

[0037] The system 100 can also include one or more databases 150. In certain embodiments, these databases can be used to store data and other information. For example, one or more of the databases 150 can be used to store information such as audio files and video files. The data stores 150 can reside at various locations. For example, a data store used by the servers 120 can be local to the servers 120 or can be remote from the servers 120 and can communicate with the servers 120 via a network-based or dedicated connection. The data stores 150 can be of different types. In certain embodiments, a data store used by the servers 120 can be a database, such as a relational database. One or more of these databases can store, update, and retrieve data to and from the databases in response to commands.

[0038] In certain embodiments, one or more of the databases 150 can also be used by the applications to store application data. Databases used by the applications can be databases of different types, such as key-value stores, object stores, or regular stores supported by file systems.

[0039] The motor vehicle 110 can comprise sensors 111 for perceiving the surrounding environment. The sensors 111 can comprise one or more of the following sensors: visual camera, infrared camera, ultrasonic sensor, millimeter wave radar, and laser radar (LiDAR). Different sensors can provide different detection accuracy and range. The camera can be installed at the front, rear or other positions of the vehicle. The visual camera can capture the situation inside and outside the vehicle in real time and present it to the driver and / or passenger. In addition, by analyzing the pictures captured by the visual camera, information such as traffic signal indication, intersection situation, other vehicle operating state, etc. can be obtained. The infrared camera can capture objects in night vision conditions. The ultrasonic sensor can be installed around the vehicle to measure the distance from the object outside the vehicle to the vehicle by taking advantage of the strong directivity of ultrasonic waves. The millimeter wave radar can be installed at the front, rear or other positions of the vehicle to measure the distance from the object outside the vehicle to the vehicle by taking advantage of the characteristics of electromagnetic waves. The laser radar can be installed at the front, rear or other positions of the vehicle to detect the edge and shape information of the object, thereby performing object recognition and tracking. Due to the Doppler effect, the radar device can also measure the speed change of the vehicle and the moving object.

[0040] The motor vehicle 110 can also comprise a communication device 112. The communication device 112 can comprise a satellite positioning module capable of receiving satellite positioning signals (e.g. Beidou, GPS, GLONASS and GALILEO) from satellites 141 and generating coordinates based on these signals. The communication device 112 can also comprise a module for communicating with mobile communication base stations 142, and the mobile communication network can implement any suitable communication technology, such as GSM / GPRS, CDMA, LTE, etc. current or developing wireless communication technology (e.g. 5G technology). The communication device 112 can also have a vehicle-to-everything (V2X) module configured to enable communication with the outside world, such as vehicle-to-vehicle (V2V) communication with other vehicles 143 and vehicle-to-infrastructure (V2I) communication with infrastructure 144. In addition, the communication device 112 can also have a module configured to communicate with user terminals 145 (including but not limited to smartphones, tablets or wearable devices such as watches) by using, for example, wireless local area networks or Bluetooth based on IEEE 802.11 standards. With the communication device 112, the motor vehicle 110 can also access the server 120 via the network 130.

[0041] The motor vehicle 110 may also include a control unit 113. The control unit 113 may include a processor, such as a central processing unit (CPU) or a graphics processing unit (GPU), or other dedicated processors, that communicates with various types of computer-readable storage devices or media. The control unit 113 may include an autonomous driving system for automatically controlling various actuators in the vehicle. The autonomous driving system is configured to control the powertrain, steering system, and braking system of the motor vehicle 110 (not shown) via multiple actuators in response to inputs from multiple sensors 111 or other input devices to control acceleration, steering, and braking respectively, without human intervention or with limited human intervention. Some processing functions of the control unit 113 can be implemented via cloud computing. For example, some processing can be performed using an onboard processor while other processing can be performed using cloud computing resources. The control unit 113 may be configured to perform an obstacle perception method according to this disclosure. Furthermore, the control unit 113 may be implemented as an example of a computing device on the motor vehicle side (client) according to this disclosure.

[0042] Figure 1 The system 100 can be configured and operated in various ways to enable the application of the various methods and apparatus described in this disclosure.

[0043] According to one aspect of this disclosure, an obstacle perception method for a vehicle is provided. For example... Figure 2 As shown, the obstacle perception method includes: step S201, determining the 3D detection boxes and categories of multiple candidate obstacles around the vehicle; step S202, determining the 2D detection boxes projected onto the vehicle's camera view based on the 3D detection boxes of the multiple candidate obstacles; step S203, filtering out occluded candidate obstacles of the same category based on the 3D and 2D detection boxes of the multiple candidate obstacles to obtain multiple target obstacles; step S204, determining the importance confidence of each of the multiple target obstacles in response to determining that the number of multiple target obstacles is greater than a preset number; and step S205, retaining the preset number of target obstacles with the highest importance confidence among the multiple target obstacles.

[0044] The disclosure first determines a three-dimensional bounding box of a plurality of candidate obstacles around the vehicle, projects the three-dimensional bounding box to the camera view angle of the vehicle to obtain a two-dimensional bounding box, and then filters out the occluded candidate obstacles of the same category in combination with the three-dimensional bounding box and the two-dimensional bounding box, and further determines the importance confidence of the plurality of target obstacles according to the number of the filtered target obstacles to perform further quantity control. Thus, by comprehensively utilizing the information of the candidate obstacles in multiple dimensions, the obstacles with low impact on intelligent driving can be effectively filtered out, while ensuring that the key obstacles affecting intelligent driving are retained, thereby realizing effective and reasonable control of the number of perceived obstacles and ensuring the stability of the perception link delay processing.

[0045] The method proposed by the disclosure can be used in an autonomous vehicle or an intelligent driving system (also referred to as a smart driving system). In the context of the disclosure, the "vehicle" referred to in the above steps is also referred to as "ego vehicle" or "host vehicle".

[0046] In some embodiments, the perception model (e.g., a Bird's-Eye-View (BEV) model) can be used for obstacle detection to obtain the relevant information of the plurality of candidate obstacles. The relevant information can include a three-dimensional bounding box and a category, and can also include orientation, speed, and other motion information and detection confidence. The detection confidence can indicate the degree of correct detection of the corresponding candidate obstacle. Further, some positionally overlapping candidate obstacles can be removed. In an exemplary embodiment, a three-dimensional non-maximum suppression (3D Non-Max Suppression, NMS) technique can be used: the intersection over union between the three-dimensional bounding boxes of candidate obstacles of the same category is calculated, and if it exceeds a preset threshold, the candidate obstacle with higher detection confidence is retained.

[0047] Figure 3 A schematic diagram of removing positionally overlapping obstacles according to an exemplary embodiment of the disclosure is shown. A plurality of candidate obstacles are detected around the ego vehicle 302, and by using the 3D NMS technique, the overlapping obstacles 304 can be removed to obtain the retained perceived obstacles 306.

[0048] It should be noted that in the disclosure, the meaning of "occluded" by B can be that the proportion of the occluded part of A is greater than a certain preset value. In other words, "occluded" does not necessarily mean completely occluded, or only a very small part is occluded, which does not necessarily be deemed as "occluded".

[0049] The reserved candidate obstacles can perform post-processing operations. The post-processing operations mainly incorporate timing information, i.e., use the information of the candidate obstacles in multiple time frames to extract more accurate motion information, such as obstacle speed, acceleration, etc. In an exemplary embodiment, the post-processing operations can include three parts of obstacle tracking, speed filtering and obstacle reporting.

[0050] The present disclosure adds multiple steps of quantity control of obstacles between obstacle detection (and 3D NMS) and post-processing, thereby achieving effective reduction of perceived obstacles affecting intelligent driving in complex scenes, while retaining key obstacles affecting intelligent driving, so that the maximum calculation amount of the perception link is controlled, thereby ensuring the processing stability of the perception link.

[0051] From the perspective of obstacle priority affecting intelligent driving, the inventors have concluded that:

[0052] 1) Unoccluded obstacles are more important than occluded obstacles;

[0053] 2) Forward obstacles of the host vehicle are more important than obstacles in other directions;

[0054] 3) Moving obstacles are more important than stationary obstacles.

[0055] The present disclosure filters and controls the quantity of obstacles based on the above objective laws.

[0056] In step S202, the three-dimensional detection boxes of the multiple candidate obstacles determined in step S201 can be projected into the camera view of the vehicle to obtain two-dimensional detection boxes. This process can be understood as determining, for a two-dimensional image captured in the camera view, a two-dimensional detection box of the three-dimensional detection box of the candidate obstacle projected on the two-dimensional image. In some embodiments, the position of the above-mentioned two-dimensional detection box can be calculated according to the camera intrinsic parameters and the three-dimensional position, size and / or orientation of the three-dimensional detection box.

[0057] According to some embodiments, as Figure 4As shown, the step S203 of filtering out the occluded candidate obstacles of the same category based on the three-dimensional bounding boxes and the two-dimensional bounding boxes of the plurality of candidate obstacles can include: a step S401 of determining three-dimensional occlusion information of each of the plurality of candidate obstacles, wherein the three-dimensional occlusion information indicates an occlusion relationship between the three-dimensional bounding box of the corresponding candidate obstacle and the three-dimensional bounding boxes of other candidate obstacles of the same category in the vehicle coordinate system; a step S402 of determining two-dimensional occlusion information of each of the plurality of candidate obstacles, wherein the two-dimensional occlusion information indicates an occlusion relationship between the two-dimensional bounding box of the corresponding candidate obstacle and the two-dimensional bounding boxes of other candidate obstacles of the same category in the camera perspective; and a step S403 of determining, for each candidate obstacle, whether to filter out the candidate obstacle based on the three-dimensional occlusion information and the two-dimensional occlusion information of the candidate obstacle.

[0058] In the present disclosure, the camera perspective is similar to the perspective formed by observing the surroundings from the vehicle, while the vehicle coordinate system is similar to the bird's eye perspective. Thus, by determining the occlusion relationship between the three-dimensional bounding boxes of the candidate obstacles of the same category in the vehicle coordinate system and determining the occlusion relationship between the two-dimensional bounding boxes of the candidate obstacles of the same category in the camera perspective, it is possible to judge the occlusion relationship between the candidate obstacles from multiple perspectives and angles, thereby more accurately and effectively filtering out the obstacles that have less impact on intelligent driving.

[0059] According to some embodiments, as Figure 5 As shown, the step S401 of determining the three-dimensional occlusion information of each of the plurality of candidate obstacles can include: a step S501 of projecting the three-dimensional bounding boxes of the plurality of candidate obstacles onto a horizontal plane in the vehicle coordinate system to obtain position information and shape information of the plurality of candidate obstacles in the horizontal plane; a step S502 of determining, for two candidate obstacles of the same category, a first included angle of an overlapping part of the two candidate obstacles with respect to the origin of the vehicle coordinate system and a second included angle of the entire two candidate obstacles with respect to the origin of the vehicle coordinate system in the horizontal plane; and a step S503 of determining a three-dimensional occlusion rate of the occluded candidate obstacle of the two candidate obstacles based on the first included angle and the second included angle. The three-dimensional occlusion information of the candidate obstacle can include the three-dimensional occlusion rate obtained in the above manner.

[0060] Thus, by the above manner, the obstacle position and shape information in the bird's eye perspective are provided, thereby being able to more reasonably quantify the occlusion degree between the candidate obstacles and the impact degree on intelligent driving.

[0061] In some embodiments, the vehicle coordinate system in step S501 may, for example, adopt a host vehicle coordinate system. The position information and shape information may include the positions of four corner points of the candidate obstacle projected on the horizontal plane (a plane defined by the vehicle head direction and the vehicle side direction) of the host vehicle coordinate system. In step S502, the maximum angles of the four corner points of each of the two candidate obstacles with the origin of the vehicle coordinate system can be calculated first: let the maximum angle of candidate obstacle A with the origin of the vehicle body be a, and the maximum angle of candidate obstacle B with the origin of the vehicle body be b, then the first angle of the overlapping part of the two candidate obstacles with the origin of the vehicle coordinate system in the horizontal plane of the vehicle coordinate system can be a∩b, and the second angle of the whole of the two candidate obstacles with the origin of the vehicle coordinate system can be a∪b.

[0062] In an example embodiment, in step S503, a∩b / a∪b can be determined as the three-dimensional occlusion rate of the occluded candidate obstacle. It can be understood that the three-dimensional occlusion rate of the candidate obstacle can also be determined in other ways, which are not limited herein. Figure 6 A schematic diagram of three-dimensional occlusion according to an example embodiment of the present disclosure is shown. After projecting candidate obstacle A and candidate obstacle B in the vehicle coordinate system, the maximum angles of the two with the origin can be determined, and then the first angle of the overlapping part and the second angle of the whole can be obtained, and the three-dimensional occlusion rate can be calculated.

[0063] According to some embodiments, step S401 can further include determining the three-dimensional occlusion information of each of the plurality of candidate obstacles, including: in response to determining that the same candidate obstacle has multiple three-dimensional occlusion rates, retaining the largest three-dimensional occlusion rate as the three-dimensional occlusion rate of the candidate obstacle.

[0064] In some embodiments, occlusion may occur between more than two candidate obstacles. The occlusion relationship or occlusion rate can be determined between any two candidate obstacles, and the maximum value among the multiple occlusion rates of the candidate obstacles is taken as the occlusion rate of the candidate obstacle, and then it is judged whether to be filtered out. In this way, the calculation complexity when multiple candidate obstacles are occluded can be reduced.

[0065] According to some embodiments, step S402, determining the two-dimensional occlusion information of each of the plurality of candidate obstacles can include: for two candidate obstacles of the same category, calculating the intersection over union of the two-dimensional detection boxes of the two candidate obstacles under the camera perspective to determine the two-dimensional occlusion rate of the occluded candidate obstacle among the two candidate obstacles.

[0066] Unlike the three-dimensional occlusion information embodying the occlusion situation under the bird's eye perspective, under the camera perspective, the two-dimensional detection boxes of different candidate obstacles may present different sizes in the lateral and longitudinal directions. Therefore, by calculating the intersection over union of the two candidate obstacles, a better occlusion situation of the candidate obstacle can be obtained.

[0067] According to some embodiments, the step S402 of determining the two-dimensional occlusion information of each of the plurality of candidate obstacles can comprise: in response to determining that one of the plurality of candidate obstacles has a plurality of two-dimensional occlusion rates, retaining the largest two-dimensional occlusion rate as the two-dimensional occlusion rate of the one candidate obstacle. In this way, the computational complexity of the occlusion of the plurality of candidate obstacles can be reduced.

[0068] According to some embodiments, the vehicle can have a plurality of cameras, and the two-dimensional bounding box of each of the plurality of candidate obstacles can comprise a plurality of two-dimensional bounding boxes projected in a plurality of camera perspectives. The two-dimensional occlusion information of each of the plurality of obstacles can comprise a plurality of two-dimensional occlusion information respectively corresponding to the plurality of camera perspectives.

[0069] In the case where the vehicle has a plurality of cameras, an obstacle that is occluded in one camera perspective can not be occluded in another camera. Therefore, by the above-mentioned way, the two-dimensional occlusion situation corresponding to each camera can be obtained, so that a more accurate occlusion situation between obstacles can be obtained. Subsequently, when filtering, more accurate obstacle filtering can be achieved by using more abundant two-dimensional occlusion information.

[0070] According to some embodiments, the step S403 of determining, for each candidate obstacle, whether to filter out the candidate obstacle based on the three-dimensional occlusion information and the two-dimensional occlusion information of the candidate obstacle can comprise: in response to determining that, in the vehicle coordinate system, the three-dimensional bounding box of the candidate obstacle is occluded by the three-dimensional bounding box of another candidate obstacle of the same category, and in response to determining that, in each camera perspective, the two-dimensional bounding box of the candidate obstacle is occluded by the two-dimensional bounding box of another candidate obstacle of the same category, determining to filter out the candidate obstacle.

[0071] Thus, by combining the two-dimensional occlusion situation corresponding to each camera and the occlusion situation in the three-dimensional overhead perspective, it can be ensured that important obstacles that are not occluded are all retained, while obstacles that have less impact on intelligent driving are effectively filtered out.

[0072] In step S204, in response to determining that the number of the plurality of target obstacles is greater than a preset number, the importance confidence of each of the plurality of target obstacles is determined. Then, in step S205, a preset number of target obstacles with the highest importance confidence are retained from the plurality of target obstacles.

[0073] After two-dimensional filtering and three-dimensional filtering, some obstacles satisfying three-dimensional occlusion and two-dimensional bounding box overlap relationship will be filtered out, but this cannot guarantee that the maximum number of obstacles is controlled. It is assumed that the maximum number of retained obstacles is N. If the number of obstacles obtained after the above filtering does not exceed N, the maximum value filtering module does not need to be processed; otherwise, the module will recalculate an importance confidence of the obstacles obtained in this stage, sort the obstacles according to the importance confidence, and only retain the first N obstacles after sorting.

[0074] According to some embodiments, the importance confidence can include a distance confidence, the distance confidence being negatively correlated with the distance between the corresponding target obstacle and the vehicle.

[0075] In one embodiment, the distance confidence can represent mapping the Euclidean distance between the obstacle and the host vehicle to the similarity, and the closer the distance, the more important. In one exemplary embodiment, the value of the distance confidence is greater than 1 for the obstacle within a preset distance (for example, 50 meters), and the value of the distance confidence is less than 1 for the obstacle beyond the preset distance. The distance confidence can be represented as:

[0076]

[0077] According to some embodiments, step S201 of determining the three-dimensional detection box and the category of each of the plurality of candidate obstacles around the vehicle can include: obtaining the three-dimensional detection box, the category and the detection confidence of each of the plurality of candidate obstacles by using a perception model, the detection confidence indicating the degree of correct detection of the corresponding candidate obstacle. The importance confidence can include the detection confidence. The perception model can be, for example, a BEV model, which can output a confidence describing the detection accuracy while outputting the obstacle information. The value of the detection confidence can be between 0 and 1, and the closer to 1, the higher the confidence that the obstacle is correctly detected by the model. The detection confidence can be represented as:

[0078] score confidence =obj confidence

[0079] wherein, obj confidence may represent the confidence output by the perception model.

[0080] According to some embodiments, the importance confidence can include a speed confidence, the speed confidence being positively correlated with the predicted speed of the corresponding target obstacle. The higher the speed value of the obstacle, the greater the impact on intelligent driving, and therefore the greater the speed confidence score. The speed confidence can be represented as:

[0081]

[0082] wherein, velL2 pred_speed = sqrt(max(100, (v - v0)2 / (v0)2)), where v0 represents the speed of the ego vehicle, v represents the speed of the target obstacle, pred_speed represents the predicted speed of the target obstacle, sqrt() is a square root operation, and max() represents taking the maximum value. It can be understood that 100 in the above formula can also be replaced by other values, which are not limited herein. In some embodiments, the predicted speed can be obtained by using a perception model.

[0083] According to some embodiments, the importance confidence can comprise: an attention region confidence, the attention region confidence being determined based on whether the corresponding target obstacle is located in an attention region of the vehicle and whether the corresponding target obstacle is occluded.

[0084] The attention region can be a long strip-shaped region in front of the vehicle (in the direction of the vehicle head). In an example embodiment, the attention region can be a long strip-shaped region with a width of 17.5 meters in front of the vehicle 3-150 meters, and the direction of the vehicle head coincides with the center line of the attention region. A very high score can be given to the unoccluded obstacles in the front attention region, to ensure that the key obstacles are ranked in the front in the importance confidence sorting. The attention region confidence can be represented as:

[0085]

[0086] If the obstacle is located in the front attention region and the unoccluded obstacle is not marked, a higher value of 1e4 (i.e. 10000) is given, otherwise a value of 1 is given. It can be understood that the attention region can be set to other shapes and sizes, and the attention region confidence of the unoccluded obstacle (and the attention region confidence of other obstacles) therein can select other values, which are not limited herein. Figure 7 A schematic diagram of the attention region according to an example embodiment of the present disclosure is shown. The attention region 704 is located in the direction of the vehicle head of the ego vehicle 702.

[0087] According to some embodiments, the importance confidence can comprise a non-attention region cone confidence. Step S205, in response to determining that the number of the plurality of target obstacles is greater than the preset number, determining the importance confidence of each of the plurality of target obstacles can comprise: for one of the plurality of target obstacles, in response to determining that the category of the one target obstacle is a cone, and in response to determining that the one target obstacle is located in a non-attention region, determining a non-attention region cone confidence of the one target obstacle as a first value; and in response to determining that the category of the one target obstacle is other than the cone, or in response to determining that the one target obstacle is located inside the attention region, determining the non-attention region cone confidence of the one target obstacle as a second value, the first value can be less than the second value.

[0088]

[0089] Figure 8A schematic diagram of a non-interest region and cones is shown according to an example embodiment of the present disclosure. There is a non-interest region 804 outside the heading direction of the ego vehicle 802, where there are a plurality of cones 806.

[0090] In some embodiments, the one or more sub-items included in the importance confidence can be multiplied, summed, weighted added or calculated using other manners to obtain a final importance confidence for ranking.

[0091] In some embodiments, after the importance confidence of each obstacle is calculated, the top N obstacles are retained after ranking from high to low, and the number of obstacles is controlled to N. The N obstacles then enter a post-processing link for tracking, filtering, reporting and other operations, and are finally transmitted to a downstream prediction and planning module. Because the maximum number of obstacles is controlled, the maximum calculation amount of the perception link is controlled, thereby ensuring the processing stability of the perception link.

[0092] According to another aspect of the present disclosure, a device for obstacle perception of a vehicle is provided. As shown in Figure 9 The device 900 includes a first determination unit 910 configured to determine a three-dimensional bounding box and a category of each of a plurality of candidate obstacles around the vehicle, a second determination unit 920 configured to determine a two-dimensional bounding box of each of the plurality of candidate obstacles projected under a camera view angle of the vehicle based on the three-dimensional bounding box of each of the plurality of candidate obstacles, a filtering unit 930 configured to filter out occluded candidate obstacles of the same category based on the three-dimensional bounding box and the two-dimensional bounding box of each of the plurality of candidate obstacles to obtain a plurality of target obstacles, a third determination unit 940 configured to determine an importance confidence of each of the plurality of target obstacles in response to determining that the number of the plurality of target obstacles is greater than a preset number, and a number control unit 950 configured to retain a preset number of target obstacles with the highest importance confidence from the plurality of target obstacles.

[0093] It can be understood that the operations and effects of the units 910-950 can be referred to the description of steps S201-S205 in the method 1000 above, and will not be repeated here. Figure 2

[0094] ​According to some embodiments, the filtering unit can include: a first determining subunit configured to determine three-dimensional occlusion information of each of the plurality of candidate obstacles, wherein the three-dimensional occlusion information indicates an occlusion relationship between a three-dimensional bounding box of the corresponding candidate obstacle and three-dimensional bounding boxes of other candidate obstacles of the same category in a vehicle coordinate system; a second determining subunit configured to determine two-dimensional occlusion information of each of the plurality of candidate obstacles, wherein the two-dimensional occlusion information indicates an occlusion relationship between a two-dimensional bounding box of the corresponding candidate obstacle and two-dimensional bounding boxes of other candidate obstacles of the same category in a camera view; and a third determining subunit configured to determine, for each candidate obstacle, whether to filter out the candidate obstacle based on the three-dimensional occlusion information and the two-dimensional occlusion information of the candidate obstacle.

[0095] According to some embodiments, the first determining subunit can include: a projection subunit configured to project the three-dimensional bounding boxes of the plurality of candidate obstacles to a horizontal plane in the vehicle coordinate system to obtain position information and shape information of the plurality of candidate obstacles in the horizontal plane; a fourth determining subunit configured to determine, for two candidate obstacles of the same category, a first included angle of an overlapping part of the two candidate obstacles with respect to an origin of the vehicle coordinate system and a second included angle of the entire two candidate obstacles with respect to the origin of the vehicle coordinate system in the horizontal plane; and a fifth determining subunit configured to determine a three-dimensional occlusion rate of an occluded candidate obstacle of the two candidate obstacles based on the first included angle and the second included angle.

[0096] According to some embodiments, the first determining subunit can include: a first reservation subunit configured to, in response to determining that a same candidate obstacle has a plurality of three-dimensional occlusion rates, reserve a largest three-dimensional occlusion rate as the three-dimensional occlusion rate of the candidate obstacle.

[0097] According to some embodiments, the second determining subunit can include: a calculation subunit configured to, for two candidate obstacles of the same category, calculate an intersection over union of the two-dimensional bounding boxes of the two candidate obstacles in the camera view to determine a two-dimensional occlusion rate of an occluded candidate obstacle of the two candidate obstacles.

[0098] According to some embodiments, the second determining subunit can include: a second reservation subunit configured to, in response to determining that one candidate obstacle of the plurality of candidate obstacles has a plurality of two-dimensional occlusion rates, reserve a largest two-dimensional occlusion rate as the two-dimensional occlusion rate of the candidate obstacle.

[0099] According to some embodiments, the vehicle can have a plurality of cameras, the two-dimensional bounding box of each of the plurality of candidate obstacles can include a plurality of two-dimensional bounding boxes projected in a plurality of camera views, and the two-dimensional occlusion information of each of the plurality of obstacles can include a plurality of two-dimensional occlusion information corresponding to the plurality of camera views, respectively.

[0100] According to some embodiments, the third determining subunit can be configured to determine to filter out the candidate obstacle in response to determining that the three-dimensional bounding box of the candidate obstacle is occluded by the three-dimensional bounding boxes of other candidate obstacles of the same category in the vehicle coordinate system, and in response to determining that the two-dimensional bounding box of the candidate obstacle is occluded by other candidate obstacles of the same category in each camera view.

[0101] According to some embodiments, the importance confidence can comprise a distance confidence, the distance confidence being negatively correlated with a distance between the corresponding target obstacle and the vehicle.

[0102] According to some embodiments, the first determining unit can be configured to acquire, by using the perception model, the three-dimensional bounding box, the category, and a detection confidence of each of the plurality of candidate obstacles, the detection confidence indicating a degree to which the corresponding candidate obstacle is correctly detected, wherein the importance confidence comprises the detection confidence.

[0103] According to some embodiments, the importance confidence can comprise a speed confidence, the speed confidence being positively correlated with a predicted speed of the corresponding target obstacle.

[0104] According to some embodiments, the importance confidence can comprise an attention region confidence, the attention region confidence being determined based on whether the corresponding target obstacle is located in an attention region of the vehicle and whether the corresponding target obstacle is occluded.

[0105] In the technical solutions of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in the technical solutions comply with relevant laws and regulations and do not violate public order and good customs.

[0106] According to embodiments of the present disclosure, an electronic device, a readable storage medium, and a computer program product are also provided.

[0107] Reference Figure 10 A block diagram of the structure of an electronic device 1000 that can be a server or a client of the present disclosure will now be described, which is an example of a hardware device that can be applied to aspects of the present disclosure. The electronic device is intended to represent a wide variety of digital electronic computing devices, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computing devices. The electronic device can also represent a wide variety of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections, and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present disclosure described and / or claimed in this document.

[0108] As Figure 10As shown, the device 1000 includes a computing unit 1001 that can perform various appropriate actions and processes in accordance with a computer program stored in a read only memory (ROM) 1002 or a computer program loaded into a random access memory (RAM) 1003 from a storage unit 1008. Various programs and data required for the operation of the device 1000 can also be stored in the RAM 1003. The computing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other through a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0109] A plurality of components in the device 1000 are connected to the I / O interface 1005, including an input unit 1006, an output unit 1007, a storage unit 1008, and a communication unit 1009. The input unit 1006 can be any type of device that can input information to the device 1000, and can receive inputted digital or character information, and generate key signal inputs related to user settings and / or function controls of the electronic device, and can include, but is not limited to, a mouse, a keyboard, a touch screen, a track pad, a track ball, a joystick, a microphone, and / or a remote controller. The output unit 1007 can be any type of device that can present information, and can include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 1008 can include, but is not limited to, a magnetic disk, an optical disk. The communication unit 1009 allows the device 1000 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth TM A device, an 802.11 device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0110] The computing unit 1001 can be various general and / or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning network algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1001 performs various methods and processes described above, such as the obstacle perception method. For example, in some embodiments, the obstacle perception method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 1008. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 1000 via the ROM 1002 and / or the communication unit 1009. When the computer program is loaded onto the RAM 1003 and executed by the computing unit 1001, one or more steps of the obstacle perception method described above can be performed. Alternatively, in other embodiments, the computing unit 1001 can be configured to perform the obstacle perception method by any other suitable means, such as by means of firmware.

[0111] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0112] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces a means for implementing the functions / acts specified in the flowcharts and / or block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0113] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0114] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0115] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0116] The computer system can include clients and servers. This relationship can be remote, such that the servers are distributed across many clients. The relationship can also be such that the server is remote from the client. The servers can be cloud servers, also known as cloud computing servers or cloud hosts, which are a host product in the cloud computing service system to solve the defects of large management difficulty and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or simply "VPS"). The servers can also be servers of a distributed system, or servers combined with a blockchain.

[0117] It should be understood that the various forms of flow shown above can be reordered, steps added or removed. For example, the steps described in the present disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, which is not limited herein.

[0118] Although embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it should be understood that the above-described methods, systems and devices are merely exemplary embodiments or examples, and the scope of the present disclosure is not limited by these embodiments or examples, but only by the granted claims and their equivalent scope. Various elements in the embodiments or examples can be omitted or replaced by equivalent elements. In addition, each step can be performed in an order different from that described in the present disclosure. Further, various elements in the embodiments or examples can be combined in various ways. It is important that many of the elements described herein can be replaced by equivalent elements that appear after the present disclosure as technology evolves.

Claims

1. A method for obstacle perception for a vehicle, comprising: determining a three-dimensional bounding box and a category for each of a plurality of candidate obstacles around the vehicle, the vehicle having a plurality of cameras; determining a two-dimensional bounding box for each of the plurality of candidate obstacles under a camera view of the vehicle based on the three-dimensional bounding box for each of the plurality of candidate obstacles, the two-dimensional bounding box for each of the plurality of candidate obstacles comprising a plurality of two-dimensional bounding boxes under a plurality of camera views, and the two-dimensional occlusion information for each of the plurality of candidate obstacles comprising a plurality of two-dimensional occlusion information corresponding to the plurality of camera views respectively; filtering out occluded candidate obstacles of a same category based on the three-dimensional bounding box and the two-dimensional bounding box for each of the plurality of candidate obstacles to obtain a plurality of target obstacles, comprising: determining three-dimensional occlusion information for each of the plurality of candidate obstacles, wherein the three-dimensional occlusion information indicates an occlusion relationship between the three-dimensional bounding box of a corresponding candidate obstacle and the three-dimensional bounding box of other candidate obstacles of the same category in a vehicle coordinate system; determining two-dimensional occlusion information for each of the plurality of candidate obstacles, wherein the two-dimensional occlusion information indicates an occlusion relationship between the two-dimensional bounding box of a corresponding candidate obstacle and the two-dimensional bounding box of other candidate obstacles of the same category under the camera view; and for each of the candidate obstacles, filtering out the candidate obstacle in response to determining that the three-dimensional bounding box of the candidate obstacle is occluded by the three-dimensional bounding box of other candidate obstacles of the same category in the vehicle coordinate system, and in response to determining that the two-dimensional bounding box of the candidate obstacle is occluded by other candidate obstacles of the same category under each of the camera views; in response to determining that a number of the plurality of target obstacles is greater than a preset number, determining an importance confidence for each of the plurality of target obstacles; and retaining a preset number of target obstacles with the highest importance confidence among the plurality of target obstacles.

2. The method of claim 1, wherein, determining the three-dimensional occlusion information for each of the plurality of candidate obstacles, comprising: projecting the three-dimensional bounding box of the plurality of candidate obstacles to a horizontal plane in the vehicle coordinate system to obtain position information and shape information of the plurality of candidate obstacles in the horizontal plane; for two candidate obstacles of the same category, determining a first included angle of an overlapping part of the two candidate obstacles with respect to an origin of the vehicle coordinate system and a second included angle of the entirety of the two candidate obstacles with respect to the origin of the vehicle coordinate system in the horizontal plane; and based on the first included angle and the second included angle, determining a three-dimensional occlusion rate of an occluded candidate obstacle among the two candidate obstacles.

3. The method of claim 2, wherein, determining the three-dimensional occlusion information for each of the plurality of candidate obstacles, comprising: in response to determining that a same candidate obstacle has a plurality of three-dimensional occlusion rates, retaining a largest three-dimensional occlusion rate as the three-dimensional occlusion rate of the candidate obstacle.

4. The method of claim 1, wherein, determining the two-dimensional occlusion information for each of the plurality of candidate obstacles, comprising: for two candidate obstacles of the same category, calculating an intersection over union of the two-dimensional bounding boxes of the two candidate obstacles under the camera view to determine a two-dimensional occlusion rate of an occluded candidate obstacle among the two candidate obstacles.

5. The method of claim 4, wherein, determining the two-dimensional occlusion information of each of the plurality of candidate obstacles comprises: in response to determining that one of the plurality of candidate obstacles has a plurality of two-dimensional occlusion rates, retaining the largest two-dimensional occlusion rate as the two-dimensional occlusion rate of the one candidate obstacle.

6. The method of any one of claims 1-5, wherein, the importance confidence comprises: a distance confidence, which is negatively correlated with the distance between the corresponding target obstacle and the vehicle.

7. The method of any one of claims 1-5, wherein, determining the three-dimensional bounding box and the category of each of the plurality of candidate obstacles around the vehicle comprises: obtaining the three-dimensional bounding box, the category and the detection confidence of each of the plurality of candidate obstacles using a perception model, the detection confidence indicating the degree to which the corresponding candidate obstacle is correctly detected, wherein the importance confidence comprises the detection confidence.

8. The method according to any one of claims 1-5, wherein, the importance confidence comprises: a speed confidence, which is positively correlated with the predicted speed of the corresponding target obstacle.

9. The method of any one of claims 1-5, wherein, the importance confidence comprises: an attention region confidence, which is determined based on whether the corresponding target obstacle is located in an attention region of the vehicle and whether the corresponding target obstacle is occluded.

10. An obstacle perception device for a vehicle, comprising: a first determination unit configured to determine the three-dimensional bounding box and the category of each of the plurality of candidate obstacles around the vehicle, the vehicle having a plurality of cameras; a second determination unit configured to determine the two-dimensional bounding box of each of the plurality of candidate obstacles projected under the camera view of the vehicle based on the three-dimensional bounding box of each of the plurality of candidate obstacles, the two-dimensional bounding box of each of the plurality of candidate obstacles comprising a plurality of two-dimensional bounding boxes projected under a plurality of camera views, and the two-dimensional occlusion information of each of the plurality of candidate obstacles comprising a plurality of two-dimensional occlusion information corresponding to the plurality of camera views respectively; a filtering unit configured to filter out occluded candidate obstacles of the same category based on the three-dimensional bounding box and the two-dimensional bounding box of each of the plurality of candidate obstacles to obtain a plurality of target obstacles; a third determination unit configured to determine the importance confidence of each of the plurality of target obstacles in response to determining that the number of the plurality of target obstacles is greater than a preset number; and a number control unit configured to retain the preset number of target obstacles with the highest importance confidence among the plurality of target obstacles, wherein the filtering unit comprises: a first determination sub-unit configured to determine the three-dimensional occlusion information of each of the plurality of candidate obstacles, wherein the three-dimensional occlusion information indicates the occlusion relationship between the three-dimensional bounding box of the corresponding candidate obstacle and the three-dimensional bounding box of other candidate obstacles of the same category in the vehicle coordinate system; a second determination sub-unit configured to determine the two-dimensional occlusion information of each of the plurality of candidate obstacles, wherein the two-dimensional occlusion information indicates the occlusion relationship between the two-dimensional bounding box of the corresponding candidate obstacle and the two-dimensional bounding box of other candidate obstacles of the same category under the camera view; and ​ The filtering subunit is configured to filter out, for each of the candidate obstacles, the candidate obstacle in response to determining that the three-dimensional bounding box of the candidate obstacle is occluded by three-dimensional bounding boxes of other candidate obstacles of the same category in the vehicle coordinate system, and in response to determining that the two-dimensional bounding box of the candidate obstacle is occluded by two-dimensional bounding boxes of other candidate obstacles of the same category in each of the camera perspectives.

11. The apparatus of claim 10, wherein, The first determining subunit includes: The projecting subunit is configured to project the three-dimensional bounding boxes of the plurality of candidate obstacles onto a horizontal plane in the vehicle coordinate system to obtain position information and shape information of the plurality of candidate obstacles on the horizontal plane; The fourth determining subunit is configured to determine, for two candidate obstacles of the same category, a first included angle of an overlapping part of the two candidate obstacles with respect to an origin of the vehicle coordinate system and a second included angle of the entirety of the two candidate obstacles with respect to the origin of the vehicle coordinate system on the horizontal plane; and The fifth determining subunit is configured to determine a three-dimensional occlusion rate of an occluded candidate obstacle of the two candidate obstacles based on the first included angle and the second included angle.

12. The apparatus of claim 11, wherein, The first determining subunit includes: The first retaining subunit is configured to retain, in response to determining that a same candidate obstacle has a plurality of three-dimensional occlusion rates, a largest three-dimensional occlusion rate as the three-dimensional occlusion rate of the candidate obstacle.

13. The apparatus of claim 10, wherein, The second determining subunit includes: The calculating subunit is configured to calculate, for two candidate obstacles of the same category, an intersection over union of two-dimensional bounding boxes of the two candidate obstacles in the camera perspective to determine a two-dimensional occlusion rate of an occluded candidate obstacle of the two candidate obstacles.

14. The apparatus of claim 13, wherein, The second determining subunit includes: The second retaining subunit is configured to retain, in response to determining that one of the plurality of candidate obstacles has a plurality of two-dimensional occlusion rates, a largest two-dimensional occlusion rate as the two-dimensional occlusion rate of the one candidate obstacle.

15. The apparatus of any of claims 10-14, wherein, The importance confidence includes: A distance confidence, the distance confidence being negatively correlated with a distance of a corresponding target obstacle from the vehicle.

16. The apparatus of any one of claims 10-14, wherein, The first determining unit is configured to acquire, by using a perception model, the three-dimensional bounding box, the category, and a detection confidence of each of the plurality of candidate obstacles, the detection confidence indicating a degree to which a corresponding candidate obstacle is correctly detected, wherein the importance confidence includes the detection confidence.

17. The apparatus of any one of claims 10-14, wherein, The importance confidence includes: A speed confidence, the speed confidence being positively correlated with a predicted speed of a corresponding target obstacle.

18. The apparatus of any one of claims 10-14, wherein, The importance confidence includes: An attention region confidence, the attention region confidence being determined based on whether a corresponding target obstacle is located in an attention region of the vehicle and whether the corresponding target obstacle is occluded.

19. An electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-9.

20. A vehicle comprising the electronic device of claim 19.

21. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are for causing a computer to perform the method of any one of claims 1-9.

22. A computer program product comprising a computer program, wherein, The computer program, when executed by a processor, implements the method of any one of claims 1-9.

Images