Obstacle detection method, vehicle control method, device, and medium

Abstract

The present disclosure relates to an obstacle detection method, a vehicle control method, a device and a medium. The method comprises: obtaining a first detection result of obstacle detection based on first perception data collected by a movable device, and a second detection result of obstacle detection based on second perception data collected by the movable device; wherein a first sampling time of the first perception data is before a second sampling time of the second perception data, the first detection result comprises a first detection object and a first confidence that the first detection object is a set obstacle; matching the first detection object in the second detection result to obtain a matching result; and adjusting the first confidence according to the matching result to obtain a second confidence that the first detection object is the set obstacle.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of intelligent driving, and more particularly, to an obstacle detection method, a vehicle control method, an electronic device, a chip, a vehicle, and a readable storage medium. BACKGROUND

[0002] In the automatic driving technology, obstacle detection can be performed based on the perception data of various sensors on the vehicle, and then the vehicle is controlled to perform decision planning and control based on the obstacle detection result. The accuracy of the obstacle detection result is a key to the stability and reliability of automatic driving, and therefore, it is necessary to provide a scheme for improving the accuracy of the obstacle detection result. SUMMARY

[0003] In view of this, the present disclosure proposes a new technical scheme for obstacle detection, which can improve the accuracy of the detection result.

[0004] According to a first aspect of an embodiment of the present disclosure, an obstacle detection method is provided, comprising:

[0005] obtaining a first detection result of obstacle detection based on first perception data collected by a movable device, and a second detection result of obstacle detection based on second perception data collected by the movable device; wherein a first sampling time of the first perception data is before a second sampling time of the second perception data, the first detection result comprises a first detection object and a first confidence that the first detection object is a set obstacle;

[0006] matching the first detection object in the second detection result to obtain a matching result;

[0007] adjusting the first confidence according to the matching result to obtain a second confidence that the first detection object is the set obstacle; wherein the second confidence is a confidence corresponding to the second sampling time.

[0008] Optionally, the adjusting the first confidence according to the matching result to obtain the second confidence that the first detection object is the set obstacle comprises:

[0009] in a case where the matching result indicates that there is a target detection object in the second detection result that matches the first detection object, increasing the first confidence by a first adjustment value to obtain the second confidence; or

[0010] in a case where the matching result indicates that there is no target detection object in the second detection result that matches the first detection object, decreasing the first confidence by a second adjustment value to obtain the second confidence.

[0011] Optionally, the matching the first detection object with the second detection object in the second detection result comprises:

[0012] querying the second detection object meeting a set condition in the second detection result, wherein the set condition is set based on at least one of a category and a position of the first detection object;

[0013] matching the first detection object with the second detection object to obtain a matching result of the first detection object for the second detection object.

[0014] Optionally, the movable device is provided with a camera and an ultrasonic probe, the first perception data comprises first visual perception data collected by the camera, the second perception data comprises ultrasonic perception data collected by the ultrasonic probe, the first detection result is determined based on the first visual perception data, and the second detection result is determined based on the ultrasonic perception data.

[0015] Optionally, the first detection result comprises a first position of the first detection object in a set coordinate system determined based on the first visual perception data, and the second detection result comprises a second position of the second detection object in the set coordinate system determined based on the ultrasonic perception data.

[0016] The matching the first detection object with the second detection object to obtain a matching result comprises:

[0017] determining a first distance between the first detection object and the second detection object according to the first position and the second position;

[0018] obtaining the matching result according to the first distance.

[0019] Optionally, the matching the first detection object with the second detection object to obtain a matching result comprises:

[0020] determining whether the first detection object is an object that can be detected by the ultrasonic probe according to a relative position between the first detection object and the ultrasonic probe; and matching the first detection object with the second detection object to obtain a matching result in a case that the ultrasonic probe can detect the first detection object.

[0021] Optionally, the matching the first detection object with the second detection object to obtain a matching result comprises:

[0022] obtaining an angular deviation of the first detection object relative to the ultrasonic probe;

[0023] In a case where the angle deviation is within a set threshold range, the first detection object is matched with the second detection object to obtain a matching result.

[0024] Optionally, the first detection result includes a first position of the first detection object in a set coordinate system determined based on the first visual perception data, and the second detection result includes a perception distance of the ultrasonic probe determined based on the ultrasonic perception data.

[0025] The matching of the first detection object with the second detection object to obtain a matching result includes:

[0026] According to the first position, a position of the movable device in the set coordinate system, and a mounting position of the ultrasonic probe on the movable device, a second distance between the first detection object and the ultrasonic probe is determined.

[0027] The matching result is obtained based on the second distance and the perception distance.

[0028] Optionally, the movable device is provided with a camera, the first perception data includes first visual perception data collected by the camera, the second perception data includes second visual perception data collected by the camera, the first detection result is determined based on the first visual perception data, and the second detection result is determined based on the second visual perception data.

[0029] The matching of the first detection object with the second detection object to obtain a matching result includes:

[0030] A first contour region of the first detection object in a set coordinate system and a second contour region of the second detection object in the set coordinate system are obtained.

[0031] According to the first contour region and the second contour region, a matching result is obtained.

[0032] Optionally, the second detection result is multiple, and the second confidence degree is multiple confidence degrees obtained by matching the first detection result with each of the second detection results respectively.

[0033] According to the second confidence degree and a matching weight corresponding to each of the multiple second detection results, a confidence degree that the first detection object is the set obstacle is obtained.

[0034] According to a second aspect of the embodiments of the present disclosure, a vehicle control method is provided, including:

[0035] A second confidence degree that a first detection object is a set obstacle is obtained.

[0036] According to the second confidence, a vehicle is controlled to perform an obstacle avoidance strategy.

[0037] The second confidence is obtained according to a first confidence of the first detected object being the set obstacle, based on a matching result obtained by matching the first detected object in the first detection result in second detection result, the second detection result being obtained by obstacle detection based on second perception data collected by the movable device, the first detection result being obtained by obstacle detection based on first perception data collected by the movable device, a first sampling time of the first perception data being before a second sampling time of the second perception data, the first confidence being a confidence corresponding to the first sampling time, and the second confidence being a confidence corresponding to the second sampling time.

[0038] According to a third aspect of embodiments of the present disclosure, an electronic device is provided, comprising a memory and a processor,

[0039] The memory is configured to store computer instructions, and the processor is configured to invoke the computer instructions from the memory to execute the method according to any one of the first aspect or the second aspect.

[0040] According to a fourth aspect of embodiments of the present disclosure, a chip is provided, comprising:

[0041] a storage unit configured to store a computer program; and

[0042] a processing unit configured to implement the method according to any one of the first aspect or the second aspect when executing the computer program stored in the storage unit.

[0043] According to a fifth aspect of embodiments of the present disclosure, a vehicle is provided, comprising a memory and a processor, the memory is configured to store computer instructions, and the processor is configured to invoke the computer instructions from the memory to execute the method according to any one of the first aspect or the second aspect.

[0044] According to a sixth aspect of embodiments of the present disclosure, a non-volatile computer readable storage medium is provided, having computer program instructions stored thereon, the computer program instructions being executed by a processor to implement the method according to any one of the first aspect or the second aspect.

[0045] According to embodiments of this disclosure, a first detection result for obstacle detection based on first sensing data collected by a mobile device, and a second detection result for obstacle detection based on second sensing data collected by the mobile device are obtained. A first detection object from the first detection result is matched with the second detection result to obtain a matching result. A first confidence level that the first detection object is a designated obstacle at a first sampling time is adjusted based on the matching result to obtain a second confidence level that the first detection object is the designated obstacle at a second sampling time. Embodiments of this disclosure enable real-time dynamic adjustment of the confidence level that the detection object is a designated obstacle, thereby improving the accuracy of obstacle detection results.

[0046] Other features and advantages of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0047] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments of the present disclosure and, together with their description, serve to explain the principles of the present disclosure.

[0048] Figure 1 This is a schematic diagram of an intelligent connected system to which the methods provided in the embodiments of this disclosure can be applied.

[0049] Figure 2 It is based on Figure 1 The illustrated embodiment provides a schematic diagram of the composition structure of a vehicle.

[0050] Figure 3 This is a schematic flowchart of an obstacle detection method provided in an embodiment of this disclosure.

[0051] Figure 4 This is a schematic diagram of the probe detection angle range provided in the embodiments of this disclosure.

[0052] Figure 5 This is a schematic diagram of the vehicle control method provided in the embodiments of this disclosure.

[0053] Figure 6 This is a schematic diagram of the composition structure of an electronic device provided in an embodiment of this disclosure.

[0054] Figure 7 This is a schematic diagram of the composition structure of another vehicle provided in an embodiment of this disclosure.

[0055] Figure 8 This is a schematic diagram of the composition structure of a chip provided in an embodiment of this disclosure. Detailed Implementation

[0056] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. Note that the relative arrangement of the components and steps illustrated in these embodiments, numerical expressions, and numerical values are merely examples for illustrating the present disclosure and do not limit the scope of the present disclosure unless specifically stated otherwise.

[0057] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the scope of the present disclosure and its applications or uses.

[0058] Techniques, methods, and devices known to those of ordinary skill in the relevant art can not be discussed in detail herein, but should be considered as part of the description if appropriate.

[0059] In all of the examples shown and discussed herein, any specific values should be interpreted as merely illustrative and not as a limitation of the exemplary embodiments. Thus, other examples of the exemplary embodiments can have different values.

[0060] Note that like reference numerals and letters indicate like items in the accompanying drawings and, as such, once an item is defined in one drawing, that

[0061] To facilitate understanding of the method of the embodiments of the present disclosure, the application scenario of the embodiments of the present disclosure is first described.

[0062] Figure 1 FIG. 1 is a schematic diagram of an intelligent network contact system 100 to which the method provided by the embodiments of the present disclosure can be applied. As shown in FIG. 1, the intelligent network contact system 100 can include a movable device 101, a server 102, and a user terminal 103. Figure 1

[0063] In some examples, the movable device 101 is, for example, a vehicle with an automatic driving function. The automatic driving, also referred to as unmanned driving or intelligent driving, is a vehicle with an automatic driving function that can realize driving tasks such as environmental perception, decision planning, and control execution. The level of automatic driving can refer to the intelligent vehicle classification standard formulated by the Society of Automotive Engineers (SAE), for example, L0 level is manual driving, L1 is assisted driving, L2 is partial automatic driving, L3 is conditional automatic driving, L4 is high automatic driving, and L5 is complete automatic driving. The above classification method of the level of automatic driving is merely an example, and the embodiments of the present disclosure do not limit the classification standard and level of automatic driving.

[0064] ​In some examples, the server 102 can be a single server or a distributed server cluster composed of multiple servers, which can be deployed in a local server or a cloud server. The server 102 can communicate with the movable device 101 and / or the user terminal 103 based on a communication network, and provide various services for the movable device 101 and / or the user terminal 103. For example, the server can receive perception data sent by a vehicle, and provide services such as high-definition map, data analysis, decision planning, etc. for the vehicle. For another example, the server can receive query instructions or control instructions sent by a user terminal, and provide corresponding services for the user.

[0065] In some examples, the user terminal 103 can be any form of electronic device providing services for users, such as a personal computer, a notebook computer, a smart tablet, a smart phone, a smart wearable device, etc. The user can interact with the vehicle or the server through a human-computer interaction terminal configured by the movable device 101, or interact with the vehicle or the server through the user terminal 103. For example, the user can query the status and / or parameters of the vehicle through the user terminal, or control the vehicle to perform a set task and / or modify a configuration parameter, etc. The user terminal runs an application based on the intelligent network system to realize the interaction with the vehicle or the server. The application can be a local application, or a web application or a mini-program, etc., which is not limited here.

[0066] In some examples, the above-mentioned application running on the user terminal can provide authentication or authorization services for the user. The user who successfully passes the authentication and is granted corresponding permissions can query and / or control the vehicle within the scope of the granted permissions.

[0067] The movable device 101, the server 102 and the user terminal 103 can communicate through a communication link provided by the communication network 104. The communication network 104 can include one or more networks of any type. For example, the communication network 104 can include an Internet, a local area network (LAN), a wide area network (WAN), a virtual private network (VPN), a public switched telephone network (PSTN), a satellite communication network, Wi-Fi, 2G, 3G, 4G, 5G, 6G, NB-IoT, eMTC, infrared, Bluetooth, NFC, etc. to provide communication, or a combination of the above networks. The communication network between the movable device 101 and the server 102, between the user terminal 103 and the server 102, and between the user terminal 103 and the movable device 101 can be the same or different.

[0068] It should be noted that, Figure 1The structure of the intelligent network contact system 100 shown in the figure is only schematic, and the intelligent network contact system in the embodiments of the present disclosure is not limited to the above structure, and can further include more or fewer devices, or the devices can be combined or split. For example, the intelligent network contact system can also not include user terminals and / or servers. For another example, the user terminals and servers can be combined and deployed.

[0069] Figure 2 According to Figure 1 A schematic diagram of a movable device 101 is provided according to the embodiments shown in the figure. As shown in the figure, Figure 2 The movable device 101 can include a perception component 1011, a computing platform 1012, an execution component 1013, etc. The perception component 1011, the computing platform 1012 and the execution component 1013 can be connected through a bus or other means.

[0070] In some examples, the perception component 1011 can be used to collect information of the vehicle itself or the outside, and the perception component 1011 can include at least one of a visual sensor unit, a radar, a positioning navigation unit, an inertial measurement unit (IMU) or other sensor units, wherein the visual sensor unit can include one or more cameras, the radar can include at least one of a laser radar, a millimeter wave radar, an ultrasonic radar or other radars, and the positioning navigation unit can include at least one of a GPS system, a Beidou system or other global positioning systems. The visual sensor unit can be used to collect visual images, and the ultrasonic radar can be used to collect detection information of an ultrasonic probe.

[0071] In some examples, the computing platform 1012 can include a device with computing capability, which is used to process the perception data collected by the perception component 1011 to obtain control information, and send corresponding control instructions to the execution component 1013, so that the execution component 1013 performs corresponding actions, thereby realizing the control of the movable device 101. For example, the computing platform 1012 can perform behaviors such as positioning mapping (SLAM), path planning, behavior decision, etc. of the vehicle, thereby realizing the autonomous control of the vehicle.

[0072] The computing platform 1012 can include at least one processor and at least one memory, each processor can execute instructions stored in the memory to implement the method provided by the embodiments of the present disclosure. The processor in the embodiments of the present disclosure can include at least one of a central processing unit (CPU), a graphic processing unit (GPU), a field programmable gate array (FPGA), a system on chip (SOC), an application specific integrated circuit (ASIC), a micro controller unit (MCU) or other processors. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. In addition to storing instructions, the memory can also store data, such as scene map, path information, position, direction and speed of the vehicle. The data stored in the memory can be obtained and used by the processor.

[0073] In some examples, the computing platform of the vehicle can independently perform the computing task, or can complete the computing task by communicating with the server. For example, the computing platform of the vehicle can cooperate with the server to complete the corresponding computing task.

[0074] The computing platform 1012 can be arranged in the movable device 101, and part or all of the computing platform 1012 can also be arranged in the server corresponding to the vehicle. For example, part of the computing platform 1012 with high real-time requirement is arranged in the vehicle, and the other part of the computing platform 1012 with low real-time requirement is arranged in the server corresponding to the vehicle.

[0075] In some examples, the execution component 1013 is configured to perform corresponding actions based on the control of the computing platform 1012, so that the movable device 101 completes the moving task. The execution component 1013 can include, for example, a power component, a brake component, a transmission component, a steering component, etc.

[0076] It should be noted that, Figure 2The structure of the mobile device 101 shown in the figure is only illustrative, and the vehicle in the embodiments of the present disclosure is not limited to the above structure, and can further include more or fewer components as needed, or the device can be combined or split. For example, the vehicle can not include the computing platform described above. For another example, the vehicle can further include a communication component, an interface component, a multimedia component, an input component, an output component, and the like.

[0077] In the automatic driving technology, obstacle detection can be performed based on the perception data of various sensors on the vehicle, and then the vehicle is controlled to perform decision planning and control based on the obstacle detection result. Among them, the accuracy of the obstacle detection result is the key to the stability and reliability of automatic driving, therefore, it is necessary to provide a scheme for improving the accuracy of the obstacle detection result. For example, the vehicle emergency escape scene can control the vehicle to perform an obstacle avoidance strategy based on the obstacle detection result to avoid or reduce the occurrence of vehicle collision phenomenon.

[0078] In the related art, the obstacle detection result is obtained by collecting perception data of the vehicle, and when the confidence that the detection result indicates that the detection object is an obstacle is greater than a confidence threshold, the vehicle is controlled to perform an obstacle avoidance strategy. However, the obstacle detection result obtained based on only a single frame of perception data is prone to misjudgment.

[0079] Based on this, the embodiments of the present disclosure provide an obstacle detection method. The embodiments of the present disclosure obtain a first detection result based on first perception data collected at a first sampling time, and obtain a second detection result based on second perception data collected at a second sampling time later than the first sampling time. The first detection result and the second detection result are matched, and the confidence corresponding to the first detection result is dynamically adjusted according to the matching result, thereby improving the accuracy of the obstacle detection result.

[0080] As shown in FIG. 1, Figure 3 The obstacle detection method of the embodiments of the present disclosure can include steps S110 to S130.

[0081] In step S110, a first detection result of obstacle detection based on first perception data collected by a mobile device and a second detection result of obstacle detection based on second perception data collected by the mobile device are obtained; wherein the first detection result includes a first detection object and a first confidence that the first detection object is a set obstacle.

[0082] The first sampling time of the first perception data in the present embodiment is before the second sampling time of the second perception data. In one example, the first perception data can be perception data collected in a historical frame, and the second perception data can be perception data collected in a current frame.

[0083] The first perception data in this embodiment can be visual perception data collected by a camera of a visual perception unit, ultrasonic perception data collected by an ultrasonic radar probe, or perception data collected by other sensing units.

[0084] The second perception data in this embodiment can be visual perception data collected by a camera of a visual perception unit, ultrasonic perception data collected by an ultrasonic radar probe, or perception data collected by other sensing units.

[0085] The set obstacle in this embodiment is an obstacle of a set type. In one example, the set obstacle can be one of a vehicle, a pedestrian, a cone, a building, etc.

[0086] In some examples, the obstacle detection based on the first perception data collected by the camera includes performing dynamic obstacle detection and static obstacle detection based on the first perception data, respectively. In one example, static detection objects and dynamic detection objects can be distinguished by analyzing absolute position coordinates of the same detection object in adjacent sampling times. In another example, detection objects that are likely to be obstacles can be identified by a machine learning algorithm (such as a convolutional neural network in deep learning), and the detection objects are classified to distinguish static detection objects and dynamic detection objects. After the obstacle detection based on the first perception data, a first detection result is obtained. The first detection result in this embodiment can include a first detection object and a first confidence that the first detection object is a set obstacle.

[0087] The confidence of the detection object being a set obstacle in this embodiment refers to the probability of the detection object being an obstacle of a set type. In one example, the confidence can be a floating point number between 0 and 1. The initial value of the first confidence can be a preset value, for example, 0. The first confidence in this embodiment is a confidence corresponding to the first sampling time.

[0088] It can be understood that the first detection result in this embodiment is a historical detection result, which can be stored in a maintenance table.

[0089] In this embodiment, the obstacle detection based on the second perception data can not detect potential obstacles, or can detect potential obstacles. That is, there can be no detection object in the second detection result in this embodiment, or there can be a detection object. When there is a detection object in the second detection result, the second detection result can only include the detection object, or the second detection result can include a second detection object and a confidence that the second detection object is a set obstacle.

[0090] In step S120, the first detection object is matched in the second detection result to obtain a matching result.

[0091] The second detection result can include second detection objects. In some examples, the second detection objects can be any detection objects obtained by obstacle detection based on the second perception data. In this example, step S120 can include matching the first detection objects with each of the second detection objects in the second detection result to obtain a matching result of the first detection objects for the second detection objects. That is, all the detection objects in the second detection result can be matched.

[0092] In other examples, step S120 can also include steps S121-S122.

[0093] Step S121: querying, in the second detection result, a second detection object that meets a set condition.

[0094] Step S122: matching the first detection objects with the second detection object to obtain a matching result of the first detection objects for the second detection object.

[0095] The set condition in this embodiment can be set based on at least one of a category and a position of the first detection objects. By screening the detection objects in the second detection result based on at least one of the type and the position of the first detection objects, the second detection object that meets the set condition is obtained, and then the second detection object that meets the set condition is matched with the first detection objects, which can improve the matching speed and the matching efficiency.

[0096] The matching result in this embodiment includes a target detection object that does not exist in the second detection result and matches the first detection objects, and a target detection object that exists in the second detection result and matches the first detection objects. In some examples, there is no detection object in the second detection result. In the case where there is no detection object in the second detection result, it is determined that there is no target detection object in the second detection result that matches the first detection objects. In other examples, there is a detection object in the second detection result. In the case where there is a detection object in the second detection result, the first detection objects are matched in the second detection result, and it is determined whether there is a target detection object in the second detection result that matches the first detection objects.

[0097] In some examples, there are multiple second detection objects that meet the set condition in the second detection result. The first detection objects are matched with each of the second detection objects to obtain multiple candidate matching results, and a final matching result is determined according to each candidate matching result. In one example, the second detection object with the highest matching degree in all the candidate matching results can be taken as the target detection object.

[0098] In some examples, when there is a second detection object in the second detection result that does not match any of the first detection objects, the second detection object and its confidence are added to the maintenance table for subsequent matching.

[0099] At step S130, the first confidence is adjusted according to the matching result to obtain a second confidence that the first detection object is the set obstacle; the second confidence is a confidence corresponding to the second sampling time.

[0100] In some examples, step S130 can include step S131 or step S132.

[0101] At step S131, in a case where the matching result indicates that there is a target detection object matching the first detection object in the second detection result, the first confidence is increased by a first adjustment value to obtain the second confidence.

[0102] At step S132, in a case where the matching result indicates that there is no target detection object matching the first detection object in the second detection result, the first confidence is decreased by a second adjustment value to obtain the second confidence.

[0103] The first adjustment value or the second adjustment value can be a pre-set fixed value. In some examples, the second detection result can only include the second detection object. In this case, it is assumed that a final obstacle detection result needs to be obtained within 10 frames, and the first confidence is a floating point number between 0 and 1. The first adjustment value or the second adjustment value can be, for example, 0.1.

[0104] The first adjustment value or the second adjustment value can also be a dynamically changing value. In some examples, the second detection result can include the second detection object and a confidence that the second detection object is the set obstacle at the second sampling time. The first adjustment value or the second adjustment value can be set based on the confidence that the second detection object is the set obstacle in the second detection result. In one example, the confidence that the second detection object is the set obstacle at the second sampling time is P1, and the first adjustment value or the second adjustment value can be, for example, a product of P1 and a pre-set fixed value.

[0105] The first adjustment value and the second adjustment value in the present embodiment can be the same or different.

[0106] The adjustment in the present embodiment is a correction based on the first confidence to obtain a more accurate confidence. The correction process is actually a fine tuning rather than a complete replacement.

[0107] In one example, when the second confidence obtained by adjusting the first confidence is greater than 1, the second confidence is set to a pre-set maximum value (such as 1). When the second confidence obtained by adjusting the first confidence is less than 0, the second confidence is set to a pre-set minimum value (such as 0).

[0108] Embodiments of the present disclosure match the first detection object in the second detection result, dynamically adjust the first confidence that the first detection object is the set obstacle based on the matching result, and obtain the second confidence that the first detection object is the set obstacle. By dynamically adjusting the first confidence that the first detection object is the set obstacle in real time at different sampling times, a more accurate second confidence can be obtained at each sampling time, thereby improving the accuracy of the obstacle detection result.

[0109] In related technologies, there are schemes for detecting obstacles through visual perception data. However, the data collected by the visual sensor is easily affected by the environment such as weather and light, resulting in false detection or inaccurate distance measurement.

[0110] Based on this, in some embodiments, the movable device is provided with a camera. The second perception data can include second visual perception data collected by the camera. The visual perception data is an image collected by the camera.

[0111] In some examples, the first detection object in the present embodiment includes dynamic potential obstacles and / or static potential obstacles. The image can be subjected to dynamic potential obstacle detection and potential impassable area detection, respectively. For dynamic potential obstacles, a first visual perception result determined based on the first visual perception data of the first sampling time can be used to predict a second visual perception result of the second sampling time according to the tracker principle, and a third visual perception result determined based on the second visual perception data of the second sampling time can be combined to obtain a first target visual perception result of the second sampling time. For potential impassable areas, a fourth visual perception result determined based on the first visual perception data and a fifth visual perception result determined based on the second visual perception data can be superimposed to obtain a second target visual perception result of the potential impassable area at the second sampling time. By temporally fusing visual perception results to obtain the visual perception result at the current sampling time, the accuracy of the visual perception result can be improved.

[0112] The visual perception result in the present embodiment can include dynamic potential obstacle detection results and potential impassable area detection results. The dynamic potential obstacle detection result can include at least one of the following: a timestamp, an ID of the detection object, a type of the detection object, a confidence that the detection object is of the type, a motion state of the detection object, an angle deviation of the detection object from the ego vehicle, and a detection box of the detection object. The motion state of the detection object can include at least one of the following: a longitudinal relative position, a lateral relative position, a longitudinal relative speed, a lateral relative speed, a longitudinal acceleration, and a lateral acceleration. The potential impassable area detection result includes a map with a fixed length and width, and a probability of existence of an obstacle and a corresponding obstacle type at each grid position in the map.

[0113] In this embodiment, the dynamic potential obstacle detection result and the potential impassable area detection result are fused to obtain a fused impassable area including dynamic potential obstacles and static potential obstacles. The profile of the potential obstacles in the fused passable area can be extracted. Each potential obstacle result can include at least one of the following: a timestamp, an ID of the first detection object, a type of the first detection object, a confidence degree of the first detection object being the type, a motion state of the first detection object, an angle deviation of the first detection object from the ego vehicle, and a profile point set of the first detection object.

[0114] In the related art, there is also a scheme for obstacle detection through ultrasonic sensing data, but the accuracy of the detection scheme for judging the height and specific position of the obstacle is not enough. For example, when the ultrasonic sensor senses a low curb or a road joint, a strong echo is generated, resulting in a misjudgment of the height. For another example, the position of the obstacle is determined by the triangulation method, and since multiple reflections of the ultrasonic waves are involved, the detection result deviates. These deviations will cause the automatic emergency braking to be triggered incorrectly, thereby affecting the passenger's ride experience.

[0115] Based on this, in some embodiments, the movable device is provided with an ultrasonic probe. The second sensing data can include ultrasonic sensing data collected by the ultrasonic probe. The ultrasonic sensing data is the detection information of the ultrasonic probe. In some examples, the detection information can include echo distance, echo height, echo width, echo confidence, wave-emitting probe, and the like.

[0116] When the wave emitted by one probe is received by itself, it is called a direct echo. Through the direct echo distance, it can be determined that there is a potential obstacle on a circle with the installation position of the probe as the center and the echo distance as the radius. When the wave emitted by one probe is received by another probe, it is called an indirect echo. Through the indirect echo distance, it can be determined that there is a potential obstacle on an ellipse with the transmitting-receiving probe as the focus and the echo distance as the major axis. When the same wave is received by different probes, the intersection of the ellipse and the circle can be obtained through the triangulation algorithm, i.e., the position point of the potential obstacle.

[0117] Since the triangulation algorithm involves multiple probe echoes, the position of the obstacle is obtained in an implicit way. A slight deviation in the detection distance of the direct echo or the indirect echo will cause a large deviation in the final result position. For automatic emergency braking, false positives caused by inaccurate positions will greatly affect the passenger's experience. Therefore, after triangulation, the position points of the obstacles determined by the historical several frames of ultrasonic sensing data are clustered in density. Only the position points surrounded by multiple triangulation results are considered as valid position points of the obstacles. By fusing the ultrasonic sensing results in time sequence to obtain the ultrasonic sensing result at the current sampling time, the accuracy of the ultrasonic sensing result can be improved.

[0118] In some embodiments, the movable device is provided with a camera and an ultrasonic probe. The first perception data comprises first visual perception data collected by the camera. The second perception data comprises ultrasonic perception data collected by the ultrasonic probe. The first detection result is determined based on the first visual perception data, and the second detection result is determined based on the ultrasonic perception data.

[0119] In the embodiment, the matching result can be obtained by matching the first detection object determined based on the first visual perception data in the second detection result determined based on the ultrasonic perception data. That is, the visual perception result at the first sampling time is matched with the ultrasonic perception result at the second sampling time to obtain the matching result. Then, the first confidence that the first detection object at the first sampling time is a set obstacle is adjusted according to the matching result. In this way, the ultrasonic perception result and the visual perception result can be fused, so as to further improve the accuracy of the matching result.

[0120] In some examples, the first detection result can include a first position of a first detection object in a set coordinate system determined based on the first visual perception data, and the second detection result can include a second position of a second detection object in the set coordinate system determined based on the ultrasonic perception data.

[0121] The set coordinate system in the embodiment can be a self-vehicle coordinate system. The self-vehicle coordinate system can be an initial coordinate system when the self-vehicle is turned on, or a self-vehicle coordinate system at a certain specific time.

[0122] In some examples, determining the first position of the first detection object in the set coordinate system based on the first visual perception data can include: determining a set of contour points of the first detection object in the set coordinate system based on the first visual perception data; and determining the first position of the first detection object based on the set of contour points.

[0123] In some examples, determining the second position of the second detection object in the set coordinate system based on the ultrasonic perception data can include: determining an ultrasonic wave point position of the second detection object in the set coordinate system based on the ultrasonic perception data; and taking the ultrasonic wave point position as the second position of the second detection object.

[0124] Optionally, the step S122 can comprise obtaining the matching result according to the first position and the second position. In some examples, the first position can be a position where a contour corresponding to the set of contour points of the first detected object is located, and the second position can be a position of the ultrasonic wave point corresponding to the second detected object. In a case where the ultrasonic wave point is located within the contour corresponding to the set of contour points of the first detected object, it is determined that the first detected object and the second detected object match. In a case where the ultrasonic wave point is located within a preset range outside the contour corresponding to the set of contour points of the first detected object, it is determined that the first detected object and the second detected object match successfully. In a case where the ultrasonic wave point is located outside the preset range outside the contour corresponding to the set of contour points of the first detected object, it is determined that the first detected object and the second detected object match unsuccessfully.

[0125] Optionally, the step S122 can comprise determining a first distance between the first detected object and the second detected object according to the first position and the second position, and obtaining the matching result according to the first distance. In some examples, in a case where the first distance is less than a threshold value, it is determined that the second detected object matches the first detected object successfully; and in a case where the first distance is greater than or equal to the threshold value, it is determined that the second detected object matches the first detected object unsuccessfully.

[0126] Considering that the ultrasonic probe can only detect objects within a limited distance, a first detected object that is far from the vehicle will not be detected by the ultrasonic probe. Based on this, in some embodiments, the step S122 can comprise determining whether the first detected object is an object that can be detected by the ultrasonic probe according to the relative position between the first detected object and the ultrasonic probe, and matching the first detected object with the second detected object to obtain the matching result in a case where the ultrasonic probe can detect the first detected object. In this way, in a case where the ultrasonic probe cannot detect the first detected object, the first detected object does not need to be matched with the second detected object, thereby improving the matching speed and the matching efficiency.

[0127] In some examples, the step S122 can comprise determining whether the first detected object is an object that can be detected by the ultrasonic probe and has the shortest relative distance to the ultrasonic probe according to the relative position between the first detected object and the ultrasonic probe, and matching the first detected object with the second detected object to obtain the matching result in a case where the first detected object is an object that can be detected by the ultrasonic probe and has the shortest relative distance to the ultrasonic probe. For the detected object that has the shortest distance to the ultrasonic probe, the perception of the ultrasonic probe will not be affected by the reflection waves of other detected objects, and the perception result is more accurate, thereby further improving the matching accuracy. Furthermore, in a case where the first detected object is not an object that can be detected by the ultrasonic probe and has the shortest relative distance to the ultrasonic probe, the first detected object does not need to be matched with the second detected object, thereby also improving the matching speed and the matching efficiency.

[0128] Optionally, a relative position of each first detection object to the ultrasonic probe can be calculated, a relative distance of each first detection object to the ultrasonic probe is determined based on the relative position, and the first detection object with the shortest relative distance to the ultrasonic probe is taken as the object with the shortest relative distance to the ultrasonic probe.

[0129] In one example, a distance between a first position of each first detection object in a set coordinate system and a mounting position of the ultrasonic probe in the set coordinate system can be calculated, and the first detection object with the shortest distance is taken as the object with the shortest relative distance to the ultrasonic probe, i.e., the object closest to the ultrasonic probe.

[0130] In one example, the first position of the first detection object can be determined by a contour corresponding to the set of contour points of the first detection object. In this example, a relative position of each contour point to the ultrasonic probe can be calculated, a relative distance of each contour point to the ultrasonic probe is determined, and the first detection object corresponding to the contour point with the shortest relative distance is taken as the object with the shortest relative distance to the ultrasonic probe.

[0131] Since the ultrasonic probe usually has a certain emission angle, a conical beam is formed when it emits, the ultrasonic detection probability of the object at the center of the beam is large, the ultrasonic detection probability of the object at the edge of the beam is small, and the ultrasonic probe cannot detect the object outside the beam range. Based on this, in some embodiments, step S122 can include: obtaining an angle deviation of the first detection object relative to the ultrasonic probe; and matching the first detection object with the second detection object to obtain a matching result, in a case where the angle deviation is within a set threshold range.

[0132] The angle deviation of the first detection object relative to the ultrasonic probe in the present embodiment is the included angle between the line connecting the first detection object and the ultrasonic probe and the direction in which the ultrasonic probe faces. The direction in which the ultrasonic probe faces is determined based on the probe mounting angle. In some examples, as shown in FIG. 6, the probe mounting direction can form an included angle with a set direction (for example, the length direction of the ego vehicle, i.e., the y direction), and this included angle is the probe mounting angle θ1. In one example, a first vector can be determined based on the line connecting the first detection object and the ultrasonic probe, a second vector is determined based on the direction in which the ultrasonic probe faces, and the included angle between the first vector and the second vector is taken as the angle deviation θ2. Figure 4 The set threshold range in the present embodiment can be determined based on the detection angle range of the ultrasonic probe. That is, the set threshold range can be determined based on the spatial angle region covered when the ultrasonic probe emits ultrasonic waves.

[0133]

[0134] ​In some examples, the matching result of the first detection object and the second detection object is marked as an invalid matching result when the angle deviation is outside the set threshold range. The marking value indicates that the matching of the first detection object and the second detection object is an invalid matching.

[0135] The embodiment of the present disclosure matches the first detection object and the second detection object only when the angle deviation is within the set threshold range, which can improve the matching speed and efficiency. When the angle deviation is outside the set threshold range, the first detection object and the second detection object do not need to be matched, which can eliminate invalid matching and avoid the accuracy of the detection result being affected by the invalid matching result.

[0136] In some embodiments, the first detection result includes a first position of the first detection object in a set coordinate system determined based on the first visual perception data, and the second detection result includes a perception distance of the ultrasonic probe determined based on the ultrasonic perception data. In the embodiment, step S122 can include: determining a second distance between the first detection object and the ultrasonic probe according to the first position of the first detection object in the set coordinate system determined based on the first visual perception data, the position of the movable device in the set coordinate system, and the installation position of the ultrasonic probe on the movable device; and obtaining the matching result based on the second distance and the perception distance of the ultrasonic probe determined based on the ultrasonic perception data.

[0137] In the embodiment, the perception distance is the detection distance of the second detection object by the ultrasonic probe, i.e., the distance between the probe and the second detection object detected by the ultrasonic probe. The second distance in the embodiment is the actual distance between the first detection object and the ultrasonic probe based on the first visual perception data. By comparing the detection distance of the ultrasonic probe and the second detection object and the actual distance between the ultrasonic probe and the first detection object, it is determined whether the first detection object and the second detection object match. In this way, the ultrasonic perception result and the visual perception result can be fused, thereby further improving the accuracy of the matching result.

[0138] The set coordinate system in the embodiment can be, for example, a self-vehicle coordinate system. The first position can be, for example, a position determined based on a set of contour points of the first detection object in the set coordinate system determined based on the visual perception data.

[0139] In some examples, the step of obtaining the matching result based on the second distance and the perception distance can include: determining that the first detection object and the second detection object match successfully when the absolute difference between the perception distance and the second distance is less than a first threshold value; or determining that the first detection object and the second detection object match unsuccessfully when the absolute difference between the perception distance and the second distance is greater than the first threshold value.

[0140] When the perceived distance is less than the second distance and the perceived distance is much less than the second distance, i.e., the distance of the probe to the second detection object is much less than the actual distance between the probe and the first detection object, it is possible that the nearest detection object is not detected by the visual sensor. In this case, the matching of the first detection object and the second detection object is considered as invalid matching. In some examples, when the perceived distance is less than the second distance and the difference between the perceived distance and the second distance is less than a second threshold, the matching result of the first detection object and the second detection object is set as a label value of invalid matching. The label value of invalid matching indicates that the matching process of the first detection object and the second detection object is invalid matching.

[0141] In the embodiment, when the matching process of the first detection object and the second detection object is invalid matching, the first confidence degree is not adjusted by the invalid matching result, so that the accuracy of the detection result is not affected by the invalid matching result.

[0142] When the perceived distance is greater than the second distance and the perceived distance is much greater than the second distance, i.e., the distance of the probe to the second detection object is much greater than the actual distance between the probe and the first detection object, it is indicated that the probe does not detect the first detection object. In this case, the matching of the first detection object and the second detection object is considered as failed. In some examples, when the perceived distance is greater than the second distance and the difference between the perceived distance and the second distance is greater than a third threshold, it is determined that the matching of the first detection object and the second detection object is failed.

[0143] In some examples, when the absolute difference between the second distance and the perceived distance is between the second threshold and the third threshold, it is determined that the matching of the first detection object and the second detection object is successful. When the perceived distance is less than the second distance and the difference between the perceived distance and the second distance is less than the second threshold, the matching result of the first detection object and the second detection object is set as a label value of invalid matching. When the perceived distance is greater than the second distance and the difference between the perceived distance and the second distance is greater than the third threshold, it is determined that the matching of the first detection object and the second detection object is failed.

[0144] In some embodiments, the movable device is provided with a camera. The first perception data includes first visual perception data collected by the camera, and the second perception data includes second visual perception data collected by the camera. The first detection result is determined based on the first visual perception data, and the second detection result is determined based on the second visual perception data. The matching result is obtained by matching the visual perception result at the first sampling time and the visual perception result at the second sampling time, and then the first confidence degree of the first detection object being the set obstacle at the first sampling time is adjusted. In this way, the visual perception results corresponding to different sampling times are fused in time sequence to obtain the final visual perception result, the matching accuracy is improved, and thus the accuracy of obstacle detection is improved.

[0145] In the embodiment, the step S122 can include: obtaining a first contour region of the first detection object in a set coordinate system, and a second contour region of the second detection object in the set coordinate system; and obtaining a matching result according to the first contour region and the second contour region.

[0146] In some examples, the step of obtaining the matching result according to the first contour region and the second contour region can include: determining an overlapping region between the first contour region and the second contour region according to the first contour region and the second contour region; and determining the matching result of the first detection object and the second detection object according to the overlapping region.

[0147] In some examples, the step of determining the matching result of the first detection object and the second detection object according to the overlapping region can include: determining that the first detection object and the second detection object match successfully in a case that an area of the overlapping region is greater than or equal to a threshold value; or determining that the first detection object and the second detection object match unsuccessfully in a case that the area of the overlapping region is less than the threshold value.

[0148] The first contour region in the embodiment is a region determined by a first contour point set of the first detection object in a set coordinate system, and the second contour region is a region determined by a second contour point set of the second detection object in the set coordinate system.

[0149] In some examples, multiple ultrasonic probes can be provided. The first detection object that can be detected by the ultrasonic probe and has an angle deviation within a set threshold range can be matched with the second detection object obtained based on the perception data of each probe. Each matching result obtained can be fused to obtain a final matching result.

[0150] In the example, a final first adjustment value or a final second adjustment value corresponding to the third matching pair can be obtained based on the first adjustment value or the second adjustment value corresponding to each probe. In one example, an indication function indicating whether the first detection object is an object that can be detected by the probe can be provided. When the first detection object is not an object that can be detected by the probe, the indication function can be set to 0. When the first detection object is an object that can be detected by the probe, the indication function can be set to 1.

[0151] In one example, an angle coefficient indicating whether the first detection object is within the detection angle range of the ultrasonic probe can also be set. When the first detection object is not within the detection angle range of the ultrasonic probe, the angle coefficient can be set as 0. When the first detection object is within the detection angle range of the ultrasonic probe, the angle coefficient can be set as a preset angle coefficient. The preset angle coefficient can be a fixed value, or a variable value related to the angle deviation of the first detection object with respect to the ultrasonic probe. When the preset angle coefficient is a variable value related to the angle deviation of the first detection object with respect to the ultrasonic probe, an adjustment value reflecting the matching degree of the first detection object and the second detection object can be obtained, so that a more accurate second confidence can be obtained through the adjustment value.

[0152] In one example, a distance coefficient indicating whether the distance deviation between the second distance between the first detection object and the ultrasonic probe and the perception distance of the ultrasonic probe meets a preset condition can also be set. When the distance deviation between the second distance between the first detection object and the ultrasonic probe and the perception distance of the ultrasonic probe does not meet the preset effective matching condition, the distance coefficient can be set as 0. When the distance deviation between the second distance between the first detection object and the ultrasonic probe and the perception distance of the ultrasonic probe meets the preset matching success condition, the distance coefficient can be set as a positive preset distance coefficient. When the distance deviation between the second distance between the first detection object and the ultrasonic probe and the perception distance of the ultrasonic probe meets the preset matching failure condition, the distance coefficient can be set as a negative preset distance coefficient. The positive preset distance coefficient and / or the negative preset distance coefficient can be a fixed value, or a variable value related to the distance deviation. When the preset distance coefficient is a variable value related to the distance deviation, an adjustment value reflecting the matching degree of the first detection object and the second detection object can be obtained, so that a more accurate second confidence can be obtained through the adjustment value.

[0153] In the present example, the first adjustment value or the second adjustment value corresponding to each probe can be determined according to the indication function, the angle coefficient and the distance coefficient. For example, the product of the indication function, the angle coefficient and the distance coefficient is taken as the first adjustment value or the second adjustment value corresponding to each probe. The final first adjustment value or the second adjustment value corresponding to the third matching pair can be the sum of the first adjustment value or the second adjustment value corresponding to each probe.

[0154] In some embodiments, the second perception data can be perception data collected by multiple sensors. Based on the second perception data collected by the movable device, multiple second detection results can be obtained. For example, the multiple second detection results are second detection result A, second detection result B and second detection result C respectively. By matching the first detection object in the first detection result with each second detection result respectively, multiple second confidences can be obtained.

[0155] In this embodiment, the second detection result is multiple, the second confidence is multiple, and the second detection result and the second confidence correspond one by one.

[0156] In some examples, the second perception data can include second visual perception data collected by a camera and ultrasonic perception data collected by an ultrasonic probe. The second detection result A obtained by detecting obstacles based on the second visual perception data can include a third position of a second detection object in a set coordinate. The second detection result B obtained by detecting obstacles based on the ultrasonic perception data can include a second position of the second detection object in the set coordinate. The second detection result C obtained by detecting obstacles based on the ultrasonic perception data can further include a perception distance of the ultrasonic probe. The first perception data can include first visual perception data collected by the camera. The first detection result obtained by detecting obstacles based on the first perception data can include a first position of a first detection object in a set coordinate system.

[0157] In this example, the first position corresponding to the first detection object is matched with the third position corresponding to the second detection object in the second detection result A to obtain a first matching result. The first position corresponding to the first detection object is matched with the second position corresponding to the second detection object in the second detection result B to obtain a second matching result. The second distance between the first detection object and the ultrasonic probe is matched with the perception distance of the ultrasonic probe in the second detection result C to obtain a third matching result. It can be understood that the first matching is a matching of the visual perception result at the first sampling time and the visual perception result at the second sampling time. The second matching and the third matching are both a matching of the visual perception result at the first sampling time and the ultrasonic perception result at the second sampling time.

[0158] In the third matching, the first detection object in this embodiment can be the first detection object with the shortest distance from the ultrasonic probe and / or within the detection angle range of the ultrasonic probe.

[0159] According to the first matching result, the first confidence corresponding to the first matching in the first detection result is adjusted to obtain a second confidence corresponding to the first matching. According to the second matching result, the first confidence corresponding to the second matching in the first detection result is adjusted to obtain a second confidence corresponding to the second matching. According to the third matching result, the first confidence corresponding to the third matching in the first detection result is adjusted to obtain a second confidence corresponding to the third matching.

[0160] In some examples, step S122 can include steps T110 to T170.

[0161] Step T110, obtaining a first contour region of the first detected object in a set coordinate system determined based on the first visual perception data, and a second contour region of the second detected object in the set coordinate system determined based on the second visual perception data.

[0162] Step T120, performing first matching according to the first contour region and the second contour region to obtain a first matching result.

[0163] Step T130, obtaining a first contour region of the first detected object in a set coordinate system determined based on the first visual perception data, and a second position of the second detected object in the set coordinate system determined based on the ultrasonic perception data.

[0164] Step T140, performing second matching according to the first contour region and the second position to obtain a second matching result.

[0165] Step T150, determining whether the first detected object is an object that can be detected by the ultrasonic probe, and whether an angle deviation of the first detected object relative to the ultrasonic probe is within a set threshold range.

[0166] Step T160, in the case that the ultrasonic probe can detect the first detected object and the angle deviation is within the set threshold range, determining a second distance between the first detected object and the ultrasonic probe according to a first position of the first detected object in the set coordinate system determined based on the first visual perception data, a position of the movable device in the set coordinate system, and a mounting position of the ultrasonic probe on the movable device.

[0167] Step T170, obtaining a third matching result based on the second distance and a perception distance of the ultrasonic probe determined based on the ultrasonic perception data.

[0168] The specific matching and confidence adjustment process in the embodiment can refer to the foregoing embodiments, which will not be described here.

[0169] To ensure the efficiency and accuracy of the system, when the second confidence of the three matching manners corresponding to the first detection object are all lower than a certain threshold, the system will no longer maintain the first detection object and remove it from the maintenance table. This mechanism helps to reduce false positives and resource waste, so that the system focuses on more reliable obstacle detection.

[0170] In some embodiments, the matching manners in which the first detection result and the second detection result are matched can only include part of the first matching, the second matching, and the third matching, which will not be described here.

[0171] In some embodiments, different matching weights can be assigned to the plurality of second detection results. Accordingly, the second confidence corresponding to different matching manners corresponds to different matching weights. For example, the matching weight of the second confidence corresponding to the first matching is a first matching weight, the matching weight of the second confidence corresponding to the second matching is a second matching weight, and the matching weight of the second confidence corresponding to the third matching is a third matching weight.

[0172] In this embodiment, the obstacle detection method can further include: obtaining a confidence that the first detection object is a set obstacle according to the plurality of second confidences and the matching weights corresponding to the plurality of second detection results. The confidence is a target confidence at the second sampling time. In some examples, the sum of the product of each second confidence and the matching weight corresponding to the second confidence can be calculated to obtain the target confidence that the first detection object is a set obstacle.

[0173] Considering that different perception manners have different detection result accuracies when detecting the same type of detection object, in some examples, different matching weights can be assigned to different second detection results of the same type of detection object obtained by different matching manners. Accordingly, the second confidence corresponding to the same type of detection object and different matching manners corresponds to different matching weights.

[0174] In this embodiment, the type of the detection object can be divided according to at least one of the area and the motion state of the detection object. The area of the detection object can be the area of the detection object in the top view obtained based on the visual perception data. In some examples, the type of the detection object can be divided into a first type and a second type according to the area size, the area of the detection object of the first type being less than or equal to a first threshold, and the area of the detection object of the second type being greater than the first threshold. For example, the detection object of the first type is a pedestrian, and the detection object of the second type is a vehicle.

[0175] In other examples, the type of the detection object can be divided into a static type and a dynamic type according to the motion state. For example, the detection object of the static type is a static vehicle, and the detection object of the dynamic type is a dynamic vehicle.

[0176] In some examples, the types of the detected objects can be classified into a static first type, a static second type, a dynamic first type, and a dynamic second type according to the area and the motion state of the objects. The detected objects of the static first type have an area less than or equal to a first threshold, the detected objects of the static second type have an area greater than the first threshold, the detected objects of the dynamic first type have an area less than or equal to the first threshold, and the detected objects of the dynamic second type have an area greater than the first threshold. For example, the detected objects of the static first type are static pedestrians, the detected objects of the static second type are static vehicles, the detected objects of the dynamic first type are dynamic pedestrians, and the detected objects of the dynamic second type are dynamic vehicles.

[0177] In some examples, the types of the detected objects can also be classified into a dynamic white list obstacle (e.g., a dynamic vehicle), a static white list obstacle (e.g., a static cone), and a general obstacle according to whether the obstacle is in the white list and the motion state. The general obstacle is an obstacle not in the preset white list. In some examples, the type of the general obstacle can be further refined according to the area, shape, motion state, and the like of the object.

[0178] In the embodiment, different matching weights can be assigned to the second confidence of the same type of the detected object in each matching mode according to the advantages of the visual sensor in identifying dynamic objects and small volume objects and the advantages of the ultrasonic sensor in identifying static objects and large volume objects.

[0179] For a dynamic vehicle, flexible processing is needed to deal with its fast changing characteristics. Since the visual sensor has an advantage in identifying the shape and motion trajectory of the dynamic vehicle, the ultrasonic information is used to verify the existence of the vehicle to ensure the consistency of multi-perception. Therefore, for the dynamic vehicle, the matching weight corresponding to the first matching can be set to be higher than the matching weight corresponding to the second matching and the third matching.

[0180] For a dynamic pedestrian, due to its irregular motion trajectory and small volume, it is more dependent on the perception result of the visual sensor. The visual sensor can provide more accurate pedestrian detection to identify the motion direction and speed of the pedestrian. The ultrasonic sensor has poor detection effect on the pedestrian. Therefore, for the dynamic pedestrian, the matching weight corresponding to the first matching can be set to be higher than the matching weight corresponding to the second matching and the third matching. In one example, the matching weight corresponding to the second matching and the third matching can be set to be lower than that of the dynamic vehicle.

[0181] For static vehicles, which usually have clear contours and large volumes, the ranging information of the ultrasonic waves is particularly important in identifying static vehicles, which can accurately depict the contours of the vehicles, thereby reducing the false positive rate. The visual sensor is used to confirm the specific shape and position of the vehicle. Therefore, for static vehicles, the matching weight corresponding to the second matching and the third matching can be set to be higher than the matching weight corresponding to the first matching.

[0182] For static pedestrians, the visual sensor is still the main sensing means. Based on the visual sensing data, the posture and position of the pedestrian are identified, and the ultrasonic data is used as auxiliary information to help confirm the existence of the pedestrian. Therefore, for static pedestrians, the matching weight corresponding to the first matching can be set to be higher than the matching weight corresponding to the second matching and the third matching.

[0183] For static general obstacles, since these obstacles usually have fixed shapes and positions, their detection needs to rely on the accurate ranging capability of the ultrasonic waves to provide accurate distance information, which can help to construct the accurate contours of the obstacles. The visual sensor is used to supplement the identification of shape and material. Therefore, for static general obstacles, the matching weight corresponding to the second matching and the third matching can be set to be higher than the matching weight corresponding to the first matching.

[0184] In some examples, the first confidence corresponding to each matching manner in the first detection result can be adjusted according to the first matching result, the second matching result and the third matching result respectively to obtain a second confidence corresponding to each matching manner. According to the plurality of second confidences and the matching weights corresponding to the plurality of second confidences respectively, a target confidence that the first detection object is the set obstacle at the second sampling time is obtained.

[0185] The embodiments of the present disclosure comprehensively consider the accuracy of the detection results of the visual sensor and the ultrasonic sensor on different types of obstacles, and set different matching weights based on the respective advantages of the visual sensor and the ultrasonic sensor, which can more effectively balance the true positive rate and the false positive rate, improve the accuracy and reliability of the overall perception, and realize accurate identification and stable tracking of obstacles. This flexible strategy adjustment not only improves the intelligent level of the system, but also provides a safer guarantee for the emergency braking function, while reducing the false positive rate.

[0186] The embodiments of the present disclosure also provide a vehicle control method. As shown in Figure 5 The vehicle control method can include steps S310 to S320.

[0187] Step S310, obtaining a second confidence that a first detection object is a set obstacle.

[0188] Step S320, controlling the vehicle to execute an obstacle avoidance strategy according to the second confidence.

[0189] The second confidence is obtained according to a matching result of the first detection object, the matching result being a matching result obtained by matching the first detection object in the first detection result in a second detection result, the second detection result being obtained by detecting an obstacle based on second perception data collected by the movable device, the first detection result being obtained by detecting an obstacle based on first perception data collected by the movable device, a first sampling time of the first perception data being before a second sampling time of the second perception data, the first confidence being a confidence corresponding to the first sampling time, and the second confidence being a confidence corresponding to the second sampling time.

[0190] In this embodiment, the obstacle detection result corresponding to each first detection object is received, the first detection object with the maximum risk is determined according to the confidence corresponding to each first detection object, and the relative distance and the relative speed between the first detection object with the maximum risk and the ego vehicle are obtained. In combination with the system state at this time, it is determined whether to control the vehicle to execute an obstacle avoidance strategy. In the case of controlling the vehicle to execute the obstacle avoidance strategy, the vehicle braking flag is set to an active state, and a vehicle braking strategy is generated, and then the current vehicle is automatically and urgently braked according to the vehicle braking strategy.

[0191] The first detection object in this embodiment is the second confidence of the set obstacle, and the acquisition method can refer to the description in the foregoing embodiments of the present disclosure, which will not be described here.

[0192] According to the more accurate confidence obtained by the foregoing obstacle detection method of the present disclosure, the accurate obstacle detection result is obtained based on the confidence, so as to determine whether to control the vehicle to execute the obstacle avoidance strategy, which can improve the control accuracy and reduce the false triggering.

[0193] The present disclosure also provides a map display method. The map display method can include steps S410 to S420.

[0194] In step S410, an obstacle detection result of a scene area where the vehicle is located is obtained.

[0195] In step S420, a target detection object indicated in the obstacle detection result is displayed.

[0196] The obstacle detection result includes a second confidence that the first detection object is a set obstacle, the second confidence being obtained by adjusting a first confidence that the first detection object is a set obstacle according to a matching result, the matching result being a matching result obtained by matching the first detection object in the first detection result in a second detection result, the second detection result being obtained by obstacle detection based on second perception data collected by the movable device, the first detection object being obtained by obstacle detection based on first perception data collected by the movable device, a first sampling time of the first perception data being before a second sampling time of the second perception data, the first confidence being a confidence corresponding to the first sampling time, and the second confidence being a confidence corresponding to the second sampling time.

[0197] The manner of obtaining the obstacle detection result in this embodiment can refer to the description in the foregoing embodiments of the present disclosure, which will not be described here.

[0198] The target detection object in this embodiment can be any detection object whose potential risk reaches a threshold. Step S420 can include displaying the orientation information of the target detection object.

[0199] The more accurate confidence obtained by using the foregoing obstacle detection method of the present disclosure is used to obtain an accurate obstacle detection result based on the confidence, so as to obtain a high-precision target road scene map. During vehicle driving, the target road scene map corresponding to the position of the vehicle is displayed, so that the driver can accurately know the road scene corresponding to the position of the vehicle, and the driving safety is improved.

[0200] The present disclosure also provides an electronic device. As shown in Figure 6 The electronic device 1000 can include a memory 1010 and a processor 1020, the memory 1010 can be used to store computer instructions, and the processor 1020 can be used to call the computer instructions from the memory 1010 to execute all or part of the steps of any obstacle detection method in the foregoing embodiments of the present disclosure. It should be noted that the processor 1020 can include one or more processors to execute the instructions, and the memory 1010 can also include one or more memories to store the computer instructions. In one example, the electronic device 1000 can be a cloud server.

[0201] The present disclosure also provides a vehicle. As shown in Figure 7As shown, the vehicle 1100 can include a memory 1110 and a processor 1120. The memory 1110 can be configured to store computer instructions. The processor 1120 can be configured to invoke the computer instructions from the memory 1110 to perform all or part of the steps of any of the obstacle detection methods in the foregoing embodiments of the present disclosure. It should be noted that the processor 1120 can include one or more processors to execute the instructions, and the memory 1110 can also include one or more memories to store the computer instructions. The vehicle can be, for example, a vehicle with an assisted driving function such as a memory parking and / or a memory driving.

[0202] The embodiments of the present disclosure also provide a chip. As shown in Figure 8 The chip 2000 can include a storage unit 2010 and a processing unit 2020. The storage unit 2010 is configured to store a computer program. The processing unit 2020 is configured to implement the method according to any of the embodiments of the present disclosure when executing the computer program stored in the storage unit.

[0203] The embodiments of the present disclosure also provide a non-volatile computer readable storage medium having computer program instructions stored therein, which are executed by a processor to implement the method according to any of the foregoing embodiments of the present disclosure. Alternatively, the computer readable storage medium can be a non-transitory storage medium, but is not limited thereto, and can also be a transitory storage medium.

[0204] The embodiments of the present disclosure also provide a computer program product, which can include a computer program. The computer program is executed by a processor to implement any of the methods in the foregoing embodiments of the present disclosure.

[0205] The present disclosure can be a system, a method, and / or a computer program product. The computer program product can include a computer readable storage medium having computer readable program instructions embodied therewith. The computer readable program instructions can be downloaded to the computer from a computer readable storage medium or to a computer from the internet and implemented by a processor.

[0206] Computer readable storage media can be tangible storage media which can retain and store instructions for use by an instruction execution device. Computer readable storage media can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer readable storage media include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0207] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0208] Computer readable program instructions for carrying out operations of the present disclosure can be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.

[0209] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0210] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other data storage device. When the computer readable program instructions are loaded into the computer and other programmable data processing apparatus, a series of operational steps are implemented that provide processes such that the instructions which operate on the computer or other programmable data processing apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0211] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0212] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0213] Embodiments of the present disclosure have been described above, and the description is intended to be illustrative, and not restrictive, of the disclosed embodiments. Many modifications and variations of the described embodiments are possible, and all such modifications and variations are intended to be within the scope of the described embodiments. The description used herein is intended to best explain the principles of the various embodiments, the practical application, and the best mode of the present disclosure, and to enable others skilled in the art to understand the disclosure, the principles of the various embodiments, and the best mode of the present disclosure, the principles of the various embodiments, and the best mode of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. An obstacle detection method characterized by, The method comprises: obtaining a first detection result of obstacle detection based on first perception data collected by a movable device, and a second detection result of obstacle detection based on second perception data collected by the movable device; wherein a first sampling time of the first perception data is before a second sampling time of the second perception data, the second perception data comprises ultrasonic perception data collected by an ultrasonic probe, and the first detection result comprises a first detection object and a first confidence degree that the first detection object is a set obstacle; querying a second detection object meeting a set condition in the second detection result; determining whether the first detection object is an object that can be detected by the ultrasonic probe according to a relative position between the first detection object and the ultrasonic probe; in a case where the ultrasonic probe can detect the first detection object, matching the first detection object with the second detection object to obtain a matching result; adjusting the first confidence degree according to the matching result to obtain a second confidence degree that the first detection object is the set obstacle; wherein the second confidence degree is a confidence degree corresponding to the second sampling time.

2. The method of claim 1, wherein, The adjusting the first confidence degree according to the matching result to obtain the second confidence degree that the first detection object is the set obstacle comprises: in a case where the matching result indicates that there is a target detection object matching the first detection object in the second detection result, increasing the first confidence degree by a first adjustment value to obtain the second confidence degree; or in a case where the matching result indicates that there is no target detection object matching the first detection object in the second detection result, decreasing the first confidence degree by a second adjustment value to obtain the second confidence degree.

3. The method of claim 1, wherein, The set condition is set based on at least one of a category and a position of the first detection object.

4. The method of claim 1, wherein, The movable device is provided with a camera and an ultrasonic probe, the first perception data comprises first visual perception data collected by the camera, the first detection result is determined based on the first visual perception data, and the second detection result is determined based on the ultrasonic perception data.

5. The method of claim 4, wherein, The first detection result comprises a first position of the first detection object in a set coordinate system determined based on the first visual perception data, and the second detection result comprises a second position of the second detection object in the set coordinate system determined based on the ultrasonic perception data. The matching the first detection object with the second detection object to obtain a matching result comprises: determining a first distance between the first detection object and the second detection object according to the first position and the second position; obtaining the matching result according to the first distance.

6. The method of claim 1, wherein, The matching the first detection object with the second detection object to obtain a matching result comprises: obtaining an angle deviation of the first detection object relative to the ultrasonic probe; in a case where the angle deviation is within a set threshold range, matching the first detection object with the second detection object to obtain a matching result.

7. The method of claim 4, wherein, The first detection result includes a first position of the first detection object in a set coordinate system determined based on the first visual perception data, and the second detection result includes a perception distance of the ultrasonic probe determined based on the ultrasonic perception data; The matching of the first detection object and the second detection object includes: According to the first position, the position of the movable device in the set coordinate system, and the installation position of the ultrasonic probe on the movable device, a second distance between the first detection object and the ultrasonic probe is determined; Based on the second distance and the perception distance, the matching result is obtained.

8. The method of claim 1, wherein, The movable device is provided with a camera, the first perception data includes first visual perception data collected by the camera, the second perception data further includes second visual perception data collected by the camera, the first detection result is determined based on the first visual perception data, and the second detection result is determined based on the ultrasonic perception data and the second visual perception data; The matching of the first detection object and the second detection object includes: A first contour region of the first detection object in a set coordinate system and a second contour region of the second detection object in the set coordinate system are obtained; According to the first contour region and the second contour region, a matching result is obtained.

9. The method according to any one of claims 1 to 8, characterized in that, The second detection result is multiple, the second confidence is multiple, the second detection result and the second confidence correspond one by one, and the method further includes: According to multiple second confidences and multiple matching weights corresponding to multiple second detection results, a confidence that the first detection object is the set obstacle is obtained.

10. A vehicle control method characterized by, Including: A second confidence that the first detection object is the set obstacle is obtained; According to the second confidence, a vehicle is controlled to execute an obstacle avoidance strategy; The second confidence is obtained according to a matching result adjusting a first confidence that the first detection object in the first detection result is the set obstacle, the matching result is obtained by matching the first detection object and a second detection object according to a relative position between the first detection object and the ultrasonic probe, the second detection object is a detection object meeting a set condition obtained by querying the second detection result, the second detection result is obtained by obstacle detection based on second perception data collected by a movable device, the first detection result is obtained by obstacle detection based on first perception data collected by the movable device, a first sampling time of the first perception data is before a second sampling time of the second perception data, the second perception data includes ultrasonic perception data collected by the ultrasonic probe, the first confidence is a confidence corresponding to the first sampling time, and the second confidence is a confidence corresponding to the second sampling time.

11. An electronic device, comprising: The memory and the processor, The memory is configured to store computer instructions, and the processor is configured to call the computer instructions from the memory to execute the method according to any one of claims 1-10.

12. A chip, characterized by Comprising: a storage unit configured to store a computer program; and, a processing unit configured to implement the method according to any one of claims 1-10 when executing the computer program stored in the storage unit.

13. A vehicle characterized by comprising: Comprising a memory configured to store computer instructions and a processor configured to call the computer instructions from the memory to execute the method according to any one of claims 1-10.

14. A non-transitory computer readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions, when executed by a processor, implement the method according to any one of claims 1-10.

Images