Obstacle detection method, vehicle control method, equipment and medium

Abstract

The invention relates to an obstacle detection method, a vehicle control method, equipment and a medium. The method comprises the following steps: acquiring a first detection result of obstacle detection based on first sensing data acquired by a mobile device, and a second detection result of obstacle detection based on second sensing data acquired by the mobile device; wherein the first sampling time of the first sensing data is before the second sampling time of the second sensing data, and the first detection result comprises a first detection object and a first confidence coefficient that the first detection object is a set obstacle; matching the first detection object in the second detection result to obtain a matching result; and adjusting the first confidence coefficient according to the matching result to obtain a second confidence coefficient that the first detection object is the set obstacle.

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent driving technology, and more specifically, to an obstacle detection method, a vehicle control method, an electronic device, a chip, a vehicle, and a readable storage medium. Background Art

[0002] In autonomous driving technology, obstacle detection can be performed based on the perception data of various sensors on the vehicle, and then the vehicle can be controlled to perform decision planning and control based on the obstacle detection results. Among them, the accuracy of obstacle detection results is the key to the stability and reliability of autonomous driving. Therefore, it is necessary to provide a solution to improve the accuracy of obstacle detection results. Summary of the invention

[0003] In view of this, the present disclosure proposes a new technical solution for obstacle detection, which can improve the accuracy of detection results.

[0004] According to a first aspect of an embodiment of the present disclosure, there is provided an obstacle detection method, comprising:

[0005] Obtaining a first detection result of obstacle detection based on first perception data collected by a movable device, and a second detection result of obstacle detection based on second perception data collected by the movable device; wherein a first sampling time of the first perception data is before a second sampling time of the second perception data, and the first detection result includes a first detection object and a first confidence level that the first detection object is a set obstacle;

[0006] Matching the first detection object in the second detection result to obtain a matching result;

[0007] The first confidence level is adjusted according to the matching result to obtain a second confidence level that the first detection object is the set obstacle; wherein the second confidence level is a confidence level corresponding to the second sampling time.

[0008] Optionally, adjusting the first confidence level according to the matching result to obtain a second confidence level that the first detection object is the set obstacle includes:

[0009] In a case where the matching result indicates that there is a target detection object matching the first detection object in the second detection result, the first confidence level is increased by a first adjustment value to obtain the second confidence level; or

[0010] When the matching result indicates that there is no target detection object matching the first detection object in the second detection result, the first confidence level is reduced by a second adjustment value to obtain the second confidence level.

[0011] Optionally, matching the first detection object in the second detection result includes:

[0012] Searching the second detection result for a second detection object that meets a set condition; wherein the set condition is set based on at least one of the category and the position of the first detection object;

[0013] The first detection object is matched with the second detection object to obtain a matching result of the first detection object for the second detection object.

[0014] Optionally, the movable device is provided with a camera and an ultrasonic probe, the first perception data includes first visual perception data collected by the camera, the second perception data includes ultrasonic perception data collected by the ultrasonic probe, the first detection result is determined based on the first visual perception data, and the second detection result is determined based on the ultrasonic perception data.

[0015] Optionally, the first detection result includes a first position of the first detection object in a set coordinate system determined based on the first visual perception data, and the second detection result includes a second position of the second detection object in the set coordinate system determined based on the ultrasonic perception data;

[0016] The matching of the first detection object and the second detection object to obtain a matching result includes:

[0017] Determining a first distance between the first detection object and the second detection object according to the first position and the second position;

[0018] The matching result is obtained according to the first distance.

[0019] Optionally, matching the first detection object with the second detection object to obtain a matching result includes:

[0020] According to the relative position between the first detection object and the ultrasonic probe, determine whether the first detection object is an object that can be detected by the ultrasonic probe; if the ultrasonic probe can detect the first detection object, match the first detection object with the second detection object to obtain a matching result.

[0021] Optionally, matching the first detection object with the second detection object to obtain a matching result includes:

[0022] Acquire an angle deviation of the first detection object relative to the ultrasonic probe;

[0023] When the angle deviation is within a set threshold range, the first detection object is matched with the second detection object to obtain a matching result.

[0024] Optionally, the first detection result includes a first position of the first detection object in a set coordinate system determined based on the first visual perception data, and the second detection result includes a perception distance of the ultrasonic probe determined based on the ultrasonic perception data;

[0025] The matching of the first detection object with the second detection object to obtain a matching result includes:

[0026] determining a second distance between the first detection object and the ultrasonic probe according to the first position, the position of the movable device in the set coordinate system, and the installation position of the ultrasonic probe on the movable device;

[0027] The matching result is obtained based on the second distance and the perception distance.

[0028] Optionally, the movable device is provided with a camera, the first perception data includes first visual perception data collected by the camera, the second perception data includes second visual perception data collected by the camera, the first detection result is determined based on the first visual perception data, and the second detection result is determined based on the second visual perception data;

[0029] The matching of the first detection object and the second detection object to obtain a matching result includes:

[0030] Acquire a first contour area of ​​the first detection object in a set coordinate system, and a second contour area of ​​the second detection object in the set coordinate system;

[0031] A matching result is obtained according to the first contour area and the second contour area.

[0032] Optionally, there are multiple second detection results, and the second confidence level is a plurality of confidence levels obtained by matching the first detection result with each of the second detection results respectively; the method further includes:

[0033] The confidence that the first detection object is the set obstacle is obtained according to the second confidence and the matching weights corresponding to each of the plurality of second detection results.

[0034] According to a second aspect of an embodiment of the present disclosure, there is provided a vehicle control method, including:

[0035] Acquire a second confidence level that the first detection object is a set obstacle;

[0036] According to the second confidence level, controlling the vehicle to execute an obstacle avoidance strategy;

[0037] Among them, the second confidence level is obtained by adjusting the first confidence level that the first detection object is the set obstacle according to the matching result, the matching result is a matching result obtained by matching the first detection object in the first detection result in the second detection result, the second detection result is obtained by performing obstacle detection based on the second perception data collected by the movable device, the first detection result is obtained by performing obstacle detection based on the first perception data collected by the movable device, the first sampling time of the first perception data is before the second sampling time of the second perception data, the first confidence level is the confidence level corresponding to the first sampling time, and the second confidence level is the confidence level corresponding to the second sampling time.

[0038] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including a memory and a processor.

[0039] The memory is used to store computer instructions, and the processor is used to call the computer instructions from the memory to execute the method as described in any one of the first aspect or the second aspect.

[0040] According to a fourth aspect of an embodiment of the present disclosure, a chip is provided, including:

[0041] a storage unit for storing a computer program; and,

[0042] A processing unit, wherein the processing unit is configured to implement the method as described in any one of the first aspect or the second aspect when executing the computer program stored in the storage unit.

[0043] According to a fifth aspect of an embodiment of the present disclosure, a vehicle is provided, comprising a memory and a processor, wherein the memory is used to store computer instructions, and the processor is used to call the computer instructions from the memory to execute a method as described in any one of the first aspect or the second aspect.

[0044] According to a sixth aspect of an embodiment of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method as described in any one of the first aspect or the second aspect is implemented.

[0045] According to an embodiment of the present disclosure, a first detection result of obstacle detection based on first sensing data collected by a movable device and a second detection result of obstacle detection based on second sensing data collected by the movable device are obtained; the first detection object in the first detection result is matched in the second detection result to obtain a matching result; the first confidence that the first detection object is a set obstacle at the first sampling time is adjusted according to the matching result to obtain a second confidence that the first detection object is the set obstacle at the second sampling time. The embodiment of the present disclosure realizes real-time dynamic adjustment of the confidence that the detection object is a set obstacle, thereby improving the accuracy of the obstacle detection result.

[0046] Other features and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0048] Figure 1 The present invention is a schematic diagram of an intelligent network connection system to which the method provided by the embodiment of the present disclosure can be applied.

[0049] Figure 2 is based on Figure 1 The illustrated embodiment provides a schematic diagram of the structural composition of a vehicle.

[0050] Figure 3 It is a flowchart of an obstacle detection method provided by an embodiment of the present disclosure.

[0051] Figure 4 It is a schematic diagram of the detection angle range of the probe provided in an embodiment of the present disclosure.

[0052] Figure 5 It is a schematic diagram of a vehicle control method provided in an embodiment of the present disclosure.

[0053] Figure 6 It is a schematic diagram of the composition structure of an electronic device provided by an embodiment of the present disclosure.

[0054] Figure 7 It is a schematic diagram of the composition structure of another vehicle provided in an embodiment of the present disclosure.

[0055] Figure 8 It is a schematic diagram of the composition structure of a chip provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0056] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0057] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0058] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0059] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0060] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0061] In order to facilitate understanding of the method of the embodiment of the present disclosure, the application scenario of the embodiment of the present disclosure is first described.

[0062] Figure 1 1 is a schematic diagram of an intelligent network system 100 to which the method provided by the embodiment of the present disclosure can be applied. Figure 1 As shown, the intelligent network connection system 100 may include: a movable device 101, a server 102, and a user terminal 103.

[0063] In some examples, the movable device 101 is, for example, a vehicle with an autonomous driving function. Autonomous driving is also called unmanned driving or intelligent driving. A vehicle with an autonomous driving function can achieve driving tasks such as environmental perception, decision planning, and control execution. The level of autonomous driving can refer to the automotive intelligence classification standard formulated by the Society of Automotive Engineers (SAE). For example, L0 is manual driving, L1 is assisted driving, L2 is partial autonomous driving, L3 is conditional autonomous driving, L4 is highly autonomous driving, and L5 is fully autonomous driving. The above classification method of autonomous driving levels is only used as an example, and the disclosed embodiments do not limit the classification standards and levels of autonomous driving.

[0064] In some examples, the server 102 may be a single server or a distributed server cluster consisting of multiple servers, and its deployment method may include a local server or a cloud server. The server 102 may communicate with the mobile device 101 and / or the user terminal 103 based on a communication network to provide various services for the mobile device 101 and / or the user terminal 103. For example, the server may receive perception data sent by the vehicle and provide services such as high-precision maps, data analysis, and decision-making planning to the vehicle. For another example, the server may receive query instructions or control instructions sent by the user terminal and provide corresponding services to the user.

[0065] In some examples, the user terminal 103 can be any form of electronic device that provides services to users, such as a personal computer, a laptop computer, a smart tablet, a smart phone, a smart wearable device, etc. The user can interact with the vehicle or server through the human-computer interaction terminal configured by the mobile device 101, or can interact with the vehicle or server through the user terminal 103. For example, query the status and / or parameters of the vehicle through the user terminal, or control the vehicle to perform set tasks and / or modify configuration parameters, etc. Among them, the user terminal runs an application based on the intelligent network system to achieve interaction with the vehicle or server. The application can be a local application, a web application, a mini-program, etc., which is not limited here.

[0066] In some examples, the above-mentioned application running on the user terminal can provide authentication or authorization services for the user. The user who is successfully authenticated and granted the corresponding authority can query and / or control the vehicle within the scope of the granted authority.

[0067] The mobile device 101, the server 102 and the user terminal 103 can communicate through the communication link provided by the communication network 104. The communication network 104 may include one or more networks of any type. For example, the communication network 104 may include the Internet, a local area network (LAN), a wide area network (WAN), a virtual private network (VPN), a public switched telephone network (PSTN), a satellite communication network, Wi-Fi, 2G, 3G, 4G, 5G, 6G, NB-IoT, eMTC, infrared, Bluetooth, NFC and other networks providing communication, or a combination of the above multiple networks. The communication networks between the mobile device 101 and the server 102, between the user terminal 103 and the server 102, and between the user terminal 103 and the mobile device 101 may be the same or different.

[0068] It should be noted that Figure 1The structure of the intelligent network connection system 100 shown in is only illustrative. The intelligent network connection system in the embodiment of the present disclosure is not limited to the above structure, and may include more or fewer devices as needed, and may also combine or split the devices. For example, the intelligent network connection system may not include a user terminal and / or a server. For another example, the user terminal and the server may be deployed together.

[0069] Figure 2 is based on Figure 1 The embodiment shown is a schematic diagram of a mobile device 101. Figure 2 As shown, the mobile device 101 may include a perception component 1011, a computing platform 1012, an execution component 1013, etc. The perception component 1011, the computing platform 1012, and the execution component 1013 may be connected via a bus or other means.

[0070] In some examples, the perception component 1011 can be used to collect information about the vehicle itself or outside. The perception component 1011 can include at least one of a visual sensor unit, a radar, a positioning and navigation unit, an inertial measurement unit (IMU), or other sensor units, wherein the visual sensor unit can include one or more cameras, the radar can include at least one of a laser radar, a millimeter wave radar, an ultrasonic radar, or other radars, and the positioning and navigation unit can include at least one of a GPS system, a Beidou system, or other global positioning systems. The visual sensor unit can be used to collect visual images, and the ultrasonic radar can be used to collect detection information of an ultrasonic probe.

[0071] In some examples, the computing platform 1012 may include a device with computing capabilities, which is used to process the sensing data collected by the sensing component 1011 to obtain control information, and send corresponding control instructions to the execution component 1013, so that the execution component 1013 performs corresponding actions, thereby realizing control of the movable device 101. For example, the computing platform 1012 can perform Simultaneous Localization and Mapping (SLAM), path planning, behavior decision-making and other behaviors on the vehicle, thereby realizing autonomous control of the vehicle.

[0072] The computing platform 1012 may include at least one processor and at least one memory, and each processor may execute instructions stored in the memory individually or together to implement the method provided in the embodiment of the present disclosure. The processor in the embodiment of the present disclosure may include at least one of a central processing unit (CPU), a graphic processor (GPU), a field programmable gate array (FPGA), a system on chip (SOC), an application specific integrated circuit (ASIC), a microcontroller unit (MCU) or other processors. The memory may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a disk or an optical disk. In addition to storing instructions, the memory may also store data, such as scene maps, path information, vehicle location, direction, speed and other data. The data stored in the memory may be acquired and used by the processor.

[0073] In some examples, the vehicle's computing platform can independently perform computing tasks or communicate with a server to complete computing tasks. For example, the vehicle's computing platform can cooperate with the server to complete corresponding computing tasks.

[0074] The computing platform 1012 may be arranged in the removable device 101, and part or all of the computing platform 1012 may also be arranged in the server corresponding to the vehicle. For example, some functions of the computing platform 1012 with higher real-time requirements are arranged in the vehicle, and other functions with lower real-time requirements are arranged in the server corresponding to the vehicle.

[0075] In some examples, the execution component 1013 is used to execute corresponding actions based on the control of the computing platform 1012, so that the movable device 101 completes the moving task. The execution component 1013 may include, for example, a power component, a brake component, a transmission component, a steering component, and the like.

[0076] It should be noted that Figure 2The structure of the mobile device 101 shown in the figure is only illustrative. The vehicle in the embodiment of the present disclosure is not limited to the above structure, and may include more or fewer components as needed, and the devices may be combined or split. For example, the vehicle may not include the above computing platform. For another example, the vehicle may also include a communication component, an interface component, a multimedia component, an input component, an output component, etc.

[0077] In autonomous driving technology, obstacle detection can be performed based on the perception data of various sensors on the vehicle, and then the vehicle can be controlled to perform decision planning and control based on the obstacle detection results. Among them, the accuracy of obstacle detection results is the key to the stability and reliability of autonomous driving. Therefore, it is necessary to provide a solution to improve the accuracy of obstacle detection results. For example, in a vehicle emergency avoidance scenario, the vehicle can be controlled to execute an obstacle avoidance strategy based on the obstacle detection results to avoid or reduce the occurrence of vehicle collisions.

[0078] In the related technology, there is an obstacle detection result obtained by using perception data collected by the vehicle. When the detection result indicates that the confidence level of the detection object being an obstacle is greater than the confidence threshold, the vehicle is controlled to execute an obstacle avoidance strategy. However, the obstacle detection result is only obtained based on a single frame of perception data, which is prone to misjudgment.

[0079] Based on this, an embodiment of the present disclosure provides an obstacle detection method. The embodiment of the present disclosure obtains a first detection result based on first perception data collected at a first sampling time, obtains a second detection result based on second perception data collected at a second sampling time later than the first sampling time, matches the first detection result with the second detection result, and dynamically adjusts the confidence level corresponding to the first detection result according to the matching result, thereby improving the accuracy of the obstacle detection result.

[0080] like Figure 3 As shown, the obstacle detection method of the embodiment of the present disclosure may include steps S110 to S130.

[0081] Step S110, obtaining a first detection result of obstacle detection based on first perception data collected by the movable device, and a second detection result of obstacle detection based on second perception data collected by the movable device; wherein the first detection result includes a first detection object and a first confidence level that the first detection object is a set obstacle.

[0082] In this embodiment, the first sampling time of the first perception data is before the second sampling time of the second perception data. In one example, the first perception data may be perception data collected in a historical frame, and the second perception data may be perception data collected in a current frame.

[0083] The first perception data in this embodiment may be visual perception data collected by a camera of a visual perception unit, ultrasonic perception data collected by an ultrasonic radar probe, or perception data collected by other sensing units.

[0084] The second perception data in this embodiment may be visual perception data collected by a camera of the visual perception unit, ultrasonic perception data collected by an ultrasonic radar probe, or perception data collected by other sensing units.

[0085] The set obstacle in this embodiment is an obstacle of a set type. In one example, the set obstacle can be one of a vehicle, a pedestrian, a cone barrel, a building, etc.

[0086] In some examples, obstacle detection based on first perception data collected by a camera includes: performing dynamic obstacle detection and static obstacle detection based on the first perception data, respectively. In one example, static detection objects and dynamic detection objects can be distinguished by analyzing the absolute position coordinates of the same detection object within adjacent sampling times. In another example, detection objects that may be obstacles can be identified by a machine learning algorithm (such as a convolutional neural network in deep learning), and these detection objects can be classified to distinguish static detection objects from dynamic detection objects. After obstacle detection is performed based on the first perception data, a first detection result is obtained. The first detection result in this embodiment may include the first detection object and the first confidence that the first detection object is a set obstacle.

[0087] The confidence that the detected object is a set obstacle in this embodiment refers to the probability that the detected object is an obstacle of a set type. In one example, the confidence can be a floating point number from 0 to 1. The initial value of the first confidence can be a preset value, such as 0. The first confidence in this embodiment is the confidence corresponding to the first sampling time.

[0088] It can be understood that the first detection result in this embodiment is a historical detection result, which can be stored in the maintenance table.

[0089] In this embodiment, obstacle detection based on the second sensing data may not detect potential obstacles, or may detect potential obstacles. That is, in this embodiment, the second detection result may not contain a detection object, or may contain a detection object. When a detection object exists in the second detection result, the second detection result may only include the detection object, or the second detection result may also include the second detection object and the confidence that the second detection object is a set obstacle.

[0090] Step S120, matching the first detection object in the second detection result to obtain a matching result.

[0091] The second detection result may include a second detection object. In some examples, the second detection object may be any detection object obtained by performing obstacle detection based on the second perception data. In this example, step S120 may include: matching the first detection object with each second detection object in the second detection result to obtain a matching result of the first detection object for the second detection object. In other words, the detection objects in the second detection result may be fully matched.

[0092] In some other examples, step S120 may also include steps S121 and S122.

[0093] Step S121, searching the second detection result for a second detection object that meets the set condition.

[0094] Step S122, matching the first detection object with the second detection object to obtain a matching result of the first detection object with respect to the second detection object.

[0095] The setting condition in this embodiment can be set based on at least one of the type and position of the first detection object. The detection objects in the second detection result are screened by at least one of the type and position of the first detection object to obtain the second detection object that meets the setting condition, and then the second detection object that meets the setting condition is matched with the first detection object, which can improve the matching speed and matching efficiency.

[0096] The matching result in this embodiment includes that the target detection object that matches the first detection object does not exist in the second detection result, and the target detection object that matches the first detection object exists in the second detection result. In some examples, the detection object does not exist in the second detection result. In the case where the detection object does not exist in the second detection result, it is determined that the target detection object that matches the first detection object does not exist in the second detection result. In other examples, the detection object exists in the second detection result. In the case where the detection object exists in the second detection result, the first detection object is matched in the second detection result, and it is determined whether there is a target detection object that matches the first detection object in the second detection result.

[0097] In some examples, there are multiple second detection objects that meet the set conditions in the second detection result. The first detection object is matched with each second detection object to obtain multiple candidate matching results; and the final matching result is determined based on each candidate matching result. In one example, the second detection object with the highest matching degree among all candidate matching results can be used as the target detection object.

[0098] In some examples, when there is a second detection object in the second detection result that does not match any first detection object, the second detection object and its confidence level are added to the maintenance table for subsequent matching.

[0099] Step S130, adjusting the first confidence level according to the matching result to obtain a second confidence level that the first detection object is a set obstacle; wherein the second confidence level is a confidence level corresponding to the second sampling time.

[0100] In some examples, step S130 may include step S131 or step S132.

[0101] Step S131: when the matching result indicates that there is a target detection object matching the first detection object in the second detection result, the first confidence level is increased by a first adjustment value to obtain a second confidence level.

[0102] Step S132: when the matching result indicates that there is no target detection object matching the first detection object in the second detection result, the first confidence level is reduced by a second adjustment value to obtain a second confidence level.

[0103] The first adjustment value or the second adjustment value may be a preset fixed value. In some examples, the second detection result may include only the second detection object. In this example, assuming that the final obstacle detection result needs to be obtained within 10 frames, the first confidence is a floating point number between 0 and 1. The first adjustment value or the second adjustment value may be, for example, 0.1.

[0104] The first adjustment value or the second adjustment value may also be a dynamically changing value. In some examples, the second detection result may include the second detection object and the confidence that the second detection object is a set obstacle at the second sampling time. The first adjustment value or the second adjustment value may be set based on the confidence that the second detection object is a set obstacle in the second detection result. In one example, the confidence that the second detection object is a set obstacle at the second sampling time is P1, and the first adjustment value or the second adjustment value may be, for example, the product of P1 and a preset fixed value.

[0105] In this embodiment, the first adjustment value and the second adjustment value may be the same or different.

[0106] The adjustment in this embodiment is to calibrate based on the first confidence level to obtain a more accurate confidence level. This calibration process is actually a fine-tuning rather than a complete replacement.

[0107] In one example, when the second confidence obtained by adjusting the first confidence is greater than 1, the second confidence is set to a preset maximum value (such as 1). When the second confidence obtained by adjusting the first confidence is less than 0, the second confidence is set to a preset minimum value (such as 0).

[0108] The embodiment of the present disclosure matches the first detection object in the second detection result, and dynamically adjusts the first confidence that the first detection object is a set obstacle based on the matching result to obtain a second confidence that the first detection object is the set obstacle. By dynamically adjusting the first confidence that the first detection object is a set obstacle in real time at different sampling times, a more accurate second confidence can be obtained at each sampling time, thereby improving the accuracy of the obstacle detection result.

[0109] In the related technology, there is a solution for obstacle detection through visual perception data, but the data collected by visual sensors is easily affected by environmental factors such as weather and lighting, resulting in false detection or inaccurate ranging.

[0110] Based on this, in some embodiments, the mobile device is provided with a camera. The second perception data may include second visual perception data collected by the camera. The visual perception data is an image collected by the camera.

[0111] In some examples, the first detection object in this embodiment includes dynamic potential obstacles and / or static potential obstacles. Dynamic potential obstacle detection and potential impassable area detection can be performed on the image separately. For dynamic potential obstacles, the first visual perception result determined based on the first visual perception data of the first sampling time can be used to predict the second visual perception result of the second sampling time according to the tracker principle, and the first target visual perception result of the second sampling time can be obtained by combining the third visual perception result determined based on the second visual perception data of the second sampling time. For potential impassable areas, the fourth visual perception result determined based on the first visual perception data and the fifth visual perception result determined based on the second visual perception data can be superimposed to obtain the second target visual perception result of the potential impassable area at the second sampling time. The accuracy of the visual perception results can be improved by fusing the visual perception results in time sequence to obtain the visual perception results of the current sampling time.

[0112] The visual perception results in this embodiment may include dynamic potential obstacle detection results and potential impassable area detection results. The dynamic potential obstacle detection results may include at least one of the following: timestamp, ID of the detection object, type of the detection object, confidence that the detection object is of this type, motion state of the detection object, angle deviation between the detection object and the vehicle, and detection frame of the detection object. The motion state of the detection object may include at least one of the following: longitudinal relative position, lateral relative position, longitudinal relative speed, lateral relative speed, longitudinal acceleration, and lateral acceleration. The potential impassable area detection results include a map of fixed length and width, the probability of the existence of an obstacle at the corresponding position of each grid in the map, and the corresponding obstacle type.

[0113] In this embodiment, the dynamic potential obstacle detection results and the potential impassable area detection results are fused to obtain an impassable area including dynamic potential obstacles and static potential obstacles after fusion. The contour of the potential obstacle can be extracted from the fused passable area. Each potential obstacle result can include at least one of the following: timestamp, ID of the first detection object, type of the first detection object, confidence that the first detection object is of this type, motion state of the first detection object, angle deviation between the first detection object and the vehicle, and contour point set of the first detection object.

[0114] In the related technology, there is also a scheme for obstacle detection through ultrasonic sensing data, but this detection scheme is not accurate enough in judging the height and specific location of obstacles. For example, when ultrasonic waves sense low curbs and road gaps, they will produce strong echoes, resulting in misjudgment of their height. For another example, the location of obstacles is determined by triangulation positioning, but due to the reflection of multiple ultrasonic waves, there are deviations in the detection results. These deviations will cause the automatic emergency braking to be falsely triggered, thus affecting the passengers' riding experience.

[0115] Based on this, in some embodiments, the mobile device is provided with an ultrasonic probe. The second sensing data may include ultrasonic sensing data collected by the ultrasonic probe. The ultrasonic sensing data is detection information of the ultrasonic probe. In some examples, the detection information may include information such as echo distance, echo height, echo width, echo confidence, and wave-emitting probe.

[0116] When a probe's wave is received by itself, it is called a direct echo. Through the direct echo distance, it can be obtained that there are potential obstacles on the circle with the installation position of the probe as the center and the echo distance as the radius. When a probe's wave is received by other probes, it is called an indirect echo. Through the indirect echo distance, it can be obtained that there are potential obstacles on the ellipse with the transmitting and receiving probe as the focus and the echo distance as the major axis. When the same wave is received by different probes, the intersection of the ellipse and the circle can be obtained through the triangulation positioning algorithm, that is, the location point of the potential obstacle can be obtained.

[0117] Since the triangulation positioning algorithm involves echoes from multiple probes, the position of the obstacle is obtained in an implicit way. A slight deviation in the detection distance of the direct echo or indirect echo will cause the final result position to deviate further. For automatic emergency braking, false alarms due to inaccurate positions will greatly affect the passenger experience. Therefore, after triangulation positioning, the location points of the obstacles determined by several frames of historical ultrasonic sensing data will be densely clustered, and only the location points with multiple triangulation positioning results around them will be used as valid obstacle location points. The accuracy of the ultrasonic sensing results can be improved by fusing the ultrasonic sensing results in time series to obtain the ultrasonic sensing results of the current sampling time.

[0118] In some embodiments, the mobile device is provided with a camera and an ultrasonic probe. The first perception data includes first visual perception data collected by the camera. The second perception data includes ultrasonic perception data collected by the ultrasonic probe. The first detection result is determined based on the first visual perception data, and the second detection result is determined based on the ultrasonic perception data.

[0119] In this embodiment, the matching result can be obtained by matching the first detection object determined based on the first visual perception data in the second detection result determined based on the ultrasonic perception data. That is, the visual perception result of the first sampling time is matched with the ultrasonic perception result of the second sampling time to obtain the matching result. Then, the first confidence level that the first detection object is a set obstacle at the first sampling time is adjusted according to the matching result. In this way, the ultrasonic perception result and the visual perception result can be integrated, thereby further improving the accuracy of the matching result.

[0120] In some examples, the first detection result may include a first position of a first detection object in a set coordinate system determined based on the first visual perception data, and the second detection result may include a second position of a second detection object in the set coordinate system determined based on the ultrasonic perception data.

[0121] The set coordinate system in this embodiment may be the vehicle coordinate system, which may be the initial coordinate system when the vehicle is turned on, or the vehicle coordinate system at a specific moment.

[0122] In some examples, determining a first position of a first detection object in a set coordinate system based on first visual perception data may include: determining a contour point set of the first detection object in the set coordinate system based on the first visual perception data; and determining a first position of the first detection object based on the contour point set.

[0123] In some examples, determining the second position of the second detection object in the set coordinate system based on the ultrasonic sensing data may include: determining the ultrasonic wave point position of the second detection object in the set coordinate system based on the ultrasonic sensing data; and using the ultrasonic wave point position as the second position of the second detection object.

[0124] Optionally, step S122 may include: obtaining a matching result according to the first position and the second position. In some examples, the first position may be the position of the contour corresponding to the contour point set of the first detection object, and the second position may be the position of the ultrasonic wave point corresponding to the second detection object. In the case where the ultrasonic wave point is located within the contour corresponding to the contour point set of the first detection object, it is determined that the first detection object and the second detection object match. In the case where the ultrasonic wave point is located within a preset range outside the contour corresponding to the contour point set of the first detection object, it is determined that the first detection object and the second detection object match successfully. In the case where the ultrasonic wave point is located outside the preset range outside the contour corresponding to the contour point set of the first detection object, it is determined that the first detection object and the second detection object match failed.

[0125] Optionally, step S122 may include: determining a first distance between the first detection object and the second detection object according to the first position and the second position; and obtaining a matching result according to the first distance. In some examples, when the first distance is less than a threshold, it is determined that the second detection object successfully matches the first detection object; when the first distance is greater than or equal to the threshold, it is determined that the second detection object fails to match the first detection object.

[0126] Considering that the ultrasonic probe can detect a limited distance, the first detection object that is far from the vehicle will not be detected by the ultrasonic probe. Based on this, in some embodiments, step S122 may include: determining whether the first detection object is an object that the ultrasonic probe can detect based on the relative position between the first detection object and the ultrasonic probe; if the ultrasonic probe can detect the first detection object, matching the first detection object with the second detection object to obtain a matching result. In this way, if the ultrasonic probe cannot detect the first detection object, there is no need to match the first detection object with the second detection object, thereby improving the matching speed and matching efficiency.

[0127] In some examples, step S122 may include: determining whether the first detection object is the object with the shortest relative distance from the ultrasonic probe that can be detected by the ultrasonic probe according to the relative position between the first detection object and the ultrasonic probe; if the first detection object is the object with the closest relative distance from the ultrasonic probe that can be detected by the ultrasonic probe, matching the first detection object with the second detection object to obtain a matching result. For the detection object with the closest distance to the ultrasonic probe, the perception of the ultrasonic probe will not be affected by the reflected waves of other detection objects, and the perception result is more accurate, which can further improve the matching accuracy. In addition, if the first detection object is not the object with the closest relative distance from the ultrasonic probe that can be detected by the ultrasonic probe, the first detection object does not need to be matched with the second detection object, which can also improve the matching speed and matching efficiency.

[0128] Optionally, the relative position of each first detection object and the ultrasonic probe can be calculated, and the relative distance between each first detection object and the ultrasonic probe can be determined based on the relative position, and the first detection object with the shortest relative distance is taken as the object with the shortest relative distance to the ultrasonic probe.

[0129] In one example, the distance between the first position of each first detection object in the set coordinate system and the installation position of the ultrasonic probe in the set coordinate system can be calculated, and the first detection object with the shortest distance can be regarded as the object with the shortest relative distance to the ultrasonic probe, that is, the object closest to the ultrasonic probe.

[0130] In one example, the first position of the first detection object can be determined by the contour corresponding to the contour point set of the first detection object. In this example, the relative position of each contour point and the ultrasonic probe can be calculated, the relative distance between each contour point and the ultrasonic probe can be determined, and the first detection object corresponding to the contour point with the shortest relative distance is taken as the object with the shortest relative distance to the ultrasonic probe.

[0131] Since the ultrasonic probe usually has a certain transmission angle, a conical beam will be formed during transmission. The ultrasonic probe has a high probability of detecting an object at the center of the beam, and has a low probability of detecting an object at the edge of the beam. The ultrasonic probe cannot detect an object outside the beam. Based on this, in some embodiments, step S122 may include: obtaining an angle deviation of the first detection object relative to the ultrasonic probe; and when the angle deviation is within a set threshold range, matching the first detection object with the second detection object to obtain a matching result.

[0132] In this embodiment, the angle deviation of the first detection object relative to the ultrasonic probe is the angle between the line between the first detection object and the ultrasonic probe and the direction of the ultrasonic probe. The direction of the ultrasonic probe is determined based on the installation angle of the ultrasonic probe. In some examples, such as Figure 4 As shown, the probe installation direction can form an angle with the set direction (for example, the length direction of the vehicle, i.e., the y direction), and this angle is the probe installation angle θ1. In one example, the first vector can be determined based on the line between the first detection object and the ultrasonic probe, the second vector can be determined based on the direction of the ultrasonic probe, and the angle between the first vector and the second vector is used as the angle deviation θ2.

[0133] The threshold range in this embodiment can be determined based on the detection angle range of the ultrasonic probe. In other words, the threshold range can be determined based on the spatial angle region covered by the ultrasonic probe when emitting ultrasonic waves.

[0134] In some examples, when the angle deviation is outside a set threshold range, a flag value of invalid match is set for the matching result of the first detection object and the second detection object. The flag value indicates that the matching of the first detection object and the second detection object is an invalid match.

[0135] In the embodiment of the present disclosure, the first detection object is matched with the second detection object only when the angle deviation is within the set threshold range, which can improve the matching speed and matching efficiency. In the case where the angle deviation is outside the set threshold range, there is no need to match the first detection object with the second detection object, which can eliminate invalid matches and avoid the accuracy of the detection result being affected by the invalid matching results.

[0136] In some embodiments, the first detection result includes a first position of the first detection object in a set coordinate system determined based on the first visual perception data, and the second detection result includes a perception distance of the ultrasonic probe determined based on the ultrasonic perception data. In this embodiment, step S122 may include: determining a second distance between the first detection object and the ultrasonic probe according to the first position of the first detection object in the set coordinate system determined based on the first visual perception data, the position of the movable device in the set coordinate system, and the installation position of the ultrasonic probe on the movable device; obtaining a matching result based on the second distance and the perception distance of the ultrasonic probe determined based on the ultrasonic perception data.

[0137] In this embodiment, the sensing distance is the detection distance of the ultrasonic probe detecting the second detection object, that is, the distance between the ultrasonic probe and the second detection object. The second distance in this embodiment is the actual distance between the position of the first detection object determined based on the first visual perception data and the ultrasonic probe. By comparing the detection distance between the ultrasonic probe and the second detection object and the actual distance between the ultrasonic probe and the first detection object, it is determined whether the first detection object and the second detection object match. In this way, the ultrasonic perception results and the visual perception results can be fused, thereby further improving the accuracy of the matching results.

[0138] The set coordinate system in this embodiment may be, for example, the vehicle coordinate system. The first position may be, for example, a position determined by a contour point set of the first detection object in the set coordinate system based on the visual perception data.

[0139] In some examples, the step of obtaining a matching result based on the second distance and the perceived distance may include: determining that the first detection object and the second detection object are successfully matched when the absolute difference between the perceived distance and the second distance is less than a first threshold; or determining that the first detection object and the second detection object are unmatched when the absolute difference between the perceived distance and the second distance is greater than the first threshold.

[0140] Since when the perceived distance is less than the second distance and the perceived distance is much smaller than the second distance, that is, the distance at which the probe detects the second detection object is much smaller than the actual distance between the probe and the first detection object, there may be a nearest detection object that is not detected by the visual sensor, and at this time, the match is considered to be an invalid match. In some examples, when the perceived distance is less than the second distance and the difference between the perceived distance and the second distance is less than the second threshold, the match result of the first detection object and the second detection object is set to an invalid match mark value. The invalid match mark value indicates that the matching process of the first detection object and the second detection object is an invalid match.

[0141] In this embodiment, when the matching process between the first detection object and the second detection object is an invalid match, there is no need to adjust the first confidence level using the invalid matching result, thereby preventing the accuracy of the detection result from being affected by the invalid matching result.

[0142] Since when the perceived distance is greater than the second distance and the perceived distance is much greater than the second distance, that is, the distance at which the probe detects the second detection object is much greater than the actual distance between the probe and the first detection object, it is indicated that the probe has not detected the first detection object, and at this time, it is considered that the first detection object and the second detection object have failed to match. In some examples, when the perceived distance is greater than the second distance and the difference between the perceived distance and the second distance is greater than a third threshold, it is determined that the first detection object and the second detection object have failed to match.

[0143] In some examples, when the absolute difference between the second distance and the perceived distance is between the second threshold and the third threshold, it is determined that the first detection object and the second detection object are matched successfully. When the perceived distance is less than the second distance and the difference between the perceived distance and the second distance is less than the second threshold, the match result of the first detection object and the second detection object is set to an invalid match flag value. When the perceived distance is greater than the second distance and the difference between the perceived distance and the second distance is greater than the third threshold, it is determined that the first detection object and the second detection object are matched unsuccessfully.

[0144] In some embodiments, the mobile device is provided with a camera. The first perception data includes first visual perception data collected by the camera, and the second perception data includes second visual perception data collected by the camera. The first detection result is determined based on the first visual perception data, and the second detection result is determined based on the second visual perception data. By matching the visual perception result of the first sampling time with the visual perception result of the second sampling time, a matching result is obtained, and then the first confidence that the first detection object is a set obstacle at the first sampling time is adjusted, so that the visual perception results corresponding to different sampling times are fused in time sequence to obtain the final visual perception result, thereby improving the matching accuracy and thus improving the accuracy of obstacle detection.

[0145] In this embodiment, step S122 may include: acquiring a first contour area of ​​the first detection object in a set coordinate system, and a second contour area of ​​the second detection object in the set coordinate system; and obtaining a matching result based on the first contour area and the second contour area.

[0146] In some examples, the step of obtaining a matching result based on the first contour area and the second contour area may include: determining an overlapping area between the first contour area and the second contour area based on the first contour area and the second contour area; and determining a matching result of the first detection object and the second detection object based on the overlapping area.

[0147] In some examples, the step of determining the matching result of the first detection object and the second detection object based on the overlapping area may include: when the area of ​​the overlapping area is greater than or equal to a threshold, determining that the first detection object and the second detection object are successfully matched; or, when the area of ​​the overlapping area is less than the threshold, determining that the first detection object and the second detection object are unsuccessful matches.

[0148] In this embodiment, the first contour area is an area determined by a first contour point set of the first detection object in a set coordinate system, and the second contour area is an area determined by a second contour point set of the second detection object in the set coordinate system.

[0149] In some examples, there may be multiple ultrasonic probes. A first detection object that can be detected by the ultrasonic probe and whose angle deviation is within a set threshold range can be matched with a second detection object obtained based on the sensing data of each probe. Each matching result obtained is fused to obtain a final matching result.

[0150] In this example, the final first adjustment value or second adjustment value corresponding to the third match can be obtained based on the first adjustment value or second adjustment value corresponding to each probe. In one example, an indicator function can be set to indicate whether the first detection object is an object that the probe can detect. When the first detection object is not an object that the probe can detect, the indicator function can be set to 0. When the first detection object is an object that the probe can detect, the indicator function can be set to 1.

[0151] In one example, an angle coefficient indicating whether the first detection object is within the detection angle range of the ultrasonic probe may also be set. When the first detection object is not within the detection angle range of the ultrasonic probe, the angle coefficient may be set to 0. When the first detection object is within the detection angle range of the ultrasonic probe, the angle coefficient may be set to a preset angle coefficient. The preset angle coefficient may be a fixed value or a variable value related to the angle deviation of the first detection object with respect to the ultrasonic probe. When the preset angle coefficient is a variable value related to the angle deviation of the first detection object with respect to the ultrasonic probe, an adjustment value reflecting the degree of matching between the first detection object and the second detection object may be obtained, so as to obtain a more accurate second confidence through the adjustment value.

[0152] In one example, a distance coefficient may be set to indicate whether the distance deviation between the second distance between the first detection object and the ultrasonic probe and the perceived distance of the ultrasonic probe meets a preset condition. When the distance deviation between the second distance between the first detection object and the ultrasonic probe and the perceived distance of the ultrasonic probe does not meet the preset effective matching condition, the distance coefficient may be set to 0. When the distance deviation between the second distance between the first detection object and the ultrasonic probe and the perceived distance of the ultrasonic probe meets the preset matching success condition, the distance coefficient may be set to a positive preset distance coefficient. When the distance deviation between the second distance between the first detection object and the ultrasonic probe and the perceived distance of the ultrasonic probe meets the preset matching failure condition, the distance coefficient may be set to a negative preset distance coefficient. The positive preset distance coefficient and / or the negative preset distance coefficient may be a fixed value or a variable value related to the distance deviation. When the preset distance coefficient is a variable value related to the distance deviation, an adjustment value reflecting the degree of matching between the first detection object and the second detection object may be obtained, so as to obtain a more accurate second confidence through the adjustment value.

[0153] In this example, the first adjustment value or the second adjustment value corresponding to each probe can be determined according to the indicator function, the angle coefficient and the distance coefficient. For example, the product of the indicator function, the angle coefficient and the distance coefficient is used as the first adjustment value or the second adjustment value corresponding to each probe. The final first adjustment value or the second adjustment value corresponding to the third match can be the sum of the first adjustment value or the second adjustment value corresponding to each probe.

[0154] In some embodiments, the second sensed data may be sensed data collected by multiple sensors. Obstacle detection is performed based on the second sensed data collected by the mobile device, and multiple second detection results may be obtained. For example, the multiple second detection results are respectively a second detection result A, a second detection result B, and a second detection result C. Multiple second confidence levels may be obtained by matching the first detection object in the first detection result with each second detection result.

[0155] In this embodiment, there are multiple second detection results and multiple second confidence levels, and the second detection results and the second confidence levels correspond one to one.

[0156] In some examples, the second perception data may include second visual perception data collected by a camera and ultrasonic perception data collected by an ultrasonic probe. The second detection result A obtained by performing obstacle detection based on the second visual perception data includes a third position of the second detection object at the set coordinates. The second detection result B obtained by performing obstacle detection based on the ultrasonic perception data may include a second position of the second detection object at the set coordinates. The second detection result C obtained by performing obstacle detection based on the ultrasonic perception data may also include the perception distance of the ultrasonic probe. The first perception data may include the first visual perception data collected by a camera. The first detection result obtained by performing obstacle detection based on the first perception data includes a first position of the first detection object at the set coordinate system.

[0157] In this example, a first match is performed between the first position corresponding to the first detection object and the third position corresponding to the second detection object in the second detection result A to obtain a first matching result. A second match is performed between the first position corresponding to the first detection object and the second position corresponding to the second detection object in the second detection result B to obtain a second matching result. A third match is performed between the second distance between the first detection object and the ultrasonic probe and the perception distance of the ultrasonic probe in the second detection result C to obtain a third matching result. It can be understood that the first match is the match between the visual perception result of the first sampling time and the visual perception result of the second sampling time. The second match and the third match are both the match between the visual perception result of the first sampling time and the ultrasonic perception result of the second sampling time.

[0158] When performing the third matching, the first detection object in this embodiment may be the first detection object that is at the shortest relative distance to the ultrasonic probe and / or is within the detection angle range of the ultrasonic probe.

[0159] According to the first matching result, the first confidence level corresponding to the first match in the first detection result is adjusted to obtain a second confidence level corresponding to the first match. According to the second matching result, the first confidence level corresponding to the second match in the first detection result is adjusted to obtain a second confidence level corresponding to the second match. According to the third matching result, the first confidence level corresponding to the third match in the first detection result is adjusted to obtain a second confidence level corresponding to the third match.

[0160] In some examples, step S122 may include steps T110 to T170.

[0161] Step T110, obtaining a first contour area of ​​the first detection object in a set coordinate system determined based on the first visual perception data, and a second contour area of ​​the second detection object in the set coordinate system determined based on the second visual perception data.

[0162] Step T120, performing a first matching based on the first contour area and the second contour area to obtain a first matching result.

[0163] Step T130, obtaining a first contour area of ​​the first detection object in a set coordinate system determined based on the first visual perception data, and a second position of the second detection object in the set coordinate system determined based on the ultrasonic perception data.

[0164] Step T140, performing a second match based on the first contour area and the second position to obtain a second matching result.

[0165] Step T150, determining whether the first detection object is an object that can be detected by the ultrasonic probe, and whether the angle deviation of the first detection object relative to the ultrasonic probe is within a set threshold range;

[0166] Step T160, when the ultrasonic probe is able to detect the first detection object and the angle deviation is within the set threshold range, determine the second distance between the first detection object and the ultrasonic probe according to the first position of the first detection object in the set coordinate system determined based on the first visual perception data, the position of the movable device in the set coordinate system and the installation position of the ultrasonic probe on the movable device.

[0167] Step T170, obtaining a third matching result based on the second distance and the sensing distance of the ultrasonic probe determined based on the ultrasonic sensing data.

[0168] The specific matching and confidence adjustment process in this embodiment can refer to the aforementioned embodiment and will not be described again here.

[0169] To ensure the efficiency and accuracy of the system, when the second confidences of the three matching methods of the first detection object are all below a certain threshold, the system will no longer maintain the first detection object and remove it from the maintenance table. This mechanism helps reduce false alarms and resource waste, allowing the system to focus on more reliable obstacle detection.

[0170] In some embodiments, the matching method for matching the first detection result and the second detection result may only include a partial matching method among the first matching, the second matching, and the third matching, which will not be described in detail here.

[0171] In some embodiments, different matching weights may be assigned to multiple second detection results. Accordingly, the second confidences corresponding to different matching methods correspond to different matching weights. For example, the matching weight of the second confidence corresponding to the first match is the first matching weight, the matching weight of the second confidence corresponding to the second match is the second matching weight, and the matching weight of the second confidence corresponding to the third match is the third matching weight.

[0172] In this embodiment, the obstacle detection method may further include: obtaining the confidence that the first detection object is a set obstacle according to the plurality of second confidences and the matching weights corresponding to the plurality of second detection results. The confidence is the target confidence at the second sampling time. In some examples, the sum of the products of each second confidence and the matching weight corresponding to the second confidence may be calculated to obtain the target confidence that the first detection object is a set obstacle.

[0173] Considering that different sensing methods detect the same type of detection object, the detection results have different accuracy, in some examples, different matching weights can be assigned to different second detection results obtained by different matching methods for the same type of detection object. Accordingly, the second confidences corresponding to the same type of detection object and different matching methods correspond to different matching weights.

[0174] In this embodiment, the type of the detection object can be divided according to at least one of the area and the motion state of the detection object. The area of ​​the detection object can be the area of ​​the detection object in the top view obtained based on the visual perception data. In some examples, the type of the detection object can be divided into a first type and a second type according to the size of the area, the area of ​​the detection object of the first type is less than or equal to the first threshold, and the area of ​​the detection object of the second type is greater than the first threshold. For example, the detection object of the first type is a pedestrian, and the detection object of the second type is a vehicle.

[0175] In other examples, the types of detection objects can be divided into static type and dynamic type according to the motion state. For example, the static type detection object is a static vehicle, and the dynamic type detection object is a dynamic vehicle.

[0176] In other examples, the types of detected objects can be divided into static first type, static second type, dynamic first type and dynamic second type according to the area and motion state of the object. The area of ​​the static first type of detected object is less than or equal to the first threshold, the area of ​​the static second type of detected object is greater than the first threshold, the area of ​​the dynamic first type of detected object is less than or equal to the first threshold, and the area of ​​the dynamic second type of detected object is greater than the first threshold. For example, the static first type of detected object is a static pedestrian, the static second type of detected object is a static vehicle, the dynamic first type of detected object is a dynamic pedestrian, and the dynamic second type of detected object is a dynamic vehicle.

[0177] In other examples, the types of detected objects can be divided into dynamic whitelist obstacles (such as dynamic vehicles), static whitelist obstacles (such as static cones), and general obstacles according to whether they are whitelist obstacles and their motion state. Among them, general obstacles are obstacles that are not in the preset whitelist. In other examples, the general obstacle types can be further refined according to the area, shape, motion state, etc. of the object.

[0178] In this embodiment, different matching weights can be assigned to the second confidence corresponding to the same type of detection object in each matching mode based on the advantages of visual sensors in identifying dynamic objects and small objects, and the advantages of ultrasonic sensors in identifying static objects and large objects.

[0179] For dynamic vehicles, it is necessary to flexibly handle their rapidly changing characteristics. Since visual sensors have advantages in identifying the shape and movement trajectory of dynamic vehicles, ultrasonic information is used to verify the existence of the vehicle and ensure the consistency of multiple perceptions. Therefore, for dynamic vehicles, the matching weight corresponding to the first match can be set higher than the matching weights corresponding to the second and third matches.

[0180] For dynamic pedestrians, due to their irregular movement trajectories and small size, they are more dependent on the perception results of visual sensors. Visual sensors can provide relatively accurate pedestrian detection and identify their movement direction and speed. However, ultrasonic waves have poor detection effects on pedestrians. Therefore, for dynamic pedestrians, the matching weight corresponding to the first match can be set higher than the matching weights corresponding to the second and third matches. And, in one example, compared with dynamic vehicles, the matching weights corresponding to the second and third matches can be set lower.

[0181] For static vehicles, they usually have clear outlines and large volumes, so ultrasonic ranging information is particularly important in identifying static vehicles, which can accurately depict the outline of the vehicle, thereby reducing the false alarm rate. Visual sensors are used to confirm the specific shape and position of the vehicle. Therefore, for static vehicles, the matching weights corresponding to the second and third matches can be set higher than the matching weight corresponding to the first match.

[0182] For static pedestrians, visual sensors are still the main means of perception. The posture and position of pedestrians are identified based on visual perception data, and ultrasonic data is used as auxiliary information to help confirm the existence of pedestrians. Therefore, for static pedestrians, the matching weight corresponding to the first match can be set higher than the matching weights corresponding to the second and third matches.

[0183] For static universal obstacles, since these obstacles usually have fixed shapes and positions, their detection needs to rely on the accurate distance measurement capability of ultrasound to provide accurate distance information, which can help construct the accurate outline of the obstacle. Visual sensors are used to supplement the recognition of shape and material. Therefore, for static universal obstacles, the matching weights corresponding to the second and third matches can be set higher than the matching weight corresponding to the first match.

[0184] In some examples, the first confidence corresponding to each matching method in the first detection result can be adjusted according to the first matching result, the second matching result, and the third matching result, respectively, to obtain the second confidence corresponding to each matching method. According to the multiple second confidences and the matching weights corresponding to the multiple second confidences, the target confidence that the first detection object is a set obstacle at the second sampling time is obtained.

[0185] The disclosed embodiment comprehensively considers the accuracy of the detection results of visual sensors and ultrasonic sensors for different types of obstacles, and sets different matching weights based on the respective advantages of visual sensors and ultrasonic sensors, so as to more effectively balance the positive and false alarm rates, improve the accuracy and reliability of overall perception, and achieve accurate identification and stable tracking of obstacles. This flexible strategy adjustment not only improves the intelligence level of the system, but also provides a safer guarantee for the emergency braking function, while reducing the false alarm rate.

[0186] The present disclosure also provides a vehicle control method. Figure 5 As shown, the vehicle control method may include steps S310 to S320.

[0187] Step S310, obtaining a second confidence level that the first detection object is a set obstacle.

[0188] Step S320: Control the vehicle to execute an obstacle avoidance strategy according to the second confidence level.

[0189] Among them, the second confidence level is obtained by adjusting the first detection object according to the matching result to set the first confidence level of the obstacle, the matching result is the matching result obtained by matching the first detection object in the first detection result in the second detection result, the second detection result is obtained by performing obstacle detection based on the second perception data collected by the movable device, the first detection result is obtained by performing obstacle detection based on the first perception data collected by the movable device, the first sampling time of the first perception data is before the second sampling time of the second perception data, the first confidence level is the confidence level corresponding to the first sampling time, and the second confidence level is the confidence level corresponding to the second sampling time.

[0190] In this embodiment, the obstacle detection result corresponding to each first detection object is received, the first detection object with the greatest risk is determined according to the confidence level corresponding to each first detection object, and the relative distance and relative speed between the first detection object with the greatest risk and the vehicle are obtained, and combined with the system state at this time, it is determined whether to control the vehicle to execute the obstacle avoidance strategy. In the case of controlling the vehicle to execute the obstacle avoidance strategy, the vehicle braking flag is set to the activated state, and the vehicle braking strategy is generated, and then the current vehicle is automatically emergency braked according to the vehicle braking strategy.

[0191] The first detection object in this embodiment is a method for obtaining the second confidence level of the set obstacle. Please refer to the description in the previous embodiment of the present disclosure and will not be repeated here.

[0192] According to the more accurate confidence level obtained by the aforementioned obstacle detection method of the present disclosure, an accurate obstacle detection result is obtained based on the confidence level, thereby determining whether to control the vehicle to execute the obstacle avoidance strategy, which can improve control accuracy and reduce false triggering.

[0193] The embodiment of the present disclosure also provides a map display method, which may include steps S410 to S420.

[0194] Step S410, obtaining obstacle detection results in the scene area where the vehicle is located.

[0195] Step S420: display the target detection object indicated in the obstacle detection result.

[0196] Among them, the obstacle detection result includes a second confidence level that the first detection object is a set obstacle, the second confidence level is obtained by adjusting the first confidence level that the first detection object is a set obstacle according to the matching result, the matching result is a matching result obtained by matching the first detection object in the first detection result in the second detection result, the second detection result is obtained by performing obstacle detection based on second perception data collected by the movable device, the first detection object is obtained by performing obstacle detection based on first perception data collected by the movable device, the first sampling time of the first perception data is before the second sampling time of the second perception data, the first confidence level is the confidence level corresponding to the first sampling time, and the second confidence level is the confidence level corresponding to the second sampling time.

[0197] The method for obtaining the obstacle detection result in this embodiment can refer to the description in the previous embodiments of the present disclosure, and will not be repeated here.

[0198] In this embodiment, the target detection object may be any detection object whose potential risk reaches a threshold. Step S420 may include displaying the position information of the target detection object.

[0199] By using the more accurate confidence obtained by the obstacle detection method disclosed above, accurate obstacle detection results are obtained based on the confidence, thereby obtaining a high-precision target road scene map. During vehicle driving, the target road scene map corresponding to the vehicle position is displayed, so that the driver can accurately know the road scene corresponding to the vehicle position, thereby improving driving safety.

[0200] The present disclosure also provides an electronic device. Figure 6 As shown, the electronic device 1000 may include a memory 1010 and a processor 1020. The memory 1010 may be used to store computer instructions, and the processor 1020 may be used to call computer instructions from the memory 1010 to execute all or part of the steps of any obstacle detection method in the aforementioned embodiments of the present disclosure. It should be noted that the processor 1020 may include one or more processors to execute instructions, and the memory 1010 may also include one or more memories to store computer instructions. In one example, the electronic device 1000 may be a cloud server.

[0201] The present disclosure also provides a vehicle. Figure 7As shown, the vehicle 1100 may include a memory 1110 and a processor 1120. The memory 1110 may be used to store computer instructions, and the processor 1120 may be used to call computer instructions from the memory 1110 to execute all or part of the steps of any obstacle detection method in the aforementioned embodiments of the present disclosure. It should be noted that the processor 1120 may include one or more processors to execute instructions, and the memory 1110 may also include one or more memories to store computer instructions. The vehicle may be, for example, a vehicle with auxiliary driving functions such as memory parking and / or memory driving.

[0202] The present disclosure also provides a chip. Figure 8 As shown, the chip 2000 may include a storage unit 2010 and a processing unit 2020. The storage unit 2010 is used to store a computer program. The processing unit 2020 is configured to implement a method according to any embodiment of the present disclosure when executing the computer program stored in the storage unit.

[0203] The embodiment of the present disclosure also provides a non-volatile computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the method described in any one of the aforementioned embodiments of the present disclosure is implemented. Optionally, the computer-readable storage medium may be a non-transitory storage medium, but is not limited thereto, and may also be a transient storage medium.

[0204] The embodiments of the present disclosure further provide a computer program product, which may include a computer program. When the computer program is executed by a processor, any method in the aforementioned embodiments of the present disclosure may be implemented.

[0205] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement any method in the foregoing embodiments of the present disclosure.

[0206] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0207] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0208] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0209] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0210] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0211] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0212] The flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of an instruction, and the module, a program segment or a part of an instruction contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that it is equivalent to implement it by hardware, implement it by software, and implement it by combining software and hardware.

[0213] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the present disclosure is defined by the appended claims.

Claims

1. An obstacle detection method, characterized in that: include: Obtaining a first detection result of obstacle detection based on first perception data collected by a movable device, and a second detection result of obstacle detection based on second perception data collected by the movable device; wherein a first sampling time of the first perception data is before a second sampling time of the second perception data, and the first detection result includes a first detection object and a first confidence level that the first detection object is a set obstacle; Matching the first detection object in the second detection result to obtain a matching result; The first confidence level is adjusted according to the matching result to obtain a second confidence level that the first detection object is the set obstacle; wherein the second confidence level is a confidence level corresponding to the second sampling time.

2. The method according to claim 1, characterized in that The adjusting the first confidence level according to the matching result to obtain a second confidence level that the first detection object is the set obstacle includes: In a case where the matching result indicates that there is a target detection object matching the first detection object in the second detection result, the first confidence level is increased by a first adjustment value to obtain the second confidence level; or When the matching result indicates that there is no target detection object matching the first detection object in the second detection result, the first confidence level is reduced by a second adjustment value to obtain the second confidence level.

3. The method according to claim 1, characterized in that The matching the first detection object in the second detection result includes: Searching the second detection result for a second detection object that meets a set condition; wherein the set condition is set based on at least one of a category and a position of the first detection object; The first detection object is matched with the second detection object to obtain a matching result of the first detection object for the second detection object.

4. The method according to claim 3, characterized in that The movable device is provided with a camera and an ultrasonic probe, the first perception data includes first visual perception data collected by the camera, the second perception data includes ultrasonic perception data collected by the ultrasonic probe, the first detection result is determined based on the first visual perception data, and the second detection result is determined based on the ultrasonic perception data.

5. The method according to claim 4, characterized in that The first detection result includes a first position of the first detection object in a set coordinate system determined based on the first visual perception data, and the second detection result includes a second position of the second detection object in the set coordinate system determined based on the ultrasonic perception data; The matching of the first detection object and the second detection object to obtain a matching result includes: Determining a first distance between the first detection object and the second detection object according to the first position and the second position; The matching result is obtained according to the first distance.

6. The method according to claim 4, characterized in that The matching of the first detection object with the second detection object to obtain a matching result includes: According to the relative position between the first detection object and the ultrasonic probe, determine whether the first detection object is an object that can be detected by the ultrasonic probe; if the ultrasonic probe can detect the first detection object, match the first detection object with the second detection object to obtain a matching result.

7. The method according to claim 4, characterized in that The matching of the first detection object with the second detection object to obtain a matching result includes: Acquire an angle deviation of the first detection object relative to the ultrasonic probe; When the angle deviation is within a set threshold range, the first detection object is matched with the second detection object to obtain a matching result.

8. The method according to claim 4, characterized in that The first detection result includes a first position of the first detection object in a set coordinate system determined based on the first visual perception data, and the second detection result includes a perception distance of the ultrasonic probe determined based on the ultrasonic perception data; The matching of the first detection object with the second detection object to obtain a matching result includes: determining a second distance between the first detection object and the ultrasonic probe according to the first position, the position of the movable device in the set coordinate system, and the installation position of the ultrasonic probe on the movable device; The matching result is obtained based on the second distance and the perception distance.

9. The method according to claim 3, characterized in that: The movable device is provided with a camera, the first perception data includes first visual perception data collected by the camera, the second perception data includes second visual perception data collected by the camera, the first detection result is determined based on the first visual perception data, and the second detection result is determined based on the second visual perception data; The matching of the first detection object and the second detection object to obtain a matching result includes: Acquire a first contour area of ​​the first detection object in a set coordinate system, and a second contour area of ​​the second detection object in the set coordinate system; A matching result is obtained according to the first contour area and the second contour area.

10. The method according to any one of claims 1 to 9, characterized in that There are multiple second detection results, there are multiple second confidence levels, and the second detection results and the second confidence levels correspond one to one. The method further includes: The confidence that the first detection object is the set obstacle is obtained according to the matching weights corresponding to the plurality of second confidences and the plurality of second detection results.

11. A vehicle control method, characterized in that: include: Acquire a second confidence level that the first detection object is a set obstacle; According to the second confidence level, controlling the vehicle to execute an obstacle avoidance strategy; Among them, the second confidence level is obtained by adjusting the first confidence level that the first detection object is the set obstacle according to the matching result, the matching result is a matching result obtained by matching the first detection object in the first detection result in the second detection result, the second detection result is obtained by performing obstacle detection based on the second perception data collected by the movable device, the first detection result is obtained by performing obstacle detection based on the first perception data collected by the movable device, the first sampling time of the first perception data is before the second sampling time of the second perception data, the first confidence level is the confidence level corresponding to the first sampling time, and the second confidence level is the confidence level corresponding to the second sampling time.

12. An electronic device, characterized in that: including memory and processor, The memory is used to store computer instructions, and the processor is used to call the computer instructions from the memory to execute the method according to any one of claims 1 to 11.

13. A chip, characterized in that: include: A storage unit for storing a computer program; as well as, A processing unit, wherein the processing unit is configured to implement the method according to any one of claims 1 to 11 when executing the computer program stored in the storage unit.

14. A vehicle, characterized in that: The system comprises a memory and a processor, wherein the memory is used to store computer instructions, and the processor is used to call the computer instructions from the memory to execute the method according to any one of claims 1 to 11.

15. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 11 is implemented.

Images