Obstacle identification, model training method, device, equipment and storage medium

Abstract

The present disclosure provides obstacle identification, model training methods, devices, equipment and storage media, which relate to the fields of artificial intelligence technology, especially to the fields of intelligent transportation, autonomous driving, autonomous parking, Internet of Things, and deep learning technology. Among them, the obstacle identification method includes: obtaining at least two frames of detection data from an ultrasonic sensor, each frame of detection data carrying at least two echo features, determining the obstacle distance and obstacle category detected by each frame of detection data based on the at least two echo features of each frame of detection data, and determining the actual obstacle category at at least one target position based on the obstacle distance and obstacle category of at least two frames of detection data. The model training method includes: using a detection data sample set to train a preset network to obtain an obstacle recognition model, wherein each frame of detection data sample in the detection data sample set carries an obstacle label category, an obstacle label distance and at least two echo features.

Description

Technical Field

[0001] The present disclosure relates to the fields of intelligent transportation, autonomous driving, autonomous parking, the Internet of Things, and deep learning technology in artificial intelligence, and in particular to an obstacle recognition and model training method, apparatus, device, and storage medium. Background Art

[0002] With the rapid development of communications and artificial intelligence technologies, autonomous driving, as the core of intelligent driving, has become a hot topic. Obstacle recognition is the most fundamental requirement for autonomous driving. Accurately detecting obstacles around autonomous vehicles ensures accurate obstacle avoidance, ultimately ensuring safety during autonomous driving.

[0003] Ultrasonic sensors are commonly used to detect obstacles within their field of view (FOV). The principle behind using ultrasonic sensors is that they emit ultrasonic waves and detect their echoes. These echoes are then compared with an internally set detection threshold, allowing the first echo greater than the detection threshold to be detected. Furthermore, the sensor can determine the more precise location of the obstacle based on multiple detection frames from different locations or different ultrasonic waves. Because the detection threshold within an ultrasonic sensor is manually set, improperly setting it can lead to low obstacle detection accuracy. Summary of the Invention

[0004] The present disclosure provides an obstacle recognition and model training method, apparatus, device, and storage medium.

[0005] According to a first aspect of the present disclosure, there is provided an obstacle recognition method, comprising:

[0006] Acquire at least two frames of detection data from an ultrasonic sensor, each frame of detection data carrying at least two echo features;

[0007] Determine the distance and category of obstacles detected by each frame of detection data based on at least two echo features of each frame of detection data;

[0008] Determine the actual obstacle category at at least one target position based on the obstacle distance and obstacle category detected by each frame of detection data in the at least two frames of detection data, where the at least one target position is determined based on the obstacle distance detected by the at least two frames of detection data.

[0009] According to a second aspect of the present disclosure, a method for training an obstacle recognition model is provided, comprising:

[0010] Acquire a detection data sample set, wherein each frame of detection data sample in the detection data sample set carries an obstacle labeling category, an obstacle labeling distance, and at least two echo features;

[0011] Inputting each frame of detection data sample in the detection data sample set into a preset network to obtain an obstacle recognition distance and an obstacle recognition category for each frame of detection data sample, wherein the obstacle recognition distance and the obstacle recognition category are determined based on at least two echo features carried by the detection data sample;

[0012] According to the obstacle recognition category and obstacle labeling category, obstacle labeling distance and obstacle recognition distance of each frame of detection data sample, the parameters of the preset network are adjusted to obtain an obstacle recognition model.

[0013] According to a third aspect of the present disclosure, there is provided an obstacle recognition device, comprising:

[0014] an acquisition unit, configured to acquire at least two frames of detection data from the ultrasonic sensor, each frame of detection data carrying at least two echo features;

[0015] an identification unit, configured to determine the distance and type of an obstacle detected in each frame of detection data based on at least two echo features in each frame of detection data;

[0016] A determination unit is configured to determine an actual obstacle category at at least one target position based on the obstacle distance and obstacle category detected by each frame of detection data in the at least two frames of detection data, wherein the at least one target position is determined based on the obstacle distance detected by the at least two frames of detection data.

[0017] According to a fourth aspect of the present disclosure, there is provided an obstacle recognition model training device, comprising:

[0018] An acquisition unit is configured to acquire a detection data sample set, wherein each frame of detection data sample in the detection data sample set carries an obstacle labeling category, an obstacle labeling distance, and at least two echo features;

[0019] a processing unit, configured to input each frame of detection data sample in the detection data sample set into a preset network to obtain an obstacle recognition distance and an obstacle recognition category for each frame of detection data sample, wherein the obstacle recognition distance and the obstacle recognition category are determined based on at least two echo features carried by the detection data sample;

[0020] The adjustment unit adjusts the parameters of the preset network according to the obstacle recognition category and obstacle labeling category, obstacle labeling distance and obstacle recognition distance of each frame of detection data sample to obtain an obstacle recognition model.

[0021] According to a fifth aspect of the present disclosure, there is provided an electronic device, including:

[0022] at least one processor; and

[0023] a memory communicatively connected to the at least one processor; wherein,

[0024] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to execute the method described in the first aspect, or execute the method described in the second aspect.

[0025] According to a sixth aspect of the present disclosure, there is provided a vehicle, comprising: an on-board terminal;

[0026] The vehicle-mounted terminal is the obstacle recognition device provided in the third aspect above, and / or the obstacle recognition model training device provided in the fourth aspect above.

[0027] According to a seventh aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the first aspect, or execute the method described in the second aspect.

[0028] According to an eighth aspect of the present disclosure, a computer program product is provided, comprising: a computer program, wherein the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program so that the electronic device executes the method described in the first aspect, or executes the method described in the second aspect.

[0029] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0031] Figure 1 is a schematic diagram of an application scenario provided by an embodiment of the present disclosure;

[0032] Figure 2 is a flowchart of the obstacle identification method provided by the first embodiment of the present disclosure;

[0033] Figure 3 is a flowchart of an obstacle identification method provided by the second embodiment of the present disclosure;

[0034] Figure 4 1 is a flow chart of the obstacle recognition model training method provided by the first embodiment of the present disclosure;

[0035] Figure 5 2 is a flow chart of the obstacle recognition model training method provided by the second embodiment of the present disclosure;

[0036] Figure 6 is a structural diagram of an obstacle recognition device provided by an embodiment of the present disclosure;

[0037] Figure 7 Schematic diagram of the structure of an obstacle recognition model training device provided by an embodiment of the present disclosure;

[0038] Figure 8 is a schematic block diagram of an example electronic device for implementing an embodiment of the present disclosure. DETAILED DESCRIPTION

[0039] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0040] Before introducing the application scenarios and technical solutions of the present disclosure, some terms involved in the embodiments of the present disclosure are first introduced:

[0041] Single frame: refers to a single detection of the environment by a sensor (including ultrasonic sensors, cameras, etc.), such as a single ultrasonic emission or a single shot by a camera.

[0042] Single-frame detection raw data refers to the detection results directly obtained based on physical detection methods. For example, after an ultrasonic sensor transmits an ultrasonic wave once, the intensity, width, and time interval between the two echoes generated by the ultrasonic wave hitting an obstacle (i.e., the beam flight time) are all generated. For a camera, the image information is obtained by a single shutter exposure.

[0043] Single-frame detection result: The result of processing the raw data of a single detection by the sensor's internal processor. For example, an ultrasonic sensor uses the obstacle distance detected by the first echo with an intensity greater than the detection threshold as the detection distance. The camera can process this image to identify the type of obstacle in the image.

[0044] Multi-frame data association and fusion: Because single-frame detection results often cannot directly determine the specific location and size of obstacles, geometric methods are used to associate and fuse multi-frame detection results to determine the specific location and size of obstacles. Note: Multi-frame data is primarily used to determine the location and size of obstacles and cannot directly and simply determine the obstacle category. Obstacle category information is primarily derived from single-frame detection results.

[0045] In practical applications, autonomous driving is a technology that uses computer-controlled devices to enable vehicles to drive autonomously on the road. Autonomous valet parking (AVP) refers to the process of automatically parking a vehicle without human control. Both autonomous driving and AVP rely on the collaborative efforts of artificial intelligence, visual computing, radar, and positioning components. The following describes this solution using the autonomous parking application scenario.

[0046] For example, during parking, one of the key capabilities for autonomous valet parking is the ability for the vehicle to use its sensors to autonomously perceive its surroundings and determine the areas within which it can pass. The more accurate the vehicle's perception of the surrounding environment, the more detailed the description of the areas within which it can pass, and the more helpful it is in completing the task autonomously. Therefore, continuously improving environmental perception capabilities is a constant technical challenge in autonomous vehicle design.

[0047] Optionally, current parking products may include parking products based on ultrasonic sensors and parking products based on camera images.

[0048] In parking products based on ultrasonic sensors, the sensor typically uses the distance to an obstacle detected by the first echo whose intensity exceeds the detection threshold as the sensor's detection result. This is then combined with multiple frames to determine the obstacle's specific location and size. In other words, existing ultrasonic sensors distinguish the presence of obstacles based on a manually set detection threshold. Because this threshold is manually set, it cannot fully and accurately describe the presence of an obstacle.

[0049] In one possible design, the detection threshold division scheme artificially filters out the observation information of some obstacles, and the echo energy of some low obstacles is no different from that of normal obstacles. This may lead to low recognition accuracy of low obstacles such as door sills, speed bumps, and curbs.

[0050] In another possible implementation, some manufacturers can avoid detecting raised obstacles on the ground by adjusting the ultrasonic sensor's detection direction (FOV), but this approach may result in missed detection of nearby inaccessible obstacles.

[0051] In another possible implementation, ultrasonic sensors cannot clearly distinguish the extent of the traversable area. This makes it difficult to park close to the curb in automated parking scenarios, and can lead to miscontrol when negotiating low obstacles such as speed bumps. For example, in manual driving, this can result in false alarms, while in automated driving, it can lead to false braking. In other words, existing ultrasonic sensor obstacle detection solutions have a high incidence of false detections and are unable to determine the type of obstacle.

[0052] Parking products based on camera images often use machine learning methods to extract the category and position of obstacles from images, and further perform multi-frame fusion to determine the specific position, size and category of obstacles in space. This can make up for the problem that ultrasonic sensors cannot accurately determine the category of obstacles. However, the image information collected by the camera is limited by the influence of light. The detection accuracy varies greatly under different lighting conditions (such as no light, low light, backlight, strong light), and it may even fail to work in some scenarios (occlusion, no light), which greatly limits the vehicle's autonomous driving.

[0053] To address the above technical issues, the technical concept of the embodiments of the present disclosure is as follows: The inventors studied existing parking products using ultrasonic sensors and discovered that, because these products only use the detection distance represented by the first echo greater than a detection threshold, they fail to fully utilize the ultrasonic sensor's detection information. Furthermore, the threshold-based partitioning scheme used on the raw data artificially filters out some obstacle observation information, resulting in low obstacle recognition accuracy or false obstacle detection. If all detection information from the ultrasonic sensor can be fully utilized, such as the number of single-frame echoes carried in a single-frame detection data, the intensity values ​​and pairwise intensity ratios of multiple echoes in a single frame, the width values ​​and pairwise bandwidth ratios of multiple echoes in a single frame, the distance ratios of multiple echoes in a single frame, and the distance difference between multiple echoes in a single frame, obstacle detection results (e.g., obstacle distance and obstacle category) from the single-frame detection data can be determined. Furthermore, by combining the obstacle detection results from multiple frames of detection data, the actual obstacle category at at least one target location within the ultrasonic sensor's detection field of view can be determined.

[0054] Based on the above technical concept, an embodiment of the present disclosure provides an obstacle identification method, comprising: acquiring at least two frames of detection data from an ultrasonic sensor, each frame of detection data carrying at least two echo features; determining the obstacle distance and obstacle category detected by each frame of detection data based on the at least two echo features; and determining the actual obstacle category at at least one target location based on the obstacle distance and obstacle category detected by each frame of detection data in the at least two frames of detection data. The at least one target location is determined based on the obstacle distance detected by the at least two frames of detection data. This technical solution can accurately determine the detected obstacle category, improving the accuracy of obstacle category detection.

[0055] The disclosed embodiments also provide a method for training an obstacle recognition model, including obtaining a set of detection data samples from an ultrasonic sensor, wherein each frame of the detection data sample in the set carries an obstacle label category, an obstacle label distance, and at least two echo features; inputting each frame of the detection data sample in the set into a preset network to obtain an obstacle recognition distance and obstacle recognition category for each frame of the detection data sample, wherein the obstacle recognition distance and obstacle recognition category are determined based on the at least two echo features carried by the detection data sample; and adjusting parameters of the preset network based on the obstacle recognition category, obstacle label category, obstacle label distance, and obstacle recognition distance for each frame of the detection data sample to obtain an obstacle recognition model. The obstacle recognition model obtained by this technical solution, when applied in the obstacle recognition process, can improve obstacle recognition accuracy and avoid false or missed obstacle detections.

[0056] The present disclosure provides an obstacle recognition and model training method, apparatus, device, and storage medium, which are applied to the fields of intelligent transportation, autonomous driving, autonomous parking, the Internet of Things, and deep learning technology in the field of artificial intelligence to improve the recognition accuracy of obstacle categories.

[0057] It is understood that in the embodiments of the present disclosure, the "obstacle recognition model," also referred to as the "model," can receive at least two frames of detection data from the ultrasonic sensor and determine the obstacle distance and obstacle category detected in each frame of detection data based on the at least two frames of detection data received and the current model parameters. The obstacle recognition model can be a regression model, an artificial neural network (ANN), a deep neural network (DNN), a support vector machine (SVM), or other machine learning model. The embodiments of the present disclosure are not limited thereto.

[0058] It should be noted that the obstacle recognition model in this embodiment is not a recognition model for a specific obstacle and cannot reflect information about a specific obstacle. It should be noted that the at least two frames of detection data from the ultrasonic sensor and the ultrasonic sensor detection data sample set in this embodiment are all from public datasets.

[0059] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0060] For example, Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of the present disclosure. Figure 1 As shown, this application scenario may include: two stages; wherein:

[0061] The first stage is the training stage of the obstacle recognition model.

[0062] During the training phase of the obstacle recognition model, the obstacle recognition model is used to identify the distance and category of obstacles detected by each frame of detection data from the ultrasonic sensor. In the application scenario of the present disclosure, the training device trains a preset network based on each frame of detection data sample from the ultrasonic sensor to obtain the obstacle recognition model.

[0063] It can be understood that the obstacle detection results of the single-frame detection data in the embodiment of the present disclosure may include obstacle-free areas and obstacle areas, wherein the obstacle-free areas may include the ground, slopes, ground stones, bumps, and other areas that do not affect the vehicle's passability. At this time, it can be assumed that no obstacles are detected; the obstacle areas include: free-travel obstacle areas (for example, green belts, speed bumps), intravelable obstacle areas (for example, walls, columns, parking poles, building support columns, etc.), and conditionally passable obstacle areas (for example, curbs).

[0064] For example, in the embodiments of the present disclosure, see Figure 1 The training device obtains a set of detection data from N databases. The detection data set may be a set of detection data generated by ultrasonic sensors when a vehicle equipped with ultrasonic sensors is traveling in a specific scenario. Optionally, after each frame of detection data in the detection data set is annotated by the user with an obstacle category or obstacle distance, a detection data sample set can be obtained. Subsequently, a preset network is used to perform obstacle recognition on each frame of detection data in the detection data sample set to obtain an obstacle recognition distance and obstacle recognition category for each frame of detection data sample. Finally, based on the obstacle recognition category and obstacle annotation category, obstacle annotation distance, and obstacle recognition distance of each frame of detection data sample, the parameters of the preset network are adjusted to obtain an obstacle recognition model.

[0065] The second stage is the obstacle recognition stage.

[0066] For example, the obstacle recognition model can be used to identify obstacles based on the detection data of the ultrasonic sensor. Figure 1 The obstacle recognition model trained in the first phase can be loaded into the recognition device. The recognition device uses the obstacle recognition model to identify obstacles. Optionally, the recognition device can also be called an intelligent device.

[0067] It is understandable that in an autonomous driving or automatic parking scenario, the identification device can be a vehicle-mounted terminal or an identification component in the vehicle-mounted terminal.

[0068] Exemplarily, by using the obstacle model loaded in the recognition device to perform obstacle recognition on at least two frames of detection data from the ultrasonic sensor, the obstacle distance and obstacle category detected by each frame of detection data can be obtained. Then, by combining the obstacle distance and obstacle category of each frame of detection data in the at least two frames of detection data, the actual obstacle category at at least one target position can be determined.

[0069] It should be noted that Figure 1 It is only a schematic diagram of an application scenario provided by an embodiment of the present disclosure. The embodiment of the present disclosure does not limit the specific devices included in the application scenario. For example, the application scenario may also include: data acquisition devices such as ultrasonic sensors, storage devices, etc.

[0070] For example, in Figure 1 In the application scenario shown, the ultrasonic sensor can detect obstacles in a specified scene based on the received acquisition instruction, and transmit the obtained detection data to the recognition device for obstacle recognition.

[0071] Optionally, in this embodiment, the storage device may be used to store detection data of the ultrasonic sensor, may be used to store model parameters of the obstacle recognition model trained by the training device, and may be used to store recognition results of the recognition device.

[0072] Understandable, Figure 1 The positional relationship between the devices shown in the figure does not constitute any limitation. For example, when the application scenario also includes a storage device, the storage device can be an independent device or integrated into the processing platform. For example, the storage device can be an external memory relative to the training device or the recognition device. In other cases, the storage device can also be placed in the recognition device.

[0073] It should also be noted that the training device and the recognition device in the embodiments of the present disclosure can be the same device or different devices. The training device and / or the recognition device can be a terminal device, which includes but is not limited to: a smart phone, a laptop computer, a desktop computer, a platform computer, a vehicle-mounted device, a smart wearable device, etc., or a server or a virtual machine, etc., or a distributed computer system composed of one or more servers and / or computers, etc., which is not limited in the embodiments of the present disclosure. Among them, the server can be an ordinary server or a cloud server, which is also called a cloud computing server or cloud host, and is a host product in the cloud computing service system. The server can also be a server of a distributed system, or a server combined with a blockchain.

[0074] It is worth noting that the product implementation form of the present disclosure is a program code contained in machine learning and deep learning platform software and deployed on a server (which can also be hardware with computing capabilities such as a computing cloud or mobile terminal). Figure 1 In the system structure diagram shown, the program code of the present disclosure can be stored in the recognition device and the training device. During operation, the program code runs in the host memory and / or GPU memory of the server.

[0075] In the embodiments of the present disclosure, "plurality" refers to two or more. "And / or" describes the relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0076] The technical solution of the present disclosure is described in detail below through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0077] For example, the obstacle recognition process is first described in detail below with reference to several specific embodiments.

[0078] Figure 2 This is a flow chart of the obstacle identification method provided by the first embodiment of the present disclosure. The method of this embodiment can be Figure 1 The method is executed by the recognition device in the embodiment, or by the processor in the recognition device. Figure 2 As shown, the obstacle recognition method provided in this embodiment may include:

[0079] S201. Acquire at least two frames of detection data from an ultrasonic sensor, where each frame of detection data carries at least two echo features.

[0080] In the embodiments of the present disclosure, the scenario of autonomous driving or automatic parking is used for explanation. In this case, the recognition device can be an onboard terminal of a vehicle, and at least one ultrasonic sensor is deployed on the vehicle.

[0081] During the movement of the vehicle, the ultrasonic sensor deployed on the vehicle can continuously or periodically emit ultrasonic waves and receive echo data to obtain at least two frames of detection data. Correspondingly, the recognition device interacts with the ultrasonic sensor to obtain at least two frames of detection data from the ultrasonic sensor.

[0082] It's understandable that each frame of ultrasonic sensor detection data is the result of one transmission and one reception process within the detection environment. Because ultrasonic sensors operate in complex detection environments, each frame of ultrasonic sensor detection data carries at least two echo signatures, representing the sensor's internal information acquired after each detection.

[0083] Optionally, since a single obstacle may generate many echoes and there may be echoes generated by multiple obstacles around the obstacle, the at least two echo features carried by each frame of detection data may include but are not limited to the following: the number of echoes in a single frame, the intensity value of multiple echoes in a single frame and the intensity ratio of each echo, the width value of multiple echoes in a single frame and the broadband ratio of each echo, the distance ratio of multiple echoes in a single frame, the distance difference between multiple echoes in a single frame, and other echo features.

[0084] The number of single-frame echoes means that the ultrasonic sensor will receive multiple echoes after transmitting ultrasonic waves once. It can be understood that the multiple echoes may be emitted by the same obstacle or different obstacles, and this embodiment does not limit this.

[0085] Similarly, the intensity value of multiple echoes in a single frame and the intensity ratio of each pair of echoes refer to the fact that the multiple echoes in each frame of detection data have different intensity values, as well as the ratio of the intensity values ​​of each pair of echoes; the width value of multiple echoes in a single frame and the broadband ratio of each pair of echoes refer to the fact that the multiple echoes in each frame of detection data have different width values, as well as the ratio of the width values ​​of each pair of echoes; the distance ratio of multiple echoes in a single frame refers to the fact that the multiple echoes in each frame of detection data carry different obstacle distances, and the ratio between different obstacle distances is taken. The distance difference of multiple echoes in a single frame echo is the difference between different obstacle distances.

[0086] S202: Determine the obstacle distance and obstacle category detected by each frame of detection data according to at least two echo features of each frame of detection data.

[0087] In this embodiment, the distance of the detected obstacle can be calculated based on the time difference between each emission of ultrasonic waves and the reception of echoes by the ultrasonic sensor, the propagation speed of the sound waves, etc.

[0088] Furthermore, using the at least two echo features in each frame of detection data, the nature of obstacles, such as whether they are continuous, coarse or fine, or high or low, can be identified, thereby determining the obstacle category corresponding to the strongest echo among multiple echoes. In other words, in this embodiment, based on the at least two echo features in each frame of detection data, the distance and category of obstacles detected in each frame of detection data can be determined.

[0089] Optionally, for each frame of detection data, the geometric shape of the obstacle can be determined by combining the echo width and echo time of a single-frame multiple echo, and the accurate obstacle category can be determined by combining the intensity and ratio of the single-frame multiple echo, the distance ratio of the single-frame multiple echo, the distance difference, etc.

[0090] Optionally, in this embodiment, the obstacle category may be a freely passable obstacle (eg, a green belt, a speed bump), an inaccessible obstacle (a wall, a pillar), a conditionally passable obstacle (eg, a curb), etc.

[0091] In a possible implementation of this embodiment, the recognition device may use a pre-trained obstacle recognition model to process at least two echo features of each frame of detection data to determine the obstacle distance and obstacle category detected by each frame of detection data.

[0092] The obstacle recognition model is obtained by training a preset network based on multi-frame detection data samples carrying obstacle labeling categories and at least two echo features.

[0093] For example, pre-trained and stored obstacle recognition model parameters can be loaded or deployed onto the recognition device. In actual applications, after the recognition device acquires at least two frames of detection data from the ultrasonic sensor, each frame of detection data can be input into the obstacle recognition model to obtain the obstacle distance and obstacle category detected by each frame of detection data.

[0094] S203: Determine an actual obstacle category at at least one target position according to the obstacle distance and obstacle category detected in each frame of detection data in at least two frames of detection data.

[0095] Wherein, at least one target position is determined based on the distance of an obstacle detected by at least two frames of detection data.

[0096] In practical applications, since each frame of detection data contains the distance to an obstacle, the obstacle detection results from multiple frames can be combined to determine at least one target location. Furthermore, combined with the obstacle category detected in each frame, the actual obstacle category at each target location can be determined. In other words, in this step, by correlating and updating the obstacle detection results from multiple frames of ultrasonic sensors, the specific location, size, and category of each obstacle in space can be obtained.

[0097] In the embodiment of the present disclosure, at least two frames of detection data from an ultrasonic sensor are acquired, each frame of detection data carries at least two echo features, and then the obstacle distance and obstacle category detected by each frame of detection data are determined based on the at least two echo features of each frame of detection data. Finally, the obstacle distance and obstacle category detected by each frame of detection data in the at least two frames of detection data are fused to determine the actual obstacle category at at least one target position, where the at least one target position is determined based on the obstacle distance detected by the at least two frames of detection data. In this technical solution, the ultrasonic sensor can take into account obstacles that are near, far, large, and small, can reduce missed detection and false detection of obstacles, is no longer sensitive to the setting of the detection threshold, improves the identification of passable obstacles, avoids false reminders to the driver (in the case of manual driving) and false braking (in the case of automatic driving), and improves the accuracy of obstacle detection results.

[0098] In addition, since this solution adds obstacle category recognition for obstacles such as curbs that have special significance for parking, the success rate of autonomous parking close to curbs and other parking spaces is improved.

[0099] exist Figure 2 Based on the illustrated embodiment, the obstacle recognition method provided by the embodiment of the present disclosure is introduced in more detail below.

[0100] For example, Figure 3 FIG. 1 is a flow chart of the obstacle identification method provided by the second embodiment of the present disclosure. Figure 3 As shown, in the embodiment of the present disclosure, the above S203 can be implemented by the following steps:

[0101] S301: Determine at least one target position and at least two frames of target detection data corresponding to each target position based on an obstacle distance detected by each frame of detection data in at least two frames of detection data.

[0102] Each target position is determined by the same at least two obstacle distances.

[0103] For example, in this embodiment, the locations where obstacles actually exist may be determined first, and then the category of the obstacle at each location may be determined.

[0104] In practical applications, based on the principle of determining at least one point by two lines, in this embodiment, based on the obstacle distance detected by each frame of detection data in at least two frames of detection data, a target position can be determined by the intersection of at least two obstacle distances, that is, the same at least two obstacle distances can determine a target position where an obstacle exists. Therefore, at least one target position can be determined based on at least two obstacle distances, as well as at least two frames of target detection data for determining the target position.

[0105] S302 : For each target position, determine the actual obstacle category at the target position based on obstacle categories detected at the target position by at least two frames of target detection data.

[0106] For example, after determining each target position and at least two frames of target detection data corresponding to each target position, the category of the actual obstacle at each target position may be identified.

[0107] Optionally, in one possible implementation, for each target position, based on the obstacle categories detected at the target position by at least two frames of target detection data, determining the actual obstacle category at the target position can be achieved by the following steps:

[0108] A1. Determine the probability of detecting the first obstacle category at the target position in each frame of target detection data.

[0109] Optionally, for at least two frames of target detection data corresponding to a certain target position, assuming that the obstacle category at the target position is the first obstacle category, the probability of each frame of target detection data detecting the first obstacle category at the target position can be first calculated.

[0110] For example, assuming that a frame of target detection data detects an obstacle category of the first obstacle category at the target position, it can be assumed that the probability of detecting the first obstacle category at the target position is 100%, and then combined with other target detection data to detect the obstacle category at the target position to update the probability of detecting the first obstacle category at the target position.

[0111] A2. Using the Bayesian probability update method, the probability of detecting the first obstacle category at the target position in each frame of target detection data is updated to determine the probability that the first obstacle category exists at the target position.

[0112] For example, the probability of detecting the first obstacle category at the target position in each frame of target detection data can be calculated based on a Bayesian statistical model and updated with each obstacle detection based on a Bayesian probability update technique.

[0113] In one possible implementation of this embodiment, the probability distribution map of each obstacle category can be determined based on the obstacle semantic segmentation type, or the probability distribution map of each obstacle category can be represented in the form of a probability grid map. This embodiment does not limit the form of the probability of the first obstacle category existing at each target position.

[0114] A3. In response to a probability that the first obstacle category exists at the target position being greater than a detection threshold, determining that the actual obstacle category at the target position is the first obstacle category.

[0115] For example, after determining the probability that a first obstacle category exists at the target location, it can be first determined whether the probability that the first obstacle category exists at the target location is greater than or equal to a detection threshold. If so, it is determined that the actual obstacle category at the target location is the first obstacle category. Otherwise, it is determined that the actual obstacle category at the target location is not the first obstacle category.

[0116] In this embodiment, by continuously collecting obstacle detection results from multiple frames of detection data, the obstacle categories can be updated with Bayesian probabilities based on the probability grid map, thereby identifying the probabilities of various types of obstacles at different spatial locations. By setting a detection threshold for the probability of each obstacle category, it is determined whether an obstacle exists based on the probability of each obstacle category and the detection threshold.

[0117] The obstacle recognition method provided by the embodiments of the present disclosure determines at least one target position and at least two frames of target detection data corresponding to each target position based on the obstacle distance detected by each frame of at least two frames of detection data. Then, for each target position, the actual obstacle category at the target position is determined based on the obstacle category detected at the target position by the at least two frames of target detection data. This technical solution can accurately determine the actual obstacle category within the detection range of the ultrasonic sensor, thereby improving the obstacle recognition accuracy.

[0118] Furthermore, in an embodiment of the present disclosure, the obstacle identification method may further include:

[0119] The traffic condition of each target location is determined based on the actual obstacle category at each target location.

[0120] Exemplarily, for each target position, in one example, if the actual obstacle category at the target position is a freely passable obstacle (e.g., a green belt, a speed bump), then the target position is determined to be passable; in another example, if the actual obstacle category at the target position is an inaccessible obstacle (e.g., a wall, a pillar), then the target position is determined to be inaccessible; in yet another example, if the actual obstacle category at the target position is a conditionally passable obstacle (e.g., a curb), then it is necessary to determine the positional relationship between the vehicle equipped with the ultrasonic sensor and the conditionally passable obstacle. If the positional relationship meets the passable condition, then the target position is determined to be passable; if the positional relationship does not meet the passable condition, then the target position is determined to be inaccessible.

[0121] Optionally, in this embodiment, the obstacle identification method may further include the following steps:

[0122] Determine the travel plan based on the actual obstacle category at each target location;

[0123] or

[0124] Generate prompt information for each target location based on the actual obstacle category at each target location, wherein the prompt information is text information or voice information;

[0125] Output the prompt information, or send the prompt information to a preset device.

[0126] In one example of the present disclosure, in an automatic driving or automatic parking scenario, the on-board terminal can adjust or determine the travel plan according to the actual obstacle category at each target position. For example, when the actual obstacle category at the target position is a freely passable obstacle, the on-board terminal can control the vehicle to continue moving; when the actual obstacle category at the target position is an impassable obstacle, the on-board terminal adjusts the vehicle's travel strategy; when the actual obstacle category at the target position is a conditionally passable obstacle, the on-board terminal can adjust the vehicle direction and then execute the obstacle recognition scheme in the above embodiment, and then determine whether the target position is passable.

[0127] In another example of the present disclosure, in an autonomous driving or automatic parking scenario, when the recognition device is a vehicle-mounted terminal, prompt information can be generated in response to the actual obstacle category at each target location to indicate whether the target location is accessible, and the prompt information can be directly displayed or played, for example, displaying text information or playing voice information.

[0128] Optionally, after the recognition device generates the prompt information, it can also push it to a preset device, such as a user terminal, a speaker, and other devices, so as to issue a prompt.

[0129] In an embodiment of the present disclosure, prompt information for each target location is generated based on the actual obstacle category at each target location, and the prompt information is output, or a scheme for sending the prompt information to a preset device can issue prompts in a timely manner so that the driver or passengers can promptly know the obstacle category at the target location and make timely decisions.

[0130] The above embodiments describe the obstacle recognition process. The following describes the training process of the obstacle recognition model used in the obstacle recognition process in conjunction with several specific embodiments.

[0131] Figure 4 This is a flow chart of the obstacle recognition model training method provided by the first embodiment of the present disclosure. The method of this embodiment can be Figure 1 The training device in the embodiment can also execute the method. Figure 4 As shown, the obstacle recognition model training method provided in this embodiment may include:

[0132] S401: Acquire a detection data sample set, where each frame of detection data sample in the detection data sample set carries an obstacle labeling category, an obstacle labeling distance, and at least two echo features.

[0133] For example, the training device may obtain a large amount of detection data of the ultrasonic sensor from multiple databases.

[0134] In a possible design of this embodiment, each frame of detection data acquired by the training device is labeled detection data. Optionally, each labeled frame of detection data carries an obstacle labeling category, an obstacle labeling distance, and at least two echo features.

[0135] The obstacle labeling category refers to the obstacle category detected by the frame detection data, such as passable obstacle, impassable obstacle, conditionally passable obstacle, etc. The obstacle labeling distance refers to the distance information between the obstacle detected by the frame detection data and the ultrasonic sensor, etc. At least two echo features are the above Figures 2 to 3 Internal detection information of the ultrasonic sensor that performs obstacle detection in the illustrated embodiment, etc.

[0136] Exemplarily, the steps for obtaining a detection data sample set are as follows:

[0137] B1. Obtain a data collection request, where the data collection request is used to indicate a target collection scenario for detection data;

[0138] B2. Based on the data collection request, controlling a vehicle equipped with at least one ultrasonic sensor to travel to a target collection scene;

[0139] B3. Control the at least one ultrasonic sensor to detect obstacles in the target acquisition scene to obtain a detection data sample set.

[0140] It can be understood that the target collection scene can be a scene with various low obstacles that do not affect passage, a scene with various obstacles such as walls or pillars that affect passage, and a scene with conditional obstacles such as curbs. This embodiment does not limit the specific obstacle categories and specific objects in the target collection scene.

[0141] Exemplarily, one way to obtain a detection data sample set is to arrange an ultrasonic sensor on a vehicle body, adjust the detection intensity of the ultrasonic sensor, control the vehicle equipped with the ultrasonic sensor to travel in the above-mentioned target specified scene, obtain multiple frames of detection data of the ultrasonic sensor in the target specific scene and the distance information between the ultrasonic sensor and the obstacle during detection, and correspond each frame of detection data to the obstacle distance, obstacle category and scene label one by one to obtain a detection data sample set.

[0142] S402: Input each frame of detection data sample in the detection data sample set into a preset network to obtain an obstacle recognition distance and an obstacle recognition category of each frame of detection data sample.

[0143] The obstacle identification distance and obstacle identification category are determined based on at least two echo features carried by the detection data sample.

[0144] In this embodiment, during the training process of the obstacle recognition model, the training device can input each frame of detection data sample in the detection data sample set into the preset network respectively, and output the obstacle detection result of each frame of detection data sample, that is, the obstacle recognition distance and obstacle recognition category of each frame of detection data sample.

[0145] It can be understood that in this embodiment, the preset network can analyze at least two echo features carried by each frame of detection data sample and output the identified obstacle distance and obstacle category.

[0146] S403 , adjusting parameters of a preset network according to the obstacle recognition category and obstacle labeling category, obstacle labeling distance, and obstacle recognition distance of each frame of detection data sample to obtain an obstacle recognition model.

[0147] In this embodiment, the training device can compare the obstacle detection results of each frame of detection data sample with the obstacle labeling information of each frame of detection data sample to determine the consistency of the preset network's obstacle recognition for each frame of detection data sample, for example, the consistency of the obstacle category and the consistency of the obstacle distance. Then, when the consistency level (the consistency level of the obstacle category or the consistency level of the obstacle distance) is lower than the preset requirement, the parameters of the preset network are adjusted to obtain an obstacle recognition model.

[0148] Optionally, after the training device obtains the obstacle recognition model, the parameters of the obstacle recognition model can be stored in a designated location or sent to other devices, for example, Figure 1 Identify devices in the scene graph shown for subsequent use.

[0149] In an embodiment of the present disclosure, a detection data sample set is obtained, each frame of which carries an obstacle labeling category, an obstacle labeling distance, and at least two echo features. Each frame of detection data sample in the detection data sample set is input into a preset network to obtain the obstacle identification distance and obstacle identification category for each frame of detection data sample. Based on the obstacle identification category, obstacle labeling category, obstacle labeling distance, and obstacle identification distance for each frame of detection data sample, the parameters of the preset network are adjusted to obtain an obstacle identification model. In this technical solution, the obstacle category and obstacle distance are determined using the at least two echo features carried by each frame of detection data, which improves the obstacle identification model's recognition response to obstacle categories, thereby increasing the obstacle identification accuracy in the obstacle detection results.

[0150] exist Figure 4 Based on the embodiment shown, Figure 5 FIG. 1 is a flow chart of the obstacle recognition model training method provided by the second embodiment of the present disclosure. Figure 5 As shown, in the embodiment of the present disclosure, the above S403 can be implemented by the following steps:

[0151] S501: Determine the obstacle distance recognition accuracy of the preset network according to the obstacle marking distance and obstacle recognition distance of each frame of detection data sample.

[0152] Exemplarily, in an embodiment of the present disclosure, when determining the obstacle recognition distance and obstacle recognition category of each frame of detection data sample in the detection data sample set, the obstacle recognition distance of each frame of detection data sample can be first compared with the obstacle annotation distance of the frame of detection data sample to determine the number of detection data frames in the detection data sample set with correct obstacle distance recognition, and then the ratio of the number of detection data frames with correct obstacle distance recognition to the total number of detection data frames in the detection data sample set is calculated, thereby determining the obstacle distance recognition accuracy of the preset network.

[0153] S502: Determine whether the obstacle distance recognition accuracy is greater than or equal to the distance accuracy threshold; if so, execute S503; if not, execute S505 first, and then execute S402.

[0154] Optionally, a distance accuracy threshold is preset in the training device. Therefore, after determining the obstacle distance recognition accuracy of the preset network, it can be compared with the distance accuracy threshold, and then subsequent operations are determined based on the comparison result.

[0155] For example, when the obstacle distance recognition accuracy is greater than or equal to the distance accuracy threshold, S503 can be executed to determine whether the obstacle category recognition accuracy of the preset network meets the requirements; when the obstacle distance recognition accuracy is less than the distance accuracy threshold, S505 is executed to adjust the parameters of the preset network, and then S402 is executed to use the preset network to perform obstacle recognition during the training process to train the obstacle recognition model.

[0156] S503: Determine the obstacle category recognition accuracy of the preset network based on the obstacle recognition category and the obstacle labeling category of each frame of detection data sample;

[0157] As an example, in response to the obstacle distance recognition accuracy being greater than or equal to the distance accuracy threshold, it indicates that a distance recognition indicator of the preset network has met the requirement. At this time, the obstacle category recognition accuracy of the preset network is calculated.

[0158] For example, the obstacle identification category of each frame of detection data sample can be first compared with the obstacle labeling category of the detection data sample to determine the number of detection data frames in the detection data sample set with correct obstacle category identification, and then the ratio of the number of detection data frames with correct obstacle category identification to the total number of detection data sample frames in the detection data sample set can be calculated to determine the obstacle category identification accuracy of the preset network.

[0159] S504: Determine whether the obstacle category recognition accuracy is greater than or equal to the category accuracy threshold; if so, execute S506; if not, execute S505 first, and then execute S402.

[0160] Optionally, a category accuracy threshold is preset in the training device. Therefore, after determining the obstacle category recognition accuracy of the preset network, it can be compared with the category accuracy threshold, and then subsequent operations can be determined based on the comparison result.

[0161] For example, when the obstacle category recognition accuracy is greater than or equal to the category accuracy threshold, it indicates that the obstacle distance recognition index and obstacle category recognition index of the preset network both meet the requirements, and the updated preset network is the obstacle recognition model, that is, S506; when the obstacle category recognition accuracy is less than the category accuracy threshold, it is necessary to execute S505 to adjust the parameters of the preset network, and then execute S402 to use the preset network for obstacle recognition during the training process to train the obstacle recognition model.

[0162] S505: Adjust parameters of the preset network.

[0163] S506: Obtain an obstacle recognition model.

[0164] Optionally, in response to the obstacle distance recognition accuracy being less than a distance accuracy threshold and / or the obstacle category recognition accuracy being less than a category accuracy threshold, the parameters of the preset network are adjusted until the obstacle distance recognition accuracy is greater than or equal to the distance accuracy threshold and the obstacle category recognition accuracy is greater than or equal to the category accuracy threshold, thereby obtaining an obstacle recognition model.

[0165] It is understandable that the embodiments of the present disclosure do not limit the training process of the obstacle recognition model. For example, the training device may first calculate the obstacle category recognition accuracy of the preset network. When the obstacle category recognition accuracy meets the requirements, the obstacle distance recognition accuracy of the preset network is calculated, and it is determined whether the obstacle distance recognition accuracy meets the requirements. When both the obstacle category recognition accuracy and the obstacle distance recognition accuracy meet the requirements, the obstacle recognition model is obtained.

[0166] In the disclosed embodiments, by adjusting the parameters of a preset network based on obstacle recognition categories, obstacle labeling categories, obstacle labeling distances, and obstacle recognition distances, the obstacle recognition model can be trained in a supervised manner, laying the foundation for improving the accuracy of obstacle category recognition during use.

[0167] Figure 6 This is a schematic diagram of the structure of an obstacle recognition device provided by an embodiment of the present disclosure. The obstacle recognition device provided by this embodiment can be Figure 1 The identification device in the identification device or the device in the identification device, optionally, the identification device can be a vehicle-mounted terminal. Figure 6 As shown, the obstacle identification device 600 provided in the embodiment of the present disclosure may include:

[0168] An acquisition unit 601 is configured to acquire at least two frames of detection data from an ultrasonic sensor, each frame of detection data carrying at least two echo features;

[0169] an identification unit 602 for determining the distance and type of an obstacle detected in each frame of detection data based on at least two echo features in each frame of detection data;

[0170] A determination unit 603 is configured to determine an actual obstacle category at at least one target position based on the obstacle distance and obstacle category detected by each frame of detection data in the at least two frames of detection data, where the at least one target position is determined based on the obstacle distance detected by the at least two frames of detection data.

[0171] In one possible implementation of this embodiment, the identification unit 602 is specifically used to use a pre-trained obstacle recognition model to process at least two echo features of each frame of detection data to determine the obstacle distance and obstacle category detected by each frame of detection data. The obstacle recognition model is obtained by training a preset network based on multiple frames of detection data samples carrying obstacle labeled categories and at least two echo features.

[0172] In a possible implementation of this embodiment, the determining unit 603 includes:

[0173] a distance determination module, configured to determine at least one target position and at least two frames of target detection data corresponding to each target position based on the obstacle distance detected by each frame of detection data in the at least two frames of detection data, wherein each target position is determined by the same at least two obstacle distances;

[0174] The category determination module is configured to determine, for each target position, the actual obstacle category at the target position based on the obstacle categories detected at the target position by the at least two frames of target detection data.

[0175] Optionally, the category determination module includes:

[0176] A first determination submodule is configured to determine a probability that each frame of target detection data detects a first obstacle category at the target position;

[0177] a second determination submodule, configured to update the probability of detecting a first obstacle category at the target position in each frame of target detection data using a Bayesian probability update method, and determine the probability of the first obstacle category existing at the target position;

[0178] The third determining submodule is configured to determine that the actual obstacle category at the target position is the first obstacle category in response to a probability that the first obstacle category exists at the target position being greater than a detection threshold.

[0179] In a possible implementation of this embodiment, the determining unit 603 is further configured to determine the traffic condition of each target location according to the actual obstacle category at each target location.

[0180] In a possible implementation of this embodiment, the determining unit 603 is further configured to determine a travel plan according to an actual obstacle category at each target position;

[0181] or

[0182] The device further comprises:

[0183] a generating unit (not shown), configured to generate prompt information for each target location according to the actual obstacle category at each target location, wherein the prompt information is text information or voice information;

[0184] The output unit (not shown) is used to output the prompt information or send the prompt information to a preset device.

[0185] The obstacle recognition device provided in this embodiment can be used to execute the obstacle recognition method performed by the recognition device in any of the above method embodiments. Its implementation principles and technical effects are similar and will not be elaborated here.

[0186] Figure 7 This is a schematic diagram of the structure of an obstacle recognition model training device provided by an embodiment of the present disclosure. The obstacle recognition model training device provided by this embodiment can be Figure 1 A training device or a device in a training device. Figure 7 As shown, the obstacle recognition model training device 700 provided in the embodiment of the present disclosure may include:

[0187] An acquisition unit 701 is configured to acquire a detection data sample set, wherein each frame of detection data sample in the detection data sample set carries an obstacle labeling category, an obstacle labeling distance, and at least two echo features;

[0188] a processing unit 702 configured to input each frame of detection data sample in the detection data sample set into a preset network to obtain an obstacle recognition distance and an obstacle recognition category for each frame of detection data sample, wherein the obstacle recognition distance and the obstacle recognition category are determined based on at least two echo features carried by the detection data sample;

[0189] The adjustment unit 703 adjusts the parameters of the preset network according to the obstacle recognition category and obstacle labeling category, obstacle labeling distance and obstacle recognition distance of each frame of detection data sample to obtain an obstacle recognition model.

[0190] In a possible implementation of this embodiment, the adjusting unit 703 includes:

[0191] A first determination module is used to determine the obstacle distance recognition accuracy of the preset network based on the obstacle annotation distance and the obstacle recognition distance of each frame of detection data sample;

[0192] a second determining module, configured to determine, in response to the obstacle distance recognition accuracy being greater than or equal to a distance accuracy threshold, an obstacle category recognition accuracy of the preset network based on the obstacle recognition category and the obstacle labeling category of each frame of detection data sample;

[0193] an adjustment module, configured to, in response to the obstacle distance recognition accuracy being less than a distance accuracy threshold and / or the obstacle category recognition accuracy being less than a category accuracy threshold, adjust parameters of the preset network until the obstacle distance recognition accuracy is greater than or equal to the distance accuracy threshold and the obstacle category recognition accuracy is greater than or equal to the category accuracy threshold, thereby obtaining an obstacle recognition model.

[0194] In a possible implementation of this embodiment, the acquiring unit 701 includes:

[0195] An acquisition module, configured to acquire a data acquisition request, wherein the data acquisition request is used to indicate a target acquisition scenario for detection data;

[0196] A first control module is configured to control a vehicle equipped with at least one ultrasonic sensor to travel to the target collection scene based on the data collection request;

[0197] The second control module is configured to control the at least one ultrasonic sensor to detect obstacles in the target acquisition scene to obtain the detection data sample set.

[0198] The obstacle recognition model training device provided in this embodiment can be used to execute the obstacle recognition model training method performed by the training device in any of the above method embodiments. Its implementation principles and technical effects are similar and will not be elaborated here.

[0199] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0200] Optionally, in one embodiment of the present disclosure, the electronic device is a vehicle-mounted terminal.

[0201] According to an embodiment of the present disclosure, the present disclosure further provides a vehicle, comprising: an on-vehicle terminal;

[0202] The vehicle-mounted terminal is the above Figure 6 The obstacle recognition device shown, and / or the above Figure 7 The obstacle recognition model training device shown.

[0203] According to an embodiment of the present disclosure, the present disclosure also provides a computer program product, which includes: a computer program, the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device executes the solution provided by any of the above embodiments.

[0204] Figure 8 is a schematic block diagram of an example electronic device used to implement an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0205] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. Various programs and data required for the operation of the device 800 can also be stored in the RAM 803. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0206] Various components in device 800 are connected to I / O interface 805, including an input unit 806, such as a keyboard, mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, optical disk, etc.; and a communication unit 809, such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0207] The computing unit 801 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the obstacle identification method and the obstacle model training method. For example, in some embodiments, the obstacle identification method and the obstacle model training method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed onto the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the obstacle identification method and the obstacle model training method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to execute the obstacle recognition method and the obstacle model training method in any other appropriate manner (for example, by means of firmware).

[0208] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0209] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0210] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0211] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0212] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0213] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. This client-server relationship is established by computer programs running on the respective computers, establishing a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host, a host product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosts and VPS services ("Virtual Private Servers" or simply "VPS"). The server may also be a server in a distributed system or a server integrated with blockchain.

[0214] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0215] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for identifying an obstacle, comprising: Acquire at least two frames of detection data from an ultrasonic sensor, each frame of detection data carrying at least two echo features; Based on the pre-trained obstacle recognition model, the obstacle distance and obstacle category detected in each frame of detection data are determined according to at least two echo features in each frame of detection data; Determine the actual obstacle category at at least one target position based on the obstacle distance and obstacle category detected by each frame of detection data in the at least two frames of detection data, where the at least one target position is determined based on the obstacle distance detected by the at least two frames of detection data.

2. The method according to claim 1, wherein The obstacle recognition model based on the pre-training determines the obstacle distance and obstacle category detected in each frame of detection data according to at least two echo features of each frame of detection data, including: A pre-trained obstacle recognition model is used to process at least two echo features in each frame of detection data to determine the distance and category of obstacles detected in each frame of detection data. The obstacle recognition model is obtained by training a preset network based on multiple frames of detection data samples carrying obstacle labeled categories and at least two echo features.

3. The method according to claim 1 or 2, wherein: The determining, based on the obstacle distance and obstacle category detected in each frame of the at least two frames of detection data, an actual obstacle category at at least one target position includes: Determining at least one target position and at least two frames of target detection data corresponding to each target position based on the obstacle distance detected by each frame of detection data in the at least two frames of detection data, wherein each target position is determined by the same at least two obstacle distances; For each target position, the actual obstacle category at the target position is determined according to the obstacle categories detected at the target position by the at least two frames of target detection data.

4. The method according to claim 3, wherein: Determining the actual obstacle category at the target position based on the obstacle categories detected at the target position by the at least two frames of target detection data includes: Determining a probability that a first obstacle category is detected at the target location in each frame of target detection data; Using a Bayesian probability update method, the probability of detecting a first obstacle category at the target position in each frame of target detection data is updated to determine the probability that the first obstacle category exists at the target position; In response to a probability that a first obstacle category exists at the target position being greater than a detection threshold, determining that an actual obstacle category at the target position is the first obstacle category.

5. The method according to any one of claims 1 to 2 and 4, further comprising: The traffic condition of each target location is determined based on the actual obstacle category at each target location.

6. The method according to any one of claims 1 to 2 and 4, further comprising: Determine the travel plan based on the actual obstacle category at each target location; or Generate prompt information for each target location based on the actual obstacle category at each target location, wherein the prompt information is text information or voice information; Output the prompt information, or send the prompt information to a preset device.

7. A method for training an obstacle recognition model, comprising: Acquire a detection data sample set obtained based on an ultrasonic sensor, wherein each frame of detection data sample in the detection data sample set carries an obstacle labeling category, an obstacle labeling distance, and at least two echo features; Inputting each frame of detection data sample in the detection data sample set into a preset network to obtain an obstacle recognition distance and an obstacle recognition category for each frame of detection data sample, wherein the obstacle recognition distance and the obstacle recognition category are determined based on at least two echo features carried by the detection data sample; According to the obstacle recognition category and obstacle labeling category, obstacle labeling distance and obstacle recognition distance of each frame of detection data sample, the parameters of the preset network are adjusted to obtain an obstacle recognition model.

8. The method according to claim 7, wherein: The obstacle recognition model is obtained by adjusting the parameters of the preset network according to the obstacle recognition category and obstacle labeling category, obstacle labeling distance and obstacle recognition distance of each frame of detection data sample, including: Determining the obstacle distance recognition accuracy of the preset network based on the obstacle annotation distance and obstacle recognition distance of each frame of detection data sample; In response to the obstacle distance recognition accuracy being greater than or equal to a distance accuracy threshold, determining the obstacle category recognition accuracy of the preset network according to the obstacle recognition category and the obstacle labeling category of each frame of detection data sample; In response to the obstacle distance recognition accuracy being less than a distance accuracy threshold and / or the obstacle category recognition accuracy being less than a category accuracy threshold, adjusting parameters of the preset network until the obstacle distance recognition accuracy is greater than or equal to the distance accuracy threshold and the obstacle category recognition accuracy is greater than or equal to the category accuracy threshold, thereby obtaining an obstacle recognition model.

9. The method according to claim 7 or 8, wherein obtaining a detection data sample set obtained based on an ultrasonic sensor comprises: Obtaining a data collection request, where the data collection request is used to indicate a target collection scenario for detection data; Based on the data collection request, controlling a vehicle equipped with at least one ultrasonic sensor to travel to the target collection scene; The at least one ultrasonic sensor is controlled to perform obstacle detection in the target acquisition scene to obtain the detection data sample set.

10. An obstacle recognition device, comprising: an acquisition unit, configured to acquire at least two frames of detection data from the ultrasonic sensor, each frame of detection data carrying at least two echo features; an identification unit, configured to determine the distance and category of obstacles detected in each frame of detection data based on a pre-trained obstacle identification model and according to at least two echo features in each frame of detection data; A determination unit is configured to determine an actual obstacle category at at least one target position based on the obstacle distance and obstacle category detected by each frame of detection data in the at least two frames of detection data, wherein the at least one target position is determined based on the obstacle distance detected by the at least two frames of detection data.

11. The device according to claim 10, wherein The recognition unit is specifically configured to process at least two echo features in each frame of detection data using a pre-trained obstacle recognition model to determine the distance and category of obstacles detected in each frame of detection data. The obstacle recognition model is obtained by training a preset network based on samples of multiple frames of detection data that carry obstacle labeled categories and at least two echo features.

12. The device according to claim 10 or 11, wherein The determining unit includes: a distance determination module, configured to determine at least one target position and at least two frames of target detection data corresponding to each target position based on the obstacle distance detected by each frame of detection data in the at least two frames of detection data, wherein each target position is determined by the same at least two obstacle distances; The category determination module is configured to determine, for each target position, the actual obstacle category at the target position based on the obstacle categories detected at the target position by the at least two frames of target detection data.

13. The device according to claim 12, wherein The category determination module includes: A first determination submodule is configured to determine a probability that each frame of target detection data detects a first obstacle category at the target position; a second determination submodule, configured to update the probability of detecting a first obstacle category at the target position in each frame of target detection data using a Bayesian probability update method, and determine the probability of the first obstacle category existing at the target position; The third determining submodule is configured to determine that the actual obstacle category at the target position is the first obstacle category in response to a probability that the first obstacle category exists at the target position being greater than a detection threshold.

14. The device according to any one of claims 10 to 11 and 13, wherein the determining unit is further configured to determine the traffic condition of each target location according to the actual obstacle category at each target location.

15. The device according to any one of claims 10 to 11 and 13, wherein the determining unit is further configured to determine a travel plan according to an actual obstacle category at each target position; or The device further comprises: a generating unit, configured to generate prompt information for each target location according to the actual obstacle category at each target location, wherein the prompt information is text information or voice information; The output unit is used to output the prompt information or send the prompt information to a preset device.

16. An obstacle recognition model training device, comprising: An acquisition unit is configured to acquire a detection data sample set obtained based on an ultrasonic sensor, wherein each frame of detection data sample in the detection data sample set carries an obstacle labeling category, an obstacle labeling distance, and at least two echo features; a processing unit, configured to input each frame of detection data sample in the detection data sample set into a preset network to obtain an obstacle recognition distance and an obstacle recognition category for each frame of detection data sample, wherein the obstacle recognition distance and the obstacle recognition category are determined based on at least two echo features carried by the detection data sample; The adjustment unit adjusts the parameters of the preset network according to the obstacle recognition category and obstacle labeling category, obstacle labeling distance and obstacle recognition distance of each frame of detection data sample to obtain an obstacle recognition model.

17. The device according to claim 16, wherein The adjustment unit includes: A first determination module is used to determine the obstacle distance recognition accuracy of the preset network based on the obstacle annotation distance and the obstacle recognition distance of each frame of detection data sample; a second determining module, configured to determine, in response to the obstacle distance recognition accuracy being greater than or equal to a distance accuracy threshold, an obstacle category recognition accuracy of the preset network based on the obstacle recognition category and the obstacle labeling category of each frame of detection data sample; an adjustment module, configured to, in response to the obstacle distance recognition accuracy being less than a distance accuracy threshold and / or the obstacle category recognition accuracy being less than a category accuracy threshold, adjust parameters of the preset network until the obstacle distance recognition accuracy is greater than or equal to the distance accuracy threshold and the obstacle category recognition accuracy is greater than or equal to the category accuracy threshold, thereby obtaining an obstacle recognition model.

18. The apparatus according to claim 16 or 17, wherein the acquiring unit comprises: An acquisition module, configured to acquire a data acquisition request, wherein the data acquisition request is used to indicate a target acquisition scenario for detection data; A first control module is configured to control a vehicle equipped with at least one ultrasonic sensor to travel to the target collection scene based on the data collection request; The second control module is configured to control the at least one ultrasonic sensor to detect obstacles in the target acquisition scene to obtain the detection data sample set.

19. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 4.

20. The device according to claim 19, wherein the electronic device is a vehicle-mounted terminal.

21. A vehicle comprising: Vehicle-mounted terminal; The vehicle-mounted terminal is the obstacle recognition device described in any one of claims 10 to 15 above, and / or the obstacle recognition model training device described in any one of claims 16 to 18 above.

22. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable the computer to execute the method according to any one of claims 1 to 6, or to execute the method according to any one of claims 7 to 9.

23. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 6, or performs the steps of the method according to any one of claims 7 to 9.

Images