Obstacle classification method and device, storage medium and electronic equipment

Abstract

The application discloses an obstacle classification method and device, a storage medium and an electronic device, and belongs to the technical field of computers. The obstacle classification method comprises the following steps: acquiring current frame point cloud data collected by a laser radar, detecting point cloud data of an obstacle according to the current frame point cloud data, acquiring category information of the target obstacle and tracking information of the target obstacle according to the point cloud data of the target obstacle, acquiring attribute information of other obstacles within a preset distance according to the point cloud data of the other obstacles within the preset distance, inputting the category information of the target obstacle, the tracking information of the target obstacle and the attribute information of the other obstacles within the preset distance into an obstacle classification model to acquire optimized category information of the target obstacle. Therefore, the application can fully utilize various information of the obstacle, and can improve the efficiency and accuracy of obstacle classification.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to an obstacle classification method, apparatus, storage medium, and electronic device. Background Technology

[0002] With the development of intelligent technology, autonomous driving technology has begun to develop and be applied. During autonomous driving, it is necessary to detect the types of obstacles around the vehicle. In existing technologies, after acquiring point cloud data of the obstacles, this data is input into a random forest model to classify and identify the obstacle categories. Existing random forest models have many tree structures, each with multiple nodes and fields, each used to store information related to obstacle classification and identification. However, in existing technologies, when applying random forest models to classify and identify obstacle categories, a large amount of information needs to be stored in the nodes of the random forest model, thus consuming a significant amount of memory and space. Furthermore, due to the need for information storage, the entire obstacle classification and identification process is slow, inefficient, and lacks accuracy. Summary of the Invention

[0003] This application provides an obstacle classification method, apparatus, storage medium, and electronic device, which can improve the efficiency and accuracy of obstacle classification. The technical solution is as follows:

[0004] In a first aspect, embodiments of this application provide an obstacle classification method, including:

[0005] Acquire the current frame point cloud data collected by the lidar;

[0006] Point cloud data of obstacles are detected based on the current frame point cloud data; wherein, the obstacles include the target obstacle and other obstacles within a preset distance;

[0007] Based on the point cloud data of the target obstacle, obtain the category information and tracking information of the target obstacle;

[0008] The attribute information of other obstacles within the preset distance is obtained based on the point cloud data of other obstacles within the preset distance; wherein, the attribute information includes the motion information of other obstacles within the preset distance and scene information;

[0009] The target obstacle's category information, the target obstacle's tracking information, and the attribute information of other obstacles within the preset distance are input into the obstacle classification model to obtain the target obstacle's optimized category information.

[0010] Secondly, embodiments of this application provide a method for training an obstacle classification model, including:

[0011] Acquire at least one frame of point cloud data collected by the lidar;

[0012] Point cloud data of obstacles are detected based on at least one frame of point cloud data; wherein, the obstacles include the target obstacle and other obstacles within a preset distance;

[0013] Based on the point cloud data of the target obstacle, obtain the category information of the target obstacle, the tracking information of the target obstacle, and the scene information of the target obstacle;

[0014] The attribute information and tracking information of other obstacles within the preset distance are obtained based on the point cloud data of other obstacles within the preset distance; wherein, the attribute information includes the motion information and scene information of other obstacles within the preset distance;

[0015] The obstacle classification model is trained using the category information, tracking information, and scene information of the target obstacle, the attribute information and tracking information of other obstacles, and the manually labeled optimized category information as training samples.

[0016] Thirdly, embodiments of this application provide an obstacle classification device, the device comprising:

[0017] The first acquisition module is used to acquire the current frame point cloud data collected by the lidar;

[0018] The detection module is used to detect the point cloud data of obstacles based on the current frame point cloud data; wherein, the obstacles include the target obstacle and other obstacles within a preset distance;

[0019] The second acquisition module is used to acquire the category information of the target obstacle and the tracking information of the target obstacle based on the point cloud data of the target obstacle;

[0020] The third acquisition module is used to acquire attribute information of other obstacles within the preset distance based on point cloud data of other obstacles within the preset distance; wherein, the attribute information includes motion information of other obstacles within the preset distance and scene information;

[0021] The optimization module is used to input the category information of the target obstacle, the tracking information of the target obstacle, and the attribute information of other obstacles within the preset distance into the obstacle classification model to obtain the optimized category information of the target obstacle.

[0022] Fourthly, embodiments of this application provide an obstacle classification model training device, the device comprising:

[0023] The first acquisition module is used to acquire at least one frame of point cloud data collected by the lidar;

[0024] The detection module is used to detect the point cloud data of obstacles based on the at least one frame of point cloud data; wherein the obstacles include the target obstacle and other obstacles within a preset distance;

[0025] The second acquisition module is used to acquire the category information of the target obstacle, the tracking information of the target obstacle, and the scene information of the target obstacle based on the point cloud data of the target obstacle;

[0026] The third acquisition module is used to acquire attribute information and tracking information of other obstacles within the preset distance based on point cloud data of other obstacles within the preset distance; wherein, the attribute information includes motion information and scene information of other obstacles within the preset distance;

[0027] The training module is used to train the obstacle classification model using the category information, tracking information, and scene information of the target obstacle, the attribute information and tracking information of other obstacles, and the manually labeled optimized category label information as training samples.

[0028] Fifthly, embodiments of this application provide a computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the above-described method steps.

[0029] Sixthly, embodiments of this application provide an electronic device, including: a memory and a processor; wherein the memory stores a computer program adapted to be loaded by the processor and executed the above-described method steps.

[0030] The beneficial effects of the technical solutions provided in some embodiments of this application include at least the following:

[0031] When the aforementioned obstacle classification method, device, storage medium, and electronic device are operating, they acquire current frame point cloud data collected by the lidar, detect obstacle point cloud data based on the current frame point cloud data, obtain the target obstacle's category information and tracking information based on the target obstacle's point cloud data, obtain attribute information of other obstacles within a preset distance based on their point cloud data, and input the target obstacle's category information, tracking information, and attribute information of other obstacles within a preset distance into an obstacle classification model to obtain optimized category information for the target obstacle. In this embodiment, by considering that obstacles are not independent in the spatial and temporal domains, the category information, tracking information, and attribute information of other obstacles within the preset distance of the target obstacle can be fully utilized to obtain more accurate category detection results, thereby improving the efficiency and accuracy of obstacle classification. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1A This is a schematic diagram of a lidar system provided in an embodiment of this application;

[0034] Figure 1B This is a schematic diagram of another lidar system provided in an embodiment of this application;

[0035] Figure 1C This is a schematic diagram of an obstacle classification system architecture provided in an embodiment of this application;

[0036] Figure 2 This is a flowchart illustrating an obstacle classification method provided in an embodiment of this application;

[0037] Figure 3 This is another flowchart illustrating an obstacle classification method provided in an embodiment of this application;

[0038] Figure 4 This is a schematic diagram of an obstacle detection result provided in an embodiment of this application;

[0039] Figure 5 This is a schematic diagram of an attitude estimation result provided in an embodiment of this application;

[0040] Figure 6This is a schematic diagram of the structure of an obstacle classification device provided in an embodiment of this application;

[0041] Figure 7 This is a flowchart illustrating an obstacle classification model training method provided in an embodiment of this application;

[0042] Figure 8 This is a schematic diagram of the structure of an obstacle classification model training device provided in an embodiment of this application;

[0043] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0044] Figure 10 This is a schematic diagram of the structure of another electronic device provided in an embodiment of this application. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0046] In the following description of the design drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0047] In the description of this application, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0048] The autonomous vehicle 10 described in this application may include a sensing device and an on-board terminal. The sensing device includes one or more lidar sensors. When the sensing device includes multiple lidar sensors, the multiple lidar sensors can form a lidar system. The lidar sensors can typically be placed near the four corners of the vehicle, the front of the vehicle, the rear of the vehicle, the doors, the roof, etc. This application does not limit the placement location of the lidar sensors. Figure 1A An exemplary lidar system is shown, comprising three lidar units respectively mounted on the roof (101) and on both sides (102, 103) of the vehicle 10. Figure 1BAnother lidar system is illustrated, consisting of five lidar units positioned on the roof (1031), sides (1032, 1033), front (1034), and rear (1035) of the vehicle 10. It is understood that the lidar system may include more or fewer lidar units, and the lidar units may be located in other parts of the vehicle; this application does not impose any limitations in this regard. Figure 1A Taking the illustrated lidar system as an example, lidar units (101, 102, and 103) can be used to collect point cloud data within their radiation range (around the vehicle) and send the point cloud data to the vehicle-mounted terminal. Lidar unit 101 on the roof can be used to detect obstacles at a greater distance; lidar units 102 and 103 located on either side of the roof can be used to detect obstacles near the ground close to the vehicle body. The vehicle-mounted terminal can process the point cloud data sent by the lidar units (101, 102, and 103) to classify obstacles.

[0049] In existing technologies, machine learning methods are typically used for obstacle classification. However, machine learning methods require a high degree of density in the collected point cloud. If the point cloud is relatively sparse, it is impossible to accurately estimate the obstacle classification, which will affect the detection accuracy and detection range of the lidar or lidar system.

[0050] Figure 1C An exemplary schematic diagram of the architecture of an obstacle classification system provided in an embodiment of this application is shown. Figure 1C As shown, the obstacle classification system 1300 may include at least: an obstacle classification device 1310, a sensing device 1320, and an on-board terminal 1330.

[0051] It can be understood that the sensing device 1320 can be a single lidar, or it can be... Figure 1A and Figure 1B The aforementioned lidar system is used to collect point cloud data in real time.

[0052] The obstacle classification device 1310 is used to process the point cloud data transmitted by the receiving sensing device 1320 or a large amount of historical point cloud data stored therein to obtain an obstacle classification model.

[0053] The vehicle-mounted terminal 1330 is used to receive point cloud data collected in real time by the sensing device 1320, and to perform...

[0054] Point cloud data of obstacles are detected; wherein, the obstacles include the target obstacle and other obstacles within a preset distance.

[0055] The vehicle terminal 1330 obtains the category information and tracking information of the target obstacle based on the point cloud data of the target obstacle, and transmits the tracking information of the target obstacle to the obstacle classification device 1310. The obstacle classification model trained by the obstacle classification device 1310 obtains the classification result of the target obstacle.

[0056] The vehicle terminal 1330 is also used to process the point cloud data collected in real time by the sensor 1320 to obtain the point cloud data of other obstacles within a preset distance, and to obtain the attribute information of other obstacles within the preset distance based on the point cloud data of other obstacles within the preset distance.

[0057] The obstacle classification device 1310 outputs the classification results to the vehicle terminal 1330. The vehicle terminal 1330 summarizes the category information of the target obstacle, the tracking information of the target obstacle, and the attribute information of other obstacles within the preset distance, and inputs the acquired information into the obstacle classification model to obtain the optimized category information of the target obstacle. The vehicle terminal 1330 then outputs vehicle control commands based on the optimized category information of the target obstacle.

[0058] It is understood that the lidar can be a mechanical lidar, a solid-state lidar, etc., and the specific type of lidar is not limited here; or the sensing device 1320 can also be a lidar system composed of multiple lidars, and the number of lidars in the lidar system and the specific form of the lidar system are not limited here.

[0059] It is understood that the obstacle classification device 1310 can be integrated into the sensing device 1320 or exist independently of the sensing device 1320. When the obstacle classification device 1310 is integrated into the sensing device 1320, the obstacle classification device 1310 can be used to collect the first N frames of point cloud data from the sensing device 1320, using the first N frames of point cloud data as historical data to train the obstacle classification model. Furthermore, historical data can be used to verify the accuracy of the obstacle classification model's output results.

[0060] Optionally, the obstacle classification device 1310 may be fully or partially integrated into the vehicle terminal 1330.

[0061] It is understood that if the sensing device 1320 is a lidar system, the obstacle classification device 1310 can be one (i.e., the lidar system corresponds to one historical data tracking unit). Optionally, the obstacle classification device 1310 can also be multiple (i.e., each lidar in the lidar system corresponds to one obstacle classification device 1310, or multiple lidars correspond to at least two obstacle classification devices 1310).

[0062] The obstacle classification method provided in this application will be described in detail below with reference to specific embodiments. This method can be implemented using a computer program. This computer program can be integrated into an application or run as a standalone utility application.

[0063] In the following method embodiments, for ease of explanation, only the obstacle classification device is described as the subject performing each step.

[0064] The following will be combined with the appendix Figures 2 to 3 This paper provides a detailed description of the obstacle classification method provided in the embodiments of this application.

[0065] Please see Figure 2 This is a flowchart illustrating an obstacle classification method provided in this application. The method may include the following steps:

[0066] S201. Obtain the current frame point cloud data collected by the lidar.

[0067] Generally, point cloud data is the surface depth information of a measured object acquired by a LiDAR system, reflecting the three-dimensional coordinates of the object in space. Point cloud data can usually be represented by a set of vectors in a three-dimensional coordinate system. In addition to the geometric position of the measured object, point cloud data can also contain information such as color. Typically, point cloud data acquired by the LiDAR rotating at a preset angle (e.g., but not limited to 360 degrees) is defined as a frame. The LiDAR can acquire a whole frame of point cloud data at once, or it can acquire multiple frames, for example, acquiring a portion at a time (e.g., but not limited to 120 degrees), and then stitching all the portions together in the acquisition sequence to obtain a single frame of point cloud data. In one possible embodiment, the LiDAR can also acquire point cloud data using different scanning methods such as interval scanning and adaptive scanning. In the actual process of LiDAR acquiring point cloud data, due to the complex terrain and the differences in the reflectivity of the scanned object's surface, the point cloud data acquired by the LiDAR often contains a lot of noise, and the overlap rate of point cloud data between frames is relatively low due to occlusion by buildings and obstacles. These issues can directly affect the final decision on obstacle classification. Therefore, the process of LiDAR acquiring point cloud data can also include point cloud preprocessing such as point cloud filtering and point cloud registration. In the obstacle classification system of this application embodiment, the obstacle classification device first needs to acquire the current frame point cloud data acquired by the LiDAR system. The LiDAR can directly send the point cloud data to the obstacle classification device, or it can first send the point cloud data to a storage device (e.g., but not limited to a cloud platform or storage server) for data backup. Then, the obstacle classification device can perform subsequent obstacle classification processing based on the current frame point cloud data to obtain optimized category information.

[0068] S202, Detect the point cloud data of the obstacle based on the point cloud data of the current frame.

[0069] Generally, the obstacles include target obstacles and other obstacles within a preset distance. Detection can refer to identifying all obstacle information in a current frame of point cloud data through methods such as clustering and deep learning. After acquiring the current frame of point cloud data collected by the LiDAR, the obstacle classification device needs to detect the obstacle point cloud data in the current frame of point cloud data. For example, the obstacle classification device can use a pre-set local machine learning method (e.g., but not limited to the Pointnet++ deep learning model) to detect all obstacle point clouds in the current frame of point cloud data. For example, the current frame of point cloud data has point clouds numbered 1-120. The obstacle classification device detects point cloud data of points 1-30 as non-obstacles (e.g., but not limited to road signs, trees, etc.), point cloud data of points 31-80 as target obstacles (e.g., but not limited to truck a), and point cloud data of points 81-120 as other obstacles (e.g., but not limited to motorcycle a, electric vehicle a, etc.). In one possible embodiment, the obstacle classification device can also send the collected current frame point cloud data to the cloud (e.g., but not limited to multiple detection servers that can perform detection simultaneously) for detection, which helps to improve detection efficiency and reduce local storage space, and then receive the detection results from the cloud.

[0070] S203. Obtain the category information of the target obstacle and the tracking information of the target obstacle based on the point cloud data of the target obstacle.

[0071] Generally, the category information of the target obstacle can include categories such as pedestrians, bicycles, motorcycles, tricycles, cars, and trucks. The tracking information of the target obstacle can include the size, motion state, displacement direction, and lane information of the target obstacle. The size can be represented by the length and width of a rectangle, and the motion state can refer to the static state or the non-static state (e.g., but not limited to linear motion, curvilinear motion, etc.) of the target obstacle. The obstacle classification device can first preset a defined reference coordinate system, and then determine the displacement direction of the target obstacle (e.g., but not limited to the 143-degree direction) based on the reference coordinate system. Lane information can refer to the type of fast lane, slow lane, or pedestrian crossing, etc. The obstacle classification device can determine the lane information by the moving speed of the target obstacle. In the prior art, traditional algorithms (e.g., but not limited to Principal Component Analysis (PCA) algorithms) or deep learning-based methods (e.g., but not limited to Pointnet++) are usually used to analyze the category information and tracking information of the target obstacle, and then the category information is used as the final category information for subsequent processing. However, in actual use, technical problems such as low accuracy and high false negative rate will occur. In this embodiment, the obtained category information and other useful information can be used as intermediate parameters. Then, through further optimization, more accurate optimized category information can be obtained, thereby improving the detection accuracy.

[0072] S204. Obtain the attribute information of other obstacles within the preset distance based on the point cloud data of other obstacles within the preset distance.

[0073] Generally, the attribute information includes motion information of other obstacles within the preset distance and scene information. The motion information may include the size, motion state, and displacement direction of other obstacles, while the scene information may include lane information and curb distance of other obstacles. After obtaining the category information and tracking information of the target obstacle, the obstacle classification device also needs to obtain the attribute information of other obstacles within the preset distance based on the point cloud data of other obstacles within the preset distance. By obtaining multiple correlations and multi-dimensional information of the target obstacle and other obstacles, the optimized category information of the target obstacle can be detected more accurately, which helps to improve the accuracy of subsequent applications such as target obstacle trajectory tracking. For example, the obstacle classification device determines that there is another obstacle 'a' within a preset distance (e.g., but not limited to 2.5 meters). The size of this other obstacle 'a' is 2 × 1.3 meters, its motion state is linear, its displacement direction is 175 degrees, its lane information is lane 2, and its curb distance is 4.5 meters. In one possible embodiment, after obtaining the attribute information of other obstacles within the preset distance, the obstacle classification device can also upload the attribute information to the cloud for backup processing.

[0074] S205. Input the category information of the target obstacle, the tracking information of the target obstacle, and the attribute information of other obstacles within the preset distance into the obstacle classification model to obtain the optimized category information of the target obstacle.

[0075] Generally, after acquiring the attribute information of other obstacles within the preset distance, the obstacle classification device can input the category information of the target obstacle, the tracking information of the target obstacle, and the attribute information of other obstacles within the preset distance into a pre-trained obstacle classification model to obtain the optimized category information of the target obstacle. Since this embodiment requires inputting various associations and multi-dimensional information about the target obstacle and other associated obstacles, the obstacle classification device can obtain more accurate and stable optimized category information, and is insensitive to minor deviations in the detection results, possessing a certain ability to handle outliers. For example, assuming the actual category of the target obstacle is an electric vehicle, the optimized category information obtained by the obstacle classification device in this application is "electric vehicle," but the detected category information of the target obstacle may be "bicycle." In this application embodiment, the obstacle classification device can more accurately determine the possible category of the target obstacle by combining multiple types of information acquired, compared with the prior art. For example, if the obstacle is moving at a high speed, the obstacle classification device can determine that the obstacle is more likely to be a vehicle; if the obstacle's displacement direction is perpendicular to the road direction and its size is small, the obstacle classification device can determine that the obstacle is more likely to be a pedestrian crossing the road; if the obstacle is surrounded by other high-speed obstacles, the obstacle classification device can determine that the obstacle is more likely to be a vehicle; if the obstacle's location at other times is a vehicle, the obstacle classification device can determine that the obstacle is more likely to be a vehicle (e.g., if the obstacle is in the driving lane and vehicles pass through the driving lane at other times, then the obstacle is more likely to be a vehicle).

[0076] In one possible embodiment, after determining the optimized category information, the obstacle classification device can delete the processed data such as the current frame point cloud data to reduce local storage space and help improve operating efficiency, or upload it to a backup server for backup processing before deleting it.

[0077] As described above, the process involves acquiring point cloud data from the current frame collected by the lidar, detecting point cloud data of obstacles based on this data, obtaining the category information and tracking information of the target obstacle based on its point cloud data, obtaining attribute information of other obstacles within a preset distance based on their point cloud data, and inputting the category information, tracking information, and attribute information of the target obstacle into an obstacle classification model to obtain optimized category information for the target obstacle. This embodiment of the application, by considering that obstacles are not independent in both the spatial and temporal domains, can fully utilize the category information, tracking information, and attribute information of the target obstacle within a preset distance to obtain more accurate classification and detection results, thereby improving the efficiency and accuracy of obstacle classification.

[0078] Please see Figure 3 This is another flowchart illustrating an obstacle classification method provided in this application. The obstacle classification method may include the following steps:

[0079] S301. Obtain the current frame point cloud data collected by the lidar, and detect the point cloud data of the obstacle based on the current frame point cloud data.

[0080] Generally, obstacle classification devices first need to acquire the current frame point cloud data collected by lidar, and then detect the point cloud data of obstacles using obstacle recognition and segmentation algorithms. The obstacles include the target obstacle and other obstacles within a preset distance. For example... Figure 4 As shown, the obstacle classification device uses the PointNet++ algorithm to detect and process the point cloud data of the current frame in the left image, resulting in the result shown in the right image. It detects the point cloud data of obstacles such as pedestrians and cars. Generally, the point cloud data of different obstacles can be marked by different colors or different cuboids.

[0081] S302. The point cloud data of the target obstacle is processed by an attitude estimation algorithm to obtain the category information of the target obstacle.

[0082] Generally, the attitude estimation model can be a trained neural network model that calculates obstacle category information from point cloud data. After the obstacle classification device detects the point cloud data of the obstacle, it processes the point cloud data of the target obstacle (e.g., but not limited to, through the attitude estimation model) to obtain the category information of the target obstacle (e.g., but not limited to, unlicensed electric vehicles).

[0083] S303. Obtain tracking information of the target obstacle based on at least two frames of point cloud data of the target obstacle.

[0084] Generally, the tracking information of the target obstacle may include at least the size, motion state, displacement direction, and lane information of the target obstacle. After obtaining the category information of the target obstacle, the obstacle classification device can acquire the tracking information of the target obstacle through at least two frames (e.g., but not limited to at least two consecutive frames with the current frame, or at least two frames of historical point cloud data) of the target obstacle's point cloud data. For example: if the obstacle classification device determines that the size of the target obstacle is 2.4 × 1.6 meters, its position coordinates in the current frame are (93, 135), and its position coordinates in the previous frame are (94, 175), after simple calculation, it can determine that the motion state of the target obstacle is linear motion, the displacement direction is 90 degrees based on the reference coordinate system, and the moving speed is 5 meters per second, which falls within a preset low-speed range, then the lane information is determined to be a slow lane. Figure 5As shown, the obstacle classification device first determines the reference coordinate system with the horizontal direction as the x-axis, the vertical direction as the y-axis, and the top-view direction as the z-axis. It then determines the forward direction based on the longer range of the obstacle point cloud and the left-hand forward direction based on the wider range. Finally, it determines information such as size and displacement direction.

[0085] S304. Calculate the size, motion state, and displacement direction of other obstacles within the preset distance in the current frame point cloud data, and determine the motion information of the other obstacles based on their size, motion state, and displacement direction.

[0086] Generally, after acquiring the tracking information of the target obstacle, the obstacle classification device also needs to calculate the size, motion state, and displacement direction of other obstacles within the preset distance in the current frame's point cloud data. For example, the obstacle classification device can determine the size of other obstacle a as 2.5 × 0.7 meters based on the current frame, and then calculate the motion state of other obstacle a as stationary and the displacement direction as 0 based on the point cloud data of other obstacle a from at least two frames. Finally, this information can be used as the motion information of other obstacle a.

[0087] S305. Obtain the lane information and roadside distance of other obstacles within the preset distance from the point cloud data of the current frame point cloud data, and determine the scene information of the other obstacles based on the lane information and roadside distance of the other obstacles.

[0088] Generally, after determining the motion information of other obstacles, the obstacle classification device also needs to obtain the lane information and curb distance of the other obstacles based on the point cloud data of other obstacles within the preset distance in the current frame point cloud data. The lane information can be represented by lane number or other methods. For example, the scene information of other obstacle b determined by the obstacle classification device may include lane information as lane 1 and curb distance as 0.5 meters.

[0089] S306. Determine the tracking sequence of the target obstacle based on the tracking information of the target obstacle.

[0090] Generally, in one possible embodiment, after the obstacle classification device acquires the tracking information of the target obstacle, it can also determine the tracking sequence of the target obstacle based on the tracking information. The tracking sequence can be used to characterize the continuous correlation of the target obstacle in multi-frame point cloud data. For example, the obstacle classification device can number the obstacle points in the current point cloud data as 1, 2, 3, 4, and 5. There are five sequences A, B, C, D, and E in the historical tracking sequences. The two sets of sequence numbers are matched using machine learning methods such as the Hungarian algorithm or the PointNet++ algorithm to calculate the matching value. Based on the matching value, obstacle number 1 is determined to belong to sequence B, obstacle number 2 to belong to sequence D, obstacle number 3 to belong to sequence A, obstacle number 4 to belong to sequence E, and obstacle number 5 to belong to sequence C.

[0091] S307. Input the category information of the target obstacle, the tracking information of the target obstacle, and the attribute information of other obstacles within the preset distance into the obstacle classification model to obtain the optimized category information of the target obstacle.

[0092] Generally, after the obstacle classification device determines the scene information of the other obstacles, it can convert the category information of the target obstacle, the tracking information of the target obstacle, and the attribute information of the other obstacles within the preset distance into an input feature vector of preset dimensions. Then, the input feature vector is input into the trained obstacle classification model to obtain the optimized category information of the target obstacle. For example, [2,1.5,3,1,30,3,3.5,15,2,5,125,354,136,365,147,375,150,373,140,360,...] can be the input feature vector obtained by the obstacle classification device. The first element represents the category information of the target obstacle (which can be represented by 2 for bicycle category), the second and third elements represent the size (in meters), the fourth element represents the motion state (which can be represented by 1 for linear motion), the fifth element represents the displacement direction (in meters), the sixth element represents the lane information (which can be represented by lane number 3), the seventh element represents the distance to the curb, and the eighth, ninth, and so on elements represent other specific parameters in the obstacle attribute information. Then, the output obtained by the obstacle classification device can be represented as [2,95%], which determines the optimized category information of the target obstacle as bicycle category 2 with an accuracy of 95%.

[0093] In one possible embodiment, the calculation process can also include weighted processing, calculating the input feature vector using the formula: T = G * w G +H*w H +Z*w ZWherein, T represents the input feature vector, G represents the feature vector corresponding to the category information of the target obstacle, and w G The w represents the corresponding weight, H represents the feature vector corresponding to the tracking information of the target obstacle, and w represents the weight. H The corresponding weight is represented by Z, which represents the feature vector corresponding to the attribute information of other obstacles within the preset distance, and w is the weight. Z w represents the corresponding weight. G w H and w Z The input feature vector is obtained by taking real numbers greater than 0 and less than 1. The weights of each element can also be determined based on the category information of the target obstacle, the tracking information of the target obstacle, and the attribute information of other obstacles within the preset distance.

[0094] S308. Update the tracking sequence of the target obstacle according to the optimized category information of the target obstacle.

[0095] Generally, in one possible embodiment, after the obstacle classification device obtains the optimized category information of the target obstacle, it can update the tracking sequence of the target obstacle according to the optimized category information. For example, the obstacle classification device determines that target obstacle number 1 belongs to sequence B, other obstacles number 2 belong to sequence D, other obstacles number 3 belong to sequence A, other obstacles number 4 belong to sequence E, and other obstacles number 5 belong to sequence C based on the category information of the target obstacle. Then, in this step, the obstacle classification device performs matching calculations based on the optimized category information to determine that target obstacle number 1 belongs to sequence A, other obstacles number 2 belong to sequence B, other obstacles number 3 belong to sequence C, other obstacles number 4 belong to sequence D, and other obstacles number 5 belong to sequence E, which serves as the updated tracking sequence for target obstacle number 1.

[0096] S309. Determine the point cloud constraints corresponding to the target obstacle based on the optimization category information and the tracking information of the target obstacle.

[0097] Generally, the point cloud constraints can be used to determine the position of the target obstacle. The obstacle classification device can determine the size range of the target obstacle based on its optimized category information and tracking information, and then determine the corresponding point cloud constraints based on this size range. The shape of the size range can be represented by a rectangle, which may include orientation and size parameters. The point cloud constraints can be used to constrain the position of the target obstacle and can represent the coordinate range of the target obstacle's center point. The obstacle classification device can then further determine the precise center point of the target obstacle based on the aforementioned point cloud constraints.

[0098] In this embodiment, the solution acquires point cloud data of the current frame collected by a lidar, detects point cloud data of obstacles based on the current frame point cloud data, obtains the category information and tracking information of the target obstacle based on the point cloud data of the target obstacle, obtains the attribute information of other obstacles within a preset distance based on the point cloud data of other obstacles within a preset distance, and inputs the category information, tracking information, and attribute information of other obstacles within a preset distance into an obstacle classification model to obtain optimized category information of the target obstacle. This embodiment, by considering that obstacles are not independent in the spatial and temporal domains, can fully utilize the category information, tracking information, and attribute information of other obstacles within a preset distance of the target obstacle to obtain more accurate classification and detection results, thereby improving the efficiency and accuracy of obstacle classification.

[0099] In another optional embodiment of this application, the method further includes: acquiring the scene of the current frame point cloud data, and optimizing the preset distance based on the scene of the current frame point cloud data.

[0100] For example, when the scenario is determined to be a high-speed scenario, the selected preset distance should be larger because the density of surrounding obstacles is relatively low. When the scenario is determined to be a low-speed intersection scenario, the selected preset distance should be smaller because the density of surrounding obstacles is relatively high.

[0101] It is understandable that after determining the scene in which the current frame point cloud data is located, the method further includes determining the number of obstacles included in the current frame point cloud data. If the number of obstacles included in the current frame point cloud data exceeds a first preset value, the preset distance is reduced. If the number of obstacles included in the current frame point cloud data is less than the first preset value and the speed of the obstacles is greater than a second preset value, the preset distance is increased.

[0102] Through the above embodiments, the range of preset distances for analysis and calculation can be adjusted according to the scene and the density of obstacles, thereby reducing the amount of calculation and improving the monitoring speed.

[0103] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0104] Please see Figure 6 This illustration shows a schematic diagram of an obstacle sorting device provided in an exemplary embodiment of this application, hereinafter referred to as sorting device 6. Sorting device 6 can be implemented as all or part of a terminal through software, hardware, or a combination of both. It includes:

[0105] The first acquisition module 601 is used to acquire the current frame point cloud data collected by the lidar;

[0106] The detection module 602 is used to detect the point cloud data of obstacles based on the current frame point cloud data; wherein, the obstacles include the target obstacle and other obstacles within a preset distance;

[0107] The second acquisition module 603 is used to acquire the category information of the target obstacle and the tracking information of the target obstacle based on the point cloud data of the target obstacle;

[0108] The third acquisition module 604 is used to acquire attribute information of other obstacles within the preset distance based on point cloud data of other obstacles within the preset distance; wherein, the attribute information includes motion information of other obstacles within the preset distance and scene information;

[0109] The optimization module 605 is used to input the category information of the target obstacle, the tracking information of the target obstacle, and the attribute information of other obstacles within the preset distance into the obstacle classification model to obtain the optimized category information of the target obstacle.

[0110] In one possible embodiment, the second acquisition module 603 includes:

[0111] The processing unit is used to process the point cloud data of the target obstacle using an attitude estimation algorithm to obtain the category information of the target obstacle;

[0112] The acquisition unit is used to acquire tracking information of the target obstacle based on at least two frames of point cloud data of the target obstacle.

[0113] In one possible embodiment, the third acquisition module 604 includes:

[0114] The first determining unit is used to calculate the size, motion state and displacement direction of other obstacles within the preset distance in the current frame point cloud data, and to determine the motion information of other obstacles based on the size, motion state and displacement direction of other obstacles;

[0115] The second determining unit is used to obtain the lane information and roadside distance of other obstacles within the preset distance in the current frame point cloud data, and to determine the scene information of other obstacles based on the lane information and roadside distance of other obstacles.

[0116] In one possible embodiment, the sorting device 6 further includes:

[0117] The second determining module is used to determine the tracking sequence of the target obstacle based on the tracking information of the target obstacle;

[0118] An update module is used to update the tracking sequence of the target obstacle based on the optimized category information of the target obstacle.

[0119] In one possible embodiment, the sorting device 6 further includes:

[0120] The third determining module is used to determine the point cloud constraints corresponding to the target obstacle based on the optimization category information and the tracking information of the target obstacle; wherein, the point cloud constraints are used to determine the position of the target obstacle.

[0121] This application's embodiments and Figures 2 to 3 The method implementations are based on the same concept and bring about the same technical effects. The specific process can be referred to Figures 2 to 3 The method embodiments described herein will not be repeated here.

[0122] The device 6 can be a field-programmable gate array (FPGA), a dedicated integrated chip, a system on chip (SoC), a central processing unit (CPU), a network processor (NP), a digital signal processing circuit, a micro controller unit (MCU), or a programmable logic device (PLD) or other integrated chip to implement the relevant functions.

[0123] In this embodiment, the solution acquires current frame point cloud data collected by a lidar, detects obstacle point cloud data based on the current frame point cloud data, obtains the target obstacle's category information and tracking information based on the target obstacle's point cloud data, obtains the attribute information of other obstacles within a preset distance based on their point cloud data, and inputs the target obstacle's category information, tracking information, and attribute information of other obstacles within a preset distance into an obstacle classification model to obtain optimized category information for the target obstacle. This embodiment, by considering that obstacles are not independent in the spatial and temporal domains, can fully utilize the target obstacle's category information, tracking information, and attribute information of other obstacles within the preset distance to obtain more accurate category detection results, thereby improving the efficiency and accuracy of obstacle classification.

[0124] In the following method embodiments, for ease of explanation, only the obstacle classification model training device is used as the subject of execution for each step.

[0125] The following will be combined with the appendix Figure 7 This paper provides a detailed description of the obstacle classification model training method provided in the embodiments of this application.

[0126] Please see Figure 7 This is a flowchart illustrating a method for training an obstacle classification model, as provided in this application embodiment. The method may include the following steps:

[0127] S701. Acquire at least one frame of point cloud data collected by the lidar.

[0128] Generally, point cloud data is the surface depth information of a measured object acquired by a LiDAR system, reflecting the three-dimensional coordinates of the object in space. Point cloud data can usually be represented by a set of vectors in a three-dimensional coordinate system. In addition to the geometric position of the measured object, point cloud data can also contain information such as color. Typically, point cloud data acquired by the LiDAR rotating at a preset angle (e.g., but not limited to 360 degrees) is defined as a frame. The LiDAR can acquire a frame of point cloud data at once, or it can acquire multiple frames, acquiring portions (e.g., but not limited to a 90-degree range) each time, and then stitching all portions together according to the acquisition sequence to obtain a single frame of point cloud data. In one possible embodiment, the LiDAR can also acquire point cloud data using different scanning methods such as interval scanning and adaptive scanning. In the obstacle classification model training method of this application embodiment, the obstacle classification model training device first needs to acquire at least one frame of continuous point cloud data acquired by the LiDAR system. Then, it can perform point cloud preprocessing such as point cloud filtering and point cloud registration, and then use the processed point cloud data for subsequent obstacle classification model training steps.

[0129] S702, Detect the point cloud data of the obstacle based on the at least one frame of point cloud data.

[0130] Generally, the obstacles include the target obstacle and other obstacles within a preset distance. Detection can refer to identifying all obstacle information in a single frame of point cloud data using methods such as clustering and deep learning. After acquiring at least one frame of point cloud data from the LiDAR, the obstacle classification model training device needs to detect the obstacle point cloud data in that at least one frame of point cloud data. For example, the obstacle classification model training device can use a pre-set local machine learning method (e.g., but not limited to the Pointnet++ deep learning model) to detect all obstacle point clouds in the at least one frame of point cloud data. For example, in the third frame of point cloud data, there are point clouds numbered 1-100. The obstacle classification model training device detects that points 1-20 are point cloud data of non-obstacles (such as but not limited to road signs, trees, etc.), points 21-70 are point cloud data of target obstacles (such as but not limited to car a), and points 71-100 are point cloud data of other obstacles (such as but not limited to bicycle a, electric vehicle a, etc.). The point cloud data can be further numbered, such as car a being numbered 1, bicycle a being numbered 2, and electric vehicle a being numbered 3, to facilitate subsequent processing and calculation.

[0131] S703. Obtain the category information of the target obstacle, the tracking information of the target obstacle, and the scene information of the target obstacle based on the point cloud data of the target obstacle.

[0132] Generally, the tracking information of the target obstacle can include the size, motion state, displacement direction, and lane information of the target obstacle. The size can be represented by the length and width of a rectangle, and the motion state can refer to the target obstacle's stationary state or non-stationary state (e.g., but not limited to linear motion and curvilinear motion). Scene information can refer to the lane information and distance from the curb of the target obstacle. The obstacle classification model training device can first pre-determine a reference coordinate system, and then, based on this reference coordinate system, determine the displacement direction of the target obstacle (e.g., but not limited to the 30-degree direction) based on at least two frames of point cloud data of the target obstacle. Lane information can include one of the following: fast lane type, slow lane type, or pedestrian crossing type. The obstacle classification model training device can determine the lane information based on the moving speed of the target obstacle. In the prior art, traditional algorithms (e.g., but not limited to the PCA algorithm) or deep learning-based methods (e.g., but not limited to Pointnet++) are typically used to analyze the category information, tracking information, and scene information of the target obstacle, and then the category information, tracking information, and scene information of the target obstacle are used for subsequent processing. In this embodiment, the obtained category information, tracking information, and scene information can be used as intermediate parameter information. Then, through further training, a more accurate classification model can be obtained, thereby improving the stability and accuracy of model training.

[0133] S704. Obtain the attribute information and tracking information of other obstacles within the preset distance based on the point cloud data of other obstacles within the preset distance.

[0134] Generally, the tracking information can refer to the size, motion state, and displacement direction of other obstacles. The attribute information includes the motion information and scene information of other obstacles within the preset distance. The scene information can refer to the lane information and curb distance of other obstacles, and the motion information can refer to the lane information of other obstacles. After obtaining the category information, tracking information, and scene information of the target obstacle, the obstacle classification model training device also needs to obtain the attribute information and tracking information of other obstacles within the preset distance based on at least two frames of point cloud data of other obstacles within the preset distance. By obtaining multiple associations and multi-dimensional information of the target obstacle and other obstacles, a more accurate and stable obstacle classification model can be trained. For example, the obstacle classification model training device determines that there is another obstacle 'a' within a preset distance (e.g., but not limited to 3 meters). The size of this other obstacle 'a' is 1.5 × 3 meters, its motion state is linear, its displacement direction is 90 degrees, its lane information is a fast lane, its lane information is lane 3, and its curb distance is 3.5 meters.

[0135] S705. Obtain the attribute information and tracking information of other obstacles within the preset distance based on the point cloud data of other obstacles within the preset distance.

[0136] Generally, obstacle classification models can be deep neural network models. The obstacle classification model training device obtains the category information, tracking information, and scene information of the target obstacle, as well as the attribute information and tracking information of other obstacles. It then uses the category information, tracking information, and scene information of the target obstacle, the attribute information and tracking information of other obstacles, and manually labeled optimized category information as training samples to train the obstacle classification model. A loss function is calculated, and based on the calculated value of the loss function, the parameters of the obstacle classification model are updated to obtain the trained parameters.

[0137] As described above, the process involves acquiring at least one frame of point cloud data from a lidar sensor, detecting point cloud data of obstacles based on this frame, obtaining the category information, tracking information, and scene information of the target obstacle based on its point cloud data, and acquiring the attribute information and tracking information of other obstacles within a preset distance based on their point cloud data. The category information, tracking information, and scene information of the target obstacle, the attribute information and tracking information of the other obstacles, and manually labeled optimized category information are then used as training samples to train the obstacle classification model. This embodiment of the application fully utilizes point cloud data to obtain various information about obstacles as training samples, thereby making the final classification result more accurate and effectively improving the stability of the obstacle classification model training.

[0138] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0139] Please see Figure 8 This illustration shows a schematic diagram of an obstacle classification model training device provided in an exemplary embodiment of this application, hereinafter referred to as training device 8. Training device 8 can be implemented as all or part of a terminal through software, hardware, or a combination of both. It includes:

[0140] The first acquisition module 801 is used to acquire at least one frame of point cloud data collected by the lidar;

[0141] The detection module 802 is used to detect the point cloud data of obstacles based on the at least one frame of point cloud data; wherein the obstacles include the target obstacle and other obstacles within a preset distance;

[0142] The second acquisition module 803 is used to acquire the category information of the target obstacle, the tracking information of the target obstacle, and the scene information of the target obstacle based on the point cloud data of the target obstacle;

[0143] The third acquisition module 804 is used to acquire attribute information and tracking information of other obstacles within the preset distance based on point cloud data of other obstacles within the preset distance; wherein, the attribute information includes motion information and scene information of other obstacles within the preset distance;

[0144] The training module 805 is used to train the obstacle classification model using the category information, tracking information, and scene information of the target obstacle, the attribute information and tracking information of other obstacles, and the manually labeled optimized category label information as training samples.

[0145] This application's embodiments and Figure 7 The method implementations are based on the same concept and bring about the same technical effects. The specific process can be referred to Figure 7 The method embodiments described herein will not be repeated here.

[0146] The device 8 can be a field-programmable gate array (FPGA), a dedicated integrated chip, a system on chip (SoC), a central processing unit (CPU), a network processor (NP), a digital signal processing circuit, a micro controller unit (MCU), or a programmable logic device (PLD) or other integrated chips to implement the relevant functions.

[0147] In this embodiment, the solution acquires at least one frame of point cloud data collected by a LiDAR scanner. Based on this frame, point cloud data of obstacles is detected. The target obstacle's category information, tracking information, and scene information are obtained from its point cloud data. The attribute information and tracking information of other obstacles within a preset distance are obtained from their point cloud data. The target obstacle's category information, tracking information, scene information, the other obstacles' attribute information, tracking information, and manually labeled optimized category information are used as training samples to train the obstacle classification model. This embodiment fully utilizes point cloud data to acquire various information about obstacles as training samples, resulting in more accurate classification results and effectively improving the stability of the obstacle classification model training.

[0148] This application also provides a computer storage medium that can store multiple instructions adapted for loading and execution by a processor of the method steps described above. For details of the execution process, please refer to... Figure 2 or Figure 3 or Figure 7 The specific details of the illustrated embodiments will not be elaborated here.

[0149] This application also provides a computer program product that stores at least one instruction, which is loaded and executed by the processor to implement the template control method described in the above embodiments.

[0150] Please see Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 9 As shown, the electronic device 9 may include: at least one processor 901, at least one network interface 904, a user interface 903, a memory 905, and at least one communication bus 902. The communication bus 902 is used to enable communication between these components. The user interface 903 may include a display screen or a space-based radar (SBR). Optionally, the user interface 903 may also include a standard wired interface or a wireless interface. The network interface 904 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0151] The processor 901 may include one or more processing cores. The processor 901 connects to various parts within the terminal 900 using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 905, and by calling data stored in the memory 905. Optionally, the processor 901 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 901 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 901 and may be implemented as a separate chip.

[0152] The memory 905 may include random access memory (RAM) or read-only memory. Optionally, the memory 905 may include a non-transitory computer-readable storage medium. The memory 905 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 905 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 905 may also be at least one storage device located remotely from the aforementioned processor 901. Figure 9 As shown, the memory 905, as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an obstacle classification application. Figure 9In the illustrated electronic device 900, the user interface 903 is primarily used to provide an input interface for the user and to acquire user input data. The processor 901 can be used to call the obstacle classification application stored in the memory 905 and specifically perform the following operations:

[0153] Acquire the current frame point cloud data collected by the lidar;

[0154] Point cloud data of obstacles are detected based on the current frame point cloud data; wherein, the obstacles include the target obstacle and other obstacles within a preset distance;

[0155] Based on the point cloud data of the target obstacle, obtain the category information and tracking information of the target obstacle;

[0156] The attribute information of other obstacles within the preset distance is obtained based on the point cloud data of other obstacles within the preset distance; wherein, the attribute information includes the motion information of other obstacles within the preset distance and scene information;

[0157] The target obstacle's category information, the target obstacle's tracking information, and the attribute information of other obstacles within the preset distance are input into the obstacle classification model to obtain the target obstacle's optimized category information.

[0158] In one possible embodiment, when the processor 901 executes the step of obtaining the category information of the target obstacle and the tracking information of the target obstacle based on the point cloud data of the target obstacle, it specifically performs the following:

[0159] The target obstacle's category information is obtained by processing the point cloud data of the target obstacle using an attitude estimation algorithm;

[0160] The tracking information of the target obstacle is obtained based on at least two frames of point cloud data of the target obstacle.

[0161] In one possible embodiment, when the processor 901 executes the step of obtaining attribute information of other obstacles within the preset distance based on point cloud data of other obstacles within the preset distance, it specifically performs the following:

[0162] Calculate the size, motion state, and displacement direction of other obstacles within the preset distance in the current frame point cloud data, and determine the motion information of other obstacles based on their size, motion state, and displacement direction.

[0163] Based on the point cloud data of other obstacles within the preset distance in the current frame point cloud data, obtain the lane information and roadside distance of the other obstacles, and determine the scene information of the other obstacles based on the lane information and roadside distance of the other obstacles.

[0164] In one possible embodiment, after the processor 901 executes the step of obtaining the category information and tracking information of the target obstacle based on the point cloud data of the target obstacle, and before inputting the category information, tracking information, and attribute information of other obstacles within the preset distance into the obstacle classification model to obtain the optimized category information of the target obstacle, it is further configured to execute:

[0165] The tracking sequence of the target obstacle is determined based on the tracking information of the target obstacle;

[0166] After inputting the category information of the target obstacle, the tracking information of the target obstacle, and the attribute information of other obstacles within the preset distance into the obstacle classification model to obtain the optimized category information of the target obstacle, the process is further used to perform:

[0167] The tracking sequence of the target obstacle is updated based on the optimized category information of the target obstacle.

[0168] In one possible embodiment, after the processor 901 executes the step of inputting the category information of the target obstacle, the tracking information of the target obstacle, and the attribute information of other obstacles within the preset distance into the obstacle classification model to obtain the optimized category information of the target obstacle, it is further configured to execute:

[0169] The point cloud constraints corresponding to the target obstacle are determined based on the optimization category information and the tracking information of the target obstacle; wherein, the point cloud constraints are used to determine the position of the target obstacle.

[0170] The technical concept and embodiments of this application Figure 2 or Figure 3 The technical concept is the same; the specific process can be found by referring to... Figure 2 or Figure 3 The method implementation examples are not described in detail here.

[0171] In this embodiment, point cloud data of the current frame collected by a lidar is acquired. Point cloud data of obstacles is detected based on this data. Category information and tracking information of the target obstacle are obtained from the point cloud data of the target obstacle. Attribute information of other obstacles within a preset distance is obtained from their point cloud data. The category information, tracking information, and attribute information of the target obstacle are then input into an obstacle classification model to obtain optimized category information for the target obstacle. This embodiment, by considering that obstacles are not independent in both the spatial and temporal domains, can fully utilize the category information, tracking information, and attribute information of the target obstacle within the preset distance to obtain more accurate category detection results, thereby improving the efficiency and accuracy of obstacle classification.

[0172] Please see Figure 10 This provides a schematic diagram of another electronic device for embodiments of this application. Figure 10 As shown, the electronic device 10 may include: at least one processor 1001, at least one network interface 1004, a user interface 1003, a memory 1005, and at least one communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or a space-based radar (SBR). Optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0173] The processor 1001 may include one or more processing cores. The processor 1001 connects to various parts within the terminal 1000 using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 1005, and by calling data stored in the memory 1005. Optionally, the processor 1001 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 1001 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip without being integrated into the processor 1001.

[0174] The memory 1005 may include random access memory (RAM) or read-only memory. Optionally, the memory 1005 may include a non-transitory computer-readable storage medium. The memory 1005 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 1005 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 1005 may also be at least one storage device located remotely from the aforementioned processor 1001. Figure 10 As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an obstacle classification model training application. Figure 10In the illustrated electronic device 1000, the user interface 1003 is primarily used to provide an input interface for the user and to acquire user input data. The processor 1001 can be used to call the obstacle classification model training application stored in the memory 1005 and specifically perform the following operations:

[0175] Acquire at least one frame of point cloud data collected by the lidar;

[0176] Point cloud data of obstacles are detected based on at least one frame of point cloud data; wherein, the obstacles include the target obstacle and other obstacles within a preset distance;

[0177] Based on the point cloud data of the target obstacle, obtain the category information of the target obstacle, the tracking information of the target obstacle, and the scene information of the target obstacle;

[0178] The attribute information and tracking information of other obstacles within the preset distance are obtained based on the point cloud data of other obstacles within the preset distance; wherein, the attribute information includes the motion information and scene information of other obstacles within the preset distance;

[0179] The obstacle classification model is trained using the category information, tracking information, and scene information of the target obstacle, the attribute information and tracking information of other obstacles, and the manually labeled optimized category information as training samples.

[0180] The technical concept and embodiments of this application Figure 7 The technical concept is the same; the specific process can be found by referring to... Figure 7 The method implementation examples are not described in detail here.

[0181] In this embodiment, at least one frame of point cloud data collected by a lidar is acquired. Point cloud data of obstacles is detected based on this frame. The category information, tracking information, and scene information of the target obstacle are obtained from the point cloud data of the target obstacle. The attribute information and tracking information of other obstacles within a preset distance are obtained from the point cloud data of other obstacles within a preset distance. The category information, tracking information, and scene information of the target obstacle, the attribute information and tracking information of the other obstacles, and manually labeled optimized category information are used as training samples to train the obstacle classification model. This embodiment fully utilizes point cloud data to obtain various information about obstacles as training samples, making the final classification result more accurate and effectively improving the stability of the obstacle classification model training.

[0182] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory, or random access memory, etc.

[0183] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A method of obstacle classification, characterized by, The method includes: Acquire the current frame point cloud data collected by the lidar; Point cloud data of obstacles are detected based on the current frame point cloud data; wherein, the obstacles include the target obstacle and other obstacles within a preset distance; Based on the point cloud data of the target obstacle, obtain the category information and tracking information of the target obstacle; The attribute information of other obstacles within the preset distance is obtained based on the point cloud data of other obstacles within the preset distance; wherein, the attribute information includes the motion information of other obstacles within the preset distance and scene information; The target obstacle's category information, the target obstacle's tracking information, and the attribute information of other obstacles within the preset distance are input into the obstacle classification model to obtain the target obstacle's optimized category information; The step of obtaining the category information and tracking information of the target obstacle based on the point cloud data of the target obstacle includes: The target obstacle's category information is obtained by processing the point cloud data of the target obstacle using an attitude estimation algorithm; the target obstacle's tracking information is obtained based on at least two frames of the target obstacle's point cloud data.

2. The method of claim 1, wherein, The step of obtaining attribute information of other obstacles within the preset distance based on point cloud data of other obstacles within the preset distance includes: Calculate the size, motion state, and displacement direction of other obstacles within the preset distance in the current frame point cloud data, and determine the motion information of other obstacles based on their size, motion state, and displacement direction. Based on the point cloud data of other obstacles within the preset distance in the current frame point cloud data, obtain the lane information and roadside distance of the other obstacles, and determine the scene information of the other obstacles based on the lane information and roadside distance of the other obstacles.

3. The method according to claim 1, characterized in that, After obtaining the category information and tracking information of the target obstacle based on the point cloud data of the target obstacle, and before inputting the category information, tracking information, and attribute information of other obstacles within the preset distance into the obstacle classification model to obtain the optimized category information of the target obstacle, the method further includes: The tracking sequence of the target obstacle is determined based on the tracking information of the target obstacle; After inputting the category information of the target obstacle, the tracking information of the target obstacle, and the attribute information of other obstacles within the preset distance into the obstacle classification model to obtain the optimized category information of the target obstacle, the method further includes: The tracking sequence of the target obstacle is updated based on the optimized category information of the target obstacle.

4. The method of claim 1, wherein, After inputting the category information of the target obstacle, the tracking information of the target obstacle, and the attribute information of other obstacles within the preset distance into the obstacle classification model to obtain the optimized category information of the target obstacle, the method further includes: The point cloud constraints corresponding to the target obstacle are determined based on the optimization category information and the tracking information of the target obstacle; wherein, the point cloud constraints are used to determine the position of the target obstacle.

5. A method for training an obstacle classification model, the method comprising: The method includes: Acquire at least one frame of point cloud data collected by the lidar; Point cloud data of obstacles are detected based on at least one frame of point cloud data; wherein, the obstacles include the target obstacle and other obstacles within a preset distance; Based on the point cloud data of the target obstacle, obtain the category information of the target obstacle, the tracking information of the target obstacle, and the scene information of the target obstacle; The attribute information and tracking information of other obstacles within the preset distance are obtained based on the point cloud data of other obstacles within the preset distance; wherein, the attribute information includes the motion information and scene information of other obstacles within the preset distance; The obstacle classification model is trained using the category information, tracking information, and scene information of the target obstacle, the attribute information and tracking information of other obstacles, and the manually labeled optimized category information as training samples.

6. An obstacle classification device characterized by, include: The first acquisition module is used to acquire the current frame point cloud data collected by the lidar; The detection module is used to detect the point cloud data of obstacles based on the current frame point cloud data; wherein, the obstacles include the target obstacle and other obstacles within a preset distance; The second acquisition module is used to acquire the category information of the target obstacle and the tracking information of the target obstacle based on the point cloud data of the target obstacle; The third acquisition module is used to acquire attribute information of other obstacles within the preset distance based on point cloud data of other obstacles within the preset distance; wherein, the attribute information includes motion information of other obstacles within the preset distance and scene information; The optimization module is used to input the category information of the target obstacle, the tracking information of the target obstacle, and the attribute information of other obstacles within the preset distance into the obstacle classification model to obtain the optimized category information of the target obstacle; The second acquisition module includes: a processing unit, used to process the point cloud data of the target obstacle using a pose estimation algorithm to obtain the category information of the target obstacle; and an acquisition unit, used to acquire the tracking information of the target obstacle based on at least two frames of point cloud data of the target obstacle. 7.A device for training an obstacle classification model, comprising: include: The first acquisition module is used to acquire at least one frame of point cloud data collected by the lidar; The detection module is used to detect the point cloud data of obstacles based on the at least one frame of point cloud data; wherein the obstacles include the target obstacle and other obstacles within a preset distance; The second acquisition module is used to acquire the category information of the target obstacle, the tracking information of the target obstacle, and the scene information of the target obstacle based on the point cloud data of the target obstacle; The third acquisition module is used to acquire attribute information and tracking information of other obstacles within the preset distance based on point cloud data of other obstacles within the preset distance; wherein, the attribute information includes motion information and scene information of other obstacles within the preset distance; The training module is configured to train the obstacle classification model by taking the category information, tracking information and scene information of the target obstacle, the attribute information and tracking information of the other obstacles, and the manually labeled optimization category label information as training samples.

8. A computer storage medium, characterized in that The computer storage medium stores a plurality of instructions, which are adapted to be loaded and executed by the processor to perform the method steps of any one of claims 1-5.

9. An electronic device, comprising: Comprise: A memory and a processor; wherein the memory stores a computer program, and the computer program is adapted to be loaded and executed by the processor to perform the method steps of any one of claims 1-5.

Images