Object tracking method, device, equipment and storage medium

Abstract

The present application provides an object tracking method, device, equipment and storage medium, which relates to the field of multi-target object tracking. The method includes: acquiring laser radar data; the laser radar data includes a first laser point set and a second laser point set; when the first laser point set and the second laser point set meet preset conditions, the first laser point set and the second laser point set are merged into a target laser point set; the target laser point set and the target object have a corresponding relationship; according to the target laser point set and a preset Kalman filter, the first feature of the target object is predicted; the first feature of the target object includes one or more of the shape, position and speed of the target object. The method is applicable to the process of tracking an object using a laser radar, and is used to solve the problem that an object cannot be tracked based on fragmented laser radar data.

Description

Technical Field

[0001] The present application relates to the field of multi-target object tracking, and in particular to an object tracking method, device, equipment and storage medium. Background Art

[0002] A lidar system is a radar system that uses lasers to detect characteristic quantities such as the shape, position, and speed of a target object.

[0003] When performing detection, the laser radar can first emit lasers to the surroundings, receive light beams reflected by surrounding objects, process the reflected light beams, obtain laser point clouds including a plurality of laser point sets, and obtain characteristic quantities such as the shape, position, and speed of the target object based on the laser point clouds. For example, a three-dimensional digital model of the surrounding objects is calculated based on the laser point cloud, and the shape, position, and speed of the target object in the surrounding objects are predicted based on the three-dimensional digital model of the surrounding objects.

[0004] However, when the surface of a target object is smooth, the laser point set of the target object in the laser point cloud usually appears as multiple broken fragments, and the shape, position, speed and other characteristics of the target object cannot be accurately predicted based on the broken fragments. Summary of the invention

[0005] The present application provides an object tracking method, device, equipment and storage medium, which can fuse a set of laser points that meet preset conditions to accurately predict the exact features of the object.

[0006] In a first aspect, the present application provides an object tracking method, the method comprising: acquiring laser radar data; the laser radar data comprising a first laser point set and a second laser point set; when the first laser point set and the second laser point set meet a preset condition, the first laser point set and the second laser point set are merged into a target laser point set; the target laser point set and the target object have a corresponding relationship; according to the target laser point set and a preset Kalman filter, a first feature of the target object is predicted. The first feature comprises one or more of the features of the shape, position, and speed of the target object.

[0007] In a possible implementation, when the first laser point set and the second laser point set meet a preset condition, the first laser point set and the second laser point set are merged into a target laser point set, including: respectively calculating an average speed of the first laser point set and an average speed of the second laser point set; when a difference between the average speed of the first laser point set and the average speed of the second laser point set is less than a preset speed threshold, the first laser point set and the second laser point set are merged into the target laser point set.

[0008] In another possible implementation, when the first laser point set and the second laser point set meet a preset condition, the first laser point set and the second laser point set are merged into a target laser point set, including: respectively calculating average points of the first laser point set and the second laser point set; when the distance between the average point of the first laser point set and the average point of the second laser point set is less than a preset distance threshold, the first laser point set and the second laser point set are merged into the target laser point set.

[0009] In another possible implementation, the preset Kalman filters include multiple ones. Predicting the first feature of the target object according to the target laser point set and the preset Kalman filters includes: determining the target Kalman filter corresponding to the target laser point set from the multiple preset Kalman filters.

[0010] In another possible implementation, determining a target Kalman filter corresponding to a target laser point set from a plurality of preset Kalman filters comprises: calculating an average point of the target laser point set; inputting the average point of the target laser point set into each Kalman filter, and determining a candidate Kalman filter corresponding to the target laser point set by using an iterative nearest neighbor ICP algorithm; calculating a size of a bounding rectangle frame and feature point information of the target laser point set; performing ICP registration on the feature point information of the target laser point set and the feature point information of the laser point set stored in the candidate Kalman filter to obtain a registration residual; for each candidate Kalman filter, when a difference between a size of a bounding rectangle frame of the target laser point set and a size of a bounding rectangle frame of the laser point set stored in the candidate Kalman filter is less than a preset first threshold value, and a registration residual between the feature point information of the target laser point set and the feature point information of the laser point set stored in the Kalman filter is less than a preset second threshold value, determining the candidate Kalman filter as the target Kalman filter corresponding to the target laser point set; and predicting a first feature of the target object according to the target laser point set and the target Kalman filter.

[0011] In another possible implementation, for each candidate Kalman filter, when the difference between the size of the circumscribed rectangular frame of the target laser point set and the size of the circumscribed rectangular frame of the laser point set stored in the candidate Kalman filter is less than a preset first threshold, and the registration residual of the feature point information of the target laser point set and the laser point set stored in the Kalman filter is less than a preset second threshold, after determining that the candidate Kalman filter is the target Kalman filter corresponding to the target laser point set, the method further includes: updating the parameters in the target Kalman filter according to the target laser point set.

[0012] In yet another possible implementation, when all candidate Kalman filters are not determined as target Kalman filters, the method may further include: creating a target Kalman filter and initializing parameters according to the target laser point set.

[0013] In yet another possible implementation, for each candidate Kalman filter, when the number of times that the candidate Kalman filter is not determined as the target Kalman filter is greater than a preset third threshold, the candidate Kalman filter is deleted.

[0014] In another possible implementation, after acquiring the laser radar data, the method also includes: acquiring the resolution corresponding to the laser radar data and the intensity information of the laser points in the laser radar data; converting the laser radar data into coordinates in a vehicle body coordinate system according to the resolution corresponding to the laser radar data; calculating the relative distance difference between the laser points according to the coordinates in the vehicle body coordinate system; and removing flying spots in the laser radar data according to the relative distance difference between the laser points and the intensity information of the laser points.

[0015] In another possible implementation, the laser radar data obtained includes multiple frames; after obtaining the laser radar data, the method further includes: converting each frame in the multiple frames of laser radar data into coordinates in a world coordinate system; obtaining a rasterized map; the rasterized map includes multiple grids; when the number of times that the coordinates in the world coordinate system in the multiple frames of laser radar data fall into the first grid in the rasterized map is greater than a preset fourth threshold, deleting the coordinates from the laser radar data; the first grid is any one of the multiple grids included in the rasterized map. The object tracking method provided in the present application can obtain laser radar data. When the first laser point set and the second laser point set in the laser radar data meet the preset conditions, the first laser point set and the second laser point set are fused into a target laser point set. Compared with the first laser point set or the second laser point set, the fused target laser point set has a more complete shape and a higher probability of accurately representing the target object. Tracking the target object according to the laser point set with a more complete shape can more accurately obtain the characteristics of the shape, position, and speed of the target object.

[0016] In a second aspect, the present application provides an object tracking device, which includes various modules used in the method described in the first aspect or any possible implementation method of the first aspect.

[0017] In a third aspect, the present application provides a computer program product. When the computer program product is run on a computer, it enables the computer to execute the steps of the related method described in the first aspect above to implement the method described in the first aspect above.

[0018] In a fourth aspect, the present application provides an electronic device, comprising: a processor and a memory; the memory stores instructions executable by the processor; when the processor is configured to execute the instructions, the electronic device implements the method described in the first aspect above.

[0019] In a fifth aspect, the present application provides a computer-readable storage medium, which includes: computer software instructions; when the computer software instructions are executed in an electronic device, the electronic device implements the method described in the first aspect above.

[0020] The beneficial effects of the second to fifth aspects mentioned above can be referred to the first aspect and will not be elaborated on again. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A schematic diagram of an implementation environment involved in the object tracking method provided in an embodiment of the present application;

[0022] Figure 2 A flowchart of an object tracking method provided in an embodiment of the present application;

[0023] Figure 3 Another schematic diagram of a flow chart of an object tracking method provided in an embodiment of the present application;

[0024] Figure 4 A schematic diagram of another flow chart of the object tracking method provided in the embodiment of the present application;

[0025] Figure 5 A schematic diagram of another flow chart of the object tracking method provided in the embodiment of the present application;

[0026] Figure 6 A schematic diagram of another process of object tracking provided in an embodiment of the present application;

[0027] Figure 7 A schematic diagram of the composition of an object tracking device provided in an embodiment of the present application;

[0028] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] In the following, the terms "first", "second", "third", etc. are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", or "third", etc. may explicitly or implicitly include one or more of the features.

[0030] First, some concepts involved in the embodiments of the present application are introduced.

[0031] Laser spot: The laser spot formed by the laser emitted by the lidar system on the surrounding objects.

[0032] Laser point set: A laser point set may include multiple laser points. Each laser point set corresponds to a surrounding object.

[0033] Laser point cloud: A frame of laser radar data formed by all the light beams reflected by surrounding objects received by the laser radar system at a certain moment. A frame of laser point cloud can include multiple laser point sets.

[0034] World Coordinate System: The world coordinate system is an absolute coordinate system that can be selected arbitrarily and remains unchanged and unique once specified.

[0035] Vehicle body coordinate system: The vehicle body coordinate system is a relative coordinate system, for example, a coordinate system with the center of the mobile platform as the origin.

[0036] Rasterized map: A rasterized map is a map that is a result of rasterizing the world map, including multiple grids.

[0037] The object tracking method provided in the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0038] Figure 1 Schematic diagram of the implementation environment involved in the object tracking method provided in the embodiment of the present application. Figure 1 As shown, the implementation environment may include: a laser radar 10 and a computing and processing device 20. The laser radar 10 may be connected to the computing and processing device 20 via a wired network or a wireless network.

[0039] The laser radar 10 may be a single-line laser radar, a multi-line laser radar, a three-dimensional omnidirectional laser radar, etc. The embodiment of the present application does not limit the specific type of the laser radar.

[0040] The laser radar 10 can be installed on a mobile platform, for example, a tracked robot, a drone, a car, or an airplane, etc. The embodiment of the present application does not limit the specific type of the mobile platform.

[0041] The laser radar 10 can emit lasers to the surroundings, receive light beams reflected by surrounding objects, and generate laser radar data.

[0042] The computing and processing device 20 may be a device with computing functions such as a computer or a server, wherein the server may be a single server, or a server cluster consisting of multiple servers. In some implementations, the server cluster may also be a distributed cluster. The embodiments of the present application do not limit the specific form of the computing and processing device 20.

[0043] The computing and processing device 20 can be installed on the same mobile platform as the laser radar 10, or can be installed in a fixed place, such as a machine room.

[0044] The computing and processing device 20 can receive the laser radar data (also called laser point cloud) sent by the laser radar 10 through a wired network or a wireless network. The laser radar data may include multiple laser point sets. When a certain laser point set and another laser point set meet a preset condition, the computing and processing device 20 can merge the two laser point sets into a target laser point set, which is a set of all laser points formed on the target object by the laser emitted by the laser radar 10. The target laser point set and the target object have a corresponding relationship, for example, the target laser point set and the target object have a one-to-one correspondence. The target object may be any one or more of the surrounding objects.

[0045] The computing and processing device 20 can also predict the shape, position, speed and other characteristics of the target object based on the target laser point set and a preset Kalman filter.

[0046] It should be noted that the object tracking method provided in the embodiment of the present application can be applied to the above-mentioned computing and processing device 20, and can also be applied to the above-mentioned laser radar 10 and computing and processing device 20, and the laser radar 10 and computing and processing device 20 can be collectively referred to as electronic devices. The executor of the object tracking method provided in the embodiment of the present application is an object tracking device, which can be the above-mentioned electronic device; or, the device can be an application (application, APP) installed in the above-mentioned electronic device that provides an object tracking function; or, the device can be a central processing unit (CPU) in the above-mentioned electronic device; or, the device can be a functional module in the above-mentioned electronic device for executing the object tracking method. Taking an electronic device as an example, the object tracking method provided in the embodiment of the present application is described in detail below.

[0047] Figure 2 The following is a flow chart of an object tracking method provided in an embodiment of the present application. Figure 2 As shown, the method may include S101 to S103.

[0048] S101. The electronic device obtains laser radar data.

[0049] The laser radar data (also referred to as laser point cloud) may include multiple laser point sets, and each laser point set may include multiple laser points.

[0050] S102: When the first laser point set and the second laser point set meet a preset condition, the electronic device merges the first laser point set and the second laser point set into a target laser point set.

[0051] The first laser point set is any one of the multiple laser point sets in the laser radar data. The second laser point set is any one of the multiple laser point sets in the laser radar data except the first laser point set. The target laser point set and the target object tracked by the electronic device have a corresponding relationship. For example, the target laser point set and the target object may correspond one to one. The electronic device fusing the first laser point set and the second laser point set into the target laser point set means that the electronic device merges the first laser point set and the second laser point set into the target laser point set.

[0052] It should be noted that when the first laser point set and the second laser point set do not meet the preset conditions, the electronic device may not process the first laser point set and the second laser point set. That is, after the electronic device obtains the laser radar data, it can determine whether the multiple laser point sets in the laser radar data meet the preset conditions. If they meet, they are merged into the target laser point set; if not, they are not processed.

[0053] In a possible implementation, the preset condition may include: the difference between the average speeds of the two laser point sets is less than a preset speed threshold. That is, when the first laser point set and the second laser point set meet the preset condition, the electronic device merges the first laser point set and the second laser point set into a target laser point set, which may include: the electronic device calculates the average speed of the first laser point set and the average speed of the second laser point set respectively; when the difference between the average speed of the first laser point set and the average speed of the second laser point set is less than the preset speed threshold, the electronic device merges the first laser point set and the second laser point set into the target laser point set.

[0054] For example, the laser radar data may include at least two frames of data, and each frame of the at least two frames of laser radar data may include a first laser point set and a second laser point set, respectively. According to each laser point in the first laser point set in the at least two frames of laser radar data, the displacement of each laser point in the first laser point set in the at least two frames of laser radar data is determined. According to the displacement of each laser point in the first laser point set in the at least two frames of laser radar data and the preset frame rate of the laser radar data, the average speed of the laser points in the first laser point set is calculated. According to each laser point in the second laser point set in the at least two frames of laser radar data, the displacement of each laser point in the second laser point set in the at least two frames of laser radar data is determined. According to the displacement of each laser point in the second laser point set in different frames of laser radar data and the preset frame rate of the laser radar data, the average speed of the laser points in the second laser point set is calculated.

[0055] In another possible implementation, the preset condition may include: the distance between the average points of the two laser point sets is less than a preset distance threshold. That is, when the first laser point set and the second laser point set meet the preset condition, the electronic device merges the first laser point set and the second laser point set into a target laser point set, which may include: respectively calculating the average points of the first laser point set and the second laser point set; when the distance between the average points of the first laser point set and the second laser point set is less than the preset distance threshold, the electronic device merges the first laser point set and the second laser point set into the target laser point set.

[0056] The average point of the first laser point set and the second laser point set can be obtained according to the average value of the coordinates of the laser points in the first laser point set and the second laser point set.

[0057] For example, taking the first laser point set as an example, assuming that the first laser point set includes laser point 1 and laser point 2, the coordinates of laser point 1 are (x1, y1), and the coordinates of laser point 2 are (x2, y2), then the coordinates of the average point of the first laser point set can be obtained according to the average value of the coordinates of laser point 1 and laser point 2, that is, the coordinates of the average point of the first laser point set are

[0058] Optionally, when the distance between the average point of the first laser point set and the average point of the second laser point set is less than a preset distance threshold, the electronic device merges the first laser point set and the second laser point set into a target laser point set, which may include: clustering the average points of the first laser point set and the second laser point set using a clustering algorithm; when the average points of the first laser point set and the second laser point set are clustered into one category, the electronic device merges the first laser point set and the second laser point set into the target laser point set.

[0059] Optionally, the clustering algorithm may be a density-based spatial clustering of applications with noise (DBSCAN) algorithm.

[0060] S103: The electronic device predicts a first feature of the target object according to the target laser point set and a preset Kalman filter.

[0061] The Kalman filter may store state variables, the size of the circumscribed rectangular frame of the laser point set, and feature point information of the laser point set, etc. The first feature of the target object may include one or more of the features of the shape, position, and speed of the target object.

[0062] For example, the state variable may include the coordinates of the target object and the speed of the target object, etc. The size of the circumscribed rectangular frame of the laser point set may include the maximum value of the length and width of the circumscribed rectangular frame of the laser point set. The length of the circumscribed rectangular frame of the laser point set may be determined by subtracting the minimum value from the maximum value of the horizontal coordinate of the laser point set. The width of the circumscribed rectangular frame of the laser point set may be determined by subtracting the minimum value from the maximum value of the vertical coordinate of the laser point set. The feature point information of the laser point set may include the coordinates of the laser points on the contour line of the laser point set and the laser points of the key feature part, etc.

[0063] In a possible implementation, there is only one preset Kalman filter. The electronic device determines the first feature of the target object according to the target laser point set and the preset Kalman filter, which may include: the electronic device inputs the target laser point set into the preset Kalman filter to predict the first feature of the target object.

[0064] In another possible implementation, the preset Kalman filters may include multiple ones, and the electronic device may match the target laser point set with a target Kalman filter corresponding to the target laser point set from the multiple Kalman filters. Figure 3 Another flow chart of the object tracking method provided in the embodiment of the present application. Figure 3 As shown, S103 may specifically include S201 to S206.

[0065] S201. The electronic device calculates the average point of the target laser point set.

[0066] The method for calculating the average point of the target laser point set may refer to the method for calculating the average point of the first laser point set or the method for calculating the average point of the second laser point set, which will not be described in detail here.

[0067] S202, the electronic device inputs the average point of the target laser point set into each Kalman filter, and uses an iterative closest point (ICP) algorithm to determine a candidate Kalman filter corresponding to the target laser point set from multiple preset Kalman filters.

[0068] Among them, the candidate Kalman filters may include one or more.

[0069] S203, the electronic device calculates the size of the circumscribed rectangular frame and feature point information of the target laser point set.

[0070] The size of the bounding rectangle of the target laser point set can refer to the size of the bounding rectangle of the laser point set stored in the above-mentioned Kalman filter, and will not be repeated here. As described above, the feature point information of the laser point set may include the coordinates of the laser points on the contour line of the laser point set and the laser points of the key feature part. After the electronic device fuses the first laser point set and the second laser point set into the target laser point set, it can extract the coordinates of the laser points on the contour line of the target laser point set and the laser points of the key feature part as the feature point information of the target laser point set. The extraction process can also be called calculation.

[0071] S204: The electronic device performs ICP registration on the feature point information of the target laser point set and the feature point information of the laser point set stored in the candidate Kalman filter to obtain a registration residual.

[0072] It should be noted that S204 can be executed after S203, or before S203, or S204 and S203 can be executed simultaneously. The embodiment of the present application does not limit the timing relationship between the electronic device executing S203 and S204.

[0073] S205. For each candidate Kalman filter, when the difference between the size of the circumscribed rectangular frame of the target laser point set and the size of the circumscribed rectangular frame of the laser point set stored in the candidate Kalman filter is less than a preset first threshold, and the registration residual between the feature point information of the target laser point set and the feature point information of the laser point set stored in the candidate Kalman filter is less than a preset second threshold, the candidate Kalman filter is determined to be the target Kalman filter corresponding to the target laser point set.

[0074] The first threshold and the second threshold can be pre-set by the administrator in the electronic device. For example, taking the first threshold as an example, the first threshold is 1 mm, or 1 cm, or 5 cm, etc. The embodiment of the present application does not limit the specific values ​​of the first threshold and the second threshold.

[0075] It should be noted that, for each candidate Kalman filter, when the difference between the size of the bounding rectangle of the target laser point set and the size of the bounding rectangle of the laser point set stored in the candidate Kalman filter is greater than or equal to a preset first threshold, and / or the registration residual of the target laser point set and the feature point information of the laser point set stored in the Kalman filter is greater than or equal to a preset second threshold, the candidate Kalman filter cannot be used as the target Kalman filter corresponding to the target laser point set. That is, after calculating the size of the bounding rectangle of the target laser point set and the registration residual of the target laser point set and the feature point information of the laser point set stored in the Kalman filter, for each candidate Kalman filter, the electronic device can determine whether the difference between the size of the bounding rectangle of the target laser point set and the size of the bounding rectangle of the laser point set stored in the candidate Kalman filter is less than the preset first threshold, and can also determine whether the registration residual of the target laser point set and the feature point information of the laser point set stored in the Kalman filter is less than the preset second threshold.

[0076] It should also be noted that when there are multiple candidate Kalman filters that meet the screening conditions of the target Kalman filter, the candidate Kalman filter that stores the smallest difference in the size of the circumscribed rectangular frame of the target laser point set, or the candidate Kalman filter with the smallest matching residual, among the multiple candidate Kalman filters that meet the screening conditions can be selected as the target Kalman filter. The embodiment of the present application is not limited to this.

[0077] S206: The electronic device predicts a first feature of the target object based on the target laser point set and the target Kalman filter.

[0078] In the object tracking method provided in the embodiment of the present application, the electronic device can obtain laser radar data. When the first laser point set and the second laser point set in the laser radar data meet the preset conditions, the electronic device can fuse the first laser point set and the second laser point set into a target laser point set. Compared with the first laser point set or the second laser point set, the shape of the fused target laser point set is more complete, and the probability of accurately representing the target object is higher. By tracking the target object according to the laser point set with a more complete shape, the characteristics of the shape, position, and speed of the target object can be more accurately obtained.

[0079] In some possible embodiments, after determining the target Kalman filter from the plurality of candidate Kalman filters, the parameters in the target Kalman filter may be updated according to the target laser point set. That is, after S204, the method may further include: the electronic device updates the parameters in the target Kalman filter according to the target laser point set.

[0080] Exemplarily, the electronic device may update the parameters in the target Kalman filter according to the target laser point set by referring to the following formulas (1) to (5).

[0081]

[0082] In formula (1), represents the predicted value of the state vector (including the coordinates of the target object and the speed of the target object, etc.) obtained at time k. The content of the state vector is consistent with the state variables of the object stored in the above Kalman filter. A represents the known system matrix. Represents the optimal estimate of the state vector at time k-1.

[0083]

[0084] In formula (2), P represents the covariance matrix of the predicted state vector at time k. k-1 represents the covariance matrix of the best estimated state vector at time k-1. Q represents the system variance matrix.

[0085]

[0086] In formula (3), K k represents the Kalman gain matrix at time k. H represents the known observation matrix, which is the matrix that converts the observation quantity into the state vector. R represents the covariance matrix of the measurement system.

[0087]

[0088] In formula (4), represents the optimal estimate of the state vector at time k. k represents the observation vector.

[0089]

[0090] In formula (5), P k Represents the covariance matrix of the best estimated state vector at time k.

[0091] In some other possible embodiments, when all candidate Kalman filters are not used as target Kalman filters by the electronic device, that is, when the target Kalman filter determined from multiple candidate Kalman filters is empty, the method may also include: the electronic device creates a target Kalman filter and initializes parameters based on the target laser point set, and then predicts the first feature of the target object based on the created target Kalman filter.

[0092] For example, as described above, the Kalman filter may store state variables, the size of the circumscribed rectangular frame of the laser point set, and feature point information of the laser point set. The state variables may include the coordinates of the target object and the speed of the target object. The initialization parameters may include: taking the coordinates of the average point of the target laser point set as the coordinates of the target object; initializing the speed of the target object to 0; taking the size of the circumscribed rectangular frame of the target laser point set as the size of the circumscribed rectangular frame of the laser point set stored in the target Kalman filter. Acquire the feature point information of the target laser point set, and use the feature point information as the feature point information of the laser point set stored in the target Kalman filter.

[0093] In the object tracking method provided in the embodiment of the present application, when the target laser point set cannot be matched to the corresponding target Kalman filter, the electronic device creates a new target Kalman filter based on the target laser point set, and predicts the first feature of the target object based on the newly created target Kalman filter, thereby improving the flexibility of object tracking.

[0094] In some other possible embodiments, for each candidate Kalman filter, when the number of times that the candidate Kalman filter is not determined as the target Kalman filter is greater than a preset third threshold, it indicates that the candidate Kalman filter is likely to fail and is no longer suitable for continuing prediction. Therefore, the electronic device can delete the candidate Kalman filter from the preset Kalman filters.

[0095] The third threshold value may be preset in the electronic device by the administrator. For example, the third threshold value is 10 times, 20 times, or 50 times, etc. The embodiment of the present application does not limit the specific value of the third threshold value.

[0096] For example, when a candidate Kalman filter is determined as the target Kalman filter, the electronic device can update the number of unmatched times of other candidate Kalman filters except the candidate Kalman filter; when the number of unmatched times is greater than a preset third threshold, the electronic device can delete the candidate Kalman filters whose number of unmatched times is greater than the preset third threshold.

[0097] In the object tracking method provided in the embodiment of the present application, when a candidate Kalman filter has not been predicted as a target Kalman filter for a long time, the candidate Kalman filter is likely to fail. Deleting the failed candidate Kalman filter can improve the effectiveness of the Kalman filter in the electronic device.

[0098] In some embodiments, after acquiring the LiDAR data, the electronic device may also pre-process the LiDAR data to remove laser points (also referred to as flying points) that have no actual corresponding target objects. Figure 4Another flow chart of the object tracking method provided in the embodiment of the present application. Figure 4 As shown, after S101, the method may further include S301 to S304.

[0099] S301. The electronic device obtains the resolution corresponding to the laser radar data and the intensity information of the laser point in the laser radar data.

[0100] The intensity information of the laser point may include the light intensity of the reflected light beam received by the laser radar 10 .

[0101] S302. The electronic device converts the laser radar data into coordinates in the vehicle body coordinate system according to the resolution corresponding to the laser radar data.

[0102] As described above, the electronic device may include a laser radar 10, and the laser radar 10 may be installed on a mobile platform. The vehicle body coordinate system may include a coordinate system centered on the mobile platform.

[0103] S303: The electronic device calculates the relative distance difference between the laser points according to the coordinates in the vehicle body coordinate system.

[0104] S304: The electronic device removes flying spots in the laser radar data according to the relative distance difference between the laser points and the intensity information of the laser points.

[0105] Among them, a flying spot refers to a laser point in the lidar data that has no actual corresponding target object, that is, a laser point that is not corresponding to the target object.

[0106] In other embodiments, after acquiring multiple frames of lidar data, the electronic device may also remove laser points corresponding to stationary objects. Figure 5 Another flow chart of the object tracking method provided in the embodiment of the present application. Figure 5 As shown, after S101, the method may further include S401 to S403.

[0107] S401, the electronic device converts each frame of the multiple frames of laser radar data into coordinates in the world coordinate system.

[0108] S402: The electronic device obtains a rasterized map.

[0109] The rasterized map may include multiple grids.

[0110] For example, the electronic device may obtain a world map and then perform rasterization processing on the world map.

[0111] S403: When the number of times that coordinates in the world coordinate system in the multi-frame laser radar data fall into the first grid in the rasterized map is greater than a preset fourth threshold, the electronic device deletes the coordinates from the laser radar data.

[0112] The first grid is any one of the multiple grids included in the grid map. The fourth threshold can be preset in the electronic device by the administrator. For example, the fourth threshold is 5 times, 10 times, or 20 times. The embodiment of the present application does not limit the specific value of the fourth threshold.

[0113] In the object tracking method provided in the embodiment of the present application, after acquiring multiple frames of laser radar data, the electronic device can also convert each frame of the multiple frames of laser radar data into coordinates in the world coordinate system and obtain a rasterized map. When the coordinates in the world coordinate system of the laser radar data fall into a certain grid in the rasterized map too many times (greater than the preset fourth threshold), the laser radar data at the coordinates can be considered to be the laser radar data of a stationary object, and the laser radar data of the stationary object can be deleted to reduce interference in object tracking.

[0114] In some other embodiments, after acquiring the laser radar data, the electronic device may also perform clustering processing on the laser points in the laser radar data to obtain multiple laser point sets.

[0115] For example, the laser radar data may include a first laser point and a second laser point, the first laser point belongs to a first laser point set. When the distance between the second laser point and the first laser point is less than a preset fifth threshold, the electronic device may add the second laser point to the first laser point set where the first laser point is located.

[0116] The fifth threshold value may be preset in the electronic device by the administrator. For example, the fifth threshold value is 5 cm, 8 cm, or 10 cm, etc. The embodiment of the present application does not limit the actual value of the fifth threshold value.

[0117] It should be noted that when the distance between the second laser point and the first laser point is greater than the preset fifth threshold, the electronic device can add the second laser point to the laser point set outside the first laser point set. That is, after acquiring the laser radar data, the electronic device can determine whether the distance between the first laser point and the second laser point in the laser radar data is less than the preset fifth threshold. When the distance between the second laser point and the first laser point is less than the preset fifth threshold, the electronic device can add the second laser point to the first laser point set where the first laser point is located. When the distance between the second laser point and the first laser point is greater than or equal to the preset fifth threshold, the electronic device can add the second laser point and the first laser point to different laser point sets respectively. Exemplarily, taking the laser radar data including laser point 1 and laser point 2, and laser point 1 belonging to laser point set 1 as an example, assuming that the distance between laser point 2 and laser point 1 is 1 cm, and the preset fifth threshold is 5 cm. Then the electronic device can add laser point 2 to the laser point set 1 where laser point 1 is located according to the distance between laser point 2 and laser point 1 of 1 cm being less than the preset 5 cm.

[0118] Optionally, Figure 6 Another flow chart of object tracking provided in the embodiment of the present application. Figure 6 As shown, the method may include S501 to S507.

[0119] S501. The electronic device obtains laser radar data.

[0120] S501 may refer to the above-mentioned S101, which will not be described in detail here.

[0121] S502: The electronic device pre-processes the laser radar data to remove flying spots in the laser radar data.

[0122] S502 may refer to the above-mentioned S301 to S304, which will not be described in detail here.

[0123] S503: The electronic device deletes the laser point corresponding to the stationary object in the laser radar data.

[0124] S503 may refer to the above S401 to S403, which will not be described in detail here.

[0125] S504: The electronic device fuses laser points with similar distances in the laser radar data into a laser point set.

[0126] S504 may refer to the above description of adding the second laser point to the first laser point set where the first laser point is located, and will not be described in detail here.

[0127] S505: When the first laser point set and the second laser point set in the laser radar data meet a preset condition, the electronic device merges the first laser point set and the second laser point set into a target laser point set.

[0128] S505 may refer to the above-mentioned S102, which will not be described in detail here.

[0129] S506: The electronic device selects a target Kalman filter corresponding to the target laser point.

[0130] S506 can refer to the above S201 to S205, which will not be repeated here.

[0131] S507: The electronic device predicts a first feature of the target object according to the target laser point set and a preset Kalman filter.

[0132] S507 may refer to the above-mentioned S103, which will not be described in detail here.

[0133] In an exemplary embodiment, the present application also provides an object tracking device. Figure 7 This is a schematic diagram of the composition of the object tracking device provided in the embodiment of the present application. Figure 7 As shown, the object tracking device may include: an acquisition module 701 and a processing module 702, and the acquisition module 701 and the processing module 702 are connected. The acquisition module 701 is used to acquire laser radar data; the laser radar data includes a first laser point set and a second laser point set. The processing module 702 is used to merge the first laser point set and the second laser point set into a target laser point set when the first laser point set and the second laser point set meet a preset condition; the target laser point set and the target object have a corresponding relationship; according to the target laser point set and the preset Kalman filter, the first feature of the target object is predicted. Among them, the first feature may include one or more of the features of the shape, position, and speed of the target object.

[0134] In some possible embodiments, the processing module 702 is specifically configured to respectively calculate an average speed of the first laser point set and an average speed of the second laser point set; when a difference between the average speed of the first laser point set and the average speed of the second laser point set is less than a preset speed threshold, the first laser point set and the second laser point set are merged into a target laser point set.

[0135] In some other possible embodiments, the processing module 702 is specifically configured to respectively calculate the average point of the first laser point set and the average point of the second laser point set; when the distance between the average point of the first laser point set and the average point of the second laser point set is less than a preset distance threshold, the first laser point set and the second laser point set are merged into a target laser point set.

[0136] In some other possible embodiments, the preset Kalman filters include a plurality of Kalman filters. The processing module 702 is specifically configured to determine a target Kalman filter corresponding to the target laser point set from the plurality of preset Kalman filters.

[0137] In some other possible embodiments, the processing module 702 is specifically used to calculate the average point of the target laser point set; input the average point of the target laser point set into each Kalman filter, and determine the candidate Kalman filter corresponding to the target laser point set by using the iterative nearest neighbor point ICP algorithm; calculate the size of the circumscribed rectangular frame and the feature point information of the target laser point set; perform ICP registration on the feature point information of the target laser point set and the feature point information of the laser point set stored in the candidate Kalman filter to obtain the registration residual; for each candidate Kalman filter, when the difference between the size of the circumscribed rectangular frame of the target laser point set and the size of the circumscribed rectangular frame of the laser point set stored in the candidate Kalman filter is less than a preset first threshold, and the registration residual between the feature point information of the target laser point set and the feature point information of the laser point set stored in the Kalman filter is less than a preset second threshold, determine the candidate Kalman filter as the target Kalman filter corresponding to the target laser point set; and predict the first feature of the target object according to the target laser point set and the target Kalman filter.

[0138] In some other possible embodiments, the processing module 702 is further configured to update parameters in the target Kalman filter according to the target laser point set.

[0139] In some other possible embodiments, the processing module 702 is further configured to create a target Kalman filter and initialize parameters according to the target laser point set when all candidate Kalman filters are not determined as the target Kalman filter.

[0140] In some other possible embodiments, the processing module 702 is further configured to delete each candidate Kalman filter when the number of times that the candidate Kalman filter is not determined as the target Kalman filter is greater than a preset third threshold.

[0141] In some other possible embodiments, the acquisition module 701 is further used to obtain the resolution corresponding to the laser radar data and the intensity information of the laser points in the laser radar data. The processing module 702 is further used to convert the laser radar data into coordinates in the vehicle body coordinate system according to the resolution corresponding to the laser radar data; calculate the relative distance difference between the laser points according to the coordinates in the vehicle body coordinate system; and remove the flying spots in the laser radar data according to the relative distance difference between the laser points and the intensity information of the laser points.

[0142] In some other possible embodiments, the laser radar data acquired by the acquisition module 701 includes multiple frames. The processing module 702 is further used to convert each frame in the multiple frames of laser radar data into coordinates in a world coordinate system; obtain a rasterized map; the rasterized map includes multiple grids; when the number of times that the coordinates in the world coordinate system in the multiple frames of laser radar data fall into the first grid in the rasterized map is greater than a preset fourth threshold, delete the coordinates from the laser radar data; the first grid is any one of the multiple grids included in the rasterized map.

[0143] In an exemplary embodiment, the embodiment of the present application further provides a computer program product, which, when executed on a computer, enables the computer to execute the above-mentioned related method steps to implement the method described in the above-mentioned method embodiment.

[0144] In an exemplary embodiment, the present application also provides an electronic device. Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 8 As shown, the electronic device may include: a processor 801 and a memory 802; the memory 802 stores instructions executable by the processor 801; when the processor 801 is configured to execute the instructions, the electronic device implements the method described in the aforementioned method embodiment.

[0145] In an exemplary embodiment, the embodiment of the present application further provides a computer-readable storage medium on which computer program instructions are stored; when the computer program instructions are executed by a network device, the network device implements the method described in the aforementioned embodiment. The computer-readable storage medium may be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0146] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An object tracking method, characterized in that: The method comprises: Acquire laser radar data; the laser radar data includes a first laser point set and a second laser point set; When the first laser point set and the second laser point set meet a preset condition, the first laser point set and the second laser point set are merged into a target laser point set; the target laser point set and the target object have a corresponding relationship; Determine a target Kalman filter corresponding to the target laser point set from a plurality of preset Kalman filters; According to the target laser point set and the target Kalman filter corresponding to the target laser point set, a first feature of the target object is predicted; a difference between a size of a circumscribed rectangular frame of the target laser point set and a size of a circumscribed rectangular frame of the laser point set stored in the target Kalman filter is less than a preset first threshold, and a registration residual between feature point information of the target laser point set and feature point information of the laser point set stored in the target Kalman filter is less than a preset second threshold; and the first feature of the target object includes one or more of shape, position and velocity features of the target object.

2. The method according to claim 1, characterized in that When the first laser point set and the second laser point set meet a preset condition, merging the first laser point set and the second laser point set into a target laser point set comprises: respectively calculating an average speed of the first laser point set and an average speed of the second laser point set; When the difference between the average speed of the first laser point set and the average speed of the second laser point set is less than a preset speed threshold, the first laser point set and the second laser point set are merged into a target laser point set.

3. The method according to claim 1, characterized in that When the first laser point set and the second laser point set meet a preset condition, merging the first laser point set and the second laser point set into a target laser point set comprises: Calculating average points of the first laser point set and the second laser point set respectively; When the distance between the average point of the first laser point set and the average point of the second laser point set is less than a preset distance threshold, the first laser point set and the second laser point set are merged into a target laser point set.

4. The method according to claim 1, characterized in that: The method further comprises: Calculating an average point of the target laser point set; Inputting the average point of the target laser point set into each Kalman filter of a plurality of preset Kalman filters, and determining a candidate Kalman filter corresponding to the target laser point set by using an iterative nearest neighbor point ICP algorithm; Calculate the size of the circumscribed rectangular frame and feature point information of the target laser point set; Performing ICP registration on the feature point information of the target laser point set and the feature point information of the laser point set stored in the candidate Kalman filter to obtain a registration residual; For each candidate Kalman filter, when the difference between the size of the circumscribed rectangular frame of the target laser point set and the size of the circumscribed rectangular frame of the laser point set stored in the candidate Kalman filter is less than a preset first threshold, and the registration residual between the feature point information of the target laser point set and the feature point information of the laser point set stored in the candidate Kalman filter is less than a preset second threshold, the candidate Kalman filter is determined to be the target Kalman filter corresponding to the target laser point set.

5. The method according to any one of claims 1 to 4, characterized in that: After acquiring the laser radar data, the method further includes: Obtaining the resolution corresponding to the laser radar data and the intensity information of the laser points in the laser radar data; According to the resolution corresponding to the laser radar data, the laser radar data is converted into coordinates in a vehicle body coordinate system; Calculate the relative distance difference between the laser points according to the coordinates in the vehicle coordinate system; The flying spots in the laser radar data are removed according to the relative distance difference between the laser points and the intensity information of the laser points.

6. The method according to any one of claims 1 to 4, characterized in that: The laser radar data obtained includes multiple frames; after obtaining the laser radar data, the method further includes: Convert each frame of the multi-frame lidar data into coordinates in the world coordinate system; Acquire a rasterized map; the rasterized map includes a plurality of grids; When the number of times that coordinates in the world coordinate system in multiple frames of lidar data fall into the first grid in the rasterized map is greater than a preset fourth threshold, the coordinates are deleted from the lidar data; the first grid is any one of the multiple grids included in the rasterized map.

7. An object tracking device, characterized in that: The device comprises: an acquisition module and a processing module; The acquisition module is used to acquire laser radar data; the laser radar data includes a first laser point set and a second laser point set; The processing module is configured to merge the first laser point set and the second laser point set into a target laser point set when the first laser point set and the second laser point set meet a preset condition; the target laser point set and the target object have a corresponding relationship; determine a target Kalman filter corresponding to the target laser point set from a plurality of preset Kalman filters; predict a first feature of the target object according to the target laser point set and the target Kalman filter corresponding to the target laser point set; a difference between a size of an outer rectangular frame of the target laser point set and a size of an outer rectangular frame of the laser point set stored in the target Kalman filter is less than a preset first threshold, and a registration residual between feature point information of the target laser point set and feature point information of the laser point set stored in the target Kalman filter is less than a preset second threshold; and the first feature of the target object includes one or more of shape, position and speed features of the target object.

8. An electronic device, characterized in that: The electronic device comprises: a processor and a memory; The memory stores instructions executable by the processor; When the processor is configured to execute the instructions, the electronic device implements the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes: computer software instructions; When the computer software instructions are executed in an electronic device, the electronic device is enabled to implement the method according to any one of claims 1 to 6.

Images