Target tracking method and device, electronic equipment and storage medium

Abstract

The invention provides a target tracking method and device, electronic equipment and a storage medium, and relates to the technical field of the Internet of Things, and the method comprises the steps: carrying out the clustering processing of obtained multi-frame point cloud data, and obtaining a point cloud block of each frame; searching a point cloud block associated with the current point cloud block in the current frame in the point cloud blocks of at least one adjacent frame adjacent to the current frame; based on the current point cloud block in the current frame and the associated point cloud block, constructing a target point cloud feature for a current target; and identifying the category of the current target according to the target point cloud features to obtain an identification result, and tracking the target based on the identification result. According to the invention, the problem that a false target is easy to position and track in the prior art is effectively solved.

Description

Technical Field

[0001] The present application relates to the field of Internet of Things technology, and specifically, to a target tracking method, device, electronic device and storage medium. Background Art

[0002] Point cloud data is a data set composed of a large number of point clouds, each of which contains its coordinates in space and other attributes (such as color, normal vector, intensity, speed, etc.). In the field of the Internet of Things, point cloud data detected by sensor devices can be used to locate and track targets, so as to provide users with more intelligent services based on the located and tracked targets. For example, in a smart home scenario, if a person is located in the living room, the living room lights are turned on; or, if a person is tracked leaving the living room, the living room lights are turned off.

[0003] Only by accurately locating the target can we perform corresponding control operations based on the target's location information, such as turning on the living room light. Therefore, positioning and tracking are important foundations for providing intelligent services. However, when positioning and tracking targets through point cloud data, it is easy to locate and track false targets, and then mistakenly trigger the corresponding intelligent services (such as turning on the light), resulting in resource waste, miscontrol, etc.

[0004] As can be seen from the above, there is a problem in the prior art that false targets are easily located and tracked. Summary of the invention

[0005] The present application provides a target tracking method, device, electronic device and storage medium, which can solve the problem of easily locating and tracking false targets in the related art. The technical solution is as follows:

[0006] According to one aspect of the present application, a target tracking method includes: clustering the acquired multi-frame point cloud data respectively to obtain point cloud blocks of each frame; searching for point cloud blocks associated with a current point cloud block in the current frame in the point cloud blocks of at least one adjacent frame adjacent to the current frame; constructing a target point cloud feature for the current target based on the current point cloud block in the current frame and the associated point cloud blocks; identifying the category of the current target according to the target point cloud feature to obtain an identification result, and tracking the target based on the identification result.

[0007] According to one aspect of the present application, a target tracking device includes: a clustering module, which is used to perform clustering processing on the acquired multi-frame point cloud data respectively to obtain point cloud blocks of each frame; an association search module, which is used to search for point cloud blocks associated with a current point cloud block in the current frame in the point cloud blocks of at least one adjacent frame adjacent to the current frame; a feature construction module, which is used to construct a target point cloud feature for the current target based on the current point cloud block in the current frame and the associated point cloud blocks; and a recognition module, which is used to identify the category of the current target according to the target point cloud feature, obtain a recognition result, and perform target tracking based on the recognition result.

[0008] In an exemplary embodiment, the association search module includes: a center determination unit, used to determine the center of each of the point cloud blocks in each frame; a search unit, used to search for the point cloud blocks in at least one adjacent frame adjacent to the current frame, whose center distance from the current point cloud block in the current frame is within a specific range, as the point cloud block associated with the current point cloud block in the current frame.

[0009] In an exemplary embodiment, the feature construction module includes: a feature construction unit, used to construct a first feature corresponding to each point cloud block and a second feature between point cloud blocks according to the current point cloud block and the associated point cloud blocks in the current frame; and a point cloud feature acquisition unit, used to obtain the target point cloud feature for the current target according to the first feature and the second feature.

[0010] In an exemplary embodiment, the feature construction unit includes: a point cloud acquisition subunit, used to acquire a specific number of point clouds from the current point cloud block and the associated point cloud blocks in the current frame; and a feature calculation subunit, used to calculate the first features corresponding to each point cloud block and the second features between the point cloud blocks based on the acquired point clouds.

[0011] In an exemplary embodiment, the point cloud feature acquisition unit includes: a fusion subunit, configured to fuse the first feature and the second feature to obtain the target point cloud feature for the current target.

[0012] In an exemplary embodiment, the target tracking device also includes: a multiple recognition module, which is used to perform multiple recognitions on the category of the current target based on the acquired multi-frame point cloud data to obtain multiple recognition results for the current target; the recognition module includes: a number judgment unit, which is used to obtain the point cloud data of the current target if the number of times the multiple recognition results indicate that the category of the current target is a specific category exceeds a specific threshold; a trajectory creation unit, which is used to create a corresponding trajectory of the current target based on the point cloud data of the current target.

[0013] In an exemplary embodiment, the recognition is achieved by calling a target recognition model, which is a trained machine learning model that has the ability to recognize the category of the target; the recognition module includes: a prediction unit, which is used to input the target point cloud features into the target recognition model to predict the category of the current target and obtain the recognition result.

[0014] In an exemplary embodiment, the device also includes a model training module for training the target recognition model, and the model training module includes: a training data set acquisition unit, used to acquire a training data set; the training data set includes multiple training samples carrying labels; the labels are used to indicate the category of the target corresponding to the training sample; an iterative training unit, used to train a weak classifier using the training data set with a weight distribution in the current iterative training, calculate the classification error rate of the weak classifier on the training data set, calculate the weight of the weak classifier based on the classification error rate and update the weight distribution of the training data set; a weak classifier acquisition unit, used to obtain multiple weak classifiers through multiple iterative training; a strong classifier acquisition unit, used to obtain a strong classifier as the target recognition model based on the multiple weak classifiers and the corresponding weights.

[0015] In an exemplary embodiment, the device also includes: a deletion module, which is used to delete the target tracking information corresponding to the current target if it is detected that the current target leaves the set range, until it is detected that the current target enters the set range, and re-track the current target.

[0016] In an exemplary embodiment, the device also includes: a trigger condition detection module, which is used to detect whether the trigger condition in the automation scheme is met based on the target tracking information corresponding to the current target; the automation scheme is used to control the target device to perform a preset automation operation when the target is detected at a specified location; and an automation operation execution module, which is used to control the target device to perform the automation operation in the automation scheme if it is detected that the trigger condition is met.

[0017] According to one aspect of the present application, an electronic device includes at least one processor and at least one memory, wherein the memory stores computer-readable instructions; the computer-readable instructions are loaded and executed by the processor, so that the electronic device implements the target tracking method as described above.

[0018] According to one aspect of the present application, a storage medium stores computer-readable instructions thereon, and the computer-readable instructions are loaded and executed by a processor to implement the target tracking method as described above.

[0019] According to one aspect of the present application, a computer program product includes computer-readable instructions, which are stored in a storage medium. A processor of an electronic device reads the computer-readable instructions from the storage medium, loads and executes the computer-readable instructions, so that the electronic device implements the target tracking method as described above.

[0020] The beneficial effects of the technical solution provided by this application are:

[0021] In the above technical scheme, multiple frames of point cloud data are first acquired, and clustering is performed respectively to obtain multiple point cloud blocks corresponding to each frame of point cloud data. In the point cloud blocks of at least one adjacent frame adjacent to the current frame, the point cloud blocks associated with the current point cloud block in the current frame are searched. According to the current point cloud block in the current frame and the associated point cloud blocks, the target point cloud features for the current target are constructed, and then the recognition results are obtained according to the target point cloud features. Finally, the target is tracked based on the recognition results. The present application scheme utilizes at least two frames of point cloud data to identify the category of the current target, that is, constructs target point cloud features based on the current point cloud block in the current frame and the point cloud blocks associated with the current point cloud block in the current frame (at least two frames of point cloud data) and then performs category recognition based on the target point cloud features. This enables more accurate target category recognition and avoids identifying targets of other categories as specific categories. Furthermore, when tracking targets belonging to a specific category, it avoids tracking targets belonging to other categories (i.e., false targets), thereby improving target tracking accuracy and facilitating the provision of high-precision intelligent services to users, thereby effectively solving the problem of easy positioning and tracking of false targets in related technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in describing the embodiments of the present application are briefly introduced below.

[0023] Figure 1 It is a schematic diagram of the implementation environment involved in this application;

[0024] Figure 2 is a flow chart of a target tracking method according to an exemplary embodiment;

[0025] Figure 3 is a schematic diagram of clustering a frame of point cloud data according to an exemplary embodiment;

[0026] Figure 4 yes Figure 2 A flowchart of an embodiment corresponding to step 220 in an embodiment;

[0027] Figure 5 yes Figure 2A flowchart of an embodiment corresponding to step 240 in an embodiment;

[0028] Figure 6 yes Figure 5 A flowchart of an embodiment corresponding to steps 500 and 520 in an embodiment;

[0029] Figure 7 yes Figure 2 The flowchart of an embodiment after step 260 in the corresponding embodiment;

[0030] Figure 8 is a flowchart of a training process of a target recognition model according to an exemplary embodiment;

[0031] Fig. 9 It is a schematic diagram of a specific implementation of a target tracking method in an application scenario;

[0032] Fig.10 is a structural block diagram of a target tracking device according to an exemplary embodiment;

[0033] Fig.11 The present invention is a structural block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0034] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.

[0035] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0036] The following is an introduction and explanation of several terms involved in this application:

[0037] The Adaboost algorithm is a supervised learning classification algorithm and a machine learning algorithm. It needs to be trained using a labeled data set. During the training process, the Adaboost algorithm iteratively trains a series of weak classifiers and then combines them into a strong classifier to improve the accuracy of classification. The Adaboost algorithm automatically learns the features and rules in the data set to obtain the classification results.

[0038] DBSCAN, whose full name is Density-Based Spatial Clustering of Applications with Noise, means a density-based spatial clustering algorithm that is robust to noise. Its core idea is to cluster points with higher density into one category, and regard points with lower density as noise or boundary points.

[0039] Point cloud registration technology is used to find point cloud blocks belonging to the same target in different frames of point cloud data. Point cloud registration is the process of aligning multiple frames of point cloud data. Point cloud data of different frames can be aligned to the same coordinate system, so as to achieve comparison and matching of point cloud data. Its specific implementation process can be divided into the following steps: 1. De-noise and filter each point cloud data to improve the accuracy and efficiency of matching. 2. Select a point cloud data as a reference point cloud and align other point cloud data with it. You can use the ICP (Iterative Closest Point) algorithm or other registration algorithms. 3. In the registered point cloud data, you can use a clustering algorithm to group the point cloud blocks of the same target, such as using the DBSCAN algorithm. 4. According to the shape, color, texture and other features of the point cloud blocks, you can further filter out the point cloud blocks belonging to the same target. 5. Merge or fit the filtered point cloud blocks to obtain more complete target point cloud data.

[0040] As mentioned earlier, when using point cloud data for target tracking, it is easy to locate and track false targets, and then create tracks for the false targets.

[0041] When tracking a target, the related technology first clusters the point cloud data obtained by the sensor equipment to obtain point cloud blocks, and then determines in turn whether each point cloud block is associated with the existing target trajectory; if so, the associated target trajectory is updated according to the point cloud block associated with the target trajectory; if not, whether the point cloud block belongs to a new target is determined based on the number of point clouds in the point cloud block, signal echo intensity, and speed; if the point cloud block belongs to a new target, a corresponding trajectory is created for the new target, otherwise there is no need to create a trajectory.

[0042] The inventors realized that the above scheme only utilizes the point cloud data of a single frame and ignores the information between frames, which makes it easy to locate and track false targets, thereby creating false target trajectories. For example, the point cloud data generated by fans, curtains, ghost images, and multipaths are misjudged as pedestrians, resulting in the creation of false target (non-pedestrian) trajectories.

[0043] As can be seen from the above, the related technology still has the problem of easily locating and tracking false targets.

[0044] To this end, the target tracking method provided by the present application can accurately locate and track the target, reduce the occurrence of false targets, and improve the accuracy of target tracking. Accordingly, the target tracking method is suitable for a target tracking device, which can be deployed in an electronic device. The electronic device can be a sensing device, for example, the sensing device includes a radar device, or a human body sensor deployed with a millimeter-wave radar; the electronic device can also be an electronic device with a central control function or configured with a von Neumann architecture, for example, the electronic device includes a gateway, a server, etc.

[0045] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0046] Figure 1 Schematic diagram of an implementation environment involved in a target tracking method. The implementation environment at least includes a user terminal 110, a smart device 130, a server 170, and a network device. Figure 1 In the embodiment, the network devices include a gateway 150 and a router 190, which is not specifically limited herein.

[0047] Among them, the user terminal 110, which can also be considered as a user end or terminal, can deploy (also understood as install) the client associated with the smart device 130. This user terminal 110 can be an electronic device such as a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart control panel, other devices with display and control functions, etc., which are not limited here.

[0048] Among them, the client is associated with the smart device 130, which essentially means that the user registers an account in the client and configures the smart device 130 in the client. For example, the configuration includes adding a device identifier to the smart device 130, so that when the client is run in the user terminal 110, it can provide the user with functions such as device display and device control of the smart device 130. This client can be in the form of an application or a web page. Accordingly, the interface for device display on the client can be in the form of a program window or a web page, which is not limited here.

[0049] In this implementation environment, the smart device 130 includes a radar device or a human body sensor deployed with a millimeter wave radar. The smart device 130 is deployed in the gateway 150, and communicates with the gateway 150 through its own configured communication module, and is then controlled by the gateway 150. It should be understood that the smart device 130 generally refers to one of the multiple smart devices 130, and the embodiment of the present application only takes the smart device 130 as an example, that is, the embodiment of the present application does not limit the number and device type of smart devices deployed in the gateway 150. In an application scenario, the smart device 130 accesses the gateway 150 through a local area network, and is thus deployed in the gateway 150. The process of the smart device 130 accessing the gateway 150 through the local area network includes: the gateway 150 first establishes a local area network, and the smart device 130 joins the local area network established by the gateway 150 by connecting to the gateway 150. This local area network includes but is not limited to: ZIGBEE or Bluetooth. Of course, the smart device 130 can also be a smart printer, smart fax machine, smart camera, smart air conditioner, smart door lock, smart light, or a door and window sensor, temperature and humidity sensor, water immersion sensor, natural gas alarm, smoke alarm, wall switch, wall socket, wireless switch, wireless wall switch, Rubik's Cube controller, curtain motor, etc. equipped with a communication module deployed in the gateway 150.

[0050] The interaction between the user terminal 110 and the smart device 130 can be achieved through a local area network or a wide area network. In one application scenario, the user terminal 110 establishes a communication connection between the router 190 and the gateway 150 in a wired or wireless manner. For example, the wired or wireless manner includes but is not limited to WIFI, so that the user terminal 110 and the gateway 150 are deployed in the same local area network, so that the user terminal 110 can interact with the smart device 130 through the local area network path. In another application scenario, the user terminal 110 establishes a communication connection between the server 170 and the gateway 150 in a wired or wireless manner. For example, the wired or wireless manner includes but is not limited to 2G, 3G, 4G, 5G, WIFI, etc., so that the user terminal 110 and the gateway 150 are deployed in the same wide area network, so that the user terminal 110 can interact with the smart device 130 through the wide area network path.

[0051] The server end 170 may also be considered as a cloud, a cloud platform, a platform end, a service end, etc. The server end 170 may be a server, a server cluster consisting of multiple servers, or a cloud computing center consisting of multiple servers, so as to better provide background services to a large number of user terminals 110. For example, the background service includes a target tracking service.

[0052] In an application scenario, the smart device 130 (such as a radar device) detects a target in space to obtain multiple frames of point cloud data, clusters the multiple frames of point cloud data respectively, and obtains at least one point cloud block for each frame of point cloud data; searches for a point cloud block associated with a current point cloud block in the current frame in the point cloud blocks of at least one adjacent frame adjacent to the current frame; constructs a target point cloud feature for the current target based on the current point cloud block and the associated point cloud blocks in the current frame, and identifies the category of the target based on the target point cloud feature to obtain an identification result, and then can track the target based on the identification result.

[0053] After obtaining the corresponding tracking results of the target (such as the target trajectory), the smart device 130 can transmit the corresponding tracking results of the target to the user terminal 110 for display, or transmit them to the gateway 150 so that the gateway 150 can determine whether to control other smart devices 130 (such as smart lights) to perform automated operations.

[0054] Of course, in other application scenarios, the target tracking process performed by the smart device 130 can also be implemented by the server 170. In this case, after the smart device 130 detects the target in the space and obtains the multi-frame point cloud data to be processed, the server 170 obtains the multi-frame point cloud data through the wide area network path, and the server 170 obtains the corresponding recognition result based on the multi-frame point cloud data to be processed, and only performs target tracking (for example, creating a corresponding trajectory for the target) when the recognition result indicates that the target category is a specific category.

[0055] Through the above process, at least two frames of point cloud data are used to identify the target category. When the identification result indicates that the target category is a specific category, the target is tracked to avoid tracking a false target.

[0056] See also Figure 2 The present application embodiment provides a target tracking method, which is applicable to an electronic device, which may be Figure 1 The user terminal 110, gateway 150, and server 170 in the implementation environment shown may also be Figure 1 A smart device 130 (eg, a radar device) is shown in an implementation environment.

[0057] In the following method embodiments, for ease of description, the execution subject of each step of the method is taken as an electronic device as an example for illustration, but this does not constitute a specific limitation.

[0058] like Figure 2 As shown, the method may include the following steps:

[0059] Step 200 , clustering is performed on the acquired multiple frames of point cloud data to obtain point cloud blocks of each frame.

[0060] Point cloud data is obtained by sensing equipment detecting targets in space. Point cloud data is a data set composed of a large number of point clouds, each of which contains its coordinates and other attributes in space (such as color, normal vector, intensity, speed of movement, etc.). The space refers to a specific area, such as a home space, a bedroom space, etc., and can also be understood as the effective detection range of the sensing equipment; the target refers to an object that can be detected by the sensing equipment in space, and the object can be a variety of moving objects. For example, the target can specifically include at least one of a person, an animal, a robot, etc., which is not a specific limitation here.

[0061] It is explained here that the sensing device can be a sensor such as a radar device, for example, a millimeter wave radar, which is not limited here.

[0062] Clustering processing refers to dividing the point cloud in the point cloud data into multiple different parts (point cloud blocks), such as Figure 3 As shown, a frame of point cloud data is shown, and the frame of point cloud data is divided into two parts by clustering, namely point cloud block 1 and point cloud block 2. It should be noted that clustering of point clouds can be achieved by clustering algorithms based on shape, color, density, etc. For example, the clustering algorithm can be a density-based DBSCAN algorithm, a grid-based Euclidean clustering algorithm, or a hierarchical clustering algorithm, etc., which are not specifically limited here.

[0063] The DBSCAN algorithm does not require the number of clusters to be specified in advance, but controls the compactness of clusters and the tolerance of noise points through some parameters. The DBSCAN algorithm has excellent clustering effects when processing non-convex data sets with large noise interference.

[0064] Step 220, searching for a point cloud block associated with the current point cloud block in the current frame in the point cloud blocks of at least one adjacent frame adjacent to the current frame.

[0065] As mentioned above, since only using a single frame of point cloud data information will lead to ignoring the information between frames, it is easy to locate and track false targets, and then create the trajectory of false targets. In order to avoid locating and tracking false targets, in addition to using the current point cloud block in the current frame, the associated point cloud blocks are also searched in the point cloud blocks of at least one adjacent frame adjacent to the current frame, so as to capture the information between frames when constructing the target point cloud features later, thereby improving the accuracy of category recognition and avoiding locating and tracking false targets.

[0066] In one possible implementation, a point cloud block associated with a current point cloud block in the current frame is searched in a point cloud block of an adjacent frame adjacent to the current frame to obtain two point cloud blocks for constructing target point cloud features for the current target.

[0067] In one possible implementation, point cloud blocks associated with the current point cloud block in the current frame are searched in point cloud blocks of two adjacent frames adjacent to the current frame to obtain three point cloud blocks for constructing target point cloud features for the current target.

[0068] That is, the current point cloud block in the current frame and the associated point cloud blocks may include two point cloud blocks or three point cloud blocks.

[0069] In a possible implementation, the method of searching for a point cloud block associated with a current point cloud block in a current frame may be implemented based on the center of the point cloud block, or may be implemented based on a point cloud registration technology.

[0070] Take the association search based on the center of the point cloud block as an example. Figure 4 As shown, step 220 may include the following steps:

[0071] Step 400, determining the center of each point cloud block in each frame.

[0072] The center of the point cloud block can be obtained by calculating the average position of all point clouds in the point cloud block. For example, all point clouds in the point cloud block are traversed, the coordinate values ​​of each point cloud are accumulated, and the accumulated coordinate values ​​are divided by the number of point clouds in the point cloud block to obtain the average coordinate value, and the average coordinate value is taken as the center of the point cloud block.

[0073] The center of the point cloud block can also be calculated using weighted average to more accurately reflect the distribution of the point cloud block and avoid the center position offset caused by uneven distribution of point clouds within the point cloud block.

[0074] The above is only an example, and the method for determining the center of a point cloud block is not limited to the above.

[0075] Step 420, searching for a point cloud block whose center distance from the current point cloud block in the current frame is within a specific range among the point cloud blocks of at least one adjacent frame to the current frame as a point cloud block associated with the current point cloud block in the current frame.

[0076] The center distance refers to the distance between the center of the current point cloud block in the current frame and the center of the point cloud block in at least one adjacent frame.

[0077] The specific range can be flexibly adjusted according to the actual application scenario. For example, the specific range can be 1m, which is not limited here.

[0078] For example, to find the associated point cloud block in the point cloud blocks corresponding to 3 consecutive frames of point cloud data, the point cloud data of the first frame (the previous frame of the current frame) includes the point cloud block The second frame (current frame) point cloud data includes point cloud blocks The point cloud data of the third frame (the frame after the current frame) includes point cloud blocks Point Cloud Block For example, the superscript i represents the point cloud data from the i-th frame, and j represents the j-th point cloud block.

[0079] Get the point cloud block of the second frame (current frame) point cloud data For the current point cloud block in the current frame, the point cloud block in the point cloud data of the first frame (the frame before the current frame) Find the current point cloud block in the current frame The point cloud blocks within 1m of the center distance are obtained From the point cloud block in the third frame (the frame after the current frame) Find the current point cloud block in the current frame The point cloud blocks within 1m of the center distance are obtained That is, get the current point cloud block in the current frame Associated point cloud blocks

[0080] Step 240 , constructing a target point cloud feature for the current target based on the current point cloud block and the associated point cloud blocks in the current frame.

[0081] The target point cloud features include the attributes of the target (such as spatial coordinates, signal strength, and movement speed). In a possible implementation, the construction of the target point cloud features can be achieved by constructing the first feature based on the point cloud in the associated point cloud block of each frame.

[0082] In a possible implementation, the construction of the point cloud feature may be implemented by constructing a second feature based on point clouds in point cloud blocks associated with different frames.

[0083] In a possible implementation, the construction of the point cloud feature may be achieved by fusing the first feature and the second feature.

[0084] In this way, since the associated point cloud blocks include point cloud blocks from different frames, the point cloud features of the target not only consider the single-frame point cloud data, but can also utilize multi-frame point cloud data, and then extract feature information within the frame and / or between frames. When performing category detection based on the point cloud features of the target, it is beneficial to improve the accuracy of subsequent target category detection.

[0085] Step 260 , identifying the category of the current target according to the target point cloud features, obtaining a recognition result, and tracking the target based on the recognition result.

[0086] The recognition result is used to indicate the category of the current target, for example, the categories include pedestrians, curtains, fans, pets, etc. In a possible implementation, the recognition result is used to indicate whether the category of the target is a specific category, for example, the specific category may specifically refer to pedestrians, that is, the recognition result is used to indicate whether the target is a pedestrian or a non-pedestrian.

[0087] The method of identifying the category of the target can also be considered as a binary classification method, which can be implemented using a classification algorithm or a machine learning model. Among them, machine learning models include: Support Vector Machine (SVM), Decision Tree, Random Forest, Neural Network, K-Nearest Neighbor (KNN), Bayesian Classifier, Deep Learning, which is not a specific limitation here.

[0088] After obtaining the recognition result, it is possible to determine whether to perform target tracking (eg, create a corresponding track for the target) based on whether the category of the target indicated by the recognition result is a specific category.

[0089] Specifically, if the recognition result indicates that the category of the target is a specific category (pedestrian), it means that the target is a true target, and a corresponding trajectory is created for the target. If the recognition result indicates that the category of the target is not a specific category (i.e., non-pedestrian), it means that the target is a false target, and there is no need to create a corresponding trajectory for the target, thereby avoiding positioning and tracking of false targets, and fully ensuring that the created target trajectory is true and valid.

[0090] Based on accurate and effective target tracking, automation solutions can be accurately implemented, thereby improving the accuracy of equipment control and providing users with more accurate intelligent services.

[0091] It should be noted that the more point cloud blocks associated with the current point cloud block in the current frame, the more accurate the subsequent category recognition will be. Of course, the time required for target category recognition will also increase accordingly. Therefore, as many point cloud blocks associated with the current point cloud block in the current frame as possible can be used while ensuring that the time required for target category recognition is within a limited range.

[0092] In a possible implementation, after step 260, the method may further include the following steps:

[0093] Step 261 , based on the target tracking information corresponding to the current target, detect whether the triggering condition in the automation solution is met.

[0094] The automation scheme is used to control the target device to perform a preset automation operation when a target is detected at a specified location. The automation scheme at least includes a trigger condition and an automation operation.

[0095] For example, the automation solution may be to turn on the light when someone is within a certain range of the light, otherwise turn it off; in this case, the first trigger condition is that someone is within a certain range of the light, and the corresponding first automation operation is to turn on the light; the second trigger condition is that the person is not within the certain range, and the corresponding second automation operation is to turn off the light.

[0096] Step 262: If it is detected that the target tracking information meets the trigger condition, the target device is controlled to execute the automation operation in the automation solution.

[0097] For example, the trigger condition is that a person enters a certain range, and the automated operation is to turn on the light. When the corresponding trajectory of the pedestrian shows that someone has entered a certain range, the light set in the automated solution will be turned on.

[0098] In a possible implementation, after step 260, the method may further include the following steps:

[0099] Step 263, if it is detected that the current target leaves the set range, the target tracking information corresponding to the current target is deleted, until it is detected that the current target enters the set range, and target tracking is performed again for the current target.

[0100] The detection range of the range sensing device can also be a manually preset range.

[0101] After determining that the target has left a specific range, the target tracking information (such as trajectory) corresponding to the target can be deleted to avoid occupying storage resources, thereby improving resource utilization.

[0102] Through the above process, the present application scheme utilizes the current point cloud block and the associated point cloud blocks in the current frame to construct target point cloud features for category recognition. The point cloud features constructed based on the current point cloud block and the associated point cloud blocks in the current frame include not only the feature information of the single-frame point cloud block, but also the second feature information between the point cloud blocks of each frame, that is, the single-frame point cloud data information and the information between frames are used to perform target category detection. The present application scheme utilizes more feature information, so that category detection can be performed more accurately, and specific categories and other categories can be accurately identified and distinguished, avoiding the situation where other categories of targets are detected as specific categories, and then in the corresponding trajectory of the created (specific category) target, the appearance of false targets such as other categories of targets is reduced, and the accuracy of target tracking is improved, thereby effectively solving the problem of easy positioning and tracking of false targets existing in the related technology.

[0103] In addition, searching for point cloud blocks associated with the target by center distance is simple, easy and fast. Based on accurate and effective target (pedestrian) tracking, the automation solution can be realized, which improves the efficiency and convenience of equipment control. Accurate target category detection also reduces the error control rate of the automation solution and avoids the occurrence of error control.

[0104] See also Figure 5 In an exemplary embodiment, step 240 may include the following steps:

[0105] Step 500, based on the current point cloud block and the associated point cloud blocks in the current frame, respectively construct the first features corresponding to each point cloud block and the second features between each point cloud block.

[0106] Among them, the first feature is used to describe the characteristics of a frame of point cloud data itself, for example, the first feature of the first frame describes the first frame of point cloud data itself; the second feature is used to describe the relationship between two frames of point cloud data, for example, the second feature can describe the degree of correlation between the first frame of point cloud data and the second frame of point cloud data.

[0107] Taking 3 consecutive frames of point cloud data as an example, the first frame of point cloud data includes point cloud blocks The second frame of point cloud data includes point cloud blocks The third frame of point cloud data includes point cloud blocks Find the point cloud block associated with the target Get the current point cloud block in the current frame And the associated point cloud block The first features corresponding to each point cloud block include The corresponding first feature, The corresponding first feature, The corresponding first feature, the second feature between the point cloud blocks includes and The second characteristic between and The second characteristic between and The second feature between.

[0108] In one possible implementation, Figure 6 As shown, step 500 may include the following steps:

[0109] Step 600 , obtaining a specific number of point clouds from the current point cloud block and the associated point cloud blocks in the current frame respectively.

[0110] The specific number can be flexibly adjusted according to actual conditions, for example, it can be 10, and no specific limitation is made here.

[0111] The acquisition method can be random selection, selection based on signal strength, selection based on the center distance of adjacent frame point cloud data, selection based on grid, selection based on voxel, etc., which is not specifically limited here.

[0112] Continuing with the above example, the current point cloud block in the current frame And the associated point cloud block from Get 10 point clouds respectively, optionally from In the example, 10 point clouds are selected from strong to weak according to the signal strength. If If there are not 10 point clouds, copy them to 10 points according to the signal strength from strong to weak. The same operation is performed to obtain 10 selected point clouds.

[0113] Step 620, based on the acquired point clouds, calculate the first features corresponding to each point cloud block and the second features between the point cloud blocks.

[0114] Continuing with the above example, The 10 selected point clouds are subtracted from each other to obtain The first characteristic is Perform the same operation to obtain The first feature.

[0115] Will The 10 point clouds selected from The 10 point clouds selected are subtracted from each other to obtain and The second feature between The 10 point clouds selected from The 10 point clouds selected are subtracted from each other to obtain and The second feature between.

[0116] It should be understood that the above examples are not specific limitations of the present application. For example, selecting 10 point clouds may also be selecting 14 point clouds, and subtracting two point clouds from each other may also be other processing methods.

[0117] Step 520: Obtain a target point cloud feature for the current target based on the first feature and the second feature.

[0118] In one possible implementation, Figure 6 , step 520 may include the following steps:

[0119] Step 640: fuse the first feature and the second feature to obtain a target point cloud feature for the current target.

[0120] Optionally, the first features corresponding to each point cloud block and the second features between the point cloud blocks are connected (joined) in a certain order to obtain the point cloud data of the target.

[0121] Of course, in other embodiments, the fusion method is not limited to splicing, and operations such as addition may also be used, and this embodiment does not constitute a specific limitation to this.

[0122] Under the effect of the above-mentioned embodiments, a specific number of point clouds are respectively obtained from the current point cloud block in the current frame and the associated point cloud blocks to construct features, so as to avoid the failure of the subsequent classification algorithm due to insufficient or excessive point clouds, and the efficiency and accuracy of the category detection algorithm in the subsequent steps can be improved, so as to better realize the target category recognition; the first features corresponding to each point cloud block and the second features between the point cloud blocks are constructed, not only focusing on the feature information of a single frame, but also considering the feature information between frames, so as to fully obtain the features of the target, and the obtained features are richer and more effective. The first features corresponding to each point cloud block and the second features between the point cloud blocks are fused (spliced), the information of different features is integrated, the dimension of the feature space is increased, and the influence of feature selection is reduced, so as to obtain more useful and effective target point cloud features, which can improve the effect and accuracy of target category recognition in subsequent steps. For example, the target cloud features can achieve more accurate category recognition by improving the performance of machine learning models and artificial intelligence models.

[0123] See also Figure 7 In an exemplary embodiment, step 260 may be followed by the following steps:

[0124] Step 700 , performing multiple recognitions on the category of the current target based on the acquired multi-frame point cloud data to obtain multiple recognition results for the current target.

[0125] In order to further improve the accuracy of target category recognition, in this embodiment, the category of the current target is recognized multiple times based on the acquired multi-frame point cloud data.

[0126] For example, assuming that the acquired multi-frame point cloud data includes 9 consecutive frames of point cloud data, namely S1, S2, S3, S4, S5, S6, S7, S8, and S9, the current point cloud block and the associated point cloud blocks in the current frame are obtained based on S1, S2, and S3, and category recognition is performed once, category recognition is performed once based on S2, S3, and S4, and category recognition is performed once based on S3, S4, and S5, and so on. In this way, 7 recognitions can be performed and 7 recognition results for the current target are obtained respectively.

[0127] Specifically, Figure 7 In step 260, “target tracking based on the recognition result” may include the following steps:

[0128] Step 720: If the number of times that the multiple recognition results indicate that the category of the current target is a specific category exceeds a specific threshold, then the point cloud data of the current target is obtained.

[0129] The specific threshold value can be flexibly adjusted according to actual conditions.

[0130] Continuing with the above example, assuming that the acquired multi-frame point cloud data includes 9 consecutive frames of point cloud data, namely S1, S2, S3, S4, S5, S6, S7, S8, and S9, 7 recognitions are performed, and 7 recognition results for the current target are obtained respectively. The 7 recognition results indicate that the category of the current target is a specific category more than 4 times, and then the point cloud data of the current target is acquired for target tracking (for example, creating a trajectory).

[0131] Step 740: Create a trajectory corresponding to the current target based on the point cloud data of the current target.

[0132] Based on the fact that the number of times that the multiple recognition results indicate that the category of the target is a specific category exceeds a specific threshold, it can be determined that the category of the target is the specific category, and then a corresponding track is created for the target.

[0133] Optionally, the motion state of the object is inferred by comparing point cloud data at different times, thereby generating the motion trajectory of the object.

[0134] Optionally, creating a corresponding trajectory of the target based on the point cloud data of the target can also be achieved through motion estimation or motion tracking algorithms, such as ICP (Iterative Closest Point) algorithm, NDT (Normal Distribution Transform) algorithm, ESKF (Error-State Kalman Filter) algorithm, etc.

[0135] Under the effect of the above-mentioned embodiment, multiple identifications are performed. Based on the multiple identification results obtained, target tracking (for example, creating a corresponding trajectory) is performed for the target only when the multiple identification results indicate that the target category is a specific category more than a specific threshold. The category of the target is comprehensively judged based on the multiple identification results, thereby further improving the accuracy of category identification, ensuring the authenticity and effectiveness of the created target trajectory, and avoiding tracking false targets.

[0136] In an exemplary embodiment, the identification of the category of the target in step 260 is implemented by calling a target identification model, which is a trained machine learning model that has the ability to identify the category of the target.

[0137] In a possible implementation, step 260 may include the following steps:

[0138] Step 264, input the target point cloud features into the target recognition model to predict the category of the current target and obtain a recognition result.

[0139] In a possible implementation, the target recognition model is implemented based on the Adaboost algorithm.

[0140] In one possible implementation, Figure 8 As shown, the training process of the target recognition model includes the following steps:

[0141] Step 800: Obtain a training data set.

[0142] The training data set includes multiple training samples with labels, where the labels are used to indicate the categories of the targets corresponding to the training samples, and the training samples are target point cloud features.

[0143] Step 820, in the current iterative training, a weak classifier is trained using the training data set with weight distribution, the classification error rate of the weak classifier on the training data set is calculated, the weight of the weak classifier is calculated based on the classification error rate and the weight distribution of the training data set is updated.

[0144] A weak classifier can be any algorithm that can classify data, such as decision trees, support vector machines, neural networks, etc.

[0145] One iterative training can obtain a trained weak classifier, and multiple iterative training is often required to obtain multiple weak classifiers. The training data set with updated weight distribution in this iterative training is used for the next iterative training.

[0146] Optionally, before the first iteration of training, when initializing the weight distribution of the training data set, the weight of each training sample is usually set to an equal value, for example, the weight of each training sample is set to 1 / n, where n is the number of training samples.

[0147] Optionally, the weight of each training sample in the current iteration round is calculated first, and then the sum of the weights of the misclassified training samples is calculated according to the classification results of the weak classifier, and finally the classification error rate of the weak classifier is obtained.

[0148] Optionally, the weak classifier with a lower classification error rate has a larger weight, and the weak classifier with a higher classification error rate has a smaller weight.

[0149] Optionally, in order to focus on training samples with high classification error rates, when updating the weights of the training samples, the training samples with lower classification error rates have smaller weights, and the training samples with higher classification error rates have larger weights.

[0150] Step 840, obtaining multiple weak classifiers through multiple iterations of training.

[0151] For example, after the training of a current weak classifier is completed, the weight of the current weak classifier in the strong classifier is obtained, and the weight distribution of the training data set is updated; based on the training data set with updated weight distribution, continue to train another weak classifier, obtain the weight of the other weak classifier in the strong classifier, and update the weight distribution of each training sample in the training data set again; and so on, multiple weak classifiers are trained.

[0152] Step 860, obtaining a strong classifier as a target recognition model based on multiple weak classifiers and corresponding weights.

[0153] In a possible implementation, when specific conditions are met, a strong classifier is obtained according to each weak classifier that has completed training and its corresponding weights, and the strong classifier is used as a target recognition model.

[0154] The specific condition may be that a certain number of weak classifiers are trained (ie, a certain number of iterative training times are reached), or that a strong classifier reaches a certain classification accuracy. The specific condition may be set according to actual conditions and is not specifically limited here.

[0155] The training process of the target recognition model is illustrated by taking the training data set T = {(x1,y1),(x2,y2),...,(x N ,y N )},in y i ∈Y=-1,1, where x i is the training sample, y i For labels, before the first iteration of training, the weight distribution of the initial training data set is:

[0156]

[0157] Among them, w 1,i is the training sample x in the training dataset in the first iteration training i The weight of

[0158] In the first iteration training, the weak classifier G1(x) is trained using the training data set with weight distribution D1, and the classification error rate e1 of G1(x) on the training data set is calculated, which can be calculated by the formula:

[0159]

[0160] Among them, w m,i is the training sample x in the training data set in the mth iteration training i The weight of y i It's a label, G m (x i) is the predicted value of the weak classifier, I is a function, when G m (x i )≠y i If satisfied, I is 1, otherwise I is 0.

[0161] Where m = 1, 2, ..., M, the weight α1 of the weak classifier G1(x) is calculated according to the classification error rate e1, which can be calculated by the formula:

[0162]

[0163] The weight distribution of each training sample in the training data set can be calculated by the following formula:

[0164]

[0165]

[0166] Among them, α m is the weight of the weak classifier, y i It's a label, G m (x i ) is the predicted value of the weak classifier.

[0167] Get the updated weight distribution D2; in the second iterative training, use the training data set with weight distribution D2 to continue training another weak classifier G2(x), the same as the previous steps, calculate the classification error rate e2 of G2(x) on the training data set, calculate the weight α2 of the weak classifier G2(x) according to the classification error rate e2, and update the weight distribution of the training data set to D3, in the third iterative training, the training data set with weight distribution D3 can be used to train another weak classifier G3(x); repeat the training of multiple weak classifiers until the specified number of iterations is reached (each training of a weak classifier is an iterative training) or the strong classifier reaches a certain classification accuracy; the final strong classifier F(x) can be obtained by the following formula:

[0168]

[0169] Among them, α in the formula m is the weight of the weak classifier, G m (x) is a weak classifier.

[0170] Through the above process, during the training of the target recognition model, by continuously adjusting the weight distribution of the training data set and the weight of the weak classifier, a strong classifier is obtained to improve the accuracy of classification (target category recognition).

[0171] Fig. 9This is a schematic diagram of a target tracking method in an application scenario. In this application scenario, there are 9 consecutive frames of point cloud data. Take three consecutive frames of point cloud data (for example, the first frame, the second frame, and the third frame), and use the DBSCAN algorithm to cluster these three frames of point cloud data. Each frame of point cloud data will be divided into N1, N2, and N3 blocks. The time corresponding to the three frames is T1, T2, and T3, and T1>T2>T3.

[0172] The point cloud block after clustering the first frame of point cloud data is recorded as The point cloud block after clustering the second frame of point cloud data is recorded as The point cloud block after clustering the third frame of point cloud data is recorded as Calculate the center coordinates of these point cloud blocks as their centers.

[0173] right Point cloud block, calculate its center coordinates by As the center, find the center coordinates in the second frame and Point cloud blocks within 1m Find the center coordinates in the third frame and (Point Cloud Block Point cloud blocks within 1m of the center coordinates of

[0174] Depend on Construct target point cloud features: a. From Extract 10 point clouds from the image according to the signal strength from strong to weak (if there are no 10 point clouds in the image, copy them to 10 according to the signal strength from strong to weak), and subtract these 10 point clouds from each other to obtain the intra-frame feature vector f1; b. Take 10 point clouds from the frame to get the intra-frame feature vector f2; c. Same as step a from Take 10 point clouds from the frame to get the intra-frame feature vector f3; d. The 10 point clouds in Subtract the 10 point clouds in two by two to obtain the inter-frame feature vector f4; e. The 10 point clouds in The 10 point clouds in are subtracted from each other to obtain the inter-frame feature vector f5; f. The feature vectors f1, f2, f3, f4, and f5 are concatenated to form the target point cloud features.

[0175] Based on the constructed target point cloud features, the target can be classified using the target recognition model based on the adaboost algorithm to obtain the classification result. If the classification result is a pedestrian, it is considered that the current pedestrian coordinates are in Department;

[0176] For the remaining point cloud blocks of the first frame Follow the same steps to locate pedestrians. That is, for each point cloud block in the first frame, follow the above steps in sequence. Let me illustrate this with an example.

[0177] In addition, because there may be multiple people moving at the same time in the space covered by the radar, Each point cloud block generally belongs to only one person, and each point cloud block is operated according to the above steps to avoid missing possible targets.

[0178] Furthermore, in order to ensure the accuracy of category recognition (classification), 7 target category recognition (classification) are performed according to the above steps based on 9 consecutive frames of point cloud data. When more than 3 of the 7 recognitions are judged as pedestrians, the target is tracked (for example, the trajectory of the pedestrian is created).

[0179] Specifically, the first frame, the second frame, and the third frame together complete the steps more than once; the second frame, the third frame, and the fourth frame together complete the steps more than once; the third frame, the fourth frame, and the fifth frame together complete the steps more than once; during the execution of 7 times, the point cloud data used twice will overlap.

[0180] When someone disappears from a specific range, the corresponding track of this person will be eliminated and will not be retained; when he enters the specific range again, the track must be recreated.

[0181] Based on the tracking results (such as trajectories), an automation solution can be implemented. A specific example is to control the light switch. When a person is within a certain range of the light based on the detection tracking results (such as trajectories), the light is turned on, otherwise it is turned off, and non-contact switches are used to control home appliances to achieve energy saving and convenience.

[0182] In this application scenario, seven classifications are performed and the category of the target is comprehensively judged based on the seven classification results, which is conducive to improving the accuracy of target category recognition and further reducing errors.

[0183] Compared with the related art, this application can produce the following beneficial effects:

[0184] 1. In the above technical solution, multiple frames of point cloud data are first obtained, and clustering is performed to obtain multiple point cloud blocks corresponding to each frame of point cloud data. In the point cloud blocks of at least one adjacent frame adjacent to the current frame, the point cloud blocks associated with the current point cloud block in the current frame are searched. According to the current point cloud block in the current frame and the associated point cloud blocks, the target point cloud features for the current target are constructed, and then the recognition results are obtained according to the target point cloud features. Finally, the target is tracked based on the recognition results. The present application scheme utilizes at least two frames of point cloud data to identify the category of the current target, that is, constructs target point cloud features based on the current point cloud block in the current frame and the point cloud blocks associated with the current point cloud block in the current frame (at least two frames of point cloud data) and then performs category recognition based on the target point cloud features. This enables more accurate target category recognition and avoids identifying targets of other categories as specific categories. Furthermore, when tracking targets belonging to a specific category, it avoids tracking targets belonging to other categories (i.e., false targets), thereby improving target tracking accuracy and facilitating the provision of high-precision intelligent services to users, thereby effectively solving the problem of easy positioning and tracking of false targets in related technologies.

[0185] 2. Use at least two frames of point cloud data to identify the target category. Only when the recognition result indicates that the target category is a specific category, the target is tracked to avoid tracking false targets.

[0186] 3. The present application scheme utilizes the current point cloud block and the associated point cloud blocks in the current frame to construct target point cloud features for category recognition. The point cloud features constructed based on the current point cloud block and the associated point cloud blocks in the current frame include not only the feature information of the single-frame point cloud block, but also the second feature information between the point cloud blocks of each frame. That is to say, the single-frame point cloud data information and the information between frames are used to perform target category detection. The present application scheme utilizes more feature information, so that category detection can be performed more accurately, and specific categories and other categories can be accurately identified and distinguished, avoiding the situation where other categories of targets are detected as specific categories, and then in the created (specific category) target corresponding trajectory, the occurrence of false targets such as other categories of targets is reduced, and the accuracy of target tracking is improved, thereby effectively solving the problem of easy positioning and tracking of false targets existing in the related technology.

[0187] In addition, searching for point cloud blocks associated with the target by center distance is simple, easy and fast. Based on accurate and effective target (pedestrian) tracking, the automation solution can be realized, which improves the efficiency and convenience of equipment control. Accurate target category detection also reduces the error control rate of the automation solution and avoids the occurrence of error control.

[0188] 4. Obtain a specific number of point clouds from the current point cloud block and the associated point cloud blocks in the current frame to construct features, so as to avoid the failure of the subsequent classification algorithm due to insufficient or excessive point clouds, which can improve the efficiency and accuracy of the category detection algorithm in the subsequent steps, thereby better realizing target category recognition; construct the first features corresponding to each point cloud block and the second features between the point cloud blocks, not only focusing on the feature information of a single frame, but also considering the feature information between frames, fully obtaining the features of the target, and obtaining richer and more effective features. Fuse (join) the first features corresponding to each point cloud block and the second features between the point cloud blocks, integrate the information of different features, increase the dimension of the feature space, and reduce the influence of feature selection, so as to obtain more useful and effective target point cloud features, which can improve the effect and accuracy of target category recognition in subsequent steps. For example, the target cloud features can achieve more accurate category recognition by improving the performance of machine learning models and artificial intelligence models.

[0189] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0190] The following is an embodiment of the device of the present application, which can be used to execute the target tracking method involved in the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the method embodiment of the target tracking method involved in the present application.

[0191] See also Fig.10 In an embodiment of the present application, a target tracking device 1600 is provided, including but not limited to: a clustering module 1610, an association search module 1620, a feature construction module 1630 and an identification module 1640.

[0192] The clustering module 1610 is used to perform clustering processing on the acquired multiple frames of point cloud data to obtain point cloud blocks of each frame.

[0193] The associated search module 1620 is used to search for a point cloud block associated with the current point cloud block in the current frame in the point cloud blocks of at least one adjacent frame adjacent to the current frame.

[0194] The feature construction module 1630 is used to construct a target point cloud feature for the current target based on the current point cloud block and the associated point cloud blocks in the current frame.

[0195] The recognition module 1640 is used to recognize the category of the current target according to the target point cloud features, obtain a recognition result, and track the target based on the recognition result.

[0196] In an exemplary embodiment, the association search module 1620 includes: a center determination unit 1621, used to determine the center of each point cloud block in each frame; a search unit 1622, used to search for a point cloud block whose center distance from the current point cloud block in the current frame is within a specific range among the point cloud blocks of at least one adjacent frame adjacent to the current frame, as a point cloud block associated with the current point cloud block in the current frame.

[0197] In an exemplary embodiment, the feature construction module 1630 includes: a feature construction unit 1631, which is used to construct a first feature corresponding to each point cloud block and a second feature between the point cloud blocks according to the current point cloud block and the associated point cloud blocks in the current frame; a point cloud feature acquisition unit 1632, which is used to obtain a target point cloud feature for the current target according to the first feature and the second feature.

[0198] In an exemplary embodiment, the feature construction unit 1631 includes: a point cloud acquisition subunit 16311, used to acquire a specific number of point clouds from the current point cloud block and the associated point cloud blocks in the current frame; a feature calculation subunit 16312, used to calculate the first features corresponding to each point cloud block and the second features between the point cloud blocks based on the acquired point clouds.

[0199] In an exemplary embodiment, the point cloud feature acquisition unit 1632 includes: a fusion subunit 16321, which is used to fuse the first feature and the second feature to obtain a target point cloud feature for the current target.

[0200] In an exemplary embodiment, the target tracking device 1600 also includes: a multiple recognition module 1650, which is used to perform multiple recognitions on the category of the current target based on the acquired multi-frame point cloud data to obtain multiple recognition results for the current target; the recognition module 1640 includes: a number judgment unit 1641, which is used to obtain the point cloud data of the current target if the multiple recognition results indicate that the category of the current target is a specific category and the number of times exceeds a specific threshold; a trajectory creation unit 1642, which is used to create a corresponding trajectory of the current target based on the point cloud data of the current target.

[0201] In an exemplary embodiment, recognition is achieved by calling a target recognition model, which is a trained machine learning model that has the ability to recognize the category of the target; the recognition module 1640 includes: a prediction unit 1643, which is used to input the target point cloud features into the target recognition model to predict the category of the current target and obtain a recognition result.

[0202] In an exemplary embodiment, the device 1600 also includes a model training module 1660 for training a target recognition model, and the model training module 1660 includes: a training data set acquisition unit 1661, for acquiring a training data set; the training data set includes multiple training samples carrying labels; the labels are used to indicate the category of the target corresponding to the training sample; an iterative training unit 1662, for training a weak classifier using a training data set with a weight distribution in the current iterative training, calculating the classification error rate of the weak classifier on the training data set, calculating the weight of the weak classifier based on the classification error rate and updating the weight distribution of the training data set; a weak classifier acquisition unit 1663, for obtaining multiple weak classifiers through multiple iterative training; a strong classifier acquisition unit 1664, for obtaining a strong classifier as a target recognition model based on multiple weak classifiers and corresponding weights.

[0203] In an exemplary embodiment, the device 1600 further includes: a deletion module 1670, which is used to delete the target tracking information corresponding to the current target if it is detected that the current target leaves the set range, until it is detected that the current target enters the set range, and target tracking is performed again for the current target.

[0204] In an exemplary embodiment, the device 1600 also includes: a trigger condition detection module 1680, which is used to detect whether the trigger condition in the automation scheme is met based on the target tracking information corresponding to the current target; the automation scheme is used to control the target device to perform a preset automation operation when the target is detected at a specified location; and an automation operation execution module 1690, which is used to control the target device to perform the automation operation in the automation scheme if it is detected that the trigger condition is met.

[0205] It should be noted that the target tracking device provided in the above embodiment only uses the division of the above-mentioned functional modules as an example when performing target tracking. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the target tracking device will be divided into different functional modules to complete all or part of the functions described above.

[0206] In addition, the target tracking device and the target tracking method provided in the above embodiments belong to the same concept, and the specific manner in which each module performs the operation has been described in detail in the method embodiment and will not be repeated here.

[0207] See also Fig.11 In an embodiment of the present application, an electronic device 4000 is provided, and the electronic device 4000 may include: radar equipment, human body sensors deployed with millimeter wave radars, and other sensing devices.

[0208] exist Fig.11 In the embodiment, the electronic device 4000 includes at least one processor 4001 and at least one memory 4003.

[0209] The data interaction between the processor 4001 and the memory 4003 can be realized through at least one communication bus 4002. The communication bus 4002 may include a path for transmitting data between the processor 4001 and the memory 4003. The communication bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The communication bus 4002 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.11 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0210] Optionally, the electronic device 4000 may further include a transceiver 4004, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.

[0211] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0212] The memory 4003 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired computer-readable instructions in the form of instructions or data structures and can be accessed by the electronic device 4000, but is not limited to this.

[0213] The memory 4003 stores computer-readable instructions, and the processor 4001 can read the computer-readable instructions stored in the memory 4003 through the communication bus 4002 .

[0214] When the computer-readable instructions are executed by the processor 4001, the target tracking method in the above-mentioned embodiments is implemented.

[0215] In addition, an embodiment of the present application provides a storage medium on which computer-readable instructions are stored. The computer-readable instructions are loaded and executed by a processor to implement the target tracking method as described above.

[0216] A computer program product is provided in an embodiment of the present application. The computer program product includes computer-readable instructions. The computer-readable instructions are stored in a storage medium. A processor of an electronic device reads the computer-readable instructions from the storage medium, loads and executes the computer-readable instructions, so that the electronic device implements the target tracking method as described above.

[0217] The above description is only a partial implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A target tracking method, characterized in that: include: Perform clustering processing on the acquired multi-frame point cloud data respectively to obtain the point cloud blocks of each frame; In the point cloud blocks of at least one adjacent frame adjacent to the current frame, searching for a point cloud block associated with the current point cloud block in the current frame; Constructing a target point cloud feature for a current target based on the current point cloud block and the associated point cloud blocks in the current frame; The category of the current target is identified according to the target point cloud features to obtain an identification result, and the target is tracked based on the identification result.

2. The method according to claim 1, characterized in that The step of searching, in point cloud blocks of at least one adjacent frame to the current frame, a point cloud block associated with the current point cloud block in the current frame comprises: Determine the center of each point cloud block in each frame; In the point cloud blocks of at least one adjacent frame adjacent to the current frame, the point cloud blocks whose center distance from the current point cloud block in the current frame is within a specific range are searched as the point cloud blocks associated with the current point cloud block in the current frame.

3. The method according to claim 1, characterized in that The constructing a target point cloud feature for the current target based on the current point cloud block and the associated point cloud blocks in the current frame includes: According to the current point cloud block and the associated point cloud blocks in the current frame, respectively constructing a first feature corresponding to each point cloud block and a second feature between each point cloud block; The target point cloud feature for the current target is obtained according to the first feature and the second feature.

4. The method according to claim 3, characterized in that The constructing, according to the current point cloud block in the current frame and the associated point cloud blocks, first features corresponding to each point cloud block and second features between point cloud blocks respectively, comprises: Acquire a specific number of point clouds from the current point cloud block and the associated point cloud block in the current frame respectively; Based on the acquired point clouds, first features corresponding to each point cloud block and second features between the point cloud blocks are calculated.

5. The method according to claim 3, characterized in that The step of obtaining the target point cloud feature for the current target according to the first feature and the second feature includes: The first feature and the second feature are fused to obtain the target point cloud feature for the current target.

6. The method according to claim 1, characterized in that The method further comprises: Based on the acquired multi-frame point cloud data, the category of the current target is identified multiple times to obtain multiple identification results for the current target; The target tracking based on the recognition result includes: If the number of times that the recognition results indicate that the category of the current target is a specific category exceeds a specific threshold, acquiring the point cloud data of the current target; A trajectory corresponding to the current target is created based on the point cloud data of the current target.

7. The method according to claim 1, characterized in that The identification is achieved by calling a target identification model, where the target identification model is a machine learning model that has been trained and has the ability to identify the category of the target; The step of identifying the category of the current target according to the target point cloud features to obtain a recognition result includes: The target point cloud features are input into the target recognition model to predict the category of the current target to obtain the recognition result.

8. The method according to claim 7, characterized in that The training process of the target recognition model includes: Acquire a training data set; the training data set includes a plurality of training samples carrying labels; the labels are used to indicate the categories of the targets corresponding to the training samples; In a current iterative training, a weak classifier is trained using the training data set with weight distribution, a classification error rate of the weak classifier on the training data set is calculated, a weight of the weak classifier is calculated based on the classification error rate, and the weight distribution of the training data set is updated; Through multiple iterations of the training, multiple weak classifiers are obtained; According to the multiple weak classifiers and the corresponding weights, a strong classifier is obtained as the target recognition model.

9. The method according to any one of claims 1 to 8, characterized in that The method further comprises: If it is detected that the current target leaves the set range, the target tracking information corresponding to the current target is deleted until it is detected that the current target enters the set range, and target tracking is performed again for the current target.

10. The method according to any one of claims 1 to 8, characterized in that: The method further comprises: Based on the target tracking information corresponding to the current target, detecting whether the trigger condition in the automation scheme is met; the automation scheme is used to control the target device to perform a preset automation operation when the target is detected at the specified position; If it is detected that the target tracking information meets the trigger condition, the target device is controlled to execute the automation operation in the automation solution.

11. A target tracking device, characterized in that: include: The clustering module is used to perform clustering processing on the acquired multi-frame point cloud data to obtain point cloud blocks of each frame; An association search module, used to search for a point cloud block associated with a current point cloud block in the current frame in a point cloud block of at least one adjacent frame adjacent to the current frame; A feature construction module, configured to construct a target point cloud feature for a current target based on the current point cloud block and the associated point cloud blocks in the current frame; The recognition module is used to identify the category of the current target according to the target point cloud features, obtain a recognition result, and track the target based on the recognition result.

12. An electronic device, characterized in that: include: at least one processor and at least one memory, wherein: The memory has computer-readable instructions stored thereon; The computer-readable instructions are loaded and executed by the processor, so that the electronic device implements the target tracking method according to any one of claims 1 to 10.

13. A storage medium having computer-readable instructions stored thereon, characterized in that: The computer-readable instructions are loaded and executed by a processor to implement the target tracking method according to any one of claims 1 to 10.

Images