Track updating method and device, electronic equipment and storage medium

Abstract

The invention provides a trajectory updating method and device, equipment and a storage medium, and relates to the technical field of positioning and tracking. The method comprises the following steps: performing clustering processing on obtained point cloud data in a target space to obtain a plurality of point cloud blocks; associating each point cloud block with a target track corresponding to each target in the target space; constructing point cloud features of each point cloud block based on each point cloud block and historical point cloud data in the target trajectory associated with each point cloud block; recognizing the state of each target track according to the point cloud feature of each point cloud block to obtain a corresponding recognition result; each identification result is used for indicating the state change condition of each target track; and according to the identification result of each target track, updating the target track of each target. According to the invention, the problem of low target tracking and positioning accuracy in the prior art is solved.

Description

Technical Field

[0001] This application relates to the technical field of positioning and tracking. Specifically, this application relates to a trajectory update method, apparatus, electronic device, and storage medium. Background Art

[0002] Target tracking and positioning are indispensable requirements in many application scenarios. Taking smart home in the field of Internet of Things technology as an example, it is very common to use products containing millimeter-wave radar as non-contact switches to control household appliances to achieve the purpose of energy saving and convenience.

[0003] However, when multiple targets are relatively close, the trajectories between the targets will affect each other and even intersect, making the point cloud data of each target often mixed together. Through clustering, the point cloud blocks of different targets are treated as a whole, and since each point cloud block is only allowed to belong to one trajectory, some trajectories are assigned too much point cloud, and some trajectories are not assigned any point cloud, resulting in errors in target positioning and thus unable to provide accurate services for users.

[0004] As can be seen from the above, how to improve the accuracy of target positioning in the process of target tracking remains to be solved. Summary of the Invention

[0005] This application provides a trajectory update method, apparatus, electronic device, and storage medium, which can solve the problem of low accuracy of target positioning in the process of target tracking. The technical solutions are as follows:

[0006] According to one aspect of this application, a trajectory update method includes: performing clustering processing on the acquired point cloud data in the target space to obtain multiple point cloud blocks; associating each of the point cloud blocks with the target trajectories corresponding to each target in the target space; constructing the point cloud features of each of the point cloud blocks based on each of the point cloud blocks and the historical point cloud data in the target trajectories associated with each of the point cloud blocks; identifying the states of each of the target trajectories according to the point cloud features of each of the point cloud blocks, and respectively obtaining corresponding identification results; each of the identification results is used to indicate the state change of each target trajectory; and updating the target trajectories of each target according to the identification results of each target trajectory.

[0007] According to one aspect of the present application, a trajectory updating device, the device includes: a point cloud acquisition module, configured to perform clustering processing on the acquired point cloud data in the target space to obtain a plurality of point cloud blocks; a point-track association module, configured to associate each of the point cloud blocks with the target trajectories corresponding to each target in the target space; a feature extraction module, configured to construct point cloud features of each of the point cloud blocks based on each of the point cloud blocks and the historical point cloud data in the target trajectories associated with each of the point cloud blocks; a state recognition module, configured to recognize the states of each of the target trajectories according to the point cloud features of each of the point cloud blocks, and respectively obtain corresponding recognition results; each of the recognition results is used to indicate the state change of each target trajectory; a trajectory updating module, configured to update the target trajectories of each of the targets according to the recognition results of each of the target trajectories.

[0008] In an exemplary embodiment, the point-track association module includes: a position determination unit, configured to determine the current position of each target in the target space for the target trajectories corresponding to each target in the target space; a distance determination unit, configured to determine the distances between the positions of each of the point cloud blocks and the current positions of each of the targets in the target space; a point-track association unit, configured to associate the point cloud blocks whose distances meet the set distance range with each of the target trajectories.

[0009] In an exemplary embodiment, the point-track association module further includes: a trajectory creation unit, configured to create a new target trajectory according to the point cloud block if the distance between the position of the point cloud block and the current position of each of the targets in the target space does not meet the set distance range.

[0010] In an exemplary embodiment, the feature extraction module includes: a point cloud acquisition unit, configured to determine a current point cloud block based on each of the point cloud blocks, and determine the previous several frames of point cloud data based on the historical point cloud data in the target trajectories associated with each of the point cloud blocks, and respectively select a set number of point clouds from the current point cloud block and the previous several frames of point cloud data; a feature extraction unit, configured to construct an intra-frame feature and an inter-frame feature according to the selected point clouds; a feature fusion unit, configured to perform feature fusion on the intra-frame feature and the inter-frame feature to obtain the point cloud feature of the current point cloud block; until the point cloud features of each of the point cloud blocks are all constructed, the point cloud features of each of the point cloud blocks are obtained.

[0011] In an exemplary embodiment, the feature extraction unit includes: an intra-frame feature extraction unit configured to calculate a plurality of intra-frame features of the target trajectory based on the point clouds respectively selected from each current point cloud block and several previous frames of point cloud data; each of the intra-frame features corresponds to a current point cloud block or a frame of point cloud data; an inter-frame feature extraction unit configured to construct a plurality of adjacent-frame point cloud data by respectively combining each frame of point cloud data in the several previous frames of point cloud data with the current point cloud block, and calculate a plurality of inter-frame features based on the point clouds respectively selected from the adjacent-frame point cloud data; each inter-frame feature corresponds to an adjacent-frame point cloud data.

[0012] In an exemplary embodiment, the trajectory update module includes: a point cloud discarding unit configured to discard the point clouds in the current point cloud block associated with the target trajectory according to a first condition if the recognition result of the target trajectory indicates that the current state of the target trajectory has not changed compared with the historical state; the current state of the target trajectory is recognized based on the current point cloud block associated with the target trajectory; the historical state of the target trajectory is recognized based on the historical point cloud data in the target trajectory associated with the current point cloud block; if the recognition result of the target trajectory indicates that the current state of the target trajectory has changed compared with the historical state, discard the point clouds in the current point cloud block associated with the target trajectory according to a second condition; a trajectory update unit configured to update the target trajectory in the target space according to the point clouds after the discarding process until the point clouds in each of the point cloud blocks associated with the target trajectory are completely discarded.

[0013] In an exemplary embodiment, the state recognition module is further configured to respectively input the point cloud features of each of the point cloud blocks into a classifier to respectively recognize the current states of the target trajectories corresponding to different point cloud blocks; the classifier is a machine learning model that has been trained and has the ability to recognize the states of the target trajectories; based on the current states of the target trajectories corresponding to different point cloud blocks, obtain the corresponding recognition results respectively; the recognition results are used to indicate whether the current state of the target trajectory corresponding to one of the point cloud blocks has changed compared with the historical state of the target trajectory.

[0014] In an exemplary embodiment, the device further includes: a model training module, which includes: a training data acquisition unit for acquiring a training data set, where the training data set includes training samples with labels; the labels are used to indicate whether the training samples belong to the sample target and the state of the target trajectory corresponding to the sample target; a model training unit for training a weak classifier according to each of the training samples in the training data set with a weight distribution in a current iterative training to obtain a classification error rate of the weak classifier on the training data set; calculating a weight of the weak classifier based on the classification error rate of the weak classifier, and updating the weight distribution of each of the training samples in the training data set; obtaining a plurality of the weak classifiers and corresponding weights through multiple iterative trainings; and obtaining a strong classifier from the plurality of weak classifiers and their corresponding weights as the classifier that has completed training when the training is completed.

[0015] In an exemplary embodiment, the training data acquisition unit includes: a data acquisition unit for acquiring multiple frames of point cloud data of the sample target in the target space; a position calculation unit for determining the position of the sample target at the current moment in the target space based on the currently acquired frame of point cloud data, and determining the position of the sample target at a historical moment in the target space based on the previously acquired several frames of point cloud data; a state determination unit for determining the current state of the target trajectory corresponding to the sample target according to the positions of the sample target at the current moment and the historical moment in the target space; a training data set unit for annotating the point cloud data of the sample target based on the current state of the target trajectory corresponding to the sample target to obtain a training sample corresponding to the sample target; and the training data set is composed of multiple training samples corresponding to at least one sample target.

[0016] In an exemplary embodiment, the device further includes: a device control module for determining the position of the target at the current moment in the target space based on the updated target trajectory; if the position of the target at the current moment in the target space meets the trigger condition in the automation scheme, controlling the intelligent device to execute the corresponding automation operation in the automation scheme; the automation scheme is used to realize the automatic control of the intelligent device when it is monitored that the position of the target in the target space meets the trigger condition.

[0017] According to one aspect of the present application, an electronic device includes at least one processor and at least one memory, wherein computer-readable instructions are stored on the memory; the computer-readable instructions are loaded and executed by the processor, so that the electronic device implements the trajectory update method as described above.

[0018] According to one aspect of the present application, a storage medium stores computer-readable instructions, which are loaded and executed by a processor to implement the trajectory update method as described above.

[0019] According to one aspect of the present application, a computer program product includes computer-readable instructions stored in a storage medium. A processor of an electronic device reads the computer-readable instructions from the storage medium, loads and executes the computer-readable instructions, so that the electronic device implements the trajectory update method as described above.

[0020] The beneficial effects brought by the technical solution provided by the present application are:

[0021] In the above technical solution, after clustering all the point clouds in the target space to obtain multiple point cloud blocks, they are associated with the existing target trajectories. Then, classification is performed according to the point cloud features between the point cloud blocks and the point clouds of the associated target trajectories. According to the recognition result, it is indicated whether the state of the associated target trajectory changes at different times, and the target trajectory in the target space is updated. Through the above steps, after clustering the point clouds in the space, one point cloud block is allowed to be associated with multiple target trajectories, ensuring that a certain number of point clouds are associated with each target trajectory, initially confirming the possibility of the association between the point cloud block and the target trajectory. Then, the state of each target trajectory is judged by classification, and according to the recognition result, it is confirmed whether the point cloud block really belongs to the associated target trajectory, thereby reducing the probability of target positioning and tracking errors or loss caused by incorrect point-track association, and solving the problem of low accuracy of target positioning in the target tracking process. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments of the present application.

[0023] Figure 1 is a schematic diagram of the implementation environment related to the present application;

[0024] Figure 2 is a flowchart of a trajectory update method shown according to an exemplary embodiment;

[0025] Figure 3 is a schematic diagram of point cloud clustering shown according to an exemplary embodiment;

[0026] Figure 4 is Figure 2 a flowchart of step 320 in the corresponding embodiment in one embodiment;

[0027] Figure 5 is a schematic diagram of a point-track association shown according to an exemplary embodiment;

[0028] Figure 6 is Figure 2 The flowchart of step 330 in the corresponding embodiment in one embodiment;

[0029] Figure 7 is Figure 6 The flowchart of step 333 in the corresponding embodiment in another embodiment;

[0030] Figure 8 The flowchart of a classifier training process shown according to an exemplary embodiment;

[0031] Figure 9 is Figure 8 The flowchart of corresponding step 410 in one embodiment;

[0032] Figure 10 is Figure 2 The flowchart of step 350 in the corresponding embodiment in one embodiment

[0033] Figure 11 The flowchart under an application scenario shown according to an exemplary embodiment;

[0034] Figure 12 The flowchart of another trajectory update method shown according to an exemplary embodiment;

[0035] Figure 13 The structural block diagram of a trajectory update device shown according to an exemplary embodiment;

[0036] Figure 14 The hardware structure diagram of an electronic device shown according to an exemplary embodiment;

[0037] Figure 15 The structural block diagram of an electronic device shown according to an exemplary embodiment. Detailed implementation manners

[0038] The embodiments of the present application are described in detail below. The examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements with the same or similar functions from beginning to end. The embodiments described below by referring to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be construed as a limitation to the present application.

[0039] Those skilled in the art can understand that, unless specifically stated otherwise, the singular forms "a", "an", "the" and "said" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of this application means the presence of the stated features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. It should be understood that when we say that an element is "connected" or "coupled" to another element, it can be directly connected or coupled to other elements, or there may also be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more of the associated listed items.

[0040] As mentioned above, in a multi-target scenario, the trajectories between targets will affect each other and even intersect, resulting in interference during the target trajectory tracking process and causing errors in the positioning of the targets.

[0041] For example, in a smart home, millimeter-wave radar is usually used to track and position targets in the detection area. However, for millimeter-wave radar, when there are trajectories generated by multiple targets in the scene, due to the interlacing of the trajectories, it is easy to track the target trajectory incorrectly or lose track, resulting in incorrect device control.

[0042] Specifically, in the related art, the process of target positioning and tracking is as follows: First, cluster the point clouds in space to obtain multiple point cloud blocks, then associate each of the clustered point cloud blocks with the trajectory closest to it in space, and finally update the associated trajectory based on the point clouds in the point cloud block associated with the trajectory.

[0043] However, when there are trajectories of multiple targets in space, through the clustering and association method, it may associate point cloud blocks that do not belong to the trajectory with it; at the same time, since each point cloud block is only allowed to be associated with one trajectory, some trajectories are associated with too many point cloud blocks, and some trajectories are not associated with any point cloud blocks, resulting in incorrect tracking or loss of some targets.

[0044] As can be seen from the above, there is still a problem of low accuracy in multi-target tracking and positioning in the related art.

[0045] To this end, the trajectory update method provided in this application can effectively improve the accuracy of target tracking and positioning. Correspondingly, the trajectory update method is applicable to a trajectory update device, which can be deployed on an electronic device. The electronic device can be an intelligent device with the ability to detect targets. For example, the intelligent device includes a human body sensor configured with a millimeter-wave radar, etc.; the electronic device can be a computer device configured with a von Neumann architecture. For example, the computer device includes a desktop computer, a laptop computer, a server, etc.; the electronic device can also be an electronic device with a central control function. For example, the electronic device includes a gateway, etc.

[0046] To make the objectives, technical solutions, and advantages of this application clearer, the following will further describe the embodiments of this application in detail with reference to the accompanying drawings.

[0047] Figure 1 It is a schematic diagram of the implementation environment involved in a trajectory update method. The implementation environment at least includes a user terminal 110, an intelligent device 130, a server 170, and a network device. In Figure 1 it, the network device includes a gateway 150 and a router 190, but this is not a specific limitation here.

[0048] Among them, the user terminal 110, which can also be regarded as a user side or a terminal, can deploy (or understand as install) the client associated with the intelligent device 130. This user terminal 110 can be an electronic device such as a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart control panel, and other devices with display and control functions, and is not limited here.

[0049] Among them, the client, which is associated with the intelligent device 130, actually means that the user registers an account in the client and configures the intelligent device 130 in the client. For example, the configuration includes adding a device identifier to the intelligent device 130, etc., so that when the client runs in the user terminal 110, it can provide functions such as device display and device control for the user. This client can be in the form of an application program or in the form of a web page. Correspondingly, the interface for the client to display the device can be in the form of a program window or in the form of a web page, and this is not limited here either.

[0050] The intelligent device 130 is deployed in the gateway 150 and communicates with the gateway 150 through its configured communication module, and thus is controlled by the gateway 150. It should be understood that the intelligent device 130 generally refers to one of multiple intelligent devices 130. In the embodiments of the present application, only the intelligent device 130 is taken as an example for illustration. That is, the embodiments of the present application do not limit the number and device type of the intelligent devices deployed in the gateway 150. In an application scenario, the intelligent device 130 accesses the gateway 150 through a local area network, and thus is deployed in the gateway 150. The process by which the intelligent device 130 accesses the gateway 150 through the local area network includes: the gateway 150 first establishes a local area network, and the intelligent device 130 joins the local area network established by the gateway 150 by connecting to the gateway 150. The local area network includes but is not limited to: ZIGBEE or Bluetooth. Among them, the intelligent device 130 can be an intelligent printer, an intelligent fax machine, an intelligent camera, an intelligent air conditioner, an intelligent door lock, an intelligent lamp, or a human body sensor, a door and window sensor, a temperature and humidity sensor, a water immersion sensor, a natural gas alarm, a smoke alarm, a wall switch, a wall socket, a wireless switch, a wireless wall sticker switch, a magic cube controller, a curtain motor, a millimeter wave radar, etc. configured with a communication module.

[0051] The interaction between the user terminal 110 and the intelligent device 130 can be realized through a local area network or through a wide area network. In an application scenario, the user terminal 110 establishes a wired or wireless communication connection with the gateway 150 through the router 190. For example, the wired or wireless method includes but is not limited to WIFI, etc., so that the user terminal 110 and the gateway 150 are deployed in the same local area network, and thus the user terminal 110 can realize the interaction with the intelligent device 130 through the local area network path. In another application scenario, the user terminal 110 establishes a wired or wireless communication connection with the gateway 150 through the server side 170. For example, the wired or wireless method includes but is not limited to 2G, 3G, 4G, 5G, WIFI, etc., so that the user terminal 110 and the gateway 150 are deployed in the same wide area network, and thus the user terminal 110 can realize the interaction with the intelligent device 130 through the wide area network path.

[0052] Among them, the server side 170 can also be considered as the cloud, the cloud platform, the platform side, the service side, etc. This server side 170 can be a single server, or a server cluster composed of multiple servers, or a cloud computing center composed of multiple servers, so as to better provide background services for a large number of user terminals 110. For example, the background services include trajectory update services.

[0053] In an application scenario, the intelligent device 130 acquires point cloud data in the target space, clusters the acquired point cloud data to obtain multiple point cloud blocks, associates each point cloud block with at least one target trajectory in the target space, constructs point cloud features of each target trajectory according to the point cloud blocks respectively associated with each target trajectory, classifies the states of each target trajectory at different times according to the point cloud features of each target trajectory to obtain the recognition results of each target trajectory, the recognition results are used to indicate whether the states of each target trajectory change at different times, and according to the recognition results of each target trajectory, each target trajectory in the target space is updated, and then an automated configuration operation of the intelligent device is performed on the automated configuration page of the intelligent device corresponding to the user terminal 110.

[0054] Of course, in other application scenarios, the above process of updating the target trajectory completed by the intelligent device 130 can also be implemented by the server 170 / gateway 150. At this time, the intelligent device 130 reports the point cloud data in the target space to the server 170 / gateway 150 for updating the target trajectory. After the server 170 / gateway 150 updates the target trajectory, the updated target trajectory is sent to the user terminal 110 through the wide area network path / local area network path, so that the user terminal 110 can perform an automated configuration operation of the intelligent device on the automated configuration page corresponding to the target trajectory.

[0055] Please refer to Figure 2 , this application provides a trajectory update method, which is applicable to any electronic device with processing capabilities. For example, the electronic device can be Figure 1 the intelligent device 130, gateway 150, user terminal 110, server 170, etc. in the shown implementation environment, and this application does not make any limitations in this regard.

[0056] In the following method embodiments, for the sake of description, the execution subject of each step of the method is taken as an electronic device as an example for illustration, but this is not a limitation thereto.

[0057] As Figure 2 shown, the method may include the following steps:

[0058] Step 310, perform clustering processing on the acquired point cloud data in the target space to obtain multiple point cloud blocks.

[0059] Among them, the target space refers to the area range where the detection device can effectively detect the target. The target refers to any object that can be tracked within the target space. For example, the object can be a person, a pet, a robot, etc. The point cloud data under the target space is obtained by a smart device with detection capabilities detecting the target within the target space. It should be noted that the smart device with detection capabilities includes, but is not limited to, a human body sensor equipped with a millimeter-wave radar. Correspondingly, the target space can be considered as the area range where the smart device can effectively detect the target.

[0060] Taking a human body sensor equipped with a millimeter-wave radar as an example, the millimeter-wave radar is configured with an antenna array, and the antenna array includes a transmitting antenna and a receiving antenna. During the process of the human body sensor detecting the target within the target space, it will transmit a millimeter-wave signal (also considered a sensing signal) to the target through the transmitting antenna, and receive the echo signal (also considered a reflected signal) formed by the millimeter-wave signal reflected by the target through the receiving antenna. By performing relevant processing on the received echo signal, the point cloud data of the target can be determined. Here, the target space refers to the range where the transmitting antenna of the millimeter-wave radar can effectively transmit millimeter-wave signals.

[0061] Optionally, performing relevant processing on the received echo signal and determining the point cloud data of the target may include the following steps:

[0062] S1. Based on the mathematical model X = AS + N corresponding to the echo signal, determine the time-domain signal S.

[0063] Among them, the time-domain signal S is a signal related to distance, speed, and angle, expressed as f(R, v, θ).

[0064] S2. Perform a fast Fourier transform (FFT) on the time-domain signal S to obtain the frequency-domain signal S’, expressed as F(R, v, θ).

[0065] S3. Use the formula P(R, v, θ) = |F(R, v, θ)| to solve the power spectrum of the frequency-domain signal S’ and obtain the power spectrum signal, expressed as P(R, v, θ).

[0066] S4. Take the peak value of the power spectrum signal as the movement data of the target, expressed as (R, v, θ).

[0067] Among them, R represents distance, v represents speed, and θ represents angle.

[0068] Among them, the moving data is the position of the target represented in the polar coordinate system within the target space. To more accurately and effectively describe the position of the target within the target space, the above-mentioned moving data is converted to the rectangular coordinate system, thereby obtaining the point cloud (X, Y) of the target within the target space, where X = Rsin(θ) and Y = Rcos(θ). That is to say, the point cloud can be represented by the coordinates (X, Y) in the rectangular coordinate system, and this coordinate represents the position of the target in the target space.

[0069] After the above process, the point cloud data of the target can be obtained based on multiple point clouds (X, Y) of the target in the target space.

[0070] In this embodiment, after obtaining all the point cloud data in the target space, multiple point cloud blocks can be obtained by clustering according to all the point cloud data. Among them, clustering refers to finding out the point clouds with the same type in the point cloud data. These point clouds with the same type will form clusters, and the clusters can be regarded as the point cloud blocks in this application. That is to say, the point cloud block includes several point clouds with the same type.

[0071] It should be noted here that how to determine whether the point clouds have the same type is based on a specific clustering algorithm. For example, for the DBSCN algorithm, it is based on density to measure whether each point cloud belongs to the point cloud of the same type, which is not limited here.

[0072] In a possible implementation manner, clustering the obtained point cloud data to obtain point cloud blocks is implemented based on a clustering algorithm, and the clustering algorithm includes but is not limited to: one or more of the DBSCN (Density-Based Spatial Clustering of Applications with Noise) algorithm, the K-MEANS (k-means clustering algorithm) algorithm, the Mean-shift mean shift algorithm, and the AP (Affinity Propagation) algorithm.

[0073] Taking the DBSCN algorithm as an example, but not specifically limited thereto. Specifically, the method scans all the point clouds in the target space. If the number of point clouds within the radius R of a certain point cloud P is ≧ MinPoints (the number of core points), the point cloud P is included in the core point list, and a temporary clustering cluster with the point cloud P as the core point is created. For each temporary clustering cluster, check whether the points therein are core points. If so, merge the temporary clustering cluster corresponding to the point cloud and the current temporary clustering cluster to obtain a new temporary clustering cluster; repeat the above operation until each point cloud in the current temporary clustering cluster is either not in the core point list or all the points that its density reaches are already in the temporary clustering cluster, and the temporary clustering cluster is upgraded to a clustering cluster; iterate through all the temporary clustering clusters to perform the same merging operation until all the temporary clustering clusters are processed.

[0074] As Figure 3 shown, set MinPoints to 4. Through iteration, it is found that the point cloud A meets the condition for creating the core point P. At the same time, through continuous iteration, it is also found that the point clouds B and C also meet the requirements. However, the point clouds B and C are on the boundary of the entire clustering cluster, while the point cloud N can be considered as an edge point cloud and does not meet the condition for creating the core point P. Therefore, Figure 3 in the figure, the point cloud A is the core point, and thus the point clouds A, B, and C are clustered into a point cloud block with the point cloud A as the core point.

[0075] After the above process, the point cloud data of all targets in the target space can be obtained, and the obtained point cloud data is clustered to obtain several point cloud blocks. It is worth mentioning that the acquisition of point cloud data can be realized frame by frame. For example, the point cloud data obtained here can be considered as the current frame point cloud data. Correspondingly, several point cloud blocks clustered from the current frame point cloud data can be considered as the current frame point cloud blocks, and the target trajectory updated based on the current frame point cloud data will be able to determine the position of the target at the current moment in the target space.

[0076] Step 320: Associate each point cloud block with the target trajectory corresponding to each target in the target space.

[0077] As mentioned above, when multiple targets are relatively close, the point clouds generated by each target will be mixed together. According to the clustering, the point cloud blocks of different targets will be treated as a whole, and in the prior art, each block of point cloud only allows one trajectory to be associated, resulting in some trajectories being assigned too many point clouds and some trajectories not being assigned any point cloud data, ultimately leading to incorrect target tracking or loss of the target.

[0078] Therefore, the trajectory update method provided in this application allows several point cloud blocks obtained by clustering to be associated with all the trajectories in the space, ensuring that each trajectory will be associated with a certain number of point clouds, thereby reducing the situations of incorrect target tracking and loss of the target.

[0079] Step 330: Based on each point cloud block and the historical point cloud data in the target trajectory associated with each point cloud block, construct the point cloud features of each point cloud block.

[0080] Among them, the point cloud features accurately describe the position, speed and other characteristics of the target trajectory reflected by different point cloud blocks in a digital manner. It can be understood that for different point cloud blocks, the corresponding point cloud features will also be different, that is, the point cloud features can be used to uniquely identify the point cloud blocks, so as to judge whether the state of the target trajectory after point-track association has changed due to the point cloud block.

[0081] Since this application allows several point cloud blocks obtained by clustering to be associated with all trajectories in the space, then, on the premise that several point cloud blocks are associated with some trajectories in the space as soon as possible, there may be a situation where a point cloud block that does not belong to the target trajectory is wrongly associated with it. If the point cloud block does not really belong to the target trajectory, it will cause the current state of the associated target trajectory to change due to the point cloud block, for example, the current state of the target trajectory changes compared with the historical state.

[0082] Among them, the current state is related to the point cloud blocks obtained by clustering the current frame of point cloud data, and the historical state is related to the historical point cloud data in the target trajectory. Therefore, before updating the target trajectory, it is necessary to construct the point cloud features of each point cloud block according to the point cloud blocks obtained by clustering and the historical point cloud data in the target trajectories of each target in the target space, so as to judge whether the state of the target trajectory after point-track association has changed due to a certain point cloud block, and finally exclude the point cloud blocks that do not belong to the target trajectory.

[0083] In a possible implementation manner, the construction methods of the point cloud features include but are not limited to: feature extraction algorithms based on deep learning (such as feature extractors), distance-based feature extraction algorithms, and space mapping-based feature extraction algorithms.

[0084] In a possible implementation manner, the construction method of the point cloud features of the target trajectory specifically refers to constructing intra-frame features and inter-frame features based on each point cloud block and the historical point cloud data in the target trajectory associated with each point cloud block, and obtaining the point cloud features of each point cloud block from the intra-frame features and inter-frame features.

[0085] Step 340: Identify the states of each target trajectory according to the point cloud features of each point cloud block, and obtain the corresponding identification results respectively.

[0086] Among them, each identification result is used to indicate the current state change of each target trajectory. It should be noted that the state of the target trajectory can include two types: the first state and the second state. In the embodiments of this application, the first state can be stationary and the second state can be moving.

[0087] Accordingly, the state changes of the target trajectory can be divided into two categories: positive class and negative class. Among them, the positive class, that is, no change occurs. The positive class can include two types: remaining in the first state unchanged and remaining in the second state unchanged. For example, the state of the target trajectory remains stationary at different times, or the state of the target trajectory remains in motion at different times.

[0088] Among them, the negative class, that is, a change occurs. The negative class can also include two types: the first state changes to the second state, and the second state changes to the first state. For example, the state of the target trajectory changes from motion at the previous moment to stationary at the current moment, or the state of the target trajectory changes from stationary at the previous moment to motion at the current moment.

[0089] In a possible implementation, the recognition of the state of each target trajectory is achieved through classification. This classification can be implemented using a classification algorithm. For example, classification algorithms include Bayesian algorithm, decision tree classification method, classifier based on support vector machine, K-nearest neighbor method, fuzzy classification method, neural network model, etc., which are not limited here.

[0090] In a possible implementation, the classification process may include: First, calculate the probabilities of the target trajectory belonging to different states; then, determine the current state of the target trajectory according to the probabilities of the target trajectory belonging to different states; after determining the current state of the target trajectory, the corresponding recognition result can be further obtained in combination with the historical state of the target trajectory. Among them, the historical state of the target trajectory is recognized based on the historical point cloud data in the target trajectory.

[0091] For example, assume that the state categories of the target trajectory include a motion state and a stationary state. P1 represents the probability that the current state of the target trajectory is the motion state, and P2 represents the probability that the current state of the target trajectory is the stationary state. If P1 > P2, it means that the current state of the target trajectory is the motion state; conversely, if P1 < P2, it means that the current state of the target trajectory is the stationary state. That is, the current state of the target trajectory can be recognized based on the current frame point cloud data of the target trajectory.

[0092] Assume that the historical states of the target trajectory identified based on the historical point cloud data of the first two frames of the target trajectory are both in motion. If the current state of the target trajectory is in motion, then the states of the target trajectory identified from three consecutive frames of point cloud data are all in motion. In this case, the recognition result of the target trajectory is a positive class, that is, the current state of the target trajectory has not changed compared to the historical state. Conversely, if the current state of the target trajectory is stationary, then the states of the target trajectory are different in three consecutive frames. In this case, the recognition result of the target trajectory is a negative class, that is, the current state of the target trajectory has changed compared to the historical state. Of course, in other embodiments, the number of frames of historical point cloud data used for historical state recognition can be flexibly adjusted according to the actual needs of the application scenario, and this is not a specific limitation here.

[0093] Step 350: Update the target trajectories of each target according to the recognition results of each target trajectory.

[0094] After obtaining the recognition results of each target trajectory, the point cloud blocks that do not belong to the target trajectory can be excluded one by one according to the state change of each target trajectory, so as to ensure the accuracy of target trajectory update.

[0095] It should be noted here that the target trajectory indicates the positions of the target at different times in the target space. Specifically, the target trajectory records the positions obtained from the point cloud data of the frame obtained at the trajectory creation time until the position obtained from the previous frame of point cloud data of the current frame. Then, the update of the target trajectory is essentially to use the current frame of point cloud data that excludes the point cloud blocks that do not belong to the target trajectory to update the position of the target at the current moment in the target space.

[0096] Through the above steps, after clustering the point clouds in the space, a point cloud block is allowed to be associated with multiple target trajectories, ensuring that each target trajectory is associated with a certain number of point clouds, initially confirming the possibility of the association between the point cloud block and the target trajectory, and then judging the state of each target trajectory through classification, and confirming whether the point cloud block really belongs to the associated target trajectory according to the recognition result, thereby reducing the probability of target positioning and tracking errors or loss caused by incorrect point-trajectory association, and solving the problem of low accuracy of target positioning in the target tracking process.

[0097] In an exemplary embodiment, as Figure 4 shown, step 320 may include the following steps:

[0098] Step 321: For the target trajectories corresponding to each target in the target space, determine the current positions of each target in the target space.

[0099] Among them, the current position is calculated based on the previous frame of point cloud data excluding the point cloud blocks that do not belong to the target trajectory. Specifically, the mean value is calculated for all the point clouds included in the previous frame of point cloud data in the target trajectory to obtain the current position of the target corresponding to the target trajectory in the target space.

[0100] It should be noted that, as described above, the target trajectory indicates the positions of the target at different times in the target space. Then, the current position refers to the position of the target at the previous moment in the target space obtained based on the previous frame of point cloud data. This current position is different from the position of the target at the current moment in the target space obtained based on the current frame of point cloud data. And both the position at the previous moment and the position at the current moment will be used to record and form the target trajectory.

[0101] As Figure 5 shown, there are trajectory 1 and trajectory 2 in the target space. Among them, the current position of target 1 in the target space is A, and the current position of target 2 in the target space is B.

[0102] Step 323, determine the distances between the positions of each point cloud block and the current positions of each target in the target space.

[0103] The inventor realizes that although this application allows multiple point cloud blocks to be associated with the existing target trajectories, for point cloud blocks that are far apart, when they are associated with the target trajectory, it will instead cause inaccurate updates of the target's trajectory. Therefore, before performing point-trajectory association, this application first screens each point cloud block according to the distances between the positions of each point cloud block and the current positions of each target in the target space. Only the point cloud blocks that meet the set distance range will be associated with the target trajectory, thereby improving the efficiency of point-trajectory association and further improving the efficiency of trajectory update.

[0104] After determining the current positions A and B of each target in the target space, as Figure 5 shown, for the point cloud blocks obtained in step 310, such as point cloud block 1, point cloud block 2, and point cloud block 3, taking the centers of each point cloud block as the positions of each point cloud block in the target space, which are A', B', and C' respectively, and calculate the distances d between the positions of each point cloud block and the current positions of each target ij . Among them, i is the serial number of the target trajectory, j is the serial number of the point cloud block, and both i and j are integers greater than 0. For example, d11 represents the distance between the current position A of the target in the target space under target trajectory 1 and the position A' of point cloud block 1, and d12 represents the distance between the current position B of the target in the target space under target trajectory 1 and the position B' of point cloud block 2.

[0105] Step 325, associate the point cloud blocks whose distances meet the set distance range with each target trajectory.

[0106] Taking the set distance range of 1 meter as an example, if the distance d between the position of each point cloud block and the current position of each target in the target space ij is less than 1 meter, the point cloud block is associated with the corresponding target trajectory used to determine the current position.

[0107] In one embodiment, the above trajectory update method further includes:

[0108] Step 327, if the distance between the position of the point cloud block and the current position of the target in the target space does not meet the set distance range, a new target trajectory is created according to the point cloud block.

[0109] Specifically, if the distances d between all the calculated point cloud blocks and the target trajectory ij do not satisfy the set distance range, indicating that all the point cloud blocks do not belong to the target trajectory, a new target trajectory is created according to the point cloud data in all the point cloud blocks as the updated trajectory of the target in the target space.

[0110] Through the above steps, each point cloud block in the target space is associated with different target trajectories, avoiding the situation that each point cloud block is only allowed to belong to one target trajectory, resulting in some target trajectories being assigned too many point clouds while some target trajectories are not assigned any point clouds. In this way, it is ensured that each target trajectory will be associated with a certain number of point clouds, improving the accuracy of trajectory update and thus enhancing the accuracy of target tracking.

[0111] In an exemplary embodiment, as Figure 6 shown, the above step 330 may include the following steps:

[0112] Step 331, determine the current point cloud block based on each point cloud block, and determine the previous several frames of point cloud data based on the historical point cloud data in the target trajectory associated with each point cloud block, and select a set number of point clouds from the current point cloud block and the previous several frames of point cloud data respectively.

[0113] Among them, the previous several frames of point cloud data, relative to the current frame of point cloud data, include at least one frame of point cloud data. For example, if the point cloud data obtained in step 310 is the current frame of point cloud data for the millimeter-wave radar and can be considered as the third frame of point cloud data, then the first frame of point cloud data and the second frame of point cloud data are regarded as the previous several frames of point cloud data obtained from the historical point cloud data.

[0114] After obtaining the set number of point clouds, the feature construction of the point cloud block can be carried out.

[0115] Specifically, first, select the first several frames of point cloud data for feature construction from the historical point cloud data in the target trajectory, and select a set number of point clouds. For example, select the previous frame of point cloud data pc from the historical point cloud data in the target trajectory 1 and extract 10 point clouds from it in descending order of signal intensity of the point clouds (if there are less than 10 point clouds in pc 1 , then copy them to 10 in descending order of signal intensity of the existing point clouds). Select the second previous frame of point cloud data pc from the historical point cloud data in the target trajectory 2 and extract 10 point clouds from it in descending order of signal intensity of the point clouds (if there are less than 10 point clouds in pc 2 , then copy them to 10 in descending order of signal intensity of the existing point clouds).

[0116] Then, select the current point cloud block from each point cloud block, and select a set number of point clouds from the current point cloud block. For example, for the point cloud block n 1 , n 2 , n 3 ,..., n N1 , determine the current point cloud block as the point cloud block n i , and extract 10 point clouds from this point cloud block n i in descending order of signal intensity of the point clouds (if there are less than 10 point clouds in n i , then copy them to 10 in descending order of signal intensity of the existing point clouds).

[0117] Of course, in other embodiments, the selection method of the set number of point clouds can also be to select them from far to near based on the distance between the point clouds and the target trajectory, or randomly select them, and the set number can also be flexibly adjusted according to the actual needs of the application scenario. The set number is at least 2, and this embodiment does not constitute a specific limitation.

[0118] Step 333, construct intra-frame features and inter-frame features according to the selected point clouds.

[0119] Among them, the intra-frame feature refers to the feature of a specific frame of point cloud data, which is used to describe the characteristics of this specific frame of point cloud data; the inter-frame feature refers to the feature of adjacent frame point cloud data, which is used to describe the relationship between adjacent several frames of point cloud data. It can be understood that the adjacent frame point cloud data can be at least two adjacent frames of point cloud data. For example, the adjacent frames can be two adjacent frames, or 3 consecutive adjacent frames, 5 frames, 10 frames, 15 frames, etc. This application does not make a limitation on this.

[0120] For example, adjacent frame point cloud data can be constructed from the current frame point cloud data and the previous frame point cloud data. Therefore, the inter-frame features reflect the relationship between adjacent frame point cloud data; alternatively, it can also be constructed from the current frame point cloud data and the point cloud data of the previous two frames. Therefore, the inter-frame features can reflect the relationship between adjacent multiple frame point cloud data.

[0121] Specifically, as Figure 7 shown, step 333 may include the following steps:

[0122] Step 3331, calculate a plurality of intra-frame features based on the point clouds respectively selected from the current point cloud block and the point cloud data of the previous several frames.

[0123] Among them, the intra-frame features correspond to the current point cloud block or a frame of point cloud data.

[0124] For example, subtract every two of the 10 point clouds extracted from the current point cloud block n i to obtain the intra-frame feature f i corresponding to the current point cloud block n 1 ; subtract every two of the 10 point clouds extracted from the previous frame of point cloud data pc 1 to obtain the intra-frame feature f 1 corresponding to the previous frame of pc 2 point cloud data; subtract every two of the 10 point clouds extracted from the point cloud data of the previous second frame pc 2 to obtain the intra-frame feature f 2 corresponding to the point cloud data of the previous second frame pc 3 .

[0125] Step 3333, construct a plurality of adjacent frame point cloud data by respectively combining each frame of point cloud data in the previous several frames of point cloud data with the current point cloud block, and calculate a plurality of inter-frame features based on the point clouds respectively selected from each adjacent frame point cloud data.

[0126] Among them, each inter-frame feature corresponds to an adjacent frame point cloud data.

[0127] For example, subtract every two of the 10 point clouds extracted from the current point cloud block n i from the 10 point clouds extracted from the previous frame of point cloud data pc 1 to obtain the inter-frame feature f 4 .

[0128] Subtract every two of the 10 point clouds extracted from the current point cloud block n i from the 10 point clouds extracted from the point cloud data of the previous second frame pc 2 to obtain the inter-frame feature f 5 .

[0129] It is worth mentioning that the first several frame point cloud data for inter-frame construction can be the previous frame point cloud data pc 1 , or can also be the point cloud data pc 1 and pc 2 of the previous two frames, and no specific limitation is formed here.

[0130] Step 335: Perform feature fusion on the intra-frame features and inter-frame features to obtain the point cloud features of the current point cloud block.

[0131] Optionally, according to the obtained intra-frame features and inter-frame features, one or more of the methods of splicing, adding, multiplying, pooling, and deconvolution are used for fusion, and the point cloud features of the current point cloud block can be fused.

[0132] For example, after obtaining the intra-frame features f 1 , f 2 , f 3 and the inter-frame features f 4 , f 5 , fusion is performed by the splicing method, and the point cloud features of the current point cloud block can be fused, that is, {f 1 , f 2 , f 3 , f 4 , f 5}. Based on this, the point cloud features of each point cloud block are constructed according to the above steps until the feature construction of each point cloud block is completed, and the point cloud features of each point cloud block can be obtained.

[0133] Through the above process, the point cloud features of each point cloud can be constructed according to the point cloud blocks respectively associated with each target trajectory and the historical point cloud data in each target trajectory, so as to be used as the basis for whether the state of the target trajectory has changed, so that the point cloud blocks that do not belong to the target trajectory can be discarded, and thus it is beneficial to improve the accuracy of target tracking and positioning.

[0134] In an exemplary embodiment, the process of identifying the state of each target trajectory according to the point cloud features of each point cloud block in step 340 above is implemented based on a classifier.

[0135] Among them, the classifier is a machine learning model that has been trained and has the ability to identify the state of the target trajectory.

[0136] Then, the recognition process implemented based on the classifier may include the following steps: input the point cloud features of each point cloud block into the classifier respectively to identify the current states of each target trajectory corresponding to different point cloud blocks; based on the current states of each target trajectory corresponding to different point cloud blocks, obtain the corresponding recognition results respectively.

[0137] Among them, the recognition result is used to indicate whether the current state of the target trajectory corresponding to one of the point cloud blocks has changed compared to the historical state of the target trajectory.

[0138] By using the trained classifier to recognize the states of the target trajectories, the states of the target trajectories can be recognized more accurately, and thus the target trajectories of the targets can be updated more accurately and efficiently.

[0139] In an exemplary embodiment, as Figure 8 shown, the training process of the above classifier may include the following steps:

[0140] Step 410, obtain a training data set.

[0141] Among them, the training data set includes training samples with labels, and the labels are used to indicate whether the training samples belong to the sample target and the state of the target trajectory corresponding to the sample target.

[0142] Specifically, it is represented as follows:

[0143] n1 1 ,n1 2 ,……n1 k1 ;

[0144] n2 1 ,n2 2 ,……n2 k2 ;

[0145] n3 1 ,n3 2 ,……n3 k3 ;

[0146] Among them, the subscript represents the number of the point cloud in the training sample, and the values in n1, n2, and n3 represent the numbers of the training samples.

[0147] For example, n1 represents the first training sample, n11 represents the first point cloud in the first training sample, and the first training sample contains k1 point clouds.

[0148] Specifically, as Figure 9 shown, the generation process of each training sample in the training data set may include the following steps:

[0149] Step 411, collect multiple frames of point cloud data of the sample target in the target space.

[0150] When the sample target is a person, the collection process includes multiple frames of point cloud data of the person in various states. For example, the states include, but are not limited to, one or more of movement, stillness, sitting, standing, and lying.

[0151] Step 413: Based on the currently acquired point cloud data of the frame, determine the position of the sample target at the current moment in the target space, and based on the previously acquired point cloud data of several frames, determine the position of the sample target at historical moments in the target space.

[0152] Among them, the position of the sample target at historical moments in the target space is calculated based on the previously acquired point cloud data of several frames. Specifically, the average value of the positions of the first 15 frames of point cloud data in the target trajectory corresponding to the target is used as the position of the sample target at historical moments in the target space. Similarly, the position of the sample target at the current moment in the target space is calculated based on the currently acquired point cloud data.

[0153] Step 415: Based on the positions of the sample target at the current moment and historical moments in the target space, determine the current state of the target trajectory corresponding to the sample target.

[0154] If the distance between the positions of the sample target at the current moment and historical moments in the target space satisfies the set distance range (such as 0.5 meters), it is considered that the current state of the target trajectory corresponding to the sample target is stationary.

[0155] If the distance between the positions of the sample target at the current moment and historical moments in the target space does not satisfy the set distance range, it is considered that the current state of the target trajectory corresponding to the sample target is moving.

[0156] Step 417: Based on the current state of the target trajectory corresponding to the sample target, label the point cloud data of the sample target to obtain the training samples corresponding to the sample target.

[0157] The labeling process includes: based on each point cloud in the acquired point cloud data, respectively calibrate the point cloud belonging to a person, the point cloud belonging to the same person, and whether the state of the target trajectory corresponding to the person is stationary or moving, so that the training sample carries the corresponding label.

[0158] Step 419: A training data set is composed of multiple training samples corresponding to at least one sample target.

[0159] The above steps can obtain different training samples of a sample target. For several sample targets, different training samples are collected respectively, and a training data set containing several different sample targets can be obtained.

[0160] After obtaining the training data set, the classifier can be trained based on each training sample in the training data set, that is, steps 430 and 490 are executed.

[0161] Step 430: In the current iteration training, train a weak classifier according to each training sample in the training data set with a weight distribution, and obtain the classification error rate of the weak classifier on the training data set.

[0162] In this embodiment, in order to enable the classifier to have the ability to recognize the state of the target trajectory, the essence of training the classifier is the positive and negative sample features constructed based on each training sample. Among them, the positive sample features are used to describe that the current state of the target trajectory corresponding to the sample target has not changed compared with the historical state, while the negative sample features are used to describe that the current state of the target trajectory corresponding to the sample target has changed compared with the historical state.

[0163] Taking the previous example for illustration, assume that the training data set includes three training samples n1, n2, and n3.

[0164] If the labels carried by the training samples n1, n2, and n3 indicate that the states of the target trajectories corresponding to the targets are all in the moving state or the stationary state, then positive sample features are constructed based on the training samples n1, n2, and n3 according to step 333.

[0165] If the labels carried by the training samples n1 and n2 indicate that the states of the target trajectories corresponding to the targets are all in the moving state or the stationary state, while the label carried by the training sample n3 indicates that the state of the target trajectory corresponding to the target is in the stationary state or the moving state, then negative sample features are constructed based on the training samples n1, n2, and n3 according to step 333.

[0166] And so on, based on the labels carried by the training samples n1, n2, and n3 indicating the states of the target trajectories corresponding to the targets, at least one positive sample feature or negative sample feature for classifier training can be constructed.

[0167] After completing the construction of the positive and negative sample features based on the training samples n1, n2, and n3, the initial classifier can be trained using the constructed positive and negative sample features.

[0168] Taking the adaboost (Adaptive Boosting) algorithm as an example, the training of the classifier is completed based on the classification error rates of different weak classifiers on the same training data set. Among them, the classification error rate of the weak classifier is used to describe the proportion of classification errors of the weak classifier for each training sample in the training data set. It can be understood that the larger the proportion, the higher the error rate of the weak classifier and the worse the training effect.

[0169] Specifically, according to the following formula (1), the weight distribution of each training sample in the training data set is initialized:

[0170]

[0171] Among them, D1 represents the weight distribution of each training sample in the training data set during the first training, and W 1i represents the weight of the i-th training sample during the first training.

[0172] In the current (the m-th) iteration training, according to the following formula (2), use the training data set with the weight distribution D m (i.e., the weight distribution of each training sample in the training data set during the m-th training) to train the (m - 1)-th weak classifier, and calculate the classification error rate e m of the m-th weak classifier G m .

[0173]

[0174] where m = 1, 2, 3, …… M, m represents the iteration number, i.e., the m-th training, N represents the number of training samples, W m,i represents the weight of the i-th training sample x i during the m-th training, y i is the label carried by the i-th training sample x i , G m (x i ) represents the classification prediction value of the m-th weak classifier for the i-th training sample x i , I represents the difference calculation function, and I is 1 when G m (x i ) ≠ y i is satisfied, otherwise I is 0.

[0175] According to the following formula (3), based on the classification error rate e m of the m-th weak classifier G m (x), calculate the weight α m of the m-th weak classifier G m :

[0176]

[0177] Continuing to take the adaboost algorithm as an example, continue to refer to Figure 8 , the above training process may further include the following steps:

[0178] Step 450, calculate the weight of the weak classifier based on the classification error rate of the weak classifier, and update the weight distribution of each training sample in the training data set.

[0179] Among them, whether the training is completed is based on whether the classification error rate of the weak classifier meets the set model convergence condition. The set model convergence condition can be that the classification error rate meets the set value, or the number of iterations reaches the set value, which is not limited here.

[0180] Specifically, taking the set model convergence condition as the classification error rate being 0.05, if the calculated classification error rate e of the m-th weak classifierm If it is greater than 0.05, then according to the following formulas (4) and (5), update the weight distribution of each training sample in the training dataset:

[0181]

[0182]

[0183] where, W m+1,i represents the weight of the i-th training sample x i at the (m + 1)-th training, that is, the updated weight, N represents the number of training samples, and W m,i represents the weight of the i-th training sample x i at the m-th training, y i is the label carried by the i-th training sample x i , G m (x i ) represents the classification prediction value of the m-th weak classifier regarding the i-th training sample x i , and Z m represents the normalization coefficient of the m-th training.

[0184] Step 470: Through multiple iterative trainings, obtain multiple weak classifiers and their same weights.

[0185] Among them, the process of multiple iterative trainings is the same as the process of obtaining the weight of the m-th weak classifier in the above current iterative training, and will not be elaborated here.

[0186] Step 490: When the training is completed, obtain a strong classifier from multiple weak classifiers and their corresponding weights as the classifier that has completed training.

[0187] Specifically, set the model convergence condition as the classification error rate being 0.05. If the calculated classification error rate e m of the m-th weak classifier is not greater than 0.05, then according to each weak classifier and its corresponding weight, obtain a strong classifier, that is, the classifier F(x) that has completed training.

[0188] Calculate the classifier F(x) that has completed training according to the following formula (6).

[0189]

[0190] where, G m (x) represents the m-th weak classifier, the weight α m represents the weight of the m-th weak classifier, and sign represents a linear function.

[0191] Under the action of the above embodiments, the training of the classifier is realized, so that the trained classifier has the ability to classify the states of the target trajectory at different times, providing a basis for determining whether each point cloud block belongs to the target, and thus facilitating the accurate update of the target trajectory.

[0192] In an exemplary embodiment, as Figure 10 shown, the above step 350 may include the following steps:

[0193] Step 351, if the recognition result of the target trajectory indicates that the current state of the target trajectory has not changed compared with the historical state, discard the point clouds in the current point cloud block associated with the target trajectory according to the first condition.

[0194] Continuing with the previous example, if the historical states of the target trajectory recognized based on the first two frames of historical point cloud data of the target trajectory are both moving, and the current state of the target trajectory is moving, indicating that the states of the target trajectory recognized from three consecutive frames of point cloud data are all moving, then the recognition result indicates that the current state of the target trajectory has not changed compared with the historical state, indicating that the possibility that this point cloud block belongs to this target trajectory is relatively high. Then, the point clouds in the current point cloud block associated with the target trajectory will be discarded according to the first condition. Among them, the first condition can be represented by the first probability.

[0195] Taking the first probability p = 0.3 as an example, for a certain target trajectory, 0.3 * 100% of the point clouds are discarded from the current point cloud block associated with it. For example, if there are 100 point clouds in the current point cloud block, then 30% of the point clouds, that is, 30 point clouds, will be discarded.

[0196] Step 353, if the recognition result of the target trajectory indicates that the current state of the target trajectory has changed compared with the historical state, discard the point clouds in the current point cloud block associated with the target trajectory according to the second condition.

[0197] Similarly, as described above, if the historical states of the target trajectory recognized based on the first two frames of historical point cloud data of the target trajectory are both moving, and the current state of the target trajectory is stationary, indicating that the states of the target trajectory recognized from three consecutive frames of point cloud data are not completely the same, then the recognition result indicates that the current state of the target trajectory has changed compared with the historical state, indicating that the possibility that this point cloud block belongs to this target trajectory is relatively low. Then, the point clouds in the current point cloud block associated with the target trajectory are discarded according to the second condition. Among them, the second condition can be represented by the second probability. Of course, the second probability can be different from the first probability, and the second probability is greater than the first probability. This is not a specific limitation here.

[0198] Taking the second probability p' = 0.7 as an example, for a certain target trajectory, 0.7 * 100% of the point clouds are discarded from the currently associated point cloud block. For example, if there are 100 point clouds in the current point cloud block, then 70% of the point clouds, that is, 70 point clouds, will be discarded.

[0199] It is worth mentioning that the specifically discarded point clouds in the above point cloud block can be discarded from far to near according to the distance between the positions of the point clouds and the current position of the target represented by the target trajectory, or can be discarded from small to large according to the signal strength of the point clouds, which is not limited here.

[0200] Step 355, until the point clouds in each point cloud block associated with the target trajectory are completely discarded, and update the target trajectory in the target space according to the point clouds after the discard process.

[0201] It can be understood that the discard process can specifically be to remove the point clouds from the set corresponding to each associated point cloud block, and these removed point clouds do not participate in the update process of the target trajectory.

[0202] After the discard process of each point cloud block is completed, the positions of these remaining point clouds can be averaged, and the calculated average value is used as the position of the target at the current moment in the target space, and then the target trajectory in the target space is updated according to this position at the current moment.

[0203] With the cooperation of the above embodiments, the point clouds in each point cloud block are screened through probability discard, and the real point clouds belonging to the target trajectory are obtained, so that the target trajectory of the target in the target space can be updated based on these real point clouds belonging to the target trajectory, ensuring the accuracy of target tracking and positioning.

[0204] In addition, by combining the clustering method, the clustered point cloud blocks are quickly associated with the target trajectories corresponding to each target, which not only ensures that a certain number of point clouds are associated with each target trajectory, but also can exclude the point clouds in each point cloud block that do not belong to the target trajectory through the probability discard method, thus solving the problems of target tracking error or loss in the target tracking process.

[0205] Figure 11 Shows an application scenario of a trajectory update method provided by the present application.

[0206] In this application scenario, as Figure 11As shown, based on the existing trajectories in the target space, all the point cloud data in the obtained target space is clustered to obtain several point cloud blocks; based on all the obtained point cloud blocks, each qualified point cloud block (for example, the distance between the position of the point cloud block and the current position of the target corresponding to the existing trajectory is less than 1 meter) is associated with the existing trajectories in the target space; the point cloud features of each point cloud block are extracted by combining the associated point cloud blocks and the historical point cloud data in the trajectories associated with each point cloud block, and the states of each target trajectory are identified according to the extracted point cloud features of each point cloud block. This identification can also be regarded as a classification process, and the identification result is used to indicate whether the current state of each target trajectory has changed compared with the historical state.

[0207] If there is no change, it means that the point cloud block is very likely to belong to the associated target trajectory, and then the point cloud in the point cloud block associated with the target trajectory is discarded with a relatively small first probability (such as 0.3); if there is a change, it means that the point cloud block is less likely to belong to the associated target trajectory, and then the point cloud in the point cloud block associated with the target trajectory is discarded with a relatively large second probability (such as 0.7). Finally, each target trajectory can be updated based on the remaining point cloud in each point cloud block after the discard process.

[0208] Based on the updated target trajectories in the target space, the automatic control of the intelligent device can be realized.

[0209] Specifically, as Figure 12 shown, the automatic control process may include the following steps: Step 360, based on the updated trajectory, determine the position of the target at the current moment in the target space. Step 370, if the position of the target at the current moment in the target space meets the trigger condition in the automatic control scheme, then control the intelligent device to execute the corresponding automatic operation in the automatic control scheme. Among them, the automatic control scheme is used to realize the automatic control of the intelligent device by monitoring whether the position of the target in the target space meets the trigger condition.

[0210] It should be noted here that the automatic control scheme is generated based on the configuration of the user terminal. The user terminal may be Figure 1 the user terminal 110 shown. The automatic control scheme includes a trigger condition and an automatic operation. Among them, the trigger condition is related to the position of the target in the target space. Taking the induction lamp in the bedroom as an example, if an automatic control scheme is configured for the induction lamp in the user terminal, and the automatic control scheme is set with a corresponding lighting automatic control function, that is, the induction lamp is automatically turned on by detecting whether the user is in the bedroom. Then, based on this automatic control scheme, through user tracking and positioning, when the user appears in the bedroom, the induction lamp in the bedroom is automatically turned on, otherwise, the induction lamp is turned off.

[0211] Further, the pose of the target can also be recognized by the trained classifier based on the point cloud features of the constructed point cloud blocks. Specifically, after recognizing the current pose of the target, the classifier can continuously perform pose recognition on the point cloud features of each target to determine whether the pose of the target remains unchanged or changes, so as to perform pose-assisted recognition, thereby improving the accuracy of pose recognition.

[0212] In the above application scenario, when the real-time position of the target at the current moment in the target space or the pose of the target at the current moment in the target space satisfies the trigger condition, the automatic control of the intelligent device can be realized based on the automatic scheme configured in the user terminal. As a result, the automatic control scheme can be accurately and effectively executed, thus realizing more accurate automatic control.

[0213] It should be understood that although Figure 2 、 Figure 4 、 Figure 6 - 12 the steps in the flowchart are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise clearly stated in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, Figure 2 、 Figure 4 、 Figure 6 - 12 at least a part of the steps in

[0214] may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same moment, but can be executed at different moments. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps.

[0215] Please refer to Figure 13 , in the embodiment of the present application, a trajectory update device 500 is provided, including but not limited to: a point cloud acquisition module 510, a point-track association module 520, a feature extraction module 530, a state recognition module 540, and a trajectory update module 550.

[0216] Among them, the point cloud acquisition module 510 is used to perform clustering processing on the acquired point cloud data in the target space to obtain multiple point cloud blocks.

[0217] The point-track association module 520 is used to associate each point cloud block with the target trajectory corresponding to each target in the target space.

[0218] A feature extraction module 530, configured to construct point cloud features of each point cloud block based on each point cloud block and historical point cloud data in the target trajectory associated with each point cloud block.

[0219] A state recognition module 540, configured to recognize the states of each target trajectory according to the point cloud features of each point cloud block, and respectively obtain corresponding recognition results; each recognition result is used to indicate the state change of each target trajectory.

[0220] A trajectory update module 550, configured to update the target trajectories of each target according to the recognition results of each target trajectory.

[0221] In an exemplary embodiment, the point-trajectory association module includes: a position determination unit, configured to determine the current position of each target in the target space for the target trajectory corresponding to each target in the target space; a distance determination unit, configured to determine the distance between the position of each point cloud block and the current position of each target in the target space; a point-trajectory association unit, configured to associate the point cloud blocks whose distances meet the set distance range with each target trajectory.

[0222] In an exemplary embodiment, the point-trajectory association module further includes: a trajectory creation unit, configured to create a new target trajectory according to the point cloud block if the distance between the position of the point cloud block and the current position of each target in the target space does not meet the set distance range.

[0223] In an exemplary embodiment, the feature extraction module includes: a point cloud acquisition unit, configured to determine the current point cloud block based on each point cloud block, and determine the previous several frames of point cloud data based on the historical point cloud data in the target trajectory associated with each point cloud block, and respectively select a set number of point clouds from the current point cloud block and the previous several frames of point cloud data; a feature extraction unit, configured to construct intra-frame features and inter-frame features according to the selected point clouds; a feature fusion unit, configured to perform feature fusion on the intra-frame features and inter-frame features to obtain the point cloud features of the current point cloud block; until the feature construction of each point cloud block is completed, the point cloud features of each point cloud block are obtained.

[0224] In an exemplary embodiment, the feature extraction unit includes: an intra-frame feature extraction unit, configured to calculate multiple intra-frame features of the target trajectory according to the point clouds respectively selected from each frame of the current point cloud block and the previous several frames of point cloud data; each intra-frame feature corresponds to one frame of the current point cloud block or one frame of point cloud data; an inter-frame feature extraction unit, configured to respectively construct multiple adjacent-frame point cloud data by using each frame of point cloud data in the previous several frames of point cloud data and the current point cloud block, and calculate multiple inter-frame features according to the point clouds respectively selected from each adjacent-frame point cloud data; each inter-frame feature corresponds to one adjacent-frame point cloud data.

[0225] In an exemplary embodiment, the trajectory update module includes: a point cloud discarding unit, configured to discard the point cloud in the current point cloud block associated with the target trajectory according to a first condition if the recognition result of the target trajectory indicates that the current state of the target trajectory has not changed compared with the historical state; the current state of the target trajectory is recognized based on the current point cloud block associated with the target trajectory; the historical state of the target trajectory is recognized based on the historical point cloud data in the target trajectory associated with the current point cloud block; if the recognition result of the target trajectory indicates that the current state of the target trajectory has changed compared with the historical state, discard the point cloud in the current point cloud block associated with the target trajectory according to a second condition; a trajectory update unit, configured to update the target trajectory in the target space according to the point cloud after the discard processing until the point cloud in each point cloud block associated with the target trajectory is completely discarded.

[0226] In an exemplary embodiment, the state recognition module is further configured to respectively input the point cloud features of each point cloud block into a classifier to respectively recognize the current states of each target trajectory corresponding to different point cloud blocks; the classifier is a machine learning model that has been trained and has the ability to recognize the state of the target trajectory; based on the current states of each target trajectory corresponding to different point cloud blocks, corresponding recognition results are respectively obtained; the recognition result is used to indicate whether the current state of the target trajectory corresponding to one of the point cloud blocks has changed compared with the historical state of the target trajectory.

[0227] In an exemplary embodiment, the apparatus further includes: a model training module, and the model training module includes: a training data acquisition unit, configured to acquire a training data set, and the training data set includes training samples with labels; the label is used to indicate whether the training sample belongs to the sample target and the state of the target trajectory corresponding to the sample target; a model training unit, configured to, in the current iteration training, train a weak classifier according to each training sample in the training data set with a weight distribution to obtain the classification error rate of the weak classifier on the training data set; calculate the weight of the weak classifier based on the classification error rate of the weak classifier and update the weight distribution of each training sample in the training data set; through multiple iterations of training, obtain multiple weak classifiers and corresponding weights; in the case of training completion, obtain a strong classifier from the multiple weak classifiers and corresponding weights as the classifier that has completed training.

[0228] In an exemplary embodiment, the training data acquisition unit includes: a data acquisition unit for acquiring multiple frames of point cloud data of a sample target in a target space; a position calculation unit for determining the position of the sample target at the current moment in the target space based on the acquired current frame of point cloud data, and determining the position of the sample target at a historical moment in the target space based on the acquired previous several frames of point cloud data; a state determination unit for determining the current state of the target trajectory corresponding to the sample target according to the positions of the sample target at the current moment and the historical moment in the target space; a training data set unit for annotating the point cloud data of the sample target based on the current state of the target trajectory corresponding to the sample target to obtain a training sample corresponding to the sample target; and a training data set composed of multiple training samples corresponding to at least one sample target.

[0229] In an exemplary embodiment, the module training module is further configured to initialize the weight distribution of each training sample in the training data set; train an initial classifier using the training data set with the weight distribution, and calculate the classification error rate of the weak classifier on the training data set; calculate the weight of the weak classifier based on the classification error rate, and update the weight distribution of each training sample in the training data set, so as to obtain the weight of another weak classifier based on the training data set with the updated weight distribution until the training is completed, and obtain the trained classifier according to each weak classifier and its weight.

[0230] In an exemplary embodiment, the device further includes: a device control module for determining the position of the target at the current moment in the target space based on the updated target trajectory; if the position of the target at the current moment in the target space meets the trigger condition in the automation scheme, controlling the intelligent device to execute the corresponding automation operation in the automation scheme; the automation scheme is used to achieve the automatic control of the intelligent device when it is monitored that the position of the target in the target space meets the trigger condition.

[0231] It should be noted that when the above-mentioned trajectory update device updates the target trajectory, only the above-mentioned division of each functional module is used for distance description. In practical applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the target trajectory update device will be divided into different functional modules to complete all or part of the functions described above.

[0232] In addition, the above-mentioned trajectory update device and the embodiment of the trajectory update method belong to the same concept. The specific ways in which each module performs operations have been described in detail in the method embodiment, and will not be repeated here.

[0233] Figure 14 , a schematic structural diagram of an electronic device shown according to an exemplary embodiment. The electronic device can be any electronic device with processing capabilities, such as Figure 1The intelligent device 130 in the shown environment.

[0234] It should be noted that this electronic device is only an example adapted to this application and should not be considered as providing any limitation to the scope of use of this application. Nor can this electronic device be construed as needing to rely on or necessarily having Figure 14 One or more components in the shown exemplary electronic device 2000.

[0235] The hardware structure of the electronic device 2000 can vary greatly due to different configurations or performances. For example, Figure 14 As shown, the electronic device 2000 includes: a power supply 210, an interface 230, at least one memory 250, and at least one central processing unit (CPU) 270.

[0236] Specifically, the power supply 210 is used to provide operating voltage for each hardware device on the electronic device 2000.

[0237] The interface 230 includes at least one wired or wireless network interface 231 for interacting with external devices. For example, for Figure 1 The interaction between the user terminal 110 and the intelligent device 130 in the shown implementation environment.

[0238] Of course, in other examples adapted to this application, the interface 230 may further include at least one serial-to-parallel conversion interface 233, at least one input / output interface 235, and at least one USB interface 237, etc. As Figure 14 Shown, this is not a specific limitation here.

[0239] The memory 250, as a carrier for resource storage, can be a read-only memory, a random access memory, a magnetic disk, or an optical disc, etc. The resources stored thereon include an operating system 251, application programs 253, and data 255, etc. The storage method can be transient storage or permanent storage.

[0240] Among them, the operating system 251 is used to manage and control each hardware device and application program 253 on the electronic device 2000 to enable the central processing unit 270 to perform operations and processing on the massive data 255 in the memory 250. It can be Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSD TM, etc.

[0241] The application program 253 is a program instruction or code that completes at least one specific task based on the operating system 251. It can include at least one module ( Figure 14(not shown), each module may separately include program instructions or code for the electronic device 2000. For example, the trajectory update device can be regarded as an application program 253 deployed in the electronic device 2000.

[0242] The data 255 can be photos, pictures, etc. stored in a disk, or can also be the point cloud of the first target, etc., and is stored in the memory 250.

[0243] The central processing unit 270 may include one or more than one processors, and is configured to communicate with the memory 250 through at least one communication bus, so as to read the program instructions or code stored in the memory 250, and further implement the operation and processing of the massive data 255 in the memory 250. For example, the trajectory update method is completed in the form of reading a series of program instructions or code stored in the memory 250 by the central processing unit 270.

[0244] In addition, the present application can also be implemented by a hardware circuit or a combination of a hardware circuit and software. Therefore, the implementation of the present application is not limited to any specific hardware circuit, software, and the combination of the two.

[0245] Please refer to Figure 15 , in the embodiment of the present application, an electronic device 4000 is provided, and the electronic device may include: a human body sensor configured with a millimeter wave radar, etc.

[0246] In Figure 15 , the electronic device 4000 includes at least one processor 4001 and at least one memory 4003.

[0247] Among them, the data interaction between the processor 4001 and the memory 4003 can be realized through at least one communication bus 4002. The communication bus 4002 may include a path for transmitting data between the processor 4001 and the memory 4003. The communication bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The communication bus 4002 can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, Figure 15 only a thick line is used to represent it in

[0248] Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as data sending and / or data receiving, etc. It should be noted that in practical applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation to the embodiments of the present application.

[0249] The processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logic blocks, modules, and circuits described in connection with the disclosure of the present application. The processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0250] The memory 4003 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or it may also be an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium that can be used to carry or store desired program instructions or code in the form of instruction or data structures and can be accessed by the electronic device 400, but is not limited thereto.

[0251] Program instructions or code are stored on the memory 4003, and the processor 4001 can read the program instructions or code stored in the memory 4003 through the communication bus 4002.

[0252] When the program instructions or code are executed by the processor 4001, the trajectory update methods in the above embodiments are implemented.

[0253] In addition, an embodiment of the present application provides a storage medium, on which computer-readable instructions are stored. The computer-readable instructions are loaded and executed by a processor to implement the trajectory update method as described above.

[0254] An embodiment of the present application provides a computer program product, which includes computer-readable instructions. The computer-readable instructions are stored in a storage medium, and a processor of an electronic device reads the computer-readable instructions from the storage medium, loads and executes the computer-readable instructions, so that the electronic device implements the trajectory update method as described above.

[0255] Compared with the related art, the present application obtains point cloud data in a target space, and clusters the obtained point cloud data to obtain multiple point cloud blocks; associates each point cloud block with at least one target trajectory in the target space; constructs point cloud features of each target trajectory according to the point cloud blocks respectively associated with each target trajectory; classifies the states of each target trajectory at different times according to the point cloud features of each target trajectory to obtain recognition results of each target trajectory; the recognition results are used to indicate whether the states of each target trajectory at different times have changed; and updates each target trajectory in the target space according to the recognition results of each target trajectory. By clustering all the point clouds in the target space into point cloud blocks and then performing point-trajectory association with the existing trajectories, then classifying according to the point cloud features between the point cloud blocks and the point clouds of the associated trajectories, indicating whether the states of the associated target trajectories at different times have changed according to the recognition results, probabilistically discarding the point clouds in the point cloud blocks associated with the target trajectories, and updating the target trajectories in the target space according to the remaining point clouds in the point cloud blocks. Through the above steps, after clustering the point clouds in the space, it is allowed that one point cloud block is associated with multiple target trajectories, and then the state of each target trajectory is judged by classification, and the associated point cloud data is discarded according to the target trajectory state probability, ensuring that a certain number of point clouds are associated with each target trajectory, thereby reducing the probability of target positioning and tracking errors or loss caused by incorrect point-trajectory association, and thus solving the problem of low target positioning accuracy in the target tracking process.

[0256] It should be understood that although the steps in the flowchart of the accompanying drawings are shown in sequence according to the indication of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and they can be executed in other orders. Moreover, at least some of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.

[0257] The above are only some embodiments of the present application. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present application.

Claims

1. A trajectory update method, characterized in that, the method includes: Performing clustering processing on the acquired point cloud data in the target space to obtain multiple point cloud blocks; Associating each of the point cloud blocks with the target trajectories corresponding to each target in the target space; Constructing the point cloud features of each of the point cloud blocks based on each of the point cloud blocks and the historical point cloud data in the target trajectories associated with each of the point cloud blocks; Identifying the states of each of the target trajectories according to the point cloud features of each of the point cloud blocks, and respectively obtaining corresponding identification results; each of the identification results is used to indicate the state change of each target trajectory; Updating the target trajectories of each of the targets according to the identification results of each of the target trajectories.

2. The method according to claim 1, characterized in that, the associating each of the point cloud blocks with the target trajectories corresponding to each target in the target space includes: For the target trajectories corresponding to each target in the target space, determining the current positions of each target in the target space; Determining the distances between the positions of each of the point cloud blocks and the current positions of each target in the target space; Associating the point cloud blocks whose distances meet the set distance range with each of the target trajectories.

3. The method according to claim 2, characterized in that, after determining the distances between the positions of each of the point cloud blocks and the current positions of each target in the target space, the method further includes: If the distances between the positions of the point cloud blocks and the current positions of each target in the target space do not meet the set distance range, creating new target trajectories according to the point cloud blocks.

4. The method according to claim 1, characterized in that, the constructing the point cloud features of each of the point cloud blocks based on each of the point cloud blocks and the historical point cloud data in the target trajectories associated with each of the point cloud blocks includes: Determining the current point cloud block based on each of the point cloud blocks, and determining the previous several frames of point cloud data based on the historical point cloud data in the target trajectories associated with each of the point cloud blocks; Respectively selecting a set number of point clouds from the current point cloud block and the previous several frames of point cloud data; Constructing in-frame features and inter-frame features according to the selected point clouds; Performing feature fusion on the in-frame features and the inter-frame features to obtain the point cloud features of the current point cloud block; Until the feature construction of each of the point cloud blocks is completed, obtaining the point cloud features of each of the point cloud blocks.

5. The method according to claim 4, characterized in that, the constructing in-frame features and inter-frame features according to the selected point clouds includes: Calculating a plurality of in-frame features according to the point clouds respectively selected from the current point cloud block and the previous several frames of point cloud data; the in-frame features correspond to the current point cloud block or one frame of point cloud data; Respectively constructing a plurality of adjacent-frame point cloud data by using each frame of point cloud data in the previous several frames of point cloud data and the current point cloud block, and calculating a plurality of inter-frame features according to the point clouds respectively selected from each of the adjacent-frame point cloud data; each inter-frame feature corresponds to one adjacent-frame point cloud data.

6. The method according to claim 1, characterized in that, Updating the target trajectories of the targets according to the recognition results of the respective target trajectories includes: If the recognition result of the target trajectory indicates that the current state of the target trajectory has not changed compared to the historical state, discard the point clouds in the current point cloud block associated with the target trajectory according to the first condition; the current state of the target trajectory is recognized based on the current point cloud block associated with the target trajectory; the historical state of the target trajectory is recognized based on the historical point cloud data in the target trajectory associated with the current point cloud block; If the recognition result of the target trajectory indicates that the current state of the target trajectory has changed compared to the historical state, discard the point clouds in the current point cloud block associated with the target trajectory according to the second condition; Until the point clouds in each of the point cloud blocks associated with the target trajectory are completely discarded, update the target trajectory in the target space according to the discarded point clouds.

7. The method according to any one of claims 1 to 6, characterized in that recognizing the states of the respective target trajectories according to the point cloud features of the respective point cloud blocks, and respectively obtaining corresponding recognition results, including: Inputting the point cloud features of the respective point cloud blocks into a classifier respectively to recognize the current states of the respective target trajectories corresponding to different point cloud blocks; the classifier is a machine learning model that has been trained and has the ability to recognize the states of the target trajectories; Based on the current states of the respective target trajectories corresponding to different point cloud blocks, respectively obtain the corresponding recognition results; the recognition results are used to indicate whether the current state of the target trajectory corresponding to one of the point cloud blocks has changed compared to the historical state of the target trajectory.

8. The method according to claim 7, characterized in that The training process of the classifier includes: Obtaining a training data set, the training data set includes training samples with labels; the labels are used to indicate whether the training samples belong to the sample target and the state of the target trajectory corresponding to the sample target; In the current iteration training, train a weak classifier according to the respective training samples in the training data set with a weight distribution to obtain the classification error rate of the weak classifier on the training data set; Calculate the weight of the weak classifier based on the classification error rate of the weak classifier, and update the weight distribution of the respective training samples in the training data set; Through multiple iterations of training, obtain multiple weak classifiers and their corresponding weights; In the case of completing the training, obtain a strong classifier from the multiple weak classifiers and their corresponding weights as the classifier that has completed the training.

9. The method according to claim 8, characterized in that The obtaining of the training data set includes: Collecting multiple frames of point cloud data of the sample target in the target space; Based on the currently collected frame of point cloud data, determining the position of the sample target at the current moment in the target space, and based on the previously collected several frames of point cloud data, determining the position of the sample target at the historical moment in the target space; Determine the current state of the target trajectory corresponding to the sample target according to the positions of the sample target at the current moment and historical moments in the target space; Based on the current state of the target trajectory corresponding to the sample target, label the point cloud data of the sample target to obtain the training sample corresponding to the sample target; The training data set is composed of multiple training samples corresponding to at least one of the sample targets.

10. The method according to any one of claims 1 to 6, wherein, after updating the target trajectories of the targets according to the recognition results of the target trajectories, the method further includes: Determine the position of the target at the current moment in the target space based on the updated target trajectory; If the position of the target at the current moment in the target space meets the trigger condition in the automation scheme, control the intelligent device to execute the corresponding automation operation in the automation scheme; the automation scheme is used to realize the automatic control of the intelligent device when it is monitored that the position of the target in the target space meets the trigger condition.

11. A trajectory update device, wherein, the device includes: A point cloud acquisition module, configured to perform clustering processing on the acquired point cloud data in the target space to obtain a plurality of point cloud blocks; A point-trajectory association module, configured to associate each of the point cloud blocks with the target trajectories corresponding to each target in the target space; A feature extraction module, configured to construct the point cloud features of each of the point cloud blocks based on each of the point cloud blocks and the historical point cloud data in the target trajectories associated with each of the point cloud blocks; A state recognition module, configured to recognize the states of the target trajectories according to the point cloud features of each of the point cloud blocks, and respectively obtain corresponding recognition results; each of the recognition results is used to indicate the state change of each target trajectory; A trajectory update module, configured to update the target trajectories of the targets according to the recognition results of the target trajectories.

12. An electronic device, wherein, it includes: At least one processor and at least one memory, wherein, The memory stores program instructions or codes; The program instructions or codes are loaded and executed by the processor, so that the electronic device implements the trajectory update method according to any one of claims 1 to 10.

13. A storage medium, on which program instructions or codes are stored, wherein, The program instructions or codes are loaded and executed by the processor to implement the trajectory update method according to any one of claims 1 to 10.

Images