Multi-object tracking method, device, system, apparatus and computer readable medium

Abstract

The application relates to a multi-object tracking method, device, system, equipment and computer readable medium. The application determines the positions of detected objects and generates identity signature vectors by combining millimeter wave radar detection data, calculates an association cost based on the positions and the identity signature vectors to realize accurate matching of objects and state sets, arbitrates the predicted positions by using target space scene semantic information and preset behavior logic rules to determine the best predicted position, and finally outputs a stable and reliable motion trajectory, thereby solving the technical problems that the prior art lacks scene semantic constraints, resulting in unreasonable trajectory prediction and high misassociation rate.

Description

Technical Field

[0001] This application relates to the field of millimeter-wave radar technology, and in particular to a multi-object tracking method, apparatus, system, device, and computer-readable medium. Background Technology

[0002] Millimeter-wave radar, as a non-contact sensing technology, has privacy protection, is unaffected by lighting conditions, and has a certain penetration capability. It has broad application prospects in indoor personnel sensing fields such as smart homes, security monitoring, and health care, and is especially suitable for personnel status monitoring in small spaces such as bathrooms, bedrooms, and office cubicles.

[0003] In related technologies, existing millimeter-wave radar multi-object tracking technologies mostly classify targets by combining point cloud clustering with the macroscopic geometric features of the target, or perform target detection and status determination by integrating multi-dimensional information such as distance, angle, speed, and signal strength. These technologies can achieve basic personnel perception and tracking in sparse, open scenarios. However, in typical small-space scenarios characterized by close proximity, high density, and multiple obstructions, the point cloud of personnel targets is prone to severe aliasing, rendering traditional discrimination methods based on point cloud geometric features ineffective. Furthermore, existing technologies do not fully consider the inherent scene layout and behavioral patterns in small spaces, and the lack of scene semantic constraints leads to unreasonable trajectory predictions and high false association rates, making it difficult to achieve stable, continuous, and accurate tracking of multiple people in small spaces.

[0004] There is currently no effective solution to the problem that the lack of scene semantic constraints in existing technologies leads to unreasonable trajectory prediction and high false association rate. Summary of the Invention

[0005] This application provides a multi-object tracking method, apparatus, system, device, and computer-readable medium to solve the technical problems of unreasonable trajectory prediction and high false association rate caused by the lack of scene semantic constraints in the prior art.

[0006] According to one aspect of the embodiments of this application, this application provides a multi-object tracking method, comprising: upon receiving detection data from a millimeter-wave radar, determining the current positions of multiple detected objects in a target space based on the detection data, and calculating the identity signature vector of each detected object, wherein the detection data is obtained by the millimeter-wave radar detecting the target space at the current moment; calculating the association cost between each detected object and multiple state sets based on the current position and identity signature vector of each detected object, and associating each detected object with the state set with the minimum association cost, wherein the state set is a set pre-created for each detected object to record the motion trajectory and motion state of each detected object, and the detected object corresponds one-to-one with the state set; calculating multiple predicted positions of each detected object at the next moment, and arbitrating the multiple predicted positions based on scene semantic information of the target space and preset behavioral logic rules to obtain the best predicted position; and outputting the motion trajectory of each detected object based on the current position, the best predicted position, and the association results between each detected object and multiple state sets.

[0007] Optionally, determining the current position of multiple detection objects in the target space based on the detection data includes: performing a three-dimensional fast Fourier transform on the detection data to generate point cloud data, wherein the point cloud data includes the distance, azimuth, pitch angle, and Doppler velocity information of each point cloud; using a CFAR detector to detect each point cloud to select valid target points as detection objects; and calculating the current position of the detection object corresponding to the valid target point based on the distance, azimuth, pitch angle, and Doppler velocity information of the valid target point.

[0008] Optionally, calculating the identity signature vector of each detected object includes: extracting Doppler velocity information of the region where the detected object is located from multiple frames of continuous detection data, including the detection data at the current time; performing a short-time Fourier transform on the Doppler velocity information to obtain the micro-Doppler features of the detected object; inputting the micro-Doppler features into a target convolutional neural network, so that after processing the micro-Doppler features using the target convolutional neural network, the identity signature vector of the detected object is output. The target convolutional neural network is obtained by replacing the final classification layer with a fully connected layer and training it with a millimeter-wave radar micro-motion dataset.

[0009] Optionally, based on the current location and identity signature vector of each detected object, the association cost between each detected object and multiple state sets is calculated as follows: the distance between the current location of each detected object and the target location recorded in the multiple state sets is calculated as the spatial distance cost, where the target location is the location of each detected object at the previous moment; the similarity between the identity signature vector of each detected object and the identity signature vectors of each detected object recorded in the multiple state sets is calculated as the identity similarity cost; the spatial distance cost and the identity similarity cost are weighted and fused according to the preset weights of the spatial distance cost and the identity similarity cost to obtain the association cost.

[0010] Optionally, before arbitrating multiple predicted locations based on scene semantic information and preset behavioral logic rules of the target space, the method further includes: obtaining a layout map of the target space; constructing an environment model of the target space based on the layout map; creating spatial geofences for each subspace of the target space based on the environment model, wherein the spatial geofences are used to represent the scene semantic information of the target space; creating a motion model that moves between the subspaces of the target space, and configuring behavioral logic rules for the motion model to move between the subspaces.

[0011] Optionally, multiple predicted positions for each detected object at the next moment are calculated, and the multiple predicted positions are arbitrated based on the scene semantic information of the target space and preset behavioral logic rules to obtain the best predicted position. This includes: inputting the current position of each detected object and the state set associated with the detected object into an extended Kalman filter, so as to use the extended Kalman filter to predict multiple candidate positions of the detected object at the next moment based on the current position of the detected object and its historical trajectory in the state set, as multiple predicted positions; mapping the current position of the detected object and the multiple predicted positions to a spatial geofence in the target space; driving the motion model to move from the current position to the multiple predicted positions in the spatial geofence in sequence, thereby extracting the predicted position that conforms to the preset behavioral logic rules as the best predicted position.

[0012] Optionally, the method further includes: identifying a first object that existed in the previous moment and was lost in the current moment, and determining the position of the first object in the previous moment as its last appearance position; calculating the average identity signature vector of the first object based on multiple identity signature vectors of the first object in the most recent preset number of frames; storing the average identity signature vector in the lost target list and starting a loss timer; if a second object is detected before the loss timer counts down, calculating the similarity between the identity signature vector of the second object and all vectors in the lost target list, wherein the second object is a new object that appears later than the current moment within a preset range centered on the last appearance position of the first object; if the similarity between the identity signature vector of the second object and the average identity signature vector of the first object is greater than or equal to a preset threshold, associating the motion trajectory and motion state of the second object with the state set of the first object as the latest motion state of the first object, and marking the second object with the identification mark of the first object.

[0013] According to another aspect of the embodiments of this application, this application provides a multi-object tracking device, including: a determination and calculation module, configured to determine the current position of multiple detected objects in a target space based on the detection data received from millimeter-wave radar, and calculate the identity signature vector of each detected object, wherein the detection data is obtained by the millimeter-wave radar detecting the target space at the current moment; an association module, configured to calculate the association cost between each detected object and multiple state sets based on the current position and identity signature vector of each detected object, and associate each detected object with the state set with the minimum association cost, wherein the state set is a set pre-created for each detected object to record the motion trajectory and motion state of each detected object, and the detected object corresponds one-to-one with the state set; a prediction and arbitration module, configured to calculate multiple predicted positions of each detected object at the next moment, and arbitrate the multiple predicted positions based on the scene semantic information of the target space and preset behavioral logic rules to obtain the best predicted position; and an output module, configured to output the motion trajectory of each detected object based on the current position, the best predicted position, and the association results between each detected object and multiple state sets.

[0014] According to another aspect of the embodiments of this application, this application provides a multi-object tracking system, including a millimeter-wave radar sensor and a host computer, the host computer being used to execute the steps of the above method.

[0015] According to another aspect of the embodiments of this application, this application provides an electronic device, including a memory, a processor, a communication interface and a communication bus. The memory stores a computer program that can run on the processor. The memory and the processor communicate with each other through the communication bus and the communication interface. When the processor executes the computer program, it implements the steps of the above method.

[0016] According to another aspect of the embodiments of this application, this application also provides a computer-readable medium having processor-executable non-volatile program code that causes the processor to perform the above-described method.

[0017] Compared with related technologies, the technical solutions provided in this application have the following advantages: This application provides a multi-object tracking method, comprising: upon receiving detection data from a millimeter-wave radar, determining the current positions of multiple detected objects in a target space based on the detection data, and calculating the identity signature vector of each detected object, wherein the detection data is obtained by the millimeter-wave radar detecting the target space at the current moment; calculating the association cost between each detected object and multiple state sets based on the current position and identity signature vector of each detected object, and associating each detected object with the state set with the minimum association cost, wherein the state set is a set pre-created for each detected object to record the motion trajectory and motion state of each detected object, and there is a one-to-one correspondence between the detected object and the state set; calculating multiple predicted positions for each detected object at the next moment, and arbitrating the multiple predicted positions based on the scene semantic information of the target space and preset behavioral logic rules to obtain the best predicted position; and outputting the motion trajectory of each detected object based on the current position, the best predicted position, and the association results between each detected object and multiple state sets. This application determines the location of the detected object by combining millimeter-wave radar detection data and generates an identity signature vector. Based on the location and identity signature vector, it calculates the association cost to achieve accurate matching between the object and the state set. At the same time, it uses the semantic information of the target space scene and preset behavioral logic rules to arbitrate the predicted position to determine the best predicted position. Finally, it outputs a stable and reliable motion trajectory, which solves the technical problems of unreasonable trajectory prediction and high false association rate caused by the lack of scene semantic constraints in the existing technology. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the hardware environment for an optional multi-object tracking method provided according to an embodiment of this application; Figure 2 This is a schematic flowchart of an optional multi-object tracking method according to an embodiment of this application; Figure 3 This is a schematic diagram of a host computer module for an optional multi-object tracking system according to an embodiment of this application; Figure 4 This is a schematic diagram of an optional identity signature vector generation process provided according to an embodiment of this application; Figure 5 This is a schematic diagram of another multi-object tracking method provided according to an embodiment of this application; Figure 6 This is a block diagram of an optional multi-object tracking device according to an embodiment of this application; Figure 7 This is a schematic diagram of an optional electronic device structure provided in an embodiment of this application. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustration and has no specific meaning in itself. Therefore, "module" and "part" may be used interchangeably.

[0023] To address the problems mentioned in the background art, according to one aspect of the embodiments of this application, an embodiment of a multi-object tracking method is provided.

[0024] Optionally, in the embodiments of this application, the above-described multi-object tracking method can be applied to, for example... Figure 1 The hardware environment shown consists of a millimeter-wave radar 101 and a host computer 103. Figure 1 As shown, the host computer 103 is connected to the millimeter-wave radar 101 via a network and can be used to provide services to the millimeter-wave radar 101. A database 105 can be set up on the host computer or independently of the host computer to provide data storage services for the host computer 103. The aforementioned network includes, but is not limited to, a wide area network, a metropolitan area network, or a local area network.

[0025] A multi-object tracking method in this embodiment can be executed by a host computer 103, or it can be jointly executed by the host computer 103 and the millimeter-wave radar 101, such as... Figure 2 As shown, the method may include the following steps: Step S202: Upon receiving the detection data from the millimeter-wave radar, determine the current positions of multiple detected objects in the target space based on the detection data, and calculate the identity signature vector of each detected object. The detection data is obtained by the millimeter-wave radar detecting the target space at the current moment. Step S204: Based on the current position and identity signature vector of each detected object, calculate the association cost between each detected object and multiple state sets, and associate each detected object with the state set with the smallest association cost. The state set is a set created in advance for each detected object to record the motion trajectory and motion state of each detected object, and there is a one-to-one correspondence between the detected object and the state set. Step S206: Calculate multiple predicted positions for each detected object at the next moment, and arbitrate the multiple predicted positions based on the scene semantic information of the target space and the preset behavior logic rules to obtain the best predicted position. Step S208: Based on the current position, the best predicted position, and the association results between each detected object and multiple state sets, output the motion trajectory of each detected object.

[0026] This application embodiment is applied to a multi-object tracking system consisting of a millimeter-wave radar sensor and a host computer, such as... Figure 3 As shown, the host computer includes a signal processing and feature extraction module, a multi-target tracking and inference engine, a spatial geofence storage module, and a result output module. The millimeter-wave radar sensor is used to transmit radar waves and receive echo signals. The spatial geofence storage module is used to store environmental semantic information such as the environmental model of the target space and the spatial geofence. The signal processing and feature extraction module is used to generate point clouds and identity signature vectors. The multi-target tracking and inference engine is used to realize the association, prediction, and re-identification of multiple targets. The result output module is used to output the final tracking results.

[0027] In this embodiment, the detection data refers to the raw data of the target space echo received after the millimeter-wave radar emits electromagnetic waves. The identity signature vector refers to the feature vector extracted from the target's micro-Doppler features through deep learning, used to uniquely identify the target. The state set refers to the data set created for each detected object to store its historical trajectory, speed, position, and identity signature vector, among other motion state information. The association cost is a quantitative indicator used to measure the degree of matching between the current detected object and the historical state set. Scene semantic information refers to prior environmental information such as the layout of the target space, sub-region division, and physical boundaries. Preset behavioral logic rules refer to the motion rules of the target within the target space that conform to physical common sense and spatial constraints.

[0028] In step S202, the host computer receives the target space detection data collected by the millimeter-wave radar in real time, preprocesses the detection data, locates the current three-dimensional position of multiple detection objects in the target space, and extracts the target micro-Doppler features based on continuous multi-frame detection data, and generates the identity signature vector of each detection object through neural network processing.

[0029] In step S204, the host computer calculates the spatial matching degree between the current position of each detected object and all historical state sets, as well as the similarity between the identity signature vector and the identity features in each state set. The association cost is obtained by fusing these values, and the detected object is assigned to the state set with the minimum association cost, thereby achieving accurate matching between the target and the historical trajectory.

[0030] In step S206, the host computer generates multiple candidate predicted positions for the next moment for each associated detection object by combining historical motion data. The candidate positions are then substituted into the scene semantic information of the target space, and the best predicted position that has no spatial conflict and conforms to the motion law is selected according to the preset behavior logic rules.

[0031] In step S208, the host computer integrates the current position of the detected object, the best predicted position, and the association results of the target and state set to generate and output a continuous and stable motion trajectory.

[0032] Through steps S202 to S208, this application determines the location of the detected object by combining millimeter-wave radar detection data and generates an identity signature vector. Based on the location and identity signature vector, it calculates the association cost to achieve accurate matching between the object and the state set. At the same time, it uses the semantic information of the target space scene and preset behavioral logic rules to arbitrate the predicted location to determine the best predicted location. Finally, it outputs a stable and reliable motion trajectory, which solves the technical problem of unreasonable trajectory prediction and high false association rate caused by the lack of scene semantic constraints in the prior art.

[0033] In an optional embodiment, determining the current positions of multiple detected objects in the target space based on detection data includes: Step 1: Perform three-dimensional fast Fourier transform processing on the detection data to generate point cloud data, wherein the point cloud data includes the distance, azimuth angle, pitch angle and Doppler velocity information of each point cloud. Step 2: Use the CFAR detector to detect each point cloud to select valid target points as detection objects; Step 3: Calculate the current position of the detected object corresponding to the effective target point based on the distance, azimuth, pitch angle and Doppler velocity information of the effective target point.

[0034] In this embodiment, the three-dimensional fast Fourier transform refers to sequentially performing range, Doppler, and angular dimension fast Fourier transforms on the raw radar data to generate point cloud data containing spatial and motion information. The CFAR detector is a constant false alarm rate detector used to filter out valid target points from clutter and noise. A valid target point refers to the point cloud data corresponding to a detected object that is determined to exist after CFAR detection.

[0035] In this embodiment, the host computer performs a three-dimensional fast Fourier transform on the current detection data of the millimeter-wave radar, decomposing it to obtain the range, azimuth, elevation, and Doppler velocity information of all point clouds in the target space, forming a complete point cloud dataset. A CFAR detector is used to detect the point cloud data one by one, removing invalid points such as noise and clutter, retaining valid target points whose signal strength and motion characteristics match the target characteristics, and using these as the detection objects to be tracked. The host computer extracts the range, azimuth, and elevation parameters inherent in the valid target points, and calculates the three-dimensional current position of the detection object corresponding to the valid target point in the target space using a spatial coordinate transformation algorithm.

[0036] This application combines three-dimensional Fourier transform with CFAR detection to accurately extract effective targets and calculate their positions, thereby improving the accuracy and anti-interference capability of target positioning in small spaces.

[0037] In an optional embodiment, calculating the identity signature vector for each detected object includes: Step 1: Extract the Doppler velocity information of the region where the detected object is located from multiple frames of continuous detection data, including the detection data at the current moment; Step 2: Perform a short-time Fourier transform on the Doppler velocity information to obtain the micro-Doppler features of the detected object; Step 3: Input the micro-Doppler features into the target convolutional neural network, so that the target convolutional neural network processes the micro-Doppler features and outputs the identity signature vector of the detected object. The target convolutional neural network is obtained by replacing the last classification layer with a fully connected layer and training with a millimeter-wave radar micro-motion dataset.

[0038] In this embodiment, micro-Doppler features refer to the Doppler frequency modulation features generated by subtle movements of the target, such as breathing and limb micro-movements; short-time Fourier transform refers to a method of performing frame-by-frame transformation on continuous time-domain data to obtain joint time-frequency domain features. The target convolutional neural network refers to a network model trained on a millimeter-wave radar micro-motion dataset, replacing the classification layer with a fully connected layer, specifically designed for extracting micro-Doppler identity features.

[0039] In the embodiments of this application, such as Figure 4As shown, the process of calculating the identity signature vector for each detected object is as follows: The host computer receives the echo signal (i.e., detection data) uploaded by the radar. Using the current detection data as a reference, the host computer selects multiple consecutive frames of detection data to locate the spatial region where each detected object is located, and accurately extracts the Doppler velocity time-series information within that region. Then, a short-time Fourier transform is performed on the extracted Doppler velocity information to generate a time-frequency domain feature map, obtaining micro-Doppler features that can characterize the subtle motion of the target. The host computer inputs the micro-Doppler features into a pre-trained target convolutional neural network. The network performs deep encoding and dimensionality reduction on the features, outputting a fixed-dimensional, unique identity signature vector used to distinguish different detected objects.

[0040] If multiple individuals have highly similar micro-Doppler characteristics (such as similar height, synchronous state, and breathing rhythm), leading to identity signature conflicts, the following three levels of collaborative processing can be used: First, the signature features are enhanced and multidimensionalized. The identity signature vector extracted in this application does not rely solely on breathing frequency or a single micro-motion feature, but is a 128-dimensional dense feature vector extracted through deep learning based on the micro-Doppler spectrogram. This spectrogram contains superimposed information of multiple biological motion features such as breathing, heartbeat, slight limb swaying, posture changes, and gait micro-movements. Even if two people have the same breathing rhythm and height, their limb micro-movement patterns, center of gravity swing characteristics, and micro-Doppler change trajectories during posture transitions still have individual differences. Deep neural networks can capture these high-dimensional subtle differences that are difficult for humans to perceive, forming a feature representation with discriminative power.

[0041] Second, the association mechanism that integrates spatiotemporal and signature fusion. In the initialization and association of multi-target states, the association cost function simultaneously integrates spatial distance cost and identity signature cosine similarity cost. This means that even if the identity signatures of two targets are highly similar, the system will still make a joint judgment based on the differences in their spatial locations. Association conflicts will only occur when the two targets are highly similar in both spatial location and signature features, which greatly reduces the probability of false association.

[0042] Third, spatial constraints act as a fallback mechanism. In extreme cases, such as when two targets not only have highly similar micro-Doppler signatures but are also extremely close in spatial location, spatial constraints and behavioral logic rules will play an arbitration role. For example, if two targets are located in the toilet area and the sink area respectively, even if their signatures are similar, the system will refuse to incorrectly associate them based on the regional transfer logic and the rationality of trajectory prediction, ensuring the stability of tracking.

[0043] Furthermore, if it is found in actual deployment that multiple signatures are indeed difficult to distinguish in specific scenarios, the distinguishability of signatures can be further improved by increasing the collection time (such as extracting micro-Doppler temporal features for more than 10 seconds) or introducing higher-dimensional features (such as fusing distance-Doppler-angle three-dimensional tensors).

[0044] This application generates a unique identity vector based on micro-Doppler features, which can still stably distinguish targets in point cloud overlapping and occlusion scenarios, thus improving the robustness of identity recognition.

[0045] In an optional embodiment, based on the current location and identity signature vector of each detected object, the association cost between each detected object and multiple state sets is calculated, including: Step 1: Calculate the distance between the current position of each detected object and the target position recorded in multiple state sets as the spatial distance cost, where the target position is the position of each detected object in the previous time step; Step 2: Calculate the similarity between the identity signature vector of each detected object and the identity signature vectors of each detected object recorded in multiple state sets, and use this as the identity similarity cost. Step 3: According to the preset weights of spatial distance cost and identity similarity cost, perform weighted fusion of spatial distance cost and identity similarity cost to obtain the association cost.

[0046] In this embodiment, the spatial distance cost refers to the quantified spatial distance between the current position of the detected object and the previous position recorded in the state set. The identity similarity cost refers to the quantified difference between the identity signature vector of the detected object and the identity features in the state set. The preset weight refers to a pre-set proportional coefficient used to balance the influence of spatial features and identity features on the association result. In an optional embodiment, this proportional coefficient can be identity similarity cost weight: spatial distance cost weight = 7:3, and fully supports adaptive weighting in actual operation. Specifically, the system can dynamically adjust the weight according to the scene state. For example, when the detected object is in high-speed motion or in a sparse point cloud stage, the reliability of spatial location information is high, and the weight of spatial distance cost can be appropriately increased. When the detected object is occluded, the point cloud is aliased, or it is in a static state (such as sitting or lying down), the spatial location is prone to ambiguity. At this time, the system automatically increases the weight of identity signature similarity cost, making the association decision more dependent on micro-Doppler features. In addition, adaptive adjustment can be made by combining historical association confidence. For example, if the identity signature of a detected object matches stably in multiple consecutive frames, the weight of its signature cost can be temporarily increased in subsequent frames; conversely, if the signature fluctuates greatly, its weight can be reduced.

[0047] In this embodiment, the host computer traverses all historical state sets, calculates the 3D current position of the currently detected object and its Euclidean distance to the target position recorded in the previous time step in each state set, and uses this distance as the spatial distance cost. Simultaneously, the host computer calculates the cosine similarity between the identity signature vector of the currently detected object and the identity signature vectors stored in each state set, converting the similarity difference into an identity similarity cost. The spatial distance cost and the identity similarity cost are then weighted and summed according to preset weights to obtain the final association cost, which is used to determine the degree of matching between the detected object and the state set.

[0048] This application calculates the matching degree by integrating the dual costs of space and identity, avoiding false associations caused by a single feature and improving the accuracy of association of high-density targets in small spaces.

[0049] In an optional embodiment, before arbitrating multiple predicted locations based on scene semantic information of the target space and preset behavioral logic rules, the method further includes: Step 1: Obtain the layout diagram of the target space; Step 2: Construct an environmental model of the target space based on the layout diagram; Step 3: Create spatial geofences for each subspace of the target space based on the environment model, whereby the spatial geofences are used to represent the scene semantic information of the target space; Step 4: Create a motion model that moves between the various subspaces of the target space, and configure the behavior logic rules for the motion model to move between the various subspaces.

[0050] In this embodiment, the target space layout map refers to a two-dimensional / three-dimensional drawing representing the dimensions, walls, furniture, and functional zoning of the target space. The environment model refers to a digital space model constructed based on the layout map. The spatial geofence refers to the scene semantic constraints such as subspace boundaries and physical barriers defined by the environment model. The motion model refers to a mathematical model simulating the motion of a target object within the space. The preset behavioral logic rules refer to the physical constraints and motion rules that the target must follow when moving between subspaces. In an optional embodiment, taking a bathroom as an example, the bathroom can have three functional subspaces: a toilet area, a sink area, and a shower / bathtub area, and one passage area. The toilet area is a rectangular area covering the toilet itself and an area approximately 0.5 meters in front of it. The sink area is an area covering the sink and an area in front of it where standing is permitted. The shower / bathtub area is an area covering the shower or bathtub. The passage area is the passage connecting the above areas, and can have the following rules: Rule 1 (Physically Impassable): No trajectory can pass through the known location of a solid wall.

[0051] Rule 2 (Area Transfer Logic): The target cannot move instantly from the "Toilet Area" to the "Sink Area"; it must pass through the "Passage Area." The minimum transfer time from the "Toilet Area" to the "Sink Area" is set to 1 second (based on normal walking speed).

[0052] In this embodiment, after the system is powered on, the host computer obtains a complete layout diagram of the target space by importing CAD drawings or manually drawing, thus clarifying the spatial structure and functional areas. Based on the layout diagram, the host computer can perform digital modeling to reconstruct the physical information of the target space, such as walls, obstacles, and functional zones, forming a standardized environment model. The host computer then divides the target space into subspaces based on the environment model, sets boundaries and physical barriers for each subspace, and generates a scene semantic information carrier, i.e., a spatial geofence. Simultaneously, the host computer constructs a target motion model suitable for the target space and configures behavioral logic rules such as passage rules and non-crossing constraints between subspaces, providing a basis for subsequent location arbitration.

[0053] In this embodiment, the spatial geofence can be automatically generated by radar learning, without the need for manual import or drawing. The specific process is as follows: After the radar is installed, the system first enters the static environment modeling stage, continuously collecting multi-frame point cloud data when there is no human activity. By accumulating the statistical characteristics of the point cloud (such as distance, angle, signal-to-noise ratio), a three-dimensional static background model of the environment is constructed. Physical boundaries such as walls, corners, and fixed furniture are automatically identified, and multiple functional sub-regions are divided using point cloud density distribution and clustering algorithms (such as identifying the location of fixed facilities such as toilets and sinks by detecting long-term static point cloud clusters). Then, the system enters the dynamic behavior learning stage, collecting human activity trajectory data over a period of time (such as 24 hours). Through trajectory clustering and dwell point analysis, the system automatically infers the physical connection relationship between each functional sub-region (such as identifying the passage path) and the time statistical law of inter-region transfer (such as minimum transfer time), thereby generating a behavioral logic rule base. Finally, the system integrates the automatically constructed physical boundaries, functional sub-regions, connection relationships, and behavioral rules into a spatial geofence, and supports manual verification and fine-tuning.

[0054] In this embodiment, the behavioral logic rules are not completely fixed, but rather consist of a multi-layered architecture: fixed basic rules + learnable statistical rules + user-customizable rules. Specifically: Rule 1 (physical impassability) is a fixed rule, where the physical wall boundaries in the spatial geofence automatically generated by radar are directly fixed and cannot be changed; Rule 2 (area transfer logic) is a learnable rule. During the dynamic behavioral learning phase, the system continuously collects the activity trajectories of people in real-world scenarios and automatically learns parameters such as the actual travel paths, minimum transfer times, and transfer probabilities between various functional sub-areas through statistical analysis. These parameters are dynamically updated and optimized as usage data accumulates, making the rules more aligned with the actual usage habits of the space. Furthermore, users can customize rules. Users can manually correct the learning results (e.g., adjust the minimum transfer time threshold) or add special constraints (e.g., prohibit entry into a certain area during specific time periods) through configuration tools to meet personalized needs or security scenario requirements. This design ensures that spatial constraints are physically reasonable while also possessing scene adaptability and the flexibility of manual intervention.

[0055] This application provides physical common sense filtering for trajectory prediction by constructing scene semantic constraints and behavioral rules, thereby avoiding unreasonable prediction results such as passing through walls or instantly crossing regions.

[0056] In an optional embodiment, multiple predicted positions for each detected object at the next time step are calculated, and the multiple predicted positions are arbitrated based on scene semantic information of the target space and preset behavioral logic rules to obtain the optimal predicted position, including: Step 1: Input the current position of each detected object and the set of states associated with the detected object into the extended Kalman filter, so as to use the extended Kalman filter to predict multiple candidate positions of the detected object at the next moment based on the current position of the detected object and its historical trajectory in the set of states, and use these as multiple predicted positions; Step 2: Map the current location and multiple predicted locations of the detected object to the spatial geofence of the target space; Step 3: Drive the motion model to move sequentially from the current location to multiple predicted locations within the spatial geofence, thereby extracting the predicted location that conforms to the preset behavioral logic rules as the best predicted location.

[0057] In this embodiment, the Extended Kalman Filter (EPF) refers to a filtering algorithm applicable to nonlinear motion systems, used for target state prediction and trajectory smoothing. Candidate predicted positions refer to multiple possible positions for the next time step output by the EPF. Mapping refers to the operation of transforming the target position coordinates to the coordinates corresponding to a spatial geofence.

[0058] In this embodiment, the host computer inputs the current position of the detected object and its associated historical trajectory into an extended Kalman filter. The filter generates multiple candidate predicted positions for the next moment based on motion state prediction. The host computer converts the current position of the detected object and all candidate predicted positions into corresponding coordinates within a spatial geofence to match the predicted positions with scene semantic information. Then, it drives a motion model to simulate the detected object moving from its current position to each candidate predicted position, selecting the candidate positions that conform to behavioral logic rules and have no spatial conflicts as the optimal predicted positions.

[0059] Specifically, in each frame's prediction and update loop, the extended Kalman filter predicts the next position of the detected object. Subsequently, the multi-object tracking and inference engine maps the object's current position and multiple predicted positions to a pre-loaded geofence. The system then arbitrates based on behavioral logic rules. For example, if a predicted position indicates that the detected object "jumps" directly from the toilet area to the sink area, violating "Rule 2," the system will forcibly set the confidence of that predicted position to zero or reduce it to an extremely low level. Conversely, another logically sound predicted position that requires the detected object to pass through a "passage area" will be adopted. This is equivalent to using a powerful "common sense" filter to effectively eliminate a large number of unreasonable tracking results.

[0060] This application improves the rationality of prediction by using trajectory prediction and arbitration based on spatial constraints, ensuring both prediction accuracy and that the trajectory conforms to spatial constraints.

[0061] In an optional embodiment, the method further includes: Step 1: Identify the first object that existed in the previous moment but was lost in the current moment, and determine the position where the first object appeared in the previous moment as the last appearance position; Step 2: Calculate the average identity signature vector of the first object based on multiple identity signature vectors of the first object in the most recent preset number of frames; Step 3: Store the average identity signature vector into the lost target list and start the loss timer; Step 4: If a second object is detected before the countdown of the lost object timer ends, calculate the similarity between the identity signature vector of the second object and all vectors in the lost object list. The second object is a new object that appears later than the current time within a preset range centered on the last appearance position of the first object. Step 5: If the similarity between the identity signature vector of the second object and the average identity signature vector of the first object is greater than or equal to a preset threshold, the motion trajectory and motion state of the second object are associated with the state set of the first object to serve as the latest motion status of the first object, and the identification mark of the first object is used to mark the second object.

[0062] In this embodiment, the first object refers to a target that existed in the previous moment but was detected as lost in the current moment. The last appearance location refers to the spatial location of the first object before it was lost. The lost target list refers to a temporary dataset storing the identity features of lost targets. The loss timer refers to a timing tool used to limit the recoverable time after a target is lost. The second object refers to a newly detected target near its last appearance location after the target is lost.

[0063] In this embodiment, the host computer determines a first object that existed in the previous moment but is currently lost, records its last appearance position, calculates the average value of the identity signature vectors of the most recent multiple frames, stores the average vector in the lost target list, and starts a loss timer. During the loss timer's countdown, if the radar detects a new second object within a preset range of the first object's last appearance position, the host computer calculates the similarity between the second object's identity signature vector and all vectors in the lost target list in real time. When the similarity between the average identity signature vectors of the second object and the first object reaches a preset threshold, they are determined to be the same target, and the motion state and trajectory of the second object are associated with the state set of the first object, restoring its original identity identifier.

[0064] Specifically, when a detected object (e.g., A) disappears from the point cloud due to occlusion, the system does not immediately delete its tracking trajectory. Instead, it marks it as "temporarily lost" and stores its average identity signature vector from the last 5 frames in a "lost target list," while starting a 10-second (configurable) timer. During this period, if a new detected object is detected within a reasonable range (e.g., within 1 meter) of A's last known location, the system immediately calculates the cosine similarity between the new object's real-time identity signature vector and all vectors in the "lost target list." If the vector similarity with A exceeds a threshold of 0.85, the system determines it to be the same person and performs identity restoration, reassociating the new object's ID with A, thus perfectly solving the ID jump problem caused by occlusion.

[0065] In the embodiments of this application, the above parameters are all exemplary preferred values ​​that can be configured in this application, rather than fixed constraints. Their settings are all based on clear technical basis and can be flexibly adjusted according to different scenario requirements. Specifically: Regarding the dimension of the identity signature vector, 128 dimensions is the preferred dimension determined based on the balance between accuracy and computational efficiency. Too low a dimension (such as 32 dimensions) will result in insufficient discrimination, while too high a dimension (such as 512 dimensions) will increase computational overhead and pose a risk of overfitting. In practical applications, the dimension can be adjusted within the range of 64 to 256 dimensions depending on the hardware computing power and the complexity of the scenario.

[0066] Regarding the similarity threshold, 0.85 is determined by taking the median value after statistically analyzing the intra-class similarity (usually 0.90-0.98) and inter-class similarity (usually 0.60-0.80) of normal samples in offline experiments. It is an adjustable parameter. If there are few people in the scene and their individual differences are significant, the threshold can be appropriately increased to reduce the risk of misidentification. If there are highly similar people of the same gender or body type, the threshold can be appropriately decreased to improve the recall rate.

[0067] Regarding the countdown timer for lost items, the 10-second countdown is a comprehensive setting that takes into account the typical duration of a person being briefly obscured in a small space (such as one person walking around another person from behind for about 3-8 seconds) and the stability of the identity signature in a short period of time. In practical applications, it can be adjusted according to the scale of the scenario. For example, it can be shortened to 5 seconds in a smaller space (such as an elevator) and extended to 15 seconds in a larger space.

[0068] Regarding the detection space range of the new detection object, 1 meter is a reasonable setting based on the movement of people in a small space. That is, the reappearance of the target within 1 meter before and after occlusion is in line with the laws of physical movement. An excessively large range will increase the risk of false association, while an excessively small range may miss the true recovery. In actual deployment, it can be adaptively adjusted according to the scene size and radar resolution.

[0069] This application achieves rapid re-identification of occluded and lost targets through identity signature vectors, effectively solving the identity jump problem caused by occlusion in small spaces and ensuring tracking continuity.

[0070] The multi-object tracking method provided in this application not only outputs smooth trajectories and stable IDs, but also possesses rich scene semantic information, which can provide a powerful perception foundation for upper-level applications such as smart bathroom, fall detection, and energy-saving control.

[0071] The following is combined Figure 5 This paper will provide an overall description of the multi-object tracking method provided in this application.

[0072] like Figure 5As shown, the overall process of the multi-object tracking method provided in this application is as follows: First, environmental modeling and spatial geofencing are performed to obtain the target spatial layout and establish environmental models and behavioral logic rules to form scene semantic constraints. Then, radar signal processing and feature extraction are performed to generate point cloud data based on millimeter-wave radar detection data and extract micro-Doppler features to calculate the identity signature vector of each detected object. Next, multi-target state initialization and association are performed to calculate the association cost based on the current position and identity signature vector, and match the detected object to the optimal state set. After the association is completed, the trajectory prediction stage based on spatial constraints is entered. Multiple predicted positions are generated using an extended Kalman filter, and the best predicted position is obtained by arbitrating scene semantic information and behavioral logic rules. During the trajectory prediction and association process, identity signature vector matching is performed simultaneously to determine whether there is a case of lost target reappearance. If the similarity exceeds a preset threshold, association is performed and identity ID is restored. If it does not exceed the threshold, the current tracking state is maintained. Finally, multi-dimensional information fusion and output are completed, integrating the current position, the best predicted position, the association result and the identity matching result, and outputting a stable, continuous and unique motion trajectory and identity identifier for each detected object, realizing high-precision and high-robust tracking of multiple people in a small space.

[0073] The above process uses micro-Doppler identity signature as the core identity basis and spatial geofencing and behavioral logic as external constraints. It deeply integrates the micro-motion characteristics of the target with the macro-scene rules, ensuring the accuracy of target association, the rationality of trajectory prediction and the continuity of identity throughout the process. It effectively solves technical problems such as point cloud aliasing, target occlusion, unreasonable trajectory and high false association rate in small space scenarios.

[0074] According to another aspect of the embodiments of this application, such as Figure 6 As shown, a multi-object tracking device is provided, comprising: The determination and calculation module 601 is used to determine the current position of multiple detected objects in the target space based on the detection data received from the millimeter-wave radar, and to calculate the identity signature vector of each detected object. The detection data is obtained by the millimeter-wave radar detecting the target space at the current moment. The association module 603 is used to calculate the association cost between each detection object and multiple state sets based on the current position and identity signature vector of each detection object, and associate each detection object with the state set with the minimum association cost. The state set is a set created in advance for each detection object to record the motion trajectory and motion state of each detection object, and the detection object corresponds one-to-one with the state set. The prediction and arbitration module 605 is used to calculate multiple predicted positions for each detected object at the next moment, and to arbitrate the multiple predicted positions based on the scene semantic information of the target space and the preset behavior logic rules to obtain the best predicted position. The output module 607 is used to output the motion trajectory of each detected object based on the current position, the best predicted position, and the association results between each detected object and multiple state sets.

[0075] It should be noted that the determination and calculation module 601 in this embodiment can be used to execute step S202 in this application embodiment, the association module 603 in this embodiment can be used to execute step S204 in this application embodiment, the prediction and arbitration module 605 in this embodiment can be used to execute step S206 in this application embodiment, and the output module 607 in this embodiment can be used to execute step S208 in this application embodiment.

[0076] It should be noted that the examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in the above embodiments. It should also be noted that the above modules, as part of the device, can be implemented in software or hardware.

[0077] Optionally, the determination and calculation module is specifically used for: performing three-dimensional fast Fourier transform processing on the detection data to generate point cloud data, wherein the point cloud data includes the distance, azimuth, pitch angle and Doppler velocity information of each point cloud; using a CFAR detector to detect each point cloud to select valid target points as detection objects; and calculating the current position of the detection object corresponding to the valid target point based on the distance, azimuth, pitch angle and Doppler velocity information of the valid target points.

[0078] Optionally, the determination and calculation module is further configured to: extract Doppler velocity information of the region where the detected object is located from multiple frames of continuous detection data, including the detection data at the current moment; perform a short-time Fourier transform on the Doppler velocity information to obtain the micro-Doppler features of the detected object; input the micro-Doppler features into a target convolutional neural network, so that after processing the micro-Doppler features using the target convolutional neural network, the identity signature vector of the detected object is output, wherein the target convolutional neural network is obtained by replacing the final classification layer with a fully connected layer and training with a millimeter-wave radar micro-motion dataset.

[0079] Optionally, the association module is specifically used to: calculate the distance between the current position of each detected object and the target position recorded in multiple state sets, as the spatial distance cost, where the target position is the position of each detected object at the previous moment; calculate the similarity between the identity signature vector of each detected object and the identity signature vectors of each detected object recorded in multiple state sets, as the identity similarity cost; and perform weighted fusion of the spatial distance cost and the identity similarity cost according to preset weights to obtain the association cost.

[0080] Optionally, the multi-object tracking device further includes a construction and tracking module, specifically used for: obtaining a layout map of the target space before arbitrating multiple predicted locations based on scene semantic information and preset behavioral logic rules of the target space; constructing an environment model of the target space based on the layout map; creating spatial geofences for each subspace of the target space based on the environment model, wherein the spatial geofences are used to represent the scene semantic information of the target space; creating a motion model that moves between the subspaces of the target space, and configuring behavioral logic rules for the motion model to move between the subspaces.

[0081] Optionally, the prediction and arbitration module is specifically used to: input the current position of each detected object and the state set associated with the detected object into the extended Kalman filter, so as to use the extended Kalman filter to predict multiple candidate positions of the detected object at the next moment based on the current position of the detected object and its historical trajectory in the state set, as multiple predicted positions; map the current position of the detected object and the multiple predicted positions to a spatial geofence of the target space; drive the motion model to move from the current position to the multiple predicted positions in the spatial geofence in sequence, thereby extracting the predicted position that conforms to the preset behavioral logic rules as the best predicted position.

[0082] Optionally, the multi-object tracking device further includes a lost object re-identification module, specifically configured to: identify a first object that existed in the previous moment and is lost in the current moment, and determine the position of the first object in the previous moment as its last appearance position; calculate the average identity signature vector of the first object based on multiple identity signature vectors of the first object in the most recent preset number of frames; store the average identity signature vector in the lost target list and start a lost object timer; if a second object is detected before the lost object timer expires, calculate the similarity between the identity signature vector of the second object and all vectors in the lost object list, wherein the second object is a new object that appears later than the current moment within a preset range centered on the last appearance position of the first object; if the similarity between the identity signature vector of the second object and the average identity signature vector of the first object is greater than or equal to a preset threshold, associate the motion trajectory and motion state of the second object with the state set of the first object as the latest motion state of the first object, and mark the second object with the identification mark of the first object.

[0083] According to another aspect of the embodiments of this application, this application provides an electronic device, such as... Figure 7 As shown, the device includes a memory 701, a processor 703, a communication interface 705, and a communication bus 707. The memory 701 stores a computer program that can run on the processor 703. The memory 701 and the processor 703 communicate through the communication interface 705 and the communication bus 707. When the processor 703 executes the computer program, it implements the steps of the above method.

[0084] The memory and processor in the aforementioned electronic devices communicate with each other via a communication bus and a communication interface. The communication bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc.

[0085] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0086] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0087] According to another aspect of the embodiments of this application, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of any of the above embodiments.

[0088] Optionally, in embodiments of this application, the computer-readable medium is configured to store program code for the processor to perform the following steps: Upon receiving detection data from the millimeter-wave radar, the current positions of multiple detected objects in the target space are determined based on the detection data, and the identity signature vector of each detected object is calculated. The detection data is obtained by the millimeter-wave radar detecting the target space at the current moment. Based on the current position and identity signature vector of each detected object, the association cost between each detected object and multiple state sets is calculated, and each detected object is associated with the state set with the minimum association cost. The state set is a set created in advance for each detected object to record the motion trajectory and motion state of each detected object, and there is a one-to-one correspondence between the detected object and the state set. Calculate multiple predicted positions for each detected object at the next time step, and arbitrate the multiple predicted positions based on the scene semantic information of the target space and the preset behavior logic rules to obtain the best predicted position; Based on the current position, the best predicted position, and the association results between each detected object and multiple state sets, the motion trajectory of each detected object is output.

[0089] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated here.

[0090] In specific implementation, the embodiments of this application can be referred to the above embodiments and have corresponding technical effects.

[0091] It is understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described herein, or combinations thereof.

[0092] For software implementation, the techniques described herein can be implemented by units that perform the functions described herein. The software code can be stored in memory and executed by a processor. The memory can be implemented in the processor or external to the processor.

[0093] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0094] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0095] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0096] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0097] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0098] If the aforementioned function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks. It should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. In the absence of further restrictions, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0099] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A multi-object tracking method, characterized by, include: Upon receiving detection data from the millimeter-wave radar, the current positions of multiple detected objects in the target space are determined based on the detection data, and the identity signature vector of each detected object is calculated. The detection data is obtained by the millimeter-wave radar detecting the target space at the current moment. Based on the current position and identity signature vector of each of the detected objects, the association cost between each of the detected objects and multiple state sets is calculated, and each of the detected objects is associated with the state set with the smallest association cost. The state set is a set created in advance for each of the detected objects to record the motion trajectory and motion state of each of the detected objects, and the detected objects correspond one-to-one with the state set. Calculate multiple predicted positions for each of the detected objects at the next time step, and arbitrate the multiple predicted positions based on the scene semantic information of the target space and preset behavioral logic rules to obtain the optimal predicted position; Based on the current position, the best predicted position, and the association results between each detected object and multiple state sets, the motion trajectory of each detected object is output.

2. The method of claim 1, wherein, Determining the current positions of multiple detected objects in the target space based on the detection data includes: The detection data is processed by three-dimensional fast Fourier transform to generate point cloud data, wherein the point cloud data includes the distance, azimuth angle, pitch angle and Doppler velocity information of each point cloud. Each point cloud is detected using a CFAR detector to select valid target points as the detection objects; The current position of the detected object corresponding to the effective target point is calculated based on the distance, azimuth angle, pitch angle, and Doppler velocity information of the effective target point.

3. The method of claim 2, wherein, The calculation of the identity signature vector for each of the detected objects includes: From multiple frames of continuous detection data, including the detection data at the current moment, the Doppler velocity information of the region where the detected object is located is extracted; The Doppler velocity information is subjected to a short-time Fourier transform to obtain the micro-Doppler features of the detected object; The micro-Doppler features are input into a target convolutional neural network. After processing the micro-Doppler features using the target convolutional neural network, the identity signature vector of the detected object is output. The target convolutional neural network is obtained by replacing the final classification layer with a fully connected layer and training it with a millimeter-wave radar micro-motion dataset.

4. The method of claim 1, wherein, The calculation of the association cost between each detected object and multiple state sets based on the current position and identity signature vector of each detected object includes: Calculate the distance between the current position of each of the detected objects and the target positions recorded in the multiple state sets as a spatial distance cost, wherein the target positions are the positions of each detected object at the previous moment; The similarity between the identity signature vector of each detected object and the identity signature vector of each detected object recorded in the multiple state sets is calculated as the identity similarity cost; The spatial distance cost and the identity similarity cost are weighted and fused according to preset weights to obtain the association cost.

5. The method of claim 1, wherein, Before arbitrating the multiple predicted locations based on scene semantic information and preset behavioral logic rules of the target space, the method further includes: Obtain the layout diagram of the target space; Construct an environmental model of the target space based on the layout diagram; Based on the environment model, spatial geofences are created for each subspace of the target space, wherein the spatial geofences are used to represent the scene semantic information of the target space; Create a motion model that moves between the various subspaces of the target space, and configure the motion model with behavioral logic rules for traversing between the various subspaces.

6. The method of claim 5, wherein, The step of calculating multiple predicted positions for each detected object at the next moment, and arbitrating the multiple predicted positions based on the scene semantic information of the target space and preset behavioral logic rules to obtain the optimal predicted position includes: The current position of each detected object and the state set associated with the detected object are respectively input into an extended Kalman filter, so as to use the extended Kalman filter to predict multiple candidate positions of the detected object at the next moment based on the current position of the detected object and the historical trajectory in the state set, which are then used as multiple predicted positions; Map the current location and multiple predicted locations of the detected object to the spatial geofence of the target space; The motion model is driven to move sequentially from the current location to multiple predicted locations within the spatial geofence, thereby extracting the predicted location that conforms to the preset behavioral logic rules as the optimal predicted location.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: Identify the first object that existed in the previous moment but was lost in the current moment, and determine the position of the first object that appeared in the previous moment as the last appearance position; Based on the multiple identity signature vectors of the first object in the most recent preset number of frames, the average identity signature vector of the first object is calculated. Store the average identity signature vector into the lost target list and start the loss timer; If a second object is detected before the countdown of the lost object timer ends, the similarity between the identity signature vector of the second object and all vectors in the lost object list is calculated, wherein the second object is a new object that appears later than the current time within a preset range centered on the last appearance position of the first object; If the similarity between the identity signature vector of the second object and the average identity signature vector of the first object is greater than or equal to a preset threshold, the motion trajectory and motion state of the second object are associated with the state set of the first object to serve as the latest motion status of the first object, and the identification identifier of the first object is used to mark the second object.

8. A multi-object tracking device, characterized by, include: The determination and calculation module is used to determine the current position of multiple detected objects in the target space based on the detection data received from the millimeter-wave radar, and to calculate the identity signature vector of each detected object, wherein the detection data is obtained by the millimeter-wave radar detecting the target space at the current moment; The association module is used to calculate the association cost between each of the detected objects and multiple state sets based on the current position and the identity signature vector of each of the detected objects, and associate each of the detected objects with the state set with the smallest association cost. The state set is a set created in advance for each of the detected objects to record the motion trajectory and motion state of each of the detected objects, and the detected objects correspond one-to-one with the state set. The prediction and arbitration module calculates multiple predicted positions for each of the detected objects at the next moment, and arbitrates the multiple predicted positions based on the scene semantic information of the target space and preset behavioral logic rules to obtain the best predicted position. The output module is used to output the motion trajectory of each detected object based on the current position, the best predicted position, and the association results between each detected object and multiple state sets.

9. A multi-object tracking system comprising a millimeter wave radar sensor and a host computer, characterized in that, The host computer is used to execute the multi-object tracking method as described in any one of claims 1 to 7.

10. An electronic device comprising a memory, a processor, a communication interface and a communication bus, the memory having stored therein a computer program executable on the processor, the memory, the processor being in communication via the communication bus and the communication interface, characterized in that, When the processor executes the computer program, it implements the multi-object tracking method according to any one of claims 1 to 7.

11. A computer readable medium having a non-transitory program code executable by a processor, the program code comprising instructions for: The program code causes the processor to execute the multi-object tracking method according to any one of claims 1 to 7.

Images