An indoor multi-target active trajectory tracking method, device and equipment based on a movable single station

Abstract

This invention relates to the field of trajectory tracking, specifically to an indoor multi-target active trajectory tracking method, apparatus, and device based on a mobile single station. Based on the autonomous mobile single station's traversal of the current edge, the historically accumulated received signal strength sequence of each moving target is input into a preset trajectory pattern inference model to obtain the probability distribution of each preset reference trajectory pattern corresponding to each moving target. The probability distributions of all moving targets are concatenated along the moving target dimension to obtain the multi-target trajectory pattern probability. A decision state is constructed based on the multi-target trajectory pattern probability, the sequence of traversed edges since the autonomous mobile single station's starting position, and the current node position. Based on the decision state and action mask, a pre-trained reinforcement learning decision network is used to obtain the action command at the current moment. By using an autonomous mobile single station to collect wireless signals at different spatial locations indoors, the system's equipment deployment, data synchronization, and maintenance costs are reduced.

Description

Technical Field

[0001] This invention relates to the field of trajectory tracking, and specifically to an indoor multi-target active trajectory tracking method, apparatus, and equipment based on a mobile single station. Background Technology

[0002] With the development of wireless communication, mobile robots, and intelligent sensing technologies, the demand for personnel positioning, trajectory analysis, and moving target monitoring in indoor environments is constantly increasing, such as in shopping mall pedestrian monitoring, hospital staff care, warehouse target management, and underground space personnel search. Because the Global Positioning System (GPS) is significantly affected by building obstruction in indoor environments, satellite signals are easily attenuated, making it difficult to provide stable and continuous location services. Therefore, positioning and tracking technologies based on indoor wireless signals have received widespread attention.

[0003] Existing indoor positioning technologies mainly include positioning methods based on Wi-Fi, Bluetooth, UWB, RFID, ultrasound, vision, and LiDAR. Among them, Wi-Fi has advantages such as wide coverage, well-established existing infrastructure, and strong terminal compatibility, enabling target location awareness with minimal additional dedicated equipment, making it highly valuable. However, Wi-Fi positioning systems typically require the deployment of multiple fixed receiving nodes, resulting in high costs for equipment deployment, data synchronization, and maintenance. Summary of the Invention

[0004] The purpose of this invention is to provide an indoor multi-target active trajectory tracking method, device, and equipment based on a mobile single station, which solves the problems in the prior art.

[0005] This invention is achieved through the following technical solution:

[0006] In a first aspect, embodiments of the present invention provide an indoor multi-target active trajectory tracking method based on a mobile single station. The indoor space is pre-divided into a topological graph consisting of several nodes and edges connecting adjacent nodes. Nodes are locations reachable by the autonomous mobile single station, and edges are basic path units connecting adjacent nodes that can be traversed by the autonomous mobile single station. The method includes:

[0007] Based on the autonomous mobile station's traversal of the current edge, the wireless received signal strength sequence of each mobile target in the indoor space is collected.

[0008] The historical accumulated received signal strength sequence of each moving target is input into the preset trajectory pattern inference model to obtain the probability distribution of each preset reference trajectory pattern corresponding to each moving target. The trajectory pattern inference model is trained based on the received signal strength sequence of each moving target collected on each edge and its corresponding reference trajectory pattern label.

[0009] By concatenating the probability distributions of all moving targets along the dimensions of the moving targets, the probability of multi-target trajectory patterns is obtained.

[0010] The decision state is constructed based on the multi-target trajectory pattern probability, the sequence of traversed edges that the autonomous mobile single station has traversed since its starting position, and the current node position.

[0011] Based on the decision state and action mask, the action instruction at the current moment is obtained using a pre-trained reinforcement learning decision network. The action instruction is to traverse the newly determined current edge or terminate the tracking process. The action mask is used to set the selection probability of the non-executable action to zero. The reinforcement learning decision network is a policy network pre-trained using a reinforcement learning algorithm based on state transition samples generated by the interaction between the autonomous mobile station and the indoor environment. The state transition samples include the decision state, the action to be executed, the immediate reward, the next decision state, and the end-of-round flag.

[0012] Preferably, the historically accumulated received signal strength sequence of each moving target is input into a preset trajectory pattern inference model to obtain the probability distribution of each preset reference trajectory pattern corresponding to each moving target, including:

[0013] The historical accumulated received signal strength sequence of each moving target is input into the channel interaction module to obtain channel association features. The channel interaction module is used to exchange information between channels on the input features of received signal strength, sampling position and edge order.

[0014] The channel association features are input into multiple multi-scale dilated convolution branches with different kernel sizes and dilation rates to extract and fuse the local received signal intensity variation features in different time ranges, thereby obtaining multi-scale local features containing multi-time scale information.

[0015] Multi-scale local features are input into the self-attention module to obtain global correlation features;

[0016] The global association features are input into the long short-term memory network in the order of edge traversal to obtain the sequence features with cross-edge temporal dependencies.

[0017] The sequence features are input into the attention pooling module and the max pooling module to obtain the attention pooling result and the max pooling result, respectively.

[0018] The attention pooling result and the max pooling result are concatenated and then input into the linear classification module to obtain the probability distribution of each preset reference trajectory pattern corresponding to each moving target.

[0019] Preferably, the trajectory pattern inference model is trained in the following manner:

[0020] The autonomous mobile station is controlled to traverse different edges and collect the received signal strength sequence of each training mobile target on each edge.

[0021] The received signal strength sequences of each training moving target on each edge are combined in traversal order to form a cumulative fingerprint sequence;

[0022] The accumulated fingerprint sequence and the corresponding reference trajectory pattern label are used as a training sample. The reference trajectory pattern label is the number of the preset reference trajectory pattern corresponding to the actual walking trajectory of the training moving target.

[0023] Zero-padding is performed on accumulated fingerprint sequences that are shorter than the set length to generate an effective length mask to distinguish between valid data positions and zero-padding positions;

[0024] The cumulative fingerprint sequence and the effective length mask are used together as input to the model to be trained to obtain the predicted probability;

[0025] The difference between the predicted probability and the reference trajectory pattern label is calculated using cross-entropy loss. The parameters of the model to be trained are updated through backpropagation to obtain the trajectory pattern inference model.

[0026] Preferably, a decision state is constructed based on the multi-target trajectory pattern probability, the sequence of traversed edges since the starting position of the autonomous mobile station, and the current node position, including:

[0027] Determine historical path information based on the traversed edge sequence;

[0028] Determine the current location information based on the current node's location;

[0029] The path progress information is determined based on the ratio of the number of edges already traversed to the preset maximum number of edges traversed.

[0030] The probability of multi-target trajectory patterns, historical path information, current location information, and path progress information are combined to form the decision state.

[0031] Preferably, the reinforcement learning decision network is trained in the following manner:

[0032] Based on the trajectory pattern combination of each training moving target, the current training position and current training edge of the autonomous mobile station, the autonomous mobile station is controlled to traverse the current training edge from the current training position and collect the training received signal strength sequence of each training moving target on the current training edge.

[0033] By inputting the historical accumulated training received signal strength sequence of each training moving target into the trajectory pattern inference model of the frozen parameters, the probability distribution of each preset reference trajectory pattern corresponding to each training moving target and the probability of the spliced ​​multi-target trajectory pattern are obtained.

[0034] The current decision state is constructed based on the multi-target trajectory pattern probability, the sequence of traversed edges since the starting position of the autonomous mobile single station since the training start position, and the current training node position.

[0035] An action mask is generated based on the connection relationship between the current training node position and each edge in the topology graph. The current decision state and the action mask are input into the Actor network to be trained. The Actor network outputs the action instruction at the current moment. The action instruction is to select an edge adjacent to the current training node position to be traversed or to terminate the tracking process.

[0036] The current decision state is input into the Critic network to be trained, and the Critic network outputs a value estimate of the current decision state.

[0037] Execute an action command to obtain an immediate reward and the next decision state. The immediate reward is determined based on the type of action command, and the next decision state is the decision state reconstructed after executing the action command.

[0038] The current decision state, action instructions, immediate rewards, next decision state, and value estimate are combined to form a state transition sample and stored in the experience cache.

[0039] When the experience cache reaches the preset length or the current training round ends, the reward and advantage estimate are calculated based on the instantaneous reward and value estimate of each state transition sample stored in the experience cache.

[0040] The parameters of the Actor network are updated based on the reward and advantage estimates using the proximal policy optimization algorithm. The value loss is calculated using the difference between the value estimate and the reward. The parameters of the Critic network are then updated based on the value loss, resulting in a trained reinforcement learning decision network.

[0041] Preferably, the instant reward is determined in the following manner:

[0042] When the action instruction is to select an edge to be traversed that is adjacent to the current training node position, the immediate reward decreases as the ratio of the number of traversed edges to the preset maximum number of traversed edges increases.

[0043] When the action command is to select to terminate the tracking process, the immediate reward increases as the average trajectory tracking error of each training moving target decreases.

[0044] Preferably, the action mask is determined in the following way:

[0045] Get all adjacent edges of the current node in the topology graph;

[0046] Mark each adjacent edge as an executable action and the non-adjacent edge as an unexecutable action, and generate an action mask.

[0047] Preferably, after terminating the tracking process, the method further includes:

[0048] For each moving target, the multiple reference trajectories with the highest predicted probability are selected as candidate reference trajectories;

[0049] Normalized weights are calculated based on the predicted probabilities of each candidate reference trajectory;

[0050] The coordinate sequences of each candidate reference trajectory are weighted and fused according to normalized weights to obtain the estimated trajectory of each moving target.

[0051] Secondly, embodiments of the present invention provide an indoor multi-target active trajectory tracking device based on a mobile single station. The indoor space is pre-divided into a topological graph consisting of several nodes and edges connecting adjacent nodes. Nodes are locations reachable by the autonomous mobile single station, and edges are basic path units connecting adjacent nodes that can be traversed by the autonomous mobile single station. The method includes:

[0052] The acquisition module is used to acquire the wireless received signal strength sequence of each mobile target in the indoor space based on the autonomous mobile station's traversal of the current edge.

[0053] The probability distribution module is used to input the historical accumulated received signal strength sequence of each moving target into the preset trajectory pattern inference model to obtain the probability distribution of each moving target for each preset reference trajectory pattern. The trajectory pattern inference model is trained based on the received signal strength sequence collected by each moving target on each edge and its corresponding reference trajectory pattern label.

[0054] The stitching module is used to stitch together the probability distributions of all moving targets according to the moving target dimension to obtain the probability of multi-target trajectory patterns.

[0055] The construction module is used to construct the decision state based on the multi-target trajectory pattern probability, the sequence of traversed edges that the autonomous mobile station has traversed since its starting position, and the current node position.

[0056] The decision module is used to obtain the action instruction at the current moment using a pre-trained reinforcement learning decision network based on the decision state and action mask. The action instruction is to traverse the newly determined current edge or terminate the tracking process. The action mask is used to set the selection probability of the non-executable action to zero. The reinforcement learning decision network is a policy network pre-trained using a reinforcement learning algorithm based on state transition samples generated by the interaction between the autonomous mobile station and the indoor environment. The state transition samples include the decision state, the action to be executed, the immediate reward, the next decision state, and the end of the round flag.

[0057] Thirdly, embodiments of the present invention provide an electronic device, including: at least one processor, at least one memory, and computer program instructions stored in the memory, which, when executed by the processor, implement the method of the first aspect described above.

[0058] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0059] (1) Reduce reliance on multiple fixed receiving stations. By using a single autonomous mobile station to collect wireless signals at different indoor locations, the system's equipment deployment, data synchronization, and maintenance costs are reduced.

[0060] (2) Achieve synchronous trajectory tracking of multiple moving targets. By accumulating the moving target pattern fingerprint and trajectory pattern probability inference, the autonomous mobile station can obtain the estimated trajectory of multiple moving targets along a common path, avoiding repeated edge traversal caused by tracking different targets separately.

[0061] (3) Improve the effectiveness of wireless signal acquisition. It can comprehensively utilize the historical path, current location, and trajectory pattern probability of multiple moving targets of an autonomous mobile station to adaptively select effective edges to be traversed, reducing redundant observations generated by fixed paths or manual rules.

[0062] (4) Balancing trajectory tracking accuracy and system real-time performance. By jointly considering the average trajectory tracking error of multiple targets and the path length of a single mobile station, the autonomous mobile station can terminate tracking in a timely manner after obtaining sufficient trajectory information, reducing unnecessary side traversals. Attached Figure Description

[0063] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. In the drawings:

[0064] Figure 1 This is a schematic diagram of an indoor multi-moving target active trajectory tracking scenario provided by the present invention;

[0065] Figure 2 A flowchart illustrating the indoor multi-target active trajectory tracking method based on a mobile single station provided by the present invention;

[0066] Figure 3 A schematic diagram of the trajectory pattern inference model provided by the present invention;

[0067] Figure 4This is a schematic diagram of reinforcement learning decision network training provided by the present invention;

[0068] Figure 5 This is a schematic diagram of the offline training and online execution process provided by the present invention. Detailed Implementation

[0069] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0070] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0071] It should be noted that all actions involving the acquisition of signals, information, or data in this invention are carried out in compliance with the relevant data protection laws and regulations of the locality and with authorization from the owner of the relevant device.

[0072] Example 1

[0073] Please see Figure 1 This invention provides an indoor multi-target active trajectory tracking method based on a mobile single station. The indoor space is pre-divided into a topology graph consisting of several nodes and edges connecting adjacent nodes. Nodes are locations reachable by the autonomous mobile single station, and edges are basic path units connecting adjacent nodes that can be traversed by the autonomous mobile single station. For example, in an indoor corridor scenario, corridor intersections are set as nodes, and corridor segments between intersections are set as edges. The autonomous mobile single station completes the traversal of an edge by moving from one end to the other along a certain section of the corridor.

[0074] like Figure 2 As shown, the method includes:

[0075] S1. Based on the autonomous mobile station's traversal of the current edge, collect the wireless received signal strength sequence of each mobile target in the indoor space.

[0076] Here, the current edge refers to a basic path unit in the topology graph traversed by the autonomous mobile station at the current moment.

[0077] A wireless received signal strength sequence (RSSI) refers to a time series formed by an autonomous mobile station continuously collecting the Received Signal Strength Indication (RSSI) values ​​of wireless signals emitted by each mobile target at preset sampling intervals while traversing an edge. The mobile target is the tracked object carrying a wireless signal transmitting device, such as a person carrying a Wi-Fi communication device. Each mobile target corresponds to one RSSI sequence on each edge. In one embodiment, the number of sampling points on each edge is 24, and the length of the RSSI sequence for each mobile target on each edge is 24.

[0078] Specifically, an autonomous mobile station, equipped with a wireless signal receiving device, continuously scans and records the wireless signals emitted by each moving target as it moves along the edges of the topology graph. During indoor propagation, wireless signals are affected by distance, obstacle obstruction, and multipath effects, resulting in differences in received signal strength at different spatial locations. This difference reflects the spatial positional relationship of the moving target relative to the autonomous mobile station. As the autonomous mobile station traverses the current edge, the distance between the mobile station and each moving target continuously changes along the edge, causing the acquired received signal strength sequence to exhibit time-varying characteristics related to the moving target's trajectory. By acquiring the received signal strength sequence of each moving target on the current edge, current observation data is provided for subsequent trajectory pattern inference.

[0079] S2. Input the historical accumulated received signal strength sequence of each moving target into the preset trajectory pattern inference model to obtain the probability distribution of each preset reference trajectory pattern corresponding to each moving target. The trajectory pattern inference model is trained based on the received signal strength sequence collected by each moving target on each edge and its corresponding reference trajectory pattern label.

[0080] The historically accumulated received signal strength sequence refers to the set of sequences formed by piecing together the received signal strength sequences of a certain moving target collected on each edge traversed by the autonomous mobile station since its starting position, arranged in chronological order of traversal. As the autonomous mobile station continuously traverses new edges, the received signal strength sequence on the current edge is added to the historically accumulated sequence of the moving target, forming an accumulated fingerprint containing cross-edge temporal information.

[0081] The trajectory pattern inference model is a pre-trained neural network model. The model takes the historical cumulative received signal strength sequence of each moving target as input and the probability distribution of each preset reference trajectory pattern corresponding to each moving target as output.

[0082] The preset reference trajectory pattern refers to multiple typical walking trajectories pre-set in the offline stage. Each trajectory is formed by several edges in the topology graph arranged in the walking order. In one implementation, the number of preset reference trajectory patterns is 324, and each reference trajectory corresponds to a trajectory pattern category.

[0083] The reference trajectory pattern label is a unique identifier for each preset reference trajectory pattern, used to label the trajectory category of training samples during model training. The trajectory pattern inference model is trained offline based on the received signal strength sequences collected from each moving target along each edge and their corresponding reference trajectory pattern labels. This model can extract local signal variation features and cross-edge temporal dependency features from the cumulative received signal strength sequence, thereby establishing a mapping relationship between the cumulative received signal strength sequence and the reference trajectory pattern.

[0084] Specifically, each time an autonomous mobile station completes the traversal of a new edge, the currently acquired received signal strength sequence is added to the historical cumulative sequence of the corresponding mobile target. Since the actual walking paths of each mobile target are different, the received signal strength variation patterns exhibited by the same mobile target on different edges differ, and this difference information is scattered across the time dimension. The received signal strength sequences from multiple edges are accumulated in chronological order and then input into the trajectory pattern inference model. The model can comprehensively utilize the local signal features on each edge as well as the temporal transition features between edges, outputting the probability distribution of each mobile target across all preset reference trajectory patterns. The higher the probability value corresponding to a certain reference trajectory pattern, the closer the actual trajectory of the mobile target is to that reference trajectory pattern. Thus, newly added observation data on each traversed edge participates in trajectory pattern inference through cumulative input, achieving continuous updates to the trajectory tendencies of each mobile target.

[0085] In some implementations, S2 involves inputting the historically accumulated received signal strength sequence of each moving target into a preset trajectory pattern inference model to obtain the probability distribution of each preset reference trajectory pattern corresponding to each moving target, including:

[0086] S21. Input the historical accumulated received signal strength sequence of each moving target into the channel interaction module to obtain channel association features. The channel interaction module is used to exchange information between channels on the input features of received signal strength, sampling position and edge order.

[0087] The channel interaction module is the initial feature processing component in the trajectory pattern inference model. This module is used to exchange information between channels on three types of input features: received signal strength, sampling location, and edge order. Taking a single moving target as an example, when the autonomous mobile station has traversed... When the edge is defined, the historical cumulative received signal strength sequence of the moving target. Represented as:

[0088] ;

[0089] in, Indicates the first The moving target in the first A mobile target pattern fingerprint that has been traversed along an edge is composed of the sequence of received signal strengths along that edge, with the sequence length denoted as . Each sampling point carries the following features: Received Signal Strength (RSS), the position number of the sampling point within the edge (sampling position), and the traversal number of the edge to which the sampling point belongs (edge ​​order).

[0090] The channel association feature is the output of the channel interaction module. This feature incorporates the interaction information between channels while retaining the original information of each channel. In one implementation, the channel interaction module adopts a combination structure of channel-by-channel convolution and channel rearrangement. After multiple information exchanges between the received signal strength channel, sampling position channel, and side-order channel, the features of each channel are no longer independent of each other, but carry association information from other channels.

[0091] Specifically, received signal strength, sampling location, and edge order describe the information in the accumulated fingerprint sequence from different dimensions, and there is an inherent correlation among the three. For example, the changing trend of received signal strength at different sampling locations on the same edge reflects the continuous impact of distance changes on the signal as an autonomous mobile station moves along that edge; the changing received signal strength at the same sampling location between different edges reflects the changes in the signal propagation environment caused by the spatial location differences of each edge. If the three types of features are processed independently, the coupling information between the feature dimensions is lost. The channel interaction module establishes information exchange paths along the feature channel dimensions, enabling the feature expressions of each channel to complement and enhance each other, obtaining channel-related features that integrate spatial location, temporal order, and signal strength information.

[0092] S22. Input the channel association features into multiple multi-scale dilated convolution branches with different kernel sizes and dilation rates, extract the local received signal intensity change features in different time ranges and fuse them to obtain multi-scale local features containing multi-time scale information.

[0093] The multi-scale dilated convolution branch refers to multiple parallel convolutional structures employing different kernel sizes and dilation rates. For the input multi-scale local features, each branch uses a different receptive field to perform convolution operations on the channel-related features. The kernel size is denoted as... The expansion rate is denoted as Both factors together determine the receptive field of this branch. Dilated convolution expands the receptive field by inserting zeros between the kernel elements, capturing signal variation features over a longer time range without increasing the number of parameters.

[0094] Multi-scale local features refer to features obtained by concatenating the outputs of each multi-scale dilated convolution branch along the channel dimension and then fusing them. These features contain local signal variation information across multiple time scales. In one implementation, three parallel branches are set up, employing a smaller kernel and dilation rate combination to capture short-term signal fluctuations, a medium kernel and dilation rate combination to capture signal trends over medium time spans, and a larger kernel and dilation rate combination to capture long-term signal evolution patterns.

[0095] Specifically, as each moving target traverses each edge, its received signal strength sequence exhibits different variation characteristics at different time scales due to variations in walking speed and path length. Short-term fluctuations in received signal strength reflect the instantaneous impact of local position changes on the signal during the autonomous mobile station's movement. Medium-term trends reflect the signal attenuation or enhancement patterns as the moving target moves along the entire edge. Long-term evolution patterns reflect the overall trajectory characteristics of the moving target when traversing multiple edges. Using only a single-scale convolutional kernel is insufficient to simultaneously capture signal features at these multiple time scales. Multi-scale dilated convolution branches, by using convolution operations of multiple scales in parallel, extract local received signal strength variation features within different time ranges from each branch. The outputs of each branch are then concatenated and fused to obtain multi-scale local features containing information from multiple time scales.

[0096] S23. Input the multi-scale local features into the self-attention module to obtain the global correlation features;

[0097] The self-attention module is a neural network component that uses a self-attention mechanism. This module assigns learnable weights to features at different sampling positions and on different traversed edges in the input features.

[0098] The global correlation feature is the output of the self-attention module. This feature is obtained by weighted aggregation of elements in multi-scale local features, where the weights are determined by the correlation between the elements. In one implementation, the self-attention module maps the input features to a query vector, a key vector, and a value vector. It calculates the similarity between the query vector and each key vector as the attention weight, and then uses this weight to perform a weighted summation of the value vectors. This module does not rely on the absolute position of the features in the sequence, but rather calculates the relative importance between elements based on the feature content itself.

[0099] Specifically, the multi-scale local features include signal features corresponding to multiple sampling locations and multiple traversed edges, and these features contribute differently to trajectory pattern discrimination. For example, the received signal strength features at some sampling locations may be affected by environmental noise and carry less effective information, while the signal features on some edges may have unclear correspondence with the trajectory due to severe multipath effects. The self-attention module can automatically calculate the attention weight of each element based on the global correlation between features, so that features that contribute more to trajectory pattern discrimination receive higher aggregation weights, and features that contribute less receive lower aggregation weights. Thus, in the globally correlated features obtained after weighted aggregation, information related to trajectory pattern discrimination is strengthened, while irrelevant or interfering information is suppressed.

[0100] S24. Input the global association features into the long short-term memory network according to the edge traversal order to obtain the sequence features with cross-edge temporal dependencies.

[0101] Among them, Long Short-Term Memory (LSTM) network is a recurrent neural network structure that can process sequential data. This network selectively remembers or forgets information at different time steps in the sequence through a gating mechanism.

[0102] The sequential features with cross-edge temporal dependencies are the output of the Long Short-Term Memory (LSTM) network. These features preserve the temporal transition information between edges according to the edge traversal order. In one implementation, the LSM network takes the global association feature corresponding to each edge as input for one time step, processes the edges sequentially according to the traversed order, updates its hidden state edge by edge, and finally incorporates the information of all previous edges into the output sequence features.

[0103] Specifically, the trajectory of each moving target is composed of multiple edges arranged in temporal order. The spatial continuity and physical connectivity between adjacent edges create temporal correlations between the signal features of the moving target on adjacent edges. For example, when a moving target transitions from edge AB to edge BE, there is continuity between the signal features collected by the autonomous moving station at the end of the previous edge and the signal features collected at the beginning of the next edge. This continuity reflects the speed maintenance and direction change information of the moving target as it passes through node B. The self-attention module processes an unordered set of features and lacks the ability to capture sequential information. The Long Short-Term Memory (LSTM) network processes the features of each edge step by step according to the order in which the edges have been traversed. Its gating mechanism can maintain cross-edge information transmission in multi-step sequences, obtaining sequence features containing cross-edge temporal dependencies.

[0104] S25. Input the sequence features into the attention pooling module and the max pooling module to obtain the attention pooling result and the max pooling result, respectively;

[0105] The attention pooling module is a pooling component that uses an attention mechanism to weighted aggregate elements in the sequence features to obtain a fixed-length vector. This module calculates an attention weight for each element in the sequence features, which reflects the importance of that element in trajectory pattern discrimination. The attention pooling result is obtained by weighted summation of the elements based on this weight.

[0106] Max pooling is a pooling component that takes the maximum value of each element in a sequence feature along each dimension to obtain a fixed-length vector. Max pooling preserves the strongest response value in each dimension of the sequence feature.

[0107] Specifically, feature elements at different positions in a sequence feature have different importance and representations in trajectory pattern discrimination. Some elements may carry unique signal features that distinguish them from other trajectory patterns. Even if these features have low overall amplitude, they are crucial for differentiating different trajectory categories. The attention pooling module can retain these discriminative low-amplitude features through content-based attention weights. Other elements have extremely strong response values ​​in specific dimensions. The signal features corresponding to these strong response dimensions can directly indicate certain attributes of the trajectory. The max pooling module retains these strongest response features by taking the maximum value of each dimension. The two pooling methods extract key information from the sequence features from the perspectives of aggregating importance and retaining extreme values ​​of intensity, respectively. The attention pooling results and max pooling results complement each other.

[0108] S26. After concatenating the attention pooling result and the max pooling result, input them into the linear classification module to obtain the probability distribution of each preset reference trajectory pattern corresponding to each moving target.

[0109] The linear classification module refers to a neural network component consisting of one or more fully connected layers. This module maps the input feature vector to the output space, the dimension of which is equal to the total number of preset reference trajectory patterns. In one implementation, the output of the linear classification module is processed by a normalized exponential function to convert the output value into a probability distribution form.

[0110] The probability distribution of each moving target corresponding to each preset reference trajectory pattern is the output of the linear classification module. Each element in this distribution represents the probability value of a moving target corresponding to a specific preset reference trajectory pattern. For the... A moving target, whose output is a... A dimensional vector, where In one implementation method, to preset the total number of reference trajectory patterns, It equals 324.

[0111] Specifically, attention pooling retains the overall discriminative information after aggregating elements at different positions in the sequence features according to attention weights, while max pooling retains the strongest response information in each dimension of the sequence features. After concatenating the two pooling results along the feature dimension, the concatenated vector contains both complementary pooling information, and this concatenated vector serves as the input to the linear classification module. The linear classification module maps the concatenated vector to the trajectory pattern category space through a linear transformation, outputting a score corresponding to each preset reference trajectory pattern. A higher score indicates a higher degree of matching between the input historical cumulative received signal strength sequence and the reference trajectory pattern. After processing with a normalized exponential function, each score is converted into a probability form, and the sum of the probabilities of all preset reference trajectory patterns is 1. The reference trajectory pattern with the highest probability is the trajectory most likely to be followed by the moving target, inferred by the model.

[0112] like Figure 3 The diagram shows the structure of the trajectory pattern inference model. The model consists of an input layer, a channel interaction module, a dilated convolution module, a dropout layer, a self-attention module, a long short-term memory (LSTM) network module, a dual-pooling fusion module, and a linear classification module, all connected sequentially. The dual-pooling fusion module comprises parallel attention pooling and max pooling. The figure illustrates the model's macro-architecture (a), the internal structure of the channel interaction module (b), the branch structure of the dilated convolution module (c), and the internal structure of the attention pooling module (d).

[0113] The historical cumulative received signal strength sequence output from the input layer is connected to the input of the channel interaction module. The output of the channel interaction module is connected to the input of each multi-scale dilated convolution branch. The output of each multi-scale dilated convolution branch is connected to the input of its corresponding random deactivation layer. The outputs of each random deactivation layer are concatenated along the channel dimension and then connected to the input of the self-attention module. The output of the self-attention module is connected to the input of the Long Short-Term Memory (LSTM) network module in edge traversal order. The output of the LSM network module is connected to the inputs of the attention pooling module and the max pooling module. The outputs of the attention pooling module and the max pooling module are concatenated along the feature dimension and then connected to the input of the linear classification module. The output of the linear classification module is connected to the normalized exponential function (Softmax), whose output represents the probability distribution of each moving target corresponding to each preset reference trajectory pattern in the final output of the model.

[0114] The internal connection of the channel interaction module is as follows: the input features are sequentially processed by 1x1 convolution (1x1Conv), channel-wise convolution (Depthwise Conv), channel shuffle (Channel Shuffle), and 1x1 convolution before output.

[0115] The internal connections of the multi-scale dilated convolution branch are as follows: the input features pass through a one-dimensional dilated convolution layer (DilatedConv1d), a batch normalization layer (BatchNorm), a linear rectified activation function (ReLU), and a channel attention module (SEBlock) in sequence before being output.

[0116] The internal connections of the attention pooling module are as follows: the input features pass through the first linear layer (Linear), the hyperbolic tangent activation function (Tanh), the second linear layer (Linear), and the normalized exponential function (Softmax) in sequence, and then the attention weights are output.

[0117] In some implementations, the trajectory pattern inference model is trained in the following manner:

[0118] The autonomous mobile station is controlled to traverse different edges and collect the received signal strength sequence of each training mobile target on each edge.

[0119] The received signal strength sequences of each training moving target on each edge are combined in traversal order to form a cumulative fingerprint sequence;

[0120] The accumulated fingerprint sequence and the corresponding reference trajectory pattern label are used as a training sample. The reference trajectory pattern label is the number of the preset reference trajectory pattern corresponding to the actual walking trajectory of the training moving target.

[0121] Zero-padding is performed on accumulated fingerprint sequences that are shorter than the set length to generate an effective length mask to distinguish between valid data positions and zero-padding positions;

[0122] The cumulative fingerprint sequence and the effective length mask are used together as input to the model to be trained to obtain the predicted probability;

[0123] The difference between the predicted probability and the reference trajectory pattern label is calculated using cross-entropy loss. The parameters of the model to be trained are updated through backpropagation to obtain the trajectory pattern inference model.

[0124] Among them, the training moving targets refer to the moving targets set up in the offline training phase to collect training data. These targets move in the indoor space according to the preset reference trajectory pattern, and their actual walking paths are known.

[0125] A received signal strength sequence refers to a time series of received signal strength indicators (RSS) of a specific training mobile target continuously collected by an autonomous mobile station at preset sampling intervals while traversing a certain edge. The autonomous mobile station traverses different edges, collecting the RSS sequences of each training mobile target on each edge. Each training mobile target on each edge corresponds to one RSS sequence, and the length of this sequence is... In one implementation It equals 24.

[0126] The cumulative fingerprint sequence is a spliced ​​sequence formed by combining the received signal strength sequences of the same training moving target on different traversed edges in traversal order. For the first... A training moving target, when the autonomous moving single station has been traversed When there are edges, the cumulative fingerprint sequence of the target Represented as:

[0127] ;

[0128] in Indicates the first The training moving target is in the... The fingerprint of the moving target pattern has been traversed.

[0129] A reference trajectory pattern label is a unique identifier for a preset reference trajectory pattern corresponding to the actual trajectory of the training moving target. In one implementation, the number of preset reference trajectory patterns... The number is 324, and each reference trajectory corresponds to a label number ranging from 1 to 324. Pairing the accumulated fingerprint sequence with the corresponding reference trajectory pattern label constitutes a training sample.

[0130] The effective length mask is a binary vector of the same length as the cumulative fingerprint sequence, used to mark the positions of valid data and zero-padding positions in the cumulative fingerprint sequence. Since different reference trajectory patterns contain different numbers of edges, the lengths of the cumulative fingerprint sequences of each training sample are inconsistent. To facilitate batch input for model training, the length of all cumulative fingerprint sequences is standardized: cumulative fingerprint sequences shorter than the set length are padded with zeros at the end to reach the preset uniform length; simultaneously, an effective length mask is generated, with the mask positions corresponding to valid data marked as 1 and the mask positions corresponding to zero-padding data marked as 0. During model training, the cumulative fingerprint sequence and the effective length mask are used together as input to the model to be trained. The effective length mask allows the model to ignore data at zero-padding positions during feature extraction, avoiding interference from invalid data in model parameter updates.

[0131] Cross-entropy loss is a loss function used to measure the difference between the predicted probability distribution and the true label.

[0132] Specifically, after receiving the accumulated fingerprint sequence and the effective length mask, the model to be trained outputs the predicted probability distribution of each training moving target corresponding to each preset reference trajectory pattern. Each element in this distribution represents the predicted probability that the input sequence belongs to a certain reference trajectory pattern. The reference trajectory pattern label serves as the true category, representing the actual reference trajectory pattern corresponding to the training sample. The cross-entropy loss calculates the negative logarithm of the probability value corresponding to the true label position in the predicted probability; the smaller this value, the closer the predicted probability is to the true label. Using the cross-entropy loss as the optimization objective, the gradient of the loss relative to each parameter of the model to be trained is calculated through the backpropagation algorithm. The model's weights and bias parameters are updated layer by layer along the gradient descent direction, so that the model's predicted probability of the training sample gradually approaches the true label. In one implementation, the Adam optimizer is used for parameter updates, the batch size is set to 512, training is performed for 100 epochs, and the initial learning rate is set to... The training process is repeated. After each round of training, validation samples are used to evaluate the model performance. The model parameters with the minimum cross-entropy loss on the validation samples are selected as the parameters of the trained trajectory pattern inference model. The trained trajectory pattern inference model establishes a mapping relationship from the cumulative fingerprint sequence to the reference trajectory pattern. The input cumulative fingerprint sequence contains the received signal strength information of the training moving target on multiple traversed edges. The model extracts the local signal features on each edge and the temporal transition features across edges, and outputs the probability distribution of each preset reference trajectory pattern corresponding to the training moving target. The reference trajectory pattern with the highest probability is the inference result of the model on the trajectory of the training moving target.

[0133] S3. Concatenate the probability distributions of all moving targets according to the moving target dimension to obtain the multi-target trajectory pattern probability;

[0134] Here, concatenating by moving target dimension refers to combining the probability distribution vectors of all moving targets into a matrix according to their index order. In step S2, the trajectory pattern inference model outputs a probability vector for each moving target with a length equal to the total number of preset reference trajectory patterns. Each element in this vector represents the probability value of the moving target for each reference trajectory pattern. The probability vectors of all moving targets are then arranged sequentially according to their moving target numbers to form a two-dimensional matrix.

[0135] The multi-target trajectory pattern probability refers to the two-dimensional matrix obtained after splicing. The row index of this matrix corresponds to different moving targets, and the column index corresponds to different preset reference trajectory patterns. Each element in the matrix represents the probability value of a certain moving target corresponding to a certain reference trajectory pattern.

[0136] Specifically, the trajectory pattern inference model outputs a probability distribution vector for each moving target, and these vectors independently reflect the trajectory tendency of each moving target. Since the reinforcement learning decision network needs to simultaneously perceive the trajectory inference state of all moving targets in order to evaluate the amount of information acquired at the global level and plan subsequent paths accordingly, it is necessary to combine the probability distribution vectors of each moving target into a unified data structure. After concatenating them along the moving target dimension, the decision network only needs one input to obtain the complete probability information of all moving targets, avoiding the inefficient operation of making decisions for each moving target separately.

[0137] S4. Construct the decision state based on the multi-target trajectory pattern probability, the sequence of traversed edges that the autonomous mobile single station has traversed since its starting position, and the current node position.

[0138] The traversed edge sequence refers to the sequence of all edges that the autonomous mobile station has traversed in chronological order since its starting position. In one implementation, if the autonomous mobile station starts from the starting node and traverses edge AB, edge BE, and edge EF in sequence, then the traversed edge sequence is recorded as [AB, BE, EF].

[0139] The current node position refers to the node in the topology graph where the autonomous mobile station is located at the current moment. After the autonomous mobile station completes the traversal of an edge, the current node position is updated to the endpoint node of that edge.

[0140] The decision state refers to the data set used to describe the current state of an autonomous mobile station at the moment of decision-making. The decision state consists of four parts: multi-objective trajectory pattern probability, traversed edge sequence, current node position, and path progress. Path progress is the ratio of the number of traversed edges to the preset maximum number of traversed edges, expressed as a percentage. It means that, among them The number of edges already traversed. This is the preset maximum number of edges to traverse.

[0141] Specifically, the multi-target trajectory pattern probability reflects the inferred trajectory tendency of each moving target, which is the basis for determining whether sufficient trajectory information has been acquired to terminate tracking. The traversed edge sequence records the historical walking route of the autonomous mobile station, and this sequence is used to prevent the policy network from repeatedly selecting edges that have already been traversed. The current node position limits the spatial range of edges that can be selected for the next step. The path progress reflects the time consumption of the current tracking process; the more time consumed, the lower the marginal benefit of continuing to increase the number of traversed edges. Combining the above four parts of information into the decision state ensures that the input of the reinforcement learning decision network simultaneously includes information from four dimensions: trajectory inference results, historical paths, spatial location, and time progress, providing a comprehensive basis for action selection.

[0142] In some implementations, S4, based on the multi-target trajectory pattern probability, the sequence of traversed edges sequentially since the starting position of the autonomous mobile station, and the current node position, constructs a decision state, including:

[0143] S41. Determine historical path information based on the traversed edge sequence;

[0144] The traversed edge sequence refers to the sequence of all edges traversed by the autonomous mobile station in chronological order since its starting position. This sequence records the historical movement path of the autonomous mobile station. In one implementation, if the autonomous mobile station starts from the starting node and traverses the edges sequentially... ,side ,side The edge sequence that has been traversed is recorded as follows: .

[0145] Historical path information refers to the data extracted from the traversed edge sequence to describe the historical walking route of an autonomous mobile station. This information corresponds to the traversed edge sequence and includes the identifier of each traversed edge and its traversal order.

[0146] Specifically, the traversed edge sequence contains a complete record of the autonomous mobile station's movement trajectory from its starting position. When selecting the next action, the reinforcement learning decision network needs to know which edges have been traversed to avoid redundant observations caused by repeatedly selecting the same edge in subsequent path planning. Simultaneously, the path information contained in the traversed edge sequence reflects the process of the autonomous mobile station gradually exploring the indoor space, a process that corresponds to the update history of the trajectory pattern probabilities of each moving target. Inputting the traversed edge sequence as historical path information into the decision state enables the reinforcement learning decision network to select actions based on the complete path history, avoiding the repeated selection of traversed edges.

[0147] S42. Determine the current location information based on the current node's location;

[0148] Here, the current node position refers to the node identifier of the autonomous mobile station in the topology graph at the current moment. When the autonomous mobile station completes the traversal of an edge, its spatial position moves from the starting node of that edge to the ending node, and the current node position is updated to the ending node accordingly.

[0149] Current location information refers to the data extracted from the current node location that describes the current spatial location of an autonomous mobile station. This information corresponds to the current node location and is represented by the node number in the topology graph.

[0150] Specifically, the topology graph discretizes the indoor space, and the autonomous mobile station is always located at a certain node at any given time. The current node position directly limits the spatial range of edges that the autonomous mobile station can traverse in its next step. According to the definition of the topology graph, the autonomous mobile station can only move from the current node to adjacent nodes along edges connected to that node; it cannot directly jump to non-adjacent edges. Inputting the current node position as the current location information into the decision state allows the reinforcement learning decision network to explicitly know its current spatial location, thus enabling it to select actions only from edges adjacent to the current node.

[0151] S43. Determine the path progress information based on the ratio of the number of traversed edges to the preset maximum number of traversed edges;

[0152] Here, the number of traversed edges refers to the total number of edges that an autonomous mobile station has traversed since its starting position, denoted as . .

[0153] The preset maximum number of edges traversed refers to the maximum number of edges that can be traversed by an autonomously moving single station in a single tracking round, denoted as . In one implementation, Pre-set according to the size of the indoor space, the scale of the topology map, and the real-time requirements.

[0154] Path progress information refers to the ratio of the number of edges traversed to the preset maximum number of edges traversed, expressed as: .

[0155] Specifically, as the autonomous mobile station continuously traverses new edges, the number of edges already traversed... Gradually increase, path progress The more edges have been traversed, the higher the time cost of adding more edges. The marginal benefit of obtaining effective trajectory information from newly added edges typically decreases as the number of traversed edges increases. Path progress information reflects the time consumption of the current tracking process. When the path progress is close to 1, it indicates that the autonomous mobile station has approached the preset maximum number of traversed edges, and terminating tracking should be prioritized to avoid exceeding the limit. The path progress information... Inputting the decision state enables the reinforcement learning decision network to make a decision to terminate or continue based on the time resources already consumed when balancing tracking accuracy and path length.

[0156] S44. Combine the multi-target trajectory pattern probability, historical path information, current location information, and path progress information as the decision state.

[0157] The decision state refers to the complete dataset describing the current state of an autonomous mobile station at the moment of decision-making; this dataset serves as the input to the reinforcement learning decision network. The decision state is determined by the probabilities of multi-objective trajectory patterns. Historical path information Current location information and path progress information It consists of four parts, represented as:

[0158] in Indicates as of the date The edge sequence that has been traversed in each decision step This indicates the current node position of an autonomously moving single station. This represents the multi-target trajectory pattern probability matrix obtained by concatenating the probability vectors of each moving target trajectory pattern. This indicates the number of edges that have been traversed. This indicates the maximum number of edges that can be traversed.

[0159] Specifically, multi-target trajectory pattern probability This reflects the current inferred trajectory tendency distribution of each moving target, and this information is the basis for determining whether the currently acquired radio signal observations are sufficient to support trajectory estimation. Historical path information. The complete walking route of the autonomous mobile single station is recorded. This information is used to prevent the repeated selection of already traversed edges in subsequent path planning. Current location information. This limits the spatial range of edges that can be traversed in the next step. Path progress information. This reflects the time consumed in the current tracking process. The four pieces of information described above, from different dimensions, describe the current decision-making situation of the autonomous mobile station, corresponding to the four decision elements: "who to track, where they came from, where they are, and how much time is left." These four pieces of information are combined into a unified decision state. Input reinforcement learning decision network, enabling the network to simultaneously obtain information from four dimensions—trajectory inference results, historical paths, spatial location, and temporal progress—from a single input, thereby evaluating the expected benefits of each candidate action and selecting the action.

[0160] S5. Based on the decision state and action mask, the action instruction at the current moment is obtained using a pre-trained reinforcement learning decision network. The action instruction is to traverse the newly determined current edge or terminate the tracking process. The action mask is used to set the selection probability of the non-executable action to zero. The reinforcement learning decision network is a policy network pre-trained using a reinforcement learning algorithm based on state transition samples generated by the interaction between the autonomous mobile station and the indoor environment. The state transition samples include the decision state, the action to be executed, the immediate reward, the next decision state, and the end of the round flag.

[0161] The action mask is a binary vector with the same dimension as the output action space of the reinforcement learning decision network. Each element in the action space corresponds to a candidate action. The action mask is generated based on the connection relationship between the current node position and each edge in the topology graph. The action mask is determined as follows: obtain all adjacent edges of the current node position in the topology graph; mark each adjacent edge as an executable action and mark non-adjacent edges as non-executable actions to generate the action mask.

[0162] The reinforcement learning decision network is a policy network pre-trained using a reinforcement learning algorithm. The network takes the decision state and an action mask as input and outputs the probability distribution of each candidate action being selected. The action mask is used to set the selection probability of inexecutable actions to zero, ensuring that the reinforcement learning decision network selects only from the adjacent edges reachable from the current node position and the termination tracking action.

[0163] State transition samples refer to data records generated during the interaction between an autonomous mobile station and its indoor environment during reinforcement learning training. Each interaction forms a state transition sample, which includes the decision state, the action to be performed, the immediate reward, the next decision state, and the end-of-round flag. The reinforcement learning decision network is trained based on a large number of state transition samples, learning strategies for selecting actions under different decision states.

[0164] Specifically, after obtaining the current decision state, it is necessary to determine the range of executable actions. Due to the constraints of the topology graph, an autonomous mobile station can only move from its current node to the next node along a certain adjacent edge, and cannot directly jump to a non-adjacent edge. If no constraints are imposed on the action space, the reinforcement learning decision network may output selection probabilities on unreachable edges, causing the output action instructions to be inexecutable. Action masking forces the decision network to choose only between physically reachable adjacent edges and termination actions by setting the selection probability of unreachable edges to zero. The reinforcement learning decision network evaluates the expected benefits of each candidate action based on the current decision state and selects an action: if an adjacent edge is selected, that edge becomes the newly determined current edge, the autonomous mobile station performs traversal operations and enters the next round of acquisition and inference; if the termination tracking process is selected, signal acquisition stops, and the trajectory estimation results of each moving target are output.

[0165] Among them, such as Figure 4 As shown, in some implementations, the reinforcement learning decision network is trained in the following manner:

[0166] S61. Based on the trajectory pattern combination of each training moving target, the current training position and the current training edge of the autonomous mobile station, control the autonomous mobile station to traverse the current training edge from the current training position and collect the training received signal strength sequence of each training moving target on the current training edge.

[0167] Among them, the training moving target refers to the moving target set up in the offline training phase to collect training data. Each training moving target moves in the indoor space according to a preset trajectory pattern combination, and the reference trajectory pattern number corresponding to its actual walking trajectory is known.

[0168] Trajectory pattern combination refers to the set of reference trajectory patterns assigned to each training moving target in a training round. Different training moving targets can be assigned the same or different reference trajectory patterns.

[0169] The current training position refers to the node position in the topology graph where the autonomously moving single station is located at the beginning of the training round.

[0170] The current training edge refers to the first edge that the autonomous moving station will traverse at the beginning of the training round, and this edge is connected to the current training position.

[0171] The training received signal strength sequence refers to the time sequence formed by the received signal strength indication values ​​of each training mobile target collected by an autonomous mobile station at a preset sampling interval when traversing the current training edge.

[0172] Specifically, during the training phase of the reinforcement learning decision network, a large number of state transition samples covering different multi-target tracking scenarios need to be generated so that the decision network can learn strategies for selecting actions under different decision states. The combination of trajectory patterns of each training moving target determines the distribution of motion states of multiple targets in this training round. Different trajectory pattern combinations will produce different spatiotemporal distribution characteristics of wireless signals, thus affecting the received signal strength sequence collected by the autonomous mobile station when traversing each edge. The current training position and current training edge of the autonomous mobile station determine the initial spatial state of the training round. The autonomous mobile station is controlled to traverse the current training edge from the current training position, collecting the training received signal strength sequence of each training moving target on the current training edge. This sequence serves as the observation data at the current moment, providing input for subsequent trajectory pattern inference.

[0173] S62. Accumulate the historical training received signal strength sequence for each training moving target. In the trajectory pattern inference model with input freezing parameters, the probability distribution of each preset reference trajectory pattern corresponding to each training moving target and the probability of the stitched multi-target trajectory pattern are obtained.

[0174] Among them, frozen parameters refer to the fact that the model parameters of the trajectory pattern inference model (shown as MSS-MPT model in the figure) remain unchanged during the training process of the reinforcement learning decision network and do not participate in the parameter update of this stage.

[0175] Multi-target trajectory mode probability It is the probability distribution of all training moving targets. The resulting two-dimensional matrix is ​​obtained by concatenating the data along the dimensions of the moving target. The row indices of this matrix correspond to different training moving targets, and the column indices correspond to different preset reference trajectory patterns.

[0176] Specifically, during the training of a reinforcement learning decision network, the trajectory pattern inference model, as part of the environmental state perception module, is responsible for converting the original wireless signal observations into trajectory pattern probability distributions for each moving target. If the parameters of the trajectory pattern inference model are updated simultaneously with the training of the decision network, the state space distribution faced by the decision network will continuously change with the model parameters, leading to instability in the training process. Freezing the parameters of the trajectory pattern inference model ensures that the same cumulative received signal strength sequence consistently produces the same trajectory pattern probability distribution output during the decision network training process, providing a stable state space for the decision network. The historical cumulative received signal strength sequences of each training moving target are input into the trajectory pattern inference model with frozen parameters. The model outputs the probability distribution of each training moving target corresponding to each preset reference trajectory pattern. All probability distributions are concatenated along the moving target dimension to form a multi-target trajectory pattern probability, which serves as a component for subsequently constructing the decision state.

[0177] S63. Construct the current decision state based on the multi-target trajectory pattern probability, the sequence of traversed edges since the starting position of the autonomous mobile single station since training, and the current training node position.

[0178] The training starting position refers to the node position of the autonomously moving single station in the topology graph at the beginning of the current training round.

[0179] The traversed edge sequence refers to the sequence of all edges that the autonomously moving station has traversed in the current training round since its starting position, in chronological order of traversal time.

[0180] The current training node position refers to the node in the topology graph where the autonomously moving single station is located at the current moment.

[0181] The current decision state refers to the dataset describing the current state of an autonomous mobile station at the moment of decision-making. This dataset consists of four parts: multi-objective trajectory pattern probability, traversed edge sequence, current training node position, and path progress. Path progress is the ratio of the number of traversed edges to the preset maximum number of traversed edges, expressed as... ,in The number of edges already traversed. This is the preset maximum number of edges to traverse.

[0182] Specifically, the decision state is the input to the reinforcement learning decision network, and its construction method needs to be consistent between the training and online execution phases to ensure that the trained decision network can work correctly during online execution. During the training phase, the traversed edge sequence records the complete path of the autonomous mobile station from its starting position in the current training round; the current training node position reflects the current spatial location of the autonomous mobile station; the multi-object trajectory pattern probability is output by the trajectory pattern inference model with frozen parameters; and the path progress reflects the edge traversal resources consumed in the current training round. Combining these four pieces of information forms the current decision state, which fully describes the decision situation faced by the autonomous mobile station at the current moment.

[0183] S64. Generate an action mask based on the connection relationship between the current training node position and each edge in the topology graph. Input the current decision state and the action mask into the Actor network to be trained. The Actor network outputs the action instruction at the current moment. The action instruction is to select an edge adjacent to the current training node position to be traversed or to terminate the tracking process.

[0184] The Actor network is the policy network component in a reinforcement learning decision network. It takes the decision state and action mask as input and the probability distribution of each candidate action being selected as output. The network parameters are denoted as... .

[0185] The action instruction is the action that should be executed at the current moment, as output by the Actor network. This instruction can be either to select an edge adjacent to the current training node position to be traversed or to terminate the tracking process.

[0186] An edge to be traversed refers to an edge selected by the Actor network from the candidate edges adjacent to the current training node.

[0187] The termination of tracking process refers to the action of the Actor network to stop traversing new edges when it determines that the currently acquired wireless signal observation information is sufficient to support the trajectory estimation of each moving target.

[0188] Specifically, the current training node position limits the spatial range of edges that the autonomous mobile station can traverse in the next step. This is because, according to the definition of the topology graph, the autonomous mobile station can only move from the current node to an adjacent node along the edges connected to that node. Without constraints on the action space, the Actor network might output a high selection probability on unreachable edges, causing the output action command to be inexecutable. The action mask is generated based on the connection relationship between the current training node position and each edge in the topology graph. Action bits corresponding to edges adjacent to the current training node position are marked as executable, while action bits corresponding to non-adjacent edges are marked as inexecutable, and the selection probability of inexecutable actions is set to zero. The current decision state and the action mask are input into the Actor network. Under the mask constraints, the network outputs a probability distribution of the selected executable actions, and the action command at the current time is sampled based on this probability distribution.

[0189] S65. Input the current decision state into the Critic network to be trained, and the Critic network outputs the value estimate of the current decision state.

[0190] The Critic network is the value network component in a reinforcement learning decision network. This network takes the decision state as input and outputs a value estimate of the current decision state. The network parameters are denoted as follows: .

[0191] Value estimation refers to the Critic network's prediction of the expected cumulative return that can be obtained from the current decision state by following the current strategy. This value is used to evaluate the quality of the current decision state.

[0192] Specifically, in reinforcement learning, the Actor network is responsible for outputting the action selection policy, while the Critic network is responsible for evaluating the value of the current decision state. Together, they form the actor-critic framework. The value estimate output by the Critic network provides a benchmark for subsequent calculation of the advantage function, which represents the superiority or inferiority of performing a particular action relative to the average level. The current decision state is input into the Critic network, and the network calculates and outputs the value estimate of the current decision state through forward propagation. This estimate is used in subsequent steps to calculate the advantage estimate and value loss.

[0193] S66. Execute the action command to obtain an immediate reward and the next decision state. The immediate reward is determined according to the type of action command, and the next decision state is the decision state reconstructed after executing the action command.

[0194] The immediate reward refers to the scalar feedback signal obtained from the indoor environment after the autonomous mobile station executes an action command. This feedback signal is determined using different calculation methods depending on the type of action command. When the action command is to traverse a single edge to be traversed, the immediate reward is determined based on the path length feedback, which is the ratio of the number of traversed edges to the preset maximum number of traversed edges. The reward decreases as the tracking error increases. When the action command is to terminate the tracking process, the immediate reward is determined based on the average trajectory tracking error of each training moving target; the smaller the average trajectory tracking error, the higher the immediate reward.

[0195] The next decision state refers to the decision state reconstructed based on new observation data and the probability of multi-target trajectory patterns after the autonomous mobile station executes the action command.

[0196] Specifically, reinforcement learning algorithms guide the decision network to learn the optimal policy through reward signals. The path length feedback setting encourages the autonomous mobile station to terminate tracking after traversing a significant number of edges, avoiding unnecessary edge traversal. The reward based on the average trajectory tracking error upon termination ensures the decision network terminates learning promptly when trajectory estimation accuracy is sufficiently high. After executing an action command, if the action is to traverse the edge to be traversed, the autonomous mobile station moves along that edge to an adjacent node, collects a new training received signal strength sequence, and updates the historical cumulative sequence and trajectory pattern probability of each training moving target. Based on the updated information, it reconstructs the decision state as the next decision state. If the action is to terminate the tracking process, the current training round ends, and the next decision state is empty. The immediate reward and the next decision state together constitute a complete feedback mechanism from the environment to the decision network's action execution.

[0197] S67. Store the current decision state, action instructions, immediate reward, next decision state, and value estimate as a state transition sample in the experience cache.

[0198] Among them, the state transition sample refers to the complete data record generated by an autonomous mobile station during a single decision-making interaction. This sample includes the current decision state. Execution of actions Instant rewards Next decision status Value estimation of Critic network output .

[0199] The experience cache is a buffer used to store multiple state transition samples. When the cache reaches a preset length, it triggers the parameter update of the decision network.

[0200] Training reinforcement learning decision networks relies on a large amount of decision interaction data. Data generated from a single interaction only contains local information at the current decision moment and cannot be directly used for effective network parameter updates. Accumulating state transition samples from multiple interactions and storing them in an empirical cache allows for parameter updates based on batch samples once a sufficient number of samples have accumulated. This reduces the variance of parameter updates and improves training stability. The state transition samples contain complete transition information from the current state to the next state. The next decision state and immediate reward originate from environmental feedback, while the value estimate comes from the forward computation of the Critic network. This data is used in subsequent steps to calculate advantage estimation and value loss.

[0201] S68. When the experience cache reaches the preset length or the current training round ends, calculate the reward and advantage estimate based on the instantaneous reward and value estimate of each state transition sample stored in the experience cache.

[0202] Here, reward refers to the cumulative discount bonus obtained from the current decision state, following the current strategy until the end of the round, denoted as .

[0203] Advantage estimation refers to the additional cumulative reward obtained by performing a certain action compared to the average level, denoted as . The advantage estimate is calculated based on the immediate reward in the state transition sample and the value estimate output by the Critic network. Specifically, the advantage value of the action is obtained by subtracting the value estimate of the current state from the sum of the immediate reward and the value estimate of the next state. In one implementation, a generalized advantage estimation method is used to calculate the advantage estimate. This method balances the bias and variance by introducing a discount factor and an advantage decay factor.

[0204] Specifically, the state transition samples stored in the experience cache contain interaction data from multiple consecutive decision-making moments, forming a complete decision trajectory. The immediate rewards at each moment in the decision trajectory need to be discounted and accumulated to obtain the reward value at each moment; the discount factor ensures that recent rewards contribute more to the reward than long-term rewards. The advantage estimate reflects the degree of advantage of performing a specific action in a given state compared to performing an average action; a positive value indicates that the action is better than the average, and a negative value indicates that the action is worse than the average. When the experience cache reaches its preset length or the current training round ends, the reward and advantage estimates are calculated based on the immediate rewards and value estimates of each state transition sample stored in the cache, providing optimization objectives for the parameter updates of the Actor and Critic networks.

[0205] S69. The parameters of the Actor network are updated based on the reward and advantage estimates using the proximal policy optimization algorithm. The value loss is calculated using the difference between the value estimate and the reward. The parameters of the Critic network are updated based on the value loss to obtain the trained reinforcement learning decision network.

[0206] Among them, the proximal policy optimization algorithm is a reinforcement learning policy optimization method that improves training stability by limiting the magnitude of policy updates.

[0207] Value loss refers to the difference between the value estimate output by the Critic network and the return. This difference is calculated using mean squared error or a similar loss function and is used to measure the accuracy of the Critic network's prediction of state value.

[0208] Specifically, the Actor network outputs an action selection policy, with the optimization objective of maximizing the expected cumulative reward; the Critic network outputs a state value estimate, with the optimization objective of accurately predicting the expected cumulative reward of a state. The two networks have different optimization objectives and require different update methods. The Actor network's update is based on reward and advantage estimation, using the advantage estimate as the weight of the policy gradient, increasing the probability of selecting high-advantage actions and decreasing the probability of selecting low-advantage actions. The proximal policy optimization algorithm limits the magnitude of policy change in a single update by truncating the probability ratio of the new and old policies, avoiding policy performance collapse due to excessively large update steps. The Critic network's update is based on the difference between the value estimate and the reward, minimizing the value loss to gradually approximate the true expected cumulative reward. In one implementation, the Actor and Critic networks share some network parameters but each has an independent output head. Training rounds are repeatedly executed, each round generating diverse training scenarios by changing the trajectory pattern combination of the training moving target and the starting position of the autonomous moving station. The Actor and Critic networks continuously update in each round until the network converges, resulting in a trained reinforcement learning decision network.

[0209] In some implementations, the immediate reward is determined in the following way:

[0210] When the action instruction is to select an edge to be traversed that is adjacent to the current training node position, the immediate reward decreases as the ratio of the number of traversed edges to the preset maximum number of traversed edges increases.

[0211] When the action command is to select to terminate the tracking process, the immediate reward increases as the average trajectory tracking error of each training moving target decreases.

[0212] Specifically, when the action command is to select an edge to be traversed, the autonomous mobile station will move along that edge from the current node to the adjacent node. This action consumes additional movement time and signal acquisition time, increasing the path length of the entire tracking process. As the number of traversed edges increases... Increase, path progress As the path progresses, the time resources consumed increase. If new edges are continued to be added when the path progress is already significant, the improvement in trajectory estimation accuracy from these new edges is limited, while the time cost remains. Therefore, the immediate reward for continuing edge traversal is set to random. Negative values ​​that increase or decrease:

[0213] ;

[0214] in This represents the path length weighting coefficient, used to adjust the influence of path length in the reward function. Number of edges traversed. The larger the path progress The closer to 1, the greater the instant reward. The larger the absolute value of , the heavier the penalty for the action. This setting causes the decision network to tend to explore new edges to obtain trajectory information when the path progress is low, and to reduce unnecessary edge traversal when the path progress is high.

[0215] When the action command is to terminate the tracking process, the autonomous moving station stops traversing new edges, and the current training round ends. At this point, the average trajectory tracking error between the estimated trajectory and the actual reference trajectory of each training moving target is used. Determine the termination reward. A smaller error value indicates a more accurate estimation of the trajectory of each moving target based on the currently collected wireless signal observation information, making the timing of terminating tracking more appropriate; a larger error value indicates that the acquired observation information is insufficient to support accurate trajectory estimation of each moving target, and the tracking process should continue to traverse the edges to collect more signals. The immediate reward for terminating tracking is set to random. The value that decreases and increases:

[0216] ;

[0217] in This represents the normalization parameter used to unify the dimensions of the error. Average trajectory tracking error. The larger the reward, the more immediate the bonus. The larger the absolute value, the heavier the penalty for terminating tracking; the average trajectory tracking error The smaller, the instant reward The closer the value is to 0, the lighter the penalty for terminating tracking, which is equivalent to a higher reward. This setting allows the decision network to weigh the options of continuing edge traversal versus terminating tracking during the learning process: if tracking is terminated in the current decision state, the termination reward is determined by the current trajectory estimation accuracy; higher accuracy makes termination more advantageous. If edge traversal continues, the immediate reward is a path length penalty, while there is an opportunity to obtain a higher termination reward in the next step. By learning the reward distributions of these two actions, the decision network selects the action sequence that maximizes the cumulative reward, thus achieving a balance between trajectory tracking accuracy and path length.

[0218] In some implementations, after terminating the tracking process, the method further includes:

[0219] For each moving target, the multiple reference trajectories with the highest predicted probability are selected as candidate reference trajectories;

[0220] Normalized weights are calculated based on the predicted probabilities of each candidate reference trajectory;

[0221] The coordinate sequences of each candidate reference trajectory are weighted and fused according to normalized weights to obtain the estimated trajectory of each moving target.

[0222] Here, candidate reference trajectories refer to a subset of reference trajectories selected from a preset set of reference trajectory patterns for use in trajectory estimation. In one implementation, the total number of preset reference trajectory patterns... The probability distribution of each moving target corresponding to each preset reference trajectory pattern is 324, output by the trajectory pattern inference model. The model with the highest predicted probability is selected. One reference trajectory is used as a candidate reference trajectory. In one implementation, the preset positive integer is used. The probability equals 3. A higher prediction probability indicates a higher degree of match between the reference trajectory and the actual trajectory of the moving target. For a given moving target, let the probability distribution of its corresponding preset reference trajectory patterns be... ,in Indicates the first The probability values ​​corresponding to each reference trajectory pattern. Sort by probability value from largest to smallest, and select the top... Each probability value corresponds to a reference trajectory pattern number and its coordinate sequence as a candidate reference trajectory.

[0223] Normalized weights refer to the weights obtained by recalibrating the predicted probabilities of each candidate reference trajectory, such that the sum of the weights of all candidate reference trajectories equals 1. For the th... There are 10 candidate reference trajectories, and their predicted probabilities are denoted as . , The sum of the predicted probabilities of the candidate reference trajectories is . No. Normalized weights of candidate reference trajectories The calculation is as follows:

[0224] ;

[0225] The sum of the predicted probabilities of all candidate reference trajectories is less than 1 because none were selected. The probability values ​​of the reference trajectories were not included in the summation. After normalization, The sum of the weights of the candidate reference trajectories is equal to 1. The relationship between the weights is consistent with the relationship between the original probabilities. That is, the candidate reference trajectory with the higher the predicted probability receives the greater the normalized weight.

[0226] A coordinate sequence refers to an ordered set of spatial coordinates corresponding to a preset reference trajectory pattern. Each reference trajectory consists of several nodes arranged in the walking order, and each node corresponds to a two-dimensional or three-dimensional coordinate in the indoor space. In one implementation, the original coordinate sequence of the m-th reference trajectory is represented as follows: , in, This represents the number of position points contained in the original coordinate sequence of the m-th reference trajectory. Since the number of position points contained in the original coordinate sequences of different reference trajectories may be different, the original coordinate sequences of each candidate reference trajectory are resampled to contain L position points, so that position points with the same sequence number in different candidate reference trajectories correspond to the same normalized trajectory progress.

[0227] The coordinates of the estimated trajectory are:

[0228] ;

[0229] in Indicates the first The candidate reference trajectory in the 1st The coordinates of each location point The normalized weights for this candidate reference trajectory.

[0230] like Figure 5 As shown, the online tracking phase is executed in the following order: Step S1: Initialize the autonomous mobile single-station position and initial edge, traverse the initial edge and synchronously collect the RSS sequences of N moving targets, and form the FMTP of each target after preprocessing. Step S2: Add the current FMTP to the cumulative fingerprint sequence of each target, input it into the trained MSS-MPT model, and obtain the trajectory pattern probability vector and multi-target probability representation Q(t) of each target.

[0231] Step S3: Combine the historical path P(t), current position v(t), multi-target probability representation Q(t), and path progress t / T into a state s(t), and generate an action mask based on the current position and topological connections. Input the state and action mask into the trained PPO policy network to obtain the next edge to be traversed or the termination action. Step S4: If the output edge selects an action, autonomously move the single station to traverse the selected edge, collect a new RSS sequence, and return to step S2; if the output terminates the action or the maximum path length T has been reached, stop wireless signal collection.

[0232] Step S5: For each moving target, select the K reference trajectories with the highest probabilities from the final trajectory pattern probabilities, perform weighted fusion according to the normalized probabilities, and output the estimated trajectories of N moving targets respectively.

[0233] Example 2

[0234] This invention provides an indoor multi-target active trajectory tracking device based on a mobile single station. The indoor space is pre-divided into a topology graph consisting of several nodes and edges connecting adjacent nodes. Nodes are locations reachable by the autonomous mobile single station, and edges are basic path units connecting adjacent nodes that can be traversed by the autonomous mobile single station. The method includes:

[0235] The acquisition module is used to acquire the wireless received signal strength sequence of each mobile target in the indoor space based on the autonomous mobile station's traversal of the current edge.

[0236] The probability distribution module is used to input the historical accumulated received signal strength sequence of each moving target into the preset trajectory pattern inference model to obtain the probability distribution of each moving target for each preset reference trajectory pattern. The trajectory pattern inference model is trained based on the received signal strength sequence collected by each moving target on each edge and its corresponding reference trajectory pattern label.

[0237] The stitching module is used to stitch together the probability distributions of all moving targets according to the moving target dimension to obtain the probability of multi-target trajectory patterns.

[0238] The construction module is used to construct the decision state based on the multi-target trajectory pattern probability, the sequence of traversed edges that the autonomous mobile station has traversed since its starting position, and the current node position.

[0239] The decision module is used to obtain the action instruction at the current moment using a pre-trained reinforcement learning decision network based on the decision state and action mask. The action instruction is to traverse the newly determined current edge or terminate the tracking process. The action mask is used to set the selection probability of the non-executable action to zero. The reinforcement learning decision network is a policy network pre-trained using a reinforcement learning algorithm based on state transition samples generated by the interaction between the autonomous mobile station and the indoor environment. The state transition samples include the decision state, the action to be executed, the immediate reward, the next decision state, and the end of the round flag.

[0240] It should be noted that each module and unit in the indoor multi-target active trajectory tracking device based on a mobile single station in this embodiment corresponds one-to-one with each step in the indoor multi-target active trajectory tracking method based on a mobile single station in the aforementioned embodiment. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned indoor multi-target active trajectory tracking method based on a mobile single station, and will not be repeated here.

[0241] Example 3

[0242] This embodiment provides an electronic device, including at least one processor and a memory. Optionally, the device further includes a communication component. The processor, memory, and communication component are connected via a bus.

[0243] In a specific implementation, at least one processor executes computer execution instructions stored in memory, causing at least one processor to perform the above-described method.

[0244] The specific implementation process of the processor can be found in the above method embodiments, and its implementation principle and technical effect are similar, so it will not be repeated here.

[0245] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0246] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0247] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0248] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0249] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0250] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0251] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0252] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0253] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0254] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0255] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0256] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0257] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for active trajectory tracking of multiple targets indoors based on a mobile single station, characterized in that, The indoor space is pre-divided into a topological graph consisting of several nodes and edges connecting adjacent nodes. The nodes are locations reachable by an autonomous mobile station, and the edges are basic path units connecting adjacent nodes that can be traversed by the autonomous mobile station. The method includes: Based on the traversal of the current edge by the autonomous mobile station, the wireless received signal strength sequence of each mobile target in the indoor space is collected; The historical accumulated received signal strength sequence of each moving target is input into a preset trajectory pattern inference model to obtain the probability distribution of each preset reference trajectory pattern corresponding to each moving target. The trajectory pattern inference model is trained based on the received signal strength sequence of each moving target collected on each edge and its corresponding reference trajectory pattern label. By concatenating the probability distributions of all moving targets along the dimensions of the moving targets, the probability of multi-target trajectory patterns is obtained. The decision state is constructed based on the multi-target trajectory pattern probability, the sequence of traversed edges that the autonomous mobile station has traversed sequentially since its starting position, and the current node position. Based on the decision state and action mask, a pre-trained reinforcement learning decision network is used to obtain the action instruction at the current moment. The action instruction is to traverse the newly determined current edge or terminate the tracking process. The action mask is used to set the selection probability of the non-executable action to zero. The reinforcement learning decision network is a policy network pre-trained using a reinforcement learning algorithm based on the state transition samples generated by the interaction between the autonomous mobile station and the indoor environment. The state transition samples include the decision state, the action to be executed, the immediate reward, the next decision state, and the round end flag.

2. The method according to claim 1, characterized in that, The step of inputting the historically accumulated received signal strength sequence of each moving target into a preset trajectory pattern inference model to obtain the probability distribution of each preset reference trajectory pattern corresponding to each moving target includes: The historical accumulated received signal strength sequence of each moving target is input into the channel interaction module to obtain channel association features. The channel interaction module is used to exchange information between channels on the input features of received signal strength, sampling position and edge order. The channel association features are input into multiple multi-scale dilated convolution branches with different kernel sizes and dilation rates to extract and fuse local received signal intensity variation features in different time ranges, thereby obtaining multi-scale local features containing multi-time scale information. The multi-scale local features are input into the self-attention module to obtain the global correlation features; The global association features are input into the long short-term memory network in the order of edge traversal to obtain the sequence features with cross-edge temporal dependencies. The sequence features are input into the attention pooling module and the max pooling module to obtain the attention pooling result and the max pooling result, respectively. The attention pooling result and the max pooling result are concatenated and then input into the linear classification module to obtain the probability distribution of each preset reference trajectory pattern corresponding to each moving target.

3. The method according to claim 1, characterized in that, The trajectory pattern inference model is trained in the following manner: The autonomous mobile station is controlled to traverse different edges and collect the received signal strength sequence of each training mobile target on each edge. The received signal strength sequences of each training moving target on each edge are combined in traversal order to form a cumulative fingerprint sequence; The accumulated fingerprint sequence and the corresponding reference trajectory pattern label are used as a training sample. The reference trajectory pattern label is the number of the preset reference trajectory pattern corresponding to the actual trajectory of the training moving target. Zero-padding is performed on accumulated fingerprint sequences that are shorter than the set length to generate an effective length mask to distinguish between valid data positions and zero-padding positions; The accumulated fingerprint sequence and the effective length mask are used together as input to the model to be trained to obtain the predicted probability; The difference between the predicted probability and the reference trajectory pattern label is calculated using cross-entropy loss, and the parameters of the model to be trained are updated through backpropagation to obtain the trajectory pattern inference model.

4. The method according to claim 1, characterized in that, Based on the multi-target trajectory pattern probability, the sequence of traversed edges sequentially since the starting position of the autonomous mobile station, and the current node position, a decision state is constructed, including: Historical path information is determined based on the traversed edge sequence; Determine the current location information based on the current node position; The path progress information is determined based on the ratio of the number of edges already traversed to the preset maximum number of edges traversed. The multi-target trajectory pattern probability, the historical path information, the current location information, and the path progress information are combined to form the decision state.

5. The method according to claim 1, characterized in that, The reinforcement learning decision network is trained in the following manner: Based on the trajectory pattern combination of each training mobile target, the current training position and current training edge of the autonomous mobile station, the autonomous mobile station is controlled to traverse the current training edge from the current training position and collect the training received signal strength sequence of each training mobile target on the current training edge. By inputting the historical accumulated training received signal strength sequence of each training moving target into the trajectory pattern inference model of the frozen parameters, the probability distribution of each preset reference trajectory pattern corresponding to each training moving target and the probability of the spliced ​​multi-target trajectory pattern are obtained. The current decision state is constructed based on the multi-target trajectory pattern probability, the sequence of traversed edges that the autonomous mobile single station has sequentially traversed since the training start position, and the current training node position. An action mask is generated based on the connection relationship between the current training node position and each edge in the topology graph. The current decision state and the action mask are input into the Actor network to be trained. The Actor network outputs the action instruction at the current moment. The action instruction is to select an edge adjacent to the current training node position to be traversed or to terminate the tracking process. The current decision state is input into the Critic network to be trained, and the Critic network outputs a value estimate of the current decision state. Execute the action instruction to obtain an immediate reward and a next decision state. The immediate reward is determined according to the type of the action instruction, and the next decision state is a decision state reconstructed after executing the action instruction. The current decision state, the action instruction, the immediate reward, the next decision state, and the value estimate are combined to form a state transition sample and stored in the experience cache. When the experience cache reaches a preset length or the current training round ends, the reward and advantage estimate are calculated based on the instantaneous reward and value estimate of each state transition sample stored in the experience cache. The parameters of the Actor network are updated using the proximal policy optimization algorithm based on the reward and the advantage estimate. The value loss is calculated using the difference between the value estimate and the reward. The parameters of the Critic network are updated based on the value loss to obtain the trained reinforcement learning decision network.

6. The method according to claim 5, characterized in that, The instant reward is determined in the following manner: When the action instruction is to select an edge to be traversed that is adjacent to the current training node position, the instant reward decreases as the ratio of the number of traversed edges to the preset maximum number of traversed edges increases. When the action command is to select to terminate the tracking process, the immediate reward increases as the average trajectory tracking error of each training moving target decreases.

7. The method according to claim 1, characterized in that, The action mask is determined in the following way: Obtain all adjacent edges of the current node position in the topology graph; Mark each adjacent edge as an executable action and the non-adjacent edge as an unexecutable action, and generate an action mask.

8. The method according to claim 1, characterized in that, After the termination of the tracking process, the method further includes: For each moving target, the multiple reference trajectories with the highest predicted probability are selected as candidate reference trajectories; Normalized weights are calculated based on the predicted probabilities of each candidate reference trajectory; The coordinate sequences of each candidate reference trajectory are weighted and fused according to the normalized weights to obtain the estimated trajectory of each moving target.

9. An indoor multi-target active trajectory tracking device based on a mobile single station, characterized in that, The indoor space is pre-divided into a topological graph consisting of several nodes and edges connecting adjacent nodes. The nodes are locations reachable by an autonomous mobile station, and the edges are basic path units connecting adjacent nodes that can be traversed by the autonomous mobile station. The method includes: The acquisition module is used to acquire the wireless received signal strength sequence of each mobile target in the indoor space based on the traversal of the current edge by the autonomous mobile station. The probability distribution module is used to input the historically accumulated received signal strength sequence of each moving target into a preset trajectory pattern inference model to obtain the probability distribution of each moving target corresponding to each preset reference trajectory pattern. The trajectory pattern inference model is trained based on the received signal strength sequence collected by each moving target on each edge and its corresponding reference trajectory pattern label. The stitching module is used to stitch together the probability distributions of all moving targets according to the moving target dimension to obtain the probability of multi-target trajectory patterns. The construction module is used to construct the decision state based on the multi-target trajectory pattern probability, the sequence of traversed edges that the autonomous mobile station has traversed sequentially since its starting position, and the current node position. The decision module is used to obtain the action instruction at the current moment using a pre-trained reinforcement learning decision network based on the decision state and action mask. The action instruction is to traverse the newly determined current edge or terminate the tracking process. The action mask is used to set the selection probability of the non-executable action to zero. The reinforcement learning decision network is a policy network pre-trained using a reinforcement learning algorithm based on the state transition samples generated by the interaction between the autonomous mobile station and the indoor environment. The state transition samples include the decision state, the action to be executed, the immediate reward, the next decision state, and the round end flag.

10. An electronic device, characterized in that, include: At least one processor, at least one memory, and computer program instructions stored in the memory, which, when executed by the processor, implement the method as described in any one of claims 1-8.

Images