Indoor trajectory completion method based on multi-scale semantic grid map and bidirectional time sequence collaborative modeling

Abstract

The application discloses an indoor trajectory completion method based on multi-scale semantic grid map and bidirectional time sequence collaborative modeling, which comprises the following steps: 1. obtaining an indoor trajectory and a scene map, cutting the trajectory into fixed-length samples, and performing mask processing on continuous to-be-completed time coordinates; 2. performing grid division on the scene map, fusing the moving statistical features, macroscopic scene semantics and fine-grained functional semantic features of the training trajectory, and constructing a bidirectional semantic grid map with passable constraints; 3. obtaining multi-scale grid map representations by using a multi-layer graph attention network, and splicing the coordinate time representations, mask state representations and time position representations to form mask-aware trajectory tokens; 4. extracting the time sequence context of the to-be-completed time by a bidirectional gated recurrent unit, and predicting the missing coordinates by a coordinate decoder; and 5. jointly training the model by using a coordinate mean square error, a coordinate absolute error and a first-order displacement consistency error. The application can improve the spatial rationality, continuity and prediction accuracy of trajectory completion.

Description

Technical Field

[0001] This invention belongs to the field of indoor positioning and trajectory data processing technology, specifically involving an indoor trajectory completion method based on multi-scale semantic mesh graphs and bidirectional temporal collaborative modeling. Background Technology

[0002] With the development of indoor positioning and motion sensing technologies, indoor trajectory data is widely used in scenarios such as passenger flow statistics, path analysis, behavior recognition, and safety early warning. However, due to factors such as wall obstruction, signal multipath, equipment sampling interruption, and data transmission anomalies, the actual collected indoor trajectories are prone to missing coordinates at consecutive moments, which in turn affects the analysis of information such as trajectory speed, direction, displacement, and dwell area.

[0003] Existing trajectory completion methods mainly include interpolation methods, filtering methods, and data-driven methods based on neural networks. Interpolation and filtering methods typically rely on the coordinates on both sides of the missing interval or pre-defined motion patterns, which can easily lead to significant deviations when the trajectory turns, pauses, or changes in speed. Data-driven methods can learn the temporal characteristics of the trajectory, but some methods do not adequately utilize spatial information such as indoor passageways, functional areas, and impassable areas, and it is difficult to simultaneously utilize observation information before and after consecutive missing intervals. In addition, existing methods do not adequately constrain the displacement changes between adjacent predicted points, which may result in jitter, reversals, or local discontinuities in the completed trajectory. Summary of the Invention

[0004] This invention addresses the shortcomings of existing technologies by proposing an indoor trajectory completion method based on multi-scale semantic mesh graphs and bidirectional temporal collaborative modeling. The aim is to improve the accuracy, spatial rationality, and motion continuity of completing continuously missing indoor trajectories, and to reduce the impact of trajectory loss on applications such as passenger flow statistics, path analysis, behavior recognition, and safety warnings. This will provide more complete and reliable trajectory data support for indoor personnel movement analysis and spatial operation management.

[0005] To achieve the above-mentioned objectives, the present invention adopts the following technical solution: The indoor trajectory completion method based on multi-scale semantic mesh graphs and bidirectional temporal collaborative modeling of this invention is characterized by the following steps: S1. Obtain indoor trajectory and indoor scene map with horizontal coordinate, vertical coordinate and relative time. Use a sliding window to divide the indoor trajectory into trajectory samples of fixed length and perform standardization to obtain a preprocessed trajectory sample set. S2. In any preprocessed trajectory sample, determine the continuous time to be completed, replace the horizontal and vertical coordinates corresponding to the continuous time to be completed with preset mask values, retain the relative time corresponding to the continuous time to be completed, and generate a binary mask label to represent whether the coordinates of each time are masked, so as to obtain a trajectory sample to be completed containing mask coordinates, relative time and binary mask label. S3. Divide the indoor scene map into grids to obtain an indoor grid map. Based on the preprocessed trajectory sample set, statistically analyze the movement characteristics of each grid in the indoor grid map. Generate macro-level scene semantic features and fine-grained functional semantic features for each grid based on the indoor scene map; The mobile statistical features, macro-scene semantic features, and fine-grained functional semantic features are concatenated into the fused features of each grid. S4. The navigable constraint semantic mesh graph construction module uses each grid in the indoor grid map as a node, determines candidate edges based on the eight-neighbor relationship between each grid, and removes candidate edges associated with inaccessible grids based on the navigability attribute of each grid, thereby establishing bidirectional connections on the retained candidate edges to obtain a bidirectional semantic mesh graph of navigability constraints. S5. The multi-scale semantic grid graph encoding module inputs the fused features of each grid into the multi-layer graph attention network, and performs multi-head attention weighted aggregation on the neighborhood node features of each grid based on the adjacency relationship of the bidirectional semantic grid graph to obtain the node hidden state of the L layer of each grid and splice it into the multi-scale grid graph representation of each grid. S6. The mask-aware trajectory token construction module performs coordinate time encoding on the x-coordinate, y-coordinate, and relative time of each time in the preprocessed fixed-length trajectory sample that is not to be completed, as well as the mask value and relative time of each time in the continuous time to be completed, to obtain the coordinate time encoding representation corresponding to each time. Obtain a multi-scale grid diagram representation of the grid to which the x and y coordinates belong for each time point within the non-to-be-completed time point; Learnable masked grid tokens are used as the grid representations for each time step within a continuous sequence of time steps to be completed; The binary mask and the position of each time step in the fixed-length trajectory sample are embedded to obtain the mask state representation and the time position representation; The coordinate time encoding representation, mask state representation, grid representation and time position representation corresponding to each moment are concatenated to form a mask-aware trajectory token sequence; S7. Input the mask-aware trajectory token sequence into the bidirectional gated loop unit and encode it along the forward and reverse time directions respectively to obtain the forward and reverse hidden states corresponding to each time step; after concatenating the forward and reverse hidden states corresponding to each time step in the continuous time steps to be completed, input them into the coordinate decoder to obtain the standardized abscissa and standardized ordinate of each time step in the continuous time steps to be completed, and use them as the prediction coordinates. S8. Based on the predicted coordinates and the actual coordinates within the continuous time intervals to be completed, calculate the mean square error of the coordinates, the absolute error of the coordinates, and the first-order displacement consistency error respectively, and perform a weighted summation to obtain the total loss function; Using the total loss function, the indoor trajectory completion model, which consists of a multi-scale semantic grid graph encoding module, a mask-aware trajectory token construction module, a bidirectional gated recurrent unit, and a coordinate decoder, is trained end-to-end to obtain the trained indoor trajectory completion model. During inference, the trained indoor trajectory completion model outputs the predicted coordinates of each time point within the continuous time points to be completed. After de-standardizing the predicted coordinates, they are merged with the original coordinates of the non-time points to be completed in chronological order to obtain the complete indoor trajectory.

[0006] The present invention provides an indoor trajectory completion method based on multi-scale semantic mesh graphs and bidirectional temporal collaborative modeling, characterized in that, in step S4, the mesh is set... The x-coordinate and y-coordinate are respectively and The sign indicating passage is If the grid If it is passable in the indoor grid map, then let Otherwise, let And determine the grid according to equation (1). With grid Adjacency relationship between : (1) In equation (1), , Representing grids respectively x-coordinate and y-coordinate; Represents a grid The sign indicating that passage is permitted; For meshes with an adjacency relationship of 1, simultaneously create meshes. To grid and grid To grid By connecting the edges, a bidirectional semantic mesh graph that satisfies the passability constraint is constructed. Let the set of edges of the bidirectional semantic mesh graph be represented as... ,in, Represents a grid With grid The adjacency relationship between them.

[0007] Furthermore, step S6 includes: S6.1. Determine the first position in the preprocessed trajectory sample according to equations (2) and (3). The coordinate time input at each moment represents ,in, For the first The x-coordinate of the model input at each time point; For the first The ordinate of the model input at each time point; For the first The relative time of the model input at each moment: , (2) , (3) In equations (2) and (3), This represents the preset mask value; , , They represent the first The x-axis, y-axis and relative time after preprocessing at each time point; Represents the set of consecutive moments to be completed; This represents the length of a fixed-length trajectory sample. S6.2 Input coordinate time Input coordinate time encoding module Encoding is performed in the process, and the result is obtained according to equation (4). Coordinate time encoding representation at each moment : (4) In equation (4), This represents a coordinate-time encoding module that includes two layers of linear transformation and nonlinear activation. Represents a non-linear activation function. , These represent the weight matrices of the two linear transformations, respectively. , These represent the biases of the two linear transformations, respectively. S6.3, when the first When the coordinates at each moment are not masked, let Indicates the first The grid number to which the x-coordinate and y-coordinate belong at each moment. For the number The multi-scale grid diagram representation corresponding to the grid is given by... Given the mask grid token to be learned, the first one is obtained according to equation (5). The grid representation used at each time point : (5) In equation (5), Indicates the first The passability markers for each grid; Indicates the first A binary mask corresponding to the x and y coordinates at each time point, satisfying the following conditions: ; S6.4, Calculate the first... Mask-aware trajectory token at a given moment ,in, This indicates vector concatenation; Indicates mask embedding; Indicates the embedding of time position; S6.5 Arrange the mask-aware trajectory tokens corresponding to each moment in chronological order to obtain the mask-aware trajectory token sequence.

[0008] Furthermore, the total loss function in step S8 is constructed as follows: S8.1 Calculate the mean square error of the coordinates according to equation (6). : (6) In equation (6), This represents the preprocessed trajectory sample set. express Number of trajectory samples; express The total number of moments in the time frame; Indicates the first In the preprocessed trajectory sample, the first Predicted coordinates at each time point; Indicates the first In the preprocessed trajectory sample, the first The actual coordinates at any given moment; Represents the square of the L2 norm; S8.2 Calculate the absolute coordinate error using equation (7) : (7) In equation (7), Denotes the norm 1; S8.3. Suppose that the continuous interval to be completed contains K time points to be completed in chronological order. Use equation (8) to calculate the first-order displacement consistency error. : (8) In equation (8), and Let represent the predicted coordinates of the k-th and (k+1)-th time points to be completed in the b-th preprocessed trajectory sample, respectively. and Let represent the true coordinates of the k-th and (k+1)-th moments to be completed in the b-th preprocessed trajectory sample, respectively; S8.4. According to equation (9), the mean square error of coordinates, the absolute error of coordinates, and the first-order displacement consistency error are weighted and summed to obtain the total loss function. : (9) In equation (9), and These represent the non-negative weighting coefficients corresponding to the absolute coordinate error and the first-order displacement consistency error, respectively.

[0009] The present invention provides an indoor trajectory completion system based on multi-scale semantic mesh graph and bidirectional temporal collaborative modeling, characterized by including: an input and prior representation module, a spatiotemporal feature extraction and fusion module, and a trajectory completion module; The input and prior representation module is used to receive trajectory samples containing horizontal coordinates, vertical coordinates, relative time, and binary mask labels, and to divide the indoor scene map into grids. Based on the trajectory samples, the module calculates the motion statistical features of each grid, and determines the macro-scene semantic features and fine-grained functional semantic features of each grid according to the indoor scene map, thereby obtaining the fusion features of each grid. The spatiotemporal feature extraction and fusion module includes: a navigable constraint semantic mesh graph construction module, a multi-scale semantic mesh graph encoding module, and a mask-aware trajectory token construction module; Among them, the drivability constraint semantic mesh graph construction module is used to construct a bidirectional semantic mesh graph based on the eight-neighbor relationship and drivability attribute between each mesh; The multi-scale semantic grid graph encoding module is used to perform weighted aggregation of the features of the neighboring nodes of each grid through a multi-layer graph attention network, and to concatenate the hidden states of the nodes corresponding to different graph attention layers to obtain a multi-scale grid graph representation. The mask-aware trajectory token building module is used to concatenate the coordinate time encoding representation, mask state representation, grid representation, and time position representation to obtain the mask-aware trajectory token sequence; The trajectory completion module is used to encode the mask-aware trajectory token sequence through a bidirectional gated loop unit to obtain the forward and reverse hidden states corresponding to each time step. The forward and reverse hidden states of each time step within the continuous interval to be completed are then concatenated and input into the coordinate decoder to obtain the predicted coordinates of each time step to be completed.

[0010] The present invention provides an electronic device, including a memory and a processor, characterized in that the memory is used to store a program supporting the processor in performing the method described therein, and the processor is configured to execute the program stored in the memory.

[0011] The present invention discloses a computer-readable storage medium storing a computer program, characterized in that the computer program is executed by a processor to perform the steps of the method described thereon.

[0012] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention integrates the motion statistical features, macro-scene semantic features, and fine-grained functional semantic features obtained from training trajectories to obtain the fused features of each grid. This allows the grid features to simultaneously represent the historical accessibility, dwell characteristics, accessibility attributes, and functional area attributes of different indoor locations. This overcomes the problem of insufficient spatial information expression caused by existing methods that only use coordinates, grid numbers, or single scene categories, thus providing a more complete indoor scene prior for the recovery of continuously missing trajectories.

[0013] 2. This invention retains the relative time corresponding to each moment to be completed within a continuous interval to be completed, and combines the coordinate time representation, mask state representation, grid representation, and moment position representation to form a mask-aware trajectory token; through bidirectional gated loop units, encoding is performed along the forward and reverse time directions respectively, so that the coordinate recovery of each moment within the interval to be completed can simultaneously utilize the observation trajectory information before and after the interval, overcoming the problem that unidirectional time series processing is highly dependent on information on one side of the missing interval, thereby improving the accuracy of recovering the continuous missing trajectory in the middle.

[0014] 3. This invention constructs a total loss function by using coordinate mean square error, coordinate absolute error, and first-order displacement consistency error. While constraining the positional deviation between predicted coordinates and true coordinates, it also constrains the first-order displacement between adjacent predicted coordinates to be consistent with the first-order displacement of the true trajectory. This reduces the risk of local jitter, backtracking, and discontinuity in the connection of adjacent trajectory points in the completed trajectory, thereby improving the coordinate accuracy and motion continuity of the completed trajectory. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the framework for multi-scale semantic mesh graph and bidirectional temporal collaborative trajectory completion proposed in this invention; Figure 2 This is a flowchart of the method of the present invention; Figure 3 This is a schematic diagram of three types of indoor semantic grid maps in an embodiment of the present invention; Figure 4 This is a schematic diagram of the passable constraint eight-neighbor domain and graph attention aggregation in an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the four components of the mask-sensing trajectory token in this invention; Figure 6 This is a schematic diagram of the forward and reverse encoding and hidden state fusion of the bidirectional gated loop unit in this invention; Figure 7 This is a comparison of the mean square error of different trajectory completion methods in the embodiments of the present invention; Figure 8 This is a case analysis diagram of typical continuous missing trajectory completion in the embodiments of the present invention. Detailed Implementation

[0016] In this embodiment, an indoor trajectory completion system based on multi-scale semantic mesh graphs and bidirectional temporal collaborative modeling is described, such as... Figure 1 As shown, it includes: an input and prior representation module, a spatiotemporal feature extraction and fusion module, and a trajectory completion module.

[0017] The input and prior representation module is used to receive trajectory samples containing horizontal coordinates, vertical coordinates, relative time, and binary masks, and to divide the indoor scene map into grids. Based on the training trajectory, the module calculates the movement statistics of each grid, and determines the macro-scene semantic features and fine-grained functional semantic features of each grid according to the indoor scene map, thereby obtaining the fusion features of each grid.

[0018] The spatiotemporal feature extraction and fusion module includes: a navigable constraint semantic mesh graph construction module, a multi-scale semantic mesh graph encoding module, and a mask-aware trajectory token construction module. The navigable constraint semantic mesh graph construction module constructs a bidirectional semantic mesh graph based on the eight-neighborhood relationships and navigability attributes between each mesh. The multi-scale semantic mesh graph encoding module weights and aggregates the features of neighboring nodes of each mesh using a multi-layer graph attention network, and concatenates the hidden states of nodes corresponding to different graph attention layers to obtain a multi-scale mesh graph representation. The mask-aware trajectory token construction module concatenates the coordinate time encoding representation, mask state representation, mesh representation, and time-position representation to obtain a mask-aware trajectory token sequence.

[0019] The trajectory completion module is used to input the mask-aware trajectory token sequence into the bidirectional gated loop unit to obtain the forward and reverse hidden states corresponding to each time step. The forward and reverse hidden states of each time step within the continuous interval to be completed are then concatenated and input into the coordinate decoder to obtain the predicted coordinates of each time step to be completed.

[0020] like Figure 2 As shown, a multi-scale semantic mesh graph and bidirectional temporal collaborative trajectory completion method includes the following steps: S1. Obtain the x-coordinate y-axis and relative time The indoor trajectory and indoor scene map are obtained using a fixed window length. The sliding step size is The sliding window divides the indoor trajectory into trajectory samples of fixed length. The data is then standardized to obtain a preprocessed trajectory sample set. , indicating the first The trajectory points corresponding to each moment.

[0021] Using equation (1) to obtain the first The x-coordinate after standardization at each time point y-axis and the Standardized representation of relative time at each moment : (1) In equation (1), , and These represent the mean values ​​corresponding to the x-axis, y-axis, and relative time, respectively. , and These represent the corresponding standard deviations. After the above processing, the preprocessed trajectory sample set is obtained.

[0022] S2. Determine the consecutive moments to be completed in any preprocessed trajectory sample. ,in, This is the index of the starting time of the interval to be filled. The index is the end time, and the interval length is [value]. The x and y coordinates corresponding to consecutive moments to be completed are replaced with preset mask values, the relative times corresponding to consecutive moments to be completed are retained, and binary mask labels are generated to characterize whether the coordinates at each moment are masked, thus obtaining a trajectory sample to be completed containing mask coordinates, relative time, and binary mask labels.

[0023] Let the mask value be Then, using equation (2), we can obtain the first... The x-coordinate of the model input at each time point and ordinate : (2) The relative time remains known at all times, and the result is obtained using equation (3). The relative time of the model input at each moment : (3) In equation (3), This indicates the length of a fixed-length trajectory sample.

[0024] Using equation (4), we obtain the first... Binary mask label representation at each time step : (4) In equation (4), For the supervision labels of the interval to be filled, It represents the set of consecutive moments that need to be completed.

[0025] S3. For example Figure 3 As shown, an indoor scene map is obtained by dividing it into grids; an indoor grid map is generated by indexing any grid node. Based on the preprocessed trajectory sample set, the grids in the indoor grid map are statistically analyzed. Statistical characteristics of movement Including: grid Normalized dwell frequency and average length of stay .

[0026] In an indoor grid map, different colored grids represent different semantic information: corridors, indoor areas, and impassable areas. The grid is generated based on these different semantic information. Macro-scene semantic features and fine-grained functional semantic features .

[0027] The mobile statistical features, macro-level scene semantic features, and fine-grained functional semantic features are concatenated into a grid. Fusion characteristics ;in, The vector concatenation operation is represented in one specific implementation where the motion statistical features are 2-dimensional, the macro-scene semantic features are 3-dimensional, the fine-grained functional semantic features are 17-dimensional, and the mesh fusion features are 22-dimensional.

[0028] S4. For example Figure 4As shown, a bidirectional semantic mesh graph of drivability constraints is constructed using the drivability constraint semantic mesh graph construction module. Let... Figure 4 The red center point grid The x-coordinate and y-coordinate are respectively and The surrounding blue grid consists of neighboring grid nodes of the center point grid. Let the passability indicator be... If the grid If it is passable in the indoor grid map, then let Otherwise, let The grid is determined according to equation (5). With grid Adjacency relationship between : (5) In equation (5), , Representing grids respectively x-coordinate and y-coordinate; Represents a grid The sign indicating that passage is permitted.

[0029] For meshes with an adjacency relationship of 1, simultaneously create meshes. To grid and grid To grid By connecting the edges, a bidirectional semantic mesh graph with traversable constraints is constructed. The edge set of the bidirectional semantic mesh graph is represented as... ,in, Represents a grid With grid The adjacency relationship between them. Figure 4 The white corridor grid and the gray indoor area are both considered as neighboring grid nodes of the center point, while the black impassable area cannot be considered as a neighboring grid node.

[0030] S5. Using a multi-scale semantic grid graph encoding module, the fused features of each grid are input into a multi-layer graph attention network. Based on the adjacency relationship of the bidirectional semantic grid graph, the neighborhood node features of each grid are weighted and aggregated by multi-head attention to obtain the node hidden state of layer L of each grid and splice them into a multi-scale grid graph representation of each grid.

[0031] Order No. Grid nodes at the input of the attention layer of the graph Features are , where initialization When =0, let ;No. The attention layer of the graph Each attention head first performs a linear transformation on the node features using equation (6), and then calculates the node... Its neighboring nodes Unnormalized attention coefficient : (6) In equation (6), and It is the first The attention layer of the graph The learnable linear transformation matrix corresponding to each attention head Indicates the first The attention layer of the graph A learnable attention parameter vector. and Representing grid nodes respectively and grid nodes After the first The attention layer of the graph The node features corresponding to each attention head; This represents the activation function of the linear unit with leakage correction.

[0032] At the node accessible neighborhood Within the node itself, the unnormalized attention coefficient is calculated using equation (7). Perform normalization to obtain the first... The attention layer of the graph Each attention head corresponds to a normalized attention coefficient. : (7) Using equation (8), we obtain the first... The attention layer of the graph Attention head to node aggregation results and the Layer nodes Hidden state representation : (8) In equation (8), and For learnable parameters, For the first The number of attention heads per layer Presentation layer normalization operation, This represents the exponential linear unit activation function; random deactivation is applied between layers during the training phase; By concatenating the hidden states of nodes at different propagation levels, we obtain the grid nodes. Multi-scale grid diagram representation .

[0033] In one specific implementation, a three-layer graph attention network is used. The first two layers are spliced ​​with multiple heads, and the third layer is output with a single head. The hidden states of the three layers are spliced ​​together.

[0034] S6. For example Figure 5 As shown, the mask-aware trajectory token construction module is used to perform coordinate time encoding on the x-coordinate, y-coordinate, and relative time of each time in the preprocessed fixed-length trajectory sample that is not to be completed, as well as the mask value and relative time of each time in the continuous time to be completed, to obtain the coordinate time encoding representation corresponding to each time.

[0035] For each time point not within the time frame to be completed, obtain the multi-scale grid diagram representation of the corresponding horizontal and vertical coordinate grids; for each time point within the continuous time frame to be completed, use a learnable mask grid token as the corresponding grid representation; embed the binary mask and the position of each time point in the fixed-length trajectory sample respectively to obtain the mask state representation and the time position representation; concatenate the coordinate time encoding representation, mask state representation, grid representation and time position representation corresponding to each time point to form a mask-aware trajectory token sequence.

[0036] S6.1. Construct the first model based on the model input obtained in step S2. The coordinate time input vector at each moment ,in, , and They respectively represent the first determined in step S2 The standardized x-axis, standardized y-axis, and standardized relative time are the model inputs at each time point.

[0037] S6.2 Input coordinate time Input coordinate time encoding module And according to equation (9), the first one is encoded to obtain the second one. Coordinate time encoding representation at each moment : (9) In equation (9), This represents a coordinate-time encoding module that includes two layers of linear transformation and nonlinear activation. Represents a non-linear activation function. , These represent the weight matrices of the two linear transformations, respectively. , These represent the biases of the two linear transformations, respectively.

[0038] S6.3, when the first When the coordinates at each moment are not masked, let Indicates the first The grid number to which the x-coordinate and y-coordinate belong at each moment. Indicates the number is The multi-scale grid diagram representation corresponding to the grid is given by... The mask grid token to be learned is obtained according to equation (10). The grid representation used at each time point : (10) In equation (10), Indicates the first The passability markers for each grid; Indicates the first A binary mask corresponding to the x and y coordinates at each time point, satisfying the following conditions: .

[0039] S6.4, Calculate the first... Mask-aware trajectory token at a given moment ,in, This indicates vector concatenation; Indicates mask embedding; The embedding of time position is represented; the mask-aware trajectory tokens corresponding to each time point are arranged in chronological order to obtain the mask-aware trajectory token sequence.

[0040] S7. For example Figure 6 As shown, the mask-aware trajectory token sequence Input to the forward gated loop unit respectively and reverse gated loop unit Thus, by using equation (11), the first... The positive hidden state at each moment and reverse hidden state representation : (11) In equation (11), Indicates the first A mask-sensing trajectory token at a given moment. and These represent the forward-gated loop unit and the reverse-gated loop unit, respectively. and They represent the first The forward hidden state and the reverse hidden state corresponding to each time point.

[0041] Using equation (12), we obtain the first digit of the interval to be completed. Bidirectional hidden representation at each time step and predicted coordinate representation : (12) In equation (12), Indicates the first Predicted coordinates at each time point. and These represent the weights of the two linear transformation layers in the coordinate decoder; and These represent the biases of the two linear transformations in the coordinate decoder. The final output dimension of the coordinate decoder is 2, and it only extracts the bidirectional hidden states within the interval to be completed for coordinate prediction.

[0042] S8. Based on the predicted coordinates and the true coordinates within the continuous time intervals to be completed, calculate the mean square error of the coordinates, the absolute error of the coordinates, and the first-order displacement consistency error, respectively. Then, perform a weighted summation of the mean square error of the coordinates, the absolute error of the coordinates, and the first-order displacement consistency error to obtain the total loss function. Using the total loss function, perform end-to-end training on the indoor trajectory completion model composed of a multi-scale semantic grid graph encoding module, a mask-aware trajectory token construction module, a bidirectional gated recurrent unit, and a coordinate decoder to obtain the trained indoor trajectory completion model.

[0043] S8.1 Calculate the mean square error of the coordinates according to equation (13). : (13) In equation (13), This represents the preprocessed trajectory sample set. express Number of trajectory samples; express The total number of moments in the time frame; Indicates the first In the preprocessed trajectory sample, the first Predicted coordinates at each time point; Indicates the first In the preprocessed trajectory sample, the first The actual coordinates at any given moment; This represents the square of the L2 norm.

[0044] S8.2 Calculate the absolute coordinate error using equation (14) : (14) In equation (14), It represents the first norm.

[0045] S8.3. Suppose that the continuous interval to be completed contains K time points to be completed in chronological order. Use equation (15) to calculate the first-order displacement consistency error. : (15) In equation (15), and Let represent the predicted coordinates of the k-th and (k+1)-th time points to be completed in the b-th trajectory sample, respectively. and These represent the actual coordinates of the corresponding time points to be filled in.

[0046] S8.4. According to equation (16), the mean square error of coordinates, the absolute error of coordinates, and the first-order displacement consistency error are weighted and summed to obtain the total loss function. : (16) In equation (16), and These represent the non-negative weighting coefficients corresponding to the absolute coordinate error and the first-order displacement consistency error, respectively.

[0047] During inference, the trained indoor trajectory completion model outputs the predicted coordinates of each time point within the continuous time points to be completed. The predicted coordinates are de-standardized, and the de-standardized predicted coordinates are merged with the original coordinates of non-time points to be completed in chronological order to obtain the complete indoor trajectory.

[0048] In this embodiment, an electronic device includes a memory and a processor. The memory stores a program that supports the processor in executing the above-described method, and the processor is configured to execute the program stored in the memory.

[0049] In this embodiment, a computer-readable storage medium stores a computer program, which is executed by a processor to perform the steps of the above method.

[0050] To verify the completion accuracy of the indoor trajectory completion method described in this embodiment, under the same test data, continuous interval to be completed, and evaluation index, the method of this embodiment is compared with the linear interpolation method, the conditional diffusion time series completion method (CSDI), and the spatiotemporal diffusion trajectory completion method (PriSTI). The mean square error between the standardized predicted coordinates and the corresponding standardized true coordinates within the continuous interval to be completed is used as the evaluation index. The smaller the mean square error, the smaller the deviation between the coordinates to be completed and the true coordinates.

[0051] like Figure 7As shown, the mean squared errors for the linear interpolation method, CSDI method, PriSTI method, and the method of this embodiment are 0.0669, 0.0286, 0.0134, and 0.0120, respectively. The mean squared error obtained by the method of this embodiment is smaller than that of the above-mentioned comparative methods, indicating that by combining multi-scale grid diagram representation, mask-aware trajectory tokens, and bidirectional temporal context information, the error between predicted coordinates and true coordinates within a continuous interval to be completed can be reduced.

[0052] To further illustrate the effectiveness of the method in this embodiment for completing continuously missing indoor trajectories, an indoor trajectory with consecutive missing coordinates in the middle is selected as an example. The completion result is as follows: Figure 8 As shown. Figure 8 Using an indoor grid map as a background, grids with different filling patterns are used to represent different types of indoor space areas; polylines and trajectory points outside the continuous interval to be completed represent the observed trajectories; the actual trajectories within the continuous interval to be completed are used as a reference; and the trajectory segments output by the model represent the prediction results obtained by the method in this embodiment for completing the continuous interval to be completed.

[0053] Depend on Figure 8 It can be seen that the predicted trajectory segment obtained by the method in this embodiment can connect the observed trajectories on both sides of the continuous interval to be completed. Its overall movement direction is basically consistent with the corresponding real trajectory segment. The predicted trajectory is located within the passable area of ​​the indoor map, and there is no obvious coordinate jump or local backtracking phenomenon. The above results show that the method in this embodiment can utilize the observed trajectory information before and after the continuous interval to be completed, and combine it with the prior knowledge of the indoor grid space to recover the continuous missing trajectory, thereby improving the spatial rationality and motion continuity of trajectory completion.

Claims

1. An indoor trajectory completion method based on multi-scale semantic mesh graphs and bidirectional temporal collaborative modeling, characterized in that, Includes the following steps: S1. Obtain indoor trajectory and indoor scene map with horizontal coordinate, vertical coordinate and relative time. Use a sliding window to divide the indoor trajectory into trajectory samples of fixed length and perform standardization to obtain a preprocessed trajectory sample set. S2. In any preprocessed trajectory sample, determine the continuous time to be completed, replace the horizontal and vertical coordinates corresponding to the continuous time to be completed with preset mask values, retain the relative time corresponding to the continuous time to be completed, and generate a binary mask label to represent whether the coordinates of each time are masked, so as to obtain a trajectory sample to be completed containing mask coordinates, relative time and binary mask label. S3. Divide the indoor scene map into grids to obtain an indoor grid map. Based on the preprocessed trajectory sample set, statistically analyze the movement characteristics of each grid in the indoor grid map. Generate macro-level scene semantic features and fine-grained functional semantic features for each grid based on the indoor scene map; The mobile statistical features, macro-scene semantic features, and fine-grained functional semantic features are concatenated into the fused features of each grid. S4. The navigable constraint semantic mesh graph construction module uses each grid in the indoor grid map as a node, determines candidate edges based on the eight-neighbor relationship between each grid, and removes candidate edges associated with inaccessible grids based on the navigability attribute of each grid, thereby establishing bidirectional connections on the retained candidate edges to obtain a bidirectional semantic mesh graph of navigability constraints. S5. The multi-scale semantic grid graph encoding module inputs the fused features of each grid into the multi-layer graph attention network, and performs multi-head attention weighted aggregation on the neighborhood node features of each grid based on the adjacency relationship of the bidirectional semantic grid graph to obtain the node hidden state of the L layer of each grid and splice it into the multi-scale grid graph representation of each grid. S6. The mask-aware trajectory token construction module performs coordinate time encoding on the x-coordinate, y-coordinate, and relative time of each time in the preprocessed fixed-length trajectory sample that is not to be completed, as well as the mask value and relative time of each time in the continuous time to be completed, to obtain the coordinate time encoding representation corresponding to each time. Obtain a multi-scale grid diagram representation of the grid to which the x and y coordinates belong for each time point within the non-to-be-completed time point; Learnable masked grid tokens are used as the grid representations for each time step within a continuous sequence of time steps to be completed; The binary mask and the position of each time step in the fixed-length trajectory sample are embedded to obtain the mask state representation and the time position representation; The coordinate time encoding representation, mask state representation, grid representation and time position representation corresponding to each moment are concatenated to form a mask-aware trajectory token sequence; S7. Input the mask-aware trajectory token sequence into the bidirectional gated loop unit and encode it along the forward and reverse time directions respectively to obtain the forward and reverse hidden states corresponding to each time point; The forward and reverse hidden states corresponding to each time step within the continuous time steps to be completed are concatenated and then input into the coordinate decoder to obtain the standardized abscissa and ordinate of each time step within the continuous time steps to be completed, which are then used as the prediction coordinates. S8. Based on the predicted coordinates and the actual coordinates within the continuous time intervals to be completed, calculate the mean square error of the coordinates, the absolute error of the coordinates, and the first-order displacement consistency error respectively, and perform a weighted summation to obtain the total loss function; Using the total loss function, the indoor trajectory completion model, which consists of a multi-scale semantic grid graph encoding module, a mask-aware trajectory token construction module, a bidirectional gated recurrent unit, and a coordinate decoder, is trained end-to-end to obtain the trained indoor trajectory completion model. During inference, the trained indoor trajectory completion model outputs the predicted coordinates of each time point within the continuous time points to be completed. After de-standardizing the predicted coordinates, they are merged with the original coordinates of the non-time points to be completed in chronological order to obtain the complete indoor trajectory.

2. The indoor trajectory completion method based on multi-scale semantic mesh graph and bidirectional temporal collaborative modeling according to claim 1, characterized in that, In step S4, the grid is set. The x and y coordinates are respectively and The sign indicating passage is If the grid If it is passable in the indoor grid map, then let Otherwise, let And determine the grid according to equation (1). With grid Adjacency relationship between : (1) In equation (1), , Representing grids respectively x-coordinate and y-coordinate; Represents a grid The sign indicating that passage is permitted; For meshes with an adjacency relationship of 1, simultaneously create meshes. To grid and grid To grid By connecting the edges of the bidirectional semantic mesh graph, a bidirectional semantic mesh graph that satisfies the passability constraint is constructed. Let the set of edges of the bidirectional semantic mesh graph be represented as... ,in, Represents a grid With grid The adjacency relationship between them.

3. The indoor trajectory completion method based on multi-scale semantic mesh graph and bidirectional temporal collaborative modeling according to claim 2, characterized in that, Step S6 includes: S6.

1. Determine the first position in the preprocessed trajectory sample according to equations (2) and (3). The coordinate time input at each moment represents ,in, For the first The x-coordinate of the model input at each time point; For the first The ordinate of the model input at each time point; For the first The relative time of the model input at each moment: ， (2) ， (3) In equations (2) and (3), This represents the preset mask value; , , They represent the first The x-axis, y-axis and relative time after preprocessing at each time point; Represents the set of consecutive moments to be completed; This represents the length of a fixed-length trajectory sample. S6.2 Input coordinate time Input coordinate time encoding module Encoding is performed in the process, and the result is obtained according to equation (4). Coordinate time encoding representation at each moment : (4) In equation (4), This represents a coordinate-time encoding module that includes two layers of linear transformation and nonlinear activation. Represents a non-linear activation function. , These represent the weight matrices of the two linear transformations, respectively. , These represent the biases of the two linear transformations, respectively. S6.3, when the first When the coordinates at each moment are not masked, let Indicates the first The grid number to which the x-coordinate and y-coordinate belong at each moment. For the number The multi-scale grid diagram representation corresponding to the grid is given by... Given the mask grid token to be learned, the first one is obtained according to equation (5). The grid representation used at each time point : (5) In equation (5), Indicates the first The passability markers for each grid; Indicates the first A binary mask corresponding to the x and y coordinates at each time point, satisfying the following conditions: ; S6.4, Calculate the first... Mask-aware trajectory token at a given moment ,in, This indicates vector concatenation; Indicates mask embedding; Indicates the embedding of time position; S6.5 Arrange the mask-aware trajectory tokens corresponding to each moment in chronological order to obtain the mask-aware trajectory token sequence.

4. The indoor trajectory completion method based on multi-scale semantic mesh graph and bidirectional temporal collaborative modeling according to claim 3, characterized in that, The total loss function in step S8 is constructed as follows: S8.1 Calculate the mean square error of the coordinates according to equation (6). : (6) In equation (6), This represents the preprocessed trajectory sample set. express Number of trajectory samples; express The total number of moments in the time frame; Indicates the first In the preprocessed trajectory sample, the first Predicted coordinates at each time point; Indicates the first In the preprocessed trajectory sample, the first The actual coordinates at any given moment; Represents the square of the L2 norm; S8.2 Calculate the absolute coordinate error using equation (7) : (7) In equation (7), Denotes the norm 1; S8.

3. Suppose that the continuous interval to be completed contains K time points to be completed in chronological order. Use equation (8) to calculate the first-order displacement consistency error. : (8) In equation (8), and Let represent the predicted coordinates of the k-th and (k+1)-th time points to be completed in the b-th preprocessed trajectory sample, respectively. and Let represent the true coordinates of the k-th and (k+1)-th moments to be completed in the b-th preprocessed trajectory sample, respectively; S8.

4. According to equation (9), the mean square error of coordinates, the absolute error of coordinates, and the first-order displacement consistency error are weighted and summed to obtain the total loss function. : (9) In equation (9), and These represent the non-negative weighting coefficients corresponding to the absolute coordinate error and the first-order displacement consistency error, respectively.

5. An indoor trajectory completion system based on multi-scale semantic mesh graphs and bidirectional temporal collaborative modeling, characterized in that, include: The module includes an input and prior representation module, a spatiotemporal feature extraction and fusion module, and a trajectory completion module. The input and prior representation module is used to receive trajectory samples containing horizontal coordinates, vertical coordinates, relative time, and binary mask labels, and to divide the indoor scene map into grids. Based on the trajectory samples, the module calculates the motion statistical features of each grid, and determines the macro-scene semantic features and fine-grained functional semantic features of each grid according to the indoor scene map, thereby obtaining the fusion features of each grid. The spatiotemporal feature extraction and fusion module includes: a navigable constraint semantic mesh graph construction module, a multi-scale semantic mesh graph encoding module, and a mask-aware trajectory token construction module; Among them, the drivability constraint semantic mesh graph construction module is used to construct a bidirectional semantic mesh graph based on the eight-neighbor relationship and drivability attribute between each mesh; The multi-scale semantic grid graph encoding module is used to perform weighted aggregation of the features of the neighboring nodes of each grid through a multi-layer graph attention network, and to concatenate the hidden states of the nodes corresponding to different graph attention layers to obtain a multi-scale grid graph representation. The mask-aware trajectory token building module is used to concatenate the coordinate time encoding representation, mask state representation, grid representation, and time position representation to obtain the mask-aware trajectory token sequence; The trajectory completion module is used to encode the mask-aware trajectory token sequence through a bidirectional gated loop unit to obtain the forward and reverse hidden states corresponding to each time step. The forward and reverse hidden states of each time step within the continuous interval to be completed are then concatenated and input into the coordinate decoder to obtain the predicted coordinates of each time step to be completed.

6. An electronic device, comprising a memory and a processor, characterized in that, The memory is used to store a program that supports a processor in executing the method of any one of claims 1-4, the processor being configured to execute the program stored in the memory.

7. A computer-readable storage medium storing a computer program thereon, characterized in that, The computer program is executed by the processor to perform the steps of the method according to any one of claims 1-4.

Images