Real-time trajectory data compression optimization method and system for high-precision positioning

Abstract

The present application relates to high-precision positioning and real-time data compression technical field, specifically for high-precision positioning real-time trajectory data compression optimization method and system, including: collecting high-frequency original positioning data packet sequence, and dividing into trajectory segment according to terminal identification;Data quality index is calculated;The trajectory segment is compressed into semantic latent vector, and the latent distribution variance is obtained;The reconstructed trajectory segment is obtained;The reconstruction error is calculated;Normalized variance and normalized reconstruction error are obtained;The illusion risk index is solved;In response to the illusion risk index being not greater than the forensic integrity threshold, a high compression rate semantic compression mode is executed, and the semantic latent vector is output;In response to the illusion risk index being greater than the forensic integrity threshold, a fragment fidelity compression mode is executed, and the compressed trajectory segment and the data discontinuity warning flag are output;The present application realizes the dynamic data source selection of the alarm engine, and ensures the accuracy of real-time alarm.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of high-precision positioning and real-time data compression, in particular to a real-time trajectory data compression optimization method and system for high-precision positioning. BACKGROUND

[0002] With the application of high-precision positioning systems in high-frequency scenarios, a large amount of real-time trajectory data is generated. In complex network environments, such as network storms, these data streams are extremely vulnerable to severe packet loss and timing disorder during transmission.

[0003] Currently, in order to save storage costs, the industry has begun to use artificial intelligence compression models, such as variational autoencoder (VAE) for semantic compression of trajectories. However, when these AI models receive broken and incomplete trajectory data due to network problems, there is a risk of generating data hallucinations during the reconstruction process. Such false trajectories generated by the model, i.e. hallucination data, can seriously pollute the historical database, making the stored data lose forensic integrity and unable to be used for serious judicial evidence or responsibility retracement. Therefore, how to achieve high data compression rate while effectively avoiding the hallucination risk of AI models and ensuring the forensic integrity of historical data has become a technical problem that needs to be solved in the field. SUMMARY

[0004] To solve the above technical problems, the present application provides a real-time trajectory data compression optimization method and system for high-precision positioning. Specifically, the technical solution of the present application is as follows:

[0005] The real-time trajectory data compression optimization method for high-precision positioning comprises:

[0006] Collecting high-frequency original positioning data packet sequences and dividing them into trajectory segments according to terminal identifiers;

[0007] Determining the packet loss rate of the trajectory segments within a preset sliding time window;

[0008] Determining the out-of-order rate of the trajectory segments within the sliding time window;

[0009] Based on the packet loss rate and the out-of-order rate, the data quality index is calculated;

[0010] Using a pre-trained variational autoencoder, the trajectory segments are compressed into semantic latent vectors, and the latent distribution variance is obtained;

[0011] Based on the semantic latent vectors, the trajectory segments are reconstructed to obtain reconstructed trajectory segments;

[0012] Combining the trajectory segments and the reconstructed trajectory segments, the reconstruction error is calculated;

[0013] The latent distribution variance is normalized to obtain the normalized variance;

[0014] The reconstruction error is normalized to obtain the normalized reconstruction error;

[0015] The illusion risk index is calculated based on the data quality index, normalized variance, and normalized reconstruction error.

[0016] The hallucination risk index is compared with a preset legal integrity threshold;

[0017] In response to the fact that the hallucination risk index is no greater than the legal integrity threshold, a high compression ratio semantic compression mode is executed, and a semantic latent vector is output.

[0018] In response to the illusion risk index exceeding the forensic integrity threshold, the fragment fidelity compression mode is executed to perform lossless or near-lossless compression on the trajectory fragments, and the compressed trajectory fragments and data discontinuity warning flags are output.

[0019] Preferably, determining the packet loss rate of a trajectory segment within a preset sliding time window includes:

[0020] Compare the expected number of timestamps within the sliding time window with the actual number received;

[0021] Determine the out-of-order rate of trajectory segments within the sliding time window, including:

[0022] Calculate the number of inverted pairs of timestamps that are not monotonically increasing within the sliding time window.

[0023] Preferably, a data quality index is calculated based on packet loss rate and out-of-order rate, including:

[0024] Based on preset impact weights, the packet loss rate and out-of-order rate are calculated using weighted averages.

[0025] Preferably, the reconstruction error is calculated by combining the trajectory segment and the reconstructed trajectory segment, including:

[0026] Calculate the dynamic time-warped distance between the trajectory segment and the reconstructed trajectory segment.

[0027] Preferably, the variance of the latent distribution is normalized to obtain the normalized variance, including:

[0028] The variance of the potential distribution is normalized using a preset baseline variance;

[0029] The reconstruction error is normalized to obtain the normalized reconstruction error, which includes:

[0030] The reconstruction error is normalized using a preset baseline reconstruction error.

[0031] Preferably, a high compression ratio semantic compression mode is implemented, including:

[0032] Discard the trajectory fragment;

[0033] Store semantic latent vectors, or store semantic labels that match semantic latent vectors;

[0034] The output contains a data packet containing a semantic latent vector and a pattern identifier.

[0035] Preferably, a fragmented compression mode is performed, including:

[0036] Discard the semantic latent vectors and reconstructed trajectory fragments generated by the variational autoencoder;

[0037] The trajectory segment is compressed using a standard lossless or near-lossless compression algorithm to obtain the compressed trajectory segment.

[0038] Output a data packet containing compressed track segments, a data discontinuity warning flag, and a pattern identifier.

[0039] Preferred, pre-trained variational autoencoders include:

[0040] An encoder is used to compress trajectory fragments into semantic latent vectors and latent distribution variance;

[0041] Decoder used to reconstruct trajectory fragments from semantic latent vectors;

[0042] The variational autoencoder was trained offline on a forensically complete trajectory dataset.

[0043] Preferably, it also includes a parallel-running alarm engine, including:

[0044] In response to the execution of a high-compression-rate semantic compression mode, the alarm engine performs real-time analysis based on the reconstructed trajectory fragments;

[0045] In response to the execution of fragmented compression mode, the alarm engine performs real-time analysis based on trajectory fragments.

[0046] A real-time trajectory data compression and optimization system for high-precision positioning includes:

[0047] The data acquisition module is used to acquire high-frequency raw positioning data packet sequences and divide them into trajectory segments according to terminal identifiers;

[0048] The data quality module is used to determine the packet loss rate of trajectory segments within a preset sliding time window, determine the out-of-order rate of trajectory segments within the sliding time window, and calculate the data quality index based on the packet loss rate and the out-of-order rate.

[0049] The semantic compression module is used to compress trajectory fragments into semantic latent vectors using a pre-trained variational autoencoder and to obtain the variance of the latent distribution.

[0050] The trajectory reconstruction module is used to reconstruct trajectory segments based on semantic latent vectors, thus obtaining the reconstructed trajectory segments.

[0051] The risk calculation module is used to calculate the reconstruction error by combining the trajectory segment and the reconstructed trajectory segment, normalize the potential distribution variance to obtain the normalized variance, normalize the reconstruction error to obtain the normalized reconstruction error, and calculate the illusion risk index based on the data quality index, normalized variance and normalized reconstruction error.

[0052] The data triage module is used to compare the hallucination risk index with a preset forensic integrity threshold;

[0053] The data output module is used to execute a high-compression-rate semantic compression mode and output a semantic latent vector in response to the hallucination risk index not being greater than the forensic integrity threshold; and to execute a fragment-fidelity compression mode to perform lossless or near-lossless compression processing on the trajectory fragments in response to the hallucination risk index being greater than the forensic integrity threshold, and output the compressed trajectory fragments and a data discontinuity warning flag.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] 1. This invention dynamically selects compression strategies by real-time assessment of data quality and model uncertainty, and proactively avoids artificial intelligence compression in high-risk situations, thus preventing false and illusory data from entering the database and ensuring the authenticity and forensic integrity of the historical database.

[0056] 2. This invention achieves dynamic triage of compression strategies. When the data quality is high and the risk of hallucination is low, high compression ratio semantic compression is performed to save storage. When the data quality is poor and the risk is high, it switches to fidelity compression to protect the data. This achieves an effective balance between high compression ratio and forensic integrity.

[0057] 3. This invention innovatively constructs an illusion risk index, which not only quantifies the degree of packet loss and disorder of input data, but also comprehensively evaluates the uncertainty and reconstruction bias of artificial intelligence models in processing fragmented data, making the risk assessment more comprehensive and accurate.

[0058] 4. By decoupling data storage and real-time alarm logic, this invention enables dynamic data source selection for the alarm engine. Regardless of the compression mode chosen by the system to store data, alarm analysis is always based on the most reliable data source, ensuring the accuracy of real-time alarms. Attached Figure Description

[0059] The present invention will be further explained below with reference to the accompanying drawings and embodiments:

[0060] Figure 1 This is a flowchart of the method of the present invention;

[0061] Figure 2 This is a structural diagram of the system of the present invention. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0063] Example 1:

[0064] Please see Figure 1 Real-time trajectory data compression and optimization methods for high-precision positioning include:

[0065] High-frequency raw positioning data packet sequences were collected and divided into trajectory segments according to terminal identifiers;

[0066] Determine the packet loss rate of the trajectory segment within a preset sliding time window;

[0067] Determine the disorder rate of trajectory segments within the sliding time window;

[0068] The data quality index is calculated based on packet loss rate and out-of-order rate;

[0069] A pre-trained variational autoencoder is used to compress trajectory fragments into semantic latent vectors and obtain the variance of the latent distribution;

[0070] Based on semantic latent vectors, trajectory segments are reconstructed to obtain the reconstructed trajectory segments;

[0071] By combining the trajectory segments with the reconstructed trajectory segments, the reconstruction error is calculated;

[0072] The variance of the latent distribution is normalized to obtain the normalized variance;

[0073] The reconstruction error is normalized to obtain the normalized reconstruction error;

[0074] The hallucination risk index is calculated based on the data quality index, normalized variance, and normalized reconstruction error.

[0075] The hallucination risk index is compared with a preset legal integrity threshold;

[0076] In response to the fact that the hallucination risk index is no greater than the legal integrity threshold, a high-compression-ratio semantic compression mode is executed, and a semantic latent vector is output;

[0077] In response to the illusion risk index exceeding the forensic integrity threshold, the fragment fidelity compression mode is executed to perform lossless or near-lossless compression on the trajectory fragments, and the compressed trajectory fragments and data discontinuity warning flags are output.

[0078] This invention provides a real-time trajectory data compression optimization method for high-precision positioning. The method addresses the problem that traditional AI compression models, such as VAE, may produce data illusions and pollute historical databases when high-frequency positioning data encounters network storms leading to packet loss and out-of-order delivery. This method dynamically selects compression strategies by real-time evaluation of data quality and model uncertainty to achieve a balance between achieving high compression rates and ensuring the forensic integrity of the data.

[0079] Acquire high-frequency raw positioning data packet sequences ,this It is raw spatiotemporal data reported by positioning base stations or terminals such as UWB tags at high frequencies, such as 10Hz to 100Hz; in this embodiment, its basic structure can be defined as ,in It is the unique identifier of the terminal. These are spatial coordinates. It's a timestamp;

[0080] Data by terminal identifier Divided into parallel trajectory segments ;this This refers to the preset sliding time window. Within, belonging to the same terminal A set of time-series data points;

[0081] To quantify the severity of network interference in real time, this method determines trajectory segments. Packet loss rate within the preset sliding time window ;this In this embodiment, the purpose is to quantify the degree of data loss, specifically by comparing the expected number of timestamps within a sliding time window with the actual number received; for example, if the window... The expected number of data packets was 100, but only 90 were actually received. ;

[0082] Meanwhile, this method determines trajectory segments. Disorder rate within the sliding time window ;this In this embodiment, the purpose is to quantify the degree of disorder in the data time sequence, specifically by calculating the number of inverted pairs of timestamps that are not monotonically increasing within the sliding time window;

[0083] Based on the two metrics mentioned above, this method is based on packet loss rate. With disorder rate The data quality index was calculated. ;this This is a dimensionless parameter introduced in this invention for real-time quantization of data stream quality; in this embodiment, its calculation method is defined as follows:

[0084]

[0085] in: Its purpose is to assess the integrity of the data stream. The closer to 1, the higher the quality of the data stream; the closer to 0, the more fragmented the data stream. It is the packet loss rate as defined above; It is the out-of-order rate as defined above; and These are preset impact weights, derived from the network environment characteristics of the facility, such as higher sensitivity to latency than to packet loss, and satisfying the following conditions: ;

[0086] Using this formula, the present invention can quantify the fragmentation level of a data stream in real time. The index is a key input for subsequent assessment of the risk of hallucination in AI models;

[0087] Response to trajectory fragments Due to high packet loss rate, for example If the data set is determined to be empty, the system determines that the data is completely lost. It will skip the VAE compression, reconstruction, and risk assessment processes (corresponding to steps 5 to 11 in Example 1) and directly execute the fragmented fidelity compression mode (corresponding to step 13 in Example 1), i.e., start the fragmented fidelity compression mode and output a data discontinuity warning flag. ;

[0088] This method employs a pre-trained variational autoencoder (VAE) for trajectory segments. Semantic compression is performed; VAE is a deep learning model whose purpose in this invention is to learn deep semantic patterns of trajectories, such as inspection and stagnation, rather than simply fitting coordinate points; the VAE model is trained offline on the forensic gold standard trajectory dataset;

[0089] The VAE model contains an encoder. and decoder encoder trajectory segment Compression into semantic latent vectors And simultaneously obtain the variance of the latent distribution. ;

[0090] This semantic latent vector Refers to trajectory segments In low-dimensional space, the semantic representation represents the behavioral pattern of the trajectory rather than the original coordinates.

[0091] This latent distribution variance This refers to the VAE model's encoding of the input. Uncertainty measure; when input data When the pattern is broken and does not conform to the gold standard pattern seen by the model during training. The value will increase significantly;

[0092] decoder Based on semantic latent vectors Reconstruct the trajectory segment to obtain the reconstructed trajectory segment. ;

[0093] This method then combines the original trajectory fragments. With reconstructed trajectory fragments Calculate reconstruction error ;this The purpose is to quantify the reconstruction results of AI models. With the original input The deviation between them; in this embodiment, the reconstruction error Specifically, by calculating trajectory segments With reconstructed trajectory fragments The dynamic time-warped distance (DTW) between them is obtained, i.e. Using DTW can effectively overcome the drift and scaling of trajectory data over time, and more accurately assess semantic-level reconstruction distortion.

[0094] In order to assess risk within a unified framework, this method is normalized.

[0095] This method normalizes the variance of the latent distribution to obtain the normalized variance. Specifically, a preset benchmark variance is used. variance of the latent distribution Perform normalization;

[0096] Simultaneously, the reconstruction error is normalized to obtain the normalized reconstruction error. Specifically, a preset baseline reconstruction error is used. Reconstruction error Normalization is performed, and the calculation method is as follows: ;

[0097]

[0098] in: It is the baseline variance; These are baseline reconstruction errors; the two baseline values ​​are derived from the results obtained by running the VAE model on the gold standard dataset. and These are obtained through statistical methods, such as taking their maximum value or 95th percentile value, and are used as normalized scaling benchmarks.

[0099] Once all parameters are ready, this method is based on the data quality index. Normalized variance With normalized reconstruction error The hallucination risk index was calculated. ;

[0100] this This is the core innovation of the present invention, which aims to comprehensively assess the risk of AI models generating data illusions under current data quality conditions. The calculation method is as follows:

[0101]

[0102] in: It is a hallucination risk index; This item reflects the degree of damage to the data source; the worse the quality, the larger the value of this item. This term reflects the uncertainty of the VAE model in handling damaged data. The more difficult the model is to understand or the greater the bias caused by forced repair, the larger the value of this term will be. These are preset weights, obtained through grid search or experimental tuning on the validation set, used to balance model uncertainty. With reconstruction error The weighting in risk assessment; and These are preset risk source weights, which are obtained through optimization based on historical data and are used to balance the risk of data source damage. Model for handling uncertainty risk The proportion of total risk, and satisfying ;

[0103] risk Poor input data quality This is determined by the uncertainty that the model itself deals with; The higher the value, the more likely the AI's compression and restoration results are to be an illusion, and the higher the risk of damage to the Trajectory Forensic Integrity (TFI).

[0104] The system performs forensic data triage based on this risk index; this method uses the hallucination risk index. Compared with the preset legal integrity threshold Compare;

[0105] This forensic integrity threshold It is a dimensionless parameter whose specific value is selected by weighing the acceptable level of forensic risk and the data compression rate through offline testing based on the security level and judicial evidence requirements; its function is to define the upper limit of the system's tolerance for hallucination risk.

[0106] Responding to the Hallucination Risk Index Not greater than the legal integrity threshold Right now The system determines that the AI ​​compression result is reliable and then executes the high-compression-ratio semantic compression mode, mode one.

[0107] In this mode, the system aims for an extreme compression ratio, such as 99:1, discarding the original... Only output and store semantic latent vectors Or by Matched semantic tags For example, inspection route 3; the output data packet is ,in This is the pattern one identifier;

[0108] Responding to the Hallucination Risk Index Greater than the legal integrity threshold Right now The system determined that AI compression posed an unacceptable forensic risk and therefore executed fragment-fidelity compression mode two.

[0109] In this mode, the system actively sacrifices compression ratio to protect forensic integrity; the system discards all data generated by VAE. and To prevent hallucination data from being entered into the database; instead, the original trajectory fragments were processed. Perform standard lossless or near-lossless compression processing, such as differential pulse code modulation, to obtain... The final output is the compressed trajectory segment. Warning flag for data discontinuity The output data packet is ,in This is the identifier for mode two; Its purpose is to ensure that when conducting judicial evidence collection in the future, investigators can clearly know that the data for that period of time has been corrupted and has not been repaired by AI;

[0110] Furthermore, to ensure the forensic integrity of stored data without sacrificing the accuracy of real-time alerts, this method also includes a parallel-running alert engine; this engine is decoupled from the data storage path and dynamically selects the data source based on triage decisions.

[0111] when In Mode 1, the alarm engine is based on reconstructed trajectory fragments. Depend on To obtain real-time analysis;

[0112] when In Mode 2, the alarm engine directly bases its alerts on the original trajectory segments that lead to high risk. Conduct real-time analysis;

[0113] The system will generate a pattern-based data package from the triage decision in the previous stage. Include or mode 2 data packet Include and It is encapsulated into a unified heterogeneous data stream and written to a permanent historical database;

[0114] This embodiment constructs a hallucination risk index. And based on this index and the forensic integrity threshold Through comparison, dynamic triage of compression strategies was achieved; its technical effects are as follows:

[0115] When data quality is poor and the AI ​​model output has high uncertainty, The system proactively abandons AI compression and adopts a fidelity mode to ensure that the fragmented but authentic data entering the database is genuine, completely avoiding the contamination of historical databases by illusory data and guaranteeing the forensic integrity of the data (TFI).

[0116] High data quality, reliable AI compression, and low latency The system performs high-compression-ratio semantic compression, storing only low-dimensional vectors. This achieves an extreme compression ratio of, for example, 99:1, which meets the system's limited storage budget;

[0117] By decoupling the alarm engine from the data ingestion logic, the accuracy of real-time alarms, such as those for electronic fences, remains 100% regardless of the system's chosen storage mode, because it is always based on the most reliable data source. Or mode two To conduct an assessment.

[0118] Example 2:

[0119] Determine the packet loss rate of the trajectory segment within a preset sliding time window, including:

[0120] Compare the expected number of timestamps with the actual number received within the sliding time window;

[0121] Determine the out-of-order rate of trajectory segments within the sliding time window, including:

[0122] Calculate the number of inverted pairs of timestamps that are not monotonically increasing within the sliding time window.

[0123] This embodiment is based on the data quality index in Embodiment 1. The specific calculation methods for the input parameters packet loss rate and out-of-order rate;

[0124] As mentioned earlier, the packet loss rate of the trajectory segment within the preset sliding time window is determined. The steps specifically include: comparing the expected number of timestamps within the sliding time window with the actual number of times received; for example, a UWB system with a reporting frequency of 20Hz and a 3-second sliding time window. The expected number of timestamps within the window is 60; if the system actually only receives 48 data packets from the terminal within this window, then... ;

[0125] Accordingly, the disorder rate of trajectory segments within the sliding time window is determined. The steps specifically include: calculating the number of inverted pairs of timestamps that are not monotonically increasing within the sliding time window; for example, if the received timestamp sequence within a window is... ,in This forms a non-monotonic increasing inverted pair of timestamps; out-of-order rate This is the ratio of the number of all inverted pairs within the window to the total number of received pairs;

[0126] This embodiment provides an operable and computationally efficient implementation method by defining packet loss and out-of-order delivery with precise mathematical definitions; this lays the foundation for subsequent implementations. The computation provides stable and reliable input, enhancing... The index measures the accuracy of quantifying network storm disruptions, including data loss and data latency jitter.

[0127] Example 3:

[0128] Based on packet loss rate and out-of-order rate, a data quality index is calculated, including:

[0129] Based on preset impact weights, the packet loss rate and out-of-order rate are calculated using weighted averages.

[0130] This embodiment is based on the data quality index in Embodiment 1. The concretization of the calculation formula;

[0131] As mentioned earlier, the data quality index is calculated based on packet loss rate and out-of-order rate. The steps specifically include: based on preset influence weights and Regarding packet loss rate With disorder rate The weighted calculation is performed, and the specific calculation formula is as follows:

[0132]

[0133] Among them, the influence weight and It is performed offline based on the network characteristics of the deployment environment, and For example, in an environment where packet loss is mainly caused by signal obstruction but network latency is stable, such as some indoor factories, it is possible to set... ,like ,make More sensitive to packet loss rate; conversely, in network environments prone to jitter and out-of-order packets, such as Wi-Fi, settings can be configured... ;

[0134] This embodiment introduces calibrable weights. and This makes the data quality index The computational model is no longer rigid, but allows the system to be optimized according to the characteristics of the actual network environment; this greatly improves... The index demonstrates universality and quantification flexibility across different types of disturbances.

[0135] Example 4:

[0136] By combining trajectory segments with reconstructed trajectory segments, the reconstruction error is calculated, including:

[0137] Calculate the dynamic time-warped distance between the trajectory segment and the reconstructed trajectory segment.

[0138] This embodiment addresses the reconstruction error in Embodiment 1. The concretization of calculation methods;

[0139] As mentioned earlier, combining the original trajectory fragments With reconstructed trajectory fragments Calculate reconstruction error The steps specifically include: calculating the original trajectory segment With reconstructed trajectory fragments The dynamic time warp distance (DTW) between them;

[0140] this It is an algorithm used to measure the similarity between two time series of different lengths;

[0141] The reason for using DTW is: the original trajectory Time jumps may occur due to network packet loss, and the VAE reconstructs the trajectory. Typically, the time series is smooth; however, using simple Euclidean distance for point-by-point comparisons can lead to errors. It was mistakenly amplified; the DTW algorithm can find it. and The optimal nonlinear alignment path between them;

[0142] This embodiment improves reconstruction accuracy by using DTW instead of traditional distance metrics. Robustness of computation; this makes Able to reflect more accurately and The difference lies in the true form and semantics, not in the apparent differences caused by timescale drift or missing data; thus enabling the subsequent hallucination risk index. It is more sensitive to semantic distortion.

[0143] Example 5:

[0144] The variance of the latent distribution is normalized to obtain the normalized variance, which includes:

[0145] The variance of the potential distribution is normalized using a preset baseline variance;

[0146] The reconstruction error is normalized to obtain the normalized reconstruction error, which includes:

[0147] The reconstruction error is normalized using a preset baseline reconstruction error.

[0148] This embodiment is a specific implementation of the risk model normalization step in Embodiment 1;

[0149] The variance of the latent distribution is normalized to obtain the normalized variance. The steps include: using a preset benchmark variance variance of the latent distribution Normalization, i.e. ;

[0150] The reconstruction error is normalized to obtain the normalized reconstruction error. The steps include: using a preset benchmark to reconstruct the error. Reconstruction error Normalization, i.e. ;

[0151] Among them, the benchmark variance and benchmark reconstruction error The source is: by collecting all data by running a trained VAE model offline on the gold standard dataset, i.e., a forensically complete, high-quality dataset. and The statistical distribution is determined, and its 95th percentile or maximum value is taken as the benchmark value.

[0152] The purpose of this benchmark is to define a reasonable upper limit for the uncertainty and reconstruction error exhibited by the model when processing normal or high-quality data;

[0153] This embodiment thus provides a standardized method for unifying model uncertainty. and reconstruction error These two parameters have different physical meanings and dimensions; this makes them relevant to the hallucination risk index. The formula can be weighted Fair weighted summation ensures the effectiveness and comparability of risk assessments.

[0154] Example 6:

[0155] Perform high-compression-ratio semantic compression modes, including:

[0156] Discard track fragments;

[0157] Store semantic latent vectors, or store semantic labels that match semantic latent vectors;

[0158] The output contains a data packet containing a semantic latent vector and a pattern identifier.

[0159] This embodiment is a concretization of the execution logic of the high compression ratio semantic compression mode, i.e., mode one, in embodiment 1; this mode is... When triggered;

[0160] Perform high compression ratio semantic compression mode, specifically including:

[0161] Discard the original trajectory fragments The purpose of this step is to release the cache in order to achieve a high compression ratio.

[0162] Storing semantic latent vectors In some embodiments, if A clustering or matching algorithm can be used to map a data point to a known semantic label. For example, if the inspection route is No. 3 or the stagnation area is A, the system can store semantic labels that match the semantic latent vector. This enables further compression and semanticization;

[0163] The output contains semantic latent vectors or tag and pattern identifier data packets The data packet is, for example... It will be written into the permanent historical database;

[0164] This embodiment defines in detail the operational closed loop in a low-risk scenario; by storing only low-dimensional data... For example from 100 points compressed into one 8-dimensional vector or It achieves extreme data compression ratios, such as 99:1; this greatly saves the long-term storage budget required for high-frequency positioning data. This mode is suitable for most operating scenarios with high data quality and low risk of illusion.

[0165] Example 7:

[0166] Perform fragmented compression mode, including:

[0167] Discard the semantic latent vectors and reconstructed trajectory fragments generated by the variational autoencoder;

[0168] The trajectory segment is compressed using a standard lossless or near-lossless compression algorithm to obtain the compressed trajectory segment;

[0169] Output a data packet containing compressed track segments, a data discontinuity warning flag, and a pattern identifier.

[0170] This embodiment is a specific implementation of the fragment-fidelity compression mode, i.e., mode two execution logic, in embodiment 1; this mode is... When triggered;

[0171] Execute fragment-fidelity compression mode, specifically including:

[0172] Discard the semantic latent vector generated by the variational autoencoder and reconstructed trajectory fragments This step is the core risk avoidance measure of this invention, because: The system has determined and It is highly likely to be hallucination data and must be completely discarded to prevent hallucination data from being stored in the database and to prevent it from polluting the historical database.

[0173] Standard lossless or near-lossless compression algorithms, such as Differential Pulse Code Modulation (DPCM) and Lempel-Ziv-Welch LZW, are used to compress the original, fragmented trajectory segments. Compression is performed to obtain the compressed trajectory fragment. ;

[0174] Output data packets The data packet contains compressed trajectory fragments. Data discontinuity warning sign With pattern identifier ;

[0175] This embodiment defines in detail the fail-safe mechanism for high-risk scenarios. In this mode, the system proactively sacrifices compression ratio in exchange for 100% forensic integrity, ensuring that the data entering the database is fragmented but authentic. By forcibly discarding all AI-generated data, the risk of illusory data contaminating the database is completely eliminated. A data discontinuity warning flag is also included. The introduction of this feature provides a clear data quality warning for subsequent judicial evidence collection or data analysis, preventing investigators from basing their data on corrupted data. The wrong conclusion was drawn.

[0176] Example 8:

[0177] Pre-trained variational autoencoders include:

[0178] An encoder is used to compress trajectory fragments into semantic latent vectors and latent distribution variance;

[0179] A decoder is used to reconstruct trajectory fragments from semantic latent vectors;

[0180] The variational autoencoder was trained offline on a forensically complete trajectory dataset.

[0181] This embodiment is a concretization of the variational autoencoder (VAE) model structure used in Embodiment 1;

[0182] The VAE model includes: encoder ;this The goal is to map high-dimensional raw trajectory data to a low-dimensional semantic latent space; specifically, it is used to map raw trajectory fragments... Compression into semantic latent vectors representing its semantics and the potential distribution variance representing the uncertainty of the model ,Right now ;

[0183] The VAE model also includes: a decoder ;this The goal is to reconstruct high-dimensional trajectory data from low-dimensional semantic vectors; specifically, it is used to extract semantic latent vectors... Reconstructing trajectory fragments ,Right now ;

[0184] Among them, the variational autoencoder is trained offline on a legally complete trajectory dataset; this legally complete trajectory dataset, also known as the gold standard dataset, refers to a massive amount of trajectory data collected in an ideal network environment, without packet loss or out-of-order data, and manually verified, which can represent all typical behavior patterns such as inspection, stagnation, and following.

[0185] The reason for training on this gold standard data is to enable the VAE model to fully learn what a normal and reasonable trajectory should look like.

[0186] This embodiment, through training on the gold-standard dataset, ensures that the VAE model possesses powerful trajectory semantic understanding and reconstruction capabilities; this training enables the model to not only perform well with complete data but also with low latency. The key is to achieve efficient semantic compression, and more importantly, to make it highly adaptable to fragmented data not seen during training. Its output Uncertainty and The reconstruction error will increase significantly, thus becoming a reliable basis for the present invention to assess the risk of hallucination.

[0187] Example 9:

[0188] It also includes a parallel-running alarm engine, including:

[0189] In response to the execution of a high-compression-rate semantic compression mode, the alarm engine performs real-time analysis based on the reconstructed trajectory fragments;

[0190] In response to the execution of fragmented compression mode, the alarm engine performs real-time analysis based on trajectory fragments.

[0191] This embodiment is a parallel optimization of the real-time alarm logic in Embodiment 1; its purpose is to ensure that the accuracy and real-time performance of real-time alarm services such as electronic fences and crowd alarms are not affected while achieving the forensic integrity of stored data.

[0192] This method also includes a parallel-run alarm engine; the judgment logic of the alarm engine is decoupled from the data storage and compression logic, and its data source is dynamically selected based on the decision results of the data triage module.

[0193] In response to the execution of high compression ratio semantic compression mode one, At this point, the system determines that the AI ​​compression result is reliable; the alarm engine, based on the reconstructed trajectory fragments... Conduct real-time analysis; adopt The reason is that It is a complete trajectory repaired and smoothed by VAE. Using it for analysis, such as determining whether an electronic fence has been crossed, can avoid using the raw data. This reduces the possibility of false alarms caused by minor jitter or momentary packet loss, and improves the accuracy of alarms.

[0194] In response to the execution of fragmented fidelity compression mode 2, At this point, the system determines It's highly likely a hallucination; the alert engine must be based on the original trajectory fragments. Conduct real-time analysis; adopt The reason is that data based on hallucinations must be avoided. An error alarm is generated; in this case, the alarm engine processes it directly. In other words, the input to the VAE ensures that alarms are based on real, even fragmented, data. For example, even with fragmented data, the system can still identify the terminal. In the window Approximate location within;

[0195] This embodiment achieves optimization by decoupling the logic of data storage pursuing forensic integrity from the logic of real-time alarms pursuing real-time accuracy; regardless of the storage mode chosen by the system for forensic integrity. or Real-time alerts can always be smoothly based on the most reliable data source at present. or original The system ensures that the accuracy of real-time alarms remains at 100%, avoiding the risk of alarm failure or false alarms due to compression strategies.

[0196] Example 10:

[0197] Please see Figure 2 A real-time trajectory data compression and optimization system for high-precision positioning includes:

[0198] The data acquisition module is used to acquire high-frequency raw positioning data packet sequences and divide them into trajectory segments according to terminal identifiers;

[0199] The data quality module is used to determine the packet loss rate of trajectory segments within a preset sliding time window, determine the out-of-order rate of trajectory segments within the sliding time window, and calculate the data quality index based on the packet loss rate and the out-of-order rate.

[0200] The semantic compression module is used to compress trajectory fragments into semantic latent vectors using a pre-trained variational autoencoder and to obtain the variance of the latent distribution.

[0201] The trajectory reconstruction module is used to reconstruct trajectory segments based on semantic latent vectors, thus obtaining the reconstructed trajectory segments.

[0202] The risk calculation module is used to calculate the reconstruction error by combining trajectory segments and reconstructed trajectory segments, normalize the potential distribution variance to obtain the normalized variance, normalize the reconstruction error to obtain the normalized reconstruction error, and calculate the illusion risk index based on the data quality index, normalized variance, and normalized reconstruction error.

[0203] The data triage module compares the hallucination risk index with a preset forensic integrity threshold;

[0204] The data output module is used to execute a high-compression-rate semantic compression mode and output a semantic latent vector in response to the hallucination risk index not being greater than the forensic integrity threshold; and to execute a fragment-fidelity compression mode to perform lossless or near-lossless compression processing on the trajectory fragments in response to the hallucination risk index being greater than the forensic integrity threshold, and output the compressed trajectory fragments and a data discontinuity warning flag.

[0205] To achieve the above method, this embodiment of the invention also provides a real-time trajectory data compression and optimization system for high-precision positioning; the system can be deployed on edge computing nodes or central servers, and is applied to the real-time trajectory data compression and optimization method for high-precision positioning as described in any one of embodiments 1-9;

[0206] In this embodiment, the system includes:

[0207] The data acquisition module aims to acquire raw data streams; it is used to collect high-frequency raw positioning data packet sequences. and according to terminal identification Divided into trajectory segments ;

[0208] The data quality module aims to quantify network interference in real time; it receives trajectory segments from the data acquisition module. ,Sure Packet loss rate within the preset sliding time window ,Sure Disorder rate within the sliding time window The data quality index was calculated based on the packet loss rate and the out-of-order rate. For example, through ;

[0209] The semantic compression module aims to perform AI semantic compression; it incorporates a pre-trained variational autoencoder. It is used to receive trajectory segments from the data acquisition module in parallel with the data quality module. , trajectory fragment Compression into semantic latent vectors And obtain the variance of the latent distribution. ;

[0210] The trajectory reconstruction module aims to acquire AI-reconstructed trajectories; it has a built-in VAE decoder. Used for semantic latent vectors output by the semantic compression module. Reconstruct the trajectory segment to obtain the reconstructed trajectory segment. ;

[0211] The risk calculation module aims to calculate the core illusion risk index; it is used to aggregate original trajectory fragments. Reconstructing trajectory fragments Data quality module output and the output of the semantic compression module Combined with trajectory segments With reconstructed trajectory fragments Calculate reconstruction error For example, the normalized variance can be obtained by normalizing the variance of the latent distribution using DTW. The reconstruction error is normalized to obtain the normalized reconstruction error. Based on data quality index Normalized variance and normalized reconstruction error The hallucination risk index was calculated. For example, through ;

[0212] The data triage module aims to perform risk decisions; it is used to analyze the hallucination risk index. Compared with the preset legal integrity threshold Compare and base your judgment on the comparison results. or Activate the appropriate compression mode;

[0213] The data output module is designed to execute data output based on triage decisions; it is used in response to low-risk assessments by the data triage module. It performs a high-compression-ratio semantic compression mode and outputs a semantic latent vector. Packaged as ; and is used in response to high-risk assessments by the data triage module. Execute fragment-fidelity compression mode to compress the original trajectory fragments. Lossless or near-lossless compression processing is performed to obtain and output the compressed trajectory fragment. Warning flag for data discontinuity Packaged as ;

[0214] Preferably, the system further includes an alarm engine module, which operates in parallel with the data output module; the alarm engine module is connected to the data triage module and is used to dynamically select based on the decision results of the triage module. Mode 1 or Mode 2 serves as the data source for real-time analysis;

[0215] In this embodiment, the above-mentioned module may be a software program module, a logic circuit, or a combination thereof deployed on a server;

[0216] The system in this embodiment fully implements the methods described in embodiments 1-9 through the collaborative work of the above modules; through the dynamic triage mechanism, it achieves high compression rate storage optimization while ensuring the forensic integrity of high-frequency positioning data and eliminating data illusions, and at the same time ensuring the accuracy of real-time alarm services.

[0217] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A real-time trajectory data compression and optimization method for high-precision positioning, characterized in that, include: High-frequency raw positioning data packet sequences are collected and divided into trajectory segments according to terminal identifiers; Determine the packet loss rate of the trajectory segment within a preset sliding time window; Determine the disorder rate of trajectory segments within the sliding time window; The data quality index is calculated based on packet loss rate and out-of-order rate; A pre-trained variational autoencoder is used to compress trajectory fragments into semantic latent vectors and obtain the variance of the latent distribution. Based on semantic latent vectors, trajectory segments are reconstructed to obtain reconstructed trajectory segments; The reconstruction error is calculated by combining the trajectory segments with the reconstructed trajectory segments; The variance of the latent distribution is normalized to obtain the normalized variance; The reconstruction error is normalized to obtain the normalized reconstruction error; The illusion risk index is calculated based on the data quality index, normalized variance, and normalized reconstruction error. The hallucination risk index is compared with a preset legal integrity threshold; The forensic integrity threshold is a dimensionless parameter. Its specific value is selected by weighing the acceptable level of forensic risk and the data compression rate through offline testing, based on the security level and judicial evidence collection requirements. Its function is to define the upper limit of the system's tolerance for the risk of hallucination; In response to the fact that the hallucination risk index is no greater than the legal integrity threshold, a high compression ratio semantic compression mode is executed, and a semantic latent vector is output. In response to the illusion risk index exceeding the forensic integrity threshold, the fragment fidelity compression mode is executed to perform lossless or near-lossless compression on the trajectory fragments, and the compressed trajectory fragments and data discontinuity warning flags are output.

2. The real-time trajectory data compression and optimization method for high-precision positioning according to claim 1, characterized in that, Determine the packet loss rate of the trajectory segment within a preset sliding time window, including: Compare the expected number of timestamps within the sliding time window with the actual number received; Determine the out-of-order rate of trajectory segments within the sliding time window, including: Calculate the number of inverted pairs of timestamps that are not monotonically increasing within the sliding time window.

3. The real-time trajectory data compression and optimization method for high-precision positioning according to claim 1, characterized in that, Based on packet loss rate and out-of-order rate, a data quality index is calculated, including: Based on preset impact weights, the packet loss rate and out-of-order rate are calculated using weighted averages.

4. The real-time trajectory data compression and optimization method for high-precision positioning according to claim 1, characterized in that, By combining trajectory segments with reconstructed trajectory segments, the reconstruction error is calculated, including: Calculate the dynamic time-warped distance between the trajectory segment and the reconstructed trajectory segment.

5. The real-time trajectory data compression and optimization method for high-precision positioning according to claim 1, characterized in that, The variance of the latent distribution is normalized to obtain the normalized variance, which includes: The variance of the potential distribution is normalized using a preset baseline variance; The reconstruction error is normalized to obtain the normalized reconstruction error, which includes: The reconstruction error is normalized using a preset baseline reconstruction error.

6. The real-time trajectory data compression and optimization method for high-precision positioning according to claim 1, characterized in that, Perform high compression ratio semantic compression modes, including: Discard the trajectory fragment; Store semantic latent vectors, or store semantic labels that match semantic latent vectors; The output contains a data packet containing a semantic latent vector and a pattern identifier.

7. The real-time trajectory data compression and optimization method for high-precision positioning according to claim 1, characterized in that, Perform fragmented compression mode, including: Discard the semantic latent vectors and reconstructed trajectory fragments generated by the variational autoencoder; The trajectory segment is compressed using a standard lossless or near-lossless compression algorithm to obtain the compressed trajectory segment. Output a data packet containing compressed track segments, a data discontinuity warning flag, and a pattern identifier.

8. The real-time trajectory data compression and optimization method for high-precision positioning according to claim 1, characterized in that, Pre-trained variational autoencoders include: An encoder is used to compress trajectory fragments into semantic latent vectors and latent distribution variance; Decoder used to reconstruct trajectory fragments from semantic latent vectors; The variational autoencoder was trained offline on a forensically complete trajectory dataset.

9. The real-time trajectory data compression and optimization method for high-precision positioning according to claim 1, characterized in that, It also includes a parallel-running alarm engine, including: In response to the execution of a high-compression-rate semantic compression mode, the alarm engine performs real-time analysis based on the reconstructed trajectory fragments; In response to the execution of fragmented compression mode, the alarm engine performs real-time analysis based on trajectory fragments.

10. A real-time trajectory data compression and optimization system for high-precision positioning, applied to the real-time trajectory data compression and optimization method for high-precision positioning as described in any one of claims 1-9, characterized in that, include: The data acquisition module is used to acquire high-frequency raw positioning data packet sequences and divide them into trajectory segments according to terminal identifiers; The data quality module is used to determine the packet loss rate of trajectory segments within a preset sliding time window, determine the out-of-order rate of trajectory segments within the sliding time window, and calculate the data quality index based on the packet loss rate and the out-of-order rate. The semantic compression module is used to compress trajectory fragments into semantic latent vectors using a pre-trained variational autoencoder and to obtain the variance of the latent distribution. The trajectory reconstruction module is used to reconstruct trajectory segments based on semantic latent vectors, thus obtaining the reconstructed trajectory segments. The risk calculation module is used to calculate the reconstruction error by combining the trajectory segment and the reconstructed trajectory segment, normalize the potential distribution variance to obtain the normalized variance, normalize the reconstruction error to obtain the normalized reconstruction error, and calculate the illusion risk index based on the data quality index, normalized variance and normalized reconstruction error. The data triage module is used to compare the hallucination risk index with a preset forensic integrity threshold; The data output module is used to perform a high-compression-ratio semantic compression mode and output a semantic latent vector in response to the illusion risk index not being greater than the legal integrity threshold. It is used to respond to the illusion risk index being greater than the forensic integrity threshold, execute the fragment fidelity compression mode, perform lossless or near-lossless compression processing on the trajectory fragment, and output the compressed trajectory fragment and data discontinuity warning flag.

Images