A cleaning and anomaly detection method for commercial vehicle trajectory data

Abstract

The application provides a kind of commercial vehicle trajectory data cleaning and anomaly detection method, comprising the following steps: S1: import data, and partitioning;S2: clean trajectory point;S3: historical behavior modeling;S4: signal loss completion;S5: abnormal processing;S6: output cleaning result;The application is adapted to different sampling periods, significantly reduces the computing overhead, supports real-time cleaning of large-scale commercial vehicle trajectory data, avoids the misjudgment of reverse driving as normal route due to ignoring driving direction, or the misjudgment of detour as consistent route due to ignoring sequence. Missing duration hierarchical processing avoids the creation of false trajectories caused by blind completion, ensures the rationality and interpretability of completion, and multi-dimensional data output provides quantifiable and comparable data quality basis.

Description

Technical Field

[0001] This invention relates to the fields of vehicle network data processing and intelligent transportation technology, specifically to a method for cleaning and anomaly detection of commercial vehicle trajectory data. Background Technology

[0002] Commercial vehicle onboard terminals report vehicle identification, timestamps, latitude and longitude, speed, and vehicle status at set intervals. Terminals from different manufacturers or models may differ in field names, coordinate precision, time units, and sampling periods, leading to null values, zero coordinates, duplicate points, and time sequence errors after direct aggregation. Satellite signal obstruction, multipath effects in tunnels or elevated environments, communication interruptions, and terminal malfunctions can cause geographical boundary crossings, instantaneous jumps, abnormally high speeds, and long-term missing segments. If these records are directly used for trip identification, frequently used route analysis, or scheduling, the calculation results for mileage, origin-end point, and route similarity will be biased. Existing trajectory cleaning methods often use fixed latitude and longitude ranges or fixed speed thresholds for point-by-point filtering, which is difficult to adapt to differences in vehicle type, time period, and sampling period. Performing polygon calculations point-by-point with all administrative boundaries increases the computational load with the number of boundaries; using the same interpolation method for missing segments of different lengths may also generate trajectories that cross administrative boundaries, exceed speed limits, or deviate from frequently used routes. Summary of the Invention

[0003] The purpose of this invention is to address the shortcomings of existing technologies by proposing a method for cleaning and detecting anomalies in commercial vehicle trajectory data.

[0004] To achieve the above objectives, the present invention adopts the following technical solution: A method for cleaning and anomaly detection of commercial vehicle trajectory data includes the following steps: S1: Import data and partition it; The system periodically collects raw records of trajectories, receives the collected raw records, processes the raw records using distributed computing, partitions them according to vehicle identification and date, sorts them by timestamp within each partition, and appends the last valid trajectory point of the previous partition as a boundary context to the current partition; it also distributes the administrative region spatial index and rule parameters as read-only data to each computing partition. S2: Cleaning trajectory points; For each trajectory point, the coordinate validity, time sequence, geographical boundary, duplicate points, dynamic speed and dynamic distance are checked sequentially. If a check fails, it is marked as abnormal and not written into the cleaned trajectory. The dynamic distance threshold is calculated based on the dynamic speed threshold, the time difference between adjacent points and the positioning error compensation amount. S3: Historical Behavior Modeling; Based on historical valid trips, a grid coverage set and an ordered grid transfer sequence are established. The coverage similarity and sequence similarity of the trajectory window to be detected are calculated respectively, and the overall consistency is obtained by combining the speed conformity and the dwell conformity. When the overall consistency is lower than a set threshold, the trajectory window is marked as a behavior deviation. S4: Signal loss compensation; Based on the duration of the missing segment, the similarity between the endpoint context and historical representative routes, endpoint constraint interpolation, representative route constraint fitting, or missing labeling are performed on the missing segment. The generated candidate completion points are verified point-by-point by geographical boundaries, dynamic speed, and dynamic distance, and the completion method and completion confidence are recorded to obtain the completion record. S5: Exception handling; Abnormal trajectory points are corrected and marked, anomaly reports are generated, and anomaly details are saved. Continuous anomaly records are manually reviewed and confirmed. S6: Output cleaning results; The cleaning results include the cleaning trajectory, anomaly details, supplementary records, and quality indicators.

[0005] Furthermore, S1 includes: S11: Enter the original record; Each original record includes at least the record identifier (record_id), the original data source order number (source_order), the vehicle identifier (terminal_id), the GPS timestamp (gpstimestamp), the latitude, the longitude, the speed, and the status; if a trip number exists, it also includes the trip number (trip_id), and records the data source type (source_type) and the data structure version number (schema_version). S12: Partition processing; The system is partitioned according to the vehicle identifier terminal_id and the date. Within each partition, the data is sorted by the GPS timestamp gpstimestamp. Each partition reads the last valid point of the previous partition and the first valid point of the next partition as the boundary context. After detection, overlapping records are removed according to the record identifier record_id. S13: Standardize the processing of raw records; Time is converted to seconds and then to the same time zone; latitude and longitude are converted to degrees according to coordinate_scale and then to a unified coordinate system. After standardization, trajectory points are obtained. S14: Mapping and reading the spatial index and rule parameters of the administrative region; The rule parameters are obtained by reading the field mapping based on the data structure version number schema_version.

[0006] Furthermore, step S2 includes: S21: Detect the validity of the coordinates of the trajectory points; Check if the coordinates are empty or zero. If so, mark them as NULL_COORD or ZERO_COORD respectively. Do not write the marked trajectory points into the cleaning trajectory; retain the original records. S22: Filter geographic boundaries; S2 polygons are constructed based on the boundary coordinates of administrative regions, and a set of covering units is generated from the S2 polygons. Candidate administrative regions are selected from the set of covering units based on the S2 polygons corresponding to the trajectory points. Then, the S2 polygons of the candidate administrative regions are used to perform point inclusion judgment. Trajectory points that do not fall into the target administrative region are marked as OUT_OF_BOUND and enter the isolation area, and are not written into the cleaned trajectory. The original records are retained. S23: Detect dynamic speed and dynamic distance; First, check the timestamps of the same vehicle according to the original sequence number source_order of the data source. When the time of the trajectory points is in reverse order, mark TIME_REVERSE. Then sort them according to the GPS timestamp gpstimestamp and calculate the difference between adjacent trajectory points. If the difference is 0 and the coordinates are different, mark TIME_CONFLICT to indicate a time conflict. If the difference is greater than 0, calculate the distance and speed. Set detection boundaries: Set the upper limit of the vehicle's physical speed during the cold start phase; Using samples that have passed hard rule verification or been manually confirmed to be normal, after the number of normal samples reaches n_min, calculate the median speed M and median absolute deviation MAD; take Tv=min(Vphy, M+k×1.4826×max(MAD,δ)), where Vphy is the set upper limit of vehicle physical speed, k is 3 to 5, δ is the minimum speed dispersion, which is a preset value, and the upper limit of dynamic distance Td=Tv×Δt / 3.6+ε, where ε is the positioning error compensation amount; if n_min is not reached, the upper limit of vehicle physical speed is used as the upper limit of dynamic speed; If the detection speed exceeds the dynamic speed limit or the distance exceeds the dynamic distance limit, it is marked as an anomaly and removed from the cleaning trajectory or corrected. The original record is not deleted. S24: Detect duplicate points and transition points; When the same vehicle at the same time point has the same coordinates, it is marked as DUPLICATE_POINT; when the coordinates at the same time point are different, it is marked as TIME_CONFLICT. If the distance between the current point and its adjacent points exceeds their respective dynamic distance thresholds, but the distance between the two points does not exceed the dynamic distance threshold for the corresponding time span, then it is marked as DISTANCE_JUMP.

[0007] Furthermore, step S3 includes: S31: Represents the historical valid trip distance; For the same model version, a uniform GeoHash precision L is used to divide the geospace into grids of equal size. Each historical valid trip of the historical model is represented in two ways: one is the grid set G, and the other is the adjacent grid transfer sequence T. S32: Construct the trajectory window; Taking the trajectory window W as the object, W consists of m consecutive valid points or valid points within a fixed time period; S33: Calculate the similarity of routes; For the current trajectory window W to be detected, the same GeoHash precision L as the historical model is first used to generate the grid set G(W) and the adjacent grid transition sequence T(W) for the window. The coverage similarity is calculated using Jaccard similarity, and the order similarity is obtained by dividing the length of the longest common subsequence of T(W) and T(R) by the larger value of their lengths. The route similarity is calculated by weighted summation of the coverage similarity and the order similarity. S34: Obtain speed compliance; The median speed of all valid points within the current window W is set as M(W). The median speed of the corresponding vehicle is obtained as M, the median absolute deviation MAD, and the minimum speed dispersion δ. First, the standardized deviation of the window speed median relative to the historical distribution is calculated. Then, the speed conformity is obtained through the Gaussian kernel function. The speed conformity range is (0,1]. S35: Obtain the degree of compliance with the stay requirement; Preset dwell speed threshold v_stop and dwell duration threshold T_stop; define dwell points as those with speed not higher than v_stop and duration not shorter than T_stop; for all dwell points in the historical model, count the dwell frequency of each GeoHash grid and normalize it to the [0,1] interval to form the historical dwell grid frequency distribution; for the current window W, if there are no dwell points in the window, the dwell compliance Ss is 1; if there are dwell points in the window, take the average of the historical dwell frequencies of the grids where each dwell point is located in the window as the dwell compliance Ss of the window. S36: Overall consistency determination; The route similarity Sr, speed compliance Sv, and dwell compliance Ss are weighted and summed to obtain the comprehensive consistency index C of the trajectory window. When C < 0.7, the window W is marked as BEHAVIOR_DEVIATION; and this mark is written to each trajectory point within the window. S37: Continuously manage and update model versions; The GeoHash precision L, window size m, weight coefficients, dwell speed threshold v_stop, dwell duration threshold T_stop, and minimum speed discrepancy δ are all saved along with the model version number; only trajectory point samples that pass through hard rules or are manually confirmed to be normal are used.

[0008] Furthermore, step S4 includes: S41: Determine the duration of the missing information; Calculate the missing duration by obtaining the GPS timestamps of the valid trajectory points at both ends of the missing segment; S42: Hierarchical processing of missing segments; Preset a first threshold and a second threshold; When the missing duration does not exceed the first duration threshold, endpoint constraint interpolation is performed; When the missing duration exceeds the first duration threshold but does not exceed the second duration threshold, perform representative route constraint fitting; When the missing duration exceeds the second duration threshold, a missing marker is executed. Only when the similarity Sr between the endpoint context windows at both ends of the missing segment and the route of the same historical representative route is not less than 0.75, and the order of the grids containing the valid trajectory points at both ends on the route is consistent with the driving direction, sparse candidate completion points are generated at the key transfer grid.

[0009] Furthermore, step S5 includes: S51: Correct the exception and mark it; Each trajectory point record is written with one or more exception types and its handling status is given; S52: Generate an exception report; The anomaly reports are aggregated by date, vehicle, and anomaly type, and record the number of inputs, the number of anomalies, the anomaly rate, the time range of occurrence, the actual value range, the threshold range, and the number of each handling status. S53: Save exception details; The anomaly details include record identifier, vehicle identifier, timestamp, original coordinates, corrected coordinates, actual value, threshold, rule version, model version, handling status, and identifier of manual review records; S54: Manual review; Add the items to the review list based on their status and write them into the manual review form. Manually review and record the results. The manual review form includes the final judgment circle, corrected latitude, corrected longitude, reason for review, and review time.

[0010] Furthermore, exception types include exception flags for NULL_COORD, ZERO_COORD, OUT_OF_BOUND, DUPLICATE_POINT, TIME_REVERSE, TIME_CONFLICT, SPEED_EXCEED, DISTANCE_JUMP, LONG_GAP, and BEHAVIOR_DEVIATION. Normal and valid trajectory points are set to KEEP status. For invalid coordinates marked NULL_COORD or ZERO_COORD, trajectory points with time conflicts (TIME_CONFLICT) or geographical out-of-bounds (OUT_OF_BOUND) are set to EXCLUDE status. Duplicate points marked DUPLICATE_POINT are retained only once. Completed points with isolated jump points interpolated by short missing segments are set to CORRECT status. Completed points with low confidence due to continuous jumps, behavioral deviations, or medium or long missing segments are set to REVIEW status.

[0011] Furthermore, step S6 includes: S61: Calculate quality indicators; The required number of points, Nexp, should be obtained through calculation. First, calculate the original integrity Qc = min(1, Nobs / Nexp); Nobs is the number of points that have not been completed and have passed the coordinate validity check; Then, the validity Qv is calculated as Qv = Nrule_valid / Ninput, where Ninput is the number of inputs and Nrule_valid is the number of points that pass all hard rule validations. Next, the continuity is calculated as Qt = max(0, 1 - Ngap / max(1, Nadj)), where Ngap is the number of missing segments and Nadj is the total number of adjacent pairs. Calculate behavioral consistency Qs = max(0, 1 - Ndeviation_window / max(1, Nwindow)); where Ndeviation_window is the number of windows marked as behavioral deviations, and Nwindow is the total number of windows; Finally, a weighted composite score is calculated to obtain the quality index; S62: Output result; Write the cleaning trajectory, anomaly details, and quality indicators into the partition results table.

[0012] Furthermore, the change ratio of Tv and Td after each update relative to the previous one is limited to no more than ρ, where 0 < ρ < 1.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) The present invention does not use a fixed speed upper limit or a fixed distance threshold for point-by-point judgment, but dynamically calculates the speed threshold and distance threshold based on the actual time difference Δt between adjacent trajectory points. The dynamic threshold is automatically generated to adapt to vehicle type, running time and sampling period. The speed threshold is adjusted synchronously with the sampling period, avoiding the common problem that high-frequency sampling points are misjudged as stationary due to the small distance between adjacent points and low-frequency sampling points are misjudged as jumping due to the large distance between adjacent points. This allows trajectory data of different terminal models and different reporting frequencies to obtain a consistent judgment standard in the same cleaning process, without the need to configure parameters separately for each sampling period.

[0014] (2) This invention pre-constructs an S2 polygon and its covering unit set based on the administrative region boundary coordinates. The trajectory points are first quickly filtered out by the S2 units to select a small number of candidate administrative regions, and then the original polygon points are precisely included only in the candidate administrative regions. The covering units are only used to narrow down the candidate range and are not directly used as the final boundary determination result. This ensures the accuracy of the geographic boundary filtering while reducing the number of boundary operations for each trajectory point from the total number of administrative regions to a constant number of candidate regions, which significantly reduces the computational overhead and supports real-time cleaning of large-scale commercial vehicle trajectory data.

[0015] (3) This invention simultaneously stores the grid coverage set G and the temporally arranged adjacent grid transfer sequences T under the same GeoHash precision. Coverage similarity Sg measures the degree of spatial overlap, and order similarity So compares the transfer direction and order through the longest common subsequence. The two are weighted to obtain route similarity Sr. Compared with route matching methods that only compare spatial coverage, this invention can distinguish between trajectories that "pass through the same location but in opposite directions" and trajectories that "pass through the same location but in disordered order," avoiding behavioral deviation detection errors such as misjudging reverse travel as normal routes due to ignoring the travel direction or misjudging detours as consistent routes due to ignoring the order.

[0016] (4) This invention implements hierarchical processing based on the duration of missing segments and the degree of matching between endpoint context and historical route evidence. For short missing segments (not exceeding 10 minutes), endpoint-constrained spherical interpolation is used to maintain trajectory continuity; for medium missing segments (10 to 30 minutes), representative routes with consistent directions are selected from the historical route cluster to generate constrained paths, which are then smoothed using Kalman filtering; for long missing segments (more than 30 minutes), continuous trajectories are not generated by default, and sparse candidate points are only output when there is high-confidence historical route evidence, without calculating continuous mileage within the completed segment. The hierarchical strategy effectively avoids the blind interpolation generation of pseudo-trajectories that cross administrative boundaries, exceed speed limits, or deviate from the usual running route due to the lack of route evidence in long missing segments, while ensuring the rationality of short missing segment completion and the interpretability of medium missing segment completion.

[0017] (5) The present invention provides multi-dimensional data output, in which the cleaning trajectory, anomaly details and quality indicators are written into the partition result table. Low confidence completion points are marked separately and their use is determined by the downstream module. The multi-dimensional indicators quantify and evaluate the cleaning effect from four independent dimensions: data collection, rule verification, trajectory continuity and behavior rationality, providing quantifiable and comparable data quality basis for downstream applications such as trip identification, route analysis and scheduling. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the steps of a method for cleaning and detecting anomalies in commercial vehicle trajectory data. Detailed Implementation

[0019] To provide a further understanding of the purpose, structure, features, and functions of the present invention, detailed descriptions are provided below with reference to specific embodiments.

[0020] Please refer to the reference. Figure 1 This invention provides a method for cleaning and anomaly detection of commercial vehicle trajectory data, comprising the following steps: S1: Import data and partition it; The system periodically collects raw records of trajectories, receives the collected raw records, processes the raw records using distributed computing, partitions them according to vehicle identification and date, sorts them by timestamp within each partition, and appends the last valid trajectory point of the previous partition as a boundary context to the current partition; it also distributes the administrative region spatial index and rule parameters as read-only data to each computing partition. S11: Enter the original record; Each original record includes at least the record identifier (record_id), the original data source order number (source_order), the vehicle identifier (terminal_id), the GPS timestamp (gpstimestamp), the latitude, the longitude, the speed, and the status (including ignition, shutdown, driving, stopped, etc.). If a trip number exists, it also includes the trip number (trip_id), and records the data source type (source_type) (including OBD, T-Box, APP, etc.) and the data structure version number (schema_version). The original data source order number (source_order) indicates the original order of the records in the data source and is used to identify time reversal.

[0021] The source_type data source is used to distinguish the source of data collection. The accuracy and frequency of data from different sources vary.

[0022] S12: Partition processing; The system partitions the trajectory based on vehicle identifier (terminal_id) and date, and sorts the data within each partition by GPS timestamp (gpstimestamp). Each partition simultaneously reads the last valid point from the previous partition and the first valid point from the next partition as boundary context. After detection, overlapping records are removed based on record identifier (record_id). After vehicle-based trajectory partitioning, each partition independently calculates the features of adjacent points; partition boundaries carry adjacent valid points to avoid missing the first and last trajectory points.

[0023] S13: Standardize the processing of raw records; Time is converted to seconds and then to the same time zone; latitude and longitude are converted to degrees according to coordinate_scale and then to a unified coordinate system. After standardization, trajectory points are obtained.

[0024] S14: Mapping and reading the spatial index and rule parameters of the administrative region; Based on the data structure version number schema_version, read the field mapping to obtain the administrative region spatial index and rule parameters; construct a mapping table between the data structure version number schema_version and the administrative region spatial index and rule parameters, and read the table.

[0025] S2: Cleaning trajectory points; For each trajectory point, the following checks are performed sequentially: coordinate validity, time sequence, geographic boundary, duplicate points, dynamic speed, and dynamic distance. The dynamic distance threshold is calculated based on the dynamic speed threshold, the time difference between adjacent points, and the positioning error compensation. S21: Detect the validity of the coordinates of the trajectory points.

[0026] Check if the coordinates are empty or zero. If so, mark them as NULL_COORD or ZERO_COORD respectively. Do not write the marked trajectory points into the cleaning trajectory; retain the original records.

[0027] If the coordinates of a trajectory point are empty, then the trajectory point is marked as NULL_COORD; if the coordinates of a trajectory point are zero (0,0), then the trajectory point is marked as ZERO_COORD.

[0028] S22: Filter geographic boundaries; An S2 polygon is constructed based on the boundary coordinates of the administrative region, and a set of covering units is generated from the S2 polygon. Candidate administrative regions are selected from the set of covering units based on the S2 polygons corresponding to the trajectory points. Then, a point inclusion judgment is performed using the S2 polygons of the candidate administrative regions. Trajectory points that do not fall into the target administrative region are marked as OUT_OF_BOUND (outside the boundary) and placed in an isolation zone, not written into the cleaned trajectory; the original record is retained. The covering unit set is only used to narrow down the candidate range and is not directly used as the final boundary determination result. Spatial indexing first narrows down the candidate boundary range, and then performs precise inclusion judgment, reducing the number of regions involved in the boundary calculation for each trajectory point.

[0029] S2 is an open-source geographic indexing algorithm. S2 polygons are made up of multiple S2 grid cells to form a polygon representing an administrative region, ensuring extremely fast calculation (integer bitwise operations) and facilitating subsequent spatial indexing.

[0030] The isolation zone is a dedicated independent storage space for "non-compliant data." This data is fully preserved but does not participate in any business calculations, so that it can be used for subsequent investigation or manual recovery.

[0031] S23: Detect dynamic speed and dynamic distance; First, the trajectory points are checked for timestamps of the same vehicle according to the original sequence number source_order of the data source. If the current timestamp is less than the previous record, it is marked TIME_REVERSE to indicate that the time is reversed. Then, they are sorted according to the GPS timestamp gpstimestamp. For adjacent trajectory points p(i-1) and p(i), Δt=t(i)-t(i-1) is calculated. If Δt=0 and the coordinates are different, TIME_CONFLICT is marked to indicate a time conflict. If Δt>0, the distance d(i) is calculated using the Haversine formula or the S2 spherical arc length, and the speed v(i)=3.6×d(i) / Δt is calculated.

[0032] The Haversine formula is based on spherical trigonometry and uses latitude and longitude to calculate the distance between great circles; the S2 spherical arc length is a spherical distance calculation method built into the Google S2 library. It treats the Earth as a sphere and directly calculates the arc length between two points on the sphere using the Angle or Distance functions in the S2 geometry library.

[0033] Setting detection boundaries: During the cold start phase, the upper limit of the vehicle's physical speed is set to Tv=150km / h, and the upper limit of the dynamic distance is Td=Tv×Δt / 3.6+ε, where ε is the positioning error compensation amount.

[0034] Using samples that have passed hard rule verification or been manually confirmed to be normal within the last 30 days, after the number of normal samples in the rolling window reaches n_min, calculate the median velocity M and the median absolute deviation MAD; the median velocity M refers to the velocity value in the middle when the velocities of the normal samples in the rolling window are arranged in ascending order. The median absolute deviation (MAD) is calculated by first calculating the absolute deviation of each velocity value from the median velocity (M), then arranging the absolute deviations in ascending order, and taking the absolute deviation in the middle to obtain the median absolute deviation (MAD).

[0035] Based on the obtained median speed M and median absolute deviation MAD, Tv is set to min(Vphy, M + k × 1.4826 × max(MAD, δ)), where Vphy is the upper limit of the vehicle's physical speed, i.e., 150 km / h, k is 3 to 5, and δ is the minimum speed dispersion, which is a preset value within the range of 1-5 km / h. The upper limit of dynamic distance Td is Tv × Δt / 3.6 + ε. The change ratio of Tv and Td after each update relative to the previous one does not exceed ρ, 0 < ρ < 1; when the sample size is insufficient, the upper limit of the vehicle's physical speed is used.

[0036] When v(i)>Tv or d(i)>Td, the corresponding anomaly (velocity anomaly / distance anomaly) is marked, and the trajectory points marked as anomalies are excluded from or corrected in the cleaned trajectory. The original records are not deleted.

[0037] S24: Detect duplicate points and transition points; When the same vehicle at the same time point has the same coordinates, it is marked as DUPLICATE_POINT (duplicate point); when the coordinates at the same time point are different, it is marked as TIME_CONFLICT (time conflict).

[0038] For the current point p(i), compare the distances between the current point p(i) and its two adjacent points p(i-1) and p(i+1). If the distances from p(i-1) to p(i) and from p(i) to p(i+1) both exceed their respective dynamic distance thresholds, while the distance from p(i-1) to p(i+1) does not exceed the dynamic distance threshold for the corresponding time span, then mark the current point p(i) as a DISTANCE_JUMP (jump point); no additional fixed 10km jump threshold is set. Trajectory points marked as abnormal are excluded from the cleaned trajectory or corrected by supplementation, while the original records are retained.

[0039] S3: Historical Behavior Modeling; Based on historical valid trips, a grid coverage set and an ordered grid transfer sequence are established. The coverage similarity and sequence similarity of the trajectory window to be detected are calculated respectively, and the overall consistency is obtained by combining the speed conformity and the dwell conformity. When the overall consistency is lower than a set threshold, the trajectory window is marked as a behavior deviation. Within the same model version, using the same GeoHash precision, historical valid trips are represented as a GeoHash grid set and a sequence of adjacent grid transfers arranged by time. Coverage similarity is obtained based on the Jaccard similarity of the grid set, and order similarity is obtained based on the longest common subsequence of the grid transfer sequences. The coverage similarity and order similarity are weighted to obtain route similarity, and weighted with speed compliance and dwell compliance to obtain the overall consistency of the trajectory window. The historical behavior model is updated only using trajectory windows that have passed hard rule verification or have been manually confirmed to be normal.

[0040] S31: Represents the historical valid trip distance; For the same model version, a uniform GeoHash precision L is used to divide the geospatial area into grids of equal size.

[0041] Each historical valid trip in the historical model is represented in two ways: First, all trajectory points within the trip are mapped to corresponding GeoHash grids and deduplicated to form a grid set G, used to calculate route coverage. Second, the trajectory points within the trip are arranged in chronological order, and the transfer relationships between adjacent trajectory points and their corresponding grids are recorded sequentially, forming an adjacent grid transfer sequence T, used to calculate route direction and order. The number of times each GeoHash grid is covered by different trips is counted, and spatially adjacent grids with high coverage frequencies are merged into route clusters. Within each route cluster, a representative transfer sequence is generated according to the frequency of transfers between grids, forming an adjacent grid transfer sequence T. The GeoHash accuracy L, grid set G, and adjacent grid transfer sequence T are saved as historical routes with each model version, ensuring model traceability and rollback capability.

[0042] Historical models are normal trajectory points that have been manually confirmed or established through hard rules of coordinates, time, speed, and distance.

[0043] S32: Construct the trajectory window; Historical behavior detection uses a trajectory window W as its object, which consists of m consecutive valid points or valid points within a fixed time period. Whether a single point within the window is excluded is still determined by hard rules based on coordinates, time, speed, and distance.

[0044] S33: Calculate the similarity of routes; For the current trajectory window W to be detected, the same GeoHash precision L as the historical model is first used to generate the grid set G(W) and the adjacent grid transition sequence T(W) for the window.

[0045] The model version reads the historical route R of the corresponding vehicle, which includes the saved grid set G(R) under the same GeoHash precision L and the adjacent grid transfer sequence T(R); Coverage similarity is calculated using Jaccard similarity: Sg = |G(W)∩G(R)| / |G(W)∪G(R)|; where the numerator represents the number of grids traversed by the window and the historical route, and the denominator represents the total number of grids traversed by both.

[0046] The order similarity So is the length of the longest common subsequence of T(W) and T(R) divided by the larger of their lengths; that is: So=LCS(T(W),T(R)) / max(|T(W)|,|T(R)|), where LCS(T(W),T(R)) represents the length of the longest common subsequence of the adjacent grid transfer sequence of the trajectory window and the adjacent grid transfer sequence of the historical route, and the denominator is the larger of the two sequence lengths.

[0047] The route similarity is obtained by weighted summation of coverage similarity and order similarity. The route similarity Sr is calculated according to Sr=αSg+(1-α)So, where α is the weight coefficient of coverage similarity, which is a hyperparameter that is preset in the model version.

[0048] S34: Obtain speed compliance; The median speed of all valid points within the current window W is set as M(W). The median speed of the corresponding vehicle is obtained as M, the median absolute deviation MAD, and the minimum speed dispersion δ. First, calculate the standardized deviation of the median window velocity relative to the historical distribution: zv=|M(W)-M| / (1.4826×max(MAD,δ)); where 1.4826 is a constant factor for converting MAD to the standard deviation scale.

[0049] Then, the speed conformity is obtained according to the speed conformity Sv=exp(-zv² / 2); where exp() is the Gaussian kernel function; the speed conformity range is (0,1], and when the window speed is consistent with the historical speed distribution, Sv approaches 1; the larger the deviation, the closer Sv is to 0.

[0050] S35: Obtain the degree of compliance with the stay requirement; Trajectory points with a speed not exceeding v_stop and a duration not less than T_stop are defined as dwell points. Here, v_stop is the dwell speed threshold, and T_stop is the dwell duration threshold. For all dwell points in the historical model, the dwell frequency of each GeoHash grid is counted and normalized to the [0,1] interval to form a historical dwell grid frequency distribution. For the current window W, if there are no dwell points within the window, the dwell compliance Ss is set to 1, indicating no dwell behavior and no conflict with history. If there are dwell points within the window, the average historical dwell frequency of each dwell point's grid within the window is taken as the dwell compliance Ss of that window.

[0051] S36: Overall consistency determination; The route similarity Sr, speed compliance Sv, and dwell compliance Ss are weighted and summed to obtain the comprehensive consistency index C of the trajectory window: C = wr × Sr + wv × Sv + ws × Ss, where wr, wv, and ws are the weight coefficients of route similarity Sr, speed compliance Sv, and dwell compliance Ss, respectively, and wr + wv + ws = 1; these are preset and stored in the model version. When C < 0.7, the window W is marked as BEHAVIOR_DEVIATION; and this mark is added to the anomaly code list of each trajectory point within the window. Whether a single point within the window is excluded from the cleaned trajectory is still determined independently by the detection results of hard rules for coordinates, time, speed, and distance, and is not directly removed due to window marking.

[0052] S37: Continuously manage and update model versions; The GeoHash precision L, window size m (number of points or duration), weight coefficients α, wr, wv, ws, dwell speed threshold v_stop, dwell duration threshold T_stop, and minimum speed discreteness δ are all saved along with the model version number to ensure that the processing logic between different versions is traceable.

[0053] When updating the model, only trajectory point samples that pass through hard rules or are manually confirmed to be normal are used. Low-confidence completion points are not included in the update of the historical model to prevent erroneous data from polluting the historical behavior benchmark.

[0054] S4: Signal loss compensation; Based on the missing duration, endpoint context, and matching results with historical representative routes, endpoint constraint interpolation, representative route constraint fitting, or missing labeling are performed on the missing segments; the generated candidate completion points are verified point by point through geographical boundaries, dynamic speed, and dynamic distance, and the completion method and completion confidence are recorded; S41: Determine the duration of the missing information; Let p(a) and p(b) be the valid trajectory points at the two ends of the missing segment, where p(a) is the last valid point before the missing segment and p(b) is the first valid point after the missing segment. The missing duration Δt_miss of the missing segment is calculated by subtracting the reference sampling period T0 from the difference between the GPS timestamps of p(a) and p(b).

[0055] S42: Hierarchical processing of missing segments; The preset first duration threshold is 10 minutes, and the second duration threshold is 30 minutes.

[0056] When the missing duration does not exceed the first duration threshold, endpoint constraint interpolation is performed: candidate times are generated according to the baseline sampling period T0, and linear interpolation is performed on p(a) and p(b) on the sphere to obtain the candidate latitude and longitude coordinates of each candidate time; boundary, dynamic velocity, and dynamic distance checks are performed on each candidate point. Only when all checks pass is the candidate point completed and written into the cleaning trajectory. Otherwise, it is not written into the cleaning trajectory and is recorded as a high confidence level (confidence level that can be interpolated by preset constraints).

[0057] When the missing duration exceeds the first duration threshold but does not exceed the second duration threshold, representative route constraint fitting is performed. Valid points before p(a) and after p(b) are selected to form endpoint context windows. Representative routes with the highest route similarity, containing both endpoint grids and having the same direction, are selected from the historical route cluster. The similarity is recorded as the confidence level of the completion point. Candidate paths are generated according to the ordered adjacent grid transfer sequence of this route. Then, Kalman filtering is used to smooth the generated candidate paths to eliminate the jagged effect caused by grid discretization, making the completed trajectory more consistent with the continuity of actual driving. The smoothed candidate completion points are written into the cleaned trajectory.

[0058] When the missing duration exceeds the second duration threshold, a missing marker is executed: by default, no continuous trajectory is generated; only when the similarity Sr between the endpoint context windows at both ends of the missing segment and the route of the same historical representative route is not less than 0.75, and the order of the grid where p(a) is located and the grid where p(b) is located on the route is consistent with the driving direction, will sparse candidate completion points be generated at the key transfer grid and recorded as low confidence. Otherwise, only a missing marker is output and no completion points are generated.

[0059] The critical transfer grid is the grid node in the adjacent grid transfer sequence of the historical representative route that plays a decisive role in maintaining the route direction and path characteristics.

[0060] All imputed points are written with is_imputed=1, impute_method, and impute_confidence; low-confidence imputed points that have not been manually confirmed are not included in the historical model update.

[0061] Short missing segments are written into the cleaning trajectory only when all candidate points pass the boundary, speed, and distance checks; medium missing segments also require that the two endpoints fall on the same representative route and have the same direction; long missing segments only output missing markers or low-confidence sparse candidate points, and do not calculate continuous mileage.

[0062] Missing segments are processed according to the strength of the evidence for the route. When the evidence is insufficient, the missing segments are retained instead of being forcibly interpolated, thereby reducing the false trajectories generated by missing segment completion.

[0063] S5: Exception handling; Track points are anomaly corrected and marked. Records with invalid coordinates or geographical out-of-bounds entries are placed in an isolation zone. Duplicate records are deduplicated. Isolated jump points are replaced by interpolation when the constraints of the preceding and following points are met. Continuous anomaly records retain their original values ​​and are placed in the review queue. Anomaly reports are generated, and anomaly details are saved. The review queue undergoes manual review, recording the final label, corrected coordinates, and review reason. Only confirmed samples are used to update thresholds or historical models.

[0064] S51: Correct the exception and mark it; Each trajectory point record can be written with one or more anomaly types (anomaly_codes). Anomaly types include NULL_COORD, ZERO_COORD, OUT_OF_BOUND, DUPLICATE_POINT, TIME_REVERSE, TIME_CONFLICT, SPEED_EXCEED, DISTANCE_JUMP, LONG_GAP, and BEHAVIOR_DEVIATION. An action status is also provided: KEEP (keep, data is normal, enters the cleaning trajectory); EXCLUDE (excludes, only indicates not entering the cleaning trajectory, original record is still retained); CORRECT (corrects, original point is abnormal, corrected (e.g., interpolation replacement), corrected point can enter the cleaning trajectory, original record is still retained); REVIEW (manual review, manual intervention to confirm whether it is abnormal).

[0065] For valid trajectory points, the action status is set to KEEP. For trajectory points marked as anomalous, the action status is set to EXCLUDE according to the anomalous type: invalid coordinates marked as NULL_COORD or ZERO_COORD, trajectory points with time conflicts (TIME_CONFLICT) or geographical out-of-bounds (OUT_OF_BOUND). Duplicate points marked as DUPLICATE_POINT are retained only once. Isolated jump points generate completion points when the preceding and following points are valid and the short missing segment condition is met, and the action status of the completion points is set to CORRECT. For low-confidence completion points with continuous jumps, behavioral deviations, and medium or long missing segments, the action status is set to REVIEW. Original records are not overwritten. The correction records of completion points are associated through record_id and quality_weight is set. The quality_weight of original valid points is 1, and the quality_weight of completion points is impute_confidence.

[0066] S52: Generate an exception report; Anomaly reports are aggregated by date, vehicle, and anomaly type (anomaly_code), recording the number of inputs, the number of anomalies, the anomaly rate, the occurrence time range, the actual value range, the threshold range, and the number of actions for each handling status. The number of inputs refers to the total number of trajectory points from the original records.

[0067] S53: Save exception details; Save the record_id, original coordinates, corrected coordinates, rule version, model version, and review_id; the review_id is a unique identifier for the manual review record. When the action status is set to REVIEW, the trajectory point enters the review queue, and a unique identifier is generated for each trajectory point that needs review. Vehicle-level, fleet-level, and zone-level summaries are generated from the same detail table.

[0068] All records marked as abnormal will be written to the abnormal details table. The key fields saved include: record_id, vehicle identifier, timestamp, original coordinates, corrected coordinates, actual value, threshold, rule version, model version, and action status.

[0069] S54: Manual review; For trajectory points with the action status set to REVIEW, they are added to the review queue and written to the manual review table. The manual review table uses review_id as the primary key and is associated with the exception details table. Manual review is performed and recorded. The manual review table stores final_label (final judgment label), corrected_latitude (corrected latitude), corrected_longitude (corrected longitude), review_reason (review reason), and review_time (review time) as separate columns.

[0070] Only samples with a final_label of NORMAL or ANOMALY and a verified status of "confirmed" are included in the next update cycle. These samples are used to update thresholds or historical behavior models. During updates, the magnitude of single parameter changes is limited, and both the pre- and post-update versions are saved. The magnitude of single changes to thresholds and model weights is restricted, and version numbers are retained for rollback purposes. This ensures continuous model optimization while maintaining stability, preventing misclassified samples from contaminating historical statistical benchmarks or causing drastic jumps in model parameters.

[0071] S6: Output cleaning results; Output cleaning trajectory, anomaly details, completion records, and quality indicators; The cleaning trajectory is the trajectory points that have been fully verified and whose processing status is KEEP or CORRECT.

[0072] The completion record is a summary record of all completion points.

[0073] S61: Calculate quality indicators; Let the evaluation time window be [t0, t1], the baseline sampling period be T0, and the cumulative duration when the vehicle status is clearly closed be Toff. Then the effective observation duration Deff = t1 - t0 - Toff, and the number of points should be Nexp = floor(Deff / T0) + 1.

[0074] First, calculate the original completeness Qc = min(1, Nobs / Nexp), where Nobs is the number of observations that have passed the basic validity check without being imputed; it measures the sufficiency of the original valid observations, with a value of 1 indicating that the original number of points has met the required number. The number of imputed points that have passed the boundary, velocity, and distance checks is Naccepted_imputed, and the imputed coverage Qfill = Naccepted_imputed / Nexp is reported separately and is not included in Qc.

[0075] Then, the validity is calculated as Qv = Nrule_valid / Ninput, where Ninput is the number of inputs and Nrule_valid is the number of points that pass all hard rule checks, including coordinate validity, time order, geographical boundaries, speed, and distance.

[0076] Next, the continuity is calculated as Qt=max(0, 1-Ngap / max(1,Nadj)), where Ngap is the number of missing segments and Nadj is the total number of adjacent point pairs. A higher value indicates fewer trajectory breaks, which measures the degree of influence of missing segments in the trajectory on the overall continuity.

[0077] Calculate behavioral consistency Qs = max(0, 1 - Ndeviation_window / max(1, Nwindow)). Ndeviation_window is the number of windows marked as behavioral deviations, and Nwindow is the total number of windows. A higher value indicates less behavioral deviation, and measures the proportion of the trajectory window that matches the historical behavior model.

[0078] Finally, the overall score Q is calculated as Q = 100 × (0.30Qc + 0.25Qv + 0.25Qt + 0.20Qs); the numerator, denominator, and statistical time window of each item are also output.

[0079] S62: Output result; Write the cleaning trajectory, anomaly details, and quality indicators into the partition results table.

[0080] The present invention has been described in the above-described embodiments; however, these embodiments are merely examples for implementing the present invention. It must be noted that the disclosed embodiments do not limit the scope of the present invention. Conversely, any modifications and refinements made without departing from the spirit and scope of the present invention are within the scope of patent protection of the present invention.

Claims

1. A method for cleaning and anomaly detection of commercial vehicle trajectory data, characterized in that: Includes the following steps: S1: Import data and partition it; The system periodically collects raw records of trajectories, receives the collected raw records, processes the raw records using distributed computing, partitions them according to vehicle identification and date, sorts them by timestamp within each partition, and appends the last valid trajectory point of the previous partition as a boundary context to the current partition. The spatial index of administrative regions and rule parameters are distributed as read-only data to each computing partition; S2: Cleaning trajectory points; For each trajectory point, the coordinate validity, time sequence, geographical boundary, duplicate points, dynamic speed and dynamic distance are checked sequentially. If a check fails, it is marked as abnormal and not written into the cleaned trajectory. The dynamic distance threshold is calculated based on the dynamic speed threshold, the time difference between adjacent points and the positioning error compensation amount. S3: Historical Behavior Modeling; Based on historical valid trips, a grid coverage set and an ordered grid transfer sequence are established. The coverage similarity and sequence similarity of the trajectory window to be detected are calculated respectively, and the overall consistency is obtained by combining the speed conformity and the dwell conformity. When the overall consistency is lower than a set threshold, the trajectory window is marked as a behavior deviation. S4: Signal loss compensation; Based on the missing duration, the similarity between the endpoint context and the historical representative route, endpoint constraint interpolation, representative route constraint fitting, or missing labeling are performed on the missing segment; the generated candidate completion points are verified point by point through geographical boundaries, dynamic speed, and dynamic distance, and the completion method and completion confidence are recorded to obtain the completion record; S5: Exception handling; Abnormal trajectory points are corrected and marked, anomaly reports are generated, and anomaly details are saved. Continuous anomaly records are manually reviewed and confirmed. S6: Output cleaning results; The cleaning results include the cleaning trajectory, anomaly details, supplementary records, and quality indicators.

2. The method for cleaning and anomaly detection of commercial vehicle trajectory data as described in claim 1, characterized in that: S1 includes: S11: Enter the original record; Each original record includes at least the record identifier (record_id), the original data source order number (source_order), the vehicle identifier (terminal_id), the GPS timestamp (gpstimestamp), the latitude, the longitude, the speed, and the status; if a trip number exists, it also includes the trip number (trip_id), and records the data source type (source_type) and the data structure version number (schema_version). S12: Partition processing; The system is partitioned according to the vehicle identifier terminal_id and the date. Within each partition, the data is sorted by the GPS timestamp gpstimestamp. Each partition reads the last valid point of the previous partition and the first valid point of the next partition as the boundary context. After detection, overlapping records are removed according to the record identifier record_id. S13: Standardize the processing of raw records; Time is converted to seconds and then to the same time zone; latitude and longitude are converted to degrees according to coordinate_scale and then to a unified coordinate system. After standardization, trajectory points are obtained. S14: Mapping and reading the spatial index and rule parameters of the administrative region; The rule parameters are obtained by reading the field mapping based on the data structure version number schema_version.

3. The method for cleaning and anomaly detection of commercial vehicle trajectory data as described in claim 1, characterized in that: Step S2 includes: S21: Detect the validity of the coordinates of the trajectory points; Check if the coordinates are empty or zero. If so, mark them as NULL_COORD or ZERO_COORD respectively. Do not write the marked trajectory points into the cleaning trajectory; retain the original records. S22: Filter geographic boundaries; S2 polygons are constructed based on the boundary coordinates of administrative regions, and a set of covering units is generated from the S2 polygons. Candidate administrative regions are selected from the set of covering units based on the S2 polygons corresponding to the trajectory points. Then, the S2 polygons of the candidate administrative regions are used to perform point inclusion judgment. Trajectory points that do not fall into the target administrative region are marked as OUT_OF_BOUND and enter the isolation area, and are not written into the cleaned trajectory. The original records are retained. S23: Detect dynamic speed and dynamic distance; First, check the timestamps of the same vehicle according to the original sequence number source_order of the data source. When the time of the trajectory points is in reverse order, mark TIME_REVERSE. Then sort them according to the GPS timestamp gpstimestamp and calculate the difference between adjacent trajectory points. If the difference is 0 and the coordinates are different, mark TIME_CONFLICT to indicate a time conflict. If the difference is greater than 0, calculate the distance and speed. Set detection boundaries: Set the upper limit of the vehicle's physical speed during the cold start phase; Using samples that have passed hard rule verification or been manually confirmed to be normal, after the number of normal samples reaches n_min, calculate the median speed M and median absolute deviation MAD; take Tv=min(Vphy, M+k×1.4826×max(MAD,δ)), where Vphy is the set upper limit of vehicle physical speed, k is 3 to 5, δ is the minimum speed dispersion, which is a preset value, and the upper limit of dynamic distance Td=Tv×Δt / 3.6+ε, where ε is the positioning error compensation amount; if n_min is not reached, the upper limit of vehicle physical speed is used as the upper limit of dynamic speed; If the detection speed exceeds the dynamic speed limit or the distance exceeds the dynamic distance limit, it is marked as an anomaly and removed from the cleaning trajectory or corrected. The original record is not deleted. S24: Detect duplicate points and transition points; When the same vehicle at the same time point has the same coordinates, it is marked as DUPLICATE_POINT; when the coordinates at the same time point are different, it is marked as TIME_CONFLICT. If the distance between the current point and its adjacent points exceeds their respective dynamic distance thresholds, but the distance between the two points does not exceed the dynamic distance threshold for the corresponding time span, then it is marked as DISTANCE_JUMP.

4. The method for cleaning and anomaly detection of commercial vehicle trajectory data as described in claim 1, characterized in that: Step S3 includes: S31: Represents the historical valid trip distance; For the same model version, a uniform GeoHash precision L is used to divide the geospace into grids of equal size. Each historical valid trip of the historical model is represented in two ways: one is the grid set G, and the other is the adjacent grid transfer sequence T. S32: Construct the trajectory window; Taking the trajectory window W as the object, W consists of m consecutive valid points or valid points within a fixed time period; S33: Calculate the similarity of routes; For the current trajectory window W to be detected, the same GeoHash precision L as the historical model is first used to generate the grid set G(W) and the adjacent grid transition sequence T(W) for the window. The coverage similarity is calculated using Jaccard similarity, and the order similarity is obtained by dividing the length of the longest common subsequence of T(W) and T(R) by the larger value of their lengths. The route similarity is calculated by weighted summation of the coverage similarity and the order similarity. S34: Obtain speed compliance; The median speed of all valid points within the current window W is set as M(W). The median speed of the corresponding vehicle is obtained as M, the median absolute deviation MAD, and the minimum speed dispersion δ. First, the standardized deviation of the window speed median relative to the historical distribution is calculated. Then, the speed conformity is obtained through the Gaussian kernel function. The speed conformity range is (0,1]. S35: Obtain the degree of compliance with the stay requirement; Preset dwell speed threshold v_stop and dwell duration threshold T_stop; define dwell points as those with speed not higher than v_stop and duration not shorter than T_stop; for all dwell points in the historical model, count the dwell frequency of each GeoHash grid and normalize it to the [0,1] interval to form the historical dwell grid frequency distribution; for the current window W, if there are no dwell points in the window, the dwell compliance Ss is 1; if there are dwell points in the window, take the average of the historical dwell frequencies of the grids where each dwell point is located in the window as the dwell compliance Ss of the window. S36: Overall consistency determination; The route similarity Sr, speed compliance Sv, and dwell compliance Ss are weighted and summed to obtain the comprehensive consistency index C of the trajectory window. When C < 0.7, the window W is marked as BEHAVIOR_DEVIATION; and this mark is written to each trajectory point within the window. S37: Continuously manage and update model versions; The GeoHash precision L, window size m, weight coefficients, dwell speed threshold v_stop, dwell duration threshold T_stop, and minimum speed discrepancy δ are all saved along with the model version number; only trajectory point samples that pass through hard rules or are manually confirmed to be normal are used.

5. The method for cleaning and anomaly detection of commercial vehicle trajectory data as described in claim 1, characterized in that: Step S4 includes: S41: Determine the duration of the missing information; Calculate the missing duration by obtaining the GPS timestamps of the valid trajectory points at both ends of the missing segment; S42: Hierarchical processing of missing segments; Preset a first threshold and a second threshold; When the missing duration does not exceed the first duration threshold, endpoint constraint interpolation is performed; When the missing duration exceeds the first duration threshold but does not exceed the second duration threshold, perform representative route constraint fitting; When the missing duration exceeds the second duration threshold, a missing marker is executed. Only when the similarity Sr between the endpoint context windows at both ends of the missing segment and the route of the same historical representative route is not less than 0.75, and the order of the grids containing the valid trajectory points at both ends on the route is consistent with the driving direction, sparse candidate completion points are generated at the key transfer grid.

6. The method for cleaning and anomaly detection of commercial vehicle trajectory data as described in claim 1, characterized in that: Step S5 includes: S51: Correct the exception and mark it; Each trajectory point record is written with one or more exception types and its handling status is given; S52: Generate an exception report; The anomaly reports are aggregated by date, vehicle, and anomaly type, and record the number of inputs, the number of anomalies, the anomaly rate, the time range of occurrence, the actual value range, the threshold range, and the number of each handling status. S53: Save exception details; The anomaly details include record identifier, vehicle identifier, timestamp, original coordinates, corrected coordinates, actual value, threshold, rule version, model version, handling status, and identifier of manual review records; S54: Manual review; Add the items to the review list based on their status and write them into the manual review form. Manually review and record the results. The manual review form includes the final judgment circle, corrected latitude, corrected longitude, reason for review, and review time.

7. The method for cleaning and anomaly detection of commercial vehicle trajectory data as described in claim 6, characterized in that: Exception types include exception flags for NULL_COORD, ZERO_COORD, OUT_OF_BOUND, DUPLICATE_POINT, TIME_REVERSE, TIME_CONFLICT, SPEED_EXCEED, DISTANCE_JUMP, LONG_GAP, and BEHAVIOR_DEVIATION. Normal and valid trajectory points are set to KEEP status. For invalid coordinates marked NULL_COORD or ZERO_COORD, trajectory points with time conflicts (TIME_CONFLICT) or geographical out-of-bounds (OUT_OF_BOUND) are set to EXCLUDE status. Duplicate points marked DUPLICATE_POINT are retained only once. Completed points with isolated jump points interpolated by short missing segments are set to CORRECT status. Completed points with low confidence due to continuous jumps, behavioral deviations, or medium or long missing segments are set to REVIEW status.

8. The method for cleaning and anomaly detection of commercial vehicle trajectory data as described in claim 1, characterized in that: Step S6 includes: S61: Calculate quality indicators; The required number of points, Nexp, should be obtained through calculation. First, calculate the original integrity Qc = min(1, Nobs / Nexp); Nobs is the number of points that have not been completed and have passed the coordinate validity check; Then, the validity Qv is calculated as Qv = Nrule_valid / Ninput, where Ninput is the number of inputs and Nrule_valid is the number of points that pass all hard rule validations. Next, the continuity is calculated as Qt = max(0, 1 - Ngap / max(1, Nadj)), where Ngap is the number of missing segments and Nadj is the total number of adjacent pairs. Calculate behavioral consistency Qs = max(0, 1 - Ndeviation_window / max(1, Nwindow)); where Ndeviation_window is the number of windows marked as behavioral deviations, and Nwindow is the total number of windows; Finally, a weighted composite score is calculated to obtain the quality index; S62: Output result; Write the cleaning trajectory, anomaly details, and quality indicators into the partition results table.

9. The method for cleaning and anomaly detection of commercial vehicle trajectory data as described in claim 3, characterized in that: The change ratio of Tv and Td after each update relative to the previous one is limited to no more than ρ, where 0 < ρ < 1.

Images