Monitoring data analysis methods and related devices applied to digital toll stations

Abstract

This invention provides a monitoring data analysis method and related apparatus for digital toll stations. By acquiring a raw monitoring data set within the toll station area, feature extraction processing is performed on the raw monitoring data set to obtain vehicle passage features from the vehicle passage images and interaction behavior features from the transaction interaction records. A pre-trained anomaly analysis model is then invoked to perform collaborative anomaly recognition processing on the vehicle passage features and the interaction behavior features, generating anomaly recognition results for the vehicle passage images and transaction interaction records. Based on the anomaly recognition results, the types of abnormal events present in the raw monitoring data set and the spatiotemporal distribution information of these abnormal events within the toll station scenario are determined. This invention can improve the accuracy of anomaly recognition and enhance the effectiveness of management adjustments, thus better meeting the actual needs of digital toll stations for data-driven precision management.

Description

Technical Field

[0001] This invention relates to the field of data processing, and more specifically, to a method and related apparatus for analyzing monitoring data applied to digital toll stations. Background Technology

[0002] With the application of digital technology in traffic management, toll station monitoring data analysis technology has emerged. This technology analyzes monitoring data generated during toll station operations to identify abnormal events and support management adjustments. Currently, monitoring data analysis methods typically rely solely on image analysis of vehicle traffic status or record analysis of transaction processes, independently outputting anomaly identification results. This approach struggles to capture the potential correlation between vehicle traffic status and transaction interactions, easily leading to misjudgments or missed detections of abnormal events. Furthermore, it fails to provide precise spatiotemporal location information for management adjustments, resulting in a lack of targeted management measures. Summary of the Invention

[0003] This invention provides a monitoring data analysis method and related device for digital toll stations.

[0004] In a first aspect, embodiments of the present invention provide a monitoring data analysis method applied to digital toll stations, the method comprising:

[0005] Obtain the original monitoring data set within the toll station area, wherein the original monitoring data set includes continuously collected vehicle passage images with timestamps and corresponding transaction interaction records;

[0006] The original monitoring data set is subjected to feature extraction processing to obtain the vehicle passage features of the vehicle passage image and the interaction behavior features of the transaction interaction record;

[0007] A pre-trained anomaly analysis model is invoked to perform anomaly collaborative recognition processing on the vehicle passage features and the interaction behavior features, generating anomaly recognition results for the vehicle passage image and transaction interaction record;

[0008] Based on the anomaly identification results, determine the types of abnormal events present in the original monitoring data set and the spatiotemporal distribution information of the abnormal events in the toll station scenario.

[0009] Secondly, embodiments of the present invention provide a monitoring data analysis device, comprising:

[0010] A memory, wherein a computer program is stored;

[0011] A processor is used to load the computer program to implement the monitoring data analysis method applied to digital toll stations as described above.

[0012] The monitoring data analysis method for digital toll stations provided by this invention acquires a raw monitoring data set containing vehicle passage image units and transaction interaction record units, providing a multi-dimensional data foundation covering vehicle passage status and transaction interaction behavior for analysis. Vehicle passage features and interaction behavior features are extracted from the two types of data respectively, selectively preserving key visual information such as vehicle outlines and positions from the image data and behavioral sequence information such as action continuity and step matching from the interaction records. An anomaly analysis model is invoked to perform anomaly collaborative identification of the two types of features, enabling the discovery of potential correlations between vehicle passage status and transaction interaction behavior, and avoiding potential anomalies. It avoids frequent misjudgments or omissions; by identifying the types of abnormal events and their spatiotemporal distribution information in the toll station scenario based on the identification results, it not only clarifies the specific type of the abnormality, but also locates the time range and spatial location of the abnormality, providing precise problem guidance for subsequent adjustments; based on the abnormality type and spatiotemporal distribution, it generates monitoring optimization instructions containing event location identifiers and sends them to the management terminal for execution, realizing the full-process linkage from data collection to management operation, so that the analysis results can be directly transformed into targeted management measures, which not only improves the accuracy of abnormality identification, but also enhances the effectiveness of management adjustments, thus better meeting the actual needs of digital toll stations for data-driven precision management. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a flowchart of a monitoring data analysis method applied to digital toll stations, provided by an embodiment of the present invention.

[0015] Figure 2 This is a schematic diagram of the composition of a monitoring data analysis device provided in an embodiment of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Please see Figure 1 , Figure 1A flowchart of a monitoring data analysis method applied to a digital toll station, provided by an embodiment of the present invention, is included. This method can be executed by a monitoring data analysis device and includes the following steps:

[0018] Step S100: Obtain the original monitoring data set within the toll station area. The original monitoring data set includes continuously collected vehicle passage images with timestamps and corresponding transaction interaction records.

[0019] In this embodiment of the invention, the original monitoring data set refers to a dataset collected within the toll station area, encompassing vehicle passage images and transaction interaction records. Each data unit in this dataset is timestamped to indicate the data collection time. Vehicle passage images are continuously acquired using image acquisition equipment and include overall vehicle outline information and local license plate area information. Transaction interaction records are generated by combining image data of the interaction between the toll collector and the driver with transaction operation log data recorded by the toll terminal. These records contain sequence information of interactive actions and transaction operation steps, reflecting the specific behaviors and operations during the toll collection process.

[0020] As one implementation method, step S100, obtaining the original monitoring data set within the toll station area, specifically includes the following steps S110~S140:

[0021] Step S110: The first image acquisition device deployed at the entrance of the toll station continuously acquires image data of vehicles entering the toll station, and generates a vehicle passage image with a timestamp. The vehicle passage image includes the overall outline information of the vehicle and local information of the license plate area.

[0022] The first image acquisition device, such as a high-definition camera, is used to capture images of vehicles entering the toll station. It continuously acquires images at regular time intervals to ensure a complete record of the vehicle's entry into the toll station. Timestamps are used to accurately record the acquisition time of each image, facilitating subsequent data processing and analysis. Overall vehicle outline information helps identify the vehicle's type, size, and other characteristics, while local information about the license plate area helps accurately identify the license plate number.

[0023] Step S120: The second image acquisition device deployed in the toll booth continuously acquires image data of the interaction process between the toll collector and the driver. Combined with the transaction operation log data recorded by the toll terminal, a transaction interaction record with a timestamp is generated. The transaction interaction record contains the sequence information of the interaction actions and the transaction operation steps.

[0024] The second image acquisition device is used to collect image data of the interaction between the toll collector and the driver at the toll booth; it can also be a high-definition camera. The transaction operation log data recorded by the toll terminal includes various operational information during the toll collection process, such as the toll collector's operation steps, transaction amount, and payment method. Combining these two parts of data and adding a timestamp creates the transaction interaction record. The sequence of interactive actions reflects the specific interactive actions between the toll collector and the driver, such as handing over tickets and swiping cards; the transaction operation step information records each operational step in the toll collection process.

[0025] Step S130: Perform timestamp alignment processing on the vehicle passage images and transaction interaction records to ensure that the vehicle passage images and transaction interaction records at the same time point have a corresponding relationship.

[0026] Timestamp alignment is performed to ensure temporal consistency between vehicle passage images and transaction interaction records, enabling accurate correspondence between the two data points at the same time. Due to potential clock errors between different devices and differences in data acquisition and processing times, timestamp alignment is necessary to improve data accuracy and correlation.

[0027] When performing timestamp alignment, a global synchronization clock signal is first required. This signal provides a unified time reference for both the first and second image acquisition devices. This global synchronization clock signal can be provided by a high-precision clock device deployed at the toll station. Then, the original acquisition timestamps of the vehicle passage images and the original recording timestamps of the transaction interaction records are extracted, and their time offsets from the global synchronization clock signal are calculated. Based on these time offsets, the original acquisition timestamps of the vehicle passage images and the original recording timestamps of the transaction interaction records are calibrated to generate calibrated vehicle timestamps and calibrated interaction timestamps.

[0028] As one implementation method, step S130 involves performing timestamp alignment processing on the vehicle passage images and transaction interaction records to ensure a correspondence between vehicle passage images and transaction interaction records at the same time point. This specifically includes the following steps S131-S136:

[0029] Step S131: Obtain the global synchronization clock signal deployed at the toll station. The global synchronization clock signal is used to provide a unified time reference for the first image acquisition device and the second image acquisition device.

[0030] The global synchronization clock signal is a high-precision clock signal generated by a dedicated clock device deployed at the toll station. It provides a unified time reference for the first and second image acquisition devices, ensuring that the data acquired by the two devices are consistent in time.

[0031] To obtain a globally synchronized clock signal, a GPS clock device can be used. GPS clock devices provide high-precision time information by receiving satellite signals. The GPS clock device is installed at a suitable location in the toll station and connected to both the first and second image acquisition devices. In this way, both image acquisition devices can acquire the globally synchronized clock signal and use it as a reference for timestamping.

[0032] Step S132: Extract the original acquisition timestamp of the vehicle passage image and the original recording timestamp of the transaction interaction record, and calculate the time offset of the two from the global synchronization clock signal respectively.

[0033] The original acquisition timestamp of the vehicle traffic image refers to the time marked by the first image acquisition device when acquiring the image, while the original recording timestamp of the transaction interaction record is the time marked when the transaction interaction record is generated. Due to potential clock errors between different devices and differences in data acquisition and processing times, these two original timestamps may have a certain time offset from the global synchronization clock signal. When extracting timestamps, the original acquisition timestamp and the original recording timestamp can be obtained by reading the metadata of the image data and the transaction interaction record. Then, these two timestamps are compared with the time of the global synchronization clock signal, and the difference between them is calculated to obtain the time offset.

[0034] Step S133: The original acquisition timestamp of the vehicle passage image is calibrated according to the time offset to generate a calibrated vehicle timestamp; the original record timestamp of the transaction interaction record is calibrated to generate a calibrated interaction timestamp.

[0035] The calibration process involves adjusting the original acquisition timestamps of vehicle passage images and the original recording timestamps of transaction interaction records based on the calculated time offset, ensuring they are consistent with the global synchronization clock signal. The calibrated vehicle timestamps and calibrated interaction timestamps more accurately reflect the actual time of vehicle passage and transaction interactions.

[0036] Step S134: Based on the calibrated vehicle timestamp and the calibrated interaction timestamp, establish a timestamp matching window. The time span of the timestamp matching window is half of the adjacent data acquisition cycle.

[0037] The timestamp matching window is the time range used to match vehicle passage images and transaction interaction records. Based on the calibrated vehicle timestamp and calibrated interaction timestamp, a suitable time span is determined as the matching window, and data matching is performed within this window. Adjacent data acquisition cycles refer to the time interval between data acquisition by the first and second image acquisition devices. Setting the time span of the timestamp matching window to half of the adjacent data acquisition cycles can improve the accuracy and efficiency of the matching.

[0038] Step S135: In the timestamp matching window, perform one-to-one matching between the calibrated vehicle timestamp and the calibrated interaction timestamp. If there are vehicle passage images and transaction interaction records with timestamp differences less than a preset threshold, mark them as corresponding data units at the same time point.

[0039] The preset threshold is a pre-defined time difference value used to determine whether vehicle passage images and transaction interaction records belong to the same point in time. Within the timestamp matching window, the calibrated vehicle timestamp and the calibrated interaction timestamp are compared. If the time difference between the two is less than the preset threshold, the two data are considered to belong to the corresponding data unit of the same time point.

[0040] During the matching process, the time difference between all calibrated vehicle timestamps and calibrated interaction timestamps within the timestamp matching window can be calculated by iterating through them.

[0041] Step S136: For unmatched vehicle passage images and transaction interaction records, perform interpolation completion processing based on the chronological order of their timestamps to generate a set of aligned data pairs with continuous time correspondence.

[0042] Interpolation completion is a data processing method for unmatched vehicle passage images and transaction interaction records. Based on the chronological order of their timestamps, it estimates the missing data through interpolation, thereby generating a set of aligned data pairs with continuous temporal correspondence.

[0043] When performing interpolation and completion processing, a linear interpolation method can be adopted. First, sort the unmatched vehicle passing images and transaction interaction records according to the time stamps. Then, for two adjacent successfully matched data units, estimate the interpolated values for the unmatched data in between based on their time stamps and data features. For example, if the time stamps of two adjacent successfully matched data units are t1 and t2 respectively, and the corresponding vehicle passing images and transaction interaction records are I1, R1 and I2, R2 respectively, for the unmatched data with a time stamp of t (t1 < t < t2), the corresponding vehicle passing image and transaction interaction record can be estimated according to the linear interpolation formula. By doing this continuously for interpolation and completion, a set of aligned data pairs with continuous time correspondence is finally generated. An interpolation algorithm can be written in a programming language to implement this process. The program will perform interpolation calculations based on the time stamps and data features and generate a set of aligned data pairs.

[0044] Step S140: Integrate the vehicle passing images and transaction interaction records after time stamp alignment into an original monitoring data set. Each data unit in the original monitoring data set contains a vehicle passing image subunit, a transaction interaction record subunit, and the corresponding time stamp information.

[0045] The integration process is to combine the vehicle passing images and transaction interaction records after time stamp alignment to form a complete original monitoring data set. Each data unit contains a vehicle passing image subunit, a transaction interaction record subunit, and the corresponding time stamp information, which can ensure the integrity and relevance of the data.

[0046] When performing integration, it can be achieved through programming. The program will read the vehicle passing images and transaction interaction records after time stamp alignment, arrange them in chronological order, and combine the vehicle passing image subunit, the transaction interaction record subunit, and the corresponding time stamp information into a data unit. Then, store all the data units in a data file or database to form the original monitoring data set.

[0047] Step S200: Perform feature extraction processing on the original monitoring data set to obtain the vehicle passing features of the vehicle passing images and the interaction behavior features of the transaction interaction records.

[0048] Feature extraction processing is a process of extracting from the original monitoring data set the essential features that can reflect vehicle passing and transaction interaction. The vehicle passing features are extracted from the vehicle passing images and are used to describe various characteristics of vehicle passing; the interaction behavior features are extracted from the transaction interaction records and are used to describe the characteristics of the interaction behavior between the toll collector and the driver.

[0049] As one implementation method, step S200 involves performing feature extraction processing on the original monitoring data set to obtain vehicle passage features of the vehicle passage images and interaction behavior features of the transaction interaction records, specifically including the following steps S210~S240:

[0050] Step S210: Perform noise reduction and contrast enhancement processing on the vehicle traffic image to obtain a preprocessed vehicle traffic image.

[0051] Noise reduction processing aims to remove noise interference from vehicle traffic images, improving image quality and clarity. Noise may originate from electronic noise in the image acquisition equipment itself, ambient light interference, etc. Contrast enhancement processing, on the other hand, enhances the contrast between different areas of the image, making vehicle outlines and license plate areas more clearly visible.

[0052] For noise reduction, Gaussian filtering can be used. Gaussian filtering is a linear smoothing filtering method that removes noise by weighted averaging of each pixel and its neighborhood in the image. In practice, Gaussian filtering functions from image processing libraries (such as OpenCV) can be used, setting appropriate Gaussian kernel size and standard deviation to filter the vehicle traffic image. Contrast enhancement can be achieved using histogram equalization. Histogram equalization enhances image contrast by adjusting the image's grayscale histogram. It redistributes the grayscale values, making the grayscale range more uniform. Similarly, histogram equalization functions from the OpenCV library can be used to process the vehicle traffic image, resulting in a preprocessed image.

[0053] Step S220: Perform feature extraction processing on the preprocessed vehicle traffic image, identify the integrity features of the vehicle outline, the clarity features of the license plate area, and the positional relationship features between the vehicle and the toll lane boundary, and integrate the integrity features of the vehicle outline, the clarity features of the license plate area, and the positional relationship features between the vehicle and the toll lane boundary to obtain the vehicle traffic features.

[0054] The vehicle outline integrity feature reflects the completeness of the vehicle's outline in the image, the license plate area clarity feature reflects the clarity of the license plate area in the image, and the positional relationship feature between the vehicle and the toll lane boundary describes the vehicle's position in the toll lane. Integrating these three features yields a comprehensive vehicle traffic feature that describes the traffic situation.

[0055] In one implementation, step S220 specifically includes the following steps S221 to S224:

[0056] Step S221: Perform edge detection processing on the preprocessed vehicle traffic image, extract the set of boundary pixels of the vehicle contour, and calculate the continuous coverage ratio of the boundary pixel set in the image as the integrity feature of the vehicle contour.

[0057] Edge detection is a method used to detect the edges of objects in an image. Edge detection can extract the set of boundary pixels that define the vehicle's outline. The continuous coverage ratio refers to the proportion of boundary pixels that are continuously distributed in the image, reflecting the completeness of the vehicle's outline.

[0058] The Canny edge detection algorithm can be used for edge detection processing. The Canny edge detection algorithm is a multi-stage edge detection algorithm with high accuracy and noise resistance. First, Gaussian smoothing is applied to the preprocessed vehicle traffic image to remove noise. Then, the gradient magnitude and direction of the image are calculated. Next, non-maximum suppression is performed to refine the edges. Finally, double thresholding is used to determine the true edges. The Canny edge detection algorithm can obtain the set of boundary pixels of the vehicle contour.

[0059] To calculate the continuous coverage ratio of boundary pixel sets in an image, the following method can be used. First, divide the image into several small regions and count the number of boundary pixels in each small region. Then, calculate the proportion of small regions with continuous boundary pixels to the total number of small regions. For example, if the image is divided into 100 small regions, and 80 of them have continuous boundary pixels, then the continuous coverage ratio is 80%.

[0060] Step S222: Perform local feature enhancement processing on the license plate area in the vehicle traffic image, and extract the identifiability features of the license plate characters as the clarity features of the license plate area.

[0061] Local feature enhancement processing of the license plate area aims to improve the clarity and recognizability of the license plate area in the image, making the license plate characters easier to identify. The recognizability features of license plate characters are an important indicator of whether a license plate is clear and distinguishable in an image. It reflects information such as the integrity, clarity, and distinction from the background of the license plate characters.

[0062] To enhance local features in the license plate area, histogram local equalization can be used. Unlike global histogram equalization, this method performs histogram equalization on local regions of the image, enhancing local contrast while avoiding the over-enhancement issues that global equalization might cause. Specifically, the license plate area needs to be accurately located in the vehicle traffic image first. This can be achieved by training and detecting the image using deep learning-based object detection algorithms, such as the YOLO (YouOnlyLookOnce) series, to identify the license plate location. The detected license plate area is then cropped from the image, and the histogram local equalization algorithm is applied to this area.

[0063] When extracting the identifiability features of license plate characters, character segmentation and feature extraction algorithms can be used. First, character segmentation is performed on the enhanced license plate region image, dividing each character into individual image blocks. A projection-based character segmentation algorithm can be used, determining the character boundary positions by calculating the projection distribution of the image in the horizontal and vertical directions. Then, feature extraction is performed on each segmented character image block. The Histogram of Oriented Gradients (HOG) feature extraction algorithm can be used, which describes the texture and shape features of the image by calculating the gradient orientation histogram of local image regions. The extracted HOG features of each character are concatenated to form a feature vector, which serves as the identifiability feature of the license plate characters. In this way, the clarity features of the license plate region can be accurately extracted, providing strong support for subsequent analysis and processing.

[0064] Step S223: Perform semantic segmentation on the toll lane boundary in the vehicle passage image, extract the relative position coordinates of the vehicle outline boundary and the toll lane boundary, and calculate the distance difference feature between the two as the positional relationship feature between the vehicle and the toll lane boundary.

[0065] Semantic segmentation is the process of dividing and labeling different semantic regions in an image. Semantic segmentation of toll lane boundaries in vehicle traffic images can accurately identify the boundary positions of the toll lanes. The relative position coordinates of the vehicle outline boundary and the toll lane boundary describe the vehicle's specific position within the toll lane, while the distance difference between the two reflects the distance relationship between the vehicle and the toll lane boundary.

[0066] When performing semantic segmentation, deep learning-based semantic segmentation models, such as the U-Net model, can be used. The U-Net model is a convolutional neural network with an encoder-decoder structure, capable of effectively classifying images at the pixel level. First, a large number of vehicle traffic images containing toll lanes need to be collected and manually labeled, assigning specific categories to the toll lane boundary regions. Then, the U-Net model is trained using the labeled dataset, adjusting its parameters to accurately segment the toll lane boundaries. After training, the vehicle traffic images are input into the trained U-Net model to obtain the segmentation results of the toll lane boundaries.

[0067] When extracting the relative position coordinates between the vehicle outline boundary and the toll lane boundary, edge detection algorithms and semantic segmentation results can be combined. The Canny edge detection algorithm is used to extract the set of boundary pixels of the vehicle outline. Then, the coordinates of these boundary pixels are compared and calculated with the coordinates of the toll lane boundary obtained from semantic segmentation. The Euclidean distance formula can be used to calculate the distance difference feature between the two. For each pixel on the vehicle outline boundary, its shortest Euclidean distance to the toll lane boundary is calculated. Then, statistical analysis is performed on the distance values ​​of all pixels, such as calculating the average, maximum, and minimum values. These statistical values ​​are used as the distance difference feature between the vehicle and the toll lane boundary. In this way, the positional relationship between the vehicle and the toll lane boundary can be accurately described.

[0068] Step S224: Perform feature stitching on the integrity features of the vehicle outline, the clarity features of the license plate area, and the positional relationship features between the vehicle and the boundary of the toll lane to generate vehicle passage features.

[0069] When performing feature concatenation, a vector concatenation method can be used. The integrity features of the vehicle outline, the clarity features of the license plate area, and the positional relationship features between the vehicle and the toll lane boundary are each represented as a feature vector. For example, the integrity feature of the vehicle outline can be represented as a one-dimensional vector, whose value is the proportion of continuous coverage of the boundary pixel set in the image; the clarity feature of the license plate area can be represented as a multi-dimensional vector, composed of the HOG features of each character; and the positional relationship features between the vehicle and the toll lane boundary can be represented as a one-dimensional vector containing statistical values ​​of distance differences. These three feature vectors are concatenated in a certain order to form a longer feature vector, which serves as the vehicle passage feature.

[0070] Step S230: Perform behavioral sequence parsing on the transaction interaction record, extract the continuity features of the interaction actions, the integrity features of the transaction operation steps, and the time matching features between the interaction actions and the transaction operation steps, and integrate the continuity features of the interaction actions, the integrity features of the transaction operation steps, and the time matching features between the interaction actions and the transaction operation steps to obtain the interaction behavior features.

[0071] Behavioral sequence analysis is the process of analyzing and understanding the interactive actions and transaction operation steps in transaction interaction records. The continuity of interactive actions reflects the smoothness of the interaction between the toll collector and the driver; the completeness of transaction operation steps reflects whether the transaction operation was performed according to preset standard steps; and the time matching feature between interactive actions and transaction operation steps describes the consistency of the interactive actions and transaction operation steps in time. Integrating these three features yields a comprehensive description of the interactive behavior. Detailed analysis of the transaction interaction records is required during behavioral sequence analysis. The interactive action sequences and transaction operation logs in the transaction interaction records can be processed separately. The continuity of interactive actions can be extracted by analyzing the time intervals and order of the interactive action sequences. The completeness of transaction operation steps can be compared and matched with preset standard operation steps in the transaction operation logs. The time matching feature between interactive actions and transaction operation steps can be calculated by comparing the time intervals of the interactive actions and the execution time intervals of the transaction operation steps.

[0072] In one implementation, step S230 specifically includes the following steps S231 to S234:

[0073] Step S231: Divide the sequence of interactive actions in the transaction interaction record into time axis to obtain multiple continuous subsequences of interactive actions, and calculate the uniformity of the time interval between adjacent actions in each subsequence of interactive actions as the continuity feature of the interactive actions.

[0074] Timeline segmentation involves dividing the sequence of interactive actions in the transaction interaction record according to time order, forming multiple consecutive subsequences of interactive actions. The uniformity of the time interval between adjacent actions reflects the smoothness of the interaction. If the time interval is uniform, it indicates that the interaction is relatively continuous; conversely, it indicates that the interaction may be interrupted or not smooth.

[0075] When performing timeline segmentation, the segmentation can be based on the timestamp information of the interactive actions. The sequence of interactive actions is arranged chronologically, and then divided into multiple subsequences according to a certain time interval threshold. For example, if the time interval between adjacent actions exceeds a preset threshold (e.g., 1 second), the two actions are considered to belong to different subsequences. The standard deviation method can be used to calculate the uniformity of time intervals between adjacent actions in each subsequence. First, the time intervals between adjacent actions in each subsequence are calculated, resulting in a time interval sequence. Then, the standard deviation of this time interval sequence is calculated. The smaller the standard deviation, the more uniform the time intervals and the better the continuity of the interactive actions; the larger the standard deviation, the more uneven the time intervals and the worse the continuity of the interactive actions.

[0076] Step S232: Perform step matching processing on the transaction operation log in the transaction interaction record, identify the completion status features of the preset standard operation steps, and use them as the integrity features of the transaction operation steps.

[0077] Step matching is the process of comparing and matching the operation steps in the transaction operation log with preset standard operation steps. Preset standard operation steps are a series of operational procedures formulated according to the toll station's business processes and specifications to ensure the normal execution of transactions. Identifying the completion characteristics of preset standard operation steps can determine whether the transaction operation was carried out according to the standard process, thus reflecting the completeness of the transaction operation steps.

[0078] When performing step matching, the first step is to define a template for preset standard operation steps. These standard operation steps can be represented as an ordered list of steps, each containing a step name and operation requirements. Then, the transaction operation log is parsed to extract the operation step information. The extracted operation steps are compared one by one with the preset standard operation steps to determine if each standard step has been completed in the transaction operation log. String matching algorithms (such as the Levenshtein distance algorithm) can be used to compare the similarity of the operation steps. For each standard step, if there is a corresponding operation step in the transaction operation log and the similarity exceeds a preset threshold (e.g., 0.8), then the standard step is considered complete. When calculating the completeness feature of transaction operation steps, the proportion of completed standard steps to the total number of standard steps can be counted. For example, if there are 10 preset standard operation steps, and 8 have been completed in the transaction operation log, then the completeness feature of the transaction operation steps is 80%. In this way, the completeness feature of transaction operation steps can be accurately identified.

[0079] Step S233: Extract the overlap ratio feature between the time interval of each interaction action subsequence and the execution time interval of the corresponding transaction operation step, as the time matching feature between the interaction action and the transaction operation step.

[0080] The time interval of each interactive action subsequence refers to the time range from the start time of the first action to the end time of the last action in the subsequence. The execution time interval corresponding to the trading operation step refers to the time range from the start of the execution of the operation step to the completion of the execution. The overlap ratio feature reflects the degree of temporal consistency between the interactive action and the trading operation step. The higher the overlap ratio, the better the temporal matching degree between the two.

[0081] When extracting the overlap ratio feature, it is first necessary to determine the time interval of each interactive action subsequence and the execution time interval of the corresponding trading operation step. The timestamp information of each interactive action can be extracted from the trading interaction record to determine the time interval of the interactive action subsequence; for the execution time interval of the trading operation step, the start time and end time of the operation step can be obtained from the trading operation log. Then, calculate the overlapping part of the two time intervals. The start time and end time of the overlapping part can be determined by comparing the start time and end time of the two time intervals. If the two time intervals do not overlap, the length of the overlapping part is 0; if there is an overlap, calculate the time length of the overlapping part. Finally, calculate the ratio of the time length of the overlapping part to the total length of the two time intervals as the overlap ratio feature.

[0082] For example, assume that the time interval of an interactive action subsequence is [t1, t2], and the execution time interval of the corresponding trading operation step is [t3, t4]. If t2 < t3 or t4 < t1, the length of the overlapping part is 0; if t3 <= t1 <= t4, the start time of the overlapping part is t1; if t1 <= t3 <= t2, the start time of the overlapping part is t3. Similarly, the end time of the overlapping part can be determined. After calculating the time length of the overlapping part, divide it by the total length of the two time intervals (i.e., (t2 - t1) + (t4 - t3) - the length of the overlapping part) to obtain the overlap ratio feature. A function can be written in a programming language to implement this calculation process. This function accepts two time intervals as input and returns the overlap ratio feature value.

[0083] Step S234: Perform feature fusion processing on the continuity feature of the interactive action, the integrity feature of the trading operation step, and the temporal matching feature between the interactive action and the trading operation step to generate an interactive behavior feature.

[0084] Feature fusion processing is a process of integrating different types of features to form a comprehensive feature. By fusing the continuity feature of the interactive action, the integrity feature of the trading operation step, and the temporal matching feature between the interactive action and the trading operation step, an interactive behavior feature that comprehensively describes the trading interaction behavior can be obtained.

[0085] When performing feature fusion processing, a weighted summation method can be used. First, assign a weight to each feature, the magnitude of which reflects the importance of the feature in describing the interaction behavior. For example, based on actual business needs and experience, a weight of 0.3 can be assigned to the feature of continuity of interaction actions, a weight of 0.4 can be assigned to the feature of completeness of transaction operation steps, and a weight of 0.3 can be assigned to the feature of time matching between interaction actions and transaction operation steps.

[0086] Then, the value of each feature is multiplied by its corresponding weight, and the results are summed to obtain the interaction behavior feature. Assuming the continuity feature of the interaction action is C, the integrity feature of the transaction operation steps is I, and the time matching feature of the interaction action and the transaction operation steps is M, then the interaction behavior feature value F can be expressed as: F = 0.3 × C + 0.4 × I + 0.3 × M.

[0087] Step S240: Perform feature dimension alignment processing on vehicle passage features and interaction behavior features so that they have the same feature dimension representation.

[0088] Feature dimension alignment is performed to ensure that vehicle traffic features and interaction behavior features are consistent in dimension for subsequent collaborative analysis and processing. Different feature extraction methods may result in these two features having different dimensions, while they need to have the same dimensional representation when performing anomaly collaborative identification.

[0089] When aligning feature dimensions, feature selection and feature imputation methods can be used. First, feature analysis is performed on vehicle traffic features and interaction behavior features to determine their respective number of dimensions and feature meanings. If one feature has more dimensions than another, feature selection methods can be used to choose representative dimensions to make the number of dimensions for both features equal. For example, Principal Component Analysis (PCA) can be used to reduce the dimensionality of high-dimensional features, extracting the principal components and reducing the number of dimensions.

[0090] If one feature has fewer dimensions than another, the missing dimensional information can be supplemented using feature imputation. Appropriate imputation values ​​can be chosen based on the meaning of the feature and the distribution of the data. For example, if the missing dimension is a numerical feature, the mean or median of that feature can be used; if it is a categorical feature, the most common category can be used.

[0091] By using feature selection and feature filling methods, vehicle passage features and interaction behavior features can be represented with the same feature dimension.

[0092] Step S300: Call the pre-trained anomaly analysis model to perform anomaly collaborative recognition processing on vehicle passage features and interaction behavior features, and generate anomaly recognition results for vehicle passage images and transaction interaction records.

[0093] The pre-trained anomaly analysis model, trained on a large amount of data, can analyze and judge vehicle traffic characteristics and interaction behavior characteristics to identify whether anomalies exist. Collaborative anomaly recognition processing comprehensively considers the correlation and interaction between vehicle traffic characteristics and interaction behavior characteristics, judging the existence of anomalies from multiple perspectives. The generated anomaly recognition results include anomaly judgment information for vehicle traffic images and transaction interaction records, such as whether anomalies exist and their types. When calling the pre-trained anomaly analysis model, vehicle traffic characteristics and interaction behavior characteristics must first be input into the model. The model processes and analyzes the input features, generating anomaly recognition results through a series of calculations and judgments. The model training process requires a large amount of historical data, including normal and abnormal data. By learning from this data, the model can master the characteristic patterns of normal and abnormal situations, thereby accurately identifying anomalies.

[0094] As one implementation method, step S300 specifically includes the following steps S310~S350:

[0095] Step S310: Input the vehicle passage features and interaction behavior features into the association mapping module of the anomaly analysis model, and generate an association feature group that reflects the strength of the association between the two through feature co-occurrence relationship analysis. The association feature group includes the synchronization and conflict indicators of vehicle features and interaction features.

[0096] The association mapping module of the anomaly analysis model is used to analyze the correlation between vehicle traffic characteristics and interaction behavior characteristics. Feature co-occurrence analysis studies the frequency and pattern of the simultaneous occurrence of these two features in the data, thereby determining the strength of their correlation. The associated feature group consists of synchronicity and conflict indicators of vehicle features and interaction features. The synchronicity indicator reflects the degree of consistency in their changing trends, while the conflict indicator reflects the contradictions and inconsistencies between them.

[0097] When performing feature co-occurrence analysis, association rule mining algorithms (such as the Apriori algorithm) can be used. First, vehicle traffic features and interaction behavior features are discretized, transforming continuous features into discrete features. Then, the Apriori algorithm is used to mine association rules between features. For each association rule, its support and confidence can be calculated. Support represents the frequency of the rule's occurrence in the data, and confidence represents the probability of the consequent occurring given that the antecedent is satisfied.

[0098] Based on the support and confidence of association rules, determine the synchronicity and conflict indicators for vehicle features and interaction features. If an association rule has both high support and confidence, it indicates that vehicle features and interaction features have strong synchronicity under that rule, and its synchronicity indicator is set to high. If an association rule has high support but low confidence, it indicates that there may be a conflict between the two, and its conflict indicator is set to high. The synchronicity and conflict indicators are then combined to form an association feature group.

[0099] Step S320: Perform scene context enhancement processing on the associated feature group, and combine the reference feature patterns of similar scenes in the historical monitoring data of the toll station to generate a context-enhanced feature group containing scene background information.

[0100] The purpose of context-enhanced processing is to combine associated feature groups with reference feature patterns from similar scenarios in historical toll station monitoring data. This allows the associated feature groups to integrate with the contextual information, generating more targeted and accurate context-enhanced feature groups. Historical toll station monitoring data records various feature patterns from similar scenarios in the past. These patterns reflect the characteristic patterns of vehicle traffic and transaction interactions under normal circumstances. By combining these reference feature patterns, it is possible to better determine whether the current associated feature group conforms to a normal scenario, thereby improving the accuracy of anomaly identification.

[0101] As one implementation method, step S320 specifically includes the following steps S321 to S326:

[0102] Step S321: Extract a subset of historical data from the toll station's historical monitoring database that matches the current original monitoring data set's traffic environment.

[0103] The toll station's historical monitoring database stores a large amount of historical monitoring data, including vehicle passage images and transaction records from different times and under different traffic conditions. To accurately find historical data that matches the current original monitoring data set under specific traffic conditions, a detailed definition and matching of the traffic environment is required. The traffic environment can include time factors (such as different time periods like weekdays, weekends, and holidays), weather conditions (sunny, rainy, foggy, etc.), traffic flow (peak hours, off-peak hours, etc.), and the specific location and layout of the toll station.

[0104] When extracting subsets of historical data, data matching certain criteria can be filtered by writing queries. For example, SQL statements in a database management system (such as MySQL) can be used to set filtering conditions based on various factors of the current environment. By executing the queries, a subset of historical data matching the current environment can be extracted from the historical monitoring database, providing a data foundation for subsequent reference pattern mining.

[0105] Step S322: Perform reference pattern mining processing on a subset of historical data to extract the co-occurrence frequency distribution and conflict frequency distribution of vehicle traffic features and interaction behavior features in normal scenarios, and generate a reference feature pattern set.

[0106] Reference pattern mining involves in-depth analysis of extracted historical data subsets to identify co-occurrence and conflict patterns between vehicle traffic features and interaction behavior features in normal scenarios. The co-occurrence frequency distribution describes the frequency with which vehicle traffic features and interaction behavior features appear simultaneously, reflecting their synergistic relationship; the conflict frequency distribution indicates the frequency of conflicts between these two features, reflecting their contradictory relationship.

[0107] To perform reference pattern mining, data mining algorithms, such as frequent pattern mining algorithms (e.g., FP-growth algorithm), can be used. First, vehicle traffic features and interaction behavior features in a subset of historical data are encoded and converted into a format suitable for algorithm processing. Then, the FP-growth algorithm is used to mine frequent itemsets, i.e., feature combinations that frequently occur together. By statistically analyzing the occurrence frequency of these frequent itemsets, the co-occurrence frequency distribution of vehicle traffic features and interaction behavior features is obtained.

[0108] The conflict frequency distribution can be calculated by comparing the differences between vehicle passage characteristics and interaction behavior characteristics. For example, if a vehicle passage characteristic indicates that a vehicle passes through quickly, while the corresponding interaction behavior characteristic shows that the toll collector performs an operation for a long time, this may be considered a conflict situation. By statistically analyzing the frequency of such conflict situations in a subset of historical data, the conflict frequency distribution can be obtained.

[0109] The co-occurrence frequency distribution and conflict frequency distribution are combined to generate a reference feature pattern set. This set contains typical patterns of vehicle traffic features and interaction behavior features in normal scenarios, providing a reference for subsequent matching analysis.

[0110] Step S323: Perform a matching analysis between the synchronicity identifiers in the associated feature group and the co-occurrence frequency distribution in the reference feature pattern set, and calculate the synchronicity matching degree.

[0111] Synchronization identifiers reflect the degree of synchronization between vehicle traffic characteristics and interaction behavior characteristics, while the co-occurrence frequency distribution in the reference feature pattern set represents the frequency pattern of these two features appearing simultaneously under normal circumstances. By matching the synchronization identifiers with the co-occurrence frequency distribution, it can be determined whether the current synchronization situation conforms to the normal pattern, thereby calculating the synchronization matching degree.

[0112] When performing matching analysis, similarity calculation methods can be used. For example, the synchronicity identifier can be represented as a vector, with each dimension of the vector corresponding to the synchronicity level of a feature combination; the co-occurrence frequency distribution can also be represented as a vector, with each dimension corresponding to the co-occurrence frequency of a feature combination. Then, the cosine similarity algorithm is used to calculate the similarity between these two vectors. The calculated cosine similarity value is the synchronicity matching degree. The closer the similarity value is to 1, the better the synchronicity identifier matches the co-occurrence frequency distribution, meaning the current synchronization situation is more in line with the normal pattern; the closer the similarity value is to 0, the lower the matching degree, and the more likely there may be an anomaly.

[0113] Step S324: Perform a matching analysis between the conflict identifiers in the associated feature group and the conflict frequency distribution in the reference feature pattern set, and calculate the conflict matching degree.

[0114] Conflict flags reflect the conflicts between vehicle traffic characteristics and interaction behavior characteristics, while the conflict frequency distribution in the reference feature pattern set records the frequency patterns of conflicts between these two features under normal scenarios. By matching the conflict flags with the conflict frequency distribution, we can assess whether the current conflict situation is within the normal range, and then calculate the conflict matching degree.

[0115] The same similarity calculation method is used for matching analysis. Conflict identifiers are represented as vectors, with each dimension corresponding to the conflict level of a feature combination; conflict frequency distributions are also represented as vectors, with each dimension corresponding to the conflict frequency of a feature combination. Then, the Euclidean distance algorithm is used to calculate the distance d(A,B) between these two vectors. The distance is converted into a matching degree, for example, matching degree D = 1 / (1+d(A,B)). The closer the matching degree value is to 1, the better the match between the conflict identifier and the conflict frequency distribution, meaning the current conflict situation is more in line with the normal pattern; the closer the matching degree value is to 0, the lower the matching degree, and the more likely there may be abnormal conflict situations.

[0116] Step S325: Generate scene background weight coefficients based on synchronization matching degree and conflict matching degree. The scene background weight coefficients are positively correlated with synchronization matching degree and negatively correlated with conflict matching degree.

[0117] The scene background weight coefficient is used to measure the degree of conformity between the current associated feature group and the normal scene background. Since the synchronization matching degree reflects the degree of matching between the coordination between vehicle passage features and interaction behavior features and the normal pattern, while the conflict matching degree reflects the degree of matching between the conflict situation and the normal pattern, it is necessary to consider both matching degrees to generate the scene background weight coefficient.

[0118] To generate scene background weight coefficients, pre-set weight coefficients for synchronization matching and conflict matching (these can be set according to actual needs, such as based on experiments or experience) can be used to adjust the influence of synchronization matching and conflict matching. Then, a weighted average is calculated based on these pre-set weight coefficients. Finally, the weighted synchronization matching is subtracted from the weighted conflict matching, and the result is used as the scene background weight coefficient. Based on this calculation, a higher synchronization matching degree results in a larger scene background weight coefficient, and a higher conflict matching degree results in a smaller scene background weight coefficient. This achieves a positive correlation between the scene background weight coefficient and synchronization matching degree, and a negative correlation with conflict matching degree.

[0119] Step S326: Perform weighted adjustment processing on the associated feature group based on the scene background weight coefficient to generate a context-enhanced feature group that integrates scene background information.

[0120] The weighted adjustment process aims to incorporate scene background information into the associated feature group, enabling it to better reflect the actual situation in the current traffic environment. By weighting the associated feature group with scene background weight coefficients, features with a high degree of matching with normal scene backgrounds can be highlighted, while features with a low degree of matching can be weakened.

[0121] Specifically, each feature value in the associated feature group is multiplied by a scene background weight coefficient to obtain a weighted feature value. Through this weighted adjustment process, the generated context-enhanced feature group incorporates scene background information, which can more accurately reflect the current vehicle traffic and transaction interaction situation, providing a more reliable feature input for subsequent temporal continuity detection processing.

[0122] Step S330: Perform temporal continuity detection processing on the context-enhanced feature group, extract the changing trend features of features under continuous timestamps through time window sliding analysis, and generate a temporal feature sequence that reflects the stability of the time dimension.

[0123] While context-enhanced feature sets incorporate scene background information, temporal continuity detection is still required to further analyze feature changes over time. Sliding time window analysis is a commonly used method. It defines a fixed-size time window, slides it across consecutive timestamps, and analyzes the features within the window to extract their changing trends.

[0124] When performing sliding window analysis, the first step is to determine the size of the time window. The size of the time window should be determined based on the specific application scenario and data characteristics. For example, if the data changes slowly, a larger time window can be chosen; if the data changes rapidly, a smaller time window should be chosen. Let's assume the time window size is T timestamps.

[0125] For each time window, statistics of the feature within the window, such as mean and standard deviation, are calculated. These statistics reflect the average level and fluctuation of the feature within that time window. As the time window slides across consecutive timestamps, a series of statistical values ​​are obtained. Arranging these statistical values ​​in chronological order forms a time-series feature sequence reflecting the trend of feature changes. By generating a time-series feature sequence, the trend of feature changes over consecutive timestamps can be observed to determine its stability. If the fluctuation of the time-series feature sequence is small, it indicates that the feature is relatively stable over time; if the fluctuation is large, there may be abnormal changes.

[0126] Step S340: Perform anomaly discrimination processing on the time-series feature sequence, and generate a discrimination result containing anomaly probability values ​​based on the pattern matching degree of the feature sequence according to the preset abnormal behavior pattern library.

[0127] The pre-defined abnormal behavior pattern library stores various known abnormal behavior patterns, which are derived from the analysis and summarization of a large amount of historical abnormal data. The purpose of anomaly detection processing on time-series feature sequences is to determine whether the sequence conforms to a certain pattern in the abnormal behavior pattern library, thereby determining whether an anomaly exists.

[0128] Dynamic Time Warping (DTW) can be used for pattern matching. DTW is a method for calculating the similarity between two time series, capable of handling the scaling and warping of time series along the time axis. The temporal feature sequence is matched with each pattern in an anomalous behavior pattern library using DTW, and the similarity between them is calculated.

[0129] For each matching result, an anomaly probability value can be calculated based on similarity. A common method is to convert the similarity value into anomaly probability. For example, the closer the similarity value is to 1, the more similar the temporal feature sequence is to the abnormal behavior pattern, and the higher the anomaly probability; the closer the similarity value is to 0, the less similar it is, and the lower the anomaly probability. By matching the temporal feature sequence with each pattern in the abnormal behavior pattern library and calculating the anomaly probability, a series of anomaly probability values ​​are obtained. These anomaly probability values ​​are used as part of the discrimination result, while the matching situation with each abnormal behavior pattern is recorded, forming a discrimination result that includes the anomaly probability values.

[0130] Step S350: Based on the feature sequence whose abnormal probability value exceeds the threshold in the discrimination result, match it with the preset abnormal type label library to generate an abnormal identification result containing an abnormal type identifier. The abnormal type identifier is used to distinguish between abnormal vehicle passage or abnormal transaction interaction.

[0131] The pre-defined anomaly type label library contains labels for various anomaly types and corresponding feature pattern descriptions. When the anomaly probability value in the discrimination result exceeds a pre-defined threshold (e.g., 0.8), it indicates that the feature sequence may contain anomalies. In this case, the feature sequence needs to be matched with the anomaly type label library to determine the specific anomaly type.

[0132] Matching can be achieved through feature pattern comparison. Each anomaly type in the anomaly type tag library has a corresponding feature pattern description. Feature sequences with anomaly probability values ​​exceeding a threshold are compared with these descriptions to find the best-matching anomaly type. For example, if a feature sequence shows a sudden and drastic change in vehicle traffic characteristics within a certain time period, and matches the feature pattern description of "abnormal vehicle traffic speed" in the anomaly type tag library, then the anomaly type is identified as "abnormal vehicle traffic speed." The generated anomaly identification result includes an anomaly type identifier, which clearly distinguishes between vehicle traffic anomalies and transaction interaction anomalies. The anomaly identification result can be represented as a list, where each element contains an anomaly type identifier and its corresponding anomaly probability value.

[0133] Step S400: Determine the types of abnormal events present in the original monitoring data set and the spatiotemporal distribution information of the abnormal events in the toll station scenario based on the anomaly identification results.

[0134] The anomaly identification results provide a preliminary assessment of the anomaly, including anomaly type identifiers and anomaly probability values. To gain a more comprehensive understanding of the anomaly, it is necessary to further determine its specific type and its spatiotemporal distribution within the toll station setting. The anomaly type helps staff quickly pinpoint the nature of the problem, while the spatiotemporal distribution information helps determine the exact time and location of the anomaly, enabling targeted measures to be taken.

[0135] In one implementation, step S400 specifically includes the following steps S410~S450:

[0136] Step S410: Parse the anomaly type identifier in the anomaly identification result, match it with the preset anomaly event type library, and determine the anomaly event type of the anomaly event. The anomaly event types include anomaly where the vehicle does not fully enter the toll lane, anomaly where license plate recognition is difficult, anomaly where the interactive action is interrupted, or anomaly where the transaction operation steps are missing.

[0137] The pre-defined exception event type library is a list of predefined exception event types, each with its own clear definition and characteristic description. Parsing the exception type identifier in the exception identification results involves comparing the exception type identifier recorded in the exception identification results with the types in the exception event type library to find the matching exception event type.

[0138] For example, if the anomaly type in the anomaly identification result is "vehicle_not_fully_entered", the corresponding anomaly type "vehicle not fully entered the toll lane anomaly" can be found in the preset anomaly event type library. Through this matching process, the specific type of the anomaly event is accurately determined, providing a clear target for subsequent processing and analysis.

[0139] Step S420: Extract the timestamp information of the corresponding abnormal event from the anomaly identification results, and combine it with the timestamp-aligned vehicle passage images and transaction interaction records in the original monitoring data set to determine the start and end times of the abnormal event.

[0140] The anomaly identification results include timestamp information for the corresponding anomaly events, which helps to pinpoint the approximate time range in which the anomaly occurred. Simultaneously, the vehicle passage images and transaction interaction records in the original monitoring data set, after being aligned with timestamps, provide detailed time information.

[0141] When determining the start and end times of an abnormal event, the first step is to locate the corresponding vehicle passage images and transaction interaction records in the original monitoring data set based on the timestamp information in the anomaly identification results. Then, through detailed analysis of these records, the specific start and end times of the abnormal event are identified. For example, if the abnormal event involves a vehicle not fully entering the toll lane, the start time can be determined by observing the vehicle passage images and identifying the point when the vehicle begins to enter the toll lane but does not fully enter, while the end time is determined by the point when the vehicle finally leaves the toll lane or the problem is resolved.

[0142] Step S430: Based on the start and end times of the abnormal event, extract vehicle passage images and transaction interaction records within the corresponding time interval, and analyze the spatial location of the abnormal event in the toll station scenario, including the specific location of the vehicle in the toll lane and the location of the booth area where the interaction occurred.

[0143] After determining the start and end times of the abnormal event, vehicle traffic images and transaction interaction records for that time interval are extracted from the original monitoring data set. These records contain detailed information about the abnormal event, and analysis of this information can determine the spatial location of the abnormal event within the toll station scenario.

[0144] The exact location of a vehicle within the toll lane can be determined by analyzing vehicle traffic images. Object detection and localization algorithms, such as the deep learning-based Faster R-CNN algorithm, can be used to detect and locate vehicles in the images, determining their coordinates within the toll lane. Furthermore, by combining this with the image's scale information, the coordinates can be converted into their actual physical location.

[0145] To determine the location of the toll booth area where the interaction occurred, image data from the transaction interaction records can be analyzed. By identifying and locating the positions of the toll collector and driver in the images, as well as the area where the interaction occurred, the specific toll booth area where the interaction took place can be determined.

[0146] Step S440: Based on the start and end times and spatial location of the abnormal event, construct a spatiotemporal trajectory model of the abnormal event under continuous timestamps. The spatiotemporal trajectory model is used to represent the continuous process of the abnormal event in the time dimension and the positional change in the spatial dimension.

[0147] Spatiotemporal trajectory models can intuitively display the dynamic changes of abnormal events in time and space, providing strong support for in-depth analysis of the development process and impact scope of abnormal events. Building this model based on the start and end times and spatial location of abnormal events allows for the integration and visualization of their spatiotemporal information, helping toll station managers better understand the full picture of abnormal events.

[0148] In one implementation, step S440 specifically includes the following steps S441 to S446:

[0149] Step S441: Divide the start and end times of the abnormal event into time axis segments to generate a time series containing multiple time intervals.

[0150] To analyze the changes of anomalous events over time in greater detail, it is necessary to divide the time interval consisting of the start and end times of the anomalous events. Time axis segmentation can divide this time interval into a series of equally or unequally spaced time intervals, forming a time series.

[0151] In practice, the size of the time interval can be determined based on specific needs and data characteristics. If you want to capture the details of changes in anomalies more precisely, you can choose a smaller time interval; if you only need to understand the general trend of the anomaly, you can choose a larger time interval. For example, if the duration of an anomaly is 10 minutes, it can be divided into 10 one-minute time intervals, generating a time series containing 10 time interval points: start time, start time + 1 minute, start time + 2 minutes... start time + 9 minutes, and end time.

[0152] Step S442: Extract the spatial location of the vehicle passage image and transaction interaction record corresponding to each time interval point to obtain the spatial location coordinates of the abnormal event at each time interval point.

[0153] After generating the time series, for each time interval, it is necessary to extract the corresponding vehicle passage images and transaction interaction records from the original monitoring data set, and obtain the spatial location information of the abnormal event at that time point.

[0154] For images of passing vehicles, object detection and localization algorithms (such as Mask R-CNN) can be used to identify the vehicle's location. Mask R-CNN can not only detect the vehicle's bounding box but also generate a pixel-level mask, thus determining the vehicle's location more accurately. By processing the passing vehicle image, information such as the vehicle's center point coordinates or the vertex coordinates of its bounding box can be extracted as the vehicle's spatial location coordinates at that time interval.

[0155] For interactive actions recorded in transaction logs, if the interaction involves toll collectors and drivers within the toll booth, image analysis technology can be used to identify the area where the interaction occurred and determine the center coordinates or feature point coordinates of that area as the spatial location coordinates of the interaction. For example, background subtraction algorithms can be used to remove background information from the image, highlighting the foreground area where the interaction occurs. Then, morphological operations and contour detection methods can be used to determine the location of the interaction. By processing the images and records corresponding to each time interval, the spatial location coordinates of the abnormal event at each time point can be obtained.

[0156] Step S443: Associate and map the time series with the corresponding spatial location coordinates to generate a set of time-space coordinate pairs.

[0157] Association mapping involves mapping each time interval in a time series to its corresponding spatial coordinates, creating a set of coordinate pairs containing both temporal and spatial information. This set clearly shows the spatial location of anomaly events at different times, providing foundational data for subsequent trajectory analysis. Association mapping can be implemented using data structures such as dictionaries or lists. For example, a Python dictionary can be used, storing time intervals as keys and their corresponding spatial coordinates as values. This associates the time series data with spatial coordinates, generating a set of time-space coordinate pairs, facilitating subsequent analysis and processing of the spatiotemporal trajectories of anomaly events.

[0158] Step S444: Perform trajectory smoothing on the time-space coordinate pair set to eliminate abnormal fluctuations in coordinate points and generate a continuous spatiotemporal trajectory curve.

[0159] Due to factors such as errors in image acquisition equipment, the accuracy of target detection algorithms, and environmental conditions, coordinate points in a set of time-space coordinate pairs may exhibit abnormal fluctuations. These fluctuations can affect the accuracy of spatiotemporal trajectories and the visualization effect. Therefore, trajectory smoothing processing of coordinate points is necessary to eliminate these abnormal fluctuations and generate continuous spatiotemporal trajectory curves.

[0160] Trajectory smoothing methods include moving averages, Gaussian filtering, and spline interpolation. Taking moving averages as an example, the trajectory is smoothed by calculating the local average value of coordinate points. For each coordinate point, a certain number of its immediate and subsequent neighboring points are taken, and the average of these points is calculated as the smoothed value for that point. For example, using a 3-point moving average, the smoothed coordinate value for the i-th coordinate point is the average of the (i-1), i, and i+1th coordinate points. Through trajectory smoothing, the set of time-space coordinate pairs is transformed into a continuous spatiotemporal trajectory curve, making the spatiotemporal changes of abnormal events more intuitive and easier to analyze.

[0161] Step S445: Calculate the duration characteristics of the abnormal event in the time dimension and the rate of change of its position in the spatial dimension through the spatiotemporal trajectory curve. The duration characteristic is the time difference between the end time point and the start time point, and the rate of change of position characteristic is the ratio of the difference in spatial position coordinates between adjacent time interval points to the time interval.

[0162] The spatiotemporal trajectory curve provides detailed information about anomalies in both time and space. Based on this curve, the duration and rate of change of location of the anomaly can be calculated. The duration reflects the length of time the anomaly takes from start to finish, while the rate of change of location reflects the speed at which the anomaly moves in the spatial dimension.

[0163] The duration characteristic can be calculated by subtracting the start time from the end time of the abnormal event. For example, if the start time of the abnormal event is 9:00 AM and the end time is 9:10 AM, then the duration characteristic is 10 minutes.

[0164] Calculating the rate of position change characteristics requires analyzing adjacent time intervals in the spatiotemporal trajectory curve. For each time interval, the spatial coordinate difference (which can be Euclidean distance) between it and its adjacent time intervals is calculated. This difference is then divided by the time interval to obtain the rate of position change within that time interval. By calculating the rate of position change of the anomalous event at all adjacent time intervals on the spatiotemporal trajectory curve, the rate of position change characteristics of the anomalous event in different time periods can be obtained. These characteristics can help analyze the development speed and dynamic changes of the anomalous event.

[0165] Step S446: Construct a spatiotemporal trajectory model based on the spatiotemporal trajectory curve, duration characteristics, and location change rate characteristics.

[0166] A spatiotemporal trajectory model is a comprehensive model that integrates various characteristic information of anomalous events in the temporal and spatial dimensions, including spatiotemporal trajectory curves, duration characteristics, and location change rate characteristics. By combining and encapsulating these features, a complete spatiotemporal trajectory model can be constructed to describe the spatiotemporal changes of anomalous events over consecutive timestamps.

[0167] The spatiotemporal trajectory curve, duration characteristics, and location change rate characteristics can be stored in a data structure, such as a Python class or dictionary, serving as a concrete implementation of the spatiotemporal trajectory model. In practical applications, by creating instances of this class and passing in the spatiotemporal trajectory curve, duration characteristics, and location change rate characteristics, a specific spatiotemporal trajectory model can be constructed. By analyzing the duration characteristics and location change rate characteristics, the severity and scope of impact of abnormal events can be determined, allowing for appropriate measures to be taken.

[0168] Step S450: Generate spatiotemporal distribution information containing time interval range and spatial location coordinates through the spatiotemporal trajectory model.

[0169] The spatiotemporal trajectory model already contains detailed information about the anomalous event in both time and space. By processing this model, spatiotemporal distribution information containing time intervals and spatial coordinates can be generated. The time intervals specify the time period in which the anomalous event occurred, while the spatial coordinates show the specific location of the anomalous event at different points in time.

[0170] For time intervals, the start and end times of the abnormal event can be directly obtained from the spatiotemporal trajectory model and combined into a time interval. For example, if the start time is 9:00 AM and the end time is 9:10 AM, then the time interval is [9:00 AM, 9:10 AM].

[0171] For spatial coordinates, the spatial coordinates corresponding to each time point can be extracted from the spatiotemporal trajectory curve of the spatiotemporal trajectory model. These coordinates can be arranged in chronological order to form a spatial coordinate sequence. For example, the spatiotemporal trajectory curve may contain the spatial coordinates of an abnormal event at 10 time points; arranging these coordinates sequentially yields a sequence containing 10 coordinates. Combining the time interval range and the spatial coordinate sequence forms spatiotemporal distribution information. This information can be stored and displayed in the form of a list, dictionary, or other data structures.

[0172] As one implementation method, the method provided in this embodiment of the invention further includes generating a monitoring optimization instruction containing an event location identifier based on the abnormal event type and spatiotemporal distribution information, and sending the monitoring optimization instruction to the toll station management terminal to perform management adjustment operations, specifically including the following steps S510~S540:

[0173] Step S510: Parse the preset optimization strategy library corresponding to the abnormal event type, and extract the optimization strategy identifier and adjustment parameter information associated with the abnormal event type.

[0174] The preset optimization strategy library is a predefined database that stores optimization strategies and adjustment parameter information corresponding to different types of abnormal events. Each type of abnormal event has its own specific optimization strategy, used to resolve or improve the problems caused by that type of abnormal event.

[0175] When parsing the preset optimization strategy library, the system first searches for the corresponding record in the library based on the type of abnormal event. For example, if the abnormal event type is "vehicle not fully entering the toll lane," the corresponding record for that type is found in the preset optimization strategy library. This record contains the optimization strategy identifier and adjustment parameter information associated with that abnormal event type. The optimization strategy identifier can be a string or a number, used to uniquely identify the optimization strategy. For example, "OP-001" indicates adjusting the shooting angle of the image acquisition device to better monitor vehicle entry. The adjustment parameter information details the specific parameters required to execute the optimization strategy, such as the specific degree of adjustment of the shooting angle and the specific content of the updated transaction operation process prompts.

[0176] Step S520: Generate an event location identifier that includes a time interval identifier and a spatial location identifier based on the time interval range and spatial location coordinates in the spatiotemporal distribution information.

[0177] The time interval range and spatial coordinates in the spatiotemporal distribution information provide detailed information for accurately determining the location and time of anomalies. To facilitate the location and management of anomalies, it is necessary to generate event location identifiers based on this information.

[0178] A time interval identifier can be an encoded representation of a time interval range to facilitate processing and recognition by computer systems. For example, a time interval range can be converted into a timestamp range string, such as "2024-01-01 09:00:00-2024-01-01 09:10:00".

[0179] Spatial location identification can be a simplified or encoded representation of spatial location coordinates. Based on the specific layout and coordinate system of the toll station, spatial location coordinates can be converted into a designated area number or location code. For example, if the toll station is divided into multiple areas, each with a unique number, the area number can be used as the spatial location identifier to determine which area the abnormal event occurred in.

[0180] By combining a time interval identifier and a spatial location identifier, an event location identifier is formed. For example, an event location identifier could be “T202401010900-0910-S003”, where “T202401010900-0910” represents the time interval identifier and “S003” represents the spatial location identifier. By generating event location identifiers, abnormal events can be located more clearly.

[0181] Step S530: Integrate the optimization strategy identifier, adjustment parameter information, and event location identifier to generate a monitoring optimization instruction that includes the optimization strategy type, adjustment parameter value, and event location.

[0182] Information integration processing combines and encapsulates optimization strategy identifiers, adjustment parameter information, and event location identifiers to form a complete monitoring and optimization command. This command contains all the key information required to execute management and adjustment operations, including the optimization strategy type, adjustment parameter values, and event location.

[0183] The optimization strategy type can be parsed from the optimization strategy identifier. For example, if the optimization strategy identifier is "OP-001", according to the preset encoding rules, the optimization strategy type corresponding to "OP-001" is to adjust the shooting angle of the image acquisition device.

[0184] Adjustment parameter values ​​are obtained directly from the adjustment parameter information. For example, the adjustment parameter information might include a specific angle adjustment of 30 degrees; this value will be used as the adjustment parameter value.

[0185] The event location uses the event location identifier generated in step S520.

[0186] By combining optimization strategy types, parameter adjustments, and event locations, monitoring optimization instructions can be represented using JSON format. Through this information integration process, the generated monitoring optimization instructions contain clear optimization strategies and adjustment information, as well as the precise location and time of the abnormal event.

[0187] Step S540: The monitoring optimization instruction is sent to the toll station management terminal through the internal communication network of the toll station. The toll station management terminal executes the corresponding management adjustment operation according to the optimization strategy type and adjustment parameter value in the monitoring optimization instruction. The management adjustment operation includes adjusting the shooting angle of the image acquisition device, updating the transaction operation process prompt information, and triggering the manual review process of abnormal events.

[0188] The toll station's internal communication network is a dedicated network for communication between devices within the toll station. It ensures that monitoring and optimization commands can be securely and reliably transmitted to the toll station management terminal. The toll station management terminal is responsible for receiving and processing monitoring and optimization commands. It executes corresponding management and adjustment operations based on the optimization strategy type and adjustment parameter values ​​in the command.

[0189] Before sending monitoring optimization instructions, the instructions need to undergo a series of processing steps to ensure their accuracy and completeness.

[0190] In one implementation, step S540 specifically includes the following steps S541 to S546:

[0191] Step S541: Perform data integrity verification on the monitoring optimization command and generate a verification identifier containing the hash value of the command content.

[0192] Data integrity verification is performed to ensure that monitoring and optimization instructions are not tampered with or corrupted during transmission. A hash value is a fixed-length string obtained by encrypting data content; it is unique, meaning different data content will produce different hash values. By calculating the hash value of the monitoring and optimization instructions, the integrity of the instructions can be verified at the receiving end. Hash algorithms (such as the SHA-256 algorithm) can be used to calculate the hash value of the instruction content.

[0193] Step S542: Encapsulate the monitoring optimization instructions and verification identifiers to generate an instruction data packet to be transmitted.

[0194] Data encapsulation processing combines monitoring optimization instructions and verification identifiers into a complete instruction data packet for transmission within the toll station's internal communication network.

[0195] Data can be encapsulated using JSON format. The monitoring optimization instructions and verification identifiers are stored as two fields within a single JSON object. This data encapsulation ensures that the instruction data packet contains monitoring optimization instructions along with identification information used to verify their integrity, guaranteeing the accuracy and verifiability of the transmitted data. During the encapsulation process, it is crucial to ensure that each field is formatted correctly and that the data is accurate so that the receiving end can correctly parse and process the data packet.

[0196] Step S543: Send the instruction data packet to the receiving interface of the toll station management terminal through the dedicated transmission channel of the toll station's internal communication network.

[0197] The dedicated transmission channel of the internal communication network of the toll station is set up to ensure the stable, secure, and efficient transmission of instruction data packets. This channel can use wired or wireless network technology, selected according to the actual layout and needs of the toll station.

[0198] Step S544: After receiving the instruction data packet at the toll station management terminal, the instruction data packet is decapsulated to extract the monitoring optimization instruction and verification identifier.

[0199] When the toll station management terminal receives the instruction data packet, it first performs decapsulation processing. This is to separate the previously encapsulated monitoring optimization instructions and verification identifiers for subsequent processing and verification. The decapsulation process is the reverse of the rules used during the initial encapsulation. Since instruction data packets are usually encapsulated in JSON format, the management terminal uses a corresponding JSON parsing tool to parse the data packet. The parsing tool identifies the various fields in the data packet and extracts the monitoring optimization instructions and verification identifiers separately.

[0200] Step S545: Recalculate the hash value of the extracted monitoring optimization instruction and perform a consistency comparison with the hash value in the verification identifier. If the comparison results are consistent, the instruction is marked as valid; if the comparison results are inconsistent, the instruction is marked as invalid and a retransmission request is triggered.

[0201] To ensure that the received monitoring optimization instructions have not been tampered with or corrupted during transmission, integrity verification is required. This verification involves recalculating the hash value of the extracted monitoring optimization instructions and comparing it with the hash value in the verification identifier.

[0202] When recalculating the hash value, the same hash algorithm as the sender (such as SHA-256) is used. The extracted monitoring optimization instructions are encoded according to the specified encoding format (such as UTF-8), and then hashed to obtain a new hash value. This new hash value is then compared with the hash value in the verification identifier.

[0203] If the two hash values ​​are exactly the same, it means the monitoring optimization instruction has not changed during transmission, and the instruction is marked as valid and can be executed subsequently. Conversely, if the two hash values ​​are inconsistent, it means the instruction may have encountered a problem during transmission, such as interference or tampering, and in this case, the instruction is marked as invalid. The management terminal will trigger a retransmission request, sending a request to the sender via the toll station's internal communication network to obtain the correct monitoring optimization instruction.

[0204] Step S546: For valid instructions, parse the optimization strategy type and adjustment parameter values ​​in the monitoring optimization instruction, call the preset strategy execution module, and start the corresponding execution process according to the optimization strategy type: if the optimization strategy type is image acquisition device adjustment, then adjust the shooting angle parameters of the first image acquisition device or the second image acquisition device; if the optimization strategy type is transaction process prompt update, then update the transaction operation prompt information on the toll booth display screen; if the optimization strategy type is manual review trigger, then send a review notification information containing the event location identifier to the designated management personnel terminal.

[0205] Once a monitoring optimization command is verified as valid, the toll station management terminal will further analyze it. The analysis process mainly involves extracting two key pieces of information from the command: the optimization strategy type and the adjusted parameter values. The optimization strategy type clarifies the specific operational direction to be executed, while the adjusted parameter values ​​provide a precise basis for the specific operation.

[0206] The preset strategy execution modules are a series of pre-developed program modules, each corresponding to a different optimization strategy type. Based on the parsed optimization strategy type, the management terminal will call the corresponding strategy execution module to start the execution process.

[0207] If the optimization strategy type is image acquisition device adjustment, the strategy execution module will communicate with either the first or second image acquisition device. Through the device's control interface, adjustment parameter values ​​(such as specific shooting angles in degrees) are sent to the device. Upon receiving the instruction, the device will automatically adjust the shooting angle to meet monitoring requirements. For example, if the license plate area in a vehicle traffic image is found to be unclear, it may be necessary to adjust the shooting angle of the first image acquisition device to enable it to capture the license plate more clearly.

[0208] If the optimization strategy type is transaction process prompt update, the strategy execution module will interact with the toll booth display screen. It will send the new transaction operation prompts included in the adjusted parameter values ​​to the display screen, which will then update its content to provide toll collectors and drivers with the latest transaction operation guidance. For example, timely updates to the prompts can prevent toll collector errors and improve transaction efficiency when the transaction process changes.

[0209] If the optimization strategy type is triggered by manual review, the strategy execution module will send a review notification message containing an event location identifier to the designated administrator's terminal. This notification message can be sent via SMS, instant messaging software, or a dedicated management system message. After receiving the notification, the administrator's terminal can quickly understand the time and location of the abnormal event based on the event location identifier, and promptly perform manual review of the abnormal event to ensure that the problem is properly handled.

[0210] It is understood that the various algorithms involved in the above descriptions of the embodiments of the present invention, such as Euclidean distance algorithm, cosine distance algorithm, hash algorithm, etc., can all be obtained from relevant content in the prior art. To save space, they will not be elaborated on in the embodiments of the present invention. In addition, those skilled in the art can supplement the details based on common knowledge in the art when implementing the solution of the present invention. For example, based on common knowledge in the art, normalization can be used to eliminate dimensional conflicts before feature fusion, interpolation can be used to eliminate dimensional differences, thresholds can be reasonably set in combination with historical data, experience or business scenario requirements, the model can be trained based on a general model training method, the number of layers in the model structure can be set according to actual needs, activation functions can be selected, etc. The present invention will not provide redundant descriptions of overly detailed implementation processes.

[0211] Please see Figure 2 , Figure 2 This is a schematic diagram of a monitoring data analysis device provided in an embodiment of the present invention. The monitoring data analysis device can be a computer system installed at a toll station or a server communicating remotely with the monitoring device. The monitoring data analysis device includes at least a processor 101, a communication interface 102, and a memory 103. The processor 101, communication interface 102, and memory 103 can be connected via a bus or other means. The processor 101 (or Central Processing Unit, CPU) is the computing and control core of the monitoring data analysis device, capable of parsing various instructions and processing various data within the monitoring data analysis device. The communication interface 102 may optionally include a standard wired interface or a wireless interface (such as Wi-Fi, mobile communication interface, etc.), and can be used to send and receive data under the control of the processor 101; the communication interface 102 can also be used for the transmission and interaction of data within the monitoring data analysis device. The memory 103 is a storage device in the monitoring data analysis device, used to store programs and data. It is understood that the memory 103 here may include the built-in memory of the monitoring data analysis device, or it may include the extended memory supported by the monitoring data analysis device. The memory 103 provides storage space for the operating system of the monitoring data analysis device, which is not limited in this respect. In one embodiment, the processor 101 executes the monitoring data analysis method for digital toll stations provided in the above embodiments of the present invention by running the computer program in the memory 103.

Claims

1. A monitoring data analysis method applied to digital toll stations, characterized in that, The method includes: Obtain the original monitoring data set within the toll station area, wherein the original monitoring data set includes continuously collected vehicle passage images with timestamps and corresponding transaction interaction records; The original monitoring data set is subjected to feature extraction processing to obtain the vehicle passage features of the vehicle passage image and the interaction behavior features of the transaction interaction record; The process involves calling a pre-trained anomaly analysis model to perform anomaly collaborative recognition processing on the vehicle passage features and the interaction behavior features, generating anomaly recognition results for the vehicle passage images and transaction interaction records. Specifically, this includes: inputting the vehicle passage features and the interaction behavior features into the association mapping module of the anomaly analysis model; generating an association feature group reflecting the strength of the association between the two through feature co-occurrence relationship analysis; the association feature group containing synchronization and conflict indicators of vehicle features and interaction features; extracting a subset of historical data from the toll station's historical monitoring database that matches the current original monitoring data set's passage environment; performing reference pattern mining processing on the historical data subset to extract the co-occurrence frequency distribution and conflict frequency distribution of vehicle passage features and interaction behavior features under normal scenarios, generating a reference feature pattern set; performing matching analysis between the synchronization indicators in the association feature group and the co-occurrence frequency distribution in the reference feature pattern set to calculate the synchronization matching degree; and performing matching analysis between the conflict indicators in the association feature group and the reference feature pattern set. The system performs a matching analysis on the conflict frequency distribution in the data to calculate the conflict matching degree. Based on the synchronization matching degree and the conflict matching degree, it generates a scene background weight coefficient, which is positively correlated with the synchronization matching degree and negatively correlated with the conflict matching degree. Based on the scene background weight coefficient, it performs a weighted adjustment on the associated feature group to generate a context-enhanced feature group that integrates scene background information. It then performs temporal continuity detection on the context-enhanced feature group, extracting the trend features of feature changes under continuous timestamps through time window sliding analysis to generate a temporal feature sequence reflecting the stability of the time dimension. Finally, it performs anomaly discrimination on the temporal feature sequence, matching the pattern matching degree of the feature sequence based on a preset abnormal behavior pattern library to generate a discrimination result containing anomaly probability values. Based on the feature sequences in the discrimination result whose anomaly probability values ​​exceed a threshold, it matches them with a preset anomaly type label library to generate an anomaly identification result containing anomaly type identifiers, which are used to distinguish between abnormal vehicle passage and abnormal transaction interaction. Based on the anomaly identification results, determine the types of abnormal events present in the original monitoring data set and the spatiotemporal distribution information of the abnormal events in the toll station scenario.

2. The monitoring data analysis method applied to digital toll stations according to claim 1, characterized in that, The acquisition of the original monitoring data set within the toll station area includes: The first image acquisition device deployed at the entrance of the toll station continuously acquires image data of vehicles entering the toll station, and generates vehicle passage images with timestamps. The vehicle passage images include the overall outline information of the vehicle and local information of the license plate area. By continuously collecting image data of the interaction process between the toll collector and the driver through the second image acquisition device deployed at the toll booth, and combining it with the transaction operation log data recorded by the toll terminal, a transaction interaction record with a timestamp is generated. The transaction interaction record includes the sequence information of the interaction actions and the transaction operation steps. The vehicle passage images and transaction interaction records are timestamped to ensure that vehicle passage images and transaction interaction records at the same time point have a corresponding relationship. The timestamp-aligned vehicle passage images and transaction interaction records are integrated into the original monitoring data set. Each data unit in the original monitoring data set includes a vehicle passage image subunit, a transaction interaction record subunit, and corresponding timestamp information.

3. The monitoring data analysis method applied to digital toll stations according to claim 1, characterized in that, The step of performing feature extraction processing on the original monitoring data set to obtain vehicle passage features of the vehicle passage images and interaction behavior features of the transaction interaction records includes: The vehicle traffic image is subjected to noise reduction and contrast enhancement processing to obtain a preprocessed vehicle traffic image. Feature extraction processing is performed on the preprocessed vehicle passage image to identify the integrity features of the vehicle outline, the clarity features of the license plate area, and the positional relationship features between the vehicle and the toll lane boundary. The integrity features of the vehicle outline, the clarity features of the license plate area, and the positional relationship features between the vehicle and the toll lane boundary are integrated to obtain the vehicle passage features. The transaction interaction record is subjected to behavior sequence parsing processing to extract the continuity features of the interaction actions, the integrity features of the transaction operation steps, and the time matching features between the interaction actions and the transaction operation steps. The continuity features of the interaction actions, the integrity features of the transaction operation steps, and the time matching features between the interaction actions and the transaction operation steps are integrated to obtain the interaction behavior features. The vehicle passage features and the interaction behavior features are aligned in terms of feature dimensions so that they have the same feature dimension representation.

4. The monitoring data analysis method applied to digital toll stations according to claim 3, characterized in that, The preprocessed vehicle passage image undergoes feature extraction processing to identify the integrity features of the vehicle outline, the clarity features of the license plate area, and the positional relationship features between the vehicle and the toll lane boundary. These features are then integrated to obtain the vehicle passage features, including: Edge detection processing is performed on the preprocessed vehicle traffic image to extract the set of boundary pixels of the vehicle outline, and the continuous coverage ratio of the set of boundary pixels in the image is calculated as the integrity feature of the vehicle outline. Local feature enhancement processing is performed on the license plate area in the vehicle passage image to extract the identifiability features of the license plate characters, which are used as the clarity features of the license plate area. Semantic segmentation processing is performed on the toll lane boundary in the vehicle passage image to extract the relative position coordinates between the vehicle outline boundary and the toll lane boundary, and the distance difference feature between the two is calculated as the positional relationship feature between the vehicle and the toll lane boundary. The vehicle's outline integrity features, the license plate area clarity features, and the positional relationship features between the vehicle and the toll lane boundary are combined to generate the vehicle passage features.

5. The monitoring data analysis method applied to digital toll stations according to claim 3, characterized in that, The process involves parsing the transaction interaction records to extract the continuity features of the interaction actions, the completeness features of the transaction operation steps, and the time matching features between the interaction actions and the transaction operation steps. These features are then integrated to obtain the interaction behavior features, including: The interaction action sequence in the transaction interaction record is divided into multiple consecutive interaction action subsequences by time axis, and the uniformity feature of the time interval between adjacent actions in each interaction action subsequence is calculated as the continuity feature of the interaction action. The transaction operation log in the transaction interaction record is subjected to step matching processing to identify the completion status characteristics of the preset standard operation steps, which are used as the integrity characteristics of the transaction operation steps. The overlap ratio between the time interval of each interactive action subsequence and the execution time interval of the corresponding transaction operation step is extracted as the time matching feature between the interactive action and the transaction operation step. The interaction behavior features are generated by performing feature fusion processing on the continuity features of the interaction actions, the integrity features of the transaction operation steps, and the time matching features between the interaction actions and the transaction operation steps.

6. The monitoring data analysis method applied to digital toll stations according to claim 1, characterized in that, The step of determining the types of abnormal events present in the original monitoring data set and the spatiotemporal distribution information of the abnormal events in the toll station scenario based on the anomaly identification results includes: The abnormal type identifier in the abnormal identification result is analyzed, matched with a preset abnormal event type library, and the abnormal event type is determined. The abnormal event type includes abnormality of vehicle not fully entering the toll lane, abnormality of license plate recognition difficulty, abnormality of interrupted interactive action, or abnormality of missing transaction operation steps. Extract the timestamp information of the corresponding abnormal event from the anomaly identification results, and combine it with the timestamp-aligned vehicle passage images and transaction interaction records in the original monitoring data set to determine the start and end times of the abnormal event. Based on the start and end times of the abnormal event, vehicle passage images and transaction interaction records within the corresponding time interval are extracted, and the spatial location of the abnormal event in the toll station scenario is analyzed, including the specific location of the vehicle in the toll lane and the location of the booth area where the interaction occurred. Based on the start and end times and spatial location of the abnormal event, a spatiotemporal trajectory model of the abnormal event under continuous timestamps is constructed. The spatiotemporal trajectory model is used to represent the continuous process of the abnormal event in the time dimension and the positional change in the spatial dimension. The spatiotemporal trajectory model generates spatiotemporal distribution information containing time interval ranges and spatial location coordinates.

7. The monitoring data analysis method applied to digital toll stations according to claim 6, characterized in that, The construction of a spatiotemporal trajectory model of the abnormal event under continuous timestamps, based on the start and end times and spatial location of the abnormal event, includes: The start and end times of the abnormal event are divided into time axis segments to generate a time series containing multiple time intervals. Spatial location extraction is performed on the vehicle traffic images and transaction interaction records corresponding to each time interval point to obtain the spatial location coordinates of the abnormal event at each time interval point; The time series is associated with the corresponding spatial location coordinates to generate a set of time-space coordinate pairs. The time-space coordinate pair set is subjected to trajectory smoothing processing to eliminate abnormal fluctuations in coordinate points and generate a continuous spatiotemporal trajectory curve. The duration of the abnormal event in the time dimension and the rate of change of its position in the spatial dimension are calculated using the spatiotemporal trajectory curve. The duration is the time difference between the end time point and the start time point, and the rate of change of position is the ratio of the difference in spatial coordinates between adjacent time interval points to the time interval. The spatiotemporal trajectory model is constructed based on the spatiotemporal trajectory curve, duration characteristics, and location change rate characteristics.

8. A monitoring data analysis device, characterized in that, include: A memory, wherein a computer program is stored; A processor is configured to load the computer program to implement the monitoring data analysis method for digital toll stations as described in any one of claims 1-7.

Images