A method and system for monitoring electric vehicle violations based on positioning and image fusion

Abstract

The application relates to the technical field of electric vehicle violation detection, and provides an electric vehicle violation monitoring method and system based on positioning and image fusion, which is applied to campus electric vehicle management and comprises the following steps: arranging RFID readers and cameras, and installing RFID tags on campus electric vehicles; collecting RFID signal data, processing the RFID signal data, obtaining electric vehicle spatial position coordinates, performing filtering processing and trajectory optimization on the electric vehicle spatial position coordinates, and obtaining RFID positioning data; collecting video image data, extracting position information, license plate information and behavior characteristic data of the electric vehicle; performing data fusion based on a multi-level fusion method to obtain real-time positioning fusion data; establishing a violation behavior judgment model; identifying the real-time positioning fusion data through the violation behavior judgment model to generate violation records of the campus electric vehicle. The application realizes automatic identification of campus electric vehicle violation behaviors and association of rider identities, and improves the management efficiency of campus electric vehicle violation monitoring.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electric vehicle violation detection, and particularly relates to an electric vehicle violation monitoring method and system based on positioning and image fusion. BACKGROUND

[0002] With the rapid growth of the number of electric vehicles on campus, safety hazards such as illegal driving are increasingly prominent. Currently, campus electric vehicle management mainly relies on manual patrol or a single video monitoring system. The traditional video monitoring system is easily affected by environmental factors such as light and weather, resulting in unstable monitoring effect. Although a single RFID positioning system can achieve electric vehicle positioning, the positioning accuracy is limited and vehicle behavior characteristics cannot be obtained. The existing monitoring systems generally lack effective data fusion mechanisms and are difficult to comprehensively analyze monitoring data from different sources, resulting in low accuracy of illegal behavior determination.

[0003] In the prior art, the independence of RFID positioning data and video image data leads to inaccurate monitoring of electric vehicle behavior and determination of illegal behavior, especially in complex scenarios such as multiple electric vehicles passing through the monitoring area at the same time, which easily causes data inconsistency and behavior recognition errors. SUMMARY

[0004] Therefore, the present application provides an electric vehicle violation monitoring method and system based on positioning and image fusion to solve the problem of inaccurate monitoring of electric vehicle behavior and determination of illegal behavior in the prior art, which easily causes data inconsistency and behavior recognition errors in complex scenarios.

[0005] The technical solution of the present application is as follows: In a first aspect, the present application provides an electric vehicle violation monitoring method based on positioning and image fusion, applied to campus electric vehicle management, comprising the following steps:

[0006] RFID readers and cameras are arranged in the campus monitoring area, and RFID tags are installed on the campus electric vehicles;

[0007] RFID signal data of the campus electric vehicles are collected by the RFID readers, the RFID signal data are processed by a triangular positioning algorithm to obtain spatial position coordinates of the electric vehicles, the spatial position coordinates of the electric vehicles are filtered and optimized to obtain RFID positioning data;

[0008] Video image data are collected by the cameras, the video image data are preprocessed and target detected to extract position information, license plate information and behavior characteristic data of the electric vehicles;

[0009] The RFID positioning data and the location information are spatio-temporally aligned, and combined with license plate information and behavior characteristic data, data fusion is performed based on a multi-level fusion method to obtain real-time positioning fusion data of the campus electric vehicle;

[0010] A violation behavior judgment model is established based on historical positioning fusion data;

[0011] The real-time positioning fusion data is identified through the violation behavior judgment model, and a violation record of the campus electric vehicle is generated and associated with rider identity information.

[0012] On the basis of the above technical solutions, preferably, the RFID readers and cameras are arranged in the monitoring area of the campus, and the RFID tags are installed on the campus electric vehicles, and specifically comprising:

[0013] According to the passing route of the campus electric vehicle and the frequently violated area, the range of the monitoring area is determined, the RFID readers and cameras are arranged in the monitoring area, the coverage areas of the RFID readers overlap with each other, and the visual range of the cameras covers the monitoring area;

[0014] When the campus electric vehicle is registered, the license plate information and rider identity information of the campus electric vehicle are collected, and the RFID tags with unique identification codes are installed at fixed positions on the body of the campus electric vehicle.

[0015] On the basis of the above technical solutions, preferably, the RFID signal data of the campus electric vehicle is collected through the RFID reader, the RFID signal data is processed through a triangulation positioning algorithm to obtain the spatial position coordinates of the electric vehicle, the spatial position coordinates of the electric vehicle are filtered and optimized to obtain the RFID positioning data, and specifically comprising:

[0016] The RFID signal data of the campus electric vehicle is collected through multiple RFID readers simultaneously, and the spatial position coordinates of the campus electric vehicle are calculated based on a triangulation positioning algorithm;

[0017] The calculation formula of the spatial position coordinates is:

[0018]

[0019] d i =c·Δt i ;

[0020]

[0021] Wherein, (x, y, z) is the spatial position coordinates of the campus electric vehicle, N is the number of RFID readers participating in positioning, (x i ,y i ,z i) is the spatial coordinate of the ith RFID reader, d i is the distance between the ith RFID reader and the target RFID tag, Δt i is the signal read-write time difference of the ith RFID reader, c is the signal propagation speed, w i is the signal strength weight coefficient of the ith RFID reader, RSSI i is the signal strength value received by the ith RFID reader;

[0022] The spatial position coordinates are filtered by using a Kalman filtering algorithm, and trajectory smoothing optimization is performed in combination with historical trajectory data to obtain RFID positioning data;

[0023] The spatial position coordinates are filtered based on a Kalman filtering algorithm, an adaptive factor is introduced to dynamically adjust the process noise covariance matrix, a cubic spline interpolation method is used to smooth the historical trajectory data, and the trajectory curve is optimized in combination with motion constraints;

[0024] The calculation formula of the adaptive factor is:

[0025]

[0026] where λ1 is the adaptive factor, α1 is the basic weight coefficient, β1 is the attenuation coefficient, is the residual vector, and σ1 is the measurement noise standard deviation.

[0027] On the basis of the above technical solutions, preferably, the video image data is collected by a camera, preprocessed and target detected, and position information, license plate information and behavior feature data of the electric vehicle are extracted, specifically including:

[0028] The video image data is preprocessed, and the preprocessing includes image enhancement, noise suppression and illumination compensation, and the electric vehicle target is detected by a YOLOv5 deep learning model;

[0029] Based on the detected target region, the position information, license plate information and behavior feature data of the campus electric vehicle are extracted by a multi-branch deep neural network.

[0030] On the basis of the above technical solutions, preferably, the RFID positioning data and the position information are spatio-temporally aligned, and the license plate information and behavior feature data are combined to perform data fusion based on a multi-level fusion method to obtain real-time positioning fusion data of the campus electric vehicle, specifically including:

[0031] The RFID positioning data and the position information are spatio-temporally aligned;

[0032] The license plate information and behavior characteristic data and the spatiotemporal aligned RFID positioning data are fused based on a multi-level fusion method to obtain real-time positioning fusion data of the campus electric vehicle.

[0033] Based on the above technical solutions, preferably, the historical positioning fusion data is used to establish a violation behavior judgment model, which specifically includes:

[0034] The historical positioning fusion data is time-sequentially segmented and processed for abnormal values, spatiotemporal characteristics, speed characteristics, trajectory characteristics and behavior mode characteristics of the historical positioning fusion data are extracted, a sliding time window method is used to construct a violation behavior characteristic sequence, principal component analysis and a feature selection algorithm are used to optimize a feature vector to obtain a violation behavior feature vector, and a violation behavior sample library is established.

[0035] The calculation formula of the violation behavior feature vector is:

[0036]

[0037] wherein, is a violation behavior feature vector at a tt moment, w is a time window size, λ b is a time attenuation factor, P b , V b , T b , B b are position characteristics, speed characteristics, trajectory characteristics and behavior characteristics respectively.

[0038] Based on the violation behavior sample library, a long short-term memory network structure is used to construct a time sequence judgment model, a violation behavior judgment model is obtained by introducing an attention mechanism and a residual connection through the time sequence judgment model.

[0039] The weight calculation formula of the attention mechanism is:

[0040]

[0041] wherein, is an attention weight at a tt moment, h tt is a hidden state at the tt moment, s tt-1 is a state at a tt-1 moment, W c is an attention mapping weight matrix, W h is a hidden state conversion weight matrix, W s is a historical state weight matrix, is an attention bias.

[0042] The calculation formula of the violation behavior judgment model is:

[0043]

[0044] wherein, P(y tt | F tt ) is the probability of the campus electric vehicle violating the behavior y tt at the time tt under the condition of the given behavior feature vector F tt , LSTM(F tt ) is the output of the long short-term memory network processing of the behavior feature vector F tt , W q is the output layer weight matrix, is the output layer bias, is the attention weight at the time tt, and h tt is the hidden state at the time tt.

[0045] On the basis of the above technical scheme, preferably, the real-time positioning fusion data is identified by the violation behavior judgment model, the violation record of the campus electric vehicle is generated, and the rider identity information is associated, and specifically, the method comprises the following steps:

[0046] The real-time positioning fusion data is input into the violation behavior judgment model to obtain the violation behavior type and the violation probability, and when the violation probability exceeds a preset threshold, a violation event is triggered;

[0047] The violation record is generated based on the violation event, the violation record comprises the violation time, the violation location, the violation type, and the evidence information, and the rider identity information is associated through a double verification mode of RFID tag number and license plate information.

[0048] In a second aspect, the application further provides an electric vehicle violation monitoring system based on positioning and image fusion, the system comprising:

[0049] A tag monitoring module is configured to arrange an RFID reader and a camera in a campus monitoring area, and install an RFID tag on a campus electric vehicle;

[0050] A signal acquisition module is configured to acquire RFID signal data of the campus electric vehicle through the RFID reader, process the RFID signal data through a triangular positioning algorithm to obtain spatial position coordinates of the electric vehicle, and perform filtering processing and trajectory optimization on the spatial position coordinates of the electric vehicle to obtain RFID positioning data;

[0051] A feature extraction module is configured to acquire video image data through the camera, pre-process and target detect the video image data, and extract position information, license plate information, and behavior feature data of the electric vehicle;

[0052] A data fusion module is configured to perform spatio-temporal alignment on the RFID positioning data and the position information, combine the license plate information and the behavior feature data, perform data fusion based on a multi-level fusion method, and obtain real-time positioning fusion data of the campus electric vehicle.

[0053] The model building module is used to build a violation judgment model based on historical location fusion data.

[0054] The violation identification module is used to identify the real-time location fusion data through the violation behavior judgment model, generate violation records of campus electric vehicles, and associate them with rider identity information.

[0055] Thirdly, the present invention also provides an electronic device, comprising: at least one processor, at least one memory, a communication interface, and a bus;

[0056] The processor, memory, and communication interface communicate with each other through the bus. The memory stores program instructions that can be executed by the processor. The processor calls the program instructions to implement the steps of a method for monitoring electric vehicle violations based on positioning and image fusion.

[0057] Fourthly, the present invention also provides a computer-readable storage medium storing computer instructions that enable a computer to perform steps such as those in a method for monitoring electric vehicle violations based on positioning and image fusion.

[0058] The electric vehicle violation monitoring method and system based on localization and image fusion of the present invention has the following advantages over the prior art:

[0059] (1) By using RFID readers and cameras in synergy, combining triangulation algorithm, filtering and trajectory optimization to obtain accurate positioning data, and at the same time obtaining location information, license plate information and behavioral feature data through image processing, multi-level fusion method is adopted to fuse multi-source data, and a violation judgment model is established based on historical data, realizing automatic identification of campus electric vehicle violations and rider identity association, and improving the management efficiency of campus electric vehicle violation monitoring.

[0060] (2) By working together with multiple RFID readers and calculating signal strength weights, combined with the triangulation algorithm, and introducing the Kalman filter algorithm with adaptive factor dynamic adjustment to filter the coordinates, and using the cubic spline interpolation method combined with motion constraints to optimize trajectory smoothing, the accurate calculation of spatial position is realized, and the accuracy of RFID positioning data is improved.

[0061] (3) By performing time-series segmentation and outlier processing on historical location fusion data, multi-dimensional features are extracted using the sliding time window method, and a time-series judgment model is constructed using a long short-term memory network. Attention mechanism and residual connection are introduced to enhance model performance. At the same time, cross-validation and early stopping strategies are used to prevent overfitting, thereby realizing the identification and judgment of violations and improving the reliability of violation judgment. Attached Figure Description

[0062] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 This is a flowchart of a method for monitoring electric vehicle violations based on localization and image fusion according to the present invention.

[0064] Figure 2 This is a structural diagram of an electric vehicle violation monitoring system based on positioning and image fusion according to the present invention. Detailed Implementation

[0065] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0066] Please see Figure 1 This invention provides a method for monitoring electric vehicle violations based on localization and image fusion, applicable to campus electric vehicle management, comprising the following steps:

[0067] RFID readers and cameras are deployed in the campus monitoring area, and RFID tags are installed on electric vehicles on campus.

[0068] RFID signal data of electric vehicles on campus are collected by RFID readers. The RFID signal data is processed by a triangulation algorithm to obtain the spatial coordinates of the electric vehicle. The spatial coordinates of the electric vehicle are then filtered and the trajectory is optimized to obtain RFID positioning data.

[0069] Video image data is collected by a camera, and the video image data is preprocessed and target detection is performed to extract the location information, license plate information and behavioral feature data of the electric vehicle.

[0070] The RFID positioning data and the location information are spatiotemporally aligned, and combined with license plate information and behavioral feature data, data fusion is performed based on a multi-level fusion method to obtain real-time positioning fusion data of electric vehicles on campus.

[0071] A violation determination model is established based on historical location fusion data;

[0072] The violation determination model identifies the real-time location fusion data, generates violation records for campus electric vehicles, and associates them with rider identity information.

[0073] Specifically, this embodiment uses RFID readers and cameras in tandem, combining triangulation algorithms, filtering, and trajectory optimization to obtain accurate positioning data. At the same time, it obtains location information, license plate information, and behavioral feature data through image processing. It uses a multi-level fusion method to fuse multi-source data and establishes a violation judgment model based on historical data. This enables automatic identification of violations by electric vehicles on campus and rider identity association, thereby improving the management efficiency of monitoring violations by electric vehicles on campus.

[0074] The deployment of RFID readers and cameras in the campus monitoring area, and the installation of RFID tags on electric vehicles on campus, specifically includes:

[0075] Based on the routes of electric vehicles on campus and areas with frequent violations, the scope of the monitoring area is determined. RFID readers and cameras are deployed in the monitoring area, with the coverage areas of the RFID readers overlapping and the field of view of the cameras covering the monitoring area.

[0076] RFID readers are deployed at intersections, corners, and building entrances within the monitoring area, with an overlap rate of at least 30% between the coverage areas of adjacent RFID readers; cameras are deployed within the coverage area of ​​the RFID readers, with an installation height of at least 3 meters and a field of view covering the electric vehicle passage area.

[0077] When registering electric vehicles on campus, the license plate information and rider identity information of the electric vehicles are collected, and RFID tags with unique identification codes are installed at fixed positions on the electric vehicles.

[0078] The RFID tag is fixedly installed at a fixed position on the front of the electric vehicle on campus. The RFID tag is associated with the license plate information and rider identity information, and the association is stored in a database.

[0079] Specifically, this embodiment ensures monitoring coverage of key areas by deploying RFID readers at key locations such as intersections, corners, and building entrances and exits. The overlap rate of more than 30% between the coverage areas of adjacent RFID readers avoids monitoring blind spots. The installation height of the cameras at more than 3 meters and the reasonable field of view ensure effective video surveillance coverage.

[0080] The overlapping coverage of RFID readers ensures signal continuity and reliability, the collaborative work of multiple readers improves positioning accuracy, and the high-position installation of cameras reduces obstruction and improves image acquisition quality.

[0081] During vehicle registration, license plate information and rider identity information are collected simultaneously. The fixed installation locations of RFID tags ensure the stability of signal collection and establish a correspondence between RFID tags, license plate information, and rider identity information. This correspondence is stored in a database for easy information retrieval and management, achieving integrated management of vehicle, tag, and rider information and providing a reliable data foundation for monitoring violations.

[0082] The process involves collecting RFID signal data from electric vehicles on campus using an RFID reader, processing the RFID signal data using a triangulation algorithm to obtain the spatial coordinates of the electric vehicle, and then filtering and optimizing the spatial coordinates to obtain RFID positioning data. Specifically, this includes:

[0083] The RFID signal data of the campus electric vehicles are collected simultaneously by multiple RFID readers, and the spatial coordinates of the campus electric vehicles are calculated based on the triangulation algorithm.

[0084] The formula for calculating the spatial position coordinates is:

[0085]

[0086] d i =c·Δt i ;

[0087]

[0088] Where (x,y,z) are the spatial coordinates of the electric vehicles on campus, and N is the number of RFID readers participating in the positioning. i ,y i ,z i Let d be the spatial coordinates of the i-th RFID reader / writer. i Let Δt be the distance between the i-th RFID reader and the target RFID tag. i Let be the signal read / write time difference for the i-th RFID reader, c be the signal propagation speed, and w be the signal propagation speed. i RSSI is the signal strength weighting coefficient for the i-th RFID reader / writer. i Let be the signal strength value received by the i-th RFID reader / writer;

[0089] The spatial location coordinates are filtered using the Kalman filter algorithm, and trajectory smoothing optimization is performed by combining historical trajectory data to obtain RFID positioning data.

[0090] The spatial position coordinates are filtered based on the Kalman filter algorithm, the process noise covariance matrix is ​​dynamically adjusted by introducing an adaptive factor, the historical trajectory data is smoothed by cubic spline interpolation, and the trajectory curve is optimized by combining motion constraints.

[0091] The formula for calculating the adaptive factor is:

[0092]

[0093] Where λ1 is the adaptive factor, α1 is the basic weight coefficient, and β1 is the attenuation coefficient. Let σ1 be the residual vector and σ1 be the standard deviation of the measurement noise.

[0094] Specifically, this embodiment collects signals simultaneously through multiple RFID readers, providing multi-dimensional positioning data. The spatial location calculation based on the triangulation algorithm takes into account signal strength weights, improving positioning accuracy. The introduction of signal propagation time difference calculation further optimizes the positioning results.

[0095] Kalman filtering is employed to filter spatial position coordinates, effectively suppressing random errors. An adaptive factor is introduced to dynamically adjust the process noise covariance matrix, improving the adaptability of the filtering. Adaptive optimization of data processing is achieved through the calculation of residual vectors and measurement noise standard deviation. Trajectory smoothing optimization is performed by combining historical trajectory data, using cubic spline interpolation to achieve trajectory smoothing. Motion constraints are considered to make the optimized trajectory more consistent with actual motion characteristics.

[0096] The process of acquiring video image data via a camera, preprocessing and detecting targets in the video image data, and extracting the electric vehicle's location information, license plate information, and behavioral feature data specifically includes:

[0097] The video image data is preprocessed, including image enhancement, noise suppression and illumination compensation, and electric vehicle target detection is performed using the YOLOv5 deep learning model.

[0098] Adaptive histogram equalization is used for image enhancement, combined with bilateral filtering for noise suppression; image brightness and contrast are automatically adjusted based on ambient light intensity; attention mechanism and spatial pyramid pooling module are introduced into the YOLOv5 model.

[0099] Based on the detected target area, the location information, license plate information and behavioral feature data of electric vehicles on campus are extracted by a multi-branch deep neural network. The behavioral features include vehicle speed, driving trajectory and driving behavior features.

[0100] Location information, including the target's coordinates and scale in the image, is extracted using a deep residual network; license plate information, including license plate number and color, is extracted using a license plate recognition network; and behavioral features, including real-time velocity estimation, trajectory prediction, and abnormal behavior recognition, are extracted using a spatiotemporal convolutional network.

[0101] The loss function for the location information is calculated as follows:

[0102] L pos =α2·L reg +β2·L iou +γ2·L conf ;

[0103] Among them, L pos L reg For bounding box regression loss, L iou To compare the loss between intersection and union, L conf For confidence loss, α2, β2, and γ2 are the weighting coefficients for bounding box regression loss, intersection-union ratio loss, and confidence loss, respectively.

[0104] The calculation formula for the spatiotemporal attention mechanism that extracts the behavioral features is as follows:

[0105]

[0106] Where f is the input feature map, W q W is the first learned weight matrix. k W is the second learning weight matrix. v For the third learning weight matrix, d k Let σ be the feature dimension, and σ(·) be the softmax function.

[0107] Specifically, this embodiment improves image contrast and clarity through adaptive histogram equalization, effectively suppresses image noise by using bilateral filtering while preserving edge details, and automatically adjusts image parameters based on ambient light intensity to improve the adaptability of image quality.

[0108] The YOLOv5 model is introduced to achieve real-time object detection. The attention mechanism is used to improve the attention of the detection, and the spatial pyramid pooling module enhances the feature extraction capability, thereby improving the accuracy and real-time performance of electric vehicle object detection.

[0109] Parallel extraction of multi-dimensional features is achieved through multi-branch deep neural networks, precise location information is extracted through deep residual networks, license plate features are extracted using dedicated license plate recognition networks, complete behavioral features are extracted through spatiotemporal convolutional networks, and spatiotemporal attention mechanisms are introduced to enhance the focus of feature extraction.

[0110] The process of aligning RFID positioning data and location information in time and space, and combining license plate information and behavioral feature data, to perform data fusion based on a multi-level fusion method to obtain real-time positioning fusion data for electric vehicles on campus, specifically includes:

[0111] The RFID positioning data and the location information are spatiotemporally aligned.

[0112] The timestamps in the RFID positioning data and the location information are synchronized and calibrated to ensure data consistency in the time dimension; the spatial coordinates in the RFID positioning data and the location information are uniformly converted to a preset standard reference coordinate system to ensure data consistency in the spatial dimension.

[0113] The formula for calculating the spatiotemporal alignment is:

[0114] T syn =T RFID -ΔT;

[0115]

[0116] Among them, T syn T is the synchronized timestamp. RFID Here, ΔT is the original timestamp of the RFID positioning data, ΔT is the time offset between the camera and the RFID reader, (x,y,z) are the spatial coordinates of the electric vehicle on campus, R is the rotation matrix, and (t) is the time offset between the camera and the RFID reader. x ,t y ,t z (x', y', z') is the translation vector, and (x', y', z') is the spatial coordinate of the electric vehicle on campus after transformation in the standard reference coordinate system;

[0117] The license plate information and behavioral feature data are fused with the spatiotemporally aligned RFID positioning data based on the multi-level fusion method to obtain real-time positioning fusion data of electric vehicles on campus.

[0118] The RFID positioning data, location information, license plate information and behavioral feature data are integrated at the feature level using a weighted fusion algorithm, and a fused feature vector is generated by methods such as feature splicing or weighted averaging.

[0119] Based on the fused feature vector, a multi-model ensemble algorithm, including random forest, support vector machine or neural network, is used to comprehensively analyze and judge the real-time location data of electric vehicles on campus, and generate the final real-time location fusion data.

[0120] The formula for calculating the fused feature vector is:

[0121]

[0122] Among them, F fusion To fuse feature vectors, F represents the weighting coefficient for the a-th class of data. a Let A be the feature vector of the a-th data class, and A be the number of data classes.

[0123] The formula for calculating the real-time location fusion data is:

[0124] P fusion =α3·P RFID +β3·P Image +γ3·P Behavior +δ3·P License ;

[0125] Among them, P fusion To achieve real-time location fusion data, P RFID For RFID location data, P Image For image location information, P Behavior P is location data obtained based on behavioral characteristics. License For license plate recognition location data, α3, β3, γ3, and δ3 are the weighting coefficients for RFID positioning data, image location information, location data obtained based on behavioral features, and license plate recognition location data, respectively.

[0126] Specifically, this embodiment ensures the consistency of RFID positioning data and location information in the time dimension through timestamp synchronization calibration, taking into account the time offset between the camera and the RFID reader, and achieving accurate time alignment of multi-source data.

[0127] Spatial coordinates from different sources are uniformly transformed to a standard reference coordinate system. The precise transformation of the coordinate system is achieved through rotation matrices and translation vectors, ensuring data consistency across spatial dimensions.

[0128] A multi-level fusion method is adopted to achieve the organic integration of data. The weights of different data sources are reasonably allocated through a weighted fusion algorithm, and the fusion feature vector is generated by feature splicing or weighted averaging. This achieves the effective fusion of RFID positioning data, location information, license plate information and behavioral feature data.

[0129] By using a multi-model fusion algorithm to comprehensively analyze the fused data and taking into account the weighting coefficients of various data types, more accurate and reliable real-time positioning fusion data is generated.

[0130] The violation determination model based on historical location fusion data specifically includes:

[0131] Historical location fusion data is segmented into time series and outlier processing is performed. Spatiotemporal features, velocity features, trajectory features and behavioral pattern features of historical location fusion data are extracted. A sliding time window method is used to construct a sequence of violation behavior features. Principal component analysis and feature selection algorithms are used to optimize the feature vector to obtain the violation behavior feature vector. A violation behavior sample library is then established.

[0132] The formula for calculating the feature vector of the violation is:

[0133]

[0134] in, Let be the feature vector of the violation at time t, w be the size of the time window, and λ be the value of the violation. b P is the time decay factor. b V b T b B b These are location features, velocity features, trajectory features, and behavioral features, respectively.

[0135] Based on the aforementioned violation sample library, a temporal judgment model is constructed using a long short-term memory network structure. Through the temporal judgment model, an attention mechanism and residual connections are introduced to obtain a violation judgment model. Cross-validation and early stopping strategies are used to prevent overfitting.

[0136] The weight calculation formula for the attention mechanism is as follows:

[0137]

[0138] in, Let h be the attention weight at time tt. tt Let s be the hidden state at time tt. tt-1 Let W be the state at time tt-1. c W is the attention mapping weight matrix. h W is the hidden state transition weight matrix. s The historical state weight matrix, For attentional bias;

[0139] The calculation formula for the violation determination model is as follows:

[0140]

[0141] Wherein, P(y tt |F tt Given a feature vector F of violation behavior tt Under certain conditions, a campus electric vehicle commits a violation at time tt. tt The probability of LSTM(F) tt ) represents the feature vector F of the violation behavior in the Long Short-Term Memory network.tt The processing output, W q This is the output layer weight matrix. For output layer bias, Let h be the attention weight at time tt. tt Let t be the hidden state at time tt.

[0142] Specifically, this embodiment improves data quality through time-series segmentation and outlier processing, extracts multi-dimensional features such as spatiotemporal features, velocity features, trajectory features and behavioral pattern features, and uses a sliding time window method to ensure the continuity of the feature sequence.

[0143] A comprehensive sample database of violations was established based on historical data. A temporal judgment model was constructed using a Long Short-Term Memory (LSTM) network, and an attention mechanism was introduced to enhance the identification of key information. Residual connections were used to strengthen the model's learning ability. The temporal judgment model considers historical state information, dynamically adjusts feature weights using the attention mechanism, and accurately calculates the probability of violations. Cross-validation was used to evaluate model performance, and an early stopping strategy was employed to avoid overtraining, thereby improving the model's generalization ability.

[0144] The step of identifying the real-time location fusion data through the violation determination model, generating violation records for campus electric vehicles, and associating them with rider identity information specifically includes:

[0145] The real-time location fusion data is input into the violation behavior determination model to obtain the violation behavior type and violation probability. When the violation probability exceeds a preset threshold, a violation event is triggered.

[0146] Feature extraction and normalization are performed on real-time location fusion data. The probability distribution of various violation types is calculated through a violation judgment model. Dynamic violation judgment thresholds are set and the threshold parameters are adjusted according to different violation types and scenarios. When the probability of any violation type exceeds the corresponding threshold, the violation type, probability value and key feature data are recorded.

[0147] Based on the aforementioned violation, a violation record is generated. The violation record includes the time of violation, the location of violation, the type of violation, and evidence information. The rider's identity information is linked through a dual verification method using RFID tag number and license plate information.

[0148] Violation record generation: Records the precise timestamp and geographic coordinates of the violation, saves the type description and probability value of the violation, automatically captures video clips before and after the violation as evidence using a camera, and records environmental parameters such as weather conditions and lighting conditions;

[0149] Identity information association: Electric vehicle registration information is queried by RFID tag number, vehicle information is verified by license plate recognition results, a mapping relationship between violation records and rider identity is established, and rider violation record files are updated in real time.

[0150] Specifically, this embodiment improves data quality through feature extraction and normalization, adopts dynamic violation judgment thresholds to adapt to different scenarios, considers the probability distribution of multiple violation types, and flexibly adjusts threshold parameters according to different violation types and scenarios.

[0151] Record the precise timestamp and geographic coordinates of the violation, save the violation type description and violation probability value, automatically capture video evidence before and after the violation through the camera, and record environmental parameters, including weather conditions and lighting conditions, to form a complete chain of evidence of the violation.

[0152] The system employs dual verification using RFID tag numbers and license plate information, and achieves accurate matching of vehicle information through database queries. This establishes a precise mapping relationship between violation records and rider identities, and updates rider violation record files in real time.

[0153] Please see Figure 2 The present invention also provides an electric vehicle violation monitoring system based on positioning and image fusion, the system comprising:

[0154] The tag monitoring module is used to deploy RFID readers and cameras in the campus monitoring area and to install RFID tags on electric vehicles on campus.

[0155] The signal acquisition module is used to collect RFID signal data of electric vehicles on campus through an RFID reader, process the RFID signal data through a triangulation algorithm to obtain the spatial coordinates of the electric vehicle, and perform filtering and trajectory optimization on the spatial coordinates of the electric vehicle to obtain RFID positioning data.

[0156] The feature extraction module is used to collect video image data through a camera, preprocess the video image data and detect targets, and extract the electric vehicle's location information, license plate information and behavioral feature data;

[0157] The data fusion module is used to align the RFID positioning data and the location information in time and space, and combine them with license plate information and behavioral feature data to perform data fusion based on a multi-level fusion method to obtain real-time positioning fusion data of electric vehicles on campus.

[0158] The model building module is used to build a violation judgment model based on historical location fusion data.

[0159] The violation identification module is used to identify the real-time location fusion data through the violation behavior judgment model, generate violation records of campus electric vehicles, and associate them with rider identity information.

[0160] Specifically, this embodiment of an electric vehicle violation monitoring system based on positioning and image fusion achieves fully automated management of the entire process of campus electric vehicle violation monitoring through a tag monitoring module, a signal acquisition module, a feature extraction module, a data fusion module, a model building module, and a violation identification module. The tag monitoring module provides a reliable hardware foundation, the signal acquisition module achieves accurate RFID positioning, the feature extraction module ensures high-quality processing of video data, the data fusion module effectively integrates multi-source data, the model building module constructs accurate violation judgment capabilities, and the violation identification module ensures timely detection and recording of violations, improving the system's maintainability and scalability. Close collaboration between modules enhances system operating efficiency, ultimately forming a complete, reliable, and efficient campus electric vehicle violation monitoring system.

[0161] The present invention also discloses an electronic device, comprising: at least one processor, at least one memory communication interface and bus; wherein the processor, memory and communication interface communicate with each other through the bus; the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement a method for monitoring electric vehicle violations based on positioning and image fusion.

[0162] This invention also discloses a computer-readable storage medium storing computer instructions that enable the computer to implement all or part of the steps of the electric vehicle violation monitoring method based on positioning and image fusion described in this embodiment of the invention. The storage medium includes various media capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0163] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for monitoring electric vehicle traffic violations based on localization and image fusion, characterized in that, The application to campus electric vehicle management includes the following steps: RFID readers and cameras are deployed in the campus monitoring area, and RFID tags are installed on electric vehicles on campus. RFID signal data of electric vehicles on campus are collected by RFID readers. The RFID signal data is processed by a triangulation algorithm to obtain the spatial coordinates of the electric vehicle. The spatial coordinates of the electric vehicle are then filtered and the trajectory is optimized to obtain RFID positioning data. The process involves collecting RFID signal data from electric vehicles on campus using an RFID reader, processing the RFID signal data using a triangulation algorithm to obtain the spatial coordinates of the electric vehicle, and then filtering and optimizing the spatial coordinates to obtain RFID positioning data. Specifically, this includes: The RFID signal data of the campus electric vehicles are collected simultaneously by multiple RFID readers, and the spatial coordinates of the campus electric vehicles are calculated based on the triangulation algorithm. The spatial location coordinates are filtered using the Kalman filter algorithm, and trajectory smoothing optimization is performed by combining historical trajectory data to obtain RFID positioning data. The spatial position coordinates are filtered based on the Kalman filter algorithm, the process noise covariance matrix is ​​dynamically adjusted by introducing an adaptive factor, the historical trajectory data is smoothed by cubic spline interpolation, and the trajectory curve is optimized by combining motion constraints. Video image data is collected by a camera, and the video image data is preprocessed and target detection is performed to extract the location information, license plate information and behavioral feature data of the electric vehicle. The RFID positioning data and the location information are spatiotemporally aligned, and combined with license plate information and behavioral feature data, data fusion is performed based on a multi-level fusion method to obtain real-time positioning fusion data of electric vehicles on campus. A violation determination model is established based on historical location fusion data; The violation determination model based on historical location fusion data specifically includes: Historical location fusion data is segmented into time series and outlier processing is performed. Spatiotemporal features, velocity features, trajectory features and behavioral pattern features of historical location fusion data are extracted. A sliding time window method is used to construct a sequence of violation behavior features. Principal component analysis and feature selection algorithms are used to optimize the feature vector to obtain the violation behavior feature vector. A violation behavior sample library is then established. Based on the aforementioned violation sample database, a temporal judgment model is constructed using a long short-term memory network structure. Through the temporal judgment model, an attention mechanism and residual connections are introduced to obtain the violation judgment model. The violation determination model identifies the real-time location fusion data, generates violation records for campus electric vehicles, and associates them with rider identity information.

2. The method for monitoring electric vehicle violations based on localization and image fusion as described in claim 1, characterized in that, The deployment of RFID readers and cameras in the campus monitoring area, and the installation of RFID tags on electric vehicles on campus, specifically includes: Based on the routes of electric vehicles on campus and areas with frequent violations, the scope of the monitoring area is determined. RFID readers and cameras are deployed in the monitoring area, with the coverage areas of the RFID readers overlapping and the field of view of the cameras covering the monitoring area. When registering electric vehicles on campus, the license plate information and rider identity information of the electric vehicles are collected, and RFID tags with unique identification codes are installed at fixed positions on the electric vehicles.

3. The method for monitoring electric vehicle violations based on localization and image fusion as described in claim 1, characterized in that, The formula for calculating the spatial position coordinates is: ； ； ； in, The spatial coordinates of the electric vehicles on campus. N The number of RFID readers participating in the positioning. For the first i Spatial coordinates of an RFID reader / writer For the first i The distance between an RFID reader and the target RFID tag For the first i The signal read / write time difference of an RFID reader / writer c For signal propagation speed, For the first i Signal strength weighting coefficient for each RFID reader / writer For the first i The signal strength value received by the RFID reader; The formula for calculating the adaptive factor is: ； in, As an adaptive factor, Based on the weighting coefficient, The attenuation coefficient is... For the residual vector, To measure the standard deviation of noise.

4. The method for monitoring electric vehicle violations based on localization and image fusion as described in claim 1, characterized in that, The process of acquiring video image data via a camera, preprocessing and detecting targets in the video image data, and extracting the electric vehicle's location information, license plate information, and behavioral feature data specifically includes: The video image data is preprocessed, including image enhancement, noise suppression and illumination compensation, and electric vehicle target detection is performed using the YOLOv5 deep learning model. Based on the detected target area, the location information, license plate information and behavioral feature data of electric vehicles on campus are extracted by a multi-branch deep neural network.

5. The method for monitoring electric vehicle violations based on localization and image fusion as described in claim 1, characterized in that, The process of aligning RFID positioning data and location information in time and space, and combining license plate information and behavioral feature data, to perform data fusion based on a multi-level fusion method to obtain real-time positioning fusion data for electric vehicles on campus, specifically includes: The RFID positioning data and the location information are spatiotemporally aligned. The license plate information and behavioral feature data are fused with the spatiotemporally aligned RFID positioning data using a multi-level fusion method to obtain real-time positioning fusion data for electric vehicles on campus.

6. The method for monitoring electric vehicle violations based on localization and image fusion as described in claim 5, characterized in that, The formula for calculating the feature vector of the violation is: ； in, for tt The feature vector of the violation at any given time. w The size of the time window. The time decay factor, , , , These are location features, velocity features, trajectory features, and behavioral features, respectively. The weight calculation formula for the attention mechanism is as follows: ； in, for tt Attention weight at any moment for tt The hidden state at any given moment. for tt The state at time -1 For attention mapping weight matrix, This is the hidden state transition weight matrix. The historical state weight matrix, For attentional bias; The calculation formula for the violation determination model is as follows: ； in, Given a feature vector of violation behavior Under certain conditions, a violation occurred on a campus electric vehicle at time tt. The probability, For Long Short-Term Memory networks, the feature vector of violation behavior The processing output, This is the output layer weight matrix. For output layer bias, for tt Attention weight at any moment for tt The hidden state at any given moment.

7. The method for monitoring electric vehicle violations based on localization and image fusion as described in claim 6, characterized in that, The step of identifying the real-time location fusion data through the violation determination model, generating violation records for campus electric vehicles, and associating them with rider identity information specifically includes: The real-time location fusion data is input into the violation behavior determination model to obtain the violation behavior type and violation probability. When the violation probability exceeds a preset threshold, a violation event is triggered. Based on the aforementioned violation, a violation record is generated. The violation record includes the time of violation, the location of violation, the type of violation, and evidence information. The rider's identity information is linked through a dual verification method using RFID tag number and license plate information.

8. A system for monitoring electric vehicle violations based on localization and image fusion, used to execute the method for monitoring electric vehicle violations based on localization and image fusion as described in any one of claims 1-7, characterized in that, The system includes: The tag monitoring module is used to deploy RFID readers and cameras in the campus monitoring area and to install RFID tags on electric vehicles on campus. The signal acquisition module is used to collect RFID signal data of electric vehicles on campus through an RFID reader, process the RFID signal data through a triangulation algorithm to obtain the spatial coordinates of the electric vehicle, and perform filtering and trajectory optimization on the spatial coordinates of the electric vehicle to obtain RFID positioning data. The feature extraction module is used to collect video image data through a camera, preprocess the video image data and detect targets, and extract the electric vehicle's location information, license plate information and behavioral feature data; The data fusion module is used to align the RFID positioning data and the location information in time and space, and combine them with license plate information and behavioral feature data to perform data fusion based on a multi-level fusion method to obtain real-time positioning fusion data of electric vehicles on campus. The model building module is used to build a violation judgment model based on historical location fusion data. The violation identification module is used to identify the real-time location fusion data through the violation behavior judgment model, generate violation records of campus electric vehicles, and associate them with rider identity information.

9. An electronic device, characterized in that, include: At least one processor, at least one memory, a communication interface, and a bus; The processor, memory, and communication interface communicate with each other through the bus. The memory stores program instructions that are executed by the processor, and the processor calls the program instructions to implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause the computer to perform the method as described in any one of claims 1 to 7.

Images