Electric vehicle violation monitoring method and system based on positioning and image fusion

Abstract

The invention relates to the technical field of electric vehicle violation detection, and provides an electric vehicle violation monitoring method and system based on positioning and image fusion, the method and system are applied to campus electric vehicle management, and the method comprises the following steps: arranging an RFID reader-writer and a camera, and installing an RFID tag on a campus electric vehicle; the method comprises the following steps: acquiring RFID signal data, processing the RFID signal data to obtain spatial position coordinates of an electric vehicle, and carrying out filtering processing and track optimization on the spatial position coordinates of the electric vehicle to obtain RFID positioning data; collecting video image data, and extracting position information, license plate information and behavior characteristic data of the electric vehicle; performing data fusion based on a multi-level fusion method to obtain real-time positioning fusion data; establishing an illegal behavior judgment model; and identifying the real-time positioning fusion data through an illegal behavior judgment model, and generating an illegal record of the campus electric vehicle. According to the invention, automatic identification of campus electric vehicle violation behaviors and association of rider identities are realized, and the management efficiency of campus electric vehicle violation monitoring is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of electric vehicle violation detection, and in particular to an electric vehicle violation monitoring method and system based on positioning and image fusion. Background Art

[0002] With the rapid growth of the number of electric vehicles on campus, safety hazards such as illegal driving are becoming increasingly prominent. At present, the management of electric vehicles on campus mainly relies on manual inspections or a single video surveillance system. The traditional video surveillance system is easily affected by environmental factors such as light and weather, resulting in unstable monitoring results; although a single RFID positioning system can achieve electric vehicle positioning, the positioning accuracy is limited and the vehicle behavior characteristics cannot be obtained; the existing monitoring systems generally lack an effective data fusion mechanism, making it difficult to conduct a comprehensive analysis of monitoring data from different sources, resulting in a low accuracy rate in determining violations.

[0003] In the existing technology, the independence of RFID positioning data and video image data leads to inaccurate monitoring of electric vehicle behavior and determination of violations, especially in complex scenarios, such as when multiple electric vehicles pass through the monitoring area at the same time, which easily leads to data inconsistency and behavior recognition errors. Summary of the invention

[0004] In view of this, the present invention proposes an electric vehicle violation monitoring method and system based on positioning and image fusion, which is used to solve the problem that the existing technology is not accurate enough in monitoring the behavior of electric vehicles and determining violations, and is prone to data inconsistency and behavior recognition errors in complex scenarios.

[0005] The technical solution of the present invention is implemented as follows: In the first aspect, the present invention provides an electric vehicle violation monitoring method based on positioning and image fusion, which is applied to campus electric vehicle management, comprising the following steps:

[0006] Deploy RFID readers and cameras in the campus monitoring area, and install RFID tags on campus electric vehicles;

[0007] The RFID signal data of the campus electric vehicle is collected by an RFID reader, the RFID signal data is processed by a triangulation positioning algorithm to obtain the spatial position coordinates of the electric vehicle, and the spatial position coordinates of the electric vehicle are filtered and trajectory optimized to obtain RFID positioning data;

[0008] Collect video image data through a camera, perform preprocessing and target detection on the video image data, and extract location information, license plate information and behavior feature data of the electric vehicle;

[0009] The RFID positioning data and the location information are aligned in time and space, and the license plate information and behavior feature data are combined to perform data fusion based on a multi-level fusion method to obtain real-time positioning fusion data of campus electric vehicles;

[0010] Establish a violation determination model based on historical positioning fusion data;

[0011] The real-time positioning fusion data is identified through the violation determination model to generate violation records of campus electric vehicles and associate them with the rider's identity information.

[0012] On the basis of the above technical solution, preferably, the RFID reader and camera are arranged in the campus monitoring area, and the RFID tags are installed on the campus electric vehicles, specifically including:

[0013] According to the routes of electric vehicles on campus and the areas where violations are frequent, the scope of the monitoring area is determined, and RFID readers and cameras are deployed in the monitoring area. The coverage areas of the RFID readers overlap with each other, and the field of view of the camera covers the monitoring area;

[0014] When registering campus electric vehicles, the license plate information and rider identity information of the campus electric vehicle are collected, and an RFID tag with a unique identification code is installed at a fixed position on the body of the campus electric vehicle.

[0015] On the basis of the above technical solution, preferably, the RFID signal data of the campus electric vehicle is collected by the RFID reader, the RFID signal data is processed by the triangulation positioning algorithm to obtain the spatial position coordinates of the electric vehicle, and the spatial position coordinates of the electric vehicle are filtered and trajectory optimized to obtain the RFID positioning data, which specifically includes:

[0016] The RFID signal data of the campus electric vehicle is collected by multiple RFID readers and writers at the same time, and the spatial position coordinates of the campus electric vehicle are calculated based on the triangulation positioning algorithm;

[0017] The calculation formula of the spatial position coordinates is:

[0018]

[0019] d i =c·Δt i ;

[0020]

[0021] Among them, (x, y, z) is the spatial position coordinates of the campus electric vehicle, N is the number of RFID readers involved in positioning, (x i ,y i ,z i) is the spatial coordinate of the i-th RFID reader, d i is the distance between the i-th RFID reader and the target RFID tag, Δt i is the signal reading and writing time difference of the i-th RFID reader, c is the signal propagation speed, w i is the signal strength weight coefficient of the i-th RFID reader, RSSI i is the signal strength value received by the i-th RFID reader;

[0022] The spatial position coordinates are filtered using a Kalman filter algorithm, and the trajectory is optimized by combining historical trajectory data to obtain RFID positioning data;

[0023] The spatial position coordinates are filtered based on the Kalman filter algorithm, the adaptive factor is introduced to dynamically adjust the process noise covariance matrix, the cubic spline interpolation method is used to smooth the historical trajectory data, and the trajectory curve is optimized in combination with motion constraints;

[0024] The calculation formula of the adaptive factor is:

[0025]

[0026] Among them, λ1 is the adaptive factor, α1 is the basic weight coefficient, β1 is the attenuation coefficient, is the residual vector, and σ1 is the standard deviation of the measurement noise.

[0027] On the basis of the above technical solution, preferably, the video image data is collected by a camera, the video image data is preprocessed and the target is detected, and the location information, license plate information and behavior feature data of the electric vehicle are extracted, which specifically includes:

[0028] Preprocessing the video image data includes image enhancement, noise suppression, and illumination compensation, and performing electric vehicle target detection using a YOLOv5 deep learning model;

[0029] Based on the detected target area, the location information, license plate information and behavioral feature data of campus electric vehicles are extracted respectively through a multi-branch deep neural network.

[0030] On the basis of the above technical solution, preferably, the RFID positioning data and the location information are aligned in time and space, and combined with the license plate information and behavior feature data, data fusion is performed based on a multi-level fusion method to obtain real-time positioning fusion data of campus electric vehicles, specifically including:

[0031] Performing spatiotemporal alignment processing on the RFID positioning data and the location information;

[0032] Based on the multi-level fusion method, the license plate information and behavior feature data are fused with the RFID positioning data after time and space alignment to obtain the real-time positioning fusion data of campus electric vehicles.

[0033] On the basis of the above technical solutions, preferably, the establishment of a violation behavior determination model based on historical positioning fusion data specifically includes:

[0034] Perform time series segmentation and outlier processing on historical positioning fusion data, extract the spatiotemporal characteristics, speed characteristics, trajectory characteristics and behavior pattern characteristics of historical positioning fusion data, use the sliding time window method to construct the violation behavior feature sequence, use principal component analysis and feature selection algorithm to optimize the feature vector, obtain the violation behavior feature vector, and establish a violation behavior sample library;

[0035] The calculation formula of the illegal behavior feature vector is:

[0036]

[0037] in, is the feature vector of the violation at time tt, w is the time window size, λ b is the time decay factor, P b 、V b , T b , B b They are position features, speed features, trajectory features and behavior features;

[0038] Based on the violation sample library, a time series judgment model is constructed by using a long short-term memory network structure, and an attention mechanism and a residual connection are introduced through the time series judgment model to obtain a violation judgment model;

[0039] The weight calculation formula of the attention mechanism is:

[0040]

[0041] in, is the attention weight at time tt, h tt is the hidden state at time tt, s tt-1 is the state at time tt-1, W c is the attention mapping weight matrix, W h is the hidden state conversion weight matrix, W s is the historical state weight matrix, For attention bias;

[0042] The calculation formula of the violation determination model is:

[0043]

[0044] Among them, P(y tt |F tt ) is the given violation feature vector F tt Under the condition that the campus electric vehicle violates the rules at time tt, tt The probability of LSTM(F tt ) is the feature vector F of the violation behavior of the long short-term memory network tt The processed output, W q is the output layer weight matrix, is the output layer bias, is the attention weight at time tt, h tt is the hidden state at time tt.

[0045] On the basis of the above technical solution, preferably, the real-time positioning fusion data is identified by the violation behavior determination model, and the violation record of the campus electric vehicle is generated and associated with the rider identity information, specifically including:

[0046] Inputting the real-time positioning fusion data into a violation behavior determination model to obtain the violation behavior type and violation probability, and triggering a violation event when the violation probability exceeds a preset threshold;

[0047] A violation record is generated based on the violation event, and the violation record includes the violation time, violation location, violation type, and evidence information, and is associated with the rider's identity information through a dual verification method of RFID tag number and license plate information.

[0048] In a second aspect, the present invention further provides an electric vehicle violation monitoring system based on positioning and image fusion, the system comprising:

[0049] Tag monitoring module, used to deploy RFID readers and cameras in the campus monitoring area and install RFID tags on campus electric vehicles;

[0050] A signal acquisition module is used to collect RFID signal data of campus electric vehicles through an RFID reader / writer, process the RFID signal data through a triangulation positioning algorithm to obtain the spatial position coordinates of the electric vehicle, filter the spatial position coordinates of the electric vehicle and optimize the trajectory to obtain RFID positioning data;

[0051] A feature extraction module is used to collect video image data through a camera, perform preprocessing and target detection on the video image data, and extract the location information, license plate information and behavior feature data of the electric vehicle;

[0052] A data fusion module is used to align the RFID positioning data with the location information in time and space, and combine the license plate information and behavior feature data to perform data fusion based on a multi-level fusion method to obtain real-time positioning fusion data of campus electric vehicles;

[0053] Model building module, used to build a violation determination model based on historical positioning fusion data;

[0054] The violation identification module is used to identify the real-time positioning fusion data through the violation behavior determination model, generate violation records of campus electric vehicles and associate them with the rider's identity information.

[0055] In a third aspect, the present invention further provides an electronic device, comprising: at least one processor, at least one memory, a communication interface and a bus;

[0056] Among them, the processor, memory, and communication interface communicate with each other through the bus, the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement steps such as an electric vehicle violation monitoring method based on positioning and image fusion.

[0057] In a fourth aspect, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions enable a computer to implement steps of a method for monitoring electric vehicle violations based on positioning and image fusion.

[0058] Compared with the prior art, the electric vehicle violation monitoring method and system based on positioning and image fusion of the present invention has the following beneficial effects:

[0059] (1) Through the coordinated use of RFID readers and cameras, combined with triangulation positioning algorithms, filtering processing and trajectory optimization, accurate positioning data is obtained. At the same time, image processing is used to obtain location information, license plate information and behavioral feature data. Multi-source data is integrated using a multi-level fusion method, and a violation judgment model is established based on historical data. The automatic identification of campus electric vehicle violations and the association with the rider's identity are realized, which improves the management efficiency of campus electric vehicle violation monitoring.

[0060] (2) Through the collaborative work of multiple RFID readers and writers and the calculation of signal strength weights, combined with the triangulation positioning algorithm, the Kalman filter algorithm with dynamic adjustment of adaptive factors is introduced to filter the coordinates, and the cubic spline interpolation method is combined with motion constraints to perform trajectory smoothing optimization, thus achieving accurate calculation of spatial position and improving the accuracy of RFID positioning data;

[0061] (3) By performing time series segmentation and outlier processing on the historical positioning fusion data, the sliding time window method is combined to extract multi-dimensional features, and the long short-term memory network is used to build a time series judgment model. The attention mechanism and residual connection are introduced to enhance the model performance. At the same time, cross-validation and early stopping strategies are used to prevent overfitting. This realizes the identification and judgment of violations and improves the reliability of violation judgment. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0063] Figure 1 This is a flow chart of an electric vehicle violation monitoring method based on positioning and image fusion according to the present invention;

[0064] Figure 2 The present invention is a structural diagram of an electric vehicle violation monitoring system based on positioning and image fusion. DETAILED DESCRIPTION

[0065] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0066] See also Figure 1 The present invention provides an electric vehicle violation monitoring method based on positioning and image fusion, which is applied to campus electric vehicle management and includes the following steps:

[0067] Deploy RFID readers and cameras in the campus monitoring area, and install RFID tags on campus electric vehicles;

[0068] The RFID signal data of the campus electric vehicle is collected by an RFID reader, the RFID signal data is processed by a triangulation positioning algorithm to obtain the spatial position coordinates of the electric vehicle, and the spatial position coordinates of the electric vehicle are filtered and trajectory optimized to obtain RFID positioning data;

[0069] Collect video image data through a camera, perform preprocessing and target detection on the video image data, and extract location information, license plate information and behavior feature data of the electric vehicle;

[0070] The RFID positioning data and the location information are aligned in time and space, and the license plate information and behavior feature data are combined to perform data fusion based on a multi-level fusion method to obtain real-time positioning fusion data of campus electric vehicles;

[0071] Establish a violation determination model based on historical positioning fusion data;

[0072] The real-time positioning fusion data is identified through the violation determination model to generate violation records of campus electric vehicles and associate them with the rider's identity information.

[0073] Specifically, this embodiment uses RFID readers and cameras in a coordinated manner, combines triangulation algorithms, filtering processing and trajectory optimization to obtain accurate positioning data, and simultaneously obtains location information, license plate information and behavioral characteristic data through image processing. It uses a multi-level fusion method to fuse multi-source data, and establishes a violation judgment model based on historical data. It realizes the automatic identification of violations of campus electric vehicles and the association with the identity of the rider, thereby improving the management efficiency of violation monitoring of campus electric vehicles.

[0074] The method of deploying RFID readers and cameras in the campus monitoring area and installing RFID tags on campus electric vehicles specifically includes:

[0075] According to the routes of electric vehicles on campus and the areas where violations are frequent, the scope of the monitoring area is determined, and RFID readers and cameras are deployed in the monitoring area. The coverage areas of the RFID readers overlap with each other, and the field of view of the camera covers the monitoring area;

[0076] RFID readers are deployed at intersections, corners and building entrances and exits in the monitoring area, and the coverage area overlap rate of adjacent RFID readers is not less than 30%; cameras are deployed in the coverage area of ​​RFID readers, and the installation height of the cameras is not less than 3 meters, and the field of view angle covers the electric vehicle traffic area;

[0077] When registering campus electric vehicles, collect the license plate information and rider identity information of the campus electric vehicles, and install RFID tags with unique identification codes at fixed locations on the body of the campus electric vehicles;

[0078] The RFID tag is fixedly installed at a fixed position on the front of the campus electric vehicle body, and a corresponding relationship is established between the RFID tag, the license plate information and the rider identity information, and the corresponding relationship is stored in a database.

[0079] Specifically, this embodiment ensures monitoring coverage of key areas by deploying RFID readers at key locations such as intersections, corners, and building entrances and exits. The overlapping rate of adjacent RFID readers is more than 30%, avoiding monitoring blind spots. The installation height of the camera is more than 3 meters and the reasonable field of view angle ensures effective coverage of video surveillance.

[0080] The overlapping coverage of RFID readers ensures the continuity and reliability of the signal. The collaborative work of multiple readers improves the positioning accuracy. The high-position installation of the camera reduces occlusion and improves the image acquisition quality.

[0081] The license plate information and rider identity information are collected simultaneously during vehicle registration. The fixed installation position of the RFID tag ensures the stability of signal collection and establishes the corresponding relationship between the RFID tag, license plate information and rider identity information. The corresponding relationship is stored in the database to facilitate information query and management, realize the integrated management of vehicle, tag and rider information, and provide a reliable data basis for monitoring violations.

[0082] The RFID signal data of the campus electric vehicle is collected by the RFID reader, the RFID signal data is processed by the triangulation positioning algorithm to obtain the spatial position coordinates of the electric vehicle, and the spatial position coordinates of the electric vehicle are filtered and trajectory optimized to obtain the RFID positioning data, specifically including:

[0083] The RFID signal data of the campus electric vehicle is collected by multiple RFID readers and writers at the same time, and the spatial position coordinates of the campus electric vehicle are calculated based on the triangulation positioning algorithm;

[0084] The calculation formula of the spatial position coordinates is:

[0085]

[0086] d i =c·Δt i ;

[0087]

[0088] Among them, (x, y, z) is the spatial position coordinates of the campus electric vehicle, N is the number of RFID readers involved in positioning, (x i ,y i ,z i ) is the spatial coordinate of the i-th RFID reader, d i is the distance between the i-th RFID reader and the target RFID tag, Δt i is the signal reading and writing time difference of the i-th RFID reader, c is the signal propagation speed, w i is the signal strength weight coefficient of the i-th RFID reader, RSSI i is the signal strength value received by the i-th RFID reader;

[0089] The spatial position coordinates are filtered using a Kalman filter algorithm, and the trajectory is optimized by combining historical trajectory data to obtain RFID positioning data;

[0090] The spatial position coordinates are filtered based on the Kalman filter algorithm, the adaptive factor is introduced to dynamically adjust the process noise covariance matrix, the cubic spline interpolation method is used to smooth the historical trajectory data, and the trajectory curve is optimized in combination with motion constraints;

[0091] The calculation formula of the adaptive factor is:

[0092]

[0093] Among them, λ1 is the adaptive factor, α1 is the basic weight coefficient, β1 is the attenuation coefficient, is the residual vector, and σ1 is the standard deviation of the measurement noise.

[0094] Specifically, this embodiment collects signals simultaneously through multiple RFID readers and writers, provides multi-dimensional positioning data, calculates the spatial position based on the triangulation positioning algorithm, takes into account the signal strength weight, improves the positioning accuracy, introduces the signal propagation time difference calculation, and further optimizes the positioning result.

[0095] The Kalman filter algorithm is used to filter the spatial position coordinates, which effectively suppresses random errors. The adaptive factor is introduced to dynamically adjust the process noise covariance matrix to improve the adaptability of the filter. The adaptive optimization of data processing is achieved by calculating the residual vector and the standard deviation of the measurement noise. The trajectory smoothing optimization is combined with the historical trajectory data, and the cubic spline interpolation method is used to achieve trajectory smoothing. Considering the motion constraints, the optimized trajectory is more in line with the actual motion characteristics.

[0096] The method of collecting video image data through a camera, preprocessing and detecting targets on the video image data, and extracting the location information, license plate information, and behavior feature data of the electric vehicle specifically includes:

[0097] Preprocessing the video image data includes image enhancement, noise suppression, and illumination compensation, and performing electric vehicle target detection using a YOLOv5 deep learning model;

[0098] Adaptive histogram equalization is used for image enhancement, combined with bilateral filtering for noise suppression; image brightness and contrast are automatically adjusted based on ambient light intensity; attention mechanism and spatial pyramid pooling module are introduced into the YOLOv5 model;

[0099] Based on the detected target area, the location information, license plate information and behavior feature data of campus electric vehicles are extracted through a multi-branch deep neural network, where the behavior features include vehicle speed, driving trajectory and driving behavior characteristics;

[0100] The deep residual network is used to extract location information, including the coordinates and scale information of the target in the image. The license plate recognition network is used to extract license plate information, including license plate number and license plate color. The spatiotemporal convolutional network is used to extract behavior features, including real-time speed estimation, trajectory prediction, and abnormal behavior recognition.

[0101] The loss function calculation formula of the position information is:

[0102] L pos =α2·L reg +β2·L iou +γ2·L conf ;

[0103] Among them, L pos is the loss of location information of campus electric vehicles, L reg is the bounding box regression loss, L iou is the intersection-over-union loss, L conf is the confidence loss, α2, β2, and γ2 are the weight coefficients of bounding box regression loss, intersection-over-union loss, and confidence loss, respectively;

[0104] The calculation formula of the spatiotemporal attention mechanism for extracting the behavioral features is:

[0105]

[0106] Among them, f is the input feature map, W q is the first learning weight matrix, W k is the second learning weight matrix, W v is the third learning weight matrix, d k is the feature dimension, and σ(·) is the softmax function.

[0107] Specifically, this embodiment improves the contrast and clarity of the image through adaptive histogram equalization, uses bilateral filtering to effectively suppress image noise while maintaining edge details, and automatically adjusts image parameters based on ambient light intensity to improve the adaptability of image quality.

[0108] The YOLOv5 model is introduced to realize real-time target detection, the attention mechanism is used to improve the detection focus, and the spatial pyramid pooling module enhances the feature extraction capability, thereby improving the accuracy and real-time performance of electric vehicle target detection.

[0109] Through a multi-branch deep neural network, multi-dimensional features can be extracted in parallel. The deep residual network extracts precise location information. A dedicated license plate recognition network is used to extract license plate features. The spatiotemporal convolutional network extracts complete behavioral features. The spatiotemporal attention mechanism is introduced to enhance the focus of feature extraction.

[0110] The RFID positioning data and the location information are aligned in time and space, and the license plate information and behavior feature data are combined to perform data fusion based on a multi-level fusion method to obtain real-time positioning fusion data of campus electric vehicles, specifically including:

[0111] Performing spatiotemporal alignment processing on the RFID positioning data and the location information;

[0112] Synchronize and calibrate the timestamps in the RFID positioning data and the location information to ensure the consistency of the data in the time dimension; uniformly convert the spatial coordinates in the RFID positioning data and the location information into a preset standard reference coordinate system to ensure the consistency of the data in the spatial dimension;

[0113] The calculation formula for the spatiotemporal alignment is:

[0114] T syn =T RFID -ΔT;

[0115]

[0116] Among them, T syn is the timestamp after synchronization, T RFID is the original timestamp of the RFID positioning data, ΔT is the time offset between the camera and the RFID reader, (x, y, z) is the spatial position coordinates of the campus electric vehicle, R is the rotation matrix, (t x ,t y ,t z ) is the translation vector, (x', y', z') is the spatial position coordinates of the campus electric vehicle after transformation in the standard reference coordinate system;

[0117] Based on the multi-level fusion method, the license plate information and behavior feature data are fused with the RFID positioning data after time and space alignment to obtain the real-time positioning fusion data of the campus electric vehicle;

[0118] The RFID positioning data, location information, license plate information and behavior feature data are integrated at the feature level by using a weighted fusion algorithm, and a fusion feature vector is generated by methods such as feature concatenation or weighted averaging;

[0119] Based on the fusion feature vector, a multi-model integration algorithm, including random forest, support vector machine or neural network, is used to comprehensively analyze and judge the real-time positioning data of campus electric vehicles to generate the final real-time positioning fusion data;

[0120] The calculation formula of the fused feature vector is:

[0121]

[0122] Among them, F fusion is the fusion feature vector, is the weight coefficient of the a-th type of data, F a is the feature vector of the a-th type of data, and A is the number of data categories;

[0123] The calculation formula of the real-time positioning fusion data is:

[0124] P fusion =α3·P RFID +β3·P Image +γ3·P Behavior +δ3·P License ;

[0125] Among them, P fusion To locate the fused data in real time, P RFID is the RFID positioning data, P Image is the image position information, P Behavior is the location data obtained based on behavioral characteristics, P License is the license plate recognition position data, α3, β3, γ3, and δ3 are the weight coefficients of RFID positioning data, image position information, position data based on behavioral characteristics, and license plate recognition position data, respectively.

[0126] Specifically, this embodiment ensures the consistency of RFID positioning data and position information in the time dimension through timestamp synchronization calibration, takes into account the time offset between the camera and the RFID reader, and realizes accurate time alignment of multi-source data.

[0127] The spatial coordinates from different sources are uniformly converted to a standard reference coordinate system, and the precise conversion of the coordinate system is achieved through rotation matrices and translation vectors, ensuring data consistency in the spatial dimension.

[0128] A multi-level fusion method is used to achieve organic integration of data. The weights of different data sources are reasonably allocated through a weighted fusion algorithm. The fusion feature vector is generated using feature splicing or weighted average method, thus achieving effective fusion of RFID positioning data, location information, license plate information and behavioral feature data.

[0129] The fused data is comprehensively analyzed through a multi-model integration algorithm, taking into account the weight coefficients of various types of data to generate more accurate and reliable real-time positioning fusion data.

[0130] The establishment of a violation behavior determination model based on historical positioning fusion data specifically includes:

[0131] Perform time series segmentation and outlier processing on historical positioning fusion data, extract the spatiotemporal characteristics, speed characteristics, trajectory characteristics and behavior pattern characteristics of historical positioning fusion data, use the sliding time window method to construct the violation behavior feature sequence, use principal component analysis and feature selection algorithm to optimize the feature vector, obtain the violation behavior feature vector, and establish a violation behavior sample library;

[0132] The calculation formula of the illegal behavior feature vector is:

[0133]

[0134] in, is the feature vector of the violation at time tt, w is the time window size, λ b is the time decay factor, P b 、V b , T b , B b They are position features, speed features, trajectory features and behavior features;

[0135] Based on the violation sample library, a time series judgment model is constructed by using a long short-term memory network structure. Through the time series judgment model, an attention mechanism and a residual connection are introduced to obtain a violation judgment model, and a cross-validation and early stopping strategy are used to prevent overfitting.

[0136] The weight calculation formula of the attention mechanism is:

[0137]

[0138] in, is the attention weight at time tt, h tt is the hidden state at time tt, s tt-1 is the state at time tt-1, W c is the attention mapping weight matrix, W h is the hidden state conversion weight matrix, W s is the historical state weight matrix, For attention bias;

[0139] The calculation formula of the violation determination model is:

[0140]

[0141] Among them, P(y tt |F tt ) is the given violation feature vector F tt Under the condition that the campus electric vehicle violates the rules at time tt, tt The probability of LSTM(F tt ) is the feature vector F of the violation behavior of the long short-term memory networktt The processed output, W q is the output layer weight matrix, is the output layer bias, is the attention weight at time tt, h tt is the hidden state at time tt.

[0142] Specifically, this embodiment improves data quality through time series segmentation and outlier processing, extracts multi-dimensional features such as spatiotemporal features, speed features, trajectory features, and behavior pattern features, and uses a sliding time window method to ensure the continuity of the feature sequence.

[0143] A complete violation sample library is established based on historical data, and a time series judgment model is constructed using a long short-term memory network. The attention mechanism is introduced to improve the recognition ability of key information, and the learning ability of the model is enhanced through residual connections. The time series judgment model takes into account historical state information, uses the attention mechanism to dynamically adjust feature weights, and accurately calculates the probability of violation behavior. Cross-validation is used to evaluate model performance, and an early stopping strategy is used to avoid overtraining, which improves the generalization ability of the model.

[0144] The method of identifying the real-time positioning fusion data through the violation behavior determination model, generating a violation record of the campus electric vehicle and associating the rider identity information specifically includes:

[0145] Inputting the real-time positioning fusion data into a violation behavior determination model to obtain the violation behavior type and violation probability, and triggering a violation event when the violation probability exceeds a preset threshold;

[0146] Perform feature extraction and normalization on real-time positioning fusion data, calculate the probability distribution of various violation types through the violation judgment model, set dynamic violation judgment thresholds, adjust threshold parameters according to different violation types and scenarios, and record the violation type, probability value and key feature data when the probability of any violation type exceeds the corresponding threshold;

[0147] Generate a violation record based on the violation event, the violation record includes the violation time, violation location, violation type, and evidence information, and associates the rider's identity information through a dual verification method of RFID tag number and license plate information;

[0148] Generation of violation records: Record the exact timestamp and geographic coordinates of the violation, save the type description and probability value of the violation, automatically capture video clips before and after the violation as evidence through the camera, and record environmental parameters such as weather conditions, lighting conditions, etc.

[0149] Identity information association: query the electric vehicle registration information through the RFID tag number, use the license plate recognition result to verify the vehicle information, establish a mapping relationship between violation records and rider identities, and update the rider's violation record file in real time.

[0150] Specifically, this embodiment improves data quality through feature extraction and normalization processing, adopts dynamic violation determination thresholds to adapt to different scenarios, considers the probability distribution of multiple violation types, and flexibly adjusts threshold parameters according to different violation types and scenarios.

[0151] Record the precise timestamp and geographic location coordinates of the violation, save the violation type description and violation probability value, automatically capture video evidence before and after the violation through the camera, and record environmental parameters, including weather conditions, lighting conditions, etc., to form a complete chain of evidence of the violation.

[0152] Adopting dual verification of RFID tag number and license plate information, accurate matching of vehicle information is achieved through database query, precise mapping relationship between violation records and rider identities is established, and the rider's violation record files are updated in real time.

[0153] See also Figure 2 The present invention also provides an electric vehicle violation monitoring system based on positioning and image fusion, the system comprising:

[0154] Tag monitoring module, used to deploy RFID readers and cameras in the campus monitoring area and install RFID tags on campus electric vehicles;

[0155] A signal acquisition module is used to collect RFID signal data of campus electric vehicles through an RFID reader / writer, process the RFID signal data through a triangulation positioning algorithm to obtain the spatial position coordinates of the electric vehicle, filter the spatial position coordinates of the electric vehicle and optimize the trajectory to obtain RFID positioning data;

[0156] A feature extraction module is used to collect video image data through a camera, perform preprocessing and target detection on the video image data, and extract the location information, license plate information and behavior feature data of the electric vehicle;

[0157] A data fusion module is used to align the RFID positioning data with the location information in time and space, and combine the license plate information and behavior feature data to perform data fusion based on a multi-level fusion method to obtain real-time positioning fusion data of campus electric vehicles;

[0158] Model building module, used to build a violation determination model based on historical positioning fusion data;

[0159] The violation identification module is used to identify the real-time positioning fusion data through the violation behavior determination model, generate violation records of campus electric vehicles and associate them with the rider's identity information.

[0160] Specifically, an electric vehicle violation monitoring system based on positioning and image fusion in this embodiment realizes the full-process automated management of campus electric vehicle violation monitoring through a tag monitoring module, a signal acquisition module, a feature extraction module, a data fusion module, a model building module and a violation identification module. The tag monitoring module provides a reliable hardware foundation, the signal acquisition module realizes accurate RFID positioning, the feature extraction module ensures high-quality processing of video data, the data fusion module realizes effective integration of multi-source data, the model building module builds accurate violation judgment capabilities, and the violation identification module ensures timely discovery and recording of violations, thereby improving the maintainability and scalability of the system, and improving the system operation efficiency through close collaboration between modules, ultimately forming a complete, reliable and efficient campus electric vehicle violation monitoring system.

[0161] The present invention also discloses an electronic device, comprising: at least one processor, at least one memory communication interface and a bus: wherein the processor, memory and communication interface communicate with each other through the bus; the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement an electric vehicle violation monitoring method based on positioning and image fusion.

[0162] The present invention also discloses a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, wherein the computer instructions enable the computer to implement all or part of the steps of the electric vehicle violation monitoring method based on positioning and image fusion described in the embodiment of the present invention. The storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0163] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for monitoring electric vehicle violations based on positioning and image fusion, characterized in that: Applied to campus electric vehicle management, including the following steps: Deploy RFID readers and cameras in the campus monitoring area, and install RFID tags on campus electric vehicles; The RFID signal data of the campus electric vehicle is collected by an RFID reader, the RFID signal data is processed by a triangulation positioning algorithm to obtain the spatial position coordinates of the electric vehicle, and the spatial position coordinates of the electric vehicle are filtered and trajectory optimized to obtain RFID positioning data; Collect video image data through a camera, perform preprocessing and target detection on the video image data, and extract location information, license plate information and behavior feature data of the electric vehicle; The RFID positioning data and the location information are aligned in time and space, and the license plate information and behavior feature data are combined to perform data fusion based on a multi-level fusion method to obtain real-time positioning fusion data of campus electric vehicles; Establish a violation determination model based on historical positioning fusion data; The real-time positioning fusion data is identified through the violation determination model to generate violation records of campus electric vehicles and associate them with the rider's identity information.

2. The electric vehicle violation monitoring method based on positioning and image fusion as claimed in claim 1, characterized in that: The method of deploying RFID readers and cameras in the campus monitoring area and installing RFID tags on campus electric vehicles specifically includes: According to the routes of electric vehicles on campus and the areas where violations are frequent, the scope of the monitoring area is determined, and RFID readers and cameras are deployed in the monitoring area. The coverage areas of the RFID readers overlap with each other, and the field of view of the camera covers the monitoring area; When registering campus electric vehicles, the license plate information and rider identity information of the campus electric vehicle are collected, and an RFID tag with a unique identification code is installed at a fixed position on the body of the campus electric vehicle.

3. The electric vehicle violation monitoring method based on positioning and image fusion as claimed in claim 1, characterized in that: The RFID signal data of the campus electric vehicle is collected by the RFID reader, the RFID signal data is processed by the triangulation positioning algorithm to obtain the spatial position coordinates of the electric vehicle, and the spatial position coordinates of the electric vehicle are filtered and trajectory optimized to obtain the RFID positioning data, specifically including: The RFID signal data of the campus electric vehicle is collected by multiple RFID readers and writers at the same time, and the spatial position coordinates of the campus electric vehicle are calculated based on the triangulation positioning algorithm; The calculation formula of the spatial position coordinates is: d i =c·Δt i ; Among them, (x, y, z) is the spatial position coordinates of the campus electric vehicle, N is the number of RFID readers involved in positioning, (x i ,y i ,z i ) is the spatial coordinate of the i-th RFID reader, d i is the distance between the i-th RFID reader and the target RFID tag, Δt i is the signal reading and writing time difference of the i-th RFID reader, c is the signal propagation speed, w i is the signal strength weight coefficient of the i-th RFID reader, RSSI i is the signal strength value received by the i-th RFID reader; The spatial position coordinates are filtered using a Kalman filter algorithm, and the trajectory is optimized by combining historical trajectory data to obtain RFID positioning data; The spatial position coordinates are filtered based on the Kalman filter algorithm, the adaptive factor is introduced to dynamically adjust the process noise covariance matrix, the cubic spline interpolation method is used to smooth the historical trajectory data, and the trajectory curve is optimized in combination with motion constraints; The calculation formula of the adaptive factor is: Among them, λ1 is the adaptive factor, α1 is the basic weight coefficient, β1 is the attenuation coefficient, is the residual vector, and σ1 is the standard deviation of the measurement noise.

4. The electric vehicle violation monitoring method based on positioning and image fusion as claimed in claim 1, characterized in that: The method of collecting video image data through a camera, preprocessing and detecting targets on the video image data, and extracting the location information, license plate information, and behavior feature data of the electric vehicle specifically includes: Preprocessing the video image data includes image enhancement, noise suppression, and illumination compensation, and performing electric vehicle target detection using a YOLOv5 deep learning model; Based on the detected target area, the location information, license plate information and behavioral feature data of campus electric vehicles are extracted respectively through a multi-branch deep neural network.

5. The electric vehicle violation monitoring method based on positioning and image fusion as claimed in claim 1, characterized in that: The RFID positioning data and the location information are aligned in time and space, and the license plate information and behavior feature data are combined to perform data fusion based on a multi-level fusion method to obtain real-time positioning fusion data of campus electric vehicles, specifically including: Performing spatiotemporal alignment processing on the RFID positioning data and the location information; Based on the multi-level fusion method, the license plate information and behavior feature data are fused with the RFID positioning data after time and space alignment to obtain the real-time positioning fusion data of campus electric vehicles.

6. The electric vehicle violation monitoring method based on positioning and image fusion as claimed in claim 5, characterized in that: The establishment of a violation behavior determination model based on historical positioning fusion data specifically includes: Perform time series segmentation and outlier processing on historical positioning fusion data, extract the spatiotemporal characteristics, speed characteristics, trajectory characteristics and behavior pattern characteristics of historical positioning fusion data, use the sliding time window method to construct the violation behavior feature sequence, use principal component analysis and feature selection algorithm to optimize the feature vector, obtain the violation behavior feature vector, and establish a violation behavior sample library; The calculation formula of the illegal behavior feature vector is: in, is the feature vector of the violation at time tt, w is the time window size, λ b is the time decay factor, P b 、V b 、T b , B b They are position features, speed features, trajectory features and behavior features; Based on the violation sample library, a time series judgment model is constructed by using a long short-term memory network structure, and an attention mechanism and a residual connection are introduced through the time series judgment model to obtain a violation judgment model; The weight calculation formula of the attention mechanism is: in, is the attention weight at time tt, h tt is the hidden state at time tt, s tt-1 is the state at time tt-1, W c is the attention mapping weight matrix, W h is the hidden state conversion weight matrix, W s is the historical state weight matrix, For attention bias; The calculation formula of the violation determination model is: Among them, P(y tt |F tt ) is the given violation feature vector F tt Under the condition that the campus electric vehicle violates the rules at time tt, tt The probability of LSTM(F tt ) is the feature vector F of the violation behavior of the long short-term memory network tt The processed output, W q is the output layer weight matrix, is the output layer bias, is the attention weight at time tt, h tt is the hidden state at time tt.

7. The electric vehicle violation monitoring method based on positioning and image fusion as claimed in claim 6, characterized in that: The method of identifying the real-time positioning fusion data through the violation behavior determination model, generating a violation record of the campus electric vehicle and associating the rider identity information specifically includes: Inputting the real-time positioning fusion data into a violation behavior determination model to obtain the violation behavior type and violation probability, and triggering a violation event when the violation probability exceeds a preset threshold; A violation record is generated based on the violation event, and the violation record includes the violation time, violation location, violation type, and evidence information, and is associated with the rider's identity information through a dual verification method of RFID tag number and license plate information.

8. An electric vehicle violation monitoring system based on positioning and image fusion, characterized in that: The system comprises: Tag monitoring module, used to deploy RFID readers and cameras in the campus monitoring area and install RFID tags on campus electric vehicles; A signal acquisition module is used to collect RFID signal data of campus electric vehicles through an RFID reader / writer, process the RFID signal data through a triangulation positioning algorithm to obtain the spatial position coordinates of the electric vehicle, filter the spatial position coordinates of the electric vehicle and optimize the trajectory to obtain RFID positioning data; A feature extraction module is used to collect video image data through a camera, perform preprocessing and target detection on the video image data, and extract the location information, license plate information and behavior feature data of the electric vehicle; A data fusion module is used to align the RFID positioning data and the location information in time and space, and combine the license plate information and behavior feature data to perform data fusion based on a multi-level fusion method to obtain real-time positioning fusion data of campus electric vehicles; Model building module, used to build a violation determination model based on historical positioning fusion data; The violation identification module is used to identify the real-time positioning fusion data through the violation behavior determination model, generate violation records of campus electric vehicles and associate them with the rider's identity information.

9. An electronic device, characterized in that: include: at least one processor, at least one memory, a communication interface, and a bus; The processor, memory, and communication interface communicate with each other via the bus, the memory stores program instructions executable by the processor, and the processor calls the program instructions to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions enable a computer to implement the method according to any one of claims 1 to 7.

Images