Multi-mode vehicle security monitoring method and system based on charging pile

Abstract

The invention relates to a charging pile multi-mode vehicle security monitoring method and system, and the method comprises the steps: collecting vehicle contour point cloud, environment parameters and voltage fluctuation data through a stereoscopic vision sensor, a temperature and humidity sensor and a current sensor, and generating a multi-mode fusion data set through time-space synchronization; comprehensive scores of vehicle deformation, environment sudden change and voltage abnormity are calculated based on a dynamic weight fusion model, low-risk, medium-risk and high-risk three-level safety judgment results are generated in combination with a quantile threshold interval, and the environment adaptability defect of a traditional fixed rule is eliminated through a dynamic weight and an adaptive threshold; triggering a hierarchical control instruction according to a judgment result, and synchronously constructing a credible evidence chain containing a timestamp, an operation log and an encryption check code to ensure that the security event is traceable and the data is tamper-proof; and finally, aggregating multi-node user feedback through a federated learning framework, updating a model weight and a threshold interval in combination with a differential privacy technology, and realizing dynamic optimization of a false alarm rate while protecting user privacy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of charging security monitoring, in particular to a charging pile multi-modal vehicle security monitoring method and system. BACKGROUND

[0002] With the popularization of new energy vehicles, the safety protection capability of charging piles as core infrastructure directly affects user property and public safety. The charging pile security system needs to monitor the vehicle state, environmental changes and charging parameters in real time, identify potential risks such as abnormal displacement of vehicles, environmental mutations or electrical faults through multi-dimensional data fusion, and trigger graded protection measures. Such multi-modal monitoring methods can more comprehensively capture safety hazards through the cooperation of visual, environmental and electrical sensors, but existing technologies still have significant bottlenecks in practical application.

[0003] Traditional charging pile security solutions usually rely on a single type of sensor such as vision or current detection, or use static fusion rules with fixed thresholds to judge risks. Due to the large differences in the relevance of vehicle deformation, temperature and humidity interference, and voltage fluctuations in different scenarios, static models are difficult to adapt to complex environmental changes, resulting in high false alarm rates. In addition, existing methods lack a closed-loop learning mechanism for user feedback, and cannot optimize detection logic based on historical false or missed alarm events. On the other hand, after a security incident occurs, the correlation between the operation instructions and the original data is insufficient, making it difficult to trace responsibility, and data collaboration across charging piles may cause privacy leakage risks. SUMMARY

[0004] Based on this, the purpose of the present application is to provide a charging pile multi-modal vehicle security monitoring method and system that can dynamically adapt to environmental changes, accurately respond to different levels of risk, and realize secure and collaborative optimization of data.

[0005] The purpose of the present application is achieved by the following scheme:

[0006] In a first aspect, the present application provides a charging pile multi-modal vehicle security monitoring method, comprising the following steps:

[0007] S1: Real-time acquisition of vehicle contour point cloud data, environmental temperature and humidity parameters, and charging voltage fluctuation data through the integrated stereo vision sensor, temperature and humidity sensor, and current sensor of the charging pile, and spatiotemporal synchronization of the contour point cloud data, environmental temperature and humidity parameters, and charging voltage fluctuation data to generate a multi-modal fusion data set;

[0008] S2: Comprehensive anomaly calculation and anomaly identification of the multi-modal fusion data set based on a pre-set dynamic weight fusion model and a pre-set quantile threshold interval to generate a security judgment result, the security judgment result including one of a low-risk judgment result, a medium-risk judgment result, and a high-risk judgment result;

[0009] S3: identifying and judging the safety judgment result based on the preset alarm rule, generating corresponding hierarchical control instructions, and data binding and encrypting the hierarchical control instructions with the multi-modal fusion data set to construct a trusted evidence chain containing a time stamp, an operation log and an encrypted check code; the hierarchical control instructions are used to control the charging pile to execute one of the low-risk control instructions, the medium-risk control instructions and the high-risk control instructions;

[0010] S4: sending the trusted evidence chain to the user terminal and obtaining user feedback data fed back by the user terminal, processing the user feedback data based on a federated learning framework, and updating the dynamic weight parameters of the dynamic weight fusion model and the threshold intervals of the preset quantile threshold interval.

[0011] In a second aspect, the present application provides a charging pile multi-modal vehicle security monitoring system, which is configured with the following modules:

[0012] A data acquisition and processing module is configured to acquire the profile point cloud data of the vehicle, the environmental temperature and humidity parameters and the charging voltage fluctuation data in real time through the stereo vision sensor, the temperature and humidity sensor and the current sensor integrated in the charging pile, and to perform spatiotemporal synchronization on the profile point cloud data, the environmental temperature and humidity parameters and the charging voltage fluctuation data to generate a multi-modal fusion data set.

[0013] A judgment result generation module is configured to perform comprehensive anomaly calculation and anomaly identification on the multi-modal fusion data set based on a preset dynamic weight fusion model and a preset quantile threshold interval to generate a safety judgment result, the safety judgment result including one of a low-risk judgment result, a medium-risk judgment result and a high-risk judgment result.

[0014] A hierarchical control instruction generation module is configured to identify and judge the safety judgment result based on the preset alarm rule to generate corresponding hierarchical control instructions, the hierarchical control instructions being used to control the charging pile to execute one of the low-risk control instructions, the medium-risk control instructions and the high-risk control instructions.

[0015] A trusted evidence chain construction module is configured to data bind and encrypt the hierarchical control instructions with the multi-modal fusion data set to construct a trusted evidence chain containing a time stamp, an operation log and an encrypted check code.

[0016] A data communication module is configured to send the trusted evidence chain to the user terminal and obtain user feedback data fed back by the user terminal.

[0017] A model parameter update module is configured to process the user feedback data based on a federated learning framework to update the dynamic weight parameters of the dynamic weight fusion model and the threshold intervals of the preset quantile threshold interval.

[0018] In a third aspect, the present application provides a computer device, comprising a memory and a processor, the memory storing a computer program, and the processor implementing any of the above-mentioned vehicle security monitoring methods based on multi-modal charging piles when executing the computer program.

[0019] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement any of the above-mentioned vehicle security monitoring methods based on multi-modal charging piles.

[0020] In summary, the vehicle security monitoring method based on multi-modal charging piles provided by the present application can effectively solve the misjudgment problem caused by the one-sidedness of single sensor data in the traditional charging pile security system, and overcome the poor adaptability of static threshold rules in complex scenarios. Through the cooperative collection and spatio-temporal synchronous processing of stereo vision, temperature and humidity, and current sensors, comprehensive monitoring of vehicle deformation, environmental mutation, and charging abnormalities can be achieved to construct a multi-modal data set with multi-dimensional associated features, thereby improving the coverage dimension of risk identification. Based on the dynamic weight fusion model, the multi-source heterogeneous data is weighted and calculated, the contribution weights of vehicle deformation, temperature and humidity mutation, and voltage abnormality features can be dynamically adjusted according to the historical false alarm rate, and the adaptive division of quantile threshold intervals is combined to realize the environmental adaptability of risk scoring, significantly reducing the false alarm and missed alarm probability caused by environmental noise or parameter coupling. Through the binding and encryption of hierarchical control instructions and data to generate a trusted evidence chain, traceability and tamper-proof verification of the entire process of security events can be realized to provide trusted data support for responsibility definition and judicial evidence. Further, the federal learning framework is used to aggregate multi-node user feedback data, and the model parameters and threshold intervals are updated through gradient optimization and differential privacy protection technology, which can realize collaborative learning and dynamic optimization across charging piles under the premise of protecting user privacy, thereby constructing a closed-loop security decision system for continuous improvement of monitoring accuracy and scene generalization ability. Thus, a technical closed loop from data collection, risk decision, instruction execution to feedback optimization is formed to systematically solve the core problems of single monitoring dimension, rule solidification, and privacy leakage in traditional solutions.

[0021] In order to better understand and implement, the present application will be described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 A flowchart of a vehicle security monitoring method based on multi-modal charging piles provided by an embodiment of the present application is shown in the figure;

[0023] Figure 2 A flowchart of generating a security judgment result provided by an embodiment of the present application is shown in the figure;

[0024] Figure 3A flowchart of a process of generating hierarchical control instructions and tamper-proof trusted evidence chains is provided for an embodiment of the present application.

[0025] Figure 4 A flowchart of a process of generating an updated dynamic weight fusion model and a preset quantile threshold interval is provided for an embodiment of the present application.

[0026] Figure 5 A structural diagram of a charging pile multi-modal vehicle security monitoring system is provided for another embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to facilitate the understanding of the present application, the present application will be described in more detail below with reference to the relevant drawings. The preferred embodiments of the present application are shown in the drawings. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the application more thorough and comprehensive.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used in the specification of the application are only for the purpose of describing the specific embodiments and are not intended to limit the present application. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0029] In one embodiment, as shown in Figure 1 A charging pile multi-modal vehicle security monitoring method is provided. In this embodiment, the method is applied to a terminal. It should be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction of the terminal and the server. In this embodiment, the method includes the following steps:

[0030] S1: Collecting, through a stereo vision sensor, a temperature and humidity sensor, and a current sensor integrated in a charging pile, profile point cloud data of a vehicle, environmental temperature and humidity parameters, and charging voltage fluctuation data in real time, and performing space-time synchronization on the profile point cloud data, the environmental temperature and humidity parameters, and the charging voltage fluctuation data to generate a multi-modal fusion data set.

[0031] Specifically, the system captures the vehicle's contour point cloud data through the stereo vision sensor integrated with the charging pile, which contains the precise geometric features of the vehicle in three-dimensional space, including shape, position, and relative distance of each part, providing accurate three-dimensional positioning basis for subsequent judgment of abnormal displacement of the vehicle. At the same time, the system uses temperature and humidity sensors to closely monitor the temperature and humidity changes around the charging pile. Since fluctuations in environmental temperature and humidity can affect the performance of vehicle batteries, the reliability of electronic components, and the operating status of related mechanical parts, these parameters are crucial for assessing the safety of the vehicle's environment. In addition, current sensors track real-time current changes during charging, accurately calculating the fluctuation of charging voltage. The dynamic changes of current and voltage directly reflect the electrical operating conditions of the vehicle during charging, which are key indicators for identifying potential risks such as electrical faults.

[0032] Due to the differences in time and space dimensions of data from different sensors, for example, the contour point cloud data collected by the stereo vision sensor may lag slightly in timestamp compared to the temperature and humidity sensor data, or there may be different reference points in the spatial coordinate system. Therefore, it is necessary to perform rigorous spatio-temporal synchronization processing on the contour point cloud data, environmental temperature and humidity parameters, and charging voltage fluctuation data. Specifically, the system establishes a unified time series reference model to calibrate the timestamps of various data, ensuring consistency in the time dimension. At the same time, using spatial coordinate conversion algorithms, the data collected by different sensors are unified to the spatial reference system centered on the charging pile, eliminating spatial coordinate differences, thus achieving precise alignment of data in time and space dimensions. This generates a multi-modal fusion data set that integrates three-dimensional vehicle contours, environmental temperature and humidity, and charging voltage fluctuations, providing a solid data foundation for subsequent anomaly analysis. This allows the system to gain a more comprehensive and accurate perspective on the safety status of the vehicle during charging.

[0033] S2: Based on the pre-set dynamic weight fusion model and the pre-set quantile threshold interval, comprehensive anomaly calculation and anomaly identification are performed on the multi-modal fusion data set to generate a safety judgment result. The safety judgment result includes one of low-risk judgment result, medium-risk judgment result, and high-risk judgment result.

[0034] Specifically, the system performs deep fusion calculation on various types of data in the multi-modal fusion dataset according to a preset dynamic weight fusion model. This model has intelligent characteristics and can automatically and accurately adjust the weight distribution according to the importance of each data modality in anomaly detection in the actual scene. For example, in adverse weather conditions such as heavy rain or snow, the weight of environmental temperature and humidity parameters will be significantly increased. This is because in such an environment, small changes in environmental factors can have a huge impact on vehicle and charging safety. For example, rainwater may cause electrical equipment to short circuit, and low temperature may cause battery performance to drop sharply. At this time, the system will focus on analyzing temperature and humidity data to discover potential risks in a timely manner. In the case of normal weather and stable environment, the weight of vehicle contour point cloud data and charging voltage fluctuation data will be relatively increased. Vehicle contour point cloud data can help the system monitor whether the vehicle has abnormal displacement, such as whether the vehicle tilts or moves due to external force during charging, which may endanger the connection stability of the charging pile and the vehicle; charging voltage fluctuation data is used to assess whether the electrical state of the charging system is normal, and to discover potential electrical fault risks in a timely manner, such as abnormal voltage rise or fall, which may affect battery life and even cause safety accidents.

[0035] The quantile threshold interval is determined by the system through rigorous statistical analysis of a large amount of historical data, including normal and abnormal scenarios. For each monitoring indicator, the system accurately calculates its quantile value in the current data and compares it with the preset quantile threshold interval. If the quantile value of the data exceeds the normal interval range, the system determines that the indicator is abnormal. Comprehensive analysis of multiple indicators generates a safety judgment result. The safety judgment result is divided into three levels: low risk, medium risk, and high risk. The low-risk judgment result represents that there are slight abnormal signs, but the impact on safety is relatively small, but still needs attention, and the relevant data needs to be recorded for subsequent analysis; the medium-risk judgment result indicates that there are obvious abnormal conditions, which may pose a certain threat to charging safety, and measures need to be taken to handle it in a timely manner; the high-risk judgment result means that there are serious safety hazards, which may directly threaten the safety of vehicles and personnel, and emergency protection measures must be taken immediately, such as cutting off the charging power supply, locking the charging pile, etc., to ensure safety.

[0036] When performing anomaly detection, multiple factors need to be considered comprehensively to ensure the accuracy and reliability of the judgment result. For example, when the vehicle contour point cloud data changes slightly, the system will analyze it in combination with the environmental temperature and humidity parameters and the charging voltage fluctuation data. If the environmental temperature and humidity is normal at this time, and the charging voltage is stable, the slight change of the vehicle contour may be caused by some non-safety factors of the vehicle itself, such as the movement of items in the vehicle, etc., and the system may judge it as low risk; but if abnormal fluctuations in charging voltage are found at the same time, the change of the vehicle contour may be related to electrical faults, and the system will increase the risk level. Through such multi-dimensional comprehensive analysis, the system can effectively avoid misjudgment and omission, improve the accuracy of anomaly detection, and provide reliable basis for subsequent safety handling measures.

[0037] S3: identifying and judging the safety judgment result based on the preset alarm rule, generating corresponding hierarchical control instructions, and data binding and encryption of the hierarchical control instructions and the multi-modal fusion data set, constructing a trusted evidence chain containing time stamp, operation log and encryption check code; the hierarchical control instruction is used to control the charging pile to execute one of the low-risk control instruction, the medium-risk control instruction and the high-risk control instruction.

[0038] Specifically, after determining the safety level, the system will accurately regulate the action of the charging pile according to the hierarchical control instruction. The hierarchical control instruction contains three types of low-risk, medium-risk and high-risk. When the risk is low, the system orders the charging pile to start red light flashing and buzzer alarm at a preset frequency, which is eye-catching to prompt the user that the vehicle has a slight abnormality, such as the vehicle contour point cloud data showing slight displacement of the vehicle. At this moment, the alarm reminds the user to check the vehicle surrounding condition, and the system records and encrypts the relevant data and includes it in the trusted evidence chain management. When the risk is medium, the system automatically controls the charging pile to implement the charging gun electromagnetic locking to block the abnormal displacement of the vehicle in the charging state, and synchronously adjusts the charging output frequency to the safety threshold to ensure the stability of the charging process and avoid the expansion of electrical faults. The relevant operation information is recorded and encrypted in real time and stored as a basis for subsequent tracing. When the risk is high, the system immediately triggers the charging pile to stop charging, quickly starts the charging gun physical locking device to prevent any movement of the vehicle in a dangerous state, and cuts off the fault source.

[0039] In order to protect the security and privacy of data, the system uses advanced encryption algorithm to encrypt the bound data. In the encryption process, the system uses complex key generation mechanism and data encryption algorithm to ensure the integrity and tamper resistance of data in the process of transmission and storage. On this basis, the system constructs a trusted evidence chain containing timestamp, operation log and encryption check code. The timestamp accurately records the accurate time of each operation link, the operation log records the generation, execution process and related data information of the hierarchical control instruction in detail, and the encryption check code provides an additional protection for the data to prevent malicious tampering in the process of transmission and storage. The trusted evidence chain not only provides a reliable basis for the post-tracing and responsibility definition of security events, but also effectively reduces the risk of privacy leakage in cross-charging pile data cooperation, ensures the security and credibility of the whole system, and provides a solid guarantee for the safe charging environment of the vehicle.

[0040] S4: sending the trusted evidence chain to the user terminal and obtaining user feedback data fed back by the user terminal, processing the user feedback data based on the federated learning framework, and updating the dynamic weight parameter of the dynamic weight fusion model and the threshold interval of the preset quantile threshold interval.

[0041] Specifically, after constructing the trusted evidence chain, the system will send relevant information to the user terminal for viewing and feedback. After receiving the information, the user needs to confirm whether there is a fault, and can use this opportunity to view the vehicle condition in real time through the remote monitoring system, so as to take corresponding measures such as contacting the police, remotely locking the vehicle, etc., thereby effectively improving the safety and anti-theft ability of the vehicle. The system takes the user feedback data as an important basis for optimizing the monitoring model, and adjusts and optimizes the dynamic weight parameter of the dynamic weight fusion model and the preset quantile threshold interval. At the same time, the system will reevaluate the historical data combined with the updated model to verify the effectiveness and accuracy of the new model. This continuous feedback and model updating process enables the system to continuously improve the accuracy and reliability of vehicle security monitoring, and provides a solid defense line for users in the virtual world.

[0042] In summary, the multi-modal vehicle security monitoring method based on charging pile provided by the application can effectively solve the misjudgment problem caused by the one-sidedness of single sensor data in the traditional charging pile security system, and overcome the poor adaptability of static threshold rules in complex scenarios. Through the cooperative collection and spatio-temporal synchronous processing of stereo vision, temperature and humidity, and current sensors, comprehensive monitoring of vehicle deformation, environmental mutation, and charging abnormalities can be achieved to construct a multi-modal data set of multi-dimensional associated features, thereby improving the coverage dimension of risk identification; based on the dynamic weight fusion model, the multi-source heterogeneous data is weighted and calculated, the contribution weight of vehicle deformation, temperature and humidity mutation, and voltage abnormality features can be dynamically adjusted according to the historical false alarm rate, and the adaptive division of quantile threshold interval is combined to realize the environmental adaptability of risk scoring, which significantly reduces the false alarm and missed alarm probability caused by environmental noise or parameter coupling; through the binding and encryption of hierarchical control instructions and data to generate a trusted evidence chain, the traceability and tamper-proof verification of the whole process of security events can be realized, providing trusted data support for responsibility definition and judicial evidence; further, the federal learning framework is used to aggregate multi-node user feedback data, and the model parameters and threshold intervals are updated through gradient optimization and differential privacy protection technology, which can realize collaborative learning and dynamic optimization across charging piles under the premise of protecting user privacy, thereby constructing a closed-loop security decision system, continuously improving the monitoring accuracy and scene generalization ability; thereby forming a technical closed loop from data collection, risk decision, instruction execution to feedback optimization, systematically solving the core problems of single monitoring dimension, rule solidification, and privacy leakage in traditional solutions.

[0043] In one embodiment, the S1 of the multi-modal vehicle security monitoring method based on charging pile provided by the application specifically includes the following steps:

[0044] S11: Collect the contour point cloud data of the vehicle through the stereo vision sensor integrated in the charging pile, timestamp mark the contour point cloud data frame by frame according to the preset frame rate, convert the contour point cloud data from the sensor coordinate system to the charging pile reference coordinate system based on the calibration parameters, and generate spatially aligned vehicle contour three-dimensional data.

[0045] Specifically, during data collection, the system operates strictly according to the preset frame rate, ensuring that each frame of data accurately captures the subtle changes in the vehicle's outline. To ensure the temporal traceability of the data, the system marks a timestamp on each frame of collected outline point cloud data through a clock synchronization mechanism, ensuring the accuracy and consistency of the time information. Subsequently, the system converts these outline point cloud data from the sensor's own coordinate system to a unified coordinate system based on the charging pile as the reference. The coordinate conversion process involves complex geometric transformations and error correction algorithms, requiring comprehensive consideration of the sensor's installation position, angle, and the relative position relationship between the vehicle and the charging pile. Through precise calculation and adjustment, the system can achieve precise spatial alignment of the vehicle outline in the charging pile reference coordinate system, generating detailed and accurate three-dimensional data of the vehicle outline. This data not only reflects the appearance shape of the vehicle, but also provides key visual information for subsequent vehicle state monitoring and anomaly detection.

[0046] S12: Periodically collect environmental temperature and humidity parameters through the temperature and humidity sensor, perform time series filtering on the environmental temperature and humidity parameters, and generate a continuous and smooth temperature and humidity change curve after eliminating sensor noise interference.

[0047] To improve the accuracy and reliability of the data, the system will perform time series filtering on the collected environmental temperature and humidity parameters. The filtering algorithm can effectively eliminate abnormal data points caused by sensor noise and temporary interference factors in the environment, thereby generating a continuous and smooth temperature and humidity change curve. This curve directly shows the dynamic change trend of the charging environment in terms of temperature and humidity, which helps the system to evaluate whether the environmental conditions are suitable for vehicle charging in real time. For example, when the environmental temperature shows a slow rising trend, the system can analyze the temperature and humidity change curve, combined with the charging state of the vehicle, to predict whether this temperature change may have potential impact on the performance and safety of the vehicle battery, and then take appropriate measures in advance to ensure the safety of the charging process.

[0048] S13: Real-time monitoring of voltage and current parameters output by the charging pile through the current sensor, instantaneous fluctuation detection of voltage and current parameters, extraction of abnormal fluctuation features in the charging voltage waveform, and generation of charging voltage fluctuation data.

[0049] Specifically, during the monitoring process, the system performs instantaneous fluctuation detection on the voltage and current parameters, extracts abnormal fluctuation features from complex charging voltage waveforms through advanced signal analysis algorithms, and generates charging voltage fluctuation data. This process can timely capture abnormal conditions such as instantaneous voltage drop and peak during the charging process, which are often precursors of electrical faults. For example, when there is local poor contact in the charging cable or instantaneous fluctuation in the grid voltage, the system can quickly identify these abnormal fluctuation features and issue an early warning signal, avoiding more serious safety accidents such as fire and damage to the vehicle charging system, thereby effectively ensuring the electrical safety of the charging process.

[0050] S14: Perform spatio-temporal synchronization processing on the vehicle contour three-dimensional data, temperature and humidity change curve, and charging voltage fluctuation data, align the time dimension according to the time stamp, and align the spatial dimension through the charging pile reference coordinate system, to generate a multi-modal fusion data set containing visual, environmental, and electrical parameter correlation features.

[0051] Specifically, in the time dimension, the system accurately aligns data from different sources to the same time sequence according to the time stamp information of each data, ensuring the consistency and correlation of the data in time. In the spatial dimension, the system aligns the vehicle contour three-dimensional data with other environmental and electrical parameter data in space with the help of the charging pile reference coordinate system, so that all data can be analyzed and fused in a unified spatial framework. Finally, the system generates a multi-modal fusion data set containing visual, environmental, and electrical parameter correlation features. This data set comprehensively integrates the geometric shape of the vehicle, environmental temperature and humidity conditions, and electrical parameters during the charging process, providing a rich and accurate data foundation for subsequent comprehensive analysis and anomaly detection, enabling the system to comprehensively and deeply evaluate the safety status of the vehicle during the charging process from multiple angles, achieving accurate identification and timely warning of potential risks.

[0052] In one embodiment, as shown in Figure 2 The S2 of the multi-modal vehicle security monitoring method based on the charging pile provided by the application specifically includes the following steps:

[0053] S21: Process the vehicle contour point cloud data in the multi-modal fusion data set based on geometric deformation detection technology, generate vehicle deformation parameters representing the degree of physical deformation of the vehicle by calculating the offset of the key areas on the surface of the vehicle from the initial reference contour data, and the initial reference contour data is used to indicate the contour point cloud data collected for the first time in the normal parking state of the vehicle.

[0054] Specifically, the system processes the vehicle contour point cloud data in the multi-modal fusion dataset based on a geometric deformation detection technique. By accurately calculating the offset of the key areas on the vehicle surface from the initial baseline contour data, the system generates a vehicle deformation parameter that represents the degree of physical deformation of the vehicle. The initial baseline contour data is the first contour point cloud data collected when the vehicle is normally parked, which provides a reference standard for the system to compare the changes in the vehicle's contour during subsequent charging processes. When performing geometric deformation analysis, the system focuses on capturing subtle changes on the vehicle surface, such as dents or deformations caused by collisions or impacts. These changes are quantified through offset calculations, resulting in the generation of the vehicle deformation parameter. This parameter not only reflects the changes in the vehicle's physical structure but also provides a key basis for subsequent safety judgments, enabling the system to promptly identify potential safety hazards that may arise during the charging process due to external forces.

[0055] S22: Process the environmental temperature and humidity parameters based on time series analysis techniques, calculate the instantaneous change rate of the temperature and humidity parameters through first-order derivative, and generate environmental mutation features. The environmental mutation features are used to reflect the environmental mutation risk.

[0056] Specifically, the system can use time series analysis techniques to process the environmental temperature and humidity parameters in the multi-modal fusion dataset. By calculating the instantaneous change rate of the temperature and humidity parameters through first-order derivative, the system generates environmental mutation features that accurately reflect the environmental mutation risk. In practical applications, the instantaneous change rate of environmental temperature and humidity is a key indicator that helps the system quickly identify sharp changes in environmental conditions, such as sudden temperature rises or drops caused by air conditioning system start-stop or external weather mutations, as well as rapid changes in humidity. By analyzing these instantaneous change rates, the system can predict in advance the potential impact of environmental mutations on vehicle charging safety, such as high temperature and humidity environments that may exacerbate battery self-discharge and thermal runaway risks, thereby taking timely measures to ensure the safety of the vehicle charging process.

[0057] S23: Process the charging voltage fluctuation data based on waveform feature extraction techniques, calculate the standard deviation and peak-valley difference normalization, and generate voltage abnormal fluctuation coefficients that represent the degree of voltage abnormalities.

[0058] Specifically, during the charging process, abnormal fluctuations in voltage are an important safety risk factor, which can be caused by power grid fluctuations, charging equipment failures, or vehicle battery problems. The system quantifies the dispersion of voltage fluctuations through standard deviation calculation, while using peak-to-valley difference normalization to eliminate the influence of different dimensions, thereby obtaining a dimensionless coefficient that accurately reflects the degree of voltage abnormality. This coefficient provides an important basis for the system to evaluate electrical safety during the charging process, enabling the system to timely discover and handle problems such as charging interruptions, equipment damage, and even safety accidents caused by abnormal voltage fluctuations.

[0059] S24: Based on the preset quantile threshold interval, the vehicle deformation parameter, the environmental mutation feature, and the voltage abnormal fluctuation coefficient are input into the preset dynamic weight fusion model for processing to generate a safety judgment result.

[0060] Specifically, the dynamic weight fusion model considers the weights and mutual relationships of the three key parameters based on the preset quantile threshold interval to generate a safety judgment result. The dynamic weight fusion model can dynamically adjust the weights of each parameter according to different scenarios and conditions to ensure that the system's judgment result has high accuracy and adaptability. For example, in extreme weather conditions, the weight of the environmental mutation feature may be appropriately increased, while in the initial stage of vehicle charging, the weight of the voltage abnormal fluctuation coefficient may be higher. Through this dynamic adjustment mechanism, the system can more accurately evaluate the overall safety status of the vehicle during the charging process, timely issue warning signals or take appropriate control measures, effectively preventing and reducing the occurrence of safety incidents, and ensuring the safety and reliability of the vehicle charging process. The safety judgment result is generated through the following steps:

[0061] S241: Based on historical abnormal event data, the preset dynamic weight fusion model is trained. According to the false positive contribution of the vehicle deformation parameter, the environmental mutation feature, and the voltage abnormal fluctuation coefficient in the historical abnormal event data, the dynamic weight parameters of the dynamic weight fusion model are dynamically adjusted through the gradient descent algorithm to obtain an updated dynamic weight fusion model.

[0062] Specifically, the system first trains the preset dynamic weight fusion model based on historical abnormal event data. The system deeply analyzes the vehicle deformation parameters, environmental mutation features, and voltage abnormal fluctuation coefficients in the historical abnormal event data, accurately assesses the false positive contribution of these parameters in past abnormal events. At the same time, through the gradient descent algorithm, the system dynamically adjusts the dynamic weight parameters in the dynamic weight fusion model, so that the model can more accurately reflect the actual impact of each parameter on safety risks, thereby obtaining an updated dynamic weight fusion model. This model training process fully utilizes the experience information in historical data to optimize the model, so that it can more effectively identify and distinguish real abnormal events from false positives in actual application, improving the overall detection accuracy and reliability of the system.

[0063] S242: Based on the updated dynamic weight fusion model, the vehicle deformation parameters, environmental mutation features, and voltage abnormal fluctuation coefficients are weighted and summed to generate a comprehensive abnormal score.

[0064] Specifically, the calculation formula of the comprehensive abnormal score is:

[0065]

[0066] where S is the comprehensive abnormal score, δ is the vehicle deformation parameter, is the environmental mutation feature, ΔV err is the voltage abnormal fluctuation coefficient, and α, β, and γ are the dynamic weight parameters of the vehicle deformation parameter, the environmental mutation feature, and the voltage abnormal fluctuation coefficient, respectively. In the calculation process, the system combines each feature parameter according to its importance in safety risk assessment based on the updated weight parameters, forming a comprehensive score that can fully reflect the current safety state of the vehicle. This score not only integrates information from vehicle physical state, environmental conditions, and electrical parameters, but also highlights the different impact of each parameter on safety risks in different scenarios through dynamic weight adjustment, providing a more accurate and comprehensive quantitative basis for subsequent safety judgment.

[0067] S243: Based on the quantile threshold judgment rule, the comprehensive abnormal score is processed, and the danger level is divided according to the preset threshold interval to generate a safety judgment result. The preset threshold interval includes a low-risk threshold interval, a medium-risk threshold interval, and a high-risk threshold interval.

[0068] Specifically, the preset threshold interval covers three different levels of low risk, medium risk and high risk. The system compares the calculated comprehensive anomaly score with these preset threshold intervals. When the score is in the low risk threshold interval, the system determines that the vehicle is in a low risk state. If the score enters the medium risk threshold interval, it is considered that the vehicle faces a moderate risk. Once the score reaches the high risk threshold interval, the system immediately determines that the vehicle is in a high risk state and triggers the corresponding warning or control measures. Through this quantile threshold determination rule, the system can effectively distinguish different levels of safety risks and provide clear and operable safety judgment results for the security monitoring of charging piles, ensuring the safety of vehicles during charging is timely and accurately guaranteed.

[0069] The above-provided charging pile multi-modal vehicle security monitoring method can effectively solve the problem of high misjudgment rate and poor environmental adaptability caused by static models and single-dimensional features in traditional charging pile security methods through a multi-dimensional dynamic data processing mechanism. Based on real-time monitoring of vehicle contour point clouds using geometric deformation detection technology, the degree of vehicle deformation can be dynamically quantified using initial baseline contour data to avoid the scene limitations of artificial preset fixed models. Combined with time series analysis for instantaneous change rate calculation of environmental temperature and humidity parameters, short-term environmental mutation risk features that are easily overlooked in traditional solutions can be captured. Through waveform feature extraction technology for standard deviation and peak-valley difference normalization processing of voltage fluctuation data, the degree of electrical parameter anomalies can be standardized, and the detection sensitivity of hidden electrical faults can be improved. Further inputting multi-dimensional features into a dynamic weight fusion model can dynamically allocate the weight proportion of vehicle deformation, environmental mutation and voltage anomaly based on historical false positive data, and combine the adaptive division mechanism of quantile threshold intervals to realize environmental adaptability of comprehensive risk scores and effectively suppress false positives and omissions caused by single feature misjudgment or parameter coupling. Ultimately, through the synergistic optimization of multi-modal data fusion and dynamic weight decision, the performance bottleneck of traditional static threshold rules can be broken through, significantly improving the accuracy and robustness of risk identification in complex scenarios, while providing verifiable decision basis for subsequent closed-loop optimization based on federated learning.

[0070] In one embodiment, as shown in Figure 3 The S3 of the charging pile multi-modal vehicle security monitoring method provided by the present application specifically includes the following steps:

[0071] S31: performing hierarchical judgment on the safety judgment result based on a preset alarm rule, and generating a corresponding hierarchical control instruction according to the corresponding risk determination level.

[0072] Specifically, the system classifies the safety judgment results based on preset alarm rules, identifies the corresponding danger level of the current safety state, and generates corresponding hierarchical control instructions accordingly. The hierarchical control instructions cover three types of low, medium and high danger to cope with different degrees of safety risk. When the system determines a low-risk state, the charging pile body will trigger a red light strobe and a buzzer alarm at a preset frequency. The system first sends a low-risk control signal to the control module of the charging pile. After receiving the signal, the control module activates the built-in red light strobe device in the charging pile, which flashes at a preset frequency, and starts the buzzer to emit an alarm sound. The frequency of red light strobe and buzzer alarm can timely remind the on-site personnel that the vehicle has a slight abnormality and needs preliminary attention and inspection without causing excessive panic. The system will continuously monitor the response of the on-site personnel. If no manual intervention signal is received within a certain time, the system will automatically adjust the frequency of the red light and buzzer to further improve the warning effect.

[0073] When the system determines a medium-risk state, the system controls the charging pile to perform electromagnetic locking operation of the charging gun to restrict the improper movement of the vehicle during charging. The system sends a medium-risk control signal to the charging pile. After receiving the signal, the control module activates the electromagnetic locking device of the charging gun, so that the charging gun is tightly locked with the vehicle charging interface, preventing the vehicle from being accidentally moved or the charging gun from being pulled out during charging, thereby avoiding safety problems caused by unstable charging connection. At the same time, the system automatically adjusts the charging output frequency to the safety threshold range. This process involves real-time monitoring and dynamic adjustment of the charging output frequency. The system adjusts the output parameters of the charging equipment according to the preset safety threshold range to ensure that the charging process continues in a relatively stable state and prevent further safety problems caused by abnormal electrical parameters. The system continuously monitors the electrical parameters during charging to ensure that they always remain within the safety threshold range.

[0074] When the system determines a high-risk state, it will immediately trigger the charging pile to stop charging and start the physical locking mechanism of the charging gun. The system sends a high-risk control signal to the charging pile. After receiving the signal, the control module quickly cuts off the charging power supply to ensure that the charging of the vehicle is stopped in the shortest time. At the same time, the physical locking device of the charging gun is started to lock the charging gun at the vehicle charging interface through mechanical structure to prevent the charging gun from being pulled out or the vehicle from moving, avoiding possible serious safety accidents such as arc discharge or short circuit. This series of operations aims to maximize the safety of personnel and vehicles and prevent the danger from further expanding. After executing the high-risk control instruction, the system will record detailed event logs and send alarm information to the remote monitoring center for further processing by professional personnel.

[0075] S32: Process the hierarchical control instructions and the multi-modal fusion data set based on the time series data binding technology, align the instruction generation timestamp with the corresponding sensor data collection timestamp, and generate the original evidence data package.

[0076] Specifically, at the moment of generating the hierarchical control instructions, the system records the timestamp of instruction generation and accurately aligns it with the corresponding sensor data collection timestamp in the multi-modal fusion data set. This process ensures the consistency and traceability of the instructions and data in the time dimension, so that each control instruction can be accurately corresponded to a specific sensor data collection time, generating an original evidence data package containing complete time information and operation instructions. This data package not only covers the detailed content of the hierarchical control instructions, but also integrates multi-modal information such as vehicle contour point cloud data, environmental temperature and humidity parameters, and charging voltage fluctuation data, providing detailed and accurate original data support for subsequent event tracing and responsibility definition.

[0077] S33: Process the original evidence data package based on the cryptographic integrity protection technology, generate a data verification code through the SHA-256 hash algorithm, and perform digital signature on the data package using the charging pile private key to generate a tamper-proof trusted evidence chain.

[0078] Specifically, the system uses the SHA-256 hash algorithm to perform hash operation on the original evidence data package, generating a unique data verification code. This hash algorithm has high collision resistance and one-wayness, ensuring that any minor tampering with the data package can be accurately detected. Then, the system uses the private key of the charging pile to perform digital signature operation on the data package, adding the sender's identity verification information to the data package. The private key of the charging pile is generated by an authenticated security module during the production or initialization of the charging pile and stored in the dedicated security chip of the charging pile. This chip has strong anti-physical attack capability and can prevent the private key from being illegally extracted or copied. The use of the private key is subject to strict permission management and access control, and is called through the internal operation mechanism of the security chip only when necessary operations such as digital signature are performed, ensuring that the private key is not exposed to the external environment. In addition, the system also regularly evaluates and updates the private key to deal with potential security threats.

[0079] After the data package is digitally signed by the private key of the charging pile, the generated signature information and the data verification code together constitute a tamper-proof trusted evidence chain. This process not only ensures the integrity and non-tamperability of the data during transmission and storage, but also verifies the reliability of the data source through digital signature, effectively preventing data from being forged or tampered with. The generation of the trusted evidence chain provides a solid data guarantee for the security monitoring of the charging pile, so that each operation instruction and data record can be accurately traced and verified, enhancing the security and credibility of the entire system.

[0080] The multi-modal vehicle security monitoring method based on charging piles provided above can effectively solve the security protection failure problem caused by extensive response strategy and difficult data traceability in traditional charging pile security systems through the hierarchical response and data trust processing mechanism. The hierarchical control instruction generation logic based on alarm rules can dynamically match differentiated response strategies according to low-risk, medium-risk, and high-risk determination results: through low-risk instructions to trigger non-intrusive sound and light alarms to alert on-site risks, medium-risk instructions to execute charging gun electromagnetic locking and frequency limitation to balance security protection and charging continuity, and high-risk instructions to start charging interruption and mechanical locking to block major safety hazards, thereby building a progressive risk disposal system and avoiding the user experience deterioration or insufficient protection caused by a single power-off strategy in traditional solutions; combined with the time sequence data binding technology, the timestamps of the control instructions and multi-modal data are aligned to realize accurate association of operation and monitoring data to support full-link traceability analysis of security events; further, through the cryptographic integrity protection technology, the data packet is executed for hash verification and digital signature to build a non-repudiable and tamper-proof trusted evidence chain to solve the defects of easily tampered operation logs or insufficient evidence effectiveness in traditional solutions, and to provide data support with legal effect for judicial evidence and responsibility definition. Through the synergistic effect of hierarchical control, time sequence binding, and cryptography verification, the security protection accuracy, operation traceability, and data credibility can be comprehensively improved, and the dual bottlenecks of rigid response strategy and data trust loss in existing technologies are systematically broken through.

[0081] In one embodiment, as shown in FIG. 4, the multi-modal vehicle security monitoring method based on charging piles provided by the present application comprises the following steps: Figure 4 S4: The S4 of the multi-modal vehicle security monitoring method based on charging piles provided by the present application comprises the following steps:

[0082] S41: Based on the security communication protocol, the trusted evidence chain is processed, the evidence chain data including the timestamp, operation log, and encryption verification code is sent to the user terminal, and the feedback label data returned by the user terminal is obtained, the feedback label data including the false alarm confirmation mark or the real threat confirmation mark.

[0083] Specifically, before sending the evidence chain data containing timestamps, operation logs, and encryption check codes to the user terminal, the system encrypts the data using the Advanced Encryption Standard (AES) and adds a Message Authentication Code (MAC) to verify the integrity of the data. The encrypted data is transmitted to the user terminal through a secure communication channel such as the TLS / SSL protocol. After receiving the data, the user terminal decrypts it using the system's public key and verifies the MAC to ensure that the data has not been tampered with. The user confirms the security judgment result generated by the system according to the actual situation and returns feedback label data. This feedback label data includes a false positive confirmation mark or a real threat confirmation mark. When the system receives the feedback label data, it verifies the data to ensure its reliability and that it has not been tampered with. This process not only protects the security of data during transmission but also ensures the accuracy and credibility of user feedback, thereby improving the overall security of the system.

[0084] S42: Process the feedback label data based on the federated learning framework, aggregate the local model gradients of multiple charging pile nodes, calculate the gradient update direction of the weight parameters in the dynamic weight fusion model, and generate a global weight update vector.

[0085] Specifically, the system aggregates the local model gradients of multiple charging pile nodes to calculate the gradient update direction of the weight parameters in the dynamic weight fusion model. In this process, each charging pile node calculates the model gradient based on local data and sends the encrypted gradients to the system. After receiving these encrypted gradients, the system decrypts and aggregates them through secure multi-party computation to calculate the global weight update vector. The federated learning framework ensures data privacy in this way, as the original data never leaves the respective charging pile nodes, thus protecting user privacy and data security. At the same time, the system can capture data features in different scenarios more comprehensively by aggregating gradient information from multiple nodes, improving the model's generalization ability and accuracy.

[0086] S43: Process the global weight update vector based on differential privacy technology, add Laplace noise to the gradient data for desensitization, and generate optimized weight parameters that meet privacy protection requirements.

[0087] Specifically, differential privacy desensitizes gradient data by adding Laplace noise. During processing, the system determines the strength of the noise based on the privacy budget and the sensitivity of the data. The amount of Laplace noise added is carefully calculated to ensure that privacy is protected while minimizing the negative impact on model accuracy. The system adds an appropriate amount of noise to each element of the global weight update vector, generating optimized weight parameters that meet privacy protection requirements. This processing method ensures that even if an attacker obtains partial gradient information, they cannot accurately infer the details of the original data, effectively protecting data privacy. In this way, the system optimizes model parameters while ensuring the confidentiality and security of user data, providing a guarantee for the long-term stable operation of the system.

[0088] S44: Process the preset quantile threshold interval based on the dynamic threshold iteration algorithm, adjust the threshold boundary value according to the quantile distribution of historical score data and the false positive rate of feedback labels, and generate an updated threshold interval.

[0089] Specifically, the dynamic threshold iteration algorithm dynamically adjusts the threshold boundary value according to the quantile distribution of historical score data and the false positive rate of feedback labels. The system first analyzes the distribution of historical score data to determine the quantile interval of different risk levels. By calculating the percentiles of historical data, the system can identify the distribution range of low-risk, medium-risk and high-risk states. Then, combined with the false positive rate of user feedback, the system fine-tunes the threshold boundary. If the false positive rate is too high, the system will expand the low-risk threshold interval and reduce the medium-risk and high-risk intervals to reduce false positives; if the false positive rate is too low, the system will appropriately reduce the low-risk interval and expand the medium-risk and high-risk intervals to improve the detection ability of real threats. After adjustment, the system generates an updated threshold interval. This dynamic adjustment process not only improves the accuracy of the system, but also enhances its adaptability, enabling it to better cope with changing environments and data distributions.

[0090] S45: Process the optimized weight parameters and the updated threshold interval based on the encrypted transmission protocol, update the dynamic weight parameters of the dynamic weight fusion model and the threshold interval of the preset quantile threshold interval.

[0091] Specifically, the system processes the optimized weight parameters and the updated threshold interval based on an encrypted transmission protocol, ensuring the security of updating the dynamic weight parameters of the dynamic weight fusion model and the threshold interval of the preset quantile threshold interval. Before updating, the system will encrypt these parameters. The encryption process uses advanced encryption algorithms (such as RSA or ECC) to ensure the security of the parameters during transmission and storage. The system sends the encrypted parameters to each charging pile node, and after receiving the encrypted parameters, the charging pile node uses the corresponding private key to decrypt and update the local dynamic weight fusion model and threshold interval. This process not only protects the security of model parameters and threshold updates, but also prevents parameters from being stolen or tampered with during transmission, ensuring the overall security and reliability of the system. In this way, the system can continuously optimize its performance while protecting user data and model parameter privacy, providing practical protection for users.

[0092] In summary, the charging pile multi-modal vehicle security monitoring method provided by the present application can effectively solve the problem of insufficient self-adaptive ability caused by model solidification, data islanding and privacy leakage in traditional charging pile security systems through closed-loop learning and privacy protection collaborative mechanism. Based on the secure communication protocol, the two-way interaction of trusted evidence chain and user feedback data can be realized, and the user participation false alarm verification channel can be constructed to improve the credibility of the risk criterion and the transparency of the system; through the federal learning framework, the local model gradient of multiple nodes can be aggregated to realize collaborative training across charging piles without sharing original data, which can break through the limitation of single node data samples while protecting data sovereignty; combined with differential privacy technology, noise disturbance is added to the global gradient update vector to realize privacy desensitization during model optimization, which can resist privacy attack methods such as gradient backpropagation; based on the dynamic threshold iteration algorithm, historical quantiles and feedback false alarm rates are fused to realize scene adaptive adjustment of threshold boundaries, which can solve the generalization ability defect of traditional fixed threshold in complex environments; finally, the model parameters and thresholds are updated through the encrypted transmission protocol, which can build a technical closed loop from data feedback, collaborative learning to deployment optimization, continuously improve the detection accuracy and environmental adaptability of the security model under the premise of protecting user privacy and data security, and systematically solve the core pain points of model iteration lag, privacy risk and low cross-node collaboration efficiency in existing technologies.

[0093] Preferably, as shown in Figure 5 The present application provides a charging pile multi-modal vehicle security monitoring system 500, which is configured with the following modules:

[0094] The data acquisition and processing module 510 is configured to acquire, in real time, profile point cloud data of a vehicle, environmental temperature and humidity parameters, and charging voltage fluctuation data through a stereo vision sensor, a temperature and humidity sensor, and a current sensor integrated with the charging pile, and perform space-time synchronization on the profile point cloud data, the environmental temperature and humidity parameters, and the charging voltage fluctuation data, to generate a multi-modal fusion data set.

[0095] The judgment result generation module 520 is configured to perform comprehensive anomaly calculation and anomaly identification on the multi-modal fusion data set based on a preset dynamic weight fusion model and a preset quantile threshold interval, to generate a safety judgment result, wherein the safety judgment result includes one of a low-risk judgment result, a medium-risk judgment result, and a high-risk judgment result.

[0096] The hierarchical control instruction generation module 530 is configured to identify and judge the safety judgment result based on a preset alarm rule, to generate a corresponding hierarchical control instruction, wherein the hierarchical control instruction is used to control the charging pile to execute one of a low-risk control instruction, a medium-risk control instruction, and a high-risk control instruction.

[0097] The trusted evidence chain construction module 540 is configured to data bind and encrypt the hierarchical control instruction and the multi-modal fusion data set, to construct a trusted evidence chain containing a time stamp, an operation log, and an encrypted check code.

[0098] The data communication module 550 is configured to send the trusted evidence chain to a user terminal and obtain user feedback data fed back by the user terminal.

[0099] The model parameter update module 560 is configured to process the user feedback data based on a federated learning framework, to update dynamic weight parameters of the dynamic weight fusion model and threshold intervals of the preset quantile threshold interval.

[0100] In summary, the multi-modal vehicle security monitoring system based on charging piles provided by the application can effectively solve the misjudgment problem caused by the one-sidedness of single sensor data in traditional charging pile security systems, and overcome the poor adaptability of static threshold rules in complex scenarios. Through the cooperative collection and spatio-temporal synchronous processing of stereo vision, temperature and humidity, and current sensors, comprehensive monitoring of vehicle deformation, environmental mutation, and charging abnormalities can be achieved to construct a multi-modal data set with multi-dimensional associated features, thereby improving the coverage dimension of risk identification. Based on the dynamic weight fusion model, the contribution weights of vehicle deformation, temperature and humidity mutation, and voltage abnormality features can be dynamically adjusted according to the historical false alarm rate, and the adaptive division of quantile threshold intervals can be combined to realize the environmental adaptability of risk scoring, significantly reducing the false alarm and missed alarm probability caused by environmental noise or parameter coupling. Through the binding and encryption of hierarchical control instructions to generate a trusted evidence chain, traceability and tamper-proof verification of the entire process of security events can be achieved to provide trusted data support for responsibility definition and judicial evidence. Further, by using the federated learning framework to aggregate multi-node user feedback data, the model parameters and threshold intervals can be updated through gradient optimization and differential privacy protection technology to realize collaborative learning and dynamic optimization across charging piles while ensuring user privacy, thereby constructing a closed-loop security decision system for continuous improvement of monitoring accuracy and scene generalization ability. Thus, a technical closed loop from data collection, risk decision, instruction execution to feedback optimization is formed to systematically solve the core problems of single monitoring dimension, rule solidification, and privacy leakage in traditional solutions.

[0101] Preferably, the data acquisition and processing module 510 provided by the application is configured with the following units:

[0102] A vehicle contour data generation unit for acquiring contour point cloud data of a vehicle through a stereo vision sensor integrated with a charging pile, marking the contour point cloud data with a time stamp frame by frame according to a preset frame rate, converting the contour point cloud data from a sensor coordinate system to a charging pile reference coordinate system based on calibration parameters, and generating spatially aligned vehicle contour three-dimensional data;

[0103] A temperature and humidity change curve generation unit for periodically acquiring environmental temperature and humidity parameters through a temperature and humidity sensor, performing time series filtering processing on the environmental temperature and humidity parameters, and generating a continuous and smooth temperature and humidity change curve after eliminating sensor noise interference;

[0104] A charging voltage fluctuation data generation unit for monitoring voltage and current parameters output by a charging pile in real time through a current sensor, performing instantaneous fluctuation detection on the voltage and current parameters, extracting abnormal fluctuation features in the charging voltage waveform, and generating charging voltage fluctuation data;

[0105] The multi-modal fusion dataset generation unit is configured to perform spatio-temporal synchronization processing on the vehicle contour three-dimensional data, the temperature and humidity change curve, and the charging voltage fluctuation data, align the time dimension according to the time stamp, and align the space dimension through the charging pile reference coordinate system, to generate a multi-modal fusion dataset containing visual, environmental, and electrical parameter associated features.

[0106] Preferably, the judgment result generation module 520 is configured with the following units:

[0107] The vehicle deformation parameter generation unit is configured to process the vehicle contour point cloud data in the multi-modal fusion dataset based on a geometric deformation detection technology, to generate a vehicle deformation parameter representing the degree of physical deformation of the vehicle by calculating the offset of the key regions on the surface of the vehicle from the initial reference contour data, wherein the initial reference contour data is used to indicate the contour point cloud data collected for the first time in the normal parking state of the vehicle.

[0108] The environmental mutation feature generation unit is configured to process the environmental temperature and humidity parameters based on a time series analysis technology, to generate an environmental mutation feature reflecting the environmental mutation risk by calculating the instantaneous change rate of the temperature and humidity parameters through the first derivative.

[0109] The voltage abnormal fluctuation coefficient generation unit is configured to process the charging voltage fluctuation data based on a waveform feature extraction technology, to generate a voltage abnormal fluctuation coefficient representing the degree of voltage abnormality by standard deviation calculation and peak-valley difference normalization.

[0110] The safety judgment result generation unit is configured to input the vehicle deformation parameter, the environmental mutation feature, and the voltage abnormal fluctuation coefficient into a preset dynamic weight fusion model for processing based on a preset quantile threshold interval, to generate a safety judgment result.

[0111] Preferably, the safety result judgment generation unit includes a weight fusion update subunit, an abnormal score subunit, and a safety judgment subunit. The weight fusion update subunit is configured to train the preset dynamic weight fusion model based on historical abnormal event data, to dynamically adjust the dynamic weight parameters of the dynamic weight fusion model through the gradient descent algorithm according to the false alarm contribution of the vehicle deformation parameter, the environmental mutation feature, and the voltage abnormal fluctuation coefficient in the historical abnormal event data in the historical abnormal event data, to obtain an updated dynamic weight fusion model; the abnormal score subunit is configured to perform weighted summation on the vehicle deformation parameter, the environmental mutation feature, and the voltage abnormal fluctuation coefficient based on the updated dynamic weight fusion model, to generate a comprehensive abnormal score; and the safety judgment subunit is configured to process the comprehensive abnormal score based on a quantile threshold determination rule, to divide the risk degree according to a preset threshold interval, and to generate a safety judgment result, wherein the preset threshold interval includes a low-risk threshold interval, a medium-risk threshold interval, and a high-risk threshold interval.

[0112] Preferably, the trusted evidence chain construction module 540 provided by the application is configured with the following units:

[0113] The original evidence data package generation unit is configured to process the hierarchical control instruction and the multi-modal fusion data set based on the time series data binding technology, align the instruction generation timestamp with the corresponding sensor data acquisition timestamp, and generate an original evidence data package.

[0114] The trusted evidence chain generation unit is configured to process the original evidence data package based on the cryptographic integrity protection technology, generate a data check code through the SHA-256 hash algorithm, and perform digital signature on the data package using the charging pile private key to generate a tamper-proof trusted evidence chain.

[0115] Preferably, the model parameter updating module 560 provided by the application is configured with the following units:

[0116] The global weight update vector generation unit is configured to process the feedback label data based on the federated learning framework, calculate the gradient update direction of the weight parameter in the dynamic weight fusion model by aggregating the local model gradients of multiple charging pile nodes, and generate a global weight update vector.

[0117] The optimized weight parameter generation unit is configured to process the global weight update vector based on the differential privacy technology, desensitize the gradient data by adding Laplace noise, and generate optimized weight parameters that meet privacy protection requirements.

[0118] The updated threshold interval generation unit is configured to process the preset quantile threshold interval based on the dynamic threshold iteration algorithm, adjust the threshold boundary value according to the quantile distribution of historical score data and the false positive rate of feedback labels, and generate an updated threshold interval.

[0119] The model and threshold updating unit is configured to process the optimized weight parameters and the updated threshold interval based on the encrypted transmission protocol, update the dynamic weight parameters of the dynamic weight fusion model and the threshold interval of the preset quantile threshold interval.

[0120] In one embodiment, the application also provides a computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the above-mentioned charging pile multi-modal vehicle security monitoring method when executing the computer program.

[0121] In one embodiment, the application also provides a computer readable storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the above-mentioned charging pile multi-modal vehicle security monitoring method.

[0122] In the description of the specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, the skilled person in the art can combine and combine the different embodiments or examples described in the specification and the features of the different embodiments or examples without contradiction.

[0123] For the device embodiment, since it basically corresponds to the method embodiment, the relevant part is described in the part of the method embodiment. The above described device embodiment is only schematic, wherein the components described as separate components can or can not be physically separate, and the components displayed as a unit can or can not be a physical unit, i.e. can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purpose of the present disclosure according to the actual needs. Those skilled in the art can understand and implement it without creative labor.

[0124] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited to this. Any skilled person in the art can easily think of various changes or replacements within the technical range disclosed in the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A multi-modal vehicle security monitoring method based on charging piles, characterized in that, Includes the following steps: S1: By using the stereo vision sensor, temperature and humidity sensor and current sensor integrated in the charging pile, the vehicle's contour point cloud data, environmental temperature and humidity parameters and charging voltage fluctuation data are collected in real time, and the contour point cloud data, environmental temperature and humidity parameters and charging voltage fluctuation data are spatiotemporally synchronized to generate a multimodal fusion dataset. S2: Based on a preset dynamic weight fusion model and a preset quantile threshold range, perform comprehensive anomaly calculation and anomaly identification on the multimodal fusion dataset to generate a security judgment result, which includes one of a low-risk judgment result, a medium-risk judgment result, and a high-risk judgment result; S3: Based on preset alarm rules, identify and judge the security judgment result, generate corresponding hierarchical control instructions, and bind and encrypt the hierarchical control instructions with the multimodal fusion dataset to construct a trusted evidence chain containing timestamps, operation logs and encrypted verification codes; the hierarchical control instructions are used to control the charging pile to execute one of the low-risk control instructions, medium-risk control instructions and high-risk control instructions; S4: Send the trusted evidence chain to the user terminal and obtain the user feedback data from the user terminal. Process the user feedback data based on the federated learning framework and update the dynamic weight parameters of the dynamic weight fusion model and the threshold interval of the preset quantile threshold interval.

2. The method according to claim 1, characterized in that, S1 includes: S11: Collect vehicle contour point cloud data through the stereo vision sensor integrated in the charging pile, mark the contour point cloud data frame by frame with a preset frame rate, and transform the contour point cloud data from the sensor coordinate system to the charging pile reference coordinate system based on the calibration parameters to generate spatially aligned three-dimensional vehicle contour data. S12: The ambient temperature and humidity parameters are periodically collected by the temperature and humidity sensor, and the ambient temperature and humidity parameters are subjected to time series filtering to eliminate sensor noise interference and generate a continuous and smooth temperature and humidity change curve. S13: Monitor the voltage and current parameters output by the charging pile in real time through a current sensor, perform instantaneous fluctuation detection on the voltage and current parameters, extract abnormal fluctuation characteristics in the charging voltage waveform, and generate charging voltage fluctuation data. S14: Perform spatiotemporal synchronization processing on the vehicle outline 3D data, the temperature and humidity change curve, and the charging voltage fluctuation data. Align the time dimension according to the timestamp and align the spatial dimension through the charging pile reference coordinate system to generate a multimodal fusion dataset containing visual, environmental, and electrical parameter related features.

3. The method according to claim 1, characterized in that, S2 includes: S21: Based on geometric deformation detection technology, the vehicle contour point cloud data in the multimodal fusion dataset is processed. By calculating the offset between the key areas of the vehicle surface and the initial reference contour data, vehicle deformation parameters characterizing the degree of physical deformation of the vehicle are generated. The initial reference contour data is used to indicate the contour point cloud data collected for the first time under the normal parking state of the vehicle. S22: The environmental temperature and humidity parameters are processed based on time series analysis technology. The instantaneous change rate of the temperature and humidity parameters is calculated by the first derivative to generate environmental mutation characteristics. These environmental mutation characteristics are used to reflect the risk of environmental mutation. S23: The charging voltage fluctuation data is processed based on waveform feature extraction technology, and a voltage anomaly fluctuation coefficient representing the degree of voltage anomaly is generated by standard deviation calculation and peak-valley difference normalization. S24: Based on a preset quantile threshold range, the vehicle deformation parameters, the environmental abrupt change characteristics, and the voltage anomaly fluctuation coefficient are input into a preset dynamic weight fusion model for processing to generate a safety judgment result.

4. The method according to claim 3, characterized in that, S24 includes: S241: The preset dynamic weight fusion model is trained based on historical abnormal event data. According to the vehicle deformation parameters, environmental mutation characteristics and voltage abnormal fluctuation coefficient of the historical abnormal event data, the dynamic weight parameters of the dynamic weight fusion model are dynamically adjusted by the gradient descent algorithm to obtain the updated dynamic weight fusion model. S242: Based on the updated dynamic weighted fusion model, the vehicle deformation parameters, the environmental abrupt change characteristics, and the voltage anomaly fluctuation coefficient are weighted and summed to generate a comprehensive anomaly score. The formula for calculating the comprehensive anomaly score is as follows: Where S is the comprehensive anomaly score, and δ is the vehicle deformation parameter. As a characteristic of environmental mutation, ΔV err denoted as voltage anomaly fluctuation coefficient, where α, β, and γ are dynamic weighting parameters for vehicle deformation parameters, environmental abrupt change characteristics, and voltage anomaly fluctuation coefficient, respectively. S243: The comprehensive anomaly score is processed based on the quantile threshold determination rule, and the degree of danger is divided according to the preset threshold interval to generate a safety judgment result. The preset threshold interval includes a low-risk threshold interval, a medium-risk threshold interval, and a high-risk threshold interval.

5. The method according to claim 1, characterized in that, S3 includes: S31: Based on preset alarm rules, the safety judgment result is classified and judged, and corresponding graded control instructions are generated according to the corresponding danger judgment level. S32: Based on time-series data binding technology, the hierarchical control commands and multimodal fusion datasets are processed, and the command generation timestamp is aligned with the corresponding sensor data acquisition timestamp to generate the original evidence data package; S33: The original evidence data packet is processed based on cryptographic integrity protection technology, a data verification code is generated by the SHA-256 hash algorithm, and the data packet is digitally signed using the charging pile private key to generate a tamper-proof and trustworthy evidence chain.

6. The method according to claim 5, characterized in that, The tiered control instructions include one of the following: low-risk control instructions, medium-risk control instructions, and high-risk control instructions. The low-risk control command is used to trigger the charging pile body to start red light flashing and buzzer alarm at a preset frequency; the medium-risk control command is used to control the charging pile to perform electromagnetic locking of the charging gun and adjust the charging output frequency to a safe threshold range; the high-risk control command is used to trigger the charging pile to stop charging in an emergency and start physical locking of the charging gun.

7. The method according to any one of claims 1-6, characterized in that, S4 includes: S41: Process the trusted evidence chain based on the secure communication protocol, send the evidence chain data containing timestamps, operation logs and encrypted verification codes to the user terminal, and obtain the feedback tag data returned by the user terminal. The feedback tag data includes a false alarm confirmation tag or a real threat confirmation tag. S42: The feedback label data is processed based on the federated learning framework. By aggregating the local model gradients of multiple charging pile nodes, the gradient update direction of the weight parameters in the dynamic weight fusion model is calculated, and a global weight update vector is generated. S43: The global weight update vector is processed based on differential privacy technology. The gradient data is desensitized by adding Laplacian noise to generate optimized weight parameters that meet privacy protection requirements. S44: The preset quantile threshold interval is processed based on the dynamic threshold iteration algorithm. According to the quantile distribution of historical scoring data and the false alarm rate of feedback labels, the threshold boundary value is adjusted to generate an updated threshold interval. S45: Process the optimized weight parameters and the updated threshold range based on the encrypted transmission protocol, and update the dynamic weight parameters of the dynamic weight fusion model and the threshold range of the preset quantile threshold range.

8. A multimodal vehicle security monitoring system based on charging piles, characterized in that, The system includes: The data acquisition and processing module is used to collect vehicle contour point cloud data, environmental temperature and humidity parameters, and charging voltage fluctuation data in real time through the stereo vision sensor, temperature and humidity sensor, and current sensor integrated in the charging pile, and to perform spatiotemporal synchronization of the contour point cloud data, the environmental temperature and humidity parameters, and the charging voltage fluctuation data to generate a multimodal fusion dataset. The judgment result generation module is used to perform comprehensive anomaly calculation and anomaly identification on the multimodal fusion dataset based on a preset dynamic weight fusion model and a preset quantile threshold range, and generate a security judgment result, which includes one of a low-risk judgment result, a medium-risk judgment result, and a high-risk judgment result. The hierarchical control instruction generation module is used to identify and judge the safety judgment result based on preset alarm rules, and generate corresponding hierarchical control instructions. The hierarchical control instructions are used to control the charging pile to execute one of the low-risk control instructions, medium-risk control instructions, and high-risk control instructions. A trusted evidence chain construction module is used to bind and encrypt the hierarchical control instructions with the multimodal fusion dataset to construct a trusted evidence chain containing timestamps, operation logs, and encrypted verification codes. The data communication module is used to send the trusted evidence chain to the user terminal and obtain user feedback data from the user terminal. The model parameter update module is used to process the user feedback data based on the federated learning framework and update the dynamic weight parameters of the dynamic weight fusion model and the threshold interval of the preset quantile threshold interval.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images