Real-time monitoring method and system for emotion data abuse risk

Abstract

The invention provides an emotion data abuse risk real-time monitoring method and system, and the method comprises the steps: collecting the emotion data of a passenger in real time through a vehicle-mounted terminal, and completing the anonymization processing in an edge computing device; monitoring abnormal access in real time according to an anonymization processing result, judging whether the current abnormal access is misjudged when the abnormal access occurs, and if not, performing dynamic blocking, namely performing active defense and attack countering; if yes, normal user operation judged to be abnormal is extracted from a security log generated in the monitoring process, a tracking watermark record triggered by an attacker is captured to serve as dynamic optimization data, attack behaviors and false alarm types are automatically marked, secondary confirmation is conducted on fuzzy samples through manual auditing, and training is optimized through incremental learning. According to the invention, end-side anonymization processing, encrypted storage and distribution are adopted, so that privacy protection, attack defense and continuous evolution optimization can be carried out on the vehicle-mounted emotion in the collection, transmission, storage and use processes.

Description

Technical Field

[0001] The present invention relates to the technical field of data security monitoring, and in particular to a real-time monitoring method and system for the risk of abuse of emotional data. Background Art

[0002] The misuse of emotional data can lead to personal information being obtained by criminals, which in turn can trigger targeted fraud, identity theft and other criminal activities. These activities not only threaten personal property security, but also seriously infringe on personal privacy rights.

[0003] Traditional data anonymization technologies apply global mosaicing or Gaussian blurring to facial images, resulting in low accuracy in facial emotion detection. They also apply noise reduction or pitch shifting to speech within a fixed frequency range. Monitoring is performed by setting only administrator and user roles, fixing accessible IP addresses, and establishing static rules. Network traffic monitoring captures network-layer data packets to analyze traffic characteristics, but this consumes significant resources and allows attackers to forward malicious traffic through legitimate CDN nodes.

[0004] Patent document CN118124499A discloses a vehicle-based user data processing method, apparatus, device, and medium. The method provided in embodiments of this application prevents unauthorized access to the driver's private data via Bluetooth by detecting the driver's presence and connection status with the vehicle's onboard Bluetooth. Furthermore, by acquiring control commands triggered by the driver using steering wheel buttons, the driver's operational intent can be determined and corresponding security protection actions can be executed based on the type of control command. This protects the driver's associated user data.

[0005] However, patent document CN118124499A only protects the driver's data information privacy, does not provide protection for passengers, and does not perform monitoring and defense. In addition, the level of protection of its privacy data still needs to be improved. Summary of the Invention

[0006] In view of the defects in the prior art, the purpose of the present invention is to provide a real-time monitoring method and system for the risk of abuse of emotional data.

[0007] A real-time monitoring method for the risk of emotional data abuse provided by the present invention includes:

[0008] Step S1: Collect passenger emotion data in real time through the vehicle terminal and perform anonymization processing in the edge computing device;

[0009] Step S2: Monitor abnormal access in real time based on the anonymization results. When an abnormal access occurs, determine whether the current abnormal access is a false positive. If not, dynamically block it, i.e., perform active defense and attack countermeasures. If so, execute step S3.

[0010] Step S3: Extract normal user operations that are judged to be abnormal from the security logs generated during the monitoring process, and capture the tracking watermark records triggered by the attacker as dynamic optimization data, automatically annotate the attack behavior and false alarm types, perform secondary confirmation on the fuzzy samples during manual review, and optimize training through incremental learning.

[0011] Preferably, the emotional data includes facial images, voice stream information and physiological signals;

[0012] The step S1 comprises:

[0013] Step S1.1: For the facial image data, extract the coordinates of key points used for sentiment analysis calculation using a facial key point detection model based on the MediaPipe model, and anonymize and encrypt the areas surrounding the key points;

[0014] Step S1.2: For the voice stream information, separate the mixed features into voiceprint feature vectors and emotion feature vectors based on the pre-trained DCN clustering model, and perform encryption processing;

[0015] Step S1.3: For the physiological signal data, a homomorphic encryption algorithm based on the CKKS scheme is used during collection, so that the model can directly calculate the emotion index in the encrypted state;

[0016] Step S1.4: All encrypted data are divided into blocks and stored in a local secure storage unit on the vehicle, and the encryption key is managed by the vehicle TPM chip.

[0017] Preferably, the key point coordinates include the left and right mouth corner endpoints for calculating the curvature of the mouth, and the left and right eye inner and outer corner points for calculating the eyelid opening and closing degree;

[0018] The step S1.1 includes generating a circular mask area with an adjustable radius centered on each key point, encrypting pixels outside the mask area, dividing the pixels in the encrypted area into 16*16 pixel blocks, and independently performing AES-256 encryption on each area.

[0019] Preferably, the voiceprint feature vector is encrypted using the SM2 national encryption algorithm, and the encrypted voiceprint data is stored in an isolated security domain on the vehicle. The voiceprint encryption key is derived from the user's biometrics and generated using the PBKDF2 algorithm. The key lifecycle is bound to the vehicle user account, and the key is automatically destroyed after the account is cancelled. The original voice stream is fragmented into disordered fragments, and the fragments are distributed to edge nodes and cloud servers. The fragment index table is encrypted using the SM4 national encryption algorithm and stored independently.

[0020] Preferably, the step S1.3 includes the following steps:

[0021] Step S1.3.1: Normalize the physiological signal to a numerical value, and then encrypt the normalized numerical vector using the CKKS public key to generate the ciphertext c_data;

[0022] Step S1.3.2: Perform ciphertext sentiment calculation, including calculating the homomorphic mean to obtain the encrypted mean, calculating the homomorphic standard deviation based on the encrypted mean, then calculating the encrypted standard deviation based on the homomorphic standard deviation, and calculating the sentiment value based on the encrypted standard deviation and the encrypted mean.

[0023] Step 1.3.3: Authorize the pre-deployed air conditioning control system module to call the CKKS private key through the HSM to decrypt and obtain the plaintext emotion value.

[0024] Preferably, the homomorphic mean calculation in step S1.3.2 is to perform homomorphic addition and plaintext scalar multiplication on the c_data ciphertext to obtain the encrypted mean c_mean;

[0025] The formula for calculating the homomorphic standard deviation is as follows:

[0026] C_var=C_mean_square-(C_mean)2

[0027] Where C_var represents the homomorphic variance, C_mean_square represents the homomorphic calculation of the square mean, and the encrypted standard deviation c_std is calculated by Taylor expansion;

[0028] The formula for calculating the sentiment value is as follows:

[0029] Sentiment value = 0.6*c_mean+0.4*c_std.

[0030] Preferably, during the abnormal access monitoring process, all triggered abnormal events are automatically recorded to form a security log;

[0031] The abnormal access includes unauthorized IP access, high-frequency requests, illegal protocols and low-frequency crawling.

[0032] Preferably, the active defense and attack countermeasures include the following steps:

[0033] Step S2.1: Inject false emotion data containing a tracking watermark into the data stream. When the attacker triggers the watermark, a warning is triggered and the attacker's access trajectory is recorded.

[0034] Step S2.2: The data transmission link is superimposed with random noise packets. The noise packets are consistent with the length and transmission frequency of the real data packets, thereby interfering with the attacker's data parsing and cracking process.

[0035] Step S2.3: After detecting the attack, the data encryption key and watermark ID are automatically updated, and other data are checked for leakage, and the affected nodes are automatically isolated.

[0036] Preferably, the watermark ID includes an IP address, a device fingerprint, and a timestamp;

[0037] The false emotional data includes virtual physiological signals and virtual face / voice information; the virtual physiological signals are false data that conform to normal distribution but have abnormal parameters; the virtual face / voice information is a virtual face or voice clip generated by GAN.

[0038] According to the present invention, a real-time monitoring system for the risk of emotional data abuse is provided, comprising:

[0039] Module M1: collects passenger emotion data in real time through the onboard terminal and anonymizes it in the edge computing device;

[0040] Module M2: Monitors abnormal access in real time based on the results of anonymization processing. When an abnormal access occurs, it determines whether the current abnormal access is a false positive. If not, it dynamically blocks it, that is, performs active defense and attack countermeasures. If so, it triggers module M3.

[0041] Module M3: Extracts normal user operations that are judged to be abnormal from the security logs generated during the monitoring process, captures the tracking watermark records triggered by attackers as dynamic optimization data, automatically labels attack behaviors and false positive types, performs secondary confirmation on fuzzy samples during manual review, and optimizes training through incremental learning.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] 1. The present invention can protect the privacy of in-vehicle emotions during the collection, transmission, storage and use of the data by adopting end-side anonymization processing, encrypted storage and distribution.

[0044] 2. The present invention monitors data security during the process of in-vehicle emotion recognition through abnormal access monitoring and dynamic defense, attack countermeasures and incremental learning adaptation, and performs attack defense when the data is attacked. The present invention also solves problems such as continuous evolution and optimization.

[0045] 3. The present invention can dynamically monitor data abuse and improve the accuracy of monitoring results. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments with reference to the following drawings:

[0047] Figure 1It is a schematic flow chart of the working method of the present invention. DETAILED DESCRIPTION

[0048] The present invention will be described in detail below with reference to specific embodiments. The following examples will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those skilled in the art, several changes and improvements can be made without departing from the scope of the present invention. These all fall within the scope of protection of the present invention.

[0049] The present invention is based on the dynamic security protection needs of multimodal biometric data (face, voice, physiological information), and realizes adaptive permission adjustment and attack blocking through differentiated protection of multimodal data, combined with behavioral baseline modeling and real-time context analysis.

[0050] According to a real-time monitoring method for the risk of emotional data abuse provided by the present invention, Figure 1 Shown, including:

[0051] Step S1: Collect passenger emotional data in real time through the vehicle terminal and perform anonymization processing in the edge computing device to ensure that the data cannot be reversibly associated with personal identity. The emotional data includes facial images, voice stream information and physiological signals. Step S1 includes:

[0052] Step S1.1: For the facial image data, the key point coordinates for emotion analysis calculation are extracted through the facial key point detection model based on the MediaPipe model, and the areas around the key points are anonymized and encrypted. The key point coordinates include the endpoints of the left and right corners of the mouth: MediaPipe key points No. 61 and No. 291, which are used to calculate the curvature of the mouth. The inner and outer corners of the left and right eyes: MediaPipe key points No. 33, 133, 362, and 263, which are used to calculate the degree of eyelid opening. With each key point as the center, a circular mask area with an adjustable radius is generated, and the pixels outside the mask area are encrypted. The pixels in the encrypted area are divided into 16*16 pixel blocks, and each area is independently encrypted with AES-256.

[0053] Step S1.2: For the voice stream information, the mixed features are separated into voiceprint feature vectors and emotion feature vectors based on the pre-trained DCN clustering model, and the voiceprint feature vectors are encrypted. The voiceprint feature vectors are encrypted using the SM2 national encryption algorithm, and the encrypted voiceprint data is stored in the isolated security domain on the vehicle. The voiceprint encryption key is derived from the user's biometrics (fingerprint) and generated using the PBKDF2 algorithm. The key lifecycle is bound to the vehicle user account and the key is automatically destroyed after the account is canceled. The original voice stream is fragmented into disordered fragments, which are distributed to edge nodes and cloud servers. The fragment index table is encrypted using the SM4 national encryption algorithm and stored independently.

[0054] Step S1.3: For the physiological signal data, a homomorphic encryption algorithm based on the CKKS scheme is used to collect the data, which supports the model to directly calculate the emotion index in the ciphertext state. Step S1.3 includes the following steps:

[0055] Step S1.3.1: Normalize the physiological signal into a numerical value, and then use the CKKS public key to encrypt the normalized numerical vector to generate the ciphertext c_data.

[0056] Step S1.3.2: Perform ciphertext sentiment calculation, including calculating the homomorphic mean to obtain the encrypted mean, calculating the homomorphic standard deviation based on the encrypted mean, then calculating the encrypted standard deviation based on the homomorphic standard deviation, and calculating the sentiment value based on the encrypted standard deviation and the encrypted mean. Specifically, the homomorphic mean calculation is to perform homomorphic addition and plaintext scalar multiplication on the c_data ciphertext to obtain the encrypted mean c_mean. The formula for calculating the homomorphic standard deviation is as follows:

[0057] C_var=C_mean_square-(C_mean)2

[0058] Where C_var represents the homomorphic variance, C_mean_square represents the homomorphic square mean, and the encrypted standard deviation c_std is calculated through Taylor expansion. The sentiment value calculation formula is as follows:

[0059] Sentiment value = 0.6*c_mean+0.4*c_std

[0060] Step 1.3.3: Authorize the pre-deployed air conditioning control system module to call the CKKS private key through the HSM to decrypt and obtain the plaintext emotion value.

[0061] Step S1.4: All encrypted data are divided into blocks and stored in a local secure storage unit on the vehicle, and the encryption key is managed by the vehicle TPM chip.

[0062] Step S2: Monitor abnormal access in real time based on the results of the anonymization process. When an abnormal access occurs, determine whether the current abnormal access is a misjudgment. If not, perform dynamic blocking, that is, perform active defense and attack countermeasures; if so, execute step S3. At the same time, during the abnormal access monitoring process, all triggered abnormal events are automatically recorded to form a security log. The abnormal access includes unauthorized IP access, high-frequency requests, illegal protocols, and low-frequency crawling. The high-frequency request refers to more than 50 data download requests initiated by the same account or device within 1 hour. The illegal protocol includes accessing data using a non-HTTPS protocol or an unencrypted API interface. The low-frequency crawling includes only downloading once a day, but crawling the same user data for 30 consecutive days. The monitoring method includes identifying illegal protocols and non-encrypted communications based on deep packet inspection (DPI) technology, using a distributed counter cluster to count the request frequency of accounts / devices in real time, combining a sliding time window algorithm to capture high-frequency behavior, and using the LSTM model to analyze user access timing characteristics to identify long-term behavior patterns of low-frequency crawling.

[0063] The active defense and attack countermeasures include the following steps:

[0064] Step S2.1: Inject false emotional data containing a tracking watermark into the data stream. When the attacker triggers the watermark, a warning is triggered and the attacker's access trajectory is recorded. The watermark ID includes the IP address, device fingerprint, and timestamp. For example, the watermark ID = HMAC-SHA256 (session ID + timestamp + device fingerprint). The false emotional data includes virtual physiological signals and virtual face / voice information. The virtual physiological signal is false data that conforms to the normal distribution but has abnormal parameters. The virtual face / voice information is a virtual face or voice clip generated by GAN.

[0065] Step S2.2: The data transmission link is superimposed with a random noise packet, which is consistent with the length and transmission frequency of the real data packet, thereby interfering with the attacker's data parsing and cracking process.

[0066] Step S2.3: After detecting the attack, the data encryption key and watermark ID are automatically updated, and other data are checked for leakage, and the affected nodes are automatically isolated.

[0067] Step S3: Extract normal user actions identified as abnormal by the LSTM model from security logs, capture tracking watermark records triggered by attackers as dynamic optimization data, automatically annotate attack behaviors and false positive types, perform secondary verification of fuzzy samples through manual review, and optimize the LSTM behavior detection model through incremental learning. This incremental learning involves inputting historical behavior data and new samples, freezing the underlying LSTM network, updating only the fully connected layer parameters, and regularly training a new model offline weekly. Alternatively, real-time online learning can be triggered when a new attack pattern is detected.

[0068] The real-time monitoring method for sentiment data abuse risks provided by the present invention also includes a step for ensuring data compliance. This step involves automatically traversing edge nodes and cloud backup data when a user initiates a data deletion request, completely erasing the target data based on hash value matching. Furthermore, the data storage period and encryption strength compliance are dynamically verified based on a multinational regulatory database.

[0069] There is a variation of the present invention, in which the anonymization process after collecting the multimodal emotion data further includes the following steps:

[0070] Step 1: Perform local preprocessing and multimodal alignment processing on the multimodal emotion data; the preprocessing of image modal data includes image denoising, grayscale and normalization, face alignment, expression area cropping and timestamp synchronization; the preprocessing of speech modal data includes denoising, silence detection and segmentation, amplitude normalization, short-time frame division and time alignment marking; the preprocessing of physiological modal data includes filtering and noise reduction, anomaly removal, sliding average, normalization and sampling and alignment to achieve time series alignment of cross-modal data; the multimodal alignment processing introduces a unified timestamp mechanism and performs time alignment operations on all data based on the master time source, including sliding window synchronization, interpolation / cropping strategy and synchronous data packet encapsulation.

[0071] Step 2: Extract the corresponding sensitive emotion feature information from the preprocessed multimodal emotion data through the recognition algorithm, and uniformly encapsulate the identified sensitive emotion feature information; Sensitive feature recognition of image modal data includes face area detection, facial key point extraction, facial feature vector extraction and emotion-irrelevant background feature removal, and obtain the face area position coordinates, key point index and embedded feature vector for desensitization processing; Sensitive feature recognition of speech modal data includes voiceprint feature extraction, spectral component analysis, and speech feature labeling that is unrelated to semantic content. The output includes voiceprint vector, spectral sensitive segment, and frequency band distribution parameters, which are used to control subsequent voiceprint perturbation or speech masking strategy; Sensitive feature recognition of physiological modal data includes resting heart rate feature analysis, GSR skin conductance pattern recognition and heart rate variability HRV time domain / frequency domain features, respiratory rhythm and amplitude analysis.

[0072] Step 3: Desensitize each type of emotional modal data separately, and encapsulate the desensitization results and synchronize them with the labels; desensitization of image modal data uses Gaussian blurring, mosaic occlusion, or replacing the face area with an embedded vector, including face blurring, area occlusion / replacement, embedded replacement representation, and style transfer forgery; desensitization of speech modal data includes voiceprint perturbation processing, spectral feature mapping, speech pseudo-sound transformation, and low-rank reconstruction defeatureization; desensitization of physiological modal data introduces differential privacy algorithms or dimensionality degradation processing, including normalization transformation, differential privacy perturbation, feature subspace selection, and segmented resampling;

[0073] Step 4: Deep emotion features are extracted from the desensitized multimodal emotion data through a multimodal fusion model, and the extracted deep emotion features are fused to form a unified emotion representation vector; deep emotion feature extraction from image modal data includes extracting key expression feature embeddings through a lightweight convolutional neural network or a visual Transformer model, including facial expression embedding vectors, dynamic expression change indicators, and attention heat maps; deep emotion feature extraction from speech modal data includes extracting emotion indicators using a time series modeling structure, including prosodic features, spectral features, rhythm and punctuation patterns, and emotional speech embedding vectors; deep emotion feature extraction from physiological modal data includes features related to heart rate variability, skin conductance fluctuations, and emotional states.

[0074] Step 5: Input the emotion representation vector into a pre-trained emotion recognition model, which outputs the occupant's current emotional state, including the specific emotion category and corresponding confidence level. This emotion recognition model can employ a multi-layer perceptron (MLP) architecture, a multimodal Transformer architecture, or an integrated classifier system. This architecture utilizes a Softmax fusion network or a combination of SVM and random forest models to enable post-decision integration of recognition results from different modalities.

[0075] The present invention also provides a real-time monitoring system for the risk of emotional data abuse. The real-time monitoring system for the risk of emotional data abuse can be implemented by executing the process steps of the real-time monitoring method for the risk of emotional data abuse. That is, those skilled in the art can understand the real-time monitoring method for the risk of emotional data abuse as a preferred implementation of the real-time monitoring system for the risk of emotional data abuse.

[0076] A real-time monitoring system for the risk of emotional data abuse provided by the present invention includes:

[0077] Module M1: collects passenger emotional data in real time through the vehicle terminal and completes anonymization processing in the edge computing device. The emotional data includes facial images, voice stream information and physiological signals. Module M1 includes:

[0078] Module M1.1: For the facial image data, the MediaPipe model is used to extract the key point coordinates for emotion analysis calculations through the facial key point detection model, and the areas around the key points are anonymized and encrypted. The key point coordinates include the left and right mouth corner endpoints for calculating the curvature of the mouth, and the left and right eye inner and outer corners for calculating the eyelid opening and closing. Module M1.1 includes generating a circular mask area with an adjustable radius centered on each key point, encrypting the pixels outside the mask area, dividing the pixels in the encrypted area into 16*16 pixel blocks, and independently performing AES-256 encryption on each area.

[0079] Module M1.2: For the voice stream information, the mixed features are separated into voiceprint feature vectors and emotion feature vectors based on the pre-trained DCN clustering model, and then encrypted. The voiceprint feature vectors are encrypted using the SM2 national encryption algorithm and stored in an isolated security domain onboard the vehicle. The voiceprint encryption key is derived from the user's biometrics and generated using the PBKDF2 algorithm. The key lifecycle is bound to the vehicle user account and is automatically destroyed upon account cancellation. The original voice stream is fragmented into unordered fragments, which are distributed to edge nodes and cloud servers. The fragment index table is encrypted using the SM4 national encryption algorithm and stored independently.

[0080] Module M1.3: For the physiological signal data, when collecting the homomorphic encryption algorithm based on the CKKS scheme, the model is supported to directly calculate the emotion index in the ciphertext state. The module M1.3 includes the following modules: Module M1.3.1: Normalize the physiological signal to a numerical value, and then use the CKKS public key to encrypt the normalized numerical vector to generate the ciphertext c_data. Module M1.3.2: Perform ciphertext emotion calculation, including homomorphic mean calculation to obtain the encrypted mean, calculate the homomorphic standard deviation based on the encrypted mean, and then calculate the encrypted standard deviation based on the homomorphic standard deviation, and calculate the emotion value based on the encrypted standard deviation and the encrypted mean. The homomorphic mean calculation in the module M1.3.2 triggers homomorphic addition and plaintext scalar multiplication on the c_data ciphertext to obtain the encrypted mean c_mean. The formula for calculating the homomorphic standard deviation is as follows: C_var = C_mean_square - (C_mean)², where C_var represents the homomorphic variance and C_mean_square represents the homomorphic square mean. The encrypted standard deviation c_std is calculated using Taylor expansion. The formula for calculating the sentiment value is as follows: Sentiment value = 0.6 * c_mean + 0.4 * c_std. Module 1.3.3: Authorized pre-deployed air conditioning control system modules can use the HSM to call the CKKS private key for decryption and obtain the plaintext sentiment value.

[0081] Module M1.4: All encrypted data is stored in blocks in a local secure storage unit on the vehicle, and the encryption key is managed by the on-board TPM chip.

[0082] Module M2: Monitors abnormal access in real time based on the anonymization results. When an abnormal access occurs, it determines whether it is a false positive. If not, it dynamically blocks the access, effectively implementing active defense and attack countermeasures. If so, Module M3 is triggered. During the abnormal access monitoring process, all triggered abnormal events are automatically recorded to form a security log. Abnormal access includes unauthorized IP access, high-frequency requests, illegal protocols, and low-frequency crawling. This active defense and attack countermeasures include the following modules: Module M2.1: Injects false emotional data containing a tracking watermark into the data stream. When an attacker triggers the watermark, an alert is triggered and the attacker's access trajectory is recorded. Module M2.2: Overlays random noise packets onto the data transmission link. These noise packets match the length and frequency of real data packets, thereby interfering with the attacker's data parsing and cracking process. Module M2.3: Upon detecting an attack, the data encryption key and watermark ID are automatically updated, and other data is checked for leaks, automatically isolating the affected node. The watermark ID includes the IP address, device fingerprint, and timestamp. The false emotional data includes simulated physiological signals and simulated facial / voice information. The virtual physiological signal is a false data with normal distribution but abnormal parameters. The virtual face / voice information is generated by using GAN to generate a virtual face or voice segment.

[0083] Module M3: Extracts normal user operations that are judged to be abnormal from the security logs generated during the monitoring process, captures the tracking watermark records triggered by attackers as dynamic optimization data, automatically labels attack behaviors and false positive types, performs secondary confirmation on fuzzy samples during manual review, and optimizes training through incremental learning.

[0084] Those skilled in the art will appreciate that, in addition to implementing the system and its various devices, modules, and units provided by the present invention in purely computer-readable program code, it is entirely possible to implement the same functions of the system and its various devices, modules, and units provided by the present invention in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; the devices, modules, and units for implementing various functions can also be considered as both software modules implementing the method and structures within the hardware component.

[0085] The above describes specific embodiments of the present invention. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art may make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. The embodiments of this application and the features in the embodiments may be combined with each other in any manner unless there is a conflict.

Claims

1. A real-time monitoring method for the risk of emotional data abuse, characterized in that: include: Step S1: Collect passenger emotion data in real time through the vehicle terminal and perform anonymization processing in the edge computing device; Step S2: Monitor abnormal access in real time based on the anonymization results. When an abnormal access occurs, determine whether the current abnormal access is a false positive. If not, dynamically block it, i.e., perform active defense and attack countermeasures. If so, execute step S3. Step S3: Extract normal user operations that are judged to be abnormal from the security logs generated during the monitoring process, and capture the tracking watermark records triggered by the attacker as dynamic optimization data, automatically annotate the attack behavior and false alarm types, perform secondary confirmation on the fuzzy samples during manual review, and optimize training through incremental learning.

2. The real-time monitoring method for the risk of emotional data abuse according to claim 1, characterized in that: The emotional data includes facial images, voice stream information and physiological signals; The step S1 comprises: Step S1.1: For the facial image data, extract the coordinates of key points used for sentiment analysis calculation using a facial key point detection model based on the MediaPipe model, and anonymize and encrypt the areas surrounding the key points; Step S1.2: For the voice stream information, separate the mixed features into voiceprint feature vectors and emotion feature vectors based on the pre-trained DCN clustering model, and perform encryption processing; Step S1.3: For the physiological signal data, a homomorphic encryption algorithm based on the CKKS scheme is used during collection, so that the model can directly calculate the emotion index in the encrypted state; Step S1.4: All encrypted data are divided into blocks and stored in a local secure storage unit on the vehicle, and the encryption key is managed by the vehicle TPM chip.

3. The real-time monitoring method for the risk of emotional data abuse according to claim 2, characterized in that: The key point coordinates include the left and right corners of the mouth for calculating the curvature of the mouth, and the left and right inner and outer corners of the eyes for calculating the degree of eyelid opening and closing; The step S1.1 includes generating a circular mask area with an adjustable radius centered on each key point, encrypting pixels outside the mask area, dividing the pixels in the encrypted area into 16*16 pixel blocks, and independently performing AES-256 encryption on each area.

4. The real-time monitoring method for the risk of emotional data abuse according to claim 2, characterized in that: The voiceprint feature vector is encrypted using the SM2 national encryption algorithm, and the encrypted voiceprint data is stored in an isolated security domain onboard the vehicle. The voiceprint encryption key is derived from the user's biometrics and generated using the PBKDF2 algorithm. The key lifecycle is bound to the vehicle user account and the key is automatically destroyed after the account is cancelled. The original voice stream is fragmented into disordered fragments, which are distributed to edge nodes and cloud servers. The fragment index table is encrypted using the SM4 national encryption algorithm and stored independently.

5. The real-time monitoring method for the risk of emotional data abuse according to claim 2, characterized in that: The step S1.3 includes the following steps: Step S1.3.1: Normalize the physiological signal to a numerical value, and then encrypt the normalized numerical vector using the CKKS public key to generate the ciphertext c_data; Step S1.3.2: Perform ciphertext sentiment calculation, including calculating the homomorphic mean to obtain the encrypted mean, calculating the homomorphic standard deviation based on the encrypted mean, then calculating the encrypted standard deviation based on the homomorphic standard deviation, and calculating the sentiment value based on the encrypted standard deviation and the encrypted mean. Step 1.3.3: Authorize the pre-deployed air conditioning control system module to call the CKKS private key through the HSM to decrypt and obtain the plaintext emotion value.

6. The real-time monitoring method for the risk of emotional data abuse according to claim 5, characterized in that: The homomorphic mean calculation in step S1.3.2 is to perform homomorphic addition on the c_data ciphertext and scalar multiplication on the plaintext to obtain the encrypted mean c_mean; The formula for calculating the homomorphic standard deviation is as follows: C_var=C_mean_square-(C_mean)2 Where C_var represents the homomorphic variance, C_mean_square represents the homomorphic calculation of the square mean, and the encrypted standard deviation c_std is calculated by Taylor expansion; The formula for calculating the sentiment value is as follows: Sentiment value = 0.6*c_mean+0.4*c_std.

7. The real-time monitoring method for the risk of emotional data abuse according to claim 1, characterized in that: During the abnormal access monitoring process, all triggered abnormal events are automatically recorded to form a security log; The abnormal access includes unauthorized IP access, high-frequency requests, illegal protocols and low-frequency crawling.

8. The real-time monitoring method for the risk of emotional data abuse according to claim 1, characterized in that: The active defense and attack countermeasures include the following steps: Step S2.1: Inject false emotion data containing a tracking watermark into the data stream. When the attacker triggers the watermark, a warning is triggered and the attacker's access trajectory is recorded. Step S2.2: The data transmission link is superimposed with random noise packets. The noise packets are consistent with the length and transmission frequency of the real data packets, thereby interfering with the attacker's data parsing and cracking process. Step S2.3: After detecting the attack, the data encryption key and watermark ID are automatically updated, and other data are checked for leakage, and the affected nodes are automatically isolated.

9. The real-time monitoring method for the risk of emotional data abuse according to claim 8, characterized in that: The watermark ID includes IP address, device fingerprint, and timestamp; The false emotional data includes virtual physiological signals and virtual face / voice information; the virtual physiological signals are false data that conform to normal distribution but have abnormal parameters; the virtual face / voice information is a virtual face or voice clip generated by GAN.

10. A real-time monitoring system for the risk of emotional data abuse, characterized in that: include: Module M1: collects passenger emotion data in real time through the onboard terminal and anonymizes it in the edge computing device; Module M2: Monitors abnormal access in real time based on the results of anonymization processing. When an abnormal access occurs, it determines whether the current abnormal access is a false positive. If not, it dynamically blocks it, that is, performs active defense and attack countermeasures. If so, it triggers module M3. Module M3: Extracts normal user operations that are judged to be abnormal from the security logs generated during the monitoring process, captures the tracking watermark records triggered by attackers as dynamic optimization data, automatically labels attack behaviors and false positive types, performs secondary confirmation on fuzzy samples during manual review, and optimizes training through incremental learning.

Images