Incremental cloud learning method and system for vehicle-mounted emotion data

Abstract

The invention provides an incremental cloud learning method and system for vehicle-mounted emotion data, and the method comprises the steps: 1, collecting the multi-modal emotion data of a driver in real time through a vehicle-mounted sensor, and enabling the multi-modal emotion data to comprise a facial image, a voice signal and a physiological signal; 2, preprocessing the multi-modal data, including noise removal, feature extraction and lightweight edge model analysis, and then encrypting the multi-modal data; 3, uploading the encrypted data to a cloud end, and executing incremental learning and federated learning to update the emotion recognition model; 4, verifying the performance of the model based on a preset evaluation strategy, and issuing the updated model to the vehicle-mounted terminal through a privacy protection mechanism; and 5, implementing desensitization, encryption and compliance auditing in the full life cycle of the data to ensure the compatibility of user privacy and laws and regulations. The technical problems that in a vehicle-mounted emotion recognition system, the model precision is low, continuous learning cannot be achieved, and user privacy is difficult to guarantee are solved.

Description

Technical Field

[0001] The present invention relates to the fields of smart cockpits and artificial intelligence, and in particular to a method and system for incremental cloud-based learning of in-vehicle emotion data. Background Art

[0002] Currently, many in-vehicle emotion recognition systems attempt to achieve real-time recognition of the driver's emotional state by deploying models in the vehicle. These systems typically have the following characteristics:

[0003] 1. Use static emotion recognition models; such as some local models based on CNN (convolutional neural network) or RNN (recurrent neural network) architectures, which can identify the driver's emotional state through image or voice input and provide prompts or interventions in conjunction with the ADAS system.

[0004] 2. Deployed on the vehicle or edge device, emphasizing real-time performance; most systems deploy emotion recognition models in the vehicle computer, pursuing the advantage of "local operation in the vehicle without relying on the cloud" to ensure low latency and stability.

[0005] 3. Support limited emotion type recognition and response; current mainstream systems usually only recognize limited categories such as "fatigue", "anger", "inattention", etc., and mainly respond based on rules, lacking personalized analysis.

[0006] These methods have the problems of fixed models, inability to continuously learn, lack of cloud collaboration and intelligent evolution capabilities, and weak privacy protection of emotional data.

[0007] Patent application document CN119047602A discloses a distributed federated learning method based on emotion perception, including: a central server constructs a global model of the emotion dynamic evolution graph; the central server initializes the global model of the emotion dynamic evolution graph to obtain a global model to be distributed; the central server distributes the global model to be distributed to vehicle nodes; the vehicle nodes construct local models based on the received global model to be distributed; the vehicle nodes identify the vehicle user's emotional state based on the local model to obtain an emotional state recognition result; and the vehicle nodes control the target vehicle using a pre-built dynamic emotion adaptation engine and the emotional state recognition result to adjust the interaction mode and service content of the current vehicle system. However, this patent does not completely solve the existing technical problems and does not meet the requirements of the present invention. Summary of the Invention

[0008] In view of the deficiencies in the prior art, the present invention aims to provide an incremental cloud-based learning method and system for in-vehicle emotion data.

[0009] The incremental cloud-based learning method for vehicle-mounted emotion data provided by the present invention includes:

[0010] Step 1: Collect the driver's multimodal emotional data in real time through on-board sensors, including facial images, voice signals, and physiological signals;

[0011] Step 2: Preprocessing the multimodal data, including noise removal, feature extraction, and lightweight edge model analysis, and then encrypting it;

[0012] Step 3: Upload the encrypted data to the cloud and perform incremental learning and federated learning to update the emotion recognition model;

[0013] Step 4: Verify model performance based on a pre-set evaluation strategy and deliver the updated model to the vehicle through a privacy-preserving mechanism.

[0014] Step 5: Implement desensitization, encryption, and compliance audits throughout the data lifecycle to ensure user privacy and regulatory compliance.

[0015] Preferably, the preprocessing in step 2 includes: removing camera background noise and microphone ambient noise, extracting muscle activity features from facial images, extracting speech rate and intonation features from voice, and extracting heart rate variability and skin conductance change features from physiological signals;

[0016] The lightweight edge model analysis in step 2 adopts a multimodal feature fusion lightweight network, and its emotion vector output formula is:

[0017]

[0018] Among them, f, s, p are facial, voice, and physiological, x f 、x s 、x p Respectively represent facial image input, voice signal input, and physiological signal input; W m represents the weight of mode m; represents the feature extraction function; x m is the input data; Softmax is the normalized exponential function; MobileNetV3, LSTM, and FCN are feature extraction functions for facial images, speech signals, and physiological signals.

[0019] Preferably, the incremental learning in step 3 includes:

[0020] Buffer management: Define the sample capacity and the amount of historical data. The sampling probability formula is:

[0021]

[0022] Among them, Loss(x) is the loss value of sample x, Entropy(x) is the sample information entropy, t current , t storedThey are the current time and the sample storage time respectively;

[0023] Hybrid training process: The parameter update formula is:

[0024]

[0025] Among them, θ t is the parameter of the tth iteration; η is the learning rate, which controls the new data The step size of gradient update; λ is the weight coefficient of historical data, which adjusts the buffer data Impact on parameter updates; is the objective function gradient calculated based on the new data; is the objective function value calculated based on the buffer's historical data;

[0026] Catastrophic forgetting suppression: using elastic weights to consolidate loss functions:

[0027]

[0028] Among them, F i is the diagonal term of the Fisher information matrix; θ i * is the historically optimal parameter.

[0029] Preferably, the federated learning in step 3 includes:

[0030] The vehicle calculates the gradient locally: where g i (t) is the local gradient calculated for the i-th vehicle at the t-th iteration; is the gradient operator, used to calculate the update direction of the model parameters θ; is the parameter θ (t) Next, based on vehicle local data The calculated loss value;

[0031] Perform differential privacy noise processing on the gradient;

[0032] The cloud aggregates global gradients and updates model parameters;

[0033] The updated global model is synchronized to the vehicle end at preset intervals or when there is sufficient bandwidth.

[0034] Preferably, the model update in step 4 includes: generating an incremental model and deploying a shadow model, verifying the model performance using an independent test set, marking it as a formal version and pushing it if the verification passes, otherwise rolling back to the previous version;

[0035] The updated models released include:

[0036] Push decision value = ∑wi I pass (i)≥0.85

[0037] Among them, w i is the weight coefficient; I pass To pass the indicator function, the push is triggered when the following conditions are met: the model accuracy is improved to the preset threshold, the resource consumption meets the vehicle hardware specifications, the ISO 21434 security verification is passed, and the user's historical rejection rate is within the preset range.

[0038] The incremental cloud-based learning system for vehicle-mounted emotion data provided by the present invention includes:

[0039] Module M1: collects the driver's multimodal emotional data in real time through on-board sensors, including facial images, voice signals, and physiological signals;

[0040] Module M2: preprocessing the multimodal data, including noise removal, feature extraction, and lightweight edge model analysis, and then encrypting it;

[0041] Module M3: uploads the encrypted data to the cloud and performs incremental learning and federated learning to update the emotion recognition model;

[0042] Module M4: Verifies model performance based on a preset evaluation strategy and sends the updated model to the vehicle through a privacy protection mechanism;

[0043] Module M5: Implement desensitization, encryption, and compliance audits throughout the data lifecycle to ensure user privacy and regulatory compatibility.

[0044] Preferably, the preprocessing in the module M2 includes: removing camera background noise and microphone ambient noise, extracting muscle activity features from facial images, extracting speech rate and intonation features from voice, and extracting heart rate variability and skin conductance change features from physiological signals;

[0045] The lightweight edge model analysis in the module M2 adopts a multimodal feature fusion lightweight network, and its emotion vector output formula is:

[0046]

[0047] Among them, f, s, p are facial, voice, and physiological, x f 、x s 、x p Respectively represent facial image input, voice signal input, and physiological signal input; W m represents the weight of mode m; represents the feature extraction function; x mis the input data; Softmax is the normalized exponential function; MobileNetV3, LSTM, and FCN are feature extraction functions for facial images, speech signals, and physiological signals.

[0048] Preferably, the incremental learning in the module M3 includes:

[0049] Buffer management: Define the sample capacity and the amount of historical data. The sampling probability formula is:

[0050]

[0051] Among them, Loss(x) is the loss value of sample x, Entropy(x) is the sample information entropy, t current , t stored They are the current time and the sample storage time respectively;

[0052] Hybrid training process: The parameter update formula is:

[0053]

[0054] Among them, θ t is the parameter of the tth iteration; η is the learning rate, which controls the new data The step size of gradient update; λ is the weight coefficient of historical data, which adjusts the buffer data Impact on parameter updates; is the objective function gradient calculated based on the new data; is the objective function value calculated based on the buffer's historical data;

[0055] Catastrophic forgetting suppression: using elastic weights to consolidate loss functions:

[0056]

[0057] Among them, F i is the diagonal term of the Fisher information matrix; θ i * is the historically optimal parameter.

[0058] Preferably, the federated learning in the module M3 includes:

[0059] The vehicle calculates the gradient locally: where g i (t) is the local gradient calculated for the i-th vehicle at the t-th iteration; is the gradient operator, used to calculate the update direction of the model parameters θ; is the parameter θ (t) Next, based on vehicle local data The calculated loss value;

[0060] Perform differential privacy noise processing on the gradient;

[0061] The cloud aggregates global gradients and updates model parameters;

[0062] The updated global model is synchronized to the vehicle end at preset intervals or when there is sufficient bandwidth.

[0063] Preferably, the model update in the module M4 includes: generating an incremental model and deploying a shadow model, verifying the model performance using an independent test set, marking it as a formal version and pushing it if the verification passes, otherwise rolling back to the previous version;

[0064] The updated models released include:

[0065] Push decision value = ∑w i I pass (i)≥0.85

[0066] Among them, w i is the weight coefficient; I pass To pass the indicator function, the push is triggered when the following conditions are met: the model accuracy is improved to the preset threshold, the resource consumption meets the vehicle hardware specifications, the ISO 21434 security verification is passed, and the user's historical rejection rate is within the preset range.

[0067] Compared with the prior art, the present invention has the following beneficial effects:

[0068] (1) This invention uses a cloud-based incremental learning strategy (buffer management, hybrid training, and EWC regularization) to enable the model to continuously absorb new data and retain historical knowledge, thus solving the problem of catastrophic forgetting. Federated learning supports multi-vehicle collaborative training (data does not need to be shipped out of the vehicle), utilizing distributed data to enhance model generalization capabilities while avoiding data silos.

[0069] (2) Ensure that sensitive information is not leaked through data desensitization (such as facial identity information removal and voice obfuscation) and full-process encryption (symmetric / homomorphic encryption); user-controllable reporting mechanism (switch permissions) and compliance audit logs meet the requirements of regulations such as GDPR and ISO 21434.

[0070] (3) The vehicle-mounted end uses a lightweight multimodal fusion model (MMF-LiteNet), combined with MobileNetV3, LSTM, and FCN to achieve low-latency analysis; the modular differential update strategy (shadow model verification, hot update) reduces vehicle resource usage and ensures stable and efficient model distribution.

[0071] (4) Integrate multi-dimensional data such as facial expressions, voice intonation, and physiological signals (heart rate, skin conductance) to improve the accuracy of emotion recognition; output fine-grained emotion probability vectors (such as anger, calmness, fatigue, excitement) to support personalized driving behavior intervention (such as fatigue reminders and music recommendations). BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments with reference to the following drawings:

[0073] Figure 1 Flowchart for incremental cloud-based learning of in-vehicle emotion data;

[0074] Figure 2 Implementing a process flow chart for a sentiment analysis model. DETAILED DESCRIPTION

[0075] The present invention will be described in detail below with reference to specific embodiments. The following examples will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those skilled in the art, several changes and improvements can be made without departing from the scope of the present invention. These all fall within the scope of protection of the present invention.

[0076] Example 1

[0077] like Figure 1 The present invention provides an incremental cloud-based learning system for vehicle-mounted emotion data, the implementation process of which includes:

[0078] Step 1: In-vehicle emotion monitoring module. This module is responsible for collecting the driver's emotion data and performing pre-processing and preliminary analysis.

[0079] Step 2: Cloud computing module: Receives data from the vehicle emotion monitoring module, performs in-depth analysis and model updates, and pushes the updated model to the vehicle device.

[0080] Step 3: Data privacy protection module: Protect the driver's privacy and ensure the security and confidentiality of data during collection, transmission, and storage.

[0081] The step 1 comprises the following steps:

[0082] Step 1.1: Deploy data collection equipment to collect driver emotional data. This includes using cameras to capture the driver's facial expressions, microphones to capture the driver's voice and intonation, and physiological sensors to monitor the driver's physiological signals.

[0083] The specific implementation steps of step 1.1 include using a high-resolution, low-latency camera to capture the driver's micro-expression changes in real time, using a noise-canceling microphone to ensure the clarity of the voice signal, and using physiological sensors to collect the driver's heart rate and skin conductance signals.

[0084] Step 1.2: Data preprocessing.

[0085] The specific implementation steps of step 1.2 include data cleaning and feature extraction. This involves removing noise and invalid data, such as background noise from cameras and ambient noise from microphones. Useful emotional features are then extracted from the raw data. For example, facial muscle activity features can be extracted from facial expressions; features such as intonation and speech rate can be extracted from speech; and features such as heart rate variability and skin conductance changes can be extracted from physiological signals.

[0086] Step 1.3: Preliminary data analysis.

[0087] The specific implementation steps of step 1.3 include deploying a lightweight sentiment analysis model at the edge of the vehicle computer to preliminarily judge the driver's emotional state.

[0088] The sentiment analysis model uses a multimodal feature fusion lightweight network (MMF-LiteNet), and the output sentiment vector expression is:

[0089]

[0090] Among them, x f 、x s 、x p Respectively represent facial image input, speech signal input, physiological signal input, W m represents the modal weight, Represents the feature extraction function, and Softmax is a normalized exponential function. The input of the model is facial image, speech signal, and physiological signal, and the output is: output vector = [P 愤怒 ,P 平静 ,P 疲劳 ,P 兴奋 ](∑P i =1). The specific process is as follows Figure 2 shown.

[0091] The step 2 comprises the following steps:

[0092] Step 2.1: Upload emotion data.

[0093] The specific implementation steps of step 2.1 include receiving the uploaded encrypted emotion data, decoding and storing it in the database; establishing a user distributed data warehouse, and classifying and deduplicating emotion samples.

[0094] Step 2.2: Model incremental learning.

[0095] The specific implementation steps of step 2.2 include building a deep neural network to train emotional features; using incremental learning strategies (such as online fine-tuning and replay buffer) to retain historical knowledge; and building a federated learning architecture to collaboratively train multiple vehicle models without leaving the vehicle.

[0096] The incremental learning strategy includes three key steps: buffer management, hybrid training process, and catastrophic forgetting suppression;

[0097] The capacity of the buffer management definition is the latest 2000 samples + 500 samples of historical key frames; the sampling strategy used is:

[0098]

[0099] The hybrid training process is: λ = 0.3 is the historical data weight coefficient, η = 0.001 is the learning rate;

[0100] Catastrophic forgetting suppression uses elastic weight consolidation, Among them F i represents the diagonal term of the information matrix, θ i * represents the historical optimal parameters.

[0101] The specific process of collaborative training is as follows: 1. Each vehicle calculates the gradient based on local data: 2. Differential privacy processing; 3. Gradient aggregation; 4. Global update; 5. Model distribution; Synchronize the latest global model every 24 hours or when bandwidth is sufficient.

[0102] Step 2.3: Model evaluation and distribution.

[0103] The specific implementation steps of step 2.3 include online evaluation after the model is updated, such as dividing independent test sets for sample comparison; deciding whether to push based on preset strategies; and using the OTA mechanism or socket communication to push model weights and structural configurations to designated vehicles.

[0104] The specific process of model update includes: 1. Update verification process, which consists of incremental model generation, shadow model deployment, and independent test set evaluation to verify whether the performance meets the standards. If it meets the standards, it is marked as the official version and pushed to the vehicle. If it does not meet the standards, it is rolled back to the previous version; 2. Core verification indicators, including performance comparison testing, resource consumption detection, and catastrophic forgetting detection; 3. Online hot update technology, using a modular differential update strategy.

[0105] Default strategy:

[0106] Decision-making dimensions Pass conditions Weight Performance improvements ΔAcc≥1.5%ΔAcc≥1.5% 40％ Resource constraints Meet vehicle hardware specifications 30％ Safety and Compliance ISO 21434 certified 20％ User acceptance Historical rejection rate < 15% 10％

[0107] Push decision value = ∑w i I pass (i)≥0.85, where I pass Pass the indicator function for each dimension. If the push decision value is greater than or equal to 0.85, push is performed.

[0108] The step 3 comprises the following steps:

[0109] Step 3.1: Data desensitization and encryption.

[0110] The specific implementation steps of step 3.1 include: desensitizing sensitive information on the vehicle side, such as removing identity information from facial images and blurring voice content; using symmetric encryption or homomorphic encryption mechanisms throughout the entire process to ensure that data cannot be tampered with or leaked during transmission.

[0111] Step 3.2: Local controllable data reporting.

[0112] The specific implementation steps of step 3.2 include: providing user switch permissions so that the driver can choose whether to participate in model co-construction; if the user turns off data sharing, the system can still use the local model but stops updating and uploading.

[0113] Step 3.3: Compliance and audit mechanisms.

[0114] The specific implementation steps of step 3.3 include recording the time, data type, and processing process of each data upload and model update; being compatible with regulations such as GDPR and the Cybersecurity Law, and generating auditable log records.

[0115] Example 2

[0116] The present invention also provides an incremental cloud-based learning system for in-vehicle emotional data, including: module M1: real-time collection of the driver's multimodal emotional data, including facial images, voice signals and physiological signals, through on-board sensors; module M2: pre-processing the multimodal data, including noise removal, feature extraction and lightweight edge model analysis, and then encryption; module M3: uploading the encrypted data to the cloud, performing incremental learning and federated learning to update the emotion recognition model; module M4: verifying the model performance based on a preset evaluation strategy, and sending the updated model to the in-vehicle end through a privacy protection mechanism; module M5: implementing desensitization, encryption and compliance auditing throughout the data life cycle to ensure user privacy and regulatory compatibility.

[0117] The preprocessing in module M2 includes: removing camera background noise and microphone ambient noise, extracting muscle activity features from facial images, extracting speech rate and intonation features from voice, and extracting heart rate variability and skin conductance change features from physiological signals;

[0118] The lightweight edge model analysis in the module M2 adopts a multimodal feature fusion lightweight network, and its emotion vector output formula is:

[0119]

[0120] Among them, f, s, p are facial, voice, and physiological, x f 、x s 、x p Respectively represent facial image input, voice signal input, and physiological signal input; W m represents the weight of mode m; represents the feature extraction function; x m is the input data; Softmax is the normalized exponential function; MobileNetV3, LSTM, and FCN are feature extraction functions for facial images, speech signals, and physiological signals.

[0121] The incremental learning in module M3 includes:

[0122] Buffer management: Define the sample capacity and the amount of historical data. The sampling probability formula is:

[0123]

[0124] Among them, Loss(x) is the loss value of sample x, Entropy(x) is the sample information entropy, t current , t stored They are the current time and the sample storage time respectively;

[0125] Hybrid training process: The parameter update formula is:

[0126]

[0127] Among them, θ t is the parameter of the tth iteration; η is the learning rate, which controls the new data The step size of gradient update; λ is the weight coefficient of historical data, which adjusts the buffer data Impact on parameter updates; is the objective function gradient calculated based on the new data; is the objective function value calculated based on the buffer's historical data;

[0128] Catastrophic forgetting suppression: using elastic weights to consolidate loss functions:

[0129]

[0130] Among them, F i is the diagonal term of the Fisher information matrix; θ i * is the historically optimal parameter.

[0131] The federated learning in module M3 includes:

[0132] The vehicle calculates the gradient locally: where g i (t) is the local gradient calculated for the i-th vehicle at the t-th iteration; is the gradient operator, used to calculate the update direction of the model parameters θ; is the parameter θ (t) Next, based on vehicle local data The calculated loss value;

[0133] Perform differential privacy noise processing on the gradient;

[0134] The cloud aggregates global gradients and updates model parameters;

[0135] The updated global model is synchronized to the vehicle end at preset intervals or when there is sufficient bandwidth.

[0136] The model update in module M4 includes: generating an incremental model and deploying a shadow model, verifying the model performance using an independent test set, marking it as a formal version and pushing it if the verification passes, otherwise rolling back to the previous version;

[0137] The updated models released include:

[0138] Push decision value = ∑w i I pass (i)≥0.85

[0139] Among them, w i is the weight coefficient; I pass To pass the indicator function, the push is triggered when the following conditions are met: the model accuracy is improved to the preset threshold, the resource consumption meets the vehicle hardware specifications, the ISO 21434 security verification is passed, and the user's historical rejection rate is within the preset range.

[0140] Those skilled in the art will appreciate that, in addition to implementing the system, device, and various modules provided by the present invention in purely computer-readable program code, it is entirely possible to implement the same program in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, embedded microcontrollers, and the like by logically programming the method steps. Therefore, the system, device, and various modules provided by the present invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; the modules for implementing various functions can also be considered both software programs for implementing the method and structures within the hardware component.

[0141] The above describes specific embodiments of the present invention. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art may make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. The embodiments of this application and the features in the embodiments may be combined with each other in any manner unless there is a conflict.

Claims

1. An incremental cloud-based learning method for vehicle-mounted emotion data, characterized in that: include: Step 1: Collect the driver's multimodal emotional data in real time through on-board sensors, including facial images, voice signals, and physiological signals; Step 2: Preprocessing the multimodal data, including noise removal, feature extraction, and lightweight edge model analysis, and then encrypting it; Step 3: Upload the encrypted data to the cloud and perform incremental learning and federated learning to update the emotion recognition model; Step 4: Verify model performance based on a pre-set evaluation strategy and deliver the updated model to the vehicle through a privacy-preserving mechanism. Step 5: Implement desensitization, encryption, and compliance audits throughout the data lifecycle to ensure user privacy and regulatory compliance.

2. The incremental cloud-based learning method for vehicle-mounted emotion data according to claim 1, characterized in that: The preprocessing in step 2 includes: removing camera background noise and microphone ambient noise, extracting muscle activity features from facial images, extracting speech rate and intonation features from voice, and extracting heart rate variability and skin conductance change features from physiological signals; The lightweight edge model analysis in step 2 adopts a multimodal feature fusion lightweight network, and its emotion vector output formula is: Among them, f, s, p are facial, voice, and physiological, x f 、x s 、x p Respectively represent facial image input, voice signal input, and physiological signal input; W m represents the weight of mode m; represents the feature extraction function; x m is the input data; Softmax is the normalized exponential function; MobileNetV3, LSTM, and FCN are feature extraction functions for facial images, speech signals, and physiological signals.

3. The incremental cloud-based learning method for vehicle-mounted emotion data according to claim 1, characterized in that: The incremental learning in step 3 includes: Buffer management: Define the sample capacity and the amount of historical data. The sampling probability formula is: Among them, Loss(x) is the loss value of sample x, Entropy(x) is the sample information entropy, t current , t stired They are the current time and the sample storage time respectively; Hybrid training process: The parameter update formula is: Among them, θ t is the parameter of the tth iteration; η is the learning rate, which controls the new data The step size of gradient update; λ is the weight coefficient of historical data, which adjusts the buffer data Impact on parameter updates; is the objective function gradient calculated based on the new data; is the objective function value calculated based on the buffer's historical data; Catastrophic forgetting suppression: using elastic weights to consolidate loss functions: Among them, F i is the diagonal term of the Fisher information matrix; θ i * is the historically optimal parameter.

4. The incremental cloud-based learning method for vehicle-mounted emotion data according to claim 1, characterized in that: The federated learning in step 3 includes: The vehicle calculates the gradient locally: where g i (t) is the local gradient calculated for the i-th vehicle at the t-th iteration; is the gradient operator, used to calculate the update direction of the model parameters θ; is the parameter θ (t) Next, based on vehicle local data The calculated loss value; Perform differential privacy noise processing on the gradient; The cloud aggregates global gradients and updates model parameters; The updated global model is synchronized to the vehicle end at preset intervals or when there is sufficient bandwidth.

5. The incremental cloud-based learning method for vehicle-mounted emotion data according to claim 1, characterized in that: The model update in step 4 includes: generating an incremental model and deploying a shadow model, verifying the model performance using an independent test set, marking it as a formal version and pushing it if the verification passes, otherwise rolling back to the previous version; The updated models released include: Push decision value = ∑w i I pass (i)≥0.85 Among them, w i is the weight coefficient; I pass To pass the indicator function, the push is triggered when the following conditions are met: the model accuracy is improved to the preset threshold, the resource consumption meets the vehicle hardware specifications, the ISO 21434 security verification is passed, and the user's historical rejection rate is within the preset range.

6. An incremental cloud-based learning system for vehicle-mounted emotion data, characterized in that: include: Module M1: collects the driver's multimodal emotional data in real time through on-board sensors, including facial images, voice signals, and physiological signals; Module M2: preprocessing the multimodal data, including noise removal, feature extraction, and lightweight edge model analysis, and then encrypting it; Module M3: uploads the encrypted data to the cloud and performs incremental learning and federated learning to update the emotion recognition model; Module M4: Verifies model performance based on a preset evaluation strategy and sends the updated model to the vehicle through a privacy protection mechanism; Module M5: Implement desensitization, encryption, and compliance audits throughout the data lifecycle to ensure user privacy and regulatory compatibility.

7. The incremental cloud-based learning system for vehicle-mounted emotion data according to claim 6, characterized in that: The preprocessing in module M2 includes: removing camera background noise and microphone ambient noise, extracting muscle activity features from facial images, extracting speech rate and intonation features from voice, and extracting heart rate variability and skin conductance change features from physiological signals; The lightweight edge model analysis in the module M2 adopts a multimodal feature fusion lightweight network, and its emotion vector output formula is: Among them, f, s, p are facial, voice, and physiological, x f 、x s 、x p Respectively represent facial image input, voice signal input, and physiological signal input; W m represents the weight of mode m; represents the feature extraction function; x m is the input data; Softmax is the normalized exponential function; MobileNetV3, LSTM, and FCN are feature extraction functions for facial images, speech signals, and physiological signals.

8. The incremental cloud-based learning system for vehicle-mounted emotion data according to claim 6, characterized in that: The incremental learning in module M3 includes: Buffer management: Define the sample capacity and the amount of historical data. The sampling probability formula is: Among them, Loss(x) is the loss value of sample x, Entropy(x) is the sample information entropy, t cyrrent , t stored They are the current time and the sample storage time respectively; Hybrid training process: The parameter update formula is: Among them, θ t is the parameter of the tth iteration; η is the learning rate, which controls the new data The step size of gradient update; λ is the weight coefficient of historical data, which adjusts the buffer data Impact on parameter updates; is the objective function gradient calculated based on the new data; is the objective function value calculated based on the buffer's historical data; Catastrophic forgetting suppression: using elastic weights to consolidate loss functions: Among them, F i is the diagonal term of the Fisher information matrix; θ i * is the historically optimal parameter.

9. The incremental cloud-based learning system for vehicle-mounted emotion data according to claim 6, characterized in that: The federated learning in module M3 includes: The vehicle calculates the gradient locally: where g i (t) is the local gradient calculated for the i-th vehicle at the t-th iteration; is the gradient operator, used to calculate the update direction of the model parameters θ; is the parameter θ (t) Next, based on vehicle local data The calculated loss value; Perform differential privacy noise processing on the gradient; The cloud aggregates global gradients and updates model parameters; The updated global model is synchronized to the vehicle end at preset intervals or when there is sufficient bandwidth.

10. The incremental cloud-based learning system for vehicle-mounted emotion data according to claim 6, characterized in that: The model update in module M4 includes: generating an incremental model and deploying a shadow model, verifying the model performance using an independent test set, marking it as a formal version and pushing it if the verification passes, otherwise rolling back to the previous version; The updated models released include: Push decision value = ∑w i I pass (i)≥0.85 Among them, w i is the weight coefficient; I pass To pass the indicator function, the push is triggered when the following conditions are met: the model accuracy is improved to the preset threshold, the resource consumption meets the vehicle hardware specifications, the ISO 21434 security verification is passed, and the user's historical rejection rate is within the preset range.

Images