A Distributed Federated Learning Method Based on Emotion Perception

Abstract

The present application provides a distributed federated learning method based on emotion perception, including: a central server constructs a global model of an emotion dynamic evolution graph; the central server initializes the global model of the emotion dynamic evolution graph to obtain a global model to be distributed; the central server distributes the global model to be distributed to vehicle nodes; the vehicle nodes construct local models according to the received global model to be distributed; the vehicle nodes perform emotion state recognition on vehicle users based on the local models to obtain emotion state recognition results; the vehicle nodes control the target vehicle to adjust the interaction mode and service content of the current vehicle system through a pre-constructed dynamic emotion adaptive engine and the emotion state recognition results. This method can adapt to the current user emotion and driving environment by dynamically adjusting the interaction mode and service content according to the emotion state, improving the user experience and driving safety.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and more particularly, to a distributed federated learning method based on emotion perception. Background Art

[0002] In the fields of new energy and artificial intelligence, the rapid development of intelligent vehicle systems has promoted the driving experience towards personalization and emotionalization. Modern drivers not only require cars to provide safe and efficient mobility functions, but also expect to obtain a more user-friendly and emotionally resonant interaction experience. With the continuous progress of new energy and autonomous driving technologies, vehicles are gradually transforming into intelligent partners that can understand human emotions and needs. However, although existing technologies can enhance the driving experience through augmented reality and multi-functional human-machine interaction systems (such as camera recognition, microphone voice, etc.), existing systems cannot effectively identify the emotional states of drivers or passengers and provide adaptive feedback accordingly, lacking the ability to dynamically adjust the interaction method and service content according to the user's emotions and driving environment, thus reducing the user experience and driving safety. Summary of the Invention

[0003] The purpose of the embodiments of this application is to provide a distributed federated learning method based on emotion perception, which can dynamically adjust the interaction method and service content according to the emotional state, so as to adapt to the current user's emotions and driving environment, and improve the user experience and driving safety.

[0004] In the first aspect of this application, a distributed federated learning method based on emotion perception is provided. The distributed federated learning method is applied to a distributed federated learning system based on emotion perception. The distributed federated learning system includes a central server and at least one vehicle node, where,

[0005] The central server constructs a global model of the emotion dynamic evolution graph;

[0006] The central server initializes the global model of the emotion dynamic evolution graph to obtain a global model to be distributed;

[0007] The central server distributes the global model to be distributed to the vehicle nodes;

[0008] The vehicle node constructs a local model according to the received global model to be distributed;

[0009] The vehicle node performs emotion state recognition on the vehicle user based on the local model to obtain an emotion state recognition result;

[0010] The vehicle node controls the target vehicle to adjust the interaction method and service content of the current vehicle system through a pre-constructed dynamic emotion adaptive engine and the emotion state recognition result.

[0011] In the above implementation process, the method can dynamically adjust the interaction method and service content according to the emotional state, so as to adapt to the current user's emotion and driving environment, improving the user experience and driving safety.

[0012] Furthermore, each node in the global model of the emotional dynamic evolution graph represents a specific emotional state, and each node includes an emotional intensity feature vector, an emotional duration feature vector, and an environmental feature vector;

[0013] The edge between two nodes in the global model of the emotional dynamic evolution graph represents the transition probability between the corresponding emotional states of these two nodes;

[0014] The transition probability between the emotional states is calculated by a probability model of emotional state transition; among them, the probability model of emotional state transition is constructed based on the Riemann metric and the geodesic equation;

[0015] The global model of the emotional dynamic evolution graph also includes a node feature update formula constructed based on the Riemann metric.

[0016] Furthermore, the vehicle node constructs a local model according to the received global model to be distributed, including:

[0017] The vehicle node obtains its own vehicle data and its own emotional perception requirements;

[0018] The vehicle node fine-tunes the global model to be distributed according to the vehicle's own data and its own emotional perception requirements to obtain a local model.

[0019] Furthermore, the method further includes:

[0020] The vehicle node obtains a pre-defined reinforcement learning environment, a reinforcement learning algorithm, and a reward function;

[0021] The vehicle node constructs a dynamic emotion adaptive engine according to the reinforcement learning environment, the reinforcement learning algorithm, and the reward function.

[0022] Furthermore, the reinforcement learning environment at least includes the vehicle user's emotional state, the driving environment, and the interaction method currently provided by the system;

[0023] The reinforcement learning algorithm is a Q-learning algorithm or a deep DQN network.

[0024] Furthermore, the method further includes:

[0025] The vehicle node samples multiple emotion recognition learning tasks from the distributed environment based on the meta-learning algorithm; wherein, the emotion recognition learning task includes a training set and a test set;

[0026] The vehicle node calculates the meta-learning loss and the meta-learning gradient based on the meta-learning algorithm, the emotion recognition learning task, and the local model;

[0027] The vehicle node feeds back the meta-learning loss and the meta-learning gradient to the central server;

[0028] The central server updates the outer-loop parameters of the to-be-distributed global model according to the meta-learning losses and meta-learning gradients fed back by all vehicle nodes in the distributed federated learning method, and obtains the updated to-be-distributed global model; and executes the step of distributing the to-be-distributed global model to the vehicle nodes.

[0029] Further, the vehicle node calculates the meta-learning loss and the meta-learning gradient based on the meta-learning algorithm, the emotion recognition learning task, and the local model, including:

[0030] The vehicle node updates the inner-loop parameters of the local model according to the training set, and obtains the updated local model;

[0031] The vehicle node calculates the meta-learning loss and the meta-learning gradient of the local model according to the test set.

[0032] Further, the vehicle node updates the inner-loop parameters of the local model according to the training set, and obtains the updated local model, including:

[0033] The vehicle node updates the model parameters of the local model according to the training set and the loss function on the minimization task, and obtains the updated local model.

[0034] Further, the vehicle node calculates the meta-learning loss and the meta-learning gradient of the local model according to the test set, including:

[0035] The vehicle node performs model evaluation calculation on the local model according to the test set and the differential privacy algorithm, and obtains the meta-learning loss and the meta-learning gradient.

[0036] Further, the vehicle node controls the target vehicle to adjust the interaction mode and service content of the current vehicle system through a pre-constructed dynamic emotion adaptive engine and the emotion state recognition result, including:

[0037] The vehicle node determines the optimal action strategy through a pre-constructed dynamic emotion adaptive engine and the emotion state recognition result; wherein, the optimal action strategy includes an interaction mode adjustment strategy and a service content adjustment strategy;

[0038] The vehicle node controls the target vehicle to adjust the interaction mode and service content of the current vehicle system according to the optimal action strategy.

[0039] The beneficial effects of this application are: it can better analyze the conversion dynamics between emotion states and the influence of situational factors by combining graph neural networks and Riemannian metrics;

[0040] It can introduce reinforcement learning, enabling the system to learn and adopt the optimal action strategy in a given state to maximize user satisfaction, safety, and interaction efficiency;

[0041] It can introduce differential privacy technology in the meta-learning process to form a new global model without exposing the sensitive information of any individual node, greatly improving the data security and user trust in the federated learning system;

[0042] It can improve the accuracy and flexibility of the emotion recognition model through the combination of federated learning and meta-learning, while enhancing the model's rapid adaptation ability when facing unknown emotion states, bringing significant improvements to personalized emotion recognition services. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of this application, the following will briefly introduce the drawings required to be used in the embodiments of this application. It should be understood that the following drawings only show some embodiments of this application, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0044] Figure 1 It is a schematic flowchart of a distributed federated learning method based on emotion perception provided by an embodiment of this application;

[0045] Figure 2 It is a schematic flowchart of another distributed federated learning method based on emotion perception provided by an embodiment of this application;

[0046] Figure 3 It is a schematic framework flowchart of a distributed federated learning method based on emotion perception provided by an embodiment of this application;

[0047] Figure 4 It is a schematic diagram of an intelligent vehicle interaction experience system applying a distributed federated learning method based on emotion perception provided by an embodiment of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0048] The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application.

[0049] It should be noted that similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of the present application, the terms "first", "second", etc. are only used for distinguishing descriptions and cannot be construed as indicating or implying relative importance.

[0050] Embodiment 1

[0051] Please refer to Figure 1 , Figure 1 , which is a schematic flow chart of a distributed federated learning method based on emotion perception provided in this embodiment. Among them, the distributed federated learning method based on emotion perception is applied to a distributed federated learning system based on emotion perception. The distributed federated learning system includes a central server and at least one vehicle node. The method includes:

[0052] S110. The central server constructs a global model of the emotion dynamic evolution graph.

[0053] S120. The central server initializes the global model of the emotion dynamic evolution graph to obtain a global model to be distributed.

[0054] S130. The central server distributes the global model to be distributed to the vehicle nodes.

[0055] S210. The vehicle node constructs a local model according to the received global model to be distributed.

[0056] S220. The vehicle node performs emotion state recognition on vehicle users based on the local model to obtain an emotion state recognition result.

[0057] In this embodiment, the vehicle user can be a driver or a passenger.

[0058] S230. The vehicle node controls the target vehicle to adjust the interaction mode and service content of the current vehicle system through a pre-constructed dynamic emotion adaptive engine and the emotion state recognition result.

[0059] In this embodiment, in order to enable the in-vehicle system to perceive emotions, an emotion perception adaptive algorithm (ESA) is proposed. This algorithm is actually composed of 3 frameworks:

[0060] (1) Emotion dynamic evolution graph (EDEG) model.

[0061] (2)Dynamic Adaptation Engine

[0062] (3)Federated Learning Enhanced Personalized Service (FL component)

[0063] Specifically, since emotions are not static but evolve continuously in different environments and situations. Based on this, traditional emotion recognition methods often ignore the transition dynamics between emotional states and the influence of situational factors, limiting the accuracy of emotion recognition and the depth of application.

[0064] To solve this problem, this method proposes an Emotion Dynamic Evolution Graph (EDEG) model and combines the concept of dynamic system theory, so as to use this model to capture the complex relationships between emotional states and their evolution laws over time through graph neural networks to achieve accurate prediction and response to the emotional changes of drivers.

[0065] In this embodiment, the execution subject of this method can be a computing device such as a computer or a server, and no limitation is made in this embodiment.

[0066] It can be seen that implementing the emotion-aware distributed federated learning method described in this embodiment can dynamically adjust the interaction method and service content according to the emotional state, adapt to the current user's emotion and driving environment, and improve the user experience and driving safety.

[0067] Embodiment 2

[0068] Please refer to Figure 2 , Figure 2 which is a schematic flowchart of an emotion-aware distributed federated learning method provided in this embodiment. Among them, this emotion-aware distributed federated learning method is applied to an emotion-aware distributed federated learning system. The distributed federated learning system includes a central server and at least one vehicle node. This method includes:

[0069] S110. The central server constructs a global emotion dynamic evolution graph model.

[0070] In this embodiment, each node in the global emotion dynamic evolution graph model represents a specific emotional state. Each node includes an emotional intensity feature vector, an emotional duration feature vector, and an environmental feature vector;

[0071] The edge between two nodes in the global emotion dynamic evolution graph model represents the transition probability between the two corresponding emotional states;

[0072] The transition probability between emotional states is calculated by a probability model of emotional state transition; among them, the probability model of emotional state transition is constructed based on Riemannian metric and geodesic equation;

[0073] The global model of the emotional dynamic evolution graph also includes a node feature update formula constructed based on the Riemannian metric.

[0074] In this embodiment, in the construction of the global model of the emotional dynamic evolution graph, it is mainly divided into two parts: point construction and edge construction:

[0075] (1) Point construction:

[0076] This method assigns each node v i to represent a specific emotional state. Among them, i is the index of the emotional state. The feature vector x contained in each node i can include the intensity and duration of emotions, etc., as well as environmental features such as time and location.

[0077] (2) Edge construction:

[0078] The edge e ij represents the probability of the emotional state transitioning from v i to v j . This can be obtained by analyzing historical emotional change data. For example, it is expressed using conditional probability as:

[0079] ;

[0080] where P(v i , v j ) is the probability of observing the emotional states v i and v j simultaneously, and P(v i ) is the probability of observing the emotional state v i .

[0081] In this embodiment, in order to be able to consider the complex relationships between emotional states when updating emotional state features, this method introduces the Riemannian metric. The specific method is as follows:

[0082] First, assume that the emotional manifold is a smooth manifold, and its local is represented by the feature vectors of the emotional states . Therefore, a Riemannian metric g can be defined to measure the distance on the manifold, expressed as:

[0083]

[0084] where is the tangent space on the manifold , representing the direction and speed of the change of the emotional state. The dynamic evolution of the emotional state can be represented by a vector field It is described that this vector field gives the tangent vector at each point on the manifold, that is, the direction of the change in the emotional state. The vector field can be given by the following differential equation:

[0085]

[0086] where, represents the rate of change of the emotional state over time. A connection ∇ is defined on the manifold, which provides a way to parallel transport vectors on the manifold. The connection enables an understanding of how the emotional state evolves over time and how this evolution process depends on the geometric structure of the emotional manifold.

[0087] After introducing the Riemannian metric, the update formula of the nodes is adjusted to reflect the geometric distance on the manifold. The update of the node features can include a mapping function that takes into account the geometric path length between the nodes, and the formula is adjusted to:

[0088]

[0089] where β ij is a weight calculated based on the Riemannian metric, taking into account the geometric distance on the emotional manifold between nodes i and j. This weight can be adjusted according to the geodesic distance between the nodes, thus ensuring that the geometric relationship between the emotional states is considered when information is propagated.

[0090] Calculating the geodesic distance between two points on the emotional manifold is the key to understanding the evolution of the emotional state. This can be achieved by solving the shortest path problem under the Riemannian metric, using the geodesic equation:

[0091]

[0092] where, are the Christoffel symbols, representing the connection on the manifold and used to describe the curvature of the manifold, represents the covariant derivative along the path on the manifold.

[0093] Using the Riemannian metric, this method can redefine the probability model of the emotional state transition. The transition probability not only depends on the relationship between the emotional states but is also affected by their geometric positions on the emotional manifold. Therefore, the transition probability can be dynamically adjusted by a function considering the geodesic distance between the nodes:

[0094]

[0095] where d g (v i , v j ) is the distance between nodes v i calculated based on the Riemannian metric gand v j The geodesic distance between them, where λ is an adjustment parameter. By this method, the system can consider the complex geometric relationships between emotional states when updating the emotional state features, thus providing a deeper understanding of the dynamic evolution of emotional states.

[0096] S120. The central server initializes the global model of the emotional dynamics evolution graph to obtain the global model to be distributed.

[0097] In this embodiment, steps S210 - S130 are the processes of performing parameter inner-loop update and parameter outer-loop update on the global model to be distributed in this application. Figure 2 The steps are represented by dotted lines and are optional.

[0098] In this embodiment, Figure 2 The dotted lines in the unilateral execution processes of the central server and vehicle nodes in refer to the next optional steps of the central server and vehicle nodes. The reason why they are called optional steps is that these steps need to wait for the meta-learning loss, meta-learning gradient, or the input of the model to be distributed to be implemented. Therefore, it is not advisable to execute directly and can only be executed conditionally as its optional steps.

[0099] S210. The vehicle node samples multiple emotion recognition learning tasks from the distributed environment based on the meta-learning algorithm; among them, the emotion recognition learning tasks include a training set and a test set.

[0100] S221. The vehicle node updates the model parameters of the local model according to the training set and the loss function on the minimization task to obtain the updated local model.

[0101] S222. The vehicle node performs model evaluation calculation on the local model according to the test set and the differential privacy algorithm to obtain the meta-learning loss and meta-learning gradient.

[0102] In this embodiment, introducing differential privacy in the meta-learning process can further protect the privacy of the data in each task. When calculating the meta-learning loss and its gradient, differential privacy is ensured by adding random noise:

[0103]

[0104]

[0105] where is the noise added according to the differential privacy parameters ϵ (privacy loss) and δ (failure probability).

[0106] The central server (which can be a cloud server) collects all the privacy-protected local updates and securely aggregates these updates to form a new version of the global model.

[0107]

[0108] Where N is the total number of nodes participating in the learning.

[0109] After the global model is updated, it is distributed to all nodes again. Each node can fine-tune the global model according to its own specific data and requirements to achieve better personalized adaptability.

[0110]

[0111] Where ξ is the learning rate for personalized fine-tuning.

[0112] By combining meta-learning and differential privacy, this framework not only improves the personalized adaptability and learning efficiency of the model in the federated learning environment, but also enhances data privacy protection. Meta-learning enables the global model to quickly adapt to new or rare emotional states with a small amount of data, while differential privacy ensures that the privacy of personal data is not leaked during this process. This framework provides a technical basis for realizing a truly private and efficient personalized emotion recognition service.

[0113] S230. The vehicle node feeds back the meta-learning loss and meta-learning gradient to the central server.

[0114] In this embodiment, the user data of a single vehicle cannot meet the data requirements for emotion recognition. Therefore, this method constructs an emotion recognition model across vehicles. At the same time, in order to protect the privacy of user data, this method introduces a federated learning framework to implement a distributed learning system, so as to improve the performance and personalized adaptability of the emotion recognition algorithm through cross-vehicle collaborative learning without sharing user personal data.

[0115] In this embodiment, the transmission of model parameters or gradient information is used instead of the transmission of raw data to avoid privacy leakage. Specifically, for model initialization and distribution, the central server (which can be a cloud server) first initializes the global model θ global . Then, this global model is distributed to all nodes (vehicles) participating in FL.

[0116]

[0117] Each node receives the initial version of the global model as their respective local models.

[0118] To enable each independent node to better adapt to new scenarios in distributed training, this method introduces the method of meta-learning within the framework of federated learning. Meta-learning focuses on how to train a model on multiple learning tasks so that it can quickly adapt using past experience when encountering new tasks. In the federated learning environment, meta-learning is used to optimize the learning strategy of the global model, enabling it to learn and adapt to new scenarios more effectively on distributed nodes.

[0119] S130. The central server updates the outer-loop parameters of the global model to be distributed according to the meta-learning loss and meta-learning gradients feedback by all vehicle nodes in the distributed federated learning method, obtaining the updated global model to be distributed.

[0120] For example, the update process of the meta-learning model can be as follows:

[0121] (1) Task sampling:

[0122] Meta-learning begins by sampling multiple emotion recognition learning tasks from the distributed environment. Each task is defined by a dataset and contains a training set and a test set.

[0123]

[0124] Among them, is the distribution of all possible tasks.

[0125] (2) Inner-loop update:

[0126] For each task , the model parameters θ are updated by minimizing the loss function on the task, namely the so-called inner-loop update:

[0127]

[0128] where α is the learning rate within the task.

[0129] (3) Meta-learning loss function:

[0130] The updated parameters θ i are evaluated on the test set of the task to calculate the meta-learning loss:

[0131]

[0132] (4) Outer-loop update:

[0133] Based on the meta-learning losses of all emotion recognition tasks, the global model parameters are updated, namely the so-called outer-loop update:

[0134]

[0135] Among them, β is the meta-learning rate and K is the number of tasks.

[0136] S140. The central server distributes the global model to be distributed to vehicle nodes.

[0137] S241. The vehicle node obtains its own vehicle data and its own emotion perception requirements.

[0138] S242. The vehicle node fine-tunes the global model to be distributed according to its own vehicle data and its own emotion perception requirements to obtain a local model.

[0139] S250. The vehicle node performs emotion state recognition on vehicle users based on the local model to obtain an emotion state recognition result.

[0140] S260. The vehicle node obtains a pre-defined reinforcement learning environment, a reinforcement learning algorithm, and a reward function.

[0141] In this embodiment, the reinforcement learning environment includes at least the emotion state of the vehicle user, the driving environment, and the interaction method currently provided by the system;

[0142] The reinforcement learning algorithm is a Q-learning algorithm or a deep DQN network.

[0143] In this embodiment, after obtaining the emotion prediction, in order to achieve better human-computer interaction, the method introduces a reinforcement learning method:

[0144] First, the method defines the reinforcement learning environment, which includes the emotion state, the driving environment, and the interaction method provided by the system. The environmental state s t at time t is represented as including:

[0145]

[0146] e t : The emotion state of the driver, provided by the emotion recognition model;

[0147] d t : The driving environment state, such as traffic conditions, weather, etc.;

[0148] i t : The current interaction method and service content of the system.

[0149] Then, the agent (i.e., the dynamic emotion adaptation engine) takes an action a t at each time step t, aiming to adjust the interaction method and service content of the system to adapt to the current emotion and driving environment.

[0150] The action space can be defined as a series of executable services and interaction strategies, such as:

[0151] A = {a1, a2, …, a n}

[0152] where each action a i represents a specific service adjustment or interaction strategy.

[0153] S270. The vehicle node constructs a dynamic emotion adaptive engine based on the reinforcement learning environment, the reinforcement learning algorithm, and the reward function.

[0154] In this embodiment, the goal of the agent is to learn a policy π(s t ), which selects an action a t given the state s t to maximize the future cumulative reward. The reward function R(s t , a t ) measures the immediate utility of the action a t in the state s t and can include multiple dimensions such as user satisfaction, safety, and interaction efficiency.

[0155]

[0156] where w1, w2, w3 are weight parameters used to balance the importance of different goals.

[0157] Finally, the method uses Q-learning or Deep Q-Network (DQN) as the learning algorithm to optimize the decision-making process. For DQN, the Q-function is defined as Q(s t , a t ; θ), which represents the expected return of taking the action a t in the state s t . DQN updates the network parameter θ by minimizing the following loss function:

[0158]

[0159] where γ is the discount factor, representing the current value of future rewards; represents the parameters of the target network, which are used to stabilize the learning process; D is the experience replay buffer that stores past transitions. Through the above method, the dynamic emotion adaptive engine can automatically learn and adjust the best interaction methods and service content based on real-time emotion feedback and the driving environment, thereby providing a deeply personalized and dynamically adaptable user experience.

[0160] S281. The vehicle node determines the optimal action strategy through a pre - constructed dynamic emotion adaptive engine and the emotion state recognition result; wherein, the optimal action strategy includes an interaction mode adjustment strategy and a service content adjustment strategy.

[0161] S282. The vehicle node controls the target vehicle to adjust the interaction mode and service content of the current vehicle system according to the optimal action strategy.

[0162] Please refer to Figure 3 , Figure 3 which shows a schematic framework flow diagram of a distributed federated learning method based on emotion perception.

[0163] Please refer to Figure 4 , Figure 4 which shows a schematic diagram of an intelligent vehicle interaction experience system applying a distributed federated learning method based on emotion perception.

[0164] In this embodiment, the innovation points of this method are as follows:

[0165] (1) Considering the dynamics of emotions in emotion recognition, introducing graph neural network prediction, and at the same time considering the complex relationships between emotion states when updating emotion state features, and increasing the accuracy of prediction by introducing Riemannian metric.

[0166] (2) By integrating the dynamic emotion adaptive engine with reinforcement learning methods, this integration not only utilizes the accurate emotion states provided by the emotion recognition model, but also enables the system to learn and adopt the optimal action strategy in a given state through the trial - and - error learning method of reinforcement learning, so as to maximize user satisfaction, safety, and interaction efficiency. The realization of this dynamic adjustment strategy brings unprecedented personalization and adaptability to the human - machine interaction system.

[0167] (3) Introducing the meta - learning method under the federated learning framework to form an advanced learning strategy to improve the adaptability of each independent node in distributed training to new scenarios. This combination not only allows the global model to utilize the experience obtained from multiple emotion recognition tasks to optimize its learning strategy, so as to quickly adapt when encountering new tasks, but also makes the learning process on distributed nodes more efficient and effective.

[0168] In this embodiment, the execution subject of this method can be a computing device such as a computer or a server, and no specific limitation is made in this embodiment.

[0169] It can be seen that implementing the distributed federated learning method based on emotion perception described in this embodiment can dynamically adjust the interaction mode and service content according to the emotion state, and can adapt to the current user emotion and driving environment, thus improving the user experience and driving safety.

[0170] In several embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of devices, methods, and computer program products according to multiple embodiments of this application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order from that marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0171] In addition, each functional module in various embodiments of this application can be integrated together to form an independent part, or each module can exist alone, or two or more modules can be integrated to form an independent part.

[0172] If the described function is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs, etc., which can store program codes.

[0173] The above are only embodiments of the present application and are not intended to limit the protection scope of the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the protection scope of the present application. It should be noted that similar reference numerals and letters indicate similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0174] As described above, this is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, and all should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0175] It should be noted that in this text, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or device including the said element.

Claims

1. A distributed federated learning method based on emotion perception, characterized in that The described distributed federated learning method is applied to a distributed federated learning system for emotion perception. The distributed federated learning system includes a central server and at least one vehicle node. Among them, the central server constructs a global model of the emotion dynamic evolution graph; the central server initializes the global model of the emotion dynamic evolution graph to obtain a global model to be distributed; the central server distributes the global model to be distributed to the vehicle nodes; the vehicle nodes construct local models according to the received global model to be distributed; the vehicle nodes perform emotion state recognition of vehicle users based on the local models to obtain emotion state recognition results; the vehicle nodes control the target vehicle to adjust the interaction mode and service content of the current vehicle system through a pre-constructed dynamic emotion adaptive engine and the emotion state recognition results; Among them, the vehicle nodes construct local models according to the received global model to be distributed, including: the vehicle nodes obtain their own vehicle data and their own emotion perception requirements; the vehicle nodes fine-tune the global model to be distributed according to the own vehicle data and the own emotion perception requirements to obtain local models; Among them, each node in the global model of the emotion dynamic evolution graph represents a specific emotion state, and each node includes an emotion intensity feature vector, an emotion duration feature vector, and an environmental feature vector; the edge between two nodes in the global model of the emotion dynamic evolution graph represents the transition probability between the emotion states corresponding to these two nodes; the transition probability between the emotion states is calculated by a probability model of emotion state transition; among them, the probability model of emotion state transition is constructed based on Riemannian metric and geodesic equation; the global model of the emotion dynamic evolution graph also includes a node feature update formula constructed based on Riemannian metric.

2. The distributed federated learning method based on emotion perception according to claim 1, wherein The method further includes: the vehicle nodes obtain a predefined reinforcement learning environment, a reinforcement learning algorithm, and a reward function; the vehicle nodes construct a dynamic emotion adaptive engine according to the reinforcement learning environment, the reinforcement learning algorithm, and the reward function; 3. The distributed federated learning method based on emotion perception according to claim 2, wherein the reinforcement learning environment at least includes the emotion state of the vehicle user, the driving environment, and the interaction mode currently provided by the system; the reinforcement learning algorithm is a Q-learning algorithm or a deep DQN network; 4. The distributed federated learning method based on emotion perception according to claim 1, characterized in that, The method further includes: the vehicle nodes sample multiple emotion recognition learning tasks from the distributed environment based on the meta-learning algorithm; among them, the emotion recognition learning task includes a training set and a test set; the vehicle nodes calculate the meta-learning loss and the meta-learning gradient based on the meta-learning algorithm, the emotion recognition learning task, and the local model; the vehicle nodes feedback the meta-learning loss and the meta-learning gradient to the central server; the central server updates the outer-loop parameters of the global model to be distributed according to the meta-learning loss and the meta-learning gradient feedback by all vehicle nodes in the distributed federated learning method to obtain an updated global model to be distributed; and execute the step of distributing the global model to be distributed to the vehicle nodes.

5. The distributed federated learning method based on emotion perception according to claim 4, wherein The vehicle node calculates the meta - learning loss and meta - learning gradient based on the meta - learning algorithm, the emotion recognition learning task, and the local model, including: The vehicle node updates the parameters of the local model in the inner loop according to the training set to obtain an updated local model; The vehicle node calculates the meta - learning loss and meta - learning gradient of the local model according to the test set.

6. The method for emotion-aware distributed federated learning according to claim 5, wherein The vehicle node updates the parameters of the local model in the inner loop according to the training set to obtain an updated local model, including: The vehicle node updates the model parameters of the local model according to the training set and the loss function on the minimization task to obtain an updated local model.

7. The distributed federated learning method based on emotion perception according to claim 5, wherein The vehicle node calculates the meta - learning loss and meta - learning gradient of the local model according to the test set, including: The vehicle node performs model evaluation calculations on the local model according to the test set and the differential privacy algorithm to obtain the meta - learning loss and meta - learning gradient.

8. The distributed federated learning method based on emotion perception according to claim 5, characterized in that The vehicle node controls the target vehicle to adjust the interaction method and service content of the current vehicle system through a pre - constructed dynamic emotion adaptive engine and the emotion state recognition result, including: The vehicle node determines the best action strategy through a pre - constructed dynamic emotion adaptive engine and the emotion state recognition result; wherein, the best action strategy includes an interaction method adjustment strategy and a service content adjustment strategy; The vehicle node controls the target vehicle to adjust the interaction method and service content of the current vehicle system according to the best action strategy.

Images