Distributed federal learning method based on emotion perception

Through the distributed federated learning method of emotion perception, combined with graph neural network and reinforcement learning, dynamically adjust the interaction mode and service content of the intelligent vehicle system, solving the problem that the existing system cannot recognize and deal with driver emotions, and improving user experience and security and provide guarantees on data security.

CN119047602AActive Publication Date: 2024-11-29GAC AION NEW ENERGY AUTOMOBILE CO LTD

Patent Information

Application Number
CN202411556064.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-04
Publication Date
2024-11-29
Estimated Expiration
2044-11-04

AI Technical Summary

Technical Problem

The existing intelligent on-board system cannot effectively identify the emotional state of the driver or passenger and provide adaptive feedback based on this, resulting in a reduced user experience and driving safety.

Method used

采用基于情绪感知的分布式联邦学习方法,通过情绪动态演化图全局模型和动态情绪自适应引擎,结合图神经网络和强化学习,动态调整交互方式和服务内容,适应用户情绪和驾驶环境。

Benefits of technology

Improve user experience and driving safety, enhance the accuracy and flexibility of the emotion recognition model, and ensure data security and user trust.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119047602A_ABST
    Figure CN119047602A_ABST
Patent Text Reader

Abstract

The invention provides a distributed federal learning method based on emotion perception. The method comprises the steps that a central server constructs an emotion dynamic evolution graph global model; the central server initializes the global model of the emotion dynamic evolution diagram to obtain a to-be-distributed global model; the central server distributes the global model to be distributed to the vehicle nodes; the vehicle node constructs a local model according to the received global model to be distributed; the vehicle node performs emotional state recognition of the vehicle user based on the local model to obtain an emotional state recognition result; and the vehicle node controls the target vehicle to adjust the interaction mode and the service content of the current vehicle system through a pre-constructed dynamic emotion self-adaptive engine and the emotion state recognition result. According to the method, the interaction mode and service content capability can be dynamically adjusted according to the emotional state so as to adapt to the current user emotion and driving environment capability, and the user experience and the driving safety are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and more specifically, to a distributed federated learning method based on emotion perception. Background Art

[0002] In the field of new energy and artificial intelligence, the rapid development of intelligent in-vehicle systems has driven the driving experience towards personalization and emotionalization. Modern drivers not only require cars to provide safe and efficient mobility functions, but also expect a more humane and emotionally resonant interactive experience. With the continuous advancement of new energy and autonomous driving technologies, vehicles are gradually transforming into intelligent partners that can understand human emotions and needs. However, although existing technologies can enhance the driving experience through augmented reality and multi-functional human-computer interaction systems (such as camera recognition, microphone voice, etc.), existing systems cannot effectively identify the emotional state of drivers or passengers and provide adaptive feedback accordingly. They lack the ability to dynamically adjust the interaction method and service content according to user emotions and driving environment, thereby reducing user experience and driving safety. Summary of the invention

[0003] The purpose of the embodiment of the present application is to provide a distributed federated learning method based on emotion perception, which can dynamically adjust the interaction mode and service content according to the emotional state to adapt to the current user emotion and driving environment, thereby improving user experience and driving safety.

[0004] In a first aspect, the present application provides a distributed federated learning method based on emotion perception, wherein the distributed federated learning method is applied to a distributed federated learning system based on emotion perception, wherein the distributed federated learning system includes a central server and at least one vehicle node, wherein: The central server constructs a global model of the emotion dynamic evolution graph; The central server initializes the global model of the emotion dynamic evolution graph to obtain a global model to be distributed; The central server distributes the global model to be distributed to the vehicle node; The vehicle node constructs a local model according to the received global model to be distributed; The vehicle node recognizes the emotional state of the vehicle user based on the local model to obtain an emotional state recognition result; The vehicle node controls the target vehicle to adjust the interaction mode and service content of the current vehicle system through a pre-built dynamic emotion adaptation engine and the emotion state recognition result.

[0005] In the above implementation process, the method is able to dynamically adjust the interaction mode and service content according to the emotional state to adapt to the current user emotion and driving environment, thereby improving user experience and driving safety.

[0006] Further, each node in the global model of the emotion dynamic evolution graph represents a specific emotion state, and each node includes an emotion intensity feature vector, an emotion duration feature vector, and an environment feature vector; The edge between two nodes in the global model of the emotional dynamic evolution graph represents the transition probability between the emotional states corresponding to the two nodes; The transition probability between the emotional states is calculated by a probability model of emotional state transition; wherein the probability model of emotional state transition is constructed based on Riemann metric and baseline equation; The global model of the emotion dynamic evolution graph also includes a node feature update formula constructed based on the Riemann metric.

[0007] Furthermore, the vehicle node constructs a local model according to the received global model to be distributed, including: The vehicle node acquires vehicle data of the vehicle and the emotion perception requirements of the vehicle; The vehicle node fine-tunes the global model to be distributed according to the vehicle data of the vehicle and the emotion perception requirements of the vehicle to obtain a local model.

[0008] Furthermore, the method further comprises: The vehicle node obtains a predefined reinforcement learning environment, a reinforcement learning algorithm, and a reward function; The vehicle node constructs a dynamic emotion adaptive engine according to the reinforcement learning environment, the reinforcement learning algorithm and the reward function.

[0009] Further, the reinforcement learning environment includes at least the emotional state of the vehicle user, the driving environment, and the interaction mode currently provided by the system; The reinforcement learning algorithm is a Q learning algorithm or a deep DQN network.

[0010] Furthermore, the method further comprises: The vehicle node samples multiple emotion recognition learning tasks from a distributed environment based on a meta-learning algorithm; wherein the emotion recognition learning task includes a training set and a test set; The vehicle node calculates the meta-learning loss and the meta-learning gradient based on the meta-learning algorithm, the emotion recognition learning task and the local model; The vehicle node feeds back the meta-learning loss and the meta-learning gradient to the central server; The central server updates the outer loop parameters of the global model to be distributed according to the meta-learning loss and meta-learning gradient fed back by all vehicle nodes in the distributed federated learning method to obtain an updated global model to be distributed; and executes the distribution of the global model to be distributed to the vehicle nodes.

[0011] Furthermore, the vehicle node calculates the meta-learning loss and the meta-learning gradient based on the meta-learning algorithm, the emotion recognition learning task and the local model, including: The vehicle node updates the inner loop parameters of the local model according to the training set to obtain an updated local model; The vehicle node calculates the meta-learning loss and the meta-learning gradient of the local model according to the test set.

[0012] Furthermore, the vehicle node updates the parameters of the local model in an inner loop according to the training set to obtain an updated local model, including: The vehicle node updates the model parameters of the local model according to the training set and the loss function on the minimization task to obtain an updated local model.

[0013] Furthermore, the vehicle node calculates the meta-learning loss and meta-learning gradient of the local model according to the test set, including: The vehicle node performs model evaluation calculation on the local model according to the test set and the differential privacy algorithm to obtain the meta-learning loss and the meta-learning gradient.

[0014] Furthermore, the vehicle node controls the target vehicle to adjust the interaction mode and service content of the current vehicle system through the pre-built dynamic emotion adaptation engine and the emotion state recognition result, including: The vehicle node determines the best action strategy through a pre-built dynamic emotion adaptation engine and the emotion state recognition result; wherein the best action strategy includes an interaction mode adjustment strategy and a service content adjustment strategy; The vehicle node controls the target vehicle to adjust the interaction mode and service content of the current vehicle system according to the optimal action strategy.

[0015] The beneficial effects of this application are: by combining graph neural networks and Riemann metrics, the transition dynamics between emotional states and the influence of situational factors can be better analyzed; Ability to introduce reinforcement learning, enabling the system to learn and take the best action strategy in a given state to maximize user satisfaction, safety, and interaction efficiency; It is possible to introduce differential privacy technology in the meta-learning process to form a new global model without exposing sensitive information of any individual node, which greatly improves data security and user trust in the federated learning system; By combining federated learning with meta-learning, the accuracy and flexibility of the emotion recognition model can be improved, while the model's ability to quickly adapt to unknown emotional states can be enhanced, bringing significant improvements to personalized emotion recognition services. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 A flow chart of a distributed federated learning method based on emotion perception provided in an embodiment of the present application; Figure 2 A flowchart of another distributed federated learning method based on emotion perception provided in an embodiment of the present application; Figure 3 A schematic diagram of a framework flow of a distributed federated learning method based on emotion perception provided in an embodiment of the present application; Figure 4 A schematic diagram of an intelligent car interactive experience system that applies a distributed federated learning method based on emotion perception provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0019] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0020] Example 1 Please see Figure 1 , Figure 1A flow chart of a distributed federated learning method based on emotion perception provided in this embodiment. The distributed federated learning method based on emotion perception is applied to a distributed federated learning system based on emotion perception, the distributed federated learning system includes a central server and at least one vehicle node, and the method includes: S110. The central server constructs a global model of the emotion dynamic evolution graph.

[0021] S120: The central server initializes the global model of the emotion dynamic evolution graph to obtain a global model to be distributed.

[0022] S130. The central server distributes the global model to be distributed to the vehicle nodes.

[0023] S210: The vehicle node constructs a local model according to the received global model to be distributed.

[0024] S220. The vehicle node recognizes the emotional state of the vehicle user based on the local model to obtain an emotional state recognition result.

[0025] In this embodiment, the vehicle user can be a driver or a passenger.

[0026] S230: The vehicle node controls the target vehicle to adjust the interaction mode and service content of the current vehicle system through the pre-built dynamic emotion adaptation engine and the emotion state recognition result.

[0027] In this embodiment, in order to enable the vehicle system to perceive emotions, the method proposes an emotion perception adaptive algorithm (ESA), which is actually composed of three frameworks: (1) Emotion Dynamic Evolution Graph (EDEG) model.

[0028] (2) Dynamic Adaptation Engine.

[0029] (3) Personalized services enhanced by federated learning (FL component).

[0030] Specifically, because emotions are not static but constantly evolve in different environments and situations, traditional emotion recognition methods often ignore the dynamics of transitions between emotional states and the influence of situational factors, limiting the accuracy of emotion recognition and the depth of its application.

[0031] To solve this problem, this method proposes the emotion dynamic evolution graph (EDEG) model and combines it with the concepts of dynamic system theory. Through this model, graph neural networks are used to capture the complex relationships between emotional states and their evolution over time to achieve accurate prediction and response to the driver's emotional changes.

[0032] In this embodiment, the execution subject of the method may be a computing device such as a computer or a server, and no limitation is made in this embodiment.

[0033] It can be seen that the distributed federated learning method based on emotion perception described in this embodiment can dynamically adjust the interaction mode and service content according to the emotional state to adapt to the current user emotion and driving environment, thereby improving user experience and driving safety.

[0034] Example 2 Please see Figure 2 , Figure 2 A flow chart of a distributed federated learning method based on emotion perception provided in this embodiment. The distributed federated learning method based on emotion perception is applied to a distributed federated learning system based on emotion perception, the distributed federated learning system includes a central server and at least one vehicle node, and the method includes: S110. The central server constructs a global model of the emotion dynamic evolution graph.

[0035] In this embodiment, each node in the global model of the dynamic evolution graph of emotions represents a specific emotional state, and each node includes an emotion intensity feature vector, an emotion duration feature vector, and an environment feature vector; The edge between two nodes in the global model of the emotional dynamic evolution graph represents the transition probability between the corresponding emotional states of the two nodes; The transition probability between emotional states is calculated by the probability model of emotional state transition; wherein the probability model of emotional state transition is constructed based on Riemann metric and baseline equation; The global model of the emotion dynamic evolution graph also includes a node feature update formula built based on the Riemann metric.

[0036] In this embodiment, the construction of the global model of the emotion dynamic evolution graph is mainly divided into two parts: point construction and edge construction: (1) Click Build: This method converts each node v i represents a specific emotional state. Where i is the index of the emotional state. Each node contains a feature vector x i It can include the intensity and duration of emotions, as well as environmental characteristics such as time and place.

[0037] (2) While building: Edge ij Represents emotional state from v i Convert to v j This can be obtained by analyzing historical sentiment change data, for example, using conditional probability as: ; Among them, P (v i , v j ) is the observed emotional state v i and v j The probability, P(v i ) is the observed emotional state v i probability.

[0038] In this embodiment, in order to consider the complex relationship between emotional states when updating the emotional state features, the method introduces the Riemann metric, and the specific method is as follows: First, assume that the emotion manifold is a smooth manifold whose local components are the eigenvectors of the emotional state Therefore, a Riemann metric g can be defined to measure the distance on the manifold, expressed as:

[0039] in, It is a manifold The tangent space on the tangent space represents the direction and speed of the change of emotional state. The dynamic evolution of emotional state can be expressed by a vector field To describe, the vector field gives the tangent vector of each point on the manifold, that is, the direction of the change of emotional state. The vector field can be given by the following differential equation:

[0040] in, represents the rate of change of the emotional state over time. A connection ∇ is defined on the manifold, which provides a method for translating vectors parallel to the manifold. The connection enables the method to understand how the emotional state evolves over time and how this evolution depends on the geometric structure of the emotional manifold.

[0041] After the introduction of the Riemannian metric, the update formula of the node is adjusted to reflect the geometric distance on the manifold. The update of the node feature can include a mapping function that takes into account the geometric path length between nodes, and the formula is adjusted to:

[0042] Among them, β ij is a weight calculated based on the Riemannian metric, taking into account the geometric distance between nodes i and j on the emotion manifold. This weight can be adjusted based on the geodesic distance between nodes, thus ensuring that the geometric relationship between emotional states is taken into account when information is propagated.

[0043] In the emotional manifold Computing the geodesic distance between two points is key to understanding the evolution of emotional states. This can be achieved by solving the shortest path problem under the Riemannian metric, using the geodesic equation:

[0044] in, is the Christoffel symbol, representing the manifold The connection on , used to describe the curvature of the manifold, represents the covariant derivative along a path on the manifold.

[0045] Using the Riemann metric, this method can redefine the probability model of emotional state transitions. The transition probability depends not only on the relationship between emotional states, but also on their geometric positions on the emotional manifold. Therefore, the transition probability can be dynamically adjusted by considering the function of the geodesic distance between nodes:

[0046] Among them, d g (v i , v j ) is the node v calculated based on the Riemann metric g i and v j The geodesic distance between them is λ, and λ is a tuning parameter. In this way, the system is able to consider the complex geometric relationships between emotional states when updating the emotional state features, thus providing a deeper understanding of the dynamic evolution of emotional states.

[0047] S120: The central server initializes the global model of the emotion dynamic evolution graph to obtain a global model to be distributed.

[0048] In this embodiment, steps S210 to S130 are the process of performing inner loop parameter update and outer loop parameter update on the global model to be distributed in this application. Figure 2 The dashed lines in the figure indicate that the steps are optional.

[0049] In this embodiment, Figure 2 The dotted lines in the one-sided execution flow of the central server and vehicle nodes in the figure refer to the next optional step of the central server and vehicle nodes. The reason why it is called an optional step is that it needs to wait for the input of the meta-learning loss, meta-learning gradient, or the model to be distributed before it can be implemented. Therefore, direct execution is not advisable and can only be executed as an optional step under certain conditions.

[0050] S210. The vehicle node samples multiple emotion recognition learning tasks from a distributed environment based on a meta-learning algorithm; wherein the emotion recognition learning task includes a training set and a test set.

[0051] S221. The vehicle node updates the model parameters of the local model according to the training set and the loss function on the minimization task to obtain an updated local model.

[0052] S222. The vehicle node performs model evaluation calculation on the local model based on the test set and the differential privacy algorithm to obtain the meta-learning loss and meta-learning gradient.

[0053] In this embodiment, the method introduces differential privacy in the meta-learning process, which can further protect the privacy of data in each task. When comparing y and its gradient, differential privacy is ensured by adding random noise:

[0054]

[0055] in, is the noise added according to the differential privacy parameters ϵ (privacy loss) and δ (failure probability).

[0056] A central server (which can be a cloud server) collects all privacy-preserving local updates and securely aggregates them to form a new version of the global model.

[0057]

[0058] Where N is the total number of nodes participating in learning.

[0059] After the global model is updated, it is distributed to all nodes again. Each node can fine-tune the global model according to its own specific data and needs to achieve better personalized adaptability.

[0060]

[0061] Where ξ is the learning rate for personalized fine-tuning.

[0062] By combining meta-learning and differential privacy, this framework not only improves the personalized adaptability and learning efficiency of the model in a federated learning environment, but also enhances data privacy protection. Meta-learning enables the global model to quickly adjust on a small amount of data to adapt to new or rare emotional states, while differential privacy ensures that the privacy of personal data is not leaked in this process. This framework provides a technical foundation for realizing truly private and efficient personalized emotion recognition services.

[0063] S230: The vehicle node feeds back the meta-learning loss and meta-learning gradient to the central server.

[0064] In this embodiment, the user data of a single vehicle cannot meet the data requirements for emotion recognition, so the method constructs a cross-vehicle emotion recognition model. At the same time, in order to protect the privacy of user data, the method introduces a federated learning framework to implement a distributed learning system, thereby achieving cross-vehicle collaborative learning without sharing user personal data, thereby improving the performance and personalized adaptability of the emotion recognition algorithm.

[0065] In this embodiment, the privacy leakage is avoided by transmitting the model parameters or gradient information instead of the original data. Specific model initialization and distribution The central server (which can be a cloud server) first initializes the global model θ global This global model is then distributed to all nodes (vehicles) participating in FL.

[0066]

[0067] Each node receives an initial version of the global model as their respective local model.

[0068] In order to enable each independent node in distributed training to better adapt to new scenarios, this method introduces meta-learning in the framework of federated learning. Meta-learning focuses on how to train a model on multiple learning tasks so that it can quickly adapt to new tasks using past experience. In a federated learning environment, meta-learning is used to optimize the learning strategy of the global model so that it can learn and adapt to new scenarios more effectively on distributed nodes.

[0069] S130. The central server updates the outer loop parameters of the global model to be distributed according to the meta-learning loss and meta-learning gradient fed back by all vehicle nodes in the distributed federated learning method to obtain an updated global model to be distributed.

[0070] For example, the update process of the meta-learning model can be as follows: (1) Task sampling: Meta-learning begins by sampling multiple emotion recognition learning tasks from a distributed environment. Each task By dataset Definition, including training set and test set.

[0071]

[0072] in, is the distribution over all possible tasks.

[0073] (2) Inner loop update: For each task , the model parameters θ are updated by minimizing the loss function on the task, the so-called inner loop update:

[0074] Here, α is the intra-task learning rate.

[0075] (3) Meta-learning loss function: The updated parameter θ i On Task The test set Evaluate on to compute the meta-learning loss:

[0076] (4) Outer loop update: Based on the meta-learning loss of all emotion recognition tasks, the global model parameters are updated, which is the so-called outer loop update:

[0077] Where β is the meta-learning rate and K is the number of tasks.

[0078] S140. The central server distributes the global model to be distributed to the vehicle nodes.

[0079] S241, the vehicle node obtains the vehicle data and the vehicle emotion perception requirements.

[0080] S242. The vehicle node fine-tunes the global model to be distributed according to the vehicle data and the emotion perception requirements of the vehicle to obtain a local model.

[0081] S250: The vehicle node recognizes the emotional state of the vehicle user based on the local model to obtain an emotional state recognition result.

[0082] S260, the vehicle node obtains a predefined reinforcement learning environment, a reinforcement learning algorithm, and a reward function.

[0083] In this embodiment, the reinforcement learning environment includes at least the emotional state of the vehicle user, the driving environment, and the interaction mode currently provided by the system; The reinforcement learning algorithm is the Q learning algorithm or the deep DQN network.

[0084] In this embodiment, after obtaining the emotion prediction, in order to achieve better human-computer interaction, the method introduces a reinforcement learning approach: First, the method defines the environment for reinforcement learning, which includes the emotional state, the driving environment, and the interaction methods provided by the system. t The representation at time t includes:

[0085] e t : The driver’s emotional state, provided by the emotion recognition model; d t : Driving environment status, such as traffic conditions, weather, etc.; i t : The system’s current interaction mode and service content.

[0086] Then, the agent (i.e., the dynamic emotion adaptation engine) takes action a at each time step t t , designed to adjust the system's interaction methods and service content to suit the current mood and driving environment.

[0087] The action space can be defined as a set of executable services and interaction strategies, such as: A={a1, a2, …, a n} Each action a i Represents a specific service adjustment or interaction strategy.

[0088] S270, the vehicle node builds a dynamic emotion adaptive engine according to the reinforcement learning environment, the reinforcement learning algorithm and the reward function.

[0089] In this example, the agent's goal is to learn a strategy π(s t ), in a given state s t Next select an action t , to maximize the future cumulative reward. The reward function R (s t , a t ) measures action a t In status t The immediate utility under the situation can include multiple dimensions such as user satisfaction, security and interaction efficiency.

[0090]

[0091] Among them, w1, w2, and w3 are weight parameters used to balance the importance of different objectives.

[0092] Finally, the method uses Q-learning or deep Q-network (DQN) as the learning algorithm to optimize the decision-making process. For DQN, the Q function is defined as Q(s t , a t ;θ), indicating that in state s t Take action t The expected return of . DQN updates the network parameters θ by minimizing the following loss function:

[0093] Where γ is the discount factor, which represents the current value of future rewards; Represents the parameters of the target network, which is used to stabilize the learning process; D is the experience playback buffer, which stores past transformations. Through the above methods, the dynamic emotion adaptation engine can automatically learn and adjust the best interaction mode and service content based on real-time emotion feedback and driving environment, thereby providing a deeply personalized and dynamically adaptive user experience.

[0094] S281. The vehicle node determines the best action strategy through a pre-built dynamic emotion adaptation engine and emotion state recognition results; wherein the best action strategy includes an interaction mode adjustment strategy and a service content adjustment strategy.

[0095] S282. The vehicle node controls the target vehicle according to the optimal action strategy to adjust the interaction mode and service content of the current vehicle system.

[0096] Please see Figure 3 , Figure 3 A framework flow chart of a distributed federated learning method based on emotion perception is shown.

[0097] Please see Figure 4 , Figure 4 A schematic diagram of an intelligent car interactive experience system using a distributed federated learning method based on emotion perception is shown.

[0098] In this embodiment, the innovative point of the method is: (1) Consider the dynamic nature of emotions in emotion recognition and introduce graph neural network prediction. At the same time, consider the complex relationship between emotional states when updating emotional state features and increase the accuracy of prediction by introducing Riemann metric.

[0099] (2) By combining the dynamic emotion adaptation engine with a reinforcement learning method, this integration not only utilizes the precise emotional state provided by the emotion recognition model, but also enables the system to learn and take the best action strategy in a given state through the trial-and-error learning method of reinforcement learning to maximize user satisfaction, safety, and interaction efficiency. The implementation of this dynamic adjustment strategy brings unprecedented personalization and adaptability to the human-computer interaction system.

[0100] (3) A meta-learning method is introduced under the federated learning framework to form an advanced learning strategy to improve the adaptability of each independent node to new scenarios in distributed training. This combination not only allows the global model to utilize the experience gained from multiple emotion recognition tasks to optimize its learning strategy so that it can quickly adapt when encountering new tasks, but also makes the learning process on distributed nodes more efficient and effective.

[0101] In this embodiment, the execution subject of the method may be a computing device such as a computer or a server, and no limitation is made in this embodiment.

[0102] It can be seen that the distributed federated learning method based on emotion perception described in this embodiment can dynamically adjust the interaction mode and service content according to the emotional state to adapt to the current user emotion and driving environment, thereby improving user experience and driving safety.

[0103] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0104] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0105] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.

[0106] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0107] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0108] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. A distributed federated learning method based on emotion perception, characterized in that: The distributed federated learning method is applied to a distributed federated learning system based on emotion perception, wherein the distributed federated learning system includes a central server and at least one vehicle node, wherein: The central server constructs a global model of the emotion dynamic evolution graph; The central server initializes the global model of the emotion dynamic evolution graph to obtain a global model to be distributed; The central server distributes the global model to be distributed to the vehicle node; The vehicle node constructs a local model according to the received global model to be distributed; The vehicle node recognizes the emotional state of the vehicle user based on the local model to obtain an emotional state recognition result; The vehicle node controls the target vehicle to adjust the interaction mode and service content of the current vehicle system through a pre-built dynamic emotion adaptation engine and the emotion state recognition result.

2. The distributed federated learning method based on emotion perception according to claim 1 is characterized in that: Each node in the global model of the emotional dynamic evolution graph represents a specific emotional state, and each node includes an emotional intensity feature vector, an emotional duration feature vector, and an environmental feature vector; The edge between two nodes in the global model of the emotional dynamic evolution graph represents the transition probability between the emotional states corresponding to the two nodes; The transition probability between the emotional states is calculated by a probability model of emotional state transition; wherein the probability model of emotional state transition is constructed based on Riemann metric and baseline equation; The global model of the emotion dynamic evolution graph also includes a node feature update formula constructed based on the Riemann metric.

3. The distributed federated learning method based on emotion perception according to claim 1 is characterized in that: The vehicle node constructs a local model according to the received global model to be distributed, including: The vehicle node acquires vehicle data of the vehicle and the emotion perception requirements of the vehicle; The vehicle node fine-tunes the global model to be distributed according to the vehicle data of the vehicle and the emotion perception requirements of the vehicle to obtain a local model.

4. The distributed federated learning method based on emotion perception according to claim 1 is characterized in that: The method further comprises: The vehicle node obtains a predefined reinforcement learning environment, a reinforcement learning algorithm, and a reward function; The vehicle node constructs a dynamic emotion adaptive engine according to the reinforcement learning environment, the reinforcement learning algorithm and the reward function.

5. The distributed federated learning method based on emotion perception according to claim 4 is characterized in that: The reinforcement learning environment includes at least the emotional state of the vehicle user, the driving environment, and the interaction mode currently provided by the system; The reinforcement learning algorithm is a Q learning algorithm or a deep DQN network.

6. The distributed federated learning method based on emotion perception according to claim 1, characterized in that: The method further comprises: The vehicle node samples multiple emotion recognition learning tasks from a distributed environment based on a meta-learning algorithm; wherein the emotion recognition learning task includes a training set and a test set; The vehicle node calculates the meta-learning loss and the meta-learning gradient based on the meta-learning algorithm, the emotion recognition learning task and the local model; The vehicle node feeds back the meta-learning loss and the meta-learning gradient to the central server; The central server updates the outer loop parameters of the global model to be distributed according to the meta-learning loss and meta-learning gradient fed back by all vehicle nodes in the distributed federated learning method to obtain an updated global model to be distributed; and executes the distribution of the global model to be distributed to the vehicle nodes.

7. The distributed federated learning method based on emotion perception according to claim 6 is characterized in that: The vehicle node calculates the meta-learning loss and the meta-learning gradient based on the meta-learning algorithm, the emotion recognition learning task and the local model, including: The vehicle node updates the inner loop parameters of the local model according to the training set to obtain an updated local model; The vehicle node calculates the meta-learning loss and the meta-learning gradient of the local model according to the test set.

8. The distributed federated learning method based on emotion perception according to claim 7 is characterized in that: The vehicle node updates the parameters of the local model in an inner loop according to the training set to obtain an updated local model, including: The vehicle node updates the model parameters of the local model according to the training set and the loss function on the minimization task to obtain an updated local model.

9. The distributed federated learning method based on emotion perception according to claim 7, characterized in that: The vehicle node calculates the meta-learning loss and the meta-learning gradient of the local model according to the test set, including: The vehicle node performs model evaluation calculation on the local model according to the test set and the differential privacy algorithm to obtain the meta-learning loss and the meta-learning gradient.

10. The distributed federated learning method based on emotion perception according to claim 7, characterized in that: The vehicle node controls the target vehicle to adjust the interaction mode and service content of the current vehicle system through the pre-built dynamic emotion adaptation engine and the emotion state recognition result, including: The vehicle node determines the best action strategy through a pre-built dynamic emotion adaptation engine and the emotion state recognition result; wherein the best action strategy includes an interaction mode adjustment strategy and a service content adjustment strategy; The vehicle node controls the target vehicle to adjust the interaction mode and service content of the current vehicle system according to the optimal action strategy.

Citation Information

Patent Citations

  • Emotion recognition method based on multi-modal clustering federal learning

    CN115620370A

  • Human-vehicle interaction system based on multi-sensory emotion regulation

    CN116118760A

  • Emotion prediction method based on vehicle equipment state and user driving behavior

    CN116509393A

  • Vehicle-mounted terminal control method and device and vehicle-mounted terminal

    CN117485265A

  • Metacontinuous federated learning system and method based on fast learning and knowledge accumulation

    CN117910600A

Cited By

  • Incremental cloud learning method and system for vehicle-mounted emotion data

    CN120448976A