Qt interactive game role action response control method and system fused with AI emotion recognition

Abstract

The invention relates to the technical field of game development and artificial intelligence, and discloses a Qt interactive game role action response control method fused with AI emotion recognition, which comprises the following steps: S1, acquiring multi-modal data such as facial expression, voice intonation and limb action of a player by using sensors such as a camera and a microphone on game equipment; s2, the collected multi-modal data are transmitted to an AI emotion recognition module, a deep learning algorithm is adopted to analyze and process the data, and the current emotion state of the player is recognized; and S3, obtaining current game scene information and other operation instructions input by the player through the Qt framework. According to the Qt interactive game role action response control method and system fused with AI emotion recognition, through the AI emotion recognition technology, a game role can sense the emotion state of a player and make the emotion interaction action matched with the game role, the emotion resonance between the game role and the player is enhanced, and the interaction experience and immersion of a game are greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of game development and artificial intelligence technology, and specifically to a Qt interactive game character action response control method and system integrating AI emotion recognition. Background Art

[0002] In existing game development, the action response control of game characters is mostly based on preset rules and simple trigger conditions; for example, when the player issues a specific instruction, the game character responds according to a pre-set action sequence. However, this method has many shortcomings. First, there is a lack of perception of the player's emotional state, and it is impossible to make corresponding emotional interaction actions based on the player's emotional changes, resulting in a relatively stiff interaction between the game character and the player, and the game experience is not immersive and interesting enough. This stiff interaction mode makes it difficult to establish an emotional connection between the game character and the player, greatly weakening the immersion and fun of the game experience.

[0003] At the same time, existing action response control methods lack flexibility and are difficult to adapt to complex and changing game scenarios and players' diverse operational needs. For example, when facing the personalized operations of different players, the action performances of game characters are often stereotyped, which makes it difficult to meet players' growing pursuit of unique gaming experiences. In addition, in terms of multimodal data processing, traditional technologies lack the ability to integrate and analyze non-traditional command information such as voice and expressions input by players, and are unable to fully tap the value of this information, resulting in a significant reduction in the naturalness and rationality of the game character's action response.

[0004] With the development of the gaming industry, players have higher and higher requirements for the authenticity and emotional resonance of game interactions. Traditional game character action response control methods have gradually failed to meet market demand.

[0005] Therefore, a Qt interactive game character action response control method and system integrating AI emotion recognition are proposed to solve the above-mentioned problems. Summary of the Invention

[0006] (1) Technical problems solved

[0007] In response to the shortcomings of the existing technology, the present invention provides a Qt interactive game character action response control method and system that integrates AI emotion recognition, which has the advantages of intelligent emotional interaction and enhanced immersive experience. It solves the problem of lack of perception of the player's emotional state and inability to make corresponding emotional interaction actions according to the player's emotional changes, resulting in relatively stiff interaction between game characters and players and insufficient immersion and fun in the gaming experience.

[0008] (2) Technical solution

[0009] To achieve the above-mentioned intelligent emotional interaction and enhance the immersive experience, the present invention provides the following technical solution: a Qt interactive game character action response control method integrating AI emotion recognition, comprising the following steps:

[0010] S1: Use sensors such as cameras and microphones on gaming devices to collect multimodal data such as players' facial expressions, voice intonation, and body movements;

[0011] S2: The collected multimodal data is transmitted to the AI ​​emotion recognition module, which uses a deep learning algorithm to analyze and process the data to identify the player's current emotional state;

[0012] S3: Obtain the current game scene information and other operation instructions input by the player through the Qt framework;

[0013] S4: Build an action decision model that comprehensively considers the player's emotional state, game scene information, and operation instructions, and generates appropriate action response strategies for the game character through preset rules and machine learning algorithms;

[0014] S5: Based on the action response strategy generated by the action decision model, the game character is controlled to perform corresponding actions through Qt's graphics rendering and animation system, and displayed on the game screen.

[0015] Preferably, the deep learning algorithm includes a facial expression recognition model based on a convolutional neural network and a speech emotion recognition model based on a long short-term memory network, wherein the input layer of the facial expression recognition model receives preprocessed facial image data, and the output layer is a probability distribution vector containing at least 7 basic emotion classifications; the input layer of the speech emotion recognition model receives a Mel-frequency cepstral coefficient feature vector, and the output layer is an emotion probability distribution vector of the same dimension as the facial expression recognition model.

[0016] Preferably, the emotional text semantic analysis model is constructed, and a BERT pre-trained language model is used to perform emotional polarity analysis on the player's voice commands, extracting emotional keywords (such as "hate" and "excited") in the text and their intensity values;

[0017] The attention mechanism is used to calculate the semantic relevance between emotional keywords and facial micro-expressions and voice intonation features to generate a multimodal emotional consistency score.

[0018] When the consistency score between the text sentiment polarity and non-verbal signals is less than 0.6, the sentiment verification mechanism is triggered, and secondary confirmation is performed through NPC dialogue questions (such as "Do you really mean this?").

[0019] Preferably, generating the action response strategy in step S4 further includes:

[0020] Building an emotion-plot mapping matrix Where n is the dimension of the emotional state vector, m is the number of preset plot branches, and the matrix element M_ij represents the weight value of emotional state i triggering plot branch j;

[0021] Dynamically update the matrix weights through the reinforcement learning algorithm, and calculate the cumulative reward value R_t = ∑γ^t·r_t based on the player's historical emotional feedback data (γ is the discount factor, r_t is the emotional reward at time t);

[0022] When a sudden change in the player's emotional state is detected (such as Δemotion_intensity>0.5), Kalman filtering is used to predict the optimal response path of the plot branch, so that the NPC lines generated are consistent with the continuity of emotional changes.

[0023] A Qt interactive game character action response control system integrated with AI emotion recognition, including:

[0024] Data collection module: used to collect multimodal data such as players' facial expressions, voice intonation, body movements, etc.

[0025] The AI ​​emotion recognition module receives data transmitted by the multimodal data acquisition module, uses deep learning algorithms to perform emotion recognition processing, and outputs the player's emotional state information;

[0026] The game scene and instruction acquisition module, based on the Qt framework, obtains the current game scene information and the operation instructions input by the player in real time;

[0027] The action decision module is used to build an action decision model and generate the action response strategy of the game character based on the player's emotional state, game scene information and operation instructions;

[0028] The action execution module is used to control the game character to perform corresponding actions based on the action response strategy generated by the action decision module through Qt's graphics rendering and animation system, and display them on the game screen;

[0029] Emotional plot generation module: used to dynamically generate plot content and interactive dialogues that conform to emotional logic based on the player's emotional state and game history data.

[0030] Preferably, the AI ​​emotion recognition module includes:

[0031] An image preprocessing unit for performing image cropping, histogram equalization, and normalization operations;

[0032] Feature extraction unit, including a pre-trained ResNet-50 convolutional neural network;

[0033] Sentiment classification unit, which includes a fully connected layer and a Softmax activation function and outputs a 7-dimensional sentiment probability vector;

[0034] The fusion decision unit adopts weighted average algorithm to fuse the facial expression and speech emotion recognition results.

[0035] Preferably, the action decision module includes:

[0036] A scene analysis unit, used to analyze game scene information and generate feature vectors;

[0037] Rule engine unit, storing at least 100 state transition rules;

[0038] State machine management unit, which implements the creation, monitoring and state transfer of finite state machines based on QStateMachine;

[0039] The action generation unit selects a matching action tuple from the action library according to the current state.

[0040] Preferably, the action execution module includes:

[0041] Animation control unit, which implements skeletal animation playback and mixing based on the Qt3DAnimation framework;

[0042] Expression rendering unit, which realizes facial expression animation based on vertex blending technology;

[0043] The physics simulation unit calculates the physical effects of actions based on the Bullet physics engine; the network synchronization unit uses a client-server architecture to achieve multi-player action synchronization with a synchronization frequency of no less than 60Hz.

[0044] Preferably, the system further comprises a data quality assessment module, which comprises:

[0045] a signal feature extraction circuit configured to extract time domain features, frequency domain features, and statistical features of the multimodal data;

[0046] The quality assessment model uses the support vector machine algorithm to classify and evaluate data quality, and outputs a three-dimensional quality vector including clarity, completeness, and consistency;

[0047] A data compensation unit triggers a data interpolation algorithm or a resampling algorithm to perform data compensation when any dimension in the quality vector is lower than a preset threshold;

[0048] The feedback control circuit is configured to dynamically adjust the sampling parameters of the data acquisition module according to the quality assessment result, including but not limited to the frame rate, exposure time and gain value.

[0049] (3) Beneficial effects

[0050] Compared with the existing technology, the present invention provides a Qt interactive game character action response control method and system integrated with AI emotion recognition, which has the following beneficial effects:

[0051] 1. This Qt interactive game character action response control method and system that integrates AI emotion recognition uses AI emotion recognition technology to enable game characters to perceive the player's emotional state and make emotional interaction actions that match it, enhancing the emotional resonance between the game character and the player, and greatly improving the interactive experience and immersion of the game; the action decision model comprehensively considers multiple factors and can generate diverse action response strategies for the game character based on different game scenarios, player operations and emotional states, meeting the player's personalized needs and making the game character's action performance more flexible and rich.

[0052] 2. This Qt interactive game character action response control method and system, which integrates AI emotion recognition, leverages the powerful functions of the Qt framework to achieve efficient acquisition of game scene information, accurate capture of player operation instructions, and smooth rendering and display of game character actions, ensuring the stability and efficiency of the entire system. At the same time, it fully utilizes multimodal data, combining player facial expressions, voice intonation and other information with traditional operation instructions, providing a more comprehensive and rich basis for game character action decisions, making the game character's action responses more natural and reasonable. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a flow chart of the Qt interactive game character action response control method that integrates AI emotion recognition in the present invention;

[0054] Figure 2 This is a structural diagram of the Qt interactive game character action response control system that integrates AI emotion recognition in the present invention. DETAILED DESCRIPTION

[0055] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0056] Example 1

[0057] See also Figure 1 and Figure 2 A Qt interactive game character action response control method integrating AI emotion recognition includes the following steps:

[0058] S1: Use sensors such as cameras and microphones on gaming devices to collect multimodal data such as players' facial expressions, voice intonation, and body movements;

[0059] S2: The collected multimodal data is transmitted to the AI ​​emotion recognition module, which uses deep learning algorithms to analyze and process the data and identify the player's current emotional state, such as happiness, sadness, anger, etc.

[0060] Deep learning algorithms can be based on convolutional neural networks (CNNs), recurrent neural networks (RNNs), and their variants, such as long short-term memory networks (LSTMs). By training on large amounts of labeled emotion data, they can build emotion recognition models and accurately identify players' emotional states.

[0061] S3: Obtain the current game scene information and other operation instructions input by the player through the Qt framework, such as keyboard input, mouse operation, etc.

[0062] The Qt framework provides a rich set of libraries and tools that can easily interact with the game scene, obtain various parameters and status information in the game scene in real time, and accurately capture the player's operation instructions;

[0063] S4: Build an action decision model that comprehensively considers the player's emotional state, game scene information, and operation instructions, and generates appropriate action response strategies for the game character through preset rules and machine learning algorithms;

[0064] For example, when the player is in a happy emotional state and issues a forward command in the game scene, the action decision model can select a cheerful running action for the game character; when the player is in an angry emotional state and issues an attack command, a more intense attack action is selected for the game character;

[0065] S5: Based on the action response strategy generated by the action decision model, the game character is controlled to perform the corresponding action through Qt's graphics rendering and animation system, and displayed on the game screen;

[0066] Qt's graphics rendering and animation system can efficiently process the action data of game characters, achieving smooth and natural action display effects.

[0067] A Qt interactive game character action response control system integrated with AI emotion recognition, including:

[0068] Data acquisition module: used to collect multimodal data such as players' facial expressions, voice intonation, and body movements. It consists of sensors such as cameras and microphones and corresponding data acquisition interfaces;

[0069] AI emotion recognition module: used to receive data transmitted by the multimodal data acquisition module, use deep learning algorithms to perform emotion recognition processing, and output the player's emotional state information;

[0070] This module can run on a cloud computing platform or a local high-performance computing device to ensure the efficiency and accuracy of emotion recognition.

[0071] Game scene and instruction acquisition module: Based on the Qt framework, it obtains the current game scene information and the operation instructions input by the player in real time, and transmits this information to the action decision module;

[0072] Action Decision Module: This module is used to build an action decision model, generate the game character's action response strategy based on the player's emotional state, game scene information, and operation instructions, and send the strategy to the action execution module.

[0073] Action execution module: used to control the game character to perform corresponding actions based on the action response strategy generated by the action decision module through Qt's graphics rendering and animation system, and display them on the game screen;

[0074] Emotional plot generation module: used to dynamically generate plot content and interactive dialogues that conform to emotional logic based on the player's emotional state and game history data;

[0075] Data quality assessment module: Through components such as feature analysis circuits and quality judgment units, it extracts multimodal data features and evaluates their clarity, completeness, and consistency. Based on the assessment results, it uses compensation actuators and parameter adjustment components to compensate for low-quality data and dynamically adjusts the data acquisition module parameters.

[0076] Example 2

[0077] See also Figure 1 and Figure 2 In the AI ​​emotion recognition module, the AI ​​emotion recognition module includes:

[0078] An image preprocessing unit for performing image cropping, histogram equalization, and normalization operations;

[0079] Feature extraction unit, including a pre-trained ResNet-50 convolutional neural network;

[0080] Sentiment classification unit, which includes a fully connected layer and a Softmax activation function and outputs a 7-dimensional sentiment probability vector;

[0081] The fusion decision unit adopts weighted average algorithm to fuse the facial expression and speech emotion recognition results.

[0082] In this embodiment, the AI ​​emotion recognition module crops, enhances and normalizes the original image through the image preprocessing unit, uses the pre-trained ResNet-50 network to extract deep features, and generates a 7-dimensional emotion probability vector through the fully connected layer and Softmax activation to achieve accurate classification of facial expressions; then, the voice emotion recognition results are fused through the weighted average algorithm to output the final emotional state of the comprehensive multimodal information, providing an emotional basis for the game character's action decision-making.

[0083] The action decision module includes:

[0084] A scene analysis unit, used to analyze game scene information and generate feature vectors;

[0085] Rule engine unit, storing at least 100 state transition rules;

[0086] State machine management unit, which implements the creation, monitoring and state transfer of finite state machines based on QStateMachine;

[0087] The action generation unit selects a matching action tuple from the action library according to the current state.

[0088] In this embodiment, the action decision module generates a feature vector by analyzing the game environment parameters through the scene analysis unit. Combined with more than 100 state transition rules preset in the rule engine, the state machine management unit dynamically manages the character state migration based on QStateMachine; the action generation unit then indexes the action library according to the current state, and outputs an action tuple containing animation, expression, and sound effect parameters to achieve intelligent decision-making and dynamic generation of game character behavior.

[0089] The action execution module includes:

[0090] Animation control unit, which implements skeletal animation playback and mixing based on the Qt3DAnimation framework;

[0091] Expression rendering unit, which realizes facial expression animation based on vertex blending technology;

[0092] The physics simulation unit calculates the physical effects of actions based on the Bullet physics engine; the network synchronization unit uses a client-server architecture to achieve multi-player action synchronization with a synchronization frequency of no less than 60Hz.

[0093] In this embodiment, the action execution module drives the smooth playback and fusion of skeletal animation through the Qt3DAnimation framework, combines vertex blending technology to achieve delicate rendering of facial micro-expressions, and uses the Bullet physics engine to calculate physical feedback such as collision and gravity of actions to build an immersive single-player experience; at the same time, it adopts a client-server architecture to ensure the real-time consistency of character actions in multi-player scenarios with a synchronization frequency of more than 60Hz, realizing collaborative interaction across devices.

[0094] The data quality assessment module includes:

[0095] a signal feature extraction circuit configured to extract time domain features, frequency domain features, and statistical features of the multimodal data;

[0096] The quality assessment model uses the support vector machine algorithm to classify and evaluate data quality, and outputs a three-dimensional quality vector including clarity, completeness, and consistency;

[0097] A data compensation unit triggers a data interpolation algorithm or a resampling algorithm to perform data compensation when any dimension in the quality vector is lower than a preset threshold;

[0098] The feedback control circuit is configured to dynamically adjust the sampling parameters of the data acquisition module according to the quality assessment result, including but not limited to the frame rate, exposure time and gain value.

[0099] In this embodiment, the data quality assessment module analyzes the time-frequency domain and statistical characteristics of multimodal data through a signal feature extraction circuit, and uses a support vector machine algorithm to output a three-dimensional quality vector of clarity, completeness, and consistency to quantify data reliability in real time. When quality degradation is detected, the data compensation unit immediately performs interpolation or resampling repair, and dynamically adjusts the frame rate, exposure and other parameters of the acquisition module through a feedback control circuit to form a closed-loop optimization system for data quality.

[0100] The emotional plot generation module includes:

[0101] Plot semantic network construction unit: Constructs a plot semantic network, sorts out game plot elements and logical relationships, and provides a basic framework for subsequent plot generation.

[0102] Emotional plot decision engine: Based on the player's emotional state and game progress, decisions are made to generate plot directions that conform to emotional logic.

[0103] Dialogue generation and emotion adaptation unit: Generates interactive dialogues that fit the player's emotions and plot to enhance the sense of immersion.

[0104] Plot continuity guarantee mechanism: ensure that the plot transition is natural and coherent, and avoid problems such as abrupt plots.

[0105] Dynamic resource scheduling sub-module: Rationally schedule in-game resources to ensure the smoothness of plot generation and presentation.

[0106] In this embodiment, the emotional plot generation module, through the collaboration of multiple units, brings significant advantages to the game: the plot semantic network construction unit establishes the underlying logical framework, ensuring a clear and orderly plot flow; the emotional plot decision engine dynamically generates plots based on player emotions and game progress, enhancing the resonance between the plot and the player's emotions; the dialogue generation and emotion adaptation unit creates personalized interactive dialogues, enhancing the player's sense of immersion; the plot continuity guarantee mechanism avoids plot interruptions and provides a smooth narrative experience; and the dynamic resource scheduling submodule optimizes resource allocation to ensure a smooth and uninterrupted plot presentation. These units complement each other, making the game plot both logically rigorous and emotionally interactive, significantly enhancing the player's gaming experience.

[0107] Example 3

[0108] Scenario simulation

[0109] System Setup: On a gaming PC running Windows 10, we installed the Qt 5.15 development environment and the TensorFlow 2.8 deep learning framework, connected a Logitech C920 HD camera and a HyperX Cloud Alpha high-fidelity microphone as multimodal data acquisition devices.

[0110] Multimodal data acquisition: When a player starts playing, the camera captures facial expression image data at a rate of 30 frames per second with a resolution of 640×480 pixels. The microphone collects voice data at a sampling rate of 44.1kHz and 16-bit precision. Image data is obtained through the DirectShow interface, and voice data is obtained through the Windows WAS API interface.

[0111] AI emotion recognition processing: The collected facial expression image data undergoes preprocessing, including face detection and cropping, histogram equalization, and resizing to 224×224 pixels. The preprocessed images are then fed into a convolutional neural network based on the ResNet-50 architecture. This network was pre-trained on the FER2013 dataset and fine-tuned on a custom dataset of game player expressions. After endpoint detection and noise reduction, the voice data is processed to extract Mel-Frequency Cepstral Coefficient (MFCC) features. These features are then fed into an LSTM-based voice emotion recognition model. The outputs of the two models are combined using a weighted fusion algorithm to generate the final emotion probability vector.

[0112] Game scenes and command acquisition: During the game development process, the QtQuick3D framework is used to create game scenes. The QInputEvent event system is used to obtain the player's keyboard and mouse input commands in real time. Game scene information is obtained through a custom scene manager class, including scene lighting intensity (obtained through Qt3D's QDirectionalLight component), NPC number and distribution (obtained through the game object manager), and task progress (obtained through the task system).

[0113] Action decision model construction: Build a finite state machine based on QStateMachine and define 20 basic states, such as standby, walking, running, attacking, and defending. The state transition condition is determined by the Cartesian product of the emotion probability vector and the game scene parameters. For example, when the anger value is greater than 0.7, the light intensity is less than 30 lux, and the number of NPCs is ≥3, the state transitions from the standby state to the attack preparation state. Each state is associated with 3-5 action tuples, which include the skeleton animation ID, the playback speed coefficient (0.8-1.2), the expression mapping table, and the sound effect ID.

[0114] Action response execution: The action tuple generated by the action decision module is transmitted to the action execution module. This module loads the corresponding skeletal animation resources based on the Qt3DAnimation framework, adjusts the animation playback rate according to the playback speed coefficient, and realizes facial expressions through vertex blending technology. The position of facial control points is adjusted according to the expression mapping table; the physical effects are calculated by the Bullet physics engine to ensure the physical authenticity of the action; in multiplayer game scenarios, the action data is synchronized to the server and other clients at a frequency of 60Hz via the UDP protocol.

[0115] Example 4

[0116] Scenario simulation:

[0117] In open world games, players have main quest dialogues with key NPCs:

[0118] Data collection and identification

[0119] The camera captured the player's frown (distance between eyebrows <45 pixels) and downward turn of the corners of his mouth (angle <-15°). The AI ​​determined the "anger" intensity to be 0.82, and with an increase in voice tone (fundamental frequency from 120Hz to 200Hz), the overall judgment was that the player was in a "strong anger" state.

[0120] Dynamic plot adjustments

[0121] Line generation: Based on the player's emotions, the NPC switches from the "regular dialogue tree" to the "conflict dialogue branch," generating lines like: "You think yelling like this will solve the problem? Calm down, then we can continue talking!" The lines contain provocative keywords ("yelling") and soothing intentions ("calm down"), matching the player's anger while guiding the interaction direction;

[0122] Scene feedback: The Qt framework triggers the scene system to load the "flame effect" (particle generation rate +200%), the ambient light color temperature is reduced to 3000K (warm tones enhance the sense of oppression), and the NPC action switches to a defensive posture (skeleton animation ID = 0x1B7, hands crossed in front of the chest).

[0123] Player interaction and plot progression

[0124] If the player continues to express anger (for >10 seconds), the plot triggers "combat mode", the NPC summons monsters and switches attack actions, and the UI displays a "hatred value: MAX" progress bar;

[0125] If the player's emotions calm down (the "anger" intensity drops below 0.4), the NPC's lines become gentle: "Sorry, I was too impulsive just now... Let's talk again." The "cooperative puzzle solving" branch is unlocked, rewarding the player with additional experience points.

[0126] In summary, the Qt interactive game character action response control method and system that integrates AI emotion recognition, through AI emotion recognition technology, enables the game character to perceive the player's emotional state and make emotional interaction actions that match it, enhancing the emotional resonance between the game character and the player, and greatly improving the interactive experience and immersion of the game; the action decision model comprehensively considers multiple factors and can generate diversified action response strategies for the game character according to different game scenarios, player operations and emotional states, meeting the personalized needs of players and making the game character's action performance more flexible and rich.

[0127] In addition, by leveraging the powerful functions of the Qt framework, we can efficiently obtain game scene information, accurately capture player operation instructions, and smoothly render and display game character actions, ensuring the stability and efficiency of the entire system; at the same time, we make full use of multimodal data, combining player facial expressions, voice intonation and other information with traditional operation instructions, providing a more comprehensive and rich basis for the game character's action decision-making, making the game character's action response more natural and reasonable.

[0128] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0129] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A Qt interactive game character action response control method integrating AI emotion recognition, characterized by: The following steps are involved: S1: Use sensors such as cameras and microphones on gaming devices to collect multimodal data such as players' facial expressions, voice intonation, and body movements; S2: The collected multimodal data is transmitted to the AI ​​emotion recognition module, which uses a deep learning algorithm to analyze and process the data to identify the player's current emotional state; S3: Obtain the current game scene information and other operation instructions input by the player through the Qt framework; S4: Build an action decision model that comprehensively considers the player's emotional state, game scene information, and operation instructions, and generates appropriate action response strategies for the game character through preset rules and machine learning algorithms; S5: Based on the action response strategy generated by the action decision model, the game character is controlled to perform corresponding actions through Qt's graphics rendering and animation system, and displayed on the game screen.

2. The Qt interactive game character action response control method integrated with AI emotion recognition according to claim 1 is characterized in that: The deep learning algorithm includes a facial expression recognition model based on a convolutional neural network and a speech emotion recognition model based on a long short-term memory network, wherein the input layer of the facial expression recognition model receives preprocessed facial image data, and the output layer is a probability distribution vector containing at least seven basic emotion classifications; the input layer of the speech emotion recognition model receives a Mel-frequency cepstral coefficient feature vector, and the output layer is an emotion probability distribution vector of the same dimension as the facial expression recognition model.

3. The Qt interactive game character action response control method integrated with AI emotion recognition according to claim 1 is characterized in that: The emotional text semantic analysis model is constructed, and the BERT pre-trained language model is used to perform emotional polarity analysis on the player's voice commands, extracting emotional keywords (such as "hate" and "excited") in the text and their intensity values; The attention mechanism is used to calculate the semantic relevance between emotional keywords and facial micro-expressions and voice intonation features to generate a multimodal emotional consistency score. When the consistency score between the text sentiment polarity and non-verbal signals is less than 0.6, the sentiment verification mechanism is triggered, and secondary confirmation is performed through NPC dialogue questions (such as "Do you really mean this?").

4. The Qt interactive game character action response control method integrated with AI emotion recognition according to claim 1 is characterized in that: The step S4 of generating an action response strategy further includes: Building an emotion-plot mapping matrix Where n is the dimension of the emotional state vector, m is the number of preset plot branches, and the matrix element M_ij represents the weight value of emotional state i triggering plot branch j; Dynamically update the matrix weights through the reinforcement learning algorithm, and calculate the cumulative reward value R_t = ∑γ^t·r_t based on the player's historical emotional feedback data (γ is the discount factor, r_t is the emotional reward at time t); When a sudden change in the player's emotional state is detected (such as Δemotion_intensity>0.5), Kalman filtering is used to predict the optimal response path of the plot branch, so that the NPC lines generated are consistent with the continuity of emotional changes.

5. A Qt interactive game character action response control system integrating AI emotion recognition, applied to the Qt interactive game character action response control method integrating AI emotion recognition described in claims 1-4, characterized in that: include: Data collection module: used to collect multimodal data such as players' facial expressions, voice intonation, body movements, etc. The AI ​​emotion recognition module receives data transmitted by the multimodal data acquisition module, uses deep learning algorithms to perform emotion recognition processing, and outputs the player's emotional state information; The game scene and instruction acquisition module, based on the Qt framework, obtains the current game scene information and the operation instructions input by the player in real time; The action decision module is used to build an action decision model and generate the action response strategy of the game character based on the player's emotional state, game scene information and operation instructions; The action execution module is used to control the game character to perform corresponding actions based on the action response strategy generated by the action decision module through Qt's graphics rendering and animation system, and display them on the game screen; Emotional plot generation module: used to dynamically generate plot content and interactive dialogues that conform to emotional logic based on the player's emotional state and game history data.

6. The Qt interactive game character action response control system integrated with AI emotion recognition according to claim 5, characterized in that: The AI ​​emotion recognition module includes: An image preprocessing unit for performing image cropping, histogram equalization, and normalization operations; Feature extraction unit, including a pre-trained ResNet-50 convolutional neural network; Sentiment classification unit, which includes a fully connected layer and a Softmax activation function and outputs a 7-dimensional sentiment probability vector; The fusion decision unit adopts weighted average algorithm to fuse the facial expression and speech emotion recognition results.

7. The Qt interactive game character action response control system integrated with AI emotion recognition according to claim 5, characterized in that: The action decision module includes: A scene analysis unit, used to analyze game scene information and generate feature vectors; Rule engine unit, storing at least 100 state transition rules; State machine management unit, which implements the creation, monitoring and state transfer of finite state machines based on QStateMachine; The action generation unit selects a matching action tuple from the action library according to the current state.

8. The Qt interactive game character action response control system integrated with AI emotion recognition according to claim 5, characterized in that: The action execution module includes: Animation control unit, which implements skeletal animation playback and mixing based on the Qt3DAnimation framework; Expression rendering unit, which realizes facial expression animation based on vertex blending technology; The physics simulation unit calculates the physical effects of actions based on the Bullet physics engine; the network synchronization unit uses a client-server architecture to achieve multi-player action synchronization with a synchronization frequency of no less than 60Hz.

9. The Qt interactive game character action response control system integrated with AI emotion recognition according to claim 5, characterized in that: The system further includes a data quality assessment module, which includes: a signal feature extraction circuit configured to extract time domain features, frequency domain features, and statistical features of the multimodal data; The quality assessment model uses the support vector machine algorithm to classify and evaluate data quality, and outputs a three-dimensional quality vector including clarity, completeness, and consistency; A data compensation unit triggers a data interpolation algorithm or a resampling algorithm to perform data compensation when any dimension in the quality vector is lower than a preset threshold; The feedback control circuit is configured to dynamically adjust the sampling parameters of the data acquisition module according to the quality assessment result, including but not limited to the frame rate, exposure time and gain value.

Images