Human-computer interaction method and system based on natural language large model

Abstract

The invention relates to the technical field of information processing, in particular to a man-machine interaction method and system based on a natural language large model, and the method comprises the steps: collecting and preprocessing multi-modal input data, and obtaining multi-modal fusion features; analyzing the initial intention of the user by adopting a time sequence semantic model and capturing a semantic evolution trajectory, capturing the initial emotional state of the user through a multi-mode emotional time sequence model and generating an emotional evolution trajectory, and synthesizing to obtain double-track evolution information; calculating a three-dimensional dynamic weight through a cross-scene three-dimensional linkage feedback adaptation method to obtain a scene adaptation weight set; an emotional language double-drive enhancement mechanism is adopted to obtain accurate intention emotion core information; the personalized knowledge and the general knowledge of the user are adapted through a knowledge adaptation closed-loop mechanism, a customized knowledge adaptation result is obtained, and a multi-modal personalized response is generated in combination with the scene adaptation weight set and the precise intention emotion core information. According to the scheme, through emotional language double-drive enhancement optimization, the accuracy of user intention and emotion recognition is improved.

Description

Technical Field

[0001] This invention relates to the field of information processing technology, and in particular to a human-computer interaction method and system based on a large natural language model. Background Technology

[0002] In diverse scenarios such as office collaboration, smart homes, online education, and industrial assistance, human-computer interaction driven by large natural language models has become the core link connecting people and smart devices. The naturalness, accuracy, and adaptability of this interaction directly affect user experience and task execution efficiency, representing a core upgrade requirement in the field of artificial intelligence. Currently, this field faces multiple challenges, including difficulties in multimodal data fusion, weak intent-emotion collaboration, poor cross-scenario adaptability, insufficient personalization of knowledge services, and low adaptability of response generation, which hinder the intelligent upgrade of interaction technologies.

[0003] Traditional solutions separate intent recognition from sentiment analysis, ignoring their dynamic evolution and strong correlation. This results in semantic ambiguity resolution lacking emotional context support, and sentiment recognition being susceptible to semantic scene biases, making it difficult to capture users' true needs and states. Furthermore, multimodal data fusion employs a fixed-weight strategy, where low-quality data can interfere with the extraction of core intent and sentiment-related information, weakening the foundation of recognition. In addition, insufficient cross-scene adaptation and user personalization mean that static rules cannot accommodate the differences in expression across different scenarios and users, further reducing recognition accuracy. Therefore, current human-computer interaction methods fail to meet practical needs in terms of the accuracy of user intent and sentiment recognition. Summary of the Invention

[0004] This invention enhances and optimizes through a dual-drive approach of emotion and language, achieving bidirectional collaborative correction of emotion and semantics, thereby improving the accuracy of user intent and emotion recognition.

[0005] The technical solution proposed in this invention is: a human-computer interaction method based on a large natural language model, the method comprising: Multimodal input data is collected and preprocessed to obtain multimodal fusion features; Based on multimodal fusion features, a temporal semantic model is used to parse the user's initial intent and capture the semantic evolution trajectory. A multimodal emotional temporal model is used to capture the user's initial emotional state and generate the emotional evolution trajectory, thus obtaining dual-track evolution information. By combining dual-track evolution information with user knowledge graphs, interaction capability profiles, and real-time feedback data, and calculating three-dimensional dynamic weights through a cross-scene three-dimensional linkage feedback adaptation method, a scene adaptation weight set is obtained. Based on dual-track evolutionary information and scene-adaptive weight set, an emotion-language dual-drive enhancement mechanism is adopted, combined with multimodal confidence calibration processing, to complete the bidirectional optimization of emotion and semantics, and obtain accurate intention emotion core information; Based on the core information of precise intent and emotion, a knowledge adaptation closed-loop mechanism is used to adapt user-specific knowledge and general knowledge to obtain customized knowledge adaptation results. Combined with the scene adaptation weight set and the core information of precise intent and emotion, a multimodal personalized response is generated.

[0006] Preferably, the acquisition and preprocessing process of the multimodal input data is as follows: Collect at least one multimodal input data from text, voice, image, and video, and use a differentiated and standardized preprocessing strategy adapted to the characteristics of each modality to complete data noise reduction and format regularization; The core representation features of each modality are obtained through feature extraction technology, and the dimensionality differences between modalities are eliminated by feature dimension unification technology. By integrating the core features of each modality based on the multimodal feature fusion architecture, a multimodal fusion feature with both information integrity and consistency is obtained.

[0007] Preferably, the acquisition and preprocessing process of the multimodal input data is as follows: Collect at least one multimodal input data from text, voice, image, and video, and use a differentiated and standardized preprocessing strategy adapted to the characteristics of each modality to complete data noise reduction and format regularization; The core representation features of each modality are obtained through feature extraction technology, and the dimensionality differences between modalities are eliminated by feature dimension unification technology. By integrating the core features of each modality based on the multimodal feature fusion architecture, a multimodal fusion feature with both information integrity and consistency is obtained.

[0008] Preferably, the initial emotional state capture and emotional evolution trajectory generation process is as follows: Emotion-related features are extracted from multimodal fusion features and input into a multimodal feature fusion and temporal correlation architecture; By fusing multimodal features and temporal correlation architecture to integrate emotion-related features, the system captures the user's initial emotional state and generates emotional state information corresponding to each moment. Based on the temporal correlation analysis of emotional state information at different times, an emotional evolution trajectory is generated, which is linked with the semantic evolution trajectory in a temporal manner to form a dual-track evolution information.

[0009] Preferably, the process of obtaining the scene adaptation weight set is as follows: Based on the intent characteristics, user knowledge graph matching characteristics, and user interaction capability characteristics in the dual-track evolution information, the three-dimensional initial weights of intent priority, knowledge adaptability, and interaction complexity are calculated. The initial 3D weights are dynamically calibrated by combining real-time feedback data to optimize the accuracy of the weight parameters. Extract common features across scenarios, use feature transfer technology to enable cross-scenario reuse of adaptation logic, and output a scenario adaptation weight set.

[0010] Preferably, the bidirectional optimization process of emotion and semantics is as follows: The dual-drive enhancement mechanism of emotion and language is activated, which uses emotion evolution information to help resolve semantic ambiguity, while using semantic evolution information to correct emotion recognition bias. A multimodal data credibility assessment model is constructed to dynamically quantify and assess the credibility of each modal input data and determine the confidence weight. By leveraging confidence weights to enhance the decision-making role of high-confidence data, suppressing interference from low-confidence data, and outputting accurate core information on intent and sentiment, we can achieve the desired results.

[0011] Preferably, the process for obtaining the customized knowledge adaptation result is as follows: By using knowledge balance decision-making logic, the ratio of personalized knowledge to general knowledge is dynamically adjusted to achieve the adaptation and matching of the two types of knowledge. Extract new knowledge elements from user interaction history, update the user's personalized knowledge graph, and realize the dynamic evolution of personalized knowledge; Based on the memory decay model, it periodically detects highly important knowledge that has not been used for a long time, and achieves forgetting compensation through scenario-triggered wake-up or lightweight review push, outputting customized knowledge adaptation results.

[0012] Preferably, the process for generating the multimodal personalized response is as follows: Based on the core parameters of the scenario adaptation weight set, the professionalism level of the response content and the complexity of the interaction steps are determined to ensure that the response process is accurately adapted to the user's interaction capabilities. Adjust the tone and emotional resonance of the response based on the core information of the user's precise intent and emotion to match the user's emotional state; With customized knowledge adaptation results as the core support, the core information content of the response is filled in, ensuring the accuracy and personalization of the response information.

[0013] The present invention also provides a human-computer interaction system based on a large natural language model, the system being used to execute the aforementioned human-computer interaction method based on a large natural language model.

[0014] The present invention also provides a computer-readable storage medium storing a computer program, which is executed by a processor to implement the aforementioned human-computer interaction method based on a large natural language model.

[0015] The beneficial effects of this invention are: 1. By collaborating with a temporal semantic model and a multimodal sentiment temporal model, the system simultaneously captures the evolutionary trajectories of user intent and sentiment, forming dual-track evolutionary information. This completely solves the drawback of traditional technologies that separate intent recognition and sentiment analysis. This innovation not only achieves temporal linkage representation of intent and sentiment but also tracks their dynamic changes in real time, providing a complete user state basis for subsequent interaction decisions. It effectively avoids misjudgments of needs due to the lack of information in a single dimension, significantly improving the comprehensiveness and real-time nature of human-computer interaction's perception of users' dynamic needs, making the interaction more closely aligned with the evolution of user needs and emotional fluctuations during the interaction process.

[0016] 2. Employing a dual-drive enhancement mechanism of emotion and language, this system utilizes a two-way collaborative logic—using emotional evolution information to assist in resolving semantic ambiguity and semantic evolution information to correct emotional recognition biases—combined with multimodal confidence calibration processing, to accurately solve the recognition bias problem caused by the isolation of semantics and emotion in traditional interactions. This innovation fully leverages the complementary roles of emotion and semantics, effectively suppressing interference from low-confidence modal data, significantly improving the accuracy of intent and emotion recognition. This enables the system to accurately understand the true needs behind ambiguous expressions and identify the deeper states beneath surface emotions, resulting in more natural and resonant human-computer interaction and significantly reducing interaction efficiency losses caused by misunderstandings of needs.

[0017] 3. By employing a personalized knowledge closed-loop management mechanism comprised of knowledge-balanced decision-making logic, dynamic evolution and updates, and forgetting compensation, it overcomes the limitations of traditional technologies that rely on fixed knowledge retrieval ratios and lack full lifecycle management. This innovative approach dynamically adjusts the ratio of personalized to general knowledge retrieval based on the user's knowledge level and interaction needs, promptly incorporating newly added user knowledge and reactivating long-unused important knowledge, ensuring that knowledge services always precisely match the user's dynamic needs. Simultaneously, by adapting the depth of knowledge presentation to different user knowledge backgrounds, it effectively solves the problem of one-size-fits-all knowledge delivery, significantly improving the personalization and practicality of knowledge services, making the response content more aligned with the user's cognitive level, and helping users efficiently obtain the information they need. Attached Figure Description

[0018] Figure 1 A flowchart of a human-computer interaction method based on a large natural language model; Figure 2 This is a flowchart illustrating the implementation process of a human-computer interaction method based on a large natural language model. Detailed Implementation

[0019] The following description is intended to disclose the present invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art. The basic principles of the invention defined in the following description can be applied to other embodiments, modifications, improvements, equivalents, and other technical solutions that do not depart from the spirit and scope of the invention.

[0020] It is understood that the term "a" should be understood as "at least one" or "one or more," that is, in one embodiment, the number of an element can be one, while in another embodiment, the number of the element can be multiple, and the term "a" should not be understood as a limitation on the number.

[0021] like Figure 1 and Figure 2 As shown, a multimodal input processing module collects and preprocesses multimodal data to obtain multimodal fusion features; based on a temporal semantic model and a multimodal emotional temporal model, it analyzes intent and captures emotions to generate dual-track evolutionary information; combining user knowledge graphs, interaction ability profiles, and real-time feedback data, it calculates three-dimensional dynamic weights through a cross-scene three-dimensional linkage feedback adaptation method; it optimizes and obtains accurate core information of intent and emotion using an emotion-language dual-drive enhancement mechanism; and it adapts personalized knowledge and general knowledge through a knowledge adaptation closed-loop mechanism to obtain customized knowledge adaptation results, ultimately generating a multimodal personalized response. Through multimodal data processing, dual-track evolutionary information construction, cross-scene weight adaptation, bidirectional optimization of emotional semantics, knowledge closed-loop adaptation, and personalized response generation, a full-process intelligent adaptation natural language large-scale model human-computer interaction technology system is constructed. Information; combining user knowledge graphs, interaction capability profiles, and real-time feedback data, three-dimensional dynamic weights are calculated through a cross-scenario three-dimensional linkage feedback adaptation method; an emotion-language dual-drive enhancement mechanism is adopted to optimize and obtain accurate intention and emotion core information; a knowledge-adaptation closed-loop mechanism is used to adapt personalized knowledge and general knowledge, obtain customized knowledge adaptation results, and finally generate multimodal personalized responses.

[0022] Furthermore, multimodal input data is collected and preprocessed, as detailed below: Collect at least one multimodal input data from text, speech, image, and video. Adopt a differentiated standardization preprocessing strategy adapted to the characteristics of each modality to complete data noise reduction and format regularization. Obtain the core representation features of each modality through feature extraction technology and eliminate the dimensional differences between modalities with the help of feature dimension unification technology. Integrate the core features of each modality based on the multimodal feature fusion architecture to obtain multimodal fusion features that have both information integrity and consistency.

[0023] The text data preprocessing employed word segmentation, stop word removal, and part-of-speech tagging techniques. Speech data underwent noise reduction using an adaptive noise suppression algorithm, and effective speech segments were extracted through endpoint detection, along with acoustic features such as intonation, speech rate, and energy. Image / video data used object detection algorithms to locate key regions, and 68 feature points were extracted using an expression recognition model to generate emotion-related visual features. Feature dimensions were uniformly standardized using Min-Max (the formula is: standardized value = (original value - minimum value of the modality feature) / (maximum value of the modality feature - minimum value of the modality feature)), mapping each modality feature to the [0,1] interval. The multimodal feature fusion architecture was based on a multi-head attention mechanism and a Transformer encoder layer, dynamically allocating the weights of each modality feature. After fusion, the feature dimensions were controlled between 512 and 2048 dimensions, a range determined based on the feature dimension sensitivity analysis of 100,000 sets of multimodal interaction samples, adapting to the input requirements of subsequent models.

[0024] In detail, the specific implementation logic of this step is as follows: The multimodal input processing module synchronously acquires data through multiple interfaces (text interface supports UTF-8 encoding, voice interface supports 16kHz sampling rate and 16-bit precision, image / video interface supports 1080P resolution and 30 frames / second). A hardware clock synchronization module (using PTPv2 protocol, time synchronization accuracy ≤10μs) achieves millisecond-level timestamp alignment of the multimodal data. The text interface transmits data via UART serial port (115200 baud rate), the voice interface via I2S bus, and the image / video interface via MIPICSI-2 interface. The acquisition trigger signal is uniformly generated by the FPGA (trigger delay ≤5μs). Data collection undergoes format validation and outlier removal (using the 3σ criterion, with outlier data ≤1%); text data is processed by word segmentation tools, and then word embedding technology (based on Word2Vec or BERT) is used to generate text feature vectors; speech data is denoised and converted to text, and Mel-frequency cepstral coefficients (MFCC) and fundamental frequency features are extracted simultaneously; image / video data undergoes object detection, and the extracted visual features are dimensionality reduced (using principal component analysis PCA); finally, an attention fusion network is used to weightedly fuse text, speech, and visual features to generate multimodal fusion features. The entire processing time from acquisition to fusion is ≤500ms.

[0025] Specifically, the equipment and process details for data acquisition and preprocessing are as follows: The acquisition device supports multimodal synchronous input and has a built-in low-power microprocessor (clock frequency ≥ 1GHz), integrating a text parsing unit, a speech processing unit, and an image processing unit. The process is divided into: 1. Device initialization: Start the multimodal acquisition unit and complete parameter configuration (sampling rate, resolution, feature dimension, etc.); 2. Data acquisition: Trigger corresponding modal acquisition according to user interaction scenarios, supporting single-modal or multimodal collaborative acquisition; 3. Preprocessing: Perform noise reduction, format normalization, and other operations on a modal basis. Text data additionally performs syntax correction, speech data additionally performs emotional acoustic feature extraction, and image / video data additionally performs blur detection and enhancement; 4. Feature extraction and fusion: Extract core features according to modal characteristics, input them into the fusion architecture after dimension unification, output multimodal fused features and temporarily store them (supports caching ≥ 1000 sets of feature data, storage latency ≤ 10ms); 5. Quality verification: Perform signal-to-noise ratio (SNR ≥ 25dB) and integrity verification on the fused features. Unqualified features are retried for acquisition (retry times ≤ 3 times). The equipment operates at a voltage of 3.3-5V, with a static power consumption of ≤15mA, a dynamic power consumption of ≤50mA, and an operating temperature range of -10℃. 55℃, relative humidity 20% 85% of usage scenarios.

[0026] Furthermore, the system analyzes the user's initial intent and captures the semantic evolution trajectory, as detailed below: The core language features in the multimodal fusion features are deeply correlated with the conversation history time-series data and input into a time-series modeling architecture built on a pre-trained large language model. The semantic parsing function of the time-series modeling architecture accurately identifies the user's initial intent and generates semantic state representation information corresponding to different interaction moments. The evolutionary rules of semantic state at each moment are explored using time-series correlation analysis technology, and the semantic evolution trajectory is fully captured to form structured intent time-series data. Relying on the time-series modeling capability, the full-link representation of intent from initial to dynamic evolution is realized.

[0027] The pre-trained large language model is based on the Transformer architecture with ≥1 billion parameters and has been fine-tuned with domain data (fine-tuning data volume ≥1 million interaction samples). The core language features include text semantic features, speech-to-text features, and contextual features, with feature dimensions consistent with multimodal fusion features. The conversation history time-series data is stored in the format of "timestamp-semantic content-intent label", supporting backtracking of ≥10 rounds of dialogue history, with a time-series association error ≤10ms. The time-series modeling architecture includes a semantic parsing layer, a time-series association layer, and a trajectory generation layer. The semantic parsing layer outputs intent labels (covering ≥100 common intents), the time-series association layer uses a Long Short-Term Memory (LSTM) network or a gated recurrent unit (GRU) to model time-series relationships, and the trajectory generation layer outputs structured intent time-series data (JSON format, including intent labels, semantic state vectors, and evolutionary confidence at each time point).

[0028] In detail, the specific implementation logic of this step is as follows: First, core language features are separated from the multimodal fusion features and concatenated with historical conversation time-series data (aligned by timestamp, alignment error ≤ 5ms) to form a time-series input sequence. This input sequence is then fed into the time-series modeling architecture. The semantic parsing layer uses a fine-tuned version of a pre-trained large language model, focusing on key semantic information based on an attention mechanism, and outputs initial intent labels and confidence scores (confidence score ≥ 0.8 indicates a valid intent). The time-series association layer models the semantic state at each interaction moment, generating semantic state vectors (512 dimensions). By analyzing the cosine similarity between vectors (similarity threshold ≥ 0.7), evolutionary patterns are mined to identify evolutionary types such as intent continuation, transformation, or escalation. The trajectory generation layer integrates intent labels, semantic state vectors, and evolutionary types at each moment in chronological order to form structured intent time-series data. The capture delay of the evolutionary trajectory is ≤ 200ms to ensure real-time tracking of user intent changes.

[0029] Specifically, the deployment and optimization details of the intent parsing and trajectory capture models are as follows: The temporal modeling architecture is deployed on edge computing devices or cloud servers. For edge deployment, model quantization (INT8 quantization, quantization error ≤3%) is used. The quantization toolchain employs TensorRT 8.6 combined with ONNX Runtime for model conversion and optimization, with inference time ≤300ms. For cloud deployment, inference time ≤100ms. The model training process is as follows: 1. Data preparation: Collect multi-scenario interaction data, label intent tags and evolutionary relationships, and divide the training, validation, and test sets in a 7:2:1 ratio; 2. Model initialization: Build the temporal modeling architecture based on a pre-trained large language model, initializing parameters using X... 3. Training and Optimization: The AdamW optimizer is used, with an initial learning rate of 0.0001 and ≥100 iterations. If the intent recognition accuracy on the validation set does not improve for 5 consecutive iterations, the learning rate is halved until the accuracy on the test set is ≥96%. 4. Deployment and Updates: After deployment, the model supports online incremental learning (iterative optimization is performed once every ≥10,000 new data points), and updates are pushed via OTA (size ≤20MB). 5. Performance Evaluation: The core metrics are intent recognition accuracy, evolutionary trajectory capture accuracy (≥94%), and inference latency. Model parameters are calibrated regularly (calibration cycle ≤1 month).

[0030] Furthermore, the system captures the user's initial emotional state and generates an emotional evolution trajectory, as detailed below: Emotion-related features are extracted from multimodal fusion features and input into a multimodal feature fusion and temporal correlation architecture. Emotion-related features are fused through multimodal feature fusion and temporal correlation architecture to capture the user's initial emotional state and generate emotional state information corresponding to each moment. Based on the temporal correlation analysis of emotional state information at different moments, an emotional evolution trajectory is generated and forms a temporal linkage with the semantic evolution trajectory to constitute the dual-track evolution information.

[0031] Among them, emotion-related features include text emotion word features, speech tone features (fundamental frequency, speech rate, energy fluctuation), and visual expression features (facial feature point displacement, expression intensity), with each feature having ≥64 dimensions; the multimodal feature fusion and temporal association architecture includes a feature alignment layer, a fusion layer, and a temporal modeling layer. The feature alignment layer synchronizes the emotion features of each modality according to the timestamp (alignment error ≤5ms), the fusion layer adopts a weighted summation fusion strategy (weights are dynamically allocated based on modality confidence, with an allocation error ≤2%), and the temporal modeling layer uses a bidirectional LSTM network to capture the temporal dependencies of emotion states; the initial emotion states are divided into pleasant, neutral, dissatisfied, anxious, and irritable, etc.; the emotion state information includes emotion labels, emotion intensity (0-10 points, accuracy 0.1 points), and confidence (≥0.7 is valid), with a temporal linkage error with the semantic evolution trajectory ≤10ms.

[0032] In detail, the specific implementation logic of this step is as follows: Emotion-related features are separated from multimodal fusion features and input into a multimodal feature fusion and temporal association architecture. The feature alignment layer eliminates temporal biases of features across modalities, ensuring synchronous processing of emotion features at the same time. The fusion layer dynamically assigns weights based on the emotion recognition accuracy of each modality (text modality weight 0.3-0.5, speech modality weight 0.3-0.4, visual modality weight 0.2-0.3), and weighted fusion yields a unified emotion feature vector. The temporal modeling layer performs temporal analysis on the emotion feature vector, outputting initial emotion state labels and intensities. As the interaction progresses, the emotion state information at each moment is updated in real time. By calculating the difference in emotion intensity between adjacent moments and the similarity of emotion labels, the evolution direction (strengthening, weakening, transformation) is determined, generating a complete emotion evolution trajectory. Finally, the emotion evolution trajectory and the semantic evolution trajectory are aligned by timestamps to form dual-track evolution information, where the capture delay of the emotion evolution trajectory is ≤250ms, meeting the requirements for collaborative interaction between emotion and semantics.

[0033] Specifically, the equipment and process details for emotion state capture and trajectory generation are as follows: The core processing unit uses a low-power AI chip (computing power ≥ 0.5 TOPS), integrating an emotion feature extraction module and a time-series analysis module. The process is divided into: 1. Feature separation: Extracting emotion-related features from multimodal fusion features and performing validity verification (valid feature ratio ≥ 98%); 2. Feature alignment and fusion: Synchronizing features from each modality, dynamically allocating fusion weights, and outputting a unified emotion feature vector; 3. Initial emotion recognition: Based on the fused feature vector, outputting the initial emotion state and confidence level through an emotion classification model; 4. Time-series analysis: Associating historical emotion state information, calculating evolution parameters, and generating an emotion evolution trajectory; 5. Dual-track linkage: Aligning with the semantic evolution trajectory by timestamp, constructing and storing dual-track evolution information (supporting caching ≥ 500 sets of trajectory data); 6. Quality assessment: When the emotion recognition confidence level < 0.7, re-extracting features and optimizing the fusion weights (retry times ≤ 2). The device supports collaborative work with the semantic processing unit, data transmission uses an SPI interface (clock frequency ≥ 1MHz), transmission latency ≤ 5ms, and overall power consumption ≤ 40mA.

[0034] Furthermore, the 3D dynamic weights are calculated and cross-scene adaptation is achieved, as detailed below: Based on the intent characteristics, user knowledge graph matching characteristics, and user interaction capability characteristics in the dual-track evolution information, three initial weights of intent priority, knowledge adaptability, and interaction complexity are calculated. The three initial weights are dynamically calibrated by combining real-time feedback data to optimize the accuracy of the weight parameters. Cross-scene common features are extracted, and the cross-scene reuse of adaptation logic is realized through feature transfer technology, outputting the scene adaptation weight set.

[0035] The intent characteristics include intent complexity (simple / medium / complex, weighted at 0.4) and urgency (low / medium / high, weighted at 0.6). Intent priority is calculated as a weighted sum of these two factors (range 0-10, precision 0.1). The intent complexity coefficient is determined by weighting the number of interaction steps required for the intent (≤2 steps for simple, 3-5 steps for medium, ≥6 steps for complex) and the knowledge dimensions involved (≤2 for simple, 3-4 for medium, ≥5 for complex) (each weighted at 50%). Knowledge fit is based on the matching degree between the user's knowledge graph and the knowledge related to the current intent (matching degree ≥80% for high fit, 60%-80% for medium fit, <60% for low fit), quantified with a value of 0-10. The knowledge proficiency coefficient is based on the percentage of correct applications of knowledge in this domain in the user's historical interactions (number of correct applications / total number of related interactions), linearly mapped to a value in the range of 0.6-1.0. Interaction complexity is based on... User interaction ability profiles (input proficiency, error correction frequency, and operation efficiency) are quantified, with values ​​ranging from 0 to 10. Real-time feedback data includes user error correction behavior, operation dwell time (>5 seconds for low efficiency, 2-5 seconds for medium efficiency, and <2 seconds for high efficiency), and satisfaction rating (1-5 points). The calibration coefficient is dynamically adjusted according to the feedback type (adjustment range ±0.5). The weight allocation of the feedback calibration coefficient (each accounting for 1 / 3) has been verified by 5000 sets of user interaction data. This weight allocation improves the matching degree between the calibrated weight and the user's actual needs by ≥15%, which is significantly better than other weight combinations (such as 4:3:3, 5:3:2). Cross-scenario common features include intent type and interaction scenario attributes (office / life / study, etc.). The feature transfer technology is based on Domain-Adversarial Neural Networks (DANN), and the cross-scenario adaptation latency is ≤300ms.

[0036] In detail, the specific implementation logic of this step is as follows: First, the intent characteristics in the dual-track evolution information are analyzed. Knowledge fit is calculated using the user knowledge graph, and interaction complexity is quantified based on the user's interaction capability profile. A weighted algorithm (intent priority weight 0.4, knowledge fit weight 0.3, interaction complexity weight 0.3) is used to obtain the three-dimensional initial weights. Specifically, the initial intent priority calculation uses the intent complexity coefficient and the demand urgency coefficient as core factors. The intent complexity coefficient is assigned values ​​of 0.3, 0.6, and 1 for simple, medium, and complex levels, respectively, while the demand urgency coefficient is assigned values ​​of 0.2, 0.6, and 1 for low, medium, and high levels, respectively. The two are calculated using a weighted sum, with each weighting the others at 40%. Sixty percent; the initial knowledge fit is calculated by dividing the number of matching terms between the user's knowledge graph and the current intent-related knowledge by the total number of related terms, and then multiplying by the knowledge proficiency coefficient (0.6, 0.8, and 1 for low, medium, and high respectively); the initial interaction complexity is calculated by using the input proficiency coefficient (the lower the error correction frequency, the higher the coefficient, ranging from 0.5 to 1) and the operation efficiency coefficient (the shorter the operation dwell time, the higher the coefficient, ranging from 0.5 to 1) as factors, and then multiplying by two after weighted summation; the three-dimensional initial weight synthesis is to weight and sum the intent priority, knowledge fit, and interaction complexity according to the weight ratios of 40%, 30%, and 30% respectively. Then, real-time feedback data is introduced to construct a dynamic calibration model. This model collects user error correction behavior coefficients (no correction = 1, single correction = 0.8, multiple corrections = 0.5), operation dwell time coefficients (less than two seconds = 1, two to five seconds = 0.7, greater than five seconds = 0.4), and satisfaction rating coefficients (converted to a 0-1 range on a 5-point scale). These three coefficients are weighted and summed to obtain the feedback calibration coefficient (each weighted at one-third). Finally, the calibrated weight equals the initial comprehensive weight multiplied by (one plus the feedback calibration coefficient minus 0.5), ensuring that the weight adjustment range remains within the initial value of 0.5 to 1. Between 0.5 and 0.5, the initial weights are iteratively corrected (once per round of interaction, with a correction error ≤0.3); then, the core features of the current scene are extracted and compared with the historical scene feature library to discover common features. The adaptation logic of the historical scene is transferred to the current scene through a domain adaptation algorithm to achieve adaptation logic reuse; finally, the calibrated 3D weights and cross-scene migration results are integrated to output a scene adaptation weight set (including the specific values ​​of each weight and adaptation strategy labels). The weight set update cycle is synchronized with the interaction rounds to ensure that the adaptation strategy is in real time in line with user needs and scene characteristics.

[0037] Specifically, the model and process details for weight calculation and cross-scenario adaptation are as follows: The core model employs a multi-dimensional weight calculation network and a feature transfer model working in tandem, deployed at the edge or in the cloud. The process is divided into: 1. Parameter initialization: Loading user knowledge graphs, interaction capability profiles, and scene feature libraries to initialize weight calculation parameters and transfer model parameters; 2. Initial weight calculation: Parsing input characteristic data, calculating three-dimensional initial weights according to preset formulas, and outputting a draft weight; 3. Dynamic calibration: Collecting real-time feedback data, calculating calibration coefficients, correcting initial weights, and outputting calibrated weights; 4. Cross-scene adaptation: Extracting current scene features, matching historical common features, and adjusting weight adaptation logic through feature transfer technology; 5. Weight set generation: Integrating calibration weights and adaptation logic to generate and store scene-adapted weight sets, supporting weight set backtracking ≥100 sets; 6. Adaptation verification: Verifying the effectiveness of the weight set through interaction effect evaluation (task completion rate ≥95% is considered qualified), and recalibrating if unqualified (calibration times ≤2 times). The training data for the feature transfer model covers ≥20 common scenarios, with ≥10,000 samples for each scenario, and the model generalization error is ≤4%; the overall weight calculation time is ≤200ms, meeting the requirements for real-time interaction.

[0038] Furthermore, bidirectional optimization of sentiment and semantics, as well as confidence calibration, are achieved, as detailed below: A dual-drive enhancement mechanism for emotional and semantic information is initiated (a joint optimization mechanism for resolving semantic ambiguity and correcting emotional recognition bias through bidirectional collaborative input of emotional and semantic evolution information, combined with multimodal confidence calibration). This mechanism utilizes emotional evolution information to assist in resolving semantic ambiguity while simultaneously using semantic evolution information to correct emotional recognition bias. A multimodal data credibility assessment model is constructed to dynamically quantify and evaluate the credibility of each modality's input data and determine the confidence weight. Based on the confidence weight, the decision-making role of high-credibility data is strengthened, interference from low-credibility data is suppressed, and accurate core information of intent and emotion is output.

[0039] The bidirectional optimization logic of the emotion-language dual-drive enhancement mechanism is as follows: Emotional evolution information is injected into the semantic parsing process to provide emotional context support for ambiguous semantics (e.g., "okay" combined with "annoyed" is judged as negative semantics), and a joint probability model (calculating the co-occurrence probability of emotion and semantics) is used to achieve emotion-assisted semantic disambiguation; Semantic evolution information is input into the emotion analysis process in reverse to correct the bias of single-modal emotion recognition (e.g., "really happy" combined with "failed the exam" is judged as irony), and an attention-enhanced graph neural network (using semantic entities as nodes to construct an emotion association graph) is used to achieve semantic correction of emotion recognition; The multimodal data credibility evaluation model quantifies credibility (values ​​from 0 to 1, precision 0) based on the recognition accuracy, data integrity, and noise level of each modality. 01), where the recognition accuracy = number of correctly recognized samples in this modality / total number of samples, the data integrity coefficient = actual number of effective features / standard number of features (512 dimensions for text modality, 256 dimensions for speech, and 1024 dimensions for vision), the noise suppression coefficient = (noise-free data feature entropy - actual data feature entropy) / noise-free data feature entropy, the confidence baseline for text modality ≥ 0.85, speech modality ≥ 0.8, and visual modality ≥ 0.75; the confidence weight is calculated based on confidence normalization, the sum of the weights is 1, and the weight ratio of high confidence modality (confidence ≥ 0.8) is ≥ 0.6; the core information of accurate intent emotion includes optimized intent tags, emotion tags, semantic details and emotion intensity, with information integrity ≥ 98% and accuracy ≥ 95%.

[0040] In detail, the specific implementation logic of this step is as follows: First, the dual-drive emotion-language enhancement mechanism is activated, injecting the emotion feature vectors from the emotion evolution trajectory into the semantic parsing module to help distinguish ambiguous semantics (e.g., "okay" is a positive semantic under "pleasure" but a negative semantic under "dissatisfaction"). Simultaneously, the contextual semantic information from the semantic evolution trajectory is input into the emotion analysis module to correct single-modal emotion recognition errors (e.g., speech recognition is "happy" but the semantic display is "project failed," corrected to "depressed"). During this process, the dual-drive emotion-language enhancement optimization coefficient quantifies the bidirectional optimization effect, with the emotion-assisted semantic optimization coefficient based on the emotion evolution trajectory. The stability coefficient (average sentiment similarity at adjacent time points, ranging from 0.5 to 1) and the sentiment-semantic correlation coefficient (the degree of matching between sentiment labels and semantic intent, ranging from 0.3 to 1) are used as factors, weighted summed and multiplied by 0.3 to adjust the confidence of the semantic parsing results; the semantic correction sentiment optimization coefficient is used as factors, weighted summed and multiplied by 0.3, based on the semantic evolution trajectory integrity coefficient (semantic information coverage, ranging from 0.6 to 1) and the semantic-sentiment conflict coefficient (the degree of contradiction between semantic intent and sentiment labels, ranging from 0 to 0.5) to correct the intensity value of the sentiment recognition results. Then, a multimodal data credibility assessment model is constructed. The recognition accuracy, data integrity coefficient (number of effective features divided by number of standard features), and noise suppression coefficient (similarity between ideal noise-free data features and actual data features) of each modality are input. These are weighted and summed at 50%, 30%, and 20% respectively to obtain the single-modal credibility (value range 0 to 1). The credibility values ​​of each modality are then summed to obtain the total credibility. The confidence weight of a single modality is equal to its credibility value divided by the total credibility. The sum of all modal weights is one. For modalities with a credibility lower than 0.7, the weight is multiplied by a decay coefficient of 0.8 to further suppress interference from low-credibility data. The optimization results of sentiment and semantics based on the confidence weights are weighted and fused to strengthen the influence of high-credibility modal data. Finally, the optimized intent and sentiment information are integrated to form accurate intent and sentiment core information. The optimization latency of this process is ≤250ms to ensure the real-time performance and accuracy of the interaction. Information is input into the sentiment analysis module to correct single-modal sentiment recognition errors (e.g., speech recognition is "happy" but semantics are different). Specifically, the model and process details for bidirectional optimization and confidence calibration are as follows: The core model includes a bidirectional optimization network and a credibility assessment model, deployed in the AI ​​processing unit (computing power ≥ 1 TOPS). The process is divided into: 1. Mechanism activation: Receive dual-track evolution information and scene-adaptive weight set, and activate the emotion-language dual-drive enhancement mechanism; 2. Bidirectional optimization: Perform emotion-assisted semantic parsing and semantic correction emotion recognition, quantify the optimization effect through optimization coefficients, and output preliminary optimization results; 3. Credibility assessment: Collect data quality parameters for each modality, input them into the assessment model, and obtain modality credibility values; 4. Weight calculation and fusion: Calculate confidence weights based on credibility values, weighted fuse the preliminary optimization results, and obtain the final optimization results; 5. Core information generation: Integrate the optimized intent and emotion information, and generate accurate intent and emotion core information in the format of "intent-emotion-semantic details-intensity-confidence"; 6. Verification output: Verify the accuracy of the core information (accuracy ≥ 95% is acceptable), and if unacceptable, re-execute the optimization process (retry ≤ 2 times). The model training uses ≥500,000 interaction samples labeled with sentiment-semantic conflicts; the evaluation error of the credibility assessment model is ≤3%, ensuring the rationality of weight allocation.

[0041] Furthermore, customized knowledge adaptation results are obtained, as detailed below: By using knowledge balance decision-making logic, the ratio of personalized knowledge to general knowledge is dynamically adjusted to achieve the adaptation and matching of the two types of knowledge; new knowledge elements are extracted from user interaction history to update the user's personalized knowledge graph, realizing the dynamic evolution of personalized knowledge; based on the memory decay model, highly important knowledge that has not been used for a long time is regularly tested, and forgetting compensation is achieved through scenario-triggered wake-up or lightweight review push, outputting customized knowledge adaptation results.

[0042] The knowledge-balanced decision-making logic dynamically allocates the proportion of two types of knowledge (personalized knowledge 0.3-0.7 and general knowledge 0.3-0.7) based on precise intent and sentiment core information and the user's knowledge level, with a matching error ≤5%. Personalized knowledge includes exclusive knowledge accumulated from user preferences, professional experience, and historical interactions, while general knowledge includes real-time updated public knowledge, cutting-edge information in the field, and policy hotspots (update cycle ≤24 hours). New knowledge elements extracted through knowledge evolution employ keyword extraction (accuracy ≥92%) and relationship mining techniques. The personalized knowledge graph is stored in "entity-relationship-attribute" triples, supporting incremental updates (update latency ≤10ms). The system dynamically allocates the proportion of the two types of knowledge (personalized knowledge 0.3-0.7) based on the user's knowledge level and information. -0.7, general knowledge accounts for 0.3-0.7), matching error ≤5%; personalized knowledge includes user preferences, professional experience, and exclusive knowledge accumulated from historical interactions; general knowledge includes real-time updated public knowledge, cutting-edge information in the field, policy hotspots, etc. Conflict handling adopts a knowledge confidence ranking mechanism: confidence of newly extracted knowledge = source reliability coefficient × content consistency coefficient. Source reliability is assigned a value of 0.6-1.0 according to the authority of the interaction scenario. Content consistency coefficient = similarity with historical knowledge. If the confidence is ≥0.8, it covers historical knowledge; if it is 0.5-0.8, it is marked as conflicting knowledge and the user is prompted for confirmation; if it is <0.5, it is discarded. The memory decay model is an improved Ebbinghaus model (introducing an exponential decay function of knowledge importance weight: memory retention rate = knowledge importance weight × ,in This is the duration of unused time. The decay constant is set to 30 days. Knowledge importance weights are set (use frequency ≥10 times is high importance, weight 0.7-1.0; use frequency 3-10 times is medium importance, weight 0.3-0.7; use frequency <3 times is low importance, weight 0-0.3). Long-term inactivity of ≥30 days triggers forgetting compensation, with a compensation success rate of ≥90%. Customized knowledge adaptation results include knowledge content, adaptation score (0-10 points) and presentation format suggestions. Adaptation score ≥8 points is considered excellent adaptation.

[0043] In detail, the specific implementation logic of this step is as follows: First, based on the intent type and user knowledge graph in the core information of precise intent and emotion, the knowledge balancing decision logic is initiated to analyze the demand ratio of personalized knowledge and general knowledge in the current interaction scenario and dynamically adjust the calling weights. Then, new knowledge elements (such as new preferences, professional knowledge, etc.) are extracted from the user's current and historical interaction data, and the triple information of the personalized knowledge graph is updated through relationship mining technology to ensure the timeliness of personalized knowledge. Next, based on the memory decay model, the personalized knowledge graph is scanned regularly to detect highly important and long-term unused knowledge and trigger the forgetting compensation mechanism (scenario-triggered wake-up is to actively associate the knowledge in relevant interaction scenarios, and lightweight review push is a non-intrusive short message reminder). Finally, the matched personalized knowledge and general knowledge are integrated, and the presentation depth of knowledge is adjusted according to the user's knowledge level and interaction scenario (in-depth content is pushed to professional users, and basic interpretation is pushed to novice users) to generate customized knowledge adaptation results. The knowledge adaptation latency of this process is ≤300ms.

[0044] Specifically, the process and storage details of knowledge adaptation, evolution, and forgetting compensation are as follows: Knowledge storage adopts a distributed architecture. Personalized knowledge is stored locally (encrypted, AES-128 encryption), while general knowledge is stored in a cloud-based knowledge base (supporting distributed retrieval, retrieval latency ≤50ms). The process is divided into: 1. Knowledge retrieval: Personalized and general knowledge are retrieved according to the balance decision logic, supporting real-time retrieval and cached retrieval (caching ≥500 high-frequency knowledge items); 2. Knowledge evolution: Real-time monitoring of interaction data, extraction of new knowledge elements, incremental updates to the personalized knowledge graph, and consistency verification after updates (verification pass rate ≥99%); 3. Forgetting compensation: Periodic detection according to the memory decay model, generation of a compensation list, and execution of compensation triggered by scenarios or pushed on a timed basis; 4. Adaptation result generation: Integration of the two types of knowledge, adjustment of presentation format and depth, and output of customized knowledge adaptation results; 5. Update optimization: Optimization of knowledge balance ratio and evolution strategy based on user feedback (knowledge usefulness score ≥4 is valid), iterating once every ≥1000 feedback data points. Personalized knowledge graphs can store ≥10,000 triples, while general knowledge bases can be updated daily with ≥10,000 new data entries, achieving a knowledge retrieval accuracy of ≥97%.

[0045] Furthermore, a multimodal personalized response is generated, as detailed below: Based on the core parameters of the scenario-adaptive weight set, the professionalism level of the response content and the complexity of the interaction steps are determined to ensure that the response process is accurately adapted to the user's interaction capabilities; the tone and style of the response and the degree of emotional resonance are adjusted based on the core information of precise intent and emotion to match the user's emotional state; and the core information content of the response is filled in with customized knowledge adaptation results as the core support to ensure the accuracy and personalization of the response information.

[0046] The professionalism level is divided into three levels: basic, intermediate, and professional, corresponding to low, medium, and high levels of user knowledge, with the usage rate of professional terminology being ≤20%, 20%-50%, and ≥50%, respectively. The complexity of interaction steps is determined based on the complexity of the intent and the user's interaction ability: simple intents require ≤2 steps, medium intents require ≤2 steps, and so on. Figure 2 -5 steps for complex intentions, 5-8 steps for complex intentions, with a step breakdown accuracy of ≥95%; tone style includes ≥6 categories such as friendly, professional, reassuring, and concise; emotional resonance level is adjusted according to the user's emotional intensity (≥8 points for enhanced resonance, 5-7 points for moderate resonance, and ≤4 points for neutrality); accuracy of responding to core information content is ≥98%, and personalization is ≥90% (customized based on user preferences, knowledge level, etc.); multimodal output format supports flexible switching between text, voice, and image / video, with a switching latency of ≤100ms; the naturalness of voice response synthesis is ≥95%, and the clarity of image / video response is ≥720P.

[0047] In detail, the specific implementation logic of this step is as follows: First, analyze the intent priority, knowledge compatibility, and interaction complexity parameters of the scene adaptation weight set to determine the corresponding professionalism level (knowledge compatibility ≥8 points for professional level, 5-7 points for advanced level, <5 points for basic level) and interaction step complexity (interaction complexity ≥8 points is broken down into ≤3 steps, 5-7 points into 3-5 steps, <5 points into 5-8 steps). Then, based on the emotional tags and intensity of the core information of precise intent emotion, match the corresponding tone style (pleasant emotions match friendly tone, anxious / irritable emotions match soothing tone, professional scenarios match professional tone), and adjust the emotional resonance language (e.g., adding soothing tone). The system includes phrases like "We understand your anxiety, we'll investigate step by step." Then, based on customized knowledge adaptation results, it fills in the core information of the response, adjusting the content depth according to professional level (basic level adds terminology explanations, professional level adds technical details), and breaking down the response process according to the complexity of the interaction steps. Finally, based on user interaction preferences and scenario characteristics, it selects the optimal multimodal output format (text response is suitable for complex information, voice response is suitable for convenient scenarios, and image / video response is suitable for visualization needs), generating a multimodal personalized response. The total response generation time is ≤300ms, meeting real-time interaction requirements.

[0048] Specifically, the device and process details for generating the response are as follows: The response generation module integrates text generation, speech synthesis, and image / video rendering units, supporting multimodal output interfaces. The process is divided into: 1. Parameter parsing: Receiving scene adaptation weight sets, precise intent and emotional core information, and customized knowledge adaptation results, and parsing key parameters; 2. Response planning: Determining the professionalism level, interaction steps, and tone style, and generating a response framework; 3. Content filling: Filling core content based on customized knowledge adaptation results, supplementing emotional resonance language and step-by-step guidance; 4. Multimodal adaptation: Selecting the output format according to user preferences (stored in user profiles) and scene characteristics; performing syntax optimization for text responses (optimization accuracy ≥96%), performing emotional speech synthesis for speech responses (synthesizing speech rate and intonation adapted to emotional state), and performing content rendering and compression for image / video responses (compressed size ≤5MB); 5. Quality verification: Verifying the accuracy, fluency (speech naturalness ≥95%), and adaptation of the response content; regenerating unqualified responses (retry times ≤2); 6. Output push: Pushing the response to the user's interactive terminal and recording the response log (including output format, user feedback, etc.). The device supports offline response generation (caching ≥500 sets of commonly used response templates), online response generation latency ≤300ms, offline response latency ≤100ms, and operating power consumption ≤45mA.

[0049] In addition, the system also has the functions of response iteration optimization, user preference learning, and scene adaptation: The response iteration optimization adjusts response parameters based on user feedback (clicks, duration, follow-up questions, etc.), and optimizes the response strategy once every ≥500 feedback data points; user preference learning updates user preference tags for response format, content depth, and tone style by analyzing historical interaction data, with a learning accuracy of ≥93%; scene adaptation supports automatic recognition of office, life, and study scenarios, and switches the corresponding response strategy (e.g., prioritizing professional text responses in office scenarios and prioritizing friendly voice responses in life scenarios).

[0050] The system features privacy protection, with sensitive information such as user interaction data and personalized knowledge graphs stored using AES-256 encryption and transmitted using the TLS1.3 encryption protocol. It also has device self-test and fault indication functions, performing automatic self-tests for ≤1 minute daily and providing text and sound prompts and troubleshooting suggestions when a fault is detected. Furthermore, it has a low-power optimization mechanism, entering low-power mode (power consumption ≤8mA) after 3 minutes of inactivity and waking up within ≤50ms when interaction is detected.

[0051] The processes described above with reference to the flowcharts in the embodiments disclosed in this invention can be implemented as computer software programs. The embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), it performs the functions defined in the methods of this application. It should be noted that the computer-readable medium described above in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wire segments, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless segments, wire segments, optical fibers, RF, etc., or any suitable combination thereof.

[0052] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0053] Those skilled in the art should understand that the embodiments of the present invention described above and shown in the accompanying drawings are merely examples and do not limit the present invention. The purpose of the present invention has been fully and effectively achieved. The functions and structural principles of the present invention have been shown and explained in the embodiments. Without departing from the stated principles, the implementation of the present invention may have any variations or modifications.

Claims

1. A human-computer interaction method based on a large natural language model, characterized in that, The method includes: Multimodal input data is collected and preprocessed to obtain multimodal fusion features; Based on multimodal fusion features, a temporal semantic model is used to parse the user's initial intent and capture the semantic evolution trajectory. A multimodal emotional temporal model is used to capture the user's initial emotional state and generate the emotional evolution trajectory, thus obtaining dual-track evolution information. By combining dual-track evolution information with user knowledge graphs, interaction capability profiles, and real-time feedback data, and calculating three-dimensional dynamic weights through a cross-scene three-dimensional linkage feedback adaptation method, a scene adaptation weight set is obtained. Based on dual-track evolutionary information and scene-adaptive weight set, an emotion-language dual-drive enhancement mechanism is adopted, combined with multimodal confidence calibration processing, to complete the bidirectional optimization of emotion and semantics, and obtain accurate intention emotion core information; Based on the core information of precise intent and emotion, a knowledge adaptation closed-loop mechanism is used to adapt user-specific knowledge and general knowledge to obtain customized knowledge adaptation results. Combined with the scene adaptation weight set and the core information of precise intent and emotion, a multimodal personalized response is generated.

2. The human-computer interaction method based on a large natural language model according to claim 1, characterized in that, The acquisition and preprocessing process of the multimodal input data is as follows: Collect at least one multimodal input data from text, voice, image, and video, and use a differentiated and standardized preprocessing strategy adapted to the characteristics of each modality to complete data noise reduction and format regularization; The core representation features of each modality are obtained through feature extraction technology, and the dimensionality differences between modalities are eliminated by feature dimension unification technology. By integrating the core features of each modality based on the multimodal feature fusion architecture, a multimodal fusion feature with both information integrity and consistency is obtained.

3. The human-computer interaction method based on a large natural language model according to claim 2, characterized in that, The initial intent parsing and semantic evolution trajectory capture process is as follows: The core language features in the multimodal fusion features are deeply correlated with the conversation history time series data and input into the time series modeling architecture built on a pre-trained large language model; The semantic parsing function of the temporal modeling architecture accurately identifies the user's initial intent and generates semantic state representation information corresponding to different interaction moments; By utilizing temporal correlation analysis technology, we can uncover the evolutionary patterns of semantic states at each moment, fully capture the semantic evolution trajectory, and form structured temporal data of intent. Relying on temporal modeling capabilities, we can achieve full-link representation of intent from its initial to dynamic evolution.

4. The human-computer interaction method based on a large natural language model according to claim 3, characterized in that, The process of capturing the initial emotional state and generating the emotional evolution trajectory is as follows: Emotion-related features are extracted from multimodal fusion features and input into a multimodal feature fusion and temporal correlation architecture; By fusing multimodal features and temporal correlation architecture to integrate emotion-related features, the system captures the user's initial emotional state and generates emotional state information corresponding to each moment. Based on the temporal correlation analysis of emotional state information at different times, an emotional evolution trajectory is generated, which is linked with the semantic evolution trajectory in a temporal manner to form a dual-track evolution information.

5. The human-computer interaction method based on a large natural language model according to claim 4, characterized in that, The process of obtaining the scene adaptation weight set is as follows: Based on the intent characteristics, user knowledge graph matching characteristics, and user interaction capability characteristics in the dual-track evolution information, the three-dimensional initial weights of intent priority, knowledge adaptability, and interaction complexity are calculated. The initial 3D weights are dynamically calibrated by combining real-time feedback data to optimize the accuracy of the weight parameters. Extract common features across scenarios, use feature transfer technology to enable cross-scenario reuse of adaptation logic, and output a scenario adaptation weight set.

6. The human-computer interaction method based on a large natural language model according to claim 5, characterized in that, The bidirectional optimization process of emotion and semantics is as follows: The dual-drive enhancement mechanism of emotion and language is activated, which uses emotion evolution information to help resolve semantic ambiguity, while using semantic evolution information to correct emotion recognition bias. A multimodal data credibility assessment model is constructed to dynamically quantify and assess the credibility of each modal input data and determine the confidence weight. By leveraging confidence weights to enhance the decision-making role of high-confidence data, suppressing interference from low-confidence data, and outputting accurate core information on intent and sentiment, we can achieve the desired results.

7. The human-computer interaction method based on a large natural language model according to claim 6, characterized in that, The process of obtaining the customized knowledge adaptation results is as follows: By using knowledge balance decision-making logic, the ratio of personalized knowledge to general knowledge is dynamically adjusted to achieve the adaptation and matching of the two types of knowledge. Extract new knowledge elements from user interaction history, update the user's personalized knowledge graph, and realize the dynamic evolution of personalized knowledge; Based on the memory decay model, it periodically detects highly important knowledge that has not been used for a long time, and achieves forgetting compensation through scenario-triggered wake-up or lightweight review push, outputting customized knowledge adaptation results.

8. The human-computer interaction method based on a large natural language model according to claim 7, characterized in that, The process of generating the multimodal personalized response is as follows: Based on the core parameters of the scenario adaptation weight set, the professionalism level of the response content and the complexity of the interaction steps are determined to ensure that the response process is accurately adapted to the user's interaction capabilities. Adjust the tone and emotional resonance of the response based on the core information of the user's precise intent and emotion to match the user's emotional state; With customized knowledge adaptation results as the core support, the core information content of the response is filled in, ensuring the accuracy and personalization of the response information.

9. A human-computer interaction system based on a large natural language model, characterized in that, The system is used to execute the human-computer interaction method based on a large natural language model as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that is executed by a processor to implement the human-computer interaction method based on a large natural language model as described in any one of claims 1-8.

Images