Rapid intention insight and intelligent response method and system for intelligent equipment

Abstract

The invention relates to the technical field of intelligent equipment interaction, in particular to an intelligent equipment intention rapid insight and intelligent response method, which comprises the following steps: acquiring multi-modal interaction data input into intelligent equipment by a user, and generating a standardized intention input vector; performing hierarchical feature extraction on the standardized intention input vector, and outputting a fusion intention feature matrix; inputting the fused intention feature matrix into an intention classifier to obtain initial intention probability distribution data of the user; dynamically correcting the initial intention probability distribution data based on the historical intention record to generate a final intention judgment label; the optimal response action set is matched from the intention response strategy library on the basis of the final intention judgment label, the characteristic representation capacity is improved through multi-modal fusion, intention continuity understanding is enhanced through dynamic context perception, response self-adaptive optimization is achieved through reinforcement learning, and therefore the intention recognition accuracy can be improved in the noise environment, and the intention recognition accuracy is improved. And response delay is reduced.

Description

Technical Field

[0001] The present invention relates to the field of smart device interaction technology, and in particular to a method and system for rapid intention insight and intelligent response of a smart device. Background Art

[0002] In the field of smart device interaction, as users' demands for device intelligence and convenience continue to increase, accurately understanding user intent and making intelligent responses have become key requirements. However, existing smart devices have many shortcomings in processing multimodal interaction data, identifying user intent, and optimizing response strategies.

[0003] Existing smart device intent recognition mostly relies on single-modal data (such as voice or text only), resulting in low accuracy in intent judgment in complex scenarios; and lacks a dynamic context fusion mechanism, making it impossible to optimize current decisions based on historical interactions. Summary of the Invention

[0004] The main purpose of the present invention is to provide a method and system for rapid insight into the intentions of smart devices and intelligent response, aiming to solve the technical problems raised in the above background technology.

[0005] The present invention proposes a method for rapid insight into the intention of a smart device and intelligent response, comprising:

[0006] Acquire multimodal interaction data input by a user into a smart device, the multimodal interaction data including voice information, text information, visual scene image information, and sensor timing information, and normalize the multimodal interaction data to generate a standardized intent input vector;

[0007] Based on a preset semantic feature extraction model, hierarchical feature extraction is performed on the standardized intent input vector, and a fusion intent feature matrix is output;

[0008] Inputting the fused intention feature matrix into the intention classifier to obtain the user's initial intention probability distribution data;

[0009] Obtaining historical intent records from the context memory of the smart device, dynamically revising the initial intent probability distribution data based on the historical intent records, and generating a final intent determination label;

[0010] Building an intention response strategy library, and matching an optimal response action set from the intention response strategy library based on the final intention determination label;

[0011] The optimal response action set is optimized in real time through a reinforcement learning model to generate a dynamic response instruction set and control the execution of the device;

[0012] Collect user feedback satisfaction data and device execution efficiency indicators, update the semantic feature extraction model and intent response strategy library based on the user feedback satisfaction data and device execution efficiency indicators, and perform intent response optimization.

[0013] The present invention is further configured such that the step of normalizing the multimodal interaction data to generate a standardized intention input vector includes:

[0014] Performing frame and window processing on the speech information, extracting Mel-frequency cepstral coefficients and fundamental frequency features, and generating a speech intention feature vector;

[0015] Perform word segmentation and semantic embedding on the text information, and generate a high-dimensional text semantic vector through a pre-trained language model;

[0016] Positioning key areas of the visual scene image information, extracting scene feature maps and compressing them into image feature vectors through global intent pooling;

[0017] Performing sliding window segmentation on the environmental sensor time series information, calculating statistical features and frequency domain energy distribution within the window, and generating an environmental perception feature vector;

[0018] The speech intention feature vector, the high-dimensional text semantic vector, the image feature vector and the environmental perception feature vector are mapped to a unified dimensional space to generate a standardized intention input vector.

[0019] The present invention is further configured such that the step of inputting the fused intention feature matrix into the intention classifier to obtain the user's initial intention probability distribution data includes:

[0020] Construct a multi-task intent loss function, which includes intent classification cross entropy loss and intent feature distribution regularization loss;

[0021] Based on the multi-task intent loss function, injecting Gaussian noise into the fused intent feature matrix to generate an adversarial intent feature matrix, and updating the parameters of the intent classifier through gradient back propagation;

[0022] Based on the multi-task intent loss function, a curriculum learning strategy is adopted to perform phased training according to the difficulty of the intent features in the fusion intent feature matrix, giving priority to learning the intent features with high confidence and gradually introducing the fuzzy intent features;

[0023] Using a focused loss function to dynamically adjust the weight of each of the intent features in the intent classification cross entropy loss, to obtain a trained intent classifier;

[0024] The fused intent feature matrix is input into the trained intent classifier for linear transformation, and the Softmax function is applied to the result after linear transformation to obtain the initial probability distribution data of the user intent.

[0025] The present invention is further configured such that the step of dynamically correcting the initial intention probability distribution data based on the historical intention record to generate a final intention determination label includes:

[0026] Extracting time decay factor and historical intention feature vector based on historical intention records;

[0027] Extracting a current intention feature vector from the fused intention feature matrix, and calculating the cosine similarity between the current intention feature vector and the historical intention feature vector;

[0028] Obtaining a dynamic correction coefficient of intention probability according to the time decay factor and the cosine similarity;

[0029] The initial probability distribution data of the user intention is corrected according to the dynamic correction coefficient of the intention probability, and the label corresponding to the maximum probability value is selected as the final intention determination label.

[0030] The present invention is further configured such that the step of constructing an intention response strategy library and matching an optimal response action set from the intention response strategy library based on the final intention determination tag comprises:

[0031] Define the response action set, including action type, parameter configuration and execution priority;

[0032] The knowledge graph is used to build an intent response strategy library, which includes intent judgment label nodes, target state nodes, action transfer relationship edges, edge weights, and response action sets. The edge weights are calculated based on the historical execution success rate.

[0033] Use the shortest path algorithm to search and generate several candidate response action sets from the current intention label node;

[0034] The execution efficiency, user rating and resource cost of the candidate response action sets are evaluated by a multi-objective utility function, and multiple optimal response action sets are screened.

[0035] The present invention is further configured such that the step of optimizing the optimal response action set in real time by using a reinforcement learning model, generating a dynamic response instruction set, and controlling the execution of the device comprises:

[0036] The state space is defined as the fusion intention feature matrix and the final intention determination label, and the action space is defined as the candidate response action set;

[0037] Design a composite reward function that includes user feedback score, resource saving rate, and execution delay penalty;

[0038] Based on the state space and the action space, an Actor-Critic network is trained using a deep deterministic policy gradient algorithm, a dynamic response instruction set is generated by the Actor network, and an action value is evaluated by the Critic network;

[0039] Store historical interaction data in the experience replay pool, and update the Actor-Critic network parameters according to the temporal difference error priority sampling data;

[0040] The device is controlled to execute according to the dynamic response instruction set.

[0041] The present invention is further configured such that the step of updating the semantic feature extraction model and the intention response strategy library according to the user feedback satisfaction data and the device execution performance index comprises:

[0042] Collect user feedback scores, execution latency, and accuracy metrics to build a model and update the dataset;

[0043] Calculate the model parameter update gradient by adding elastic weight constraints;

[0044] Update the semantic feature extraction model based on the model update dataset and the model parameter update gradient;

[0045] Incrementally updating the intent response strategy library based on the semantic feature extraction model, and re-encoding the graph structure relationship using a graph neural network when adding a new node;

[0046] Regularly verify the intent recognition accuracy of the optimized model.

[0047] The present invention also discloses a system for rapid insight into the intentions of smart devices and intelligent response, comprising:

[0048] A multimodal data acquisition module is used to obtain multimodal interaction data input by users into smart devices and normalize the multimodal interaction data to generate a standardized intent input vector;

[0049] A semantic feature extraction module, configured to perform hierarchical feature extraction on the standardized intent input vector based on a preset semantic feature extraction model, and output a fused intent feature matrix;

[0050] An intention classification module is used to input the fused intention feature matrix into an intention classifier to obtain the user's initial intention probability distribution data;

[0051] An intent determination module is used to obtain historical intent records in the context memory of the smart device, and dynamically modify the initial intent probability distribution data based on the historical intent records to generate a final intent determination label;

[0052] An intent response matching module is used to build an intent response strategy library and match an optimal response action set from the intent response strategy library based on the final intent determination label;

[0053] A reinforcement learning control module is used to optimize the optimal response action set in real time through a reinforcement learning model, generate a dynamic response instruction set, and control the execution of the device;

[0054] The closed-loop feedback module is used to collect user feedback satisfaction data and device execution efficiency indicators, and update the semantic feature extraction model and intent response strategy library based on the user feedback satisfaction data and device execution efficiency indicators to optimize the intent response.

[0055] The present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the steps of a method for rapid insight into the intention of an intelligent device and intelligent response.

[0056] The present invention also discloses a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of a method for rapid insight into the intentions of an intelligent device and intelligent response are implemented.

[0057] The beneficial effects of the present invention are as follows: the present invention forms a closed-loop learning system through multimodal data normalization, multi-level feature fusion, dynamic weight adjustment and reinforcement learning optimization response strategy, wherein multimodal fusion improves feature representation capabilities, dynamic context perception enhances understanding of intent continuity, and reinforcement learning realizes response adaptive optimization, thereby improving the accuracy of intent recognition in a noisy environment and reducing response delay. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is a schematic diagram of a method flow chart according to an embodiment of the present application.

[0059] Figure 2 This is a schematic diagram of the system structure of an embodiment of the present application.

[0060] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0061] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0062] like Figure 1 As shown, the present application provides a method for rapid insight into the intention of a smart device and intelligent response, which is characterized by including:

[0063] S1, obtaining multimodal interaction data input by a user into a smart device, the multimodal interaction data including voice information, text information, visual scene image information, and sensor timing information, and normalizing the multimodal interaction data to generate a standardized intent input vector;

[0064] S2, based on a preset semantic feature extraction model, performs hierarchical feature extraction on the standardized intent input vector and outputs a fused intent feature matrix; the semantic feature extraction model includes a convolutional neural network, a long short-term memory network, and an attention mechanism module, wherein the convolutional neural network is used to capture the local spatial features of the standardized intent input vector, the long short-term memory network is used to process the temporal dependency features of the local spatial features, and the attention mechanism module is used to calculate the weight of each local spatial feature based on the temporal dependency features, screen out a subset of spatial features, and finally output the fused intent feature matrix;

[0065] S3, inputting the fused intent feature matrix into the intent classifier to obtain the user's initial intent probability distribution data;

[0066] S4, obtaining historical intent records from the context memory of the smart device, dynamically revising the initial intent probability distribution data based on the historical intent records, and generating a final intent determination label;

[0067] S5, building an intention response strategy library, and matching an optimal response action set from the intention response strategy library based on the final intention determination label;

[0068] S6, optimizing the optimal response action set in real time through a reinforcement learning model, generating a dynamic response instruction set and controlling the device to execute it;

[0069] S7, collecting user feedback satisfaction data and device execution efficiency indicators, updating the semantic feature extraction model and intent response strategy library based on the user feedback satisfaction data and device execution efficiency indicators, and performing intent response optimization.

[0070] As described in the above steps S1-S7, in the field of smart device interaction, as user needs become increasingly diverse and complex, traditional single-modal interaction methods are difficult to fully and accurately understand user intentions. Although multimodal interaction is adopted, there are problems such as poor multimodal data fusion processing, low accuracy of intention recognition, insufficient intelligence and flexibility of response strategies, and untimely model optimization. In response to the above problems, the present invention proposes a technical solution for rapid insight into the intentions of smart devices and intelligent response. By obtaining multimodal interaction data such as user input, voice, text, visual scene images, and sensor timing, and normalizing them to generate standardized intention input vectors, it can provide basic data for accurate intention analysis and improve data availability. At the same time, by fusing the intention feature matrix, layered extraction can deeply mine the intentions of smart devices. By mining semantic features at different levels and fusing multiple features, we can more comprehensively represent intentions, obtain more representative intention features, and improve the accuracy of intention recognition. The fused intention feature matrix is input into the intention classifier to obtain the probability distribution data of the user's initial intention, which can preliminarily determine the possibility of various user intentions and provide a probability reference for subsequent intention judgment. Combined with historical intention records, the final intention judgment label is generated, which can utilize past experience and user habits and preferences to more accurately judge intentions and reduce misjudgments. The intention response strategy library integrates multiple response strategies and can quickly find appropriate actions based on the final intent label matching to achieve intelligent response. It also uses the reinforcement learning model to optimize the optimal response action set in real time, generate a dynamic response instruction set, and control device execution. The reinforcement learning model can learn and optimize according to real-time situations, making response actions more adaptable to actual needs and improving the flexibility and effectiveness of device responses. Finally, user feedback satisfaction data and device execution performance indicators are collected, and the semantic feature extraction model and intent response strategy library are updated accordingly to optimize intent responses. By continuously collecting feedback and indicator data, problems can be discovered in a timely manner and the model and strategy library can be optimized, which can continuously improve system performance and user experience. Through the above series of steps, the present invention realizes the effective processing and utilization of multimodal data, improves the accuracy of intent recognition and the intelligence and flexibility of response, and timely optimizes system performance, thereby solving many problems existing in multimodal interaction.

[0071] In one embodiment of the present invention, the step of normalizing the multimodal interaction data to generate a standardized intention input vector includes:

[0072] S11, performing frame and window processing on the speech information, extracting Mel-frequency cepstral coefficients and fundamental frequency features, and generating a speech intention feature vector;

[0073] S12, performing word segmentation and semantic embedding on the text information, and generating a high-dimensional text semantic vector through a pre-trained language model;

[0074] S13, locating key areas of the visual scene image information, extracting a scene feature map, and compressing it into an image feature vector through global intent pooling;

[0075] S14, performing sliding window segmentation on the environmental sensor time series information, calculating statistical features and frequency domain energy distribution within the window, and generating an environmental perception feature vector;

[0076] S15, mapping the speech intention feature vector, the high-dimensional text semantic vector, the image feature vector and the environment perception feature vector to a unified dimensional space to generate a standardized intention input vector.

[0077] As described in steps S11-S15 above, in the processing of multimodal interactive data on smart devices, different modal data have their own formats, characteristics, and dimensions, leading to difficulties in data fusion, insufficient feature extraction, and ineffective unified processing. For example, the temporal nature of voice data, the semantic complexity of text data, the spatial characteristics of image data, and the dynamic variability of sensor data make it difficult to directly integrate and utilize this data, hindering the smart device's accurate understanding and response to user intent. To address these issues, the present invention proposes a technical solution for normalizing multimodal interactive data processing: voice information is framed and windowed, Mel-frequency cepstral coefficients and fundamental frequency features are extracted, and a voice intent feature vector is generated. Framed and windowed processing divides continuous voice signals into small segments for easier analysis. Mel-frequency cepstral coefficients simulate the characteristics of human hearing, and fundamental frequency features reflect the prosodic information of speech. This processing method effectively captures the acoustic and prosodic features related to intent in the voice signal, thereby obtaining a feature vector that accurately represents the voice intent and addressing the issue of insufficient feature extraction from voice data. Furthermore, text information is segmented and semantically embedded, generating a high-dimensional text semantic vector using a pre-trained language model. Word segmentation divides text into meaningful lexical units, semantic embedding assigns semantic representations to lexical units, and pre-trained language models learn the linguistic knowledge and semantic relationships inherent in large amounts of text data. Because this process deeply explores the semantic connotations of text and converts it into computer-understandable high-dimensional semantic vectors, it effectively handles the semantic complexity of textual information and improves understanding of textual intent. Key regions of visual scene images are located, scene feature maps are extracted, and global intent pooling is used to compress them into image feature vectors. Key region location focuses on the important parts of the image relevant to intent, scene feature maps are extracted to obtain visual features, and global intent pooling compresses feature dimensions. This method highlights key image information, reduces redundancy, and produces a compact and representative image feature vector. This effectively handles the spatial characteristics of image data and facilitates subsequent fusion processing. Sliding window segmentation is performed on environmental sensor time series information, and statistical features and frequency domain energy distribution within the window are calculated to generate an environmental perception feature vector. Sliding window segmentation divides continuous time series data into segments, facilitating analysis of features at different time intervals. Statistical features and frequency domain energy distribution reflect the numerical and frequency characteristics of the data. Because this approach captures the dynamic changes in sensor data and generates accurate environmental perception feature vectors, it can address the difficulty of handling dynamic sensor data. It maps speech intent feature vectors, high-dimensional text semantic vectors, image feature vectors, and environmental perception feature vectors into a unified dimensional space to generate a standardized intent input vector. By unifying the dimensional space, the dimensional differences between feature vectors of different modalities are eliminated. This allows data from all modalities to be fused and analyzed at the same scale, effectively integrating multimodal data and facilitating subsequent intent recognition and response by smart devices.Through the above series of steps, the present invention realizes the effective feature extraction and unified processing of multimodal interaction data, solves the problems of difficult fusion of multimodal data and insufficient feature extraction, and lays the foundation for smart devices to accurately understand user intentions.

[0078] In one embodiment of the present invention, the step of inputting the fused intent feature matrix into an intent classifier to obtain the probability distribution data of the user's initial intent includes:

[0079] S31, construct a multi-task intent loss function, which includes intent classification cross entropy loss and intent feature distribution regularization loss, and its formula is:

[0080] L=λ1*L cls +λ2*L reg ;

[0081] Where L represents the value of the multi-task loss function, L cls represents the intent classification cross entropy loss, λ1 represents the weight of the intent classification cross entropy loss, L reg represents the regularization loss of the intention feature distribution, and λ2 represents the weight of the regularization loss of the intention feature distribution;

[0082] S32, based on the multi-task intent loss function, injecting Gaussian noise into the fused intent feature matrix to generate an adversarial intent feature matrix, and updating the parameters of the intent classifier through gradient back propagation;

[0083] S33, based on the multi-task intent loss function, adopting a curriculum learning strategy to perform phased training according to the difficulty of the intent features in the fusion intent feature matrix, giving priority to learning the intent features with high confidence, and gradually introducing the fuzzy intent features;

[0084] S34, using a focus loss function to dynamically adjust the weight of each of the intent features in the intent classification cross entropy loss to obtain a trained intent classifier;

[0085] S35: Input the fusion intention feature matrix into the trained intention classifier for linear transformation, apply the Softmax function to the result after linear transformation, and obtain the probability distribution data of the user's initial intention. The calculation formula is:

[0086] P(I)=Softmax(W*F+b);

[0087] Where P(I) represents the probability distribution data of the user's initial intention, W represents the weight matrix, F represents the fusion feature matrix, and b represents the bias term.

[0088] As described in steps S31-S35 above, in the field of intent classification, traditional methods face difficulties in fusing multimodal data due to differences in feature dimensions and sampling rates. Traditional methods often use simple concatenation, which can easily lose key information. To address this issue, the present invention proposes a technical solution that inputs a fused intent feature matrix into an intent classifier to obtain the user's initial intent probability distribution data. This involves constructing a multi-task intent loss function that includes an intent classification cross-entropy loss and an intent feature distribution regularization loss. The intent classification cross-entropy loss measures the difference between the model's predicted intent category and the true category, improving classification accuracy. The intent feature distribution regularization loss constrains the distribution of intent features in the feature space, making it more reasonable and preventing model overfitting. This approach simultaneously optimizes the model for both classification accuracy and feature distribution rationality, laying the foundation for subsequent training and resolving the difficulty of balancing both. Based on the multi-task intent loss function, Gaussian noise is injected into the fused intent feature matrix to generate an adversarial intent feature matrix. Gradient backpropagation is then used to update the intent classifier parameters. The injected Gaussian noise simulates interference that may be encountered in real applications, and gradient backpropagation enables the model to accurately classify even in the presence of noise. Because this enhances the model's resistance to noise and interference, it improves model robustness and addresses the issue of insufficient robustness. Based on a multi-task intent loss function, a curriculum learning strategy is employed to train the model in stages based on the difficulty of the intent features in the fused intent feature matrix. By prioritizing high-confidence intent features, the model can quickly grasp clear, easily recognizable intent patterns. Gradually introducing more ambiguous intent features allows the model to further learn to handle complex situations based on this foundation. This simple-to-complex learning sequence conforms to cognitive principles, improving model learning efficiency and addressing the inefficiency caused by an irrational learning sequence. A focus loss function is used to dynamically adjust the weight of each intent feature in the cross-entropy loss for intent classification. This weight is reduced for easily classified samples and increased for samples from difficult-to-classify and rare intent categories. This allows the model to focus more on difficult samples, mitigating the impact of sample imbalance across intent categories on classification performance. The fused intent feature matrix is input into the trained intent classifier for a linear transformation, followed by a Softmax function applied to the result. The linear transformation transforms the features, and the Softmax function converts the transformed result into a probability distribution. Because this operation converts the model output into a form that meets the requirements of the probability distribution, the initial probability distribution data of the user's intent is obtained, providing data support for subsequent intent determination. Through the above series of steps, the present invention effectively solves the accuracy, robustness, learning efficiency, and sample imbalance issues existing in intent classification, and achieves accurate acquisition of the user's initial probability distribution data of the intent.

[0089] In one embodiment of the present invention, the step of dynamically revising the initial intent probability distribution data based on the historical intent record to generate a final intent determination label includes:

[0090] S41, extracting a time decay factor and a historical intention feature vector according to the historical intention record;

[0091] S42, extracting a current intention feature vector from the fused intention feature matrix, and calculating a cosine similarity between the current intention feature vector and the historical intention feature vector;

[0092] S43, obtaining a dynamic correction coefficient of intention probability according to the time decay factor and the cosine similarity;

[0093] S44, correct the initial probability distribution data of the user intention according to the dynamic correction coefficient of the intention probability, and select the label corresponding to the maximum probability value as the final intention determination label, which is calculated as follows:

[0094]

[0095] Where P′ i The corrected probability value of the i-th type of intention, P i represents the probability value of the i-th type of intention in the initial probability distribution data, ω represents the weight of the probability value of the i-th type of intention, Sim(C curr , C j ) represents the cosine similarity between the current intention feature vector and the historical intention feature vector, C curr Represents the current intention feature vector, C j represents the j-th historical intention feature vector, β j represents the weight of the probability value of the jth historical intent, and k represents the number of intents in the historical intent record.

[0096] As described in steps S41-S44 above, determining intent on smart devices based solely on initial intent probability distribution data presents limitations. First, it fails to consider the temporal nature of user intent, and the reference value of earlier intent records may decrease over time. Second, it lacks quantitative analysis of the correlation between current and historical intent, making it difficult to accurately utilize historical information to optimize intent determination, potentially leading to inaccurate and inconsistent intent determinations. To address these issues, the present invention proposes a technical solution that dynamically modifies initial intent probability distribution data based on historical intent records to generate a final intent determination label. The method involves extracting a time decay factor and a historical intent feature vector from the historical intent records. The time decay factor reflects the change in reference value of the historical intent record over time; the longer the time from the current moment, the less likely it is to have such a significant impact. The historical intent feature vector carries characteristic information about past intents. By clarifying these two elements, the method provides a foundation for subsequent utilization of historical intent records, addressing the issues of not considering temporal variations in intent and quantifying historical intent features. The current intent feature vector is extracted from the fused intent feature matrix, and its cosine similarity with the historical intent feature vector is calculated. Cosine similarity measures the degree of directional similarity between two vectors, specifically the feature-level similarity between the current intent and historical intent. Since calculating this similarity quantifies the correlation between the current and historical intents, it provides a basis for subsequent corrections, addressing the lack of quantitative analysis of the correlation between current and historical intents. A dynamic correction coefficient for intent probability is derived based on the time decay factor and cosine similarity. The time decay factor and cosine similarity work together to comprehensively consider the timeliness of historical intents and their similarity to the current intent. This yields a correction coefficient that comprehensively reflects the impact of historical intent on the current intent, allowing for appropriate adjustments to the initial intent probability distribution. The initial probability distribution of the user's intent is then corrected based on the dynamic correction coefficient, and the label corresponding to the maximum probability value is selected as the final intent determination label. The correction coefficient adjusts the initial probability distribution data to better reflect the user's actual intent. The label with the maximum probability is then selected to determine the final intent determination. This process optimizes the initial probability distribution data using historical intent information, resulting in a more accurate final intent determination label that meets user needs. Through the above series of steps, the present invention makes full use of historical intent records, considers the changes of intent over time and its association with historical intent, effectively solves the problems existing in intent determination, and achieves more accurate determination of user intent.

[0097] In one embodiment of the present invention, the step of constructing an intent response strategy library and matching an optimal response action set from the intent response strategy library based on the final intent determination tag includes:

[0098] S51, define the response action set, including action type, parameter configuration and execution priority;

[0099] S52, using the knowledge graph to build an intent response strategy library, which includes intent determination label nodes, target state nodes, action transfer relationship edges, edge weights, and response action sets, where the edge weights are calculated based on historical execution success rates;

[0100] S53, using the shortest path algorithm to search and generate several candidate response action sets from the current intention label node;

[0101] S54 , evaluating the execution efficiency, user rating, and resource cost of the candidate response action sets through a multi-objective utility function, and screening multiple optimal response action sets.

[0102] As described in steps S51-S54 above, during the intent response process of smart devices, traditional approaches suffer from static policy libraries that are unable to adapt to dynamic scenarios and lack global optimization for response action selection. Specifically, the lack of clear definition of the specific components of response actions makes it difficult to efficiently manage and match response policies, unable to quickly filter candidate actions from a multitude of possible actions, and failing to balance efficiency, user experience, and resource costs when determining the final action to be executed. To address these issues, the present invention proposes a technical solution for constructing an intent response policy library and matching the optimal set of response actions. The response action set is clearly defined, including action type, parameter configuration, and execution priority. The action type defines the scope of the response, such as device control or information query; the parameter configuration determines the specific details of the action execution, such as the specific value for adjusting the temperature; and the execution priority is determined by combining device resource utilization and user preferences, allowing the device to rationally schedule the execution of actions. By clearly defining the key elements of the response action, response action planning is made clearer, addressing the issue of unclear response action planning. The policy library is constructed using a knowledge graph, which includes intent determination label nodes, target state nodes, action transition relationship edges, edge weights, and a set of response actions. Intent determination label nodes are associated with user intent categories, target state nodes represent the desired state achieved by an action, and action-transition edges connect intents and actions. Edge weights are calculated based on historical execution success rates. This structured approach to organizing response policies makes the policy library logically clear, easy to manage, and easy to query. By constructing the policy library in the form of a knowledge graph, the problem of irrational response policy organization is resolved, enabling efficient policy matching. A shortest path algorithm is used to search and generate several candidate response action sets from the current intent label node. The shortest path algorithm quickly finds valid paths from the current intent to possible actions within the knowledge graph network, screening out qualified candidate action sets. This algorithm's search allows for efficient selection of candidate actions from a large pool of actions, addressing the lack of efficient algorithms for candidate action selection. A multi-objective utility function is used to evaluate the execution efficiency, user ratings, and resource costs of candidate response action sets. Execution efficiency measures the speed of action execution, user ratings reflect user satisfaction, and resource costs reflect device resource consumption. By integrating these three factors, the candidate action sets can be comprehensively evaluated and selected as the optimal ones. Because the multi-objective utility function comprehensively considers multiple factors, it solves the problem of the optimal action determination not taking multiple factors into account, and the final execution action takes into account efficiency, experience, and cost. Through the above steps S51-S54, the present invention realizes the rational planning of the intelligent device's intended response actions, the effective construction of the policy library, the efficient screening of candidate actions, and the comprehensive determination of the optimal action, thereby solving many problems existing in the intelligent device's intention response process.

[0103] In one embodiment of the present invention, the step of optimizing the optimal response action set in real time by using a reinforcement learning model, generating a dynamic response instruction set, and controlling the execution of the device includes:

[0104] S61, defining the state space as the fused intention feature matrix and the final intention determination label, and the action space as the candidate response action set;

[0105] S62, design a composite reward function, including user feedback score, resource saving rate and execution delay penalty, and its calculation formula is:

[0106] R(s,a)=θ1*FS+θ2*RS+θ3*DP;

[0107] Where R(s,a) represents the reward value obtained when taking action a in state s, θ1, θ2, and θ3 are the weight coefficients of user feedback score, resource saving rate, and execution delay penalty, respectively. FS represents the user feedback score, RS represents the resource saving rate, and DP represents the execution delay penalty.

[0108] S63, training an Actor-Critic network based on the state space and the action space using a deep deterministic policy gradient algorithm, generating a dynamic response instruction set through the Actor network, and evaluating action values through the Critic network;

[0109] S64, store historical interaction data into the experience replay pool, and update the Actor-Critic network parameters according to the temporal difference error priority sampling data;

[0110] S65: Control the device to execute according to the dynamic response instruction set.

[0111] As described in steps S61-S64 above, traditional approaches to optimizing and controlling smart device response actions suffer from rigid response strategies and an inability to adapt in real time to environmental changes. For example, they fail to effectively integrate information related to device operating status and user intent to determine the basis for optimization; lack a reward mechanism that comprehensively considers multiple factors, making it difficult to fully guide model optimization; training algorithms are inefficient and ineffective, preventing the rapid and accurate generation of appropriate response instructions; historical interaction data is insufficiently utilized, lacking effective input for model learning and optimization; and device execution control is disconnected from model optimization results, making it difficult to effectively execute response instructions. To address these issues, the present invention proposes a technical solution for optimizing the optimal response action set and controlling device execution in real time using a reinforcement learning model. The state space is defined as a fused intent feature matrix containing deep semantic features and a final intent determination label that clearly defines the user intent category. The combination of the two comprehensively reflects the device's operating and user intent states. The action space is defined as a set of candidate response actions, providing a range of executable actions. By clearly defining these two spaces, the reinforcement learning model lays the foundation for optimizing response actions, resolving the issue of unclear information integration and optimization foundations. A compound reward function is designed that incorporates user feedback ratings, resource conservation rate, and execution delay penalty. The user feedback rating reflects user satisfaction, the resource conservation rate focuses on resource utilization efficiency, and the execution delay penalty constrains action execution time. By integrating these three factors, we can comprehensively measure the effectiveness of action execution and guide the model to learn a more optimal action policy. Because the compound reward function comprehensively considers multiple factors, it overcomes the problem of an incomplete reward mechanism and effectively guides model optimization. A deep deterministic policy gradient algorithm is used to train the actor-critic network based on the state space and action space. The actor network generates a dynamic response instruction set and decides on an action based on the current state; the critic network evaluates the value of an action and determines its quality. Combining the strengths of these two algorithms, the deep deterministic policy gradient algorithm efficiently learns and optimizes policies. Using this algorithm to train the network overcomes the problems of low training efficiency and poor performance, enabling rapid and accurate generation of response instructions. Historical interaction data is stored in an experience replay pool, and the actor-critic network parameters are updated based on data sampling using a temporal difference error priority method. The experience replay pool stores historical data, breaking data correlation, while temporal difference error priority sampling allows the model to prioritize learning important data. By fully leveraging historical interaction data to update network parameters, the problem of insufficient data utilization is resolved, providing effective material for model learning and optimization. By controlling device execution based on a dynamic response instruction set, model optimization results are directly applied to device operations, ensuring effective execution of response instructions. By tightly integrating device execution control with model optimization results, the disconnect between execution and optimization is resolved.Through the above steps S61-S64, the present invention effectively solves the problems existing in the optimization control of the response actions of intelligent devices, and realizes the real-time optimization of the optimal response action set and the effective control of the device.

[0112] In one embodiment of the present invention, the step of updating the semantic feature extraction model and the intent response strategy library based on the user feedback satisfaction data and the device execution performance index includes:

[0113] S71, collect user feedback scores, execution delay and accuracy indicators, and build a model update dataset;

[0114] S72, calculating the model parameter update gradient by adding elastic weight constraints;

[0115] S73, updating the semantic feature extraction model based on the model update dataset and the model parameter update gradient;

[0116] S74, incrementally updating the intent response strategy library based on the semantic feature extraction model, and re-encoding the graph structure relationship using a graph neural network when adding a new node;

[0117] S75, regularly verify the intent recognition accuracy of the optimized model.

[0118] As described in steps S71-S74 above, traditional approaches to optimizing the semantic feature extraction model and intent response policy library for smart devices face challenges with continuous learning, which can easily lead to model forgetting of old knowledge (catastrophic forgetting) and inefficient policy library updates. To address these issues, the present invention proposes a technical solution for updating the semantic feature extraction model and intent response policy library based on user feedback satisfaction data and device execution performance metrics. This involves collecting user feedback scores, execution latency, and accuracy metrics to construct a model update dataset. User feedback scores directly reflect user satisfaction with device responses, execution latency reflects device response speed, and accuracy measures the accuracy of intent recognition and action execution. Comprehensively collecting this critical data provides rich and targeted information for subsequent model and policy library optimization. By constructing a dataset containing multi-dimensional key information, the lack of an effective data collection system is overcome, providing a reliable basis for optimization. Furthermore, elastic weight constraints are added to calculate model parameter update gradients. These elastic weight constraints protect important parameters during model parameter updates, preventing excessive updates from causing the model to forget previously learned knowledge. Because this approach considers parameter stability, it effectively avoids catastrophic forgetting and addresses parameter instability during model updates. The semantic feature extraction model is updated based on the model update dataset and the model parameter update gradient. Leveraging previously collected data and calculated gradients, the model is specifically adjusted to better extract semantic features and adapt to new user intent and data patterns. By updating the model based on valid data and reasonable gradients, the model's ability to extract semantic features is enhanced, addressing the issue of the model's difficulty adapting to changes. The intent response policy library is incrementally updated based on the updated semantic feature extraction model. A graph neural network is used to re-encode graph structural relationships when new nodes are added. As the semantic feature extraction model is optimized, new user intent features and requirements are discovered, enabling incremental updates to the policy library, continuously enriching and improving it. Graph neural networks effectively process graph-structured data and re-encode structural relationships when new nodes are added, ensuring the rationality and consistency of the policy library structure. This approach to updating the policy library allows it to adapt to evolving user intent and scenario requirements, addressing the issue of a single policy library update method. The intent recognition accuracy of the optimized model is regularly verified. By continuously monitoring the model's intent recognition accuracy, problems that may arise during the optimization process can be discovered in a timely manner, and it can be evaluated whether the optimization effect meets expectations. Since a continuous verification mechanism has been established, the reliability of the optimization effect is ensured, and the problem of lack of verification of the optimized model effect is solved. Through the above steps S71-S74, the present invention effectively solves the problems existing in the optimization of the semantic feature extraction model and the intent response policy library of smart devices, and realizes the continuous optimization and performance improvement of the model and policy library.

[0119] like Figure 2As shown, the present invention also discloses a system for rapid insight into the intention of smart devices and intelligent response, including:

[0120] Multimodal data acquisition module 1, used to obtain multimodal interaction data input by users into smart devices, and normalize the multimodal interaction data to generate a standardized intention input vector;

[0121] Semantic feature extraction module 2, used for performing hierarchical feature extraction on the standardized intent input vector based on a preset semantic feature extraction model, and outputting a fused intent feature matrix;

[0122] Intent classification module 3, used to input the fused intent feature matrix into the intent classifier to obtain the user's initial intent probability distribution data;

[0123] Intent determination module 4 is used to obtain historical intent records in the context memory of the smart device, and dynamically modify the initial intent probability distribution data based on the historical intent records to generate a final intent determination label;

[0124] Intent response matching module 5, used to build an intent response strategy library and match the optimal response action set from the intent response strategy library based on the final intent determination label;

[0125] Reinforcement learning control module 6, used to optimize the optimal response action set in real time through the reinforcement learning model, generate a dynamic response instruction set and control the device execution;

[0126] The closed-loop feedback module 7 is used to collect user feedback satisfaction data and device execution efficiency indicators, and update the semantic feature extraction model and intention response strategy library based on the user feedback satisfaction data and device execution efficiency indicators to optimize the intention response.

[0127] The present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and is characterized in that when the processor executes the computer program, it implements the steps of a method for rapid insight into the intentions of an intelligent device and intelligent response.

[0128] The present invention also discloses a computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the computer program implements the steps of a method for rapid insight into the intentions of an intelligent device and intelligent response.

[0129] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, apparatus, article, or method comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, apparatus, article, or method. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, apparatus, article, or method comprising the element.

[0130] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for rapid insight into the intentions of smart devices and intelligent response, characterized in that: include: Acquire multimodal interaction data input by a user into a smart device, the multimodal interaction data including voice information, text information, visual scene image information, and sensor timing information, and normalize the multimodal interaction data to generate a standardized intent input vector; Based on a preset semantic feature extraction model, hierarchical feature extraction is performed on the standardized intent input vector, and a fusion intent feature matrix is output; Inputting the fused intention feature matrix into the intention classifier to obtain the user's initial intention probability distribution data; Obtaining historical intent records from the context memory of the smart device, dynamically revising the initial intent probability distribution data based on the historical intent records, and generating a final intent determination label; Building an intention response strategy library, and matching an optimal response action set from the intention response strategy library based on the final intention determination label; The optimal response action set is optimized in real time through a reinforcement learning model to generate a dynamic response instruction set and control the execution of the device; Collect user feedback satisfaction data and device execution efficiency indicators, update the semantic feature extraction model and intent response strategy library based on the user feedback satisfaction data and device execution efficiency indicators, and perform intent response optimization.

2. The method for rapid insight into intention and intelligent response of a smart device according to claim 1, characterized in that: The step of normalizing the multimodal interaction data to generate a standardized intention input vector includes: Performing frame and window processing on the speech information, extracting Mel-frequency cepstral coefficients and fundamental frequency features, and generating a speech intention feature vector; Perform word segmentation and semantic embedding on the text information, and generate a high-dimensional text semantic vector through a pre-trained language model; Positioning key areas of the visual scene image information, extracting scene feature maps and compressing them into image feature vectors through global intent pooling; Performing sliding window segmentation on the environmental sensor time series information, calculating statistical features and frequency domain energy distribution within the window, and generating an environmental perception feature vector; The speech intention feature vector, the high-dimensional text semantic vector, the image feature vector and the environmental perception feature vector are mapped to a unified dimensional space to generate a standardized intention input vector.

3. The method for rapid insight into intention and intelligent response of a smart device according to claim 1, characterized in that: The step of inputting the fused intention feature matrix into the intention classifier to obtain the user's initial intention probability distribution data includes: Construct a multi-task intent loss function, which includes intent classification cross entropy loss and intent feature distribution regularization loss; Based on the multi-task intent loss function, injecting Gaussian noise into the fused intent feature matrix to generate an adversarial intent feature matrix, and updating the parameters of the intent classifier through gradient back propagation; Based on the multi-task intent loss function, a curriculum learning strategy is adopted to perform phased training according to the difficulty of the intent features in the fusion intent feature matrix, giving priority to learning the intent features with high confidence and gradually introducing the fuzzy intent features; Using a focused loss function to dynamically adjust the weight of each of the intent features in the intent classification cross entropy loss, to obtain a trained intent classifier; The fused intent feature matrix is input into the trained intent classifier for linear transformation, and the Softmax function is applied to the result after linear transformation to obtain the initial probability distribution data of the user intent.

4. The method for rapid insight into intention and intelligent response of a smart device according to claim 1, characterized in that: The step of dynamically revising the initial intention probability distribution data based on the historical intention record to generate a final intention determination label includes: Extracting time decay factor and historical intention feature vector based on historical intention records; Extracting a current intention feature vector from the fused intention feature matrix, and calculating the cosine similarity between the current intention feature vector and the historical intention feature vector; Obtaining a dynamic correction coefficient of intention probability according to the time decay factor and the cosine similarity; The initial probability distribution data of the user intention is corrected according to the dynamic correction coefficient of the intention probability, and the label corresponding to the maximum probability value is selected as the final intention determination label.

5. The method for rapid insight into intention and intelligent response of a smart device according to claim 4, characterized in that: The step of constructing an intent response strategy library and matching an optimal response action set from the intent response strategy library based on the final intent determination tag includes: Define the response action set, including action type, parameter configuration and execution priority; The knowledge graph is used to build an intent response strategy library, which includes intent judgment label nodes, target state nodes, action transfer relationship edges, edge weights, and response action sets. The edge weights are calculated based on the historical execution success rate. Use the shortest path algorithm to search and generate several candidate response action sets from the current intention label node; The execution efficiency, user rating and resource cost of the candidate response action sets are evaluated by a multi-objective utility function, and multiple optimal response action sets are screened.

6. The method for rapid insight into intention and intelligent response of a smart device according to claim 1, characterized in that: The step of optimizing the optimal response action set in real time by using a reinforcement learning model, generating a dynamic response instruction set, and controlling the execution of the device includes: The state space is defined as the fusion intention feature matrix and the final intention determination label, and the action space is defined as the candidate response action set; Design a composite reward function that includes user feedback score, resource saving rate, and execution delay penalty; Based on the state space and the action space, an Actor-Critic network is trained using a deep deterministic policy gradient algorithm, a dynamic response instruction set is generated by the Actor network, and an action value is evaluated by the Critic network; Store historical interaction data in the experience replay pool, and update the Actor-Critic network parameters according to the temporal difference error priority sampling data; The device is controlled to execute according to the dynamic response instruction set.

7. The method for rapid insight into intention and intelligent response of a smart device according to claim 1, characterized in that: The step of updating the semantic feature extraction model and the intent response strategy library according to the user feedback satisfaction data and the device execution efficiency index includes: Collect user feedback scores, execution latency, and accuracy metrics to build a model and update the dataset; Calculate the model parameter update gradient by adding elastic weight constraints; Update the semantic feature extraction model based on the model update dataset and the model parameter update gradient; Incrementally updating the intent response strategy library based on the semantic feature extraction model, and re-encoding the graph structure relationship using a graph neural network when adding a new node; Regularly verify the intent recognition accuracy of the optimized model.

8. A system for rapid insight into the intentions of smart devices and intelligent response, characterized in that: include: A multimodal data acquisition module is used to obtain multimodal interaction data input by users into smart devices and normalize the multimodal interaction data to generate a standardized intent input vector; A semantic feature extraction module, configured to perform hierarchical feature extraction on the standardized intent input vector based on a preset semantic feature extraction model, and output a fused intent feature matrix; An intention classification module is used to input the fused intention feature matrix into an intention classifier to obtain the user's initial intention probability distribution data; An intent determination module is used to obtain historical intent records in the context memory of the smart device, and dynamically modify the initial intent probability distribution data based on the historical intent records to generate a final intent determination label; An intent response matching module is used to build an intent response strategy library and match an optimal response action set from the intent response strategy library based on the final intent determination label; A reinforcement learning control module is used to optimize the optimal response action set in real time through a reinforcement learning model, generate a dynamic response instruction set, and control the execution of the device; The closed-loop feedback module is used to collect user feedback satisfaction data and device execution efficiency indicators, and update the semantic feature extraction model and intent response strategy library based on the user feedback satisfaction data and device execution efficiency indicators to optimize the intent response.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images