User feedback information increment mining method and device of AI question and answer intelligent agent

Abstract

The application discloses a kind of AI question and answer intelligent agent's user feedback information increment mining method and device, method includes: synchronously collecting the initiative feedback text of user and passive operation trajectory data, and binding user identification and session identification, construct original feedback dataset, respectively extract semantic vector and the behavior vector of same dimension;Weighted splicing generates panoramic feature vector constitutes fusion dataset;Clustering analysis is carried out to fusion dataset, and clustering parameter is adaptively adjusted according to data density distribution characteristics, extract the high-frequency problem mode formed by the sample cluster with the same feedback characteristics, form incremental mining result;Quantitative evaluation is generated to high-frequency problem mode optimization priority list;Establish the mapping relationship of high-frequency problem mode and optimization action to execute optimization action.The application realizes the full collection of feedback data, depth fusion and incremental mining, and improves the iteration efficiency of intelligent agent through closed-loop optimization mechanism.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and apparatus for incremental mining of user feedback information by an AI question-answering agent. Background Technology

[0002] AI-powered question-answering agents, such as intelligent customer service representatives and dialogue assistants, provide services like information retrieval and task execution through multi-round interactions with users. In-depth mining and analysis of user feedback is a key driver for the continuous iteration and optimization of these agents. User feedback includes not only explicit textual evaluations but also implicit details of their interactive behaviors with the agent; together, they form a comprehensive user feedback profile.

[0003] Existing technologies for mining user feedback information from AI question-answering agents have significant limitations. Firstly, data collection is often limited to a single dimension, typically focusing only on user-submitted feedback text, such as reasons for dislikes or feature suggestions, neglecting passive operational data generated during use. This results in incomplete information collection and an inability to accurately reflect the full picture of the user experience. Secondly, existing mining methods often employ simple keyword statistics or text classification, failing to correlate and integrate user text feedback with their operational behaviors, making it difficult to uncover deep, high-frequency problem patterns hidden behind the surface feedback. Furthermore, mining results often remain at the level of analysis reports, lacking a closed-loop mechanism to automatically and accurately map discovered problems into specific optimization actions for the agent, such as tuning the intent recognition model or updating the knowledge base, leading to low information value utilization and iterative optimization efficiency.

[0004] Therefore, there is an urgent need for a method that can collect user feedback information from all dimensions, deeply integrate and merge it, incrementally mine it, and efficiently implement it into optimization actions. Summary of the Invention

[0005] The purpose of this invention is to provide a method, device, and terminal for incremental mining of user feedback information by an AI question-answering intelligent agent, aiming to solve the problems of existing feedback information collection having a single dimension, insufficient mining, and being disconnected from optimization actions.

[0006] In a first aspect, the present invention provides a method for incremental mining of user feedback information in an AI question-answering agent, comprising: Simultaneously collect user's active feedback text data and passive operation trajectory data to the AI ​​question-answering agent, bind the active feedback text data and the passive operation trajectory data to the user's unique identifier and session identifier, and record the feedback timestamp to construct a multi-dimensional raw feedback dataset; Semantic features are extracted from the active feedback text data to obtain a semantic vector, and behavioral features are extracted from the passive operation trajectory data to obtain a behavioral vector with the same dimension as the semantic vector. The semantic vector and the behavior vector of the same user and the same session are weighted and concatenated to generate a panoramic feature vector. The panoramic feature vector, the user's unique identifier, the session identifier, and the feedback timestamp are used to form a fusion dataset. Cluster analysis is performed on the fused dataset, and the clustering parameters are adaptively adjusted according to the density distribution characteristics of the data in the fused dataset to extract high-frequency problem patterns composed of sample clusters with the same feedback characteristics, forming incremental mining results; Based on the incremental mining results, the high-frequency problem patterns are quantitatively evaluated, and an optimization priority list is generated. Based on the optimization priority list, a mapping relationship is established between the high-frequency question patterns and the optimization actions of the AI ​​question-answering agent to execute the optimization actions; wherein, the mapping relationship is a preset optimization operation that maps the question category corresponding to the high-frequency question pattern to the corresponding component in the intent recognition model, knowledge base, answer generation module or interactive interface of the AI ​​question-answering agent.

[0007] Furthermore, the simultaneous collection of user-initiated feedback text data and passive operation trajectory data to the AI ​​question-answering agent includes: Collect the active feedback text data, which includes the user's reasons for disliking or rejecting the agent's response, the feedback content, and the function suggestion text. Collect the passive operation trajectory data, which includes session operation behavior, interaction duration characteristics, and content operation characteristics; The conversational actions include repeatedly modifying the question content, repeatedly disliking the same type of answer, continuously asking the same type of question, and the frequency and order of clicking the agent function button; The interaction duration features include the user's dwell time on a single response, the total session duration, and the trigger delay time for feedback operations; The content operation features include users' behaviors such as collecting, sharing, and copying answer content, as well as clicking on content recommended by the agent.

[0008] Further, the step of extracting semantic features from the active feedback text data to obtain a semantic vector, and extracting behavioral features from the passive operation trajectory data to obtain a behavioral vector with the same dimension as the semantic vector, includes: The proactive feedback text data is transformed into a semantic vector of a preset dimension using a pre-trained text semantic model. The passive operation trajectory data is normalized and encoded using a behavior encoding algorithm, and then transformed into a behavior vector with the same dimension as the semantic vector.

[0009] Further, the step of weightedly concatenating the semantic vectors and behavioral vectors of the same user and the same session to generate a panoramic feature vector includes: Based on the application scenario, a first weight coefficient is assigned to the semantic vector, and a second weight coefficient is assigned to the behavior vector; Based on the first weight coefficient and the second weight coefficient, a weighted concatenation operation is performed on the semantic vector and the behavior vector to generate the panoramic feature vector.

[0010] Furthermore, the clustering analysis performed on the fused dataset, adaptively adjusting the clustering parameters based on the density distribution characteristics of the data in the fused dataset, to extract high-frequency problem patterns composed of sample clusters with the same feedback characteristics, forming incremental mining results, includes: An improved DBSCAN clustering algorithm is used to adaptively adjust the neighborhood radius and minimum number of contained points by calculating the local density of the data in the fused dataset. Based on the adjusted neighborhood radius and the minimum number of contained points, the fused dataset is clustered to form multiple sample clusters; For each of the sample clusters, feature summarization is performed to generate a textual description and a unique feature identifier for the corresponding problem pattern, thereby constituting the high-frequency problem pattern; The sample clusters after feature summarization are used as the incremental mining results.

[0011] Furthermore, the step of summarizing the features of each sample cluster to generate a textual description and unique feature identifier for the corresponding problem pattern includes: By reverse parsing the panoramic feature vectors in the sample cluster using a pre-defined large language model, a textual description and unique feature identifier of the problem pattern are generated.

[0012] Furthermore, based on the incremental mining results, the high-frequency problem patterns are quantitatively evaluated to generate an optimization priority list, including: From the incremental mining results, obtain the sample size proportion, user influence range, and agent performance correlation of the high-frequency problem patterns; The optimization priority score is obtained by weighting the sample size ratio, the user influence range, and the correlation between the agent performance and the user's influence. The optimization priority list is generated by sorting the optimization priority scores from high to low.

[0013] Furthermore, establishing the mapping relationship between the high-frequency question patterns and the optimization actions of the AI ​​question-answering agent based on the optimization priority list includes: Based on the optimization priority list, high-frequency problem patterns to be processed are determined from the incremental mining results; Based on the problem category corresponding to the high-frequency problem pattern, establish a mapping relationship between the high-frequency problem pattern and at least one preset optimization action; the mapping relationship includes at least one of the following: When the high-frequency question pattern is that the AI ​​question-answering agent fails to recognize the core question intent, a mapping relationship is established between the high-frequency question pattern and the fine-tuning actions of the intent recognition model and the optimization actions of the question intent guidance rules. When the high-frequency question pattern indicates that the AI ​​question-answering agent's answer lacks authoritative basis, a mapping relationship is established between the high-frequency question pattern and the knowledge base supplementation and update actions and the answer information source verification rule adjustment actions. When the high-frequency question pattern is that the AI ​​question-answering agent's answer is logically confused or semantically biased, a mapping relationship is established between the high-frequency question pattern and the large model prompt word optimization action and the answer generation rule adjustment action; When the high-frequency question pattern is that the AI ​​question-answering agent's interactive operation is cumbersome or the feedback entry is hidden, a mapping relationship is established between the high-frequency question pattern and the functional interaction interface optimization action and the feedback entry layout adjustment action. Based on the mapping relationship, optimization instructions are generated and sent to the corresponding modules of the AI ​​question-answering agent to execute the preset optimization actions.

[0014] Secondly, the present invention provides an incremental mining device for user feedback information of an AI question-answering agent, comprising: The data acquisition module is used to synchronously collect active feedback text data and passive operation trajectory data of users to the AI ​​question-answering agent, bind the active feedback text data and the passive operation trajectory data to the user's unique identifier and session identifier, record the feedback timestamp, and construct a multi-dimensional raw feedback dataset. The feature extraction module is used to extract semantic features from the active feedback text data to obtain a semantic vector, and to extract behavioral features from the passive operation trajectory data to obtain a behavioral vector with the same dimension as the semantic vector. The feature fusion module is used to weight and concatenate the semantic vector and the behavior vector of the same user and the same session to generate a panoramic feature vector, and to combine the panoramic feature vector, the user's unique identifier, the session identifier and the feedback timestamp to form a fusion dataset. The incremental mining module is used to perform cluster analysis on the fused dataset. It adaptively adjusts the clustering parameters according to the density distribution characteristics of the data in the fused dataset to extract high-frequency problem patterns composed of sample clusters with the same feedback characteristics, thereby forming incremental mining results. The quantitative evaluation module is used to quantitatively evaluate the high-frequency problem patterns based on the incremental mining results and generate an optimization priority list. The mapping execution module is used to establish a mapping relationship between the high-frequency question patterns and the optimization actions of the AI ​​question-answering agent based on the optimization priority list, so as to execute the optimization actions; wherein, the mapping relationship is a preset optimization operation that maps the question category corresponding to the high-frequency question pattern to the corresponding component in the intent recognition model, knowledge base, answer generation module or interactive interface of the AI ​​question-answering agent.

[0015] Thirdly, the present invention provides a terminal comprising: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements a method for incremental mining of user feedback information of an AI question-answering agent as described in any one of the first aspects.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: By simultaneously collecting users' active feedback text and passive operation trajectory data, and fusing them, a panoramic feature vector that comprehensively represents the user's feedback intent is generated. Based on this, a clustering analysis method that adaptively adjusts parameters according to data density distribution characteristics is employed to incrementally mine high-frequency problem patterns from the fused data. Furthermore, a quantitative evaluation is used to generate an optimization priority list, and a precise mapping and execution mechanism is established between high-frequency problem patterns and specific optimization actions of the intelligent agent. This invention achieves a closed-loop process from feedback data acquisition and in-depth mining to optimization action implementation, solving the problems of single-dimensional data collection, insufficient mining, and disconnection from optimization actions in existing technologies. This significantly improves the utilization value of feedback information and the iterative optimization efficiency of the AI ​​question-answering agent. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a method for incremental mining of user feedback information in an AI question-answering agent, provided in an embodiment of the present invention.

[0018] Figure 2 This is a flowchart illustrating a method for incremental mining of user feedback information in an AI question-answering agent according to a specific embodiment of the present invention.

[0019] Figure 3 This is a schematic diagram of the structure of an incremental mining device for user feedback information of an AI question-answering agent provided in an embodiment of the present invention.

[0020] Figure 4 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present invention. Detailed Implementation

[0021] The technical solutions of this invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some, not all, of the embodiments of this invention. The components of this invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0022] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0023] It should be noted that the method of this embodiment can be executed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method of this embodiment, and the multiple devices will interact with each other to complete the method described.

[0024] It should be noted that the above description describes some embodiments of the present invention. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than that shown in the above embodiments and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0025] Please see Figure 1 , Figure 1 This is a flowchart illustrating a method for incremental mining of user feedback information in an AI question-answering agent, provided in an embodiment of the present invention.

[0026] An embodiment of the present invention provides a method for incremental mining of user feedback information in an AI question-answering agent, comprising the following steps: S100. Simultaneously collect user's active feedback text data and passive operation trajectory data to the AI ​​question-answering agent, bind the active feedback text data and the passive operation trajectory data to the user's unique identifier and session identifier, record the feedback timestamp, and construct a multi-dimensional raw feedback dataset.

[0027] In this embodiment, proactive feedback text data refers to text information actively input by the user during interaction with the AI ​​question-answering agent, expressing their user experience or suggestions for improvement. Passive operation trajectory data refers to a series of behavioral records generated by the user on the interactive interface, which are automatically collected by the system.

[0028] Before step S100, various parameters and thresholds can be initialized.

[0029] In one specific embodiment, step S100, which involves simultaneously collecting active feedback text data and passive operation trajectory data from the user in response to the AI ​​question-answering agent, includes the following steps: S101. Collect the active feedback text data, which includes the user's reasons for clicking or disliking the AI ​​question-answering agent's answer, the feedback content, and the function suggestion text. S102. Collect the passive operation trajectory data, which includes session operation behavior, interaction duration characteristics, and content operation characteristics.

[0030] In this embodiment, the proactive feedback text data specifically includes the feedback reason text filled in by the user when disliking the agent's answer, the rumor feedback content submitted by the user, and the optimization suggestions text proposed by the user for the agent's functions.

[0031] Passive operation trajectory data specifically includes session operation behavior, interaction duration characteristics, and content operation characteristics.

[0032] The session operation behavior records user actions such as repeatedly modifying questions, repeatedly downvoting the same type of answer, continuously asking similar questions, and the frequency and order of clicking various function buttons on the agent. Interaction duration features record the user's page dwell time on a single answer from the agent, the total duration of a single session, and the delay between the agent providing an answer and the user triggering a feedback action. Content operation features record user actions such as saving, sharing, and copying the agent's answer, as well as clicking on content recommended by the agent in the answer.

[0033] All collected data is bound to the user's unique identifier and the unique identifier of this session, ensuring that the data source is traceable and associative.

[0034] This invention enables the full collection of user feedback data, overcoming the limitation of existing technologies that only collect active feedback text. It simultaneously collects active feedback text and passive operation trajectory data, comprehensively capturing users' real feedback needs from both textual and behavioral dimensions. This addresses the problem of missing dimensions in feedback information collection and provides a sufficient data foundation for in-depth data mining.

[0035] S200. Extract semantic features from the active feedback text data to obtain a semantic vector, and extract behavioral features from the passive operation trajectory data to obtain a behavioral vector with the same dimension as the semantic vector.

[0036] In this embodiment, in order to achieve the subsequent fusion of textual information and behavioral information, it is necessary to convert both into numerical vectors of the same dimension that can be processed by computers.

[0037] In one specific embodiment, step S200 includes the following steps: S201. The proactive feedback text data is transformed into a semantic vector of a preset dimension through a pre-trained text semantic model. S202. The passive operation trajectory data is normalized and encoded using a behavior encoding algorithm, and transformed into a behavior vector with the same dimension as the semantic vector.

[0038] In this embodiment, semantic feature extraction is performed using a pre-trained text semantic model based on the Transformer architecture, which transforms active feedback text data of arbitrary length into semantic vectors of a preset dimension (e.g., 512 dimensions). This model is fine-tuned using feedback corpora related to AI question-answering agents, enabling it to more accurately capture semantic information in AI question-answering feedback scenarios. After inputting active feedback text data of arbitrary length into this model, its output layer can extract a semantic vector of a preset dimension.

[0039] Behavioral feature extraction is achieved using a behavioral coding algorithm, the specific process of which is as follows: First, frequency statistics are performed on various events in the conversation operation behavior to generate statistical values ​​including the number of times questions are modified, the number of times they are downvoted, the number of times similar questions are asked, and the frequency of function button clicks. Then, the operation sequence is sequence encoded, and the transition probability matrix between adjacent operation types is calculated.

[0040] Secondly, the duration of a single response, the total duration of the conversation, and the delay in triggering feedback are collected and recorded as characteristics of interaction duration.

[0041] Then, the content operation features such as whether to favorite, share, copy, and click on recommended content are binarized and assigned a value of 0 or 1.

[0042] After concatenating all the above statistical values, transition probability values, duration values, and binary encoded values ​​into a one-dimensional feature vector, it is normalized and nonlinearly mapped through a fully connected neural network containing an input layer, a hidden layer, and an output layer. Finally, a behavior vector with the same dimension as the semantic vector (e.g., 512 dimensions) is output. The output layer dimension of this fully connected neural network is consistent with the output vector dimension of the pre-trained text semantic model.

[0043] In one embodiment, the specific structure of the fully connected neural network used in the behavior encoding algorithm is as follows: This fully connected neural network consists of three layers: an input layer, a hidden layer, and an output layer. Each layer is fully connected, meaning that every neuron in the previous layer is connected to every neuron in the next layer.

[0044] The number of neurons in the input layer is equal to the dimension of the concatenated one-dimensional feature vector. This concatenated one-dimensional feature vector is composed of the following components concatenated sequentially: the statistics of the number of times a question was modified, the statistics of the number of times a question was disliked, the statistics of the number of times similar questions were asked, the statistics of the frequency of function button clicks, the values ​​of each element of the transition probability matrix between adjacent operation types, the recorded value of the duration of a single answer, the recorded value of the total session duration, the recorded value of the feedback trigger delay duration, the binary encoded value of whether the question was saved, the binary encoded value of whether the question was shared, the binary encoded value of whether the question was copied, and the binary encoded value of whether the recommended content was clicked. The input layer is only responsible for receiving this one-dimensional feature vector and does not perform any calculation operations.

[0045] The hidden layer contains a predetermined number of neurons, which can be set according to the input and output dimensions, usually taking the midpoint between the input and output dimensions or determined through experimental tuning. The hidden layer uses a non-linear activation function to transform the input data, commonly the ReLU or GELU function, to introduce non-linear expressive power.

[0046] The number of neurons in the output layer is the same as the dimension of the semantic vector output by the pre-trained text semantic model, to align the dimensions of the behavior vector and the semantic vector. The output layer does not have an activation function; it directly outputs the numerical vector mapped by the network, which is the behavior vector.

[0047] This fully connected neural network automatically normalizes input features of different dimensions to a unified feature space during training through nonlinear transformations in the hidden layers, and encodes the original features into behavior vectors through multi-layer mapping. Before deployment, the network parameters are trained using labeled sample data, enabling the network to learn the mapping relationship from raw behavioral statistical features to behavior vectors. After training, the network parameters are fixed and used for forward inference on subsequently acquired passive operation trajectory data, outputting the corresponding behavior vectors.

[0048] S300. The semantic vector and the behavior vector of the same user and the same session are weighted and concatenated to generate a panoramic feature vector. The panoramic feature vector, the user's unique identifier, the session identifier and the feedback timestamp are combined to form a fusion dataset.

[0049] In this embodiment, a panoramic feature vector of user feedback is generated by fusing semantic features and behavioral features. This deeply binds the user's textual feedback with their operational behavior, reflecting the underlying logic behind the feedback information and solving the problem of insufficient feedback information mining.

[0050] Specifically, based on the user's unique identifier and session identifier (i.e., session ID), the semantic vectors and behavioral vectors generated by the same user in a complete session are associated and fused into a panoramic feature vector through weighted concatenation. This vector contains both the user's explicit textual intent and implicit behavioral features, and can comprehensively reflect the user's feedback panoramic features.

[0051] In one specific embodiment, step S300 involves weighted concatenation of the semantic vector and the behavior vector for the same user and the same session to generate a panoramic feature vector, including the following steps: S301. Based on the application scenario, assign a first weight coefficient to the semantic vector and a second weight coefficient to the behavior vector; S302. Based on the first weight coefficient and the second weight coefficient, perform a weighted concatenation operation on the semantic vector and the behavior vector to generate the panoramic feature vector.

[0052] In this embodiment, weighted stitching can be implemented using preset fixed weights. The semantic vector weight is 0.6 by default, and the behavior vector weight is 0.4 by default, and can be adjusted according to the actual application scenario. Each element in the semantic vector is multiplied by its semantic vector weight, and each element in the behavior vector is multiplied by its behavior vector weight. The two weighted vectors are then stitched together end-to-end to generate a panoramic feature vector.

[0053] In another embodiment, the weighted splicing can also be implemented as follows: First, calculate the variance of each dimension value of the semantic vector, denoted as the first variance, and calculate the variance of each dimension value of the behavior vector, denoted as the second variance. Calculate the proportion of the first variance to the sum of the first and second variances, and use this proportion as the first weighting coefficient; calculate the difference between 1 and the first weighting coefficient, and use this difference as the second weighting coefficient.

[0054] Then, based on the first weight coefficient and the second weight coefficient, a weighted concatenation operation is performed on the semantic vector and the behavior vector, that is, each element in the semantic vector is multiplied by the first weight coefficient, each element in the behavior vector is multiplied by the second weight coefficient, and the two weighted vectors are concatenated end to end to generate the panoramic feature vector.

[0055] Ultimately, all the generated panoramic feature vectors, along with their associated unique user identifiers, session identifiers, and feedback timestamps, together form the fused dataset for the next step of mining.

[0056] S400. Perform cluster analysis on the fused dataset, and adaptively adjust the clustering parameters according to the density distribution characteristics of the data in the fused dataset to extract high-frequency problem patterns composed of sample clusters with the same feedback characteristics, thereby forming incremental mining results.

[0057] In this embodiment, to address the issue of uneven density in feedback feature data, an improved density-based unsupervised clustering algorithm is employed to process the fused dataset. The core advantage of this algorithm lies in its ability to dynamically adjust parameters based on the density characteristics of the data itself, without pre-setting the number of clusters. This allows it to identify sample clusters of arbitrary shapes with significant density differences, which is particularly crucial for discovering sparse but important new problem feedback patterns.

[0058] Please see Figure 2 , Figure 2 This is a flowchart illustrating a method for incremental mining of user feedback information in an AI question-answering agent according to a specific embodiment of the present invention. In one embodiment, step S400 includes the following steps: S401. An improved DBSCAN clustering algorithm is adopted to adaptively adjust the neighborhood radius and the minimum number of contained points by calculating the local density of the data in the fused dataset. S402. Based on the adjusted neighborhood radius and the minimum number of contained points, the fused dataset is clustered to form multiple sample clusters; S403. Summarize the features of each sample cluster to generate a textual description and unique feature identifier of the corresponding problem pattern, so as to constitute the high-frequency problem pattern; S404. The sample clusters after feature summarization are used as the incremental mining results.

[0059] In this embodiment, the clustering analysis process is implemented using an improved DBSCAN clustering algorithm, which can adaptively identify high-frequency problem patterns, uncover the incremental value hidden in the feedback data, improve the utilization rate of feedback information, and avoid the problem of poor clustering effect caused by fixed parameters in the traditional DBSCAN algorithm.

[0060] In one specific embodiment, step S401 includes the following steps: S4011. Calculate the distance from each data point in the fused dataset to its k-th nearest neighbor, where k is the initial value of the preset minimum number of points. Sort the k-th nearest neighbor distances of all data points in ascending order, plot the distance curve, and use the distance value corresponding to the inflection point with the largest curvature in the curve as the reference value of the global neighborhood radius.

[0061] Specifically, suppose there are N data points in the fused dataset. For each data point p, calculate its distance to the k-th nearest neighbor, denoted as d. k (p), where k is the initial value of the preset minimum number of points MinPts. This involves assigning d to all N data points. k (p) Sort the data in ascending order to generate a k-distance sequence and plot the k-distance curve. Take the distance value corresponding to the inflection point with the largest curvature in the curve as the baseline value of the global neighborhood radius, denoted as ε0. The initial value of ε0 is 0.5, and the initial value of k is 10.

[0062] S4012. For each data point, calculate the proportion of the number of data points in its neighborhood to the total number of points in the dataset, and use this as the local density of the data point and the average local density of the dataset.

[0063] Specifically, for each data point p, its neighborhood is determined with radius ε0, and the number of data points contained within this neighborhood is calculated, denoted as |N(p)|. Then, the local density ρ(p) of data point p is: ρ(p) = |N(p)| / N; Where |N(p)| is the number of data points in the neighborhood centered at p with a radius of ε0, and N is the total number of points in the dataset.

[0064] Calculate the arithmetic mean of the local densities of all data points, and use this as the average local density, denoted as ρ. avg : ρ avg =(1 / N) × ; S4013. The ratio of the local density of the data point to the average local density of the dataset is used as the density adjustment factor. The global neighborhood radius benchmark value of the data point is divided by the density adjustment factor to obtain the adaptively adjusted neighborhood radius of the data point.

[0065] Specifically, for each data point p, its density adjustment factor λ(p) is: λ(p) = ρ(p) / ρ avg ; When λ(p)>1, it indicates that the data point is in a relatively dense region; when λ(p)<1, it indicates that the data point is in a relatively sparse region.

[0066] For each data point p, its adaptively adjusted neighborhood radius ε(p) is: ε(p) = ε0 / λ(p); That is, it automatically reduces the neighborhood radius in dense areas to improve resolution, and automatically expands the neighborhood radius in sparse areas to enhance connectivity.

[0067] S4014. Multiply the density adjustment factor of the data point by the preset initial value of the minimum number of included points, and round the product up to obtain the minimum number of included points after adaptive adjustment of the data point.

[0068] Specifically, for each data point p, its adaptively adjusted minimum number of points, MinPts(p), is: MinPts(p) = ⌈λ(p) × k⌉; Where k is the preset initial value of the minimum number of contained points, and ⌈·⌉ represents rounding up. That is, the minimum number of contained points is appropriately increased in dense regions to enhance noise resistance, and the requirement is appropriately decreased in sparse regions to avoid missing valid data points.

[0069] Clustering is performed based on the adjusted parameters, aggregating data points with sufficiently high density of connections into a sample cluster. Each formed sample cluster represents a type of user feedback with similar feedback characteristics.

[0070] Specifically, based on the adaptively adjusted ε(p) and MinPts(p) of each data point p, clustering is performed, and data points that meet the density attainability condition are grouped into the same cluster to form η sample clusters.

[0071] Finally, feature summarization was performed on each sample cluster. All the summarized sample clusters together constitute the result of this incremental mining.

[0072] In one embodiment, step S403, which involves summarizing features for each sample cluster to generate a textual description and unique feature identifier for the corresponding problem pattern, includes the following steps: S4031. The panoramic feature vectors in the sample cluster are reverse-analyzed using a preset large language model to generate a textual description and unique feature identifier of the problem pattern.

[0073] In this embodiment, the panoramic feature vectors in the sample cluster are reverse-analyzed using a preset large language model to extract the common information they represent, and a natural language text description and unique feature identifier for the problem pattern are generated, thereby forming a high-frequency problem pattern, which is convenient for subsequent manual identification and processing.

[0074] Among them, the large language model can be a generative pre-trained language model based on a multi-layer Transformer decoder stack. It captures the dependencies between lexical units in the input sequence through a multi-head self-attention mechanism, performs non-linear transformations using a feedforward neural network, and predicts and generates output text word by word using an autoregressive approach. The input of this model is the original parsed text extracted from the sample cluster, and the output is a summarized problem pattern text description and a unique feature identifier.

[0075] In one specific embodiment, step S4031 specifically includes the following steps: S40311. Extract representative feature vectors from the sample cluster; S40312. Map the representative feature vector to the natural language input of the large language model; S40313. Input the original parsed text into the large language model to generate a standardized problem pattern description and a unique feature identifier.

[0076] In this embodiment, firstly, for each sample cluster formed by clustering, the geometric center of all panoramic feature vectors within the sample cluster is calculated, that is, the arithmetic mean of the corresponding dimensional values ​​of all data points in each dimension is calculated, and this geometric center is used as the baseline feature vector of the sample cluster. Alternatively, the panoramic feature vector corresponding to the actual data point closest to the geometric center is selected as the representative feature vector to ensure that the vector corresponds to a real user feedback instance.

[0077] Then, since the panoramic feature vector is a high-dimensional numerical vector formed by weighted concatenation of semantic and behavioral vectors, large language models cannot directly process this type of vector. Therefore, it is necessary to deconstruct the representative feature vector, specifically by splitting the representative feature vector according to the dimension at the time of concatenation, and restoring the corresponding semantic and behavioral vector parts.

[0078] For the semantic vector part, the inverse mapping mechanism of the pre-trained text semantic model is used to map the semantic vector to a word probability distribution. The words with the highest probability and their combinations are selected to form an approximate description of the semantic content of the feedback text.

[0079] For the behavior vector part, it is decoded in reverse according to the preset encoding rules in the behavior encoding algorithm. For example, the statistical value part is directly restored to the corresponding frequency value; the transition probability matrix part is restored to the operation sequence pattern description; and the binary encoding part is restored to the yes or no behavior description, thereby generating a textual description of the user's operation behavior.

[0080] The approximate description of the semantic content and the textual description of the operational behavior mentioned above are combined using natural language to form the original parsed text of this sample cluster.

[0081] Finally, the original parsed text is input into the pre-trained large language model, and a pre-defined prompt word template is used to guide the large language model to perform an inductive task. This prompt word template includes the following instructions: The core question types from user feedback are summarized from the original parsed text, and a standard question pattern text description of no more than a preset number of characters is generated. This description must cover the question phenomenon and the relevant intelligent agent functional modules. At the same time, a unique feature identifier is generated, which consists of a question type code and a sequence number.

[0082] Based on the instructions of the prompt word template, the large language model performs semantic understanding and summarization on the original parsed text, and outputs a standardized textual description of the question pattern and a unique feature identifier, thus forming a complete high-frequency question pattern.

[0083] In an alternative embodiment, it can also be implemented in a more direct manner as follows: From each sample cluster, select several data points closest to the geometric center of the cluster, for example, 5 to 10. Directly obtain the original active feedback text and the original records of passive operation trajectories corresponding to these data points. Input all these original texts and records as examples into the large language model. Utilize the text induction capability of the large language model to directly generate a common problem description of the user feedback problem represented by the sample cluster, and the large language model automatically generates a unique feature identifier for this problem pattern.

[0084] This approach eliminates the need for complex decoding of panoramic feature vectors. Instead, it leverages the semantic understanding and inductive capabilities of large language models to extract common information from the raw data, thereby achieving feature induction.

[0085] It should be noted that the pre-trained text semantic model used in step S201 is primarily a Transformer encoder, fine-tuned for AI question-answering feedback scenarios. Its function is to forward encode the actively fed-back text into numerical semantic vectors, serving subsequent feature fusion and clustering analysis. The large language model used in step S4031 is primarily a generative large language model. Its function is to reverse-parse the formed clusters, summarizing the numerical features or associated original text into human-readable textual descriptions and unique feature identifiers. These two models play different roles in this invention: the former is a feature encoder, completing the transformation from text to vectors; the latter is a semantic decoder, completing the transformation from vectors or data to natural language. They are sequentially connected but independent in the technical process.

[0086] In another embodiment, the clustering analysis in step S400 can also be implemented using the mean-drift clustering algorithm. In this scheme, the mean-drift clustering algorithm uses the local density estimate of each data point as a weight, guiding each data point to iteratively move in the direction of the fastest density growth. For each data point, the density-weighted average of all points in its neighborhood is calculated as its new position. This neighborhood radius is adaptively determined based on the local density of the data point; a smaller neighborhood radius is used in areas of dense data to maintain resolution, while a larger neighborhood radius is used in areas of sparse data to ensure convergence stability. All data points that converge to the same local density maxima are grouped into the same cluster.

[0087] In another embodiment, the clustering analysis in step S400 can also be implemented using hierarchical clustering combined with density analysis. In this approach, the distance between every two data points is first calculated to construct a distance matrix. Then, each data point is used as an initial cluster, and clusters that are closest in distance and have similar densities are gradually merged. The density difference between two clusters is used as part of the merging condition; merging is only allowed when the density decrease in the boundary region between the two clusters is below a set threshold, thereby avoiding the erroneous merging of independent clusters with clear density boundaries.

[0088] Furthermore, before generating the incremental mining results, the method further includes the following steps: S405. The sample clusters obtained by clustering are screened, and low-frequency sample clusters with a sample size ratio lower than a first preset threshold are removed. S406. Eliminate invalid sample clusters that are unrelated to the core performance of the AI ​​question-answering agent; S407. The retained sample clusters are identified as effective high-frequency problem patterns to form the incremental mining results.

[0089] Before forming the final incremental mining results, the initial sample clusters obtained from clustering can be screened. The first preset threshold for preliminary screening can be adjusted according to actual business needs. For example, sample clusters with a sample size ratio of less than 1% are considered low-frequency sample clusters, and sample clusters that are unrelated to the core performance of the AI ​​question-answering agent (such as intent recognition, answer accuracy, and interactive experience) are considered invalid sample clusters. Low-frequency patterns with a sample size ratio of less than the first preset threshold (such as 1%) and invalid patterns that are high-frequency but unrelated to the core question-answering performance of the agent are removed. Only valid high-frequency question patterns with optimization value are retained for the next step.

[0090] S500. Based on the incremental mining results, the high-frequency problem patterns are quantitatively evaluated, and an optimization priority list is generated.

[0091] In this embodiment, in order to focus limited development resources on the most critical issues, it is necessary to prioritize the multiple high-frequency problem patterns that have been identified.

[0092] In one specific embodiment, step S500 includes the following steps: S501. From the incremental mining results, obtain the sample size ratio, user influence range, and agent performance correlation of the high-frequency problem pattern; S502. The weighted calculation of the sample size ratio, the user influence range and the intelligent agent performance correlation is performed to obtain the optimization priority score; S503. Sort the optimization priority scores from high to low to generate the optimization priority list.

[0093] The optimization priority score for each high-frequency problem pattern is obtained by weighted calculation. The optimization priority list is generated by sorting the scores from high to low to ensure that optimization resources are tilted towards core and high-frequency problems.

[0094] The quantitative evaluation process extracts indicators from three dimensions: the sample size proportion of high-frequency problem patterns, the scope of user influence, and the correlation between agent performance and the data.

[0095] The proportion of the sample size of the high-frequency question pattern in the entire incremental mining results reflects its universality; the proportion of the number of independent users involved in the question pattern to the total number of users, i.e. the scope of user influence, reflects its impact; the correlation score between the question pattern and the core performance of the agent (such as intent recognition accuracy and response accuracy), i.e. the correlation of agent performance, reflects its importance.

[0096] The sample size percentage and user influence range are calculated proportions based on actual statistics, ranging from 0 to 1. The agent performance correlation score can be scored from 1 to 10, with higher scores indicating a stronger correlation between the problem pattern and the agent's core performance. Before weighted calculation, the agent performance correlation score is divided by 10 to normalize it to between 0 and 1, ensuring consistency with the sample size percentage and user influence range.

[0097] The quantitative evaluation is conducted using a pre-defined weighting formula, which is as follows: Optimization priority score = (sample size percentage × first weight) + (user influence range × second weight) + (agent performance correlation × third weight).

[0098] Among them, the first weight, the second weight, and the third weight are all preset constants, and the sum of the three is 1.

[0099] For example, the first weight can be set to 0.4, the second weight to 0.3, and the third weight to 0.3. Then the optimization priority score = sample size percentage × 0.4 + user influence range × 0.3 + agent performance correlation × 0.3.

[0100] Finally, all high-frequency problem patterns are sorted from highest to lowest according to this score, generating a clear list of optimization priorities.

[0101] S600. Based on the optimization priority list, establish a mapping relationship between the high-frequency question patterns and the optimization actions of the AI ​​question-answering agent to execute the optimization actions; wherein, the mapping relationship is a preset optimization operation that maps the question category corresponding to the high-frequency question pattern to the corresponding component in the intent recognition model, knowledge base, answer generation module or interactive interface of the AI ​​question-answering agent.

[0102] In this embodiment, a mapping mechanism needs to be established to automate and accurately implement the mining results into optimization actions.

[0103] In one specific embodiment, step S600 includes the following steps: S601. Based on the optimization priority list, determine the high-frequency problem patterns to be processed from the incremental mining results; S602. Based on the problem category corresponding to the high-frequency problem pattern, establish a mapping relationship between the high-frequency problem pattern and at least one preset optimization action; S603. Based on the mapping relationship, generate optimization instructions and send them to the corresponding module of the AI ​​question-answering agent to execute the preset optimization action.

[0104] First, based on the generated optimization priority list, determine the high-frequency problem patterns that need to be addressed. Then, based on the specific category to which this high-frequency problem pattern belongs, establish its association with at least one agent's optimization action.

[0105] In this embodiment, the mapping relationship can be established using a pre-set fixed mapping table. A mapping table between a problem mode category and optimization actions is pre-constructed, and this mapping table includes the following: When the question pattern is that the AI ​​question-answering agent fails to accurately identify the user's core question intent, the corresponding intent recognition model fine-tuning action and question intent guidance rule optimization action are performed. When the question pattern is that the AI ​​question-answering agent's answer lacks authoritative basis, the corresponding knowledge base will be updated with targeted content and the verification rules for the source of the answer information will be adjusted. When the question pattern manifests as the AI ​​question-answering agent's logically confused or semantically biased answers, the corresponding large language model's prompt word optimization action and answer generation rule adjustment action will be taken. When the interaction path of the AI ​​question-answering agent is too cumbersome or the negative feedback entry is hidden, the layout optimization of the front-end functional interaction interface and the position adjustment of the feedback entry should be performed accordingly.

[0106] When a mapping needs to be established, the system queries the corresponding relationship table and directly matches the corresponding preset optimization action based on the problem category of the high-frequency problem pattern.

[0107] Correspondingly, the mapping relationship includes at least one of the following: When the high-frequency question pattern is that the AI ​​question-answering agent fails to recognize the core question intent, a mapping relationship is established between the high-frequency question pattern and the fine-tuning actions of the intent recognition model and the optimization actions of the question intent guidance rules. When the high-frequency question pattern indicates that the AI ​​question-answering agent's answer lacks authoritative basis, a mapping relationship is established between the high-frequency question pattern and the knowledge base supplementation and update actions and the answer information source verification rule adjustment actions. When the high-frequency question pattern is that the AI ​​question-answering agent's answer is logically confused or semantically biased, a mapping relationship is established between the high-frequency question pattern and the large model prompt word optimization action and the answer generation rule adjustment action; When the high-frequency question pattern is characterized by cumbersome interactive operations or hidden feedback entry points of the AI ​​question-answering agent, a mapping relationship is established between the high-frequency question pattern and the actions of optimizing the functional interaction interface and adjusting the layout of the feedback entry points.

[0108] In another embodiment, the mapping relationship can also be established using an automatic recognition method based on a classification model. In this approach, a multi-label classification model is pre-trained. The model's input consists of a textual description of the problem pattern and a unique feature identifier, and its output consists of one or more optimized action labels corresponding to that problem pattern. When a mapping needs to be established, the textual description of the high-frequency problem pattern is input into the classification model, and the model outputs a set of corresponding optimized action labels. The system then establishes the mapping relationship based on this label set.

[0109] In another embodiment, the mapping relationship can also be established using a rule engine-based matching method. In this scheme, a set of matching rules is predefined, each rule containing a conditional description of the problem pattern and a corresponding optimization action. For example, a rule's condition might be that the textual description of the problem pattern contains keywords such as intent, recognition failure, or misunderstanding. When a high-frequency problem pattern meets this condition, the rule engine maps the problem pattern to the intent recognition model's fine-tuning action. When a mapping needs to be established, the system iterates through all rules, summarizes the optimization actions corresponding to all rules that meet the condition, and establishes the mapping relationship.

[0110] Once the mapping is established, specific and executable optimization instructions are generated based on the mapping relationship and automatically sent to the corresponding engine or configuration module of the AI ​​question-answering agent to complete the execution of the optimization action.

[0111] Specifically, each preset optimization action has a corresponding instruction template defined in advance. The instruction template is in a structured data format and includes the operation object, operation type, parameter list and priority identifier.

[0112] For the fine-tuning action of the intent recognition model, the operation object is the intent recognition module of the agent. The sample identifier list is obtained from the incremental mining results, and the question text of the corresponding session is extracted from the original feedback dataset as training samples and filled into the parameters of the model fine-tuning API.

[0113] For knowledge base replenishment and update actions, the operation target is the agent's knowledge base management module, which extracts frequently occurring entity words or factual information that are missing or outdated in the knowledge base from the original feedback text of the corresponding sample cluster and fills them into the parameters of the knowledge base update API.

[0114] For the large model prompt word optimization action, the operation target is the agent's answer generation module, which extracts question feature keywords from the text description of the corresponding sample cluster and fills the keywords into the constraint condition field of the prompt word template.

[0115] For the optimization of the functional interaction interface, the operation target is the front-end rendering engine of the agent. The user operation interruption point is extracted from the behavior vector decoding result of the corresponding sample cluster, and the interface element identifier and adjustment parameters corresponding to the interruption point are filled into the parameters of the interface configuration API.

[0116] Then, the instruction template with filled parameters is encapsulated into a structured optimization instruction. Each optimization instruction includes an instruction ID, target module identifier, operation function name, parameter structure, execution priority, and timestamp.

[0117] Based on the target module identifier marked in the instruction, the above structured optimization instructions are sent to the corresponding module of the AI ​​question-answering agent through internal API calls or message queues.

[0118] For instructions targeting the intent recognition module, the delivery channel is the model training platform API, the execution method is asynchronous triggering of fine-tuning tasks, and the model version is automatically updated after training is completed.

[0119] For instructions targeting the knowledge base management module, the distribution channel is the knowledge base update API, and the execution method is to synchronously write new data or update existing data entries.

[0120] For instructions targeting the answer generation module, the distribution channel is the prompt word configuration center API, and the execution method is real-time update.

[0121] For instructions targeting the front-end rendering engine, the distribution channel is the interface configuration distribution channel, and the execution method is to make them fully effective after a gray-scale release.

[0122] Upon receiving the optimization command, each target module parses the parameter structure within the command, executes the optimization actions according to the predefined operation logic, and returns an execution status code. If execution is successful, an execution log is recorded, and the process enters the closed-loop verification phase; if execution fails, an alarm is triggered, and the system automatically rolls back to the configuration version before optimization.

[0123] For example, the incremental mining results generated after cluster analysis and feature summarization contain the following high-frequency problem patterns: The first high-frequency question pattern is identified as P001, described as follows: The agent fails to accurately recognize the user's intention to ask a question when the core entity is omitted. For example, if the user only enters "How do I do that?" without specifying the target, the agent cannot connect the context to complete the entity information. This question pattern corresponds to a sample cluster containing 1247 data points, accounting for 8.2% of the total valid samples, involving 986 users.

[0124] The second high-frequency question pattern is characterized by P002, described as follows: the agent provides outdated answers to questions involving real-time data. For example, when a user asks "What is today's gold price?", the agent cites data from the previous trading day, lacking authority and timeliness. This question pattern corresponds to a sample cluster containing 968 data points, accounting for 6.4% of the total valid samples, involving 812 users.

[0125] The third high-frequency question pattern is characterized by P003, described as follows: the agent's multiple responses are contradictory; in one round, the agent confirms the user as a "Platinum Member," but in the next round, it recommends services only available to "Regular Members," demonstrating a significant deviation in logical consistency. This question pattern corresponds to a sample cluster containing 735 data points, accounting for 4.8% of the total valid samples, involving 623 users.

[0126] The fourth high-frequency problem pattern is characterized by P004, described as follows: When users provide negative feedback to an answer using the intelligent agent, they need to navigate through multiple menu levels to submit a reason for disliking the answer. Furthermore, the negative feedback entry icon is too small, and the cumbersome interaction path leads users to abandon providing feedback. This problem pattern corresponds to a sample cluster containing 589 data points, accounting for 3.9% of the total valid samples, involving 501 users.

[0127] The first high-frequency problem pattern, P001, was identified as a problem where the agent failed to accurately identify the user's core question intent. After querying the mapping table, two preset optimization actions were found: intent recognition model fine-tuning and question intent guidance rule optimization. The intent recognition model fine-tuning instruction template was invoked, extracting the user's abbreviated question and its context from the raw feedback data associated with the sample cluster corresponding to P001. A fine-tuning training set was constructed, filled into the model fine-tuning API parameters, and the generated instruction was sent to the model training platform, triggering an asynchronous fine-tuning task. Simultaneously, the question intent guidance rule optimization instruction template was invoked, filling the specific rule items of the context entity inheritance rule into the rule update API parameters. For example, when it was detected that the user input lacked a core entity and corresponding entity information existed in the current session history, the entity information from the historical context was automatically filled into the intent recognition input, and a confirmation prompt was generated. The generation instruction was then sent to the dialogue management module to update the rule configuration in real time.

[0128] The second high-frequency problem pattern P002 is determined to be a problem category where the agent's answer lacks authoritative basis. After querying the corresponding relationship table, two preset optimization actions are matched: a targeted content supplementation and update action in the knowledge base and an action to adjust the verification rules of the source of the answer information.

[0129] For the targeted content supplementation and update of the knowledge base, frequently occurring real-time data query categories were extracted from the original feedback text of the sample cluster corresponding to P002, including 12 categories such as "gold price," "exchange rate," "stock price," "weather," and "oil price." Retrieving existing data entries for these 12 query categories from the knowledge base revealed that 8 categories lacked authoritative data source configurations, and the data sources for the other 4 categories had expired. The knowledge base update API was called to add a "Real-time Data Source Mapping Table" to the knowledge base, binding the corresponding authoritative data interface addresses and update frequencies for each of the 12 query categories. For example, the "gold price" category was bound to the real-time market data interface of a gold trading platform, with an update frequency of once per minute; the "exchange rate" category was bound to the exchange rate interface of an exchange rate data platform, with an update frequency of once every five minutes; and the "weather" category was bound to the real-time weather interface of a meteorological data platform, with an update frequency of once per hour.

[0130] Regarding the adjustment to the verification rules for the source of answer information, a new timeliness verification rule has been added to the verification rule library of the answer generation module. The trigger condition for this rule is as follows: when a user's question involves any of the 12 real-time query categories mentioned above, the answer generation module must first call the corresponding authoritative data interface to obtain the latest data, and compare the timestamp of the obtained data with the existing data in the knowledge base. Only if the update time of the data returned by the interface is later than the update time of the data stored in the knowledge base will the interface data overwrite the knowledge base data and an answer be generated. This rule also requires that the data source and update time be appended at the end of the generated answer.

[0131] The structured optimization instructions for the two optimization actions mentioned above are generated respectively, and then sent to the corresponding modules for execution via the knowledge base update API and the prompt word configuration center API.

[0132] The third high-frequency problem pattern P003 is determined to be a problem category where the agent's answer logic is chaotic or there is semantic deviation. After querying the corresponding relationship table, two preset optimization actions are matched: prompt word optimization action and answer generation rule adjustment action of the large language model.

[0133] For the prompt word optimization action of the large language model, common features causing contradictions were extracted from the text descriptions and original dialogue records of the sample cluster corresponding to P003. Analysis revealed that the contradictions mainly focused on the references to user profile attributes such as user identity information, membership level, and historical order status. Typical contradiction cases were extracted from these dialogue records and transformed into negative examples in the prompt words. A consistency constraint instruction was added to the existing prompt word template. The specific content of this instruction is as follows: Before generating a response, the latest user profile information must be retrieved, including attributes such as membership level, historical orders, and usage preferences. When referencing user profile attributes in the response, it must be consistent with the latest retrieved information, and the basis for the reference must be clearly stated in the response, such as "Based on your current Platinum membership level, you can enjoy the following benefits." When user profile information changes during the conversation, the response must be regenerated based on the changed information.

[0134] Regarding the adjustment of the answer generation rules, a user profile status snapshot mechanism has been added to the multi-turn dialogue management module. The specific rules of this mechanism are as follows: at the start of each session, the complete profile information of the current user is automatically loaded from the user profile service, generating a user profile status snapshot for that session, which is stored in the session context. During the session, whenever a change in user profile information is detected, such as a membership level upgrade or order status update, the user profile status snapshot in the session context is automatically refreshed, and all subsequent answers are based on the refreshed snapshot information. Simultaneously, a consistency check submodule has been added to the output validation stage of the answer generation module. This submodule compares all user profile attributes referenced in the answer with the corresponding attributes in the current session snapshot one by one after the answer text is generated and before being returned to the user. If an inconsistency is detected, regeneration is automatically triggered.

[0135] The structured optimization instructions for the two optimization actions mentioned above are generated respectively and sent to the answer generation module for execution via the prompt word configuration center API.

[0136] The fourth high-frequency problem pattern P004 was identified as having an overly cumbersome interactive operation path for the agent or a hidden negative feedback entry design. After querying the corresponding relationship table, two preset optimization actions were found: layout optimization action for the front-end functional interaction interface and position adjustment action for the feedback entry.

[0137] For the layout optimization of the front-end functional interface, user operation interruption points were extracted from the behavior vector decoding results of the sample cluster corresponding to P004. Analysis revealed that user interruption behavior mainly concentrated in two stages: first, in the multi-level menu that popped up after clicking the "dislike" button, over 60% of users abandoned the operation at the second-level menu; second, on mobile devices, the clickable hotspot area of ​​the negative feedback entry icon was too small, leading to a high rate of accidental touches. To address these issues, the interface configuration API was called to generate the following interface layout optimization instructions: An operation bar was added directly below each agent's answer, containing four commonly used feedback icon buttons: "Like," "Dislike," "Copy," and "Share." The icon size was set to 48×48 pixels, and the spacing between adjacent icons was set to 16 pixels. After the user clicked the "Dislike" button, instead of a multi-level menu popping up, a layered overlay panel directly expanded at the current location. This panel contained rating controls from 1 to 5 stars and a feedback text box of up to 30 characters. The submit button was located at the bottom of the panel, allowing users to complete feedback submission in just two steps.

[0138] For the adjustment of the feedback entry point location, the adjustment plan is synchronized to the front-end rendering engine of the intelligent agent, and A / B testing parameters are set. The specific A / B testing configuration is as follows: first, the new layout is rolled out to 20% of the user group, while the remaining 80% of users continue to use the old interface. After continuously collecting comparative data for 7 days, the feedback submission rate of the new and old interfaces is automatically calculated. If the feedback submission rate of the new interface is higher than that of the old interface and the increase exceeds the second preset threshold, such as an increase of more than 30%, the full release process is automatically triggered, and the new interface is pushed to all users; if the increase is lower than the second preset threshold or decreases, the old interface is automatically rolled back, and an optimization invalidation report is generated for manual analysis.

[0139] The structured optimization instructions for the two optimization actions mentioned above are generated respectively and pushed to the front-end rendering engine for execution through the interface configuration distribution channel.

[0140] Further, please refer to Figure 2 To verify the optimization effect and form a complete closed-loop iterative system, the following steps are included after step S603: S604. Continuously collect subsequent user feedback data; S605. For subsequent user feedback data, repeat the steps from constructing a multi-dimensional original feedback dataset to forming incremental mining results. S606. Statistically analyze the changes in the sample size proportion corresponding to the high-frequency problem patterns; S607. If the sample size ratio does not drop to the second preset threshold, the entire process from constructing the multi-dimensional original feedback dataset to executing the optimization action is re-triggered to perform secondary incremental mining and optimization.

[0141] This invention establishes a precise mapping mechanism between incremental mining results and AI question-answering agent optimization actions, and generates an optimization priority list through quantitative evaluation to ensure that optimization resources are tilted towards core and high-frequency issues, thus solving the problem of disconnect between mining results and optimization actions in existing technologies. At the same time, a closed-loop verification mechanism for optimization effects is set up to form a complete closed loop of feedback mining, optimization implementation, and effect verification, ensuring that mining results can be efficiently implemented and achieve the expected optimization effect, greatly improving the iterative optimization efficiency and accuracy of AI question-answering agents.

[0142] Specifically, after implementing the optimization actions in the AI ​​question-answering agent, this embodiment continuously collects a new round of user feedback data, which can be collected over a period of 7 to 30 days. Steps S100 to S400 of this method are repeated to determine whether the sample size percentage of the high-frequency problem patterns targeted in the previous optimization has changed. If this percentage does not drop below a third preset threshold (e.g., 30% of the original percentage), it indicates that the problem has not been effectively resolved. The entire process from data collection to optimization action execution is then automatically retried, and the reasons for unsuccessful optimization are analyzed. Mining and optimization strategies are adjusted, and secondary incremental mining and strategy adjustments are performed until the problem is significantly improved. If the mining results are successfully implemented and the optimization effect meets the standards, the final high-frequency problem patterns and optimization implementation report are output.

[0143] Compared with the prior art, the beneficial effects of the present invention are as follows: By simultaneously collecting users' active feedback text and passive operation trajectory data, and fusing them, a panoramic feature vector that comprehensively represents the user's feedback intent is generated. Based on this, a clustering analysis method that adaptively adjusts parameters according to data density distribution characteristics is employed to incrementally mine high-frequency problem patterns from the fused data. Furthermore, a quantitative evaluation is used to generate an optimization priority list, and a precise mapping and execution mechanism is established between high-frequency problem patterns and specific optimization actions of the intelligent agent. This invention achieves a closed-loop process from feedback data acquisition and in-depth mining to optimization action implementation, solving the problems of single-dimensional data collection, insufficient mining, and disconnection from optimization actions in existing technologies. This significantly improves the utilization value of feedback information and the iterative optimization efficiency of the AI ​​question-answering agent.

[0144] Furthermore, this invention has strong versatility and adaptability, and can be applied to AI question-answering agents in various scenarios. It does not require extensive modifications to specific agents, and the entire process of data collection, fusion, mining, and deployment is configurable. Various parameters and rules can be adjusted according to actual business needs, making it suitable for the full lifecycle iterative optimization of AI question-answering agents and having broad application prospects.

[0145] Please see Figure 3 Based on the same inventive concept, and corresponding to the methods of any of the above embodiments, this invention also discloses an incremental mining device for user feedback information of an AI question-answering agent, comprising: The data acquisition module is used to synchronously collect active feedback text data and passive operation trajectory data of users to the AI ​​question-answering agent, bind the active feedback text data and the passive operation trajectory data to the user's unique identifier and session identifier, record the feedback timestamp, and construct a multi-dimensional raw feedback dataset. The feature extraction module is used to extract semantic features from the active feedback text data to obtain a semantic vector, and to extract behavioral features from the passive operation trajectory data to obtain a behavioral vector with the same dimension as the semantic vector. The feature fusion module is used to weight and concatenate the semantic vector and the behavior vector of the same user and the same session to generate a panoramic feature vector, and to combine the panoramic feature vector, the user's unique identifier, the session identifier and the feedback timestamp to form a fusion dataset. The incremental mining module is used to perform cluster analysis on the fused dataset. It adaptively adjusts the clustering parameters according to the density distribution characteristics of the data in the fused dataset to extract high-frequency problem patterns composed of sample clusters with the same feedback characteristics, thereby forming incremental mining results. The quantitative evaluation module is used to quantitatively evaluate the high-frequency problem patterns based on the incremental mining results and generate an optimization priority list. The mapping execution module is used to establish a mapping relationship between the high-frequency question patterns and the optimization actions of the AI ​​question-answering agent based on the optimization priority list, so as to execute the optimization actions; wherein, the mapping relationship is a preset optimization operation that maps the question category corresponding to the high-frequency question pattern to the corresponding component in the intent recognition model, knowledge base, answer generation module or interactive interface of the AI ​​question-answering agent.

[0146] The apparatus described above is used to implement the incremental mining method for user feedback information of the corresponding AI question-answering agent in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0147] Please see Figure 4Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this embodiment of the invention also discloses a terminal, including: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements a method for incremental mining of user feedback information of an AI question-answering agent as described in any of the above embodiments.

[0148] Specifically, the device includes: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected within the device via the bus 1050.

[0149] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0150] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0151] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device or connected externally to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0152] The communication interface 1040 is used to connect the communication module to enable communication and interaction between this device and other devices. The communication module can communicate via wired means (such as USB (Universal Serial Bus), network cable, etc.) or wireless means (such as mobile network, WIFI (Wireless Fidelity), Bluetooth, etc.).

[0153] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0154] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0155] The terminal in the above embodiments is used to implement the incremental mining method of user feedback information of the corresponding AI question-answering agent in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0156] Based on the same inventive concept, corresponding to any of the above embodiments, this invention also discloses a non-transitory computer-readable storage medium that stores computer instructions for causing a computer to execute the incremental mining method for user feedback information of the AI ​​question-answering agent described above.

[0157] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0158] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the incremental mining method of user feedback information of the AI ​​question-answering agent as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0159] The above description is merely a preferred embodiment of the present invention and the technical principles employed. The present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions that can be made by those skilled in the art will not depart from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the claims.

Claims

1. A method for incrementally mining user feedback information in an AI question-answering agent, characterized in that, include: Simultaneously collect user's active feedback text data and passive operation trajectory data to the AI ​​question-answering agent, bind the active feedback text data and the passive operation trajectory data to the user's unique identifier and session identifier, and record the feedback timestamp to construct a multi-dimensional raw feedback dataset; Semantic features are extracted from the active feedback text data to obtain a semantic vector, and behavioral features are extracted from the passive operation trajectory data to obtain a behavioral vector with the same dimension as the semantic vector. The semantic vector and the behavior vector of the same user and the same session are weighted and concatenated to generate a panoramic feature vector. The panoramic feature vector, the user's unique identifier, the session identifier, and the feedback timestamp are used to form a fusion dataset. Cluster analysis is performed on the fused dataset, and the clustering parameters are adaptively adjusted according to the density distribution characteristics of the data in the fused dataset to extract high-frequency problem patterns composed of sample clusters with the same feedback characteristics, forming incremental mining results; Based on the incremental mining results, the high-frequency problem patterns are quantitatively evaluated, and an optimization priority list is generated. Based on the optimization priority list, a mapping relationship is established between the high-frequency question patterns and the optimization actions of the AI ​​question-answering agent to execute the optimization actions; wherein, the mapping relationship is a preset optimization operation that maps the question category corresponding to the high-frequency question pattern to the corresponding component in the intent recognition model, knowledge base, answer generation module or interactive interface of the AI ​​question-answering agent.

2. The method according to claim 1, characterized in that, The synchronous collection of user-initiated text feedback data and passive operation trajectory data to the AI ​​question-answering agent includes: Collect the active feedback text data, which includes the user's reasons for disliking or rejecting the AI ​​question-answering agent's answer, the feedback content, and the function suggestion text. Collect the passive operation trajectory data, which includes session operation behavior, interaction duration characteristics, and content operation characteristics; The conversational actions include repeatedly modifying the question content, repeatedly disliking the same type of answer, continuously asking the same type of question, and the frequency and order of clicking the agent function button; The interaction duration features include the user's dwell time on a single response, the total session duration, and the trigger delay time for feedback operations; The content operation features include users' behaviors such as collecting, sharing, and copying answer content, as well as clicking on content recommended by the agent.

3. The method according to claim 1, characterized in that, The step of extracting semantic features from the active feedback text data to obtain a semantic vector, and extracting behavioral features from the passive operation trajectory data to obtain a behavioral vector with the same dimension as the semantic vector, includes: The proactive feedback text data is transformed into a semantic vector of a preset dimension using a pre-trained text semantic model. The passive operation trajectory data is normalized and encoded using a behavior encoding algorithm, and then transformed into a behavior vector with the same dimension as the semantic vector.

4. The method according to claim 1, characterized in that, The step of weightedly concatenating the semantic vectors and behavioral vectors of the same user and the same session to generate a panoramic feature vector includes: Based on the application scenario, a first weight coefficient is assigned to the semantic vector, and a second weight coefficient is assigned to the behavior vector; Based on the first weight coefficient and the second weight coefficient, a weighted concatenation operation is performed on the semantic vector and the behavior vector to generate the panoramic feature vector.

5. The method according to claim 1, characterized in that, The step involves performing cluster analysis on the fused dataset, adaptively adjusting clustering parameters based on the density distribution characteristics of the data in the fused dataset, to extract high-frequency problem patterns composed of sample clusters with the same feedback characteristics, forming incremental mining results, including: An improved DBSCAN clustering algorithm is used to adaptively adjust the neighborhood radius and minimum number of contained points by calculating the local density of the data in the fused dataset. Based on the adjusted neighborhood radius and the minimum number of contained points, the fused dataset is clustered to form multiple sample clusters; For each of the sample clusters, feature summarization is performed to generate a textual description and a unique feature identifier for the corresponding problem pattern, thereby constituting the high-frequency problem pattern; The sample clusters after feature summarization are used as the incremental mining results.

6. The method according to claim 5, characterized in that, The step of summarizing features for each sample cluster to generate a textual description and unique feature identifier for the corresponding problem pattern includes: By reverse parsing the panoramic feature vectors in the sample cluster using a pre-defined large language model, a textual description and unique feature identifier of the problem pattern are generated.

7. The method according to claim 1, characterized in that, The step of quantitatively evaluating the high-frequency problem patterns based on the incremental mining results and generating an optimization priority list includes: From the incremental mining results, obtain the sample size proportion, user influence range, and agent performance correlation of the high-frequency problem patterns; The optimization priority score is obtained by weighting the sample size ratio, the user influence range, and the correlation between the agent performance and the user's influence. The optimization priority list is generated by sorting the optimization priority scores from high to low.

8. The method according to claim 1, characterized in that, The step of establishing a mapping relationship between the high-frequency question patterns and the optimization actions of the AI ​​question-answering agent based on the optimization priority list includes: Based on the optimization priority list, high-frequency problem patterns to be processed are determined from the incremental mining results; Based on the problem category corresponding to the high-frequency problem pattern, establish a mapping relationship between the high-frequency problem pattern and at least one preset optimization action; the mapping relationship includes at least one of the following: When the high-frequency question pattern is that the AI ​​question-answering agent fails to recognize the core question intent, a mapping relationship is established between the high-frequency question pattern and the fine-tuning actions of the intent recognition model and the optimization actions of the question intent guidance rules. When the high-frequency question pattern indicates that the AI ​​question-answering agent's answer lacks authoritative basis, a mapping relationship is established between the high-frequency question pattern and the knowledge base supplementation and update actions and the answer information source verification rule adjustment actions. When the high-frequency question pattern is that the AI ​​question-answering agent's answer is logically confused or semantically biased, a mapping relationship is established between the high-frequency question pattern and the large model prompt word optimization action and the answer generation rule adjustment action; When the high-frequency question pattern is that the AI ​​question-answering agent's interactive operation is cumbersome or the feedback entry is hidden, a mapping relationship is established between the high-frequency question pattern and the functional interaction interface optimization action and the feedback entry layout adjustment action. Based on the mapping relationship, optimization instructions are generated and sent to the corresponding modules of the AI ​​question-answering agent to execute the preset optimization actions.

9. A device for incremental mining of user feedback information in an AI question-answering agent, characterized in that, include: The data acquisition module is used to synchronously collect active feedback text data and passive operation trajectory data of users to the AI ​​question-answering agent, bind the active feedback text data and the passive operation trajectory data to the user's unique identifier and session identifier, record the feedback timestamp, and construct a multi-dimensional raw feedback dataset. The feature extraction module is used to extract semantic features from the active feedback text data to obtain a semantic vector, and to extract behavioral features from the passive operation trajectory data to obtain a behavioral vector with the same dimension as the semantic vector. The feature fusion module is used to weight and concatenate the semantic vector and the behavior vector of the same user and the same session to generate a panoramic feature vector, and to combine the panoramic feature vector, the user's unique identifier, the session identifier and the feedback timestamp to form a fusion dataset. The incremental mining module is used to perform cluster analysis on the fused dataset. It adaptively adjusts the clustering parameters according to the density distribution characteristics of the data in the fused dataset to extract high-frequency problem patterns composed of sample clusters with the same feedback characteristics, thereby forming incremental mining results. The quantitative evaluation module is used to quantitatively evaluate the high-frequency problem patterns based on the incremental mining results and generate an optimization priority list. The mapping execution module is used to establish a mapping relationship between the high-frequency question patterns and the optimization actions of the AI ​​question-answering agent based on the optimization priority list, so as to execute the optimization actions; wherein, the mapping relationship is a preset optimization operation that maps the question category corresponding to the high-frequency question pattern to the corresponding component in the intent recognition model, knowledge base, answer generation module or interactive interface of the AI ​​question-answering agent.

10. A terminal, characterized in that, include: The processor, the memory, and the computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements a method for incremental mining of user feedback information of an AI question-answering agent as described in any one of claims 1 to 8.

Images