Method for evaluating importance of large model dialogue based on dynamic weight and storage medium

Abstract

The present application relates to the technical field of artificial intelligence natural language processing, in particular to a large model dialogue importance evaluation method based on dynamic weight, the method comprises the following steps: obtaining a dialogue message to be evaluated, a corresponding dialogue history and a current context, extracting five-dimensional features of time, semantics, emotion, interaction and context; then combining the basic weight of the dialogue stage and the adjustment amount of the message type to generate a dynamic weight matrix in the range of 0-1, and calculating the importance score by weighting; finally, collecting data through a feedback buffer, and dynamically adjusting the learning rate and weight parameters according to the average error; the method is suitable for multiple scenarios and can be used for dialogue window management and historical abstract compression; when the computer program on the storage medium is executed, the above evaluation method can be realized; the method solves the problems of static weight limitation, context fragmentation, timeliness loss and multi-dimensional imbalance in existing dialogue evaluation, and improves the accuracy of dialogue system history management and abstract processing.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence natural language processing technology, specifically relating to a method and storage medium for evaluating the importance of large-scale model dialogues based on dynamic weights. Background Technology

[0002] In practical applications of large language model dialogue systems, due to the fixed context window length limit of the model, when the dialogue history text exceeds this window length, it is necessary to perform summary processing on the historical dialogue to ensure the model's normal response; the accuracy of the summary processing depends on the accurate assessment of the importance of the dialogue content. Currently, in the dialogue importance assessment stage, the industry generally faces the following key technical challenges, and these challenges directly lead to assessment results that fail to meet practical needs, specifically including:

[0003] (1) Limitations of static weights: Traditional methods use fixed weights to evaluate the importance of dialogues, which cannot adapt to different dialogue scenarios;

[0004] (2) Context fragmentation: Ignoring the contextual relevance of the dialogue during evaluation leads to misjudgment of important information;

[0005] (3) Lack of timeliness: The importance weight of historical messages cannot be dynamically adjusted;

[0006] (4) Multidimensional imbalance: Single-dimensional assessment cannot fully reflect the value of dialogue.

[0007] Existing methods for assessing the importance of dialogue all have significant drawbacks. Specifically: the time decay method relies solely on time as the assessment criterion, completely ignoring the value of the dialogue content itself, resulting in a high degree of influence on the assessment results, which contrasts sharply with the requirement of this invention to integrate multiple dimensions; the keyword matching method depends on a pre-built fixed lexicon, exhibiting poor adaptability when the dialogue scenario or domain changes, resulting in a moderate degree of influence, while the dynamic semantic analysis of this invention avoids reliance on a fixed lexicon; the static weight model uses fixed weight coefficients, which cannot be dynamically adjusted with the dialogue, resulting in a high degree of influence, which is precisely the core problem that the dynamic weight calculation process of this invention needs to overcome; and the simple statistical method assesses based solely on surface statistical data, lacking semantic understanding capabilities, resulting in poor accuracy, and a moderate degree of influence.

[0008] In view of this, the present invention is hereby proposed. Summary of the Invention

[0009] To address the aforementioned technical problems in the existing technology, this invention provides a multi-process parameter optimization method. This method solves the problem of not being able to connect independent models to establish a comprehensive model, and can more accurately capture variable interactions in multi-process production processes, thereby significantly improving the overall optimization capability.

[0010] To achieve the above objectives, the technical solution of the present invention is as follows:

[0011] Firstly, a method for evaluating the importance of large-scale dialogue models based on dynamic weights includes:

[0012] S1. Obtain the dialogue message to be evaluated, the dialogue history corresponding to the dialogue message, and the current context, wherein the current context is a preset number of the latest dialogue contents in the dialogue history;

[0013] S2. Extract the multi-dimensional features of the dialogue message, including time features, semantic features, sentiment features, interaction features, and context features;

[0014] S3. Based on the dialogue history, identify the current dialogue stage and determine the basic weights of each dimension. Classify the dialogue messages by message type and determine the weight adjustment amount of each dimension. Merge the basic weights of each dimension with the corresponding weight adjustment amount to generate a dynamic weight matrix. After fusion, the weights of each dimension are limited to the range of 0-1.

[0015] S4. The feature values ​​of each dimension in the multi-dimensional features are weighted and calculated with the weights of the corresponding dimensions in the dynamic weight matrix to obtain the importance score of the dialogue message;

[0016] S5. Collect feedback data containing the predicted importance score and the actual importance score of the dialogue message, calculate the error based on the feedback data and adjust the initial weight parameters using the gradient descent method, and dynamically adjust the learning rate according to the error magnitude to achieve adaptive update of the initial weight parameters.

[0017] Furthermore, the specific process of extracting the time features includes:

[0018] If the dialogue history is empty, the feature value of the time feature is set to 1.0;

[0019] If the dialogue history is not empty, obtain the timestamp of the dialogue message, the timestamp of the latest message in the dialogue history, and the timestamp of the earliest message.

[0020] Calculate the time difference between the dialogue message and the latest message, and the time difference between the latest message and the earliest message, to obtain the feature value of the time feature. The specific calculation formula is as follows:

[0021] ).

[0022] Furthermore, the specific process of extracting the semantic features includes:

[0023] Extract the text content of the dialogue message and the text content of the current context;

[0024] The pre-trained SentenceTransformer model is invoked to encode the text content of the dialogue message and the text content of the current context, respectively, to obtain the message embedding vector and the context embedding vector.

[0025] Calculate the cosine similarity between the message embedding vector and the context embedding vector;

[0026] Extract keywords from the current context text content, count the number of keywords contained in the dialogue message text content, and calculate the ratio of the number of keywords contained to the total number of keywords as the keyword matching rate;

[0027] The feature value of the semantic feature is obtained by weighted summation of the cosine similarity and the keyword matching rate.

[0028] Furthermore, the specific process of extracting the emotional features includes:

[0029] The TextBlob sentiment analysis model is invoked to analyze the text content of the dialogue message and output the sentiment polarity value and sentiment subjectivity value.

[0030] Calculate the sum of the absolute value of the emotional polarity value and the emotional subjectivity value, and take the average of the sum as the feature value of the emotional characteristic.

[0031] Furthermore, the specific process of extracting the interaction features includes:

[0032] Obtain the user identifier corresponding to the dialogue message, count the number of messages sent by the user identifier in the dialogue history, and calculate the ratio of the number of messages sent by the user to the total number of messages in the dialogue history as the user engagement level.

[0033] The number of messages in the dialogue history that are addressed to the user is counted, and the ratio of the number of messages addressed to the user to the total number of messages in the dialogue history is calculated as the response frequency.

[0034] Determine whether the text content of the dialogue message contains a preset question identifier. If it does, set the question identifier value to 1; otherwise, set it to 0.

[0035] Calculate the average of the user engagement, response frequency, and question identifier value, and use it as the feature value of the interaction feature.

[0036] Furthermore, the specific process of extracting the contextual features includes:

[0037] The role type of the dialogue message is obtained, including system role, user role, and assistant role; the role weight value corresponding to the role type is determined according to a preset role weight table.

[0038] The position number of the dialogue message in the current context is determined, and the result is used as the position weight value.

[0039] The average of the role weight value and the position weight value is calculated and used as the feature value of the context feature.

[0040] Furthermore, the dialogue phases include: an initiation phase, a development phase, a climax phase, and a conclusion phase; the basic weights corresponding to the dialogue phases are:

[0041] Initialization phase: Time dimension 0.3, semantic dimension 0.4, emotional dimension 0.1, interaction dimension 0.1, context dimension 0.1;

[0042] Development stages: Time dimension 0.2, Semantic dimension 0.3, Emotional dimension 0.2, Interaction dimension 0.2, Context dimension 0.1;

[0043] Climax phase: Time dimension 0.1, semantic dimension 0.4, emotional dimension 0.3, interaction dimension 0.1, context dimension 0.1;

[0044] Final stage: Time dimension 0.3, semantic dimension 0.3, sentiment dimension 0.2, interaction dimension 0.1, context dimension 0.1;

[0045] The message types include: question type, answer type, clarification type, and summary type; the weight adjustment amount corresponding to each message type is:

[0046] Problem type: Semantic dimension +0.1, Interaction dimension +0.1;

[0047] Response type: Semantic dimension +0.15, Context dimension +0.05;

[0048] Clarification type: Time dimension +0.1, Interaction dimension +0.1;

[0049] Summary type: Semantic dimension +0.2, Context dimension +0.1.

[0050] Furthermore, the specific process of adaptively updating the weight parameters also includes:

[0051] Set up a feedback buffer to store the feedback data collected each time;

[0052] A preset update threshold is set, and when the amount of feedback data stored in the feedback buffer reaches the preset update threshold, a batch update is triggered.

[0053] Calculate the average of the absolute values ​​of the errors of all feedback data in the feedback buffer to obtain the average error;

[0054] If the average error is greater than 0.2, the learning rate will be increased by 10%; if the average error is less than or equal to 0.2, the learning rate will be decreased by 10%.

[0055] The adjusted initial weight parameters are limited to the range of 0.1-0.9. After the adjustment is completed, the feedback buffer is cleared.

[0056] Furthermore, the specific process of obtaining the importance score of the dialogue message includes:

[0057] The feature value of each dimension in the multidimensional feature is multiplied by the weight of the corresponding dimension in the dynamic weight matrix to obtain the weighted score of each dimension.

[0058] Calculate the sum of weighted scores for all dimensions, and the sum of dynamic weights for all dimensions;

[0059] If the sum of the dynamic weights of all dimensions is greater than 0, the ratio of the sum of weighted scores to the sum of weights is calculated and used as the importance score of the dialogue message; if the sum of the dynamic weights of all dimensions is equal to 0, the importance score is set to 0.

[0060] Secondly, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for evaluating the importance of large model dialogues based on dynamic weights.

[0061] Compared with existing technologies, the present invention provides a dynamic weight-based large-scale model dialogue importance assessment method and storage medium. The method includes acquiring the dialogue message to be evaluated, the corresponding dialogue history, and the current context; extracting five-dimensional features: time, semantics, sentiment, interaction, and context; then combining the basic weights of the dialogue stage with the adjustment amount of the message type to generate a dynamic weight matrix in the range of 0-1, and calculating the importance score by weighting; finally, collecting data through a feedback buffer and dynamically adjusting the learning rate and weight parameters according to the average error. This method is adaptable to multiple scenarios and can be used for dialogue window management and historical summary compression. When the computer program on the storage medium is executed, the above assessment method can be implemented. It solves the problems of static weight limitations, context fragmentation, lack of timeliness, and multi-dimensional imbalance in existing dialogue assessments, and improves the accuracy of dialogue system history management and summary processing. Attached Figure Description

[0062] Figure 1 A flowchart of a large-model dialogue importance assessment method provided in an embodiment of the present invention. Detailed Implementation

[0063] The technical solution of the present invention will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are not all embodiments of the present invention. All other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0064] It should be noted that, unless otherwise specifically stated, the relative arrangement and numerical expressions of the components and steps described in these embodiments should not be construed as limiting the scope of the invention.

[0065] The following description of exemplary embodiments is merely illustrative and is not intended to limit the invention or its application or use in any way. Techniques, methods, and apparatus known to those skilled in the art may not be discussed in detail herein, but where applicable, such techniques, methods, and apparatus should be considered part of this specification.

[0066] Example 1

[0067] See Figure 1 , Figure 1 This is a flowchart of the large-scale dialogue importance assessment method based on dynamic weights proposed in this invention. This method integrates features from five dimensions—time, semantics, sentiment, interaction, and context—combined with a dynamic weight matrix of "dialogue stage + message type," and introduces a feedback-based adaptive update mechanism to score the importance of any dialogue message. It is applicable to scenarios such as dialogue window management, historical filtering, and summary compression. Specific steps may include:

[0068] S1. Data Acquisition and Preprocessing: Acquire the dialogue messages to be evaluated, the corresponding dialogue history, and the current context. The current context consists of a preset number of the latest dialogue contents from the dialogue history. Specifically, this includes:

[0069] S11. Obtaining Core Data Items: The dialogue messages to be evaluated must contain multi-dimensional metadata, specifically including the message text content ( ), timestamp ( User unique identifier ( ), Character Type ( The value is ), reply with associated identifier ( (This is used to mark the reply object corresponding to the message); the dialogue history must cover all interaction records before the current message, and each record must contain metadata fields consistent with the message to be evaluated; the current context is defined as the latest dialogue content in the dialogue history of a preset number, which can be configured according to the scenario.

[0070] S12. Data Filtering and Cleaning: Perform validity filtering on the raw data, filtering out empty text messages, duplicate messages, and missing metadata (such as missing timestamps). Invalid records are removed; message text content is standardized by removing redundant whitespace characters, special control characters (such as newline characters and tab characters) and meaningless symbols; for specific domain scenarios, terminology normalization (unifying synonyms within the domain) and format conversion (standardizing formula text format) operations are added to ensure that the text content is standardized and consistent.

[0071] S13. Data structuring and organization: Sort the filtered and cleaned dialogue history and the current context in ascending order by timestamp to build a structured data list, which facilitates quick location of message time sequence relationships when extracting time features later.

[0072] right Enumerated fields are uniformly encoded to ensure consistent field values; timestamps of all messages are uniformly converted to millisecond-level format to avoid calculation errors caused by differences in time zones or time formats, providing data support for the subsequent calculation of the "relative freshness" of time features.

[0073] S2. Multi-dimensional feature extraction: Extract multi-dimensional features of dialogue messages, including time features, semantic features, sentiment features, interaction features, and context features.

[0074] S21. The specific process of extracting time features includes:

[0075] If the dialogue history is empty, the feature value of the time feature is set to 1.0;

[0076] If the dialogue history is not empty, retrieve the timestamps of the dialogue messages, the timestamps of the latest message and the earliest message in the dialogue history;

[0077] Calculate the time difference between the dialogue message and the latest message, and the time difference between the latest message and the earliest message, to obtain the feature value of the time feature. The specific calculation formula is as follows:

[0078] ).

[0079] The time feature is calculated using relative freshness normalization to ensure its value always falls within the range of [0,1], and follows the core principle of "the closer to the latest message, the higher the weight"; the specific calculation logic is divided into two scenarios: one is when the dialogue history is empty, the time feature value is directly set to 0. Secondly, when the conversation history is not empty, first set the current message timestamp to... The timestamp of the latest message in the conversation history is The earliest message timestamp is Then define two key differences;

[0080] Time difference between current news and the latest news and the time span of historical messages (in for (Using extremely small positive numbers to prevent the denominator from being 0), followed by the linear normalization formula. 1-( ) Calculate the initial time eigenvalues ​​and perform value pruning to improve robustness, i.e. ;

[0081] The explanation for this calculation logic is as follows: when the current message is the latest message... hour, ,at this time When the current message is the earliest message ( )hour, ,at this time ≈0.0; if there is only one message in the dialogue history ( ), then because ≈ All messages have a time feature value of 1.0; additionally, an optional exponential decay variant is provided to enhance the near-first priority effect, with the formula being: (in ,like This format allows for a steeper increase in the weight of recent news and a faster decrease in the weight of distant news.

[0082] S22. The specific process of extracting semantic features includes:

[0083] Extract the text content of the dialogue message and the text content of the current context; call the pre-trained SentenceTransformer model to encode the text content of the dialogue message and the text content of the current context respectively, to obtain the message embedding vector. and context embedding vector L2 normalization is performed on the vectors to ensure the numerical stability of the subsequent cosine similarity calculation.

[0084] Calculate the cosine similarity between the message embedding vector and the context embedding vector; extract keywords from the current context text content, count the number of keywords contained in the dialogue message text content, and calculate the ratio of the number of keywords contained to the total number of keywords as the keyword matching rate; obtain the feature value of the semantic feature by weighted summation based on the cosine similarity and the keyword matching rate.

[0085] In calculating cosine similarity, if the vectors are not normalized, they need to be calculated according to the formula. Calculate the original cosine similarity (value range [-1, 1]); to unify it with the keyword matching rate to the range [0, 1], the original value needs to be linearly mapped, i.e. After mapping .

[0086] In the keyword matching rate calculation process, the current context text is used as a reference. Extract the keyword set K (using methods such as TF-IDF, RAKE, KeyBERT, etc.); analyze the dialogue message text. Number of K keywords appearing in and according to the formula Calculate the matching rate (ensuring the denominator is non-zero), and the result. ; You can optionally perform synonym or word form restoration matching on keywords to improve robustness.

[0087] The final semantic feature values ​​are obtained by weighted summation: Assume the weights α and β satisfy α, β ≥ 0 and α + β = 1, then The semantic feature values ​​obtained from this .

[0088] S23. The specific process of extracting emotional features includes:

[0089] The TextBlob sentiment analysis model is invoked to analyze the text content of the dialogue message and output sentiment polarity values. (-1 represents extremely negative, 1 represents extremely positive) and subjective emotional value (0 represents objective, 1 represents subjective); based on this, the sentiment feature value is calculated:

[0090] S231. Take the absolute value of the emotional polarity value to obtain an index that only reflects the intensity of emotion (without distinguishing between positive and negative directions). (Value range [0,1]);

[0091] S232. The basic sentiment feature value is calculated using an equal-weighted average, and the formula is as follows: The result naturally falls within the range [0,1], and the boundary is clear;

[0092] S233. To further ensure robustness, the results can be pruned as needed, i.e. .

[0093] Calculate the sum of the absolute value of the emotional polarity value and the emotional subjectivity value, and take the average of the sum as the feature value of the emotional characteristic. Furthermore, enhancement schemes can be selected based on the actual scenario.

[0094] S234, Weighted Version: If it is necessary to emphasize the intensity of emotions or subjectivity, the following can be used:

[0095] ;

[0096] Preserve emotional direction: If it is necessary to distinguish between positive and negative emotions, calculate the intensity first. Then through the symbolic function The direction is determined, and the final directional score is obtained through bilateral quantification. (Value range [-1, 1]).

[0097] S24. The specific process of extracting interaction features includes:

[0098] Get the user identifier corresponding to the conversation message Define the total number of messages in the dialogue history. The number of times this user has spoken in the conversation history The number of messages in the conversation history that were replied to by this user. Based on thread / reference / mention metadata, it prioritizes determining the thread / reference field; if none is found, it falls back to matching "@username / ID" or the most recently referenced heuristic.

[0099] S241. Calculate user engagement: (use To prevent division by zero, if the dialogue history is empty... The participation rate is (If the current message is the first one and is included in the history, the value is approximately 1; otherwise, it is 0).

[0100] S242. Calculate the response frequency: If the conversation history is empty, the reply frequency is 0).

[0101] Determine whether the text content of the dialogue message contains a preset question identifier. The question identifier detection rule is to meet at least one of the following: it contains symbols such as “?” or “?”, or contains trigger words such as “please ask”, “why”, “how”, “how”, or matches a structured question template. If it contains, the question identifier value q=1; otherwise, q=0.

[0102] S243. Calculate the interaction feature value: using the equal-weighted average formula. This value naturally falls within the range of [0,1], and can be robustly pruned as needed. ).

[0103] S244. Optional enhancement options include:

[0104] Weighted version: If you need to emphasize a certain dimension, you can use:

[0105] ;in, Default is recommended .

[0106] Normalization correction: for and First Alternatively, logarithmic scaling and normalization can be used to avoid extreme dominance by popular users.

[0107] S25. The specific process of extracting contextual features includes:

[0108] S251, Role Type for Receiving Dialogue Messages Role types include system roles, user roles, and assistant roles; the role weight value corresponding to each role type is determined according to a preset role weight table. The default values ​​for the example are: system=0.8, user=1.0, assistant=0.9 (the weight table can be adjusted according to the scenario; if the role is unknown, the default value is 0.8).

[0109] S252. Determine the position number of the dialogue message in the current context. ( ,in The current context length is the total number of messages. Sequence number 1 corresponds to the earliest message. (Based on the latest news), the result calculated using the formula is used as the position weight value. :

[0110] Linear normalization (default, simple and intuitive): when hour, ;when hour, The closer the position is to the latest message (the closer i is to N), the higher the weight, and the value range is [0,1].

[0111] Exponential decay (optional, enhances near-term sensitivity): when hour, (k>0, default k=1); when N=1, The emphasis on the principle that newer information is more important is more pronounced, and the weighting of recent news increases more sharply.

[0112] S253. Calculate contextual feature values: using the equal-weighted average formula. If it is necessary to emphasize role or position factors, a weighted approach can be used. .

[0113] Boundary and robustness handling: and All values ​​are cropped to ensure they are within the range [0,1]; when N=1, forced cropping is applied. To avoid denominators of 0 and ensure the rigor of computational logic.

[0114] S3. Dynamic weight matrix generation: Based on the dialogue history, identify the current dialogue stage and determine the basic weight of each dimension. Classify the dialogue messages by message type and determine the weight adjustment amount of each dimension. Merge the basic weight of each dimension with the corresponding weight adjustment amount to generate a dynamic weight matrix. After fusion, the weight of each dimension is limited to the range of 0-1.

[0115] The dialogue phase includes: the initiation phase, the development phase, the climax phase, and the conclusion phase; the basic weights corresponding to the dialogue phases are:

[0116] Initialization phase: Time dimension 0.3, semantic dimension 0.4, emotional dimension 0.1, interaction dimension 0.1, context dimension 0.1;

[0117] Development stages: Time dimension 0.2, Semantic dimension 0.3, Emotional dimension 0.2, Interaction dimension 0.2, Context dimension 0.1;

[0118] Climax phase: Time dimension 0.1, semantic dimension 0.4, emotional dimension 0.3, interaction dimension 0.1, context dimension 0.1;

[0119] Final stage: Time dimension 0.3, semantic dimension 0.3, sentiment dimension 0.2, interaction dimension 0.1, context dimension 0.1;

[0120] Message types include: question type, answer type, clarification type, and summary type; the corresponding weight adjustment for each message type is:

[0121] Problem type: Semantic dimension +0.1, Interaction dimension +0.1;

[0122] Response type: Semantic dimension +0.15, Context dimension +0.05;

[0123] Clarification type: Time dimension +0.1, Interaction dimension +0.1;

[0124] Summary type: Semantic dimension +0.2, Context dimension +0.1.

[0125] S4. Importance Score Calculation: The importance score of the dialogue message is obtained by weighting the feature values ​​of each dimension in the multi-dimensional features with the corresponding weights in the dynamic weight matrix. The specific process of obtaining the importance score of the dialogue message includes:

[0126] S41. Multiply the feature value of each dimension in the multi-dimensional feature matrix with the corresponding weight in the dynamic weight matrix to obtain the weighted score of each dimension; where the multi-dimensional feature value vector integrates five dimensions of features: time, semantics, sentiment, interaction, and context, denoted as . , where each dimension's feature value ;

[0127] The dynamic weight vector corresponds one-to-one with the five-dimensional features, denoted as... Weights of each dimension (It is not necessary to satisfy the sum of 1 beforehand).

[0128] S42. Sum the weighted scores of all dimensions, denoted as... ; and the summation of dynamic weights over all dimensions, denoted as ;

[0129] If the sum of the dynamic weights of all dimensions is greater than 0, the ratio of the sum of the weighted scores to the sum of the weights is calculated as the importance score of the dialogue message; if the sum of the dynamic weights of all dimensions is equal to 0, the importance score is set to 0.

[0130] S5. Adaptive Update of Weight Parameters: Feedback data containing dialogue message predictions and actual importance scores is collected. Based on the feedback data, the error is calculated, and the initial weight parameters are adjusted using gradient descent. Simultaneously, the learning rate is dynamically adjusted according to the error magnitude to achieve adaptive updates of the initial weight parameters. The specific process of adaptive update of weight parameters also includes:

[0131] S51. Set up a feedback buffer to store the feedback data collected each time.

[0132] S52. Preset update threshold: When the amount of feedback data stored in the feedback buffer reaches the update threshold, batch update is triggered.

[0133] S53. Calculate the average of the absolute values ​​of the errors of all feedback data in the feedback buffer to obtain the average error;

[0134] S54. If the average error is greater than 0.2, increase the learning rate by 10%; if the average error is less than or equal to 0.2, decrease the learning rate by 10%.

[0135] S55. The adjusted initial weight parameters are limited to the range of 0.1-0.9. After the adjustment is completed, the feedback buffer is cleared.

[0136] Secondly, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-described method for evaluating the importance of large-scale model dialogues based on dynamic weights.

[0137] In summary, the present invention has the following advantages:

[0138] 1. By using a dynamic weighting mechanism that identifies the basic weights of the dialogue phase and fine-tunes the weights based on the message type, the limitations of existing static weighting technologies in adapting to different dialogue scenarios are solved.

[0139] 2. By associating semantic features with the current context to calculate similarity, combining contextual features with role and position weights, and relying on the complete dialogue history for stage recognition, we can solve the problems of existing technologies that ignore relevance and misjudge important information due to fragmented context.

[0140] 3. By collecting feedback from 10 messages using the WeightUpdateManager class to trigger batch updates, adjusting the learning rate based on error, and adjusting the initial weights using gradient descent within a limited range, the problem of lack of timeliness and inability to dynamically adjust the weights of historical messages in existing technologies is solved.

[0141] 4. By extracting five-dimensional features and weighted fusion evaluation, the problem of multi-dimensional imbalance and one-sided evaluation in existing technologies is solved.

[0142] The above specific embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to examples, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for evaluating the importance of large-scale model dialogues based on dynamic weights, characterized in that: include: S1. Obtain the dialogue message to be evaluated, the dialogue history corresponding to the dialogue message, and the current context, wherein the current context is a preset number of the latest dialogue contents in the dialogue history; S2. Extract the multi-dimensional features of the dialogue message, including time features, semantic features, sentiment features, interaction features, and context features; S3. Based on the dialogue history, identify the current dialogue stage and determine the basic weights of each dimension. Classify the dialogue messages by message type and determine the weight adjustment amount of each dimension. Merge the basic weights of each dimension with the corresponding weight adjustment amount to generate a dynamic weight matrix. After fusion, the weights of each dimension are limited to the range of 0-1. S4. The feature values ​​of each dimension in the multi-dimensional features are weighted and calculated with the weights of the corresponding dimensions in the dynamic weight matrix to obtain the importance score of the dialogue message; S5. Collect feedback data containing the predicted importance score and the actual importance score of the dialogue message, calculate the error based on the feedback data and adjust the initial weight parameters using the gradient descent method, and dynamically adjust the learning rate according to the error magnitude to achieve adaptive update of the initial weight parameters.

2. The method for evaluating the importance of large-scale model dialogues based on dynamic weights according to claim 1, characterized in that, The specific process of extracting the time features includes: If the dialogue history is empty, the feature value of the time feature is set to 1.0; If the dialogue history is not empty, obtain the timestamp of the dialogue message, the timestamp of the latest message in the dialogue history, and the timestamp of the earliest message. Calculate the time difference between the dialogue message and the latest message, and the time difference between the latest message and the earliest message, to obtain the feature value of the time feature. The specific calculation formula is as follows: )。 3. The method for evaluating the importance of large-scale model dialogues based on dynamic weights according to claim 1, characterized in that, The specific process of extracting the semantic features includes: Extract the text content of the dialogue message and the text content of the current context; The pre-trained SentenceTransformer model is invoked to encode the text content of the dialogue message and the text content of the current context, respectively, to obtain the message embedding vector and the context embedding vector. Calculate the cosine similarity between the message embedding vector and the context embedding vector; Extract keywords from the current context text content, count the number of keywords contained in the dialogue message text content, and calculate the ratio of the number of keywords contained to the total number of keywords as the keyword matching rate; The feature value of the semantic feature is obtained by weighted summation of the cosine similarity and the keyword matching rate.

4. The method for evaluating the importance of large-scale model dialogues based on dynamic weights according to claim 1, characterized in that, The specific process of extracting the emotional features includes: The TextBlob sentiment analysis model is invoked to analyze the text content of the dialogue message and output the sentiment polarity value and sentiment subjectivity value. Calculate the sum of the absolute value of the emotional polarity value and the emotional subjectivity value, and take the average of the sum as the feature value of the emotional characteristic.

5. The method for evaluating the importance of large-scale model dialogues based on dynamic weights according to claim 1, characterized in that, The specific process of extracting the interaction features includes: Obtain the user identifier corresponding to the dialogue message, count the number of messages sent by the user identifier in the dialogue history, and calculate the ratio of the number of messages sent by the user to the total number of messages in the dialogue history as the user engagement level. The number of messages in the dialogue history that are addressed to the user is counted, and the ratio of the number of messages addressed to the user to the total number of messages in the dialogue history is calculated as the response frequency. Determine whether the text content of the dialogue message contains a preset question identifier. If it does, set the question identifier value to 1; otherwise, set it to 0. Calculate the average of the user engagement, response frequency, and question identifier value, and use it as the feature value of the interaction feature.

6. The method for evaluating the importance of large-scale model dialogues based on dynamic weights according to claim 1, characterized in that, The specific process of extracting the contextual features includes: The role type of the dialogue message is obtained, including system role, user role, and assistant role; the role weight value corresponding to the role type is determined according to a preset role weight table. The position number of the dialogue message in the current context is determined, and the result is used as the position weight value. The average of the role weight value and the position weight value is calculated and used as the feature value of the context feature.

7. The method for evaluating the importance of large-scale model dialogues based on dynamic weights according to claim 1, characterized in that, The dialogue phases include: initiation phase, development phase, climax phase, and conclusion phase; the basic weights corresponding to the dialogue phases are: Initialization phase: Time dimension 0.3, semantic dimension 0.4, emotional dimension 0.1, interaction dimension 0.1, context dimension 0.1; Development stages: Time dimension 0.2, Semantic dimension 0.3, Emotional dimension 0.2, Interaction dimension 0.2, Context dimension 0.1; Climax phase: Time dimension 0.1, semantic dimension 0.4, emotional dimension 0.3, interaction dimension 0.1, context dimension 0.1; Final stage: Time dimension 0.3, semantic dimension 0.3, sentiment dimension 0.2, interaction dimension 0.1, context dimension 0.1; The message types include: question type, answer type, clarification type, and summary type; the weight adjustment amount corresponding to each message type is: Problem type: Semantic dimension +0.1, Interaction dimension +0.1; Response type: Semantic dimension +0.15, Context dimension +0.05; Clarification type: Time dimension +0.1, Interaction dimension +0.1; Summary type: Semantic dimension +0.2, Context dimension +0.

1.

8. The method for evaluating the importance of large-scale model dialogues based on dynamic weights according to claim 1, characterized in that, The specific process of adaptive updating of weight parameters also includes: Set up a feedback buffer to store the feedback data collected each time; A preset update threshold is set, and when the amount of feedback data stored in the feedback buffer reaches the preset update threshold, a batch update is triggered. Calculate the average of the absolute values ​​of the errors of all feedback data in the feedback buffer to obtain the average error; If the average error is greater than 0.2, the learning rate will be increased by 10%; if the average error is less than or equal to 0.2, the learning rate will be decreased by 10%. The adjusted initial weight parameters are limited to the range of 0.1-0.

9. After the adjustment is completed, the feedback buffer is cleared.

9. The method for evaluating the importance of large-scale model dialogues based on dynamic weights according to claim 1, characterized in that, The specific process of obtaining the importance score of the dialogue message includes: The feature value of each dimension in the multidimensional feature is multiplied by the weight of the corresponding dimension in the dynamic weight matrix to obtain the weighted score of each dimension. Calculate the sum of weighted scores for all dimensions, and the sum of dynamic weights for all dimensions; If the sum of the dynamic weights of all dimensions is greater than 0, the ratio of the sum of weighted scores to the sum of weights is calculated and used as the importance score of the dialogue message; if the sum of the dynamic weights of all dimensions is equal to 0, the importance score is set to 0.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the large model dialogue importance assessment method based on dynamic weights according to any one of claims 1-9.

Images