An implicit feedback collection and incremental training method and system based on user subsequent semantics

Abstract

The present application relates to the technical field of computer, and discloses a kind of implicit feedback collection and incremental training method and system based on user subsequent semantics, including obtaining dialogue segment, containing basic large model output text, user subsequent text, copy, regenerate, new session start behavior, time interval and conversation cumulative round;User subsequent text semantic classification gets intent category and first confidence, the intent category polarity benchmark value is multiplied with first confidence to get intent tendency score;Behavior mapping is behavior comprehensive score, relative reading time ratio is time interval score, conversation cumulative round is conversation continuation score by saturation growth function;Above-mentioned score is fused by learnable weight weighting to obtain implicit feedback comprehensive score and second confidence;According to score and threshold value, dialogue segment is screened to generate training label, and fine-tuning dataset is constructed, and basic large model is incrementally trained.The present application does not need additional feedback, reduces explicit feedback sparsity and implicit feedback noise, and improves the quality of backflow data.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and specifically to a method and system for implicit feedback acquisition and incremental training based on subsequent user semantics. Background Technology

[0002] Currently, large-scale models are widely used in scenarios such as dialogue question answering, content generation, code assistance, and knowledge services. In order to ensure that large-scale models continue to adapt to real user needs after deployment, it is usually necessary to feed back the interaction data between users and models as training samples, and then update the model parameters through incremental training or efficient parameter fine-tuning.

[0003] Existing data feedback solutions largely rely on explicit feedback, such as providing feedback options like "agree," "disagree," "like," and "dislike" on the model output interface, allowing users to actively click the feedback button to express their approval of the basic model's output text. While this approach can generate relatively direct supervisory signals, users' willingness to provide feedback is low, resulting in a sparse sample size of explicit feedback that struggles to cover the diverse question types and output states in real-world interactions. Furthermore, explicit feedback increases the user's workload and disrupts the continuity of natural dialogue.

[0004] Some solutions attempt to utilize implicit feedback, such as inferring user satisfaction with the output text of the base model based on user behaviors like whether they continue asking questions, copying answers, clicking to regenerate, and reading time. However, subsequent questions often have multiple semantic meanings: a user's continued questioning may indicate acceptance of the previous answer and a desire for more information, or it may indicate questioning of the previous answer; when a user provides partial corrections, only a portion of the previous base model's output text may be problematic. If subsequent inquiries are treated merely as binary behavioral markers of presence or absence, or if only absolute reading time is used for judgment, it is easy to mix different feedback states with the same training label, leading to sample mislabeling.

[0005] Furthermore, existing implicit feedback schemes are insufficient in their joint utilization of conversation continuity, the length of the output text of the base model, response time, copying behavior, regeneration behavior, and subsequent text semantics. There is also a lack of a stable calibration mechanism between explicit and implicit feedback. As a result, the quality of the resulting reflux samples fluctuates significantly, and the ratio of positive to negative samples is easily imbalanced, thus affecting the reliability of subsequent incremental training of the large model.

[0006] Therefore, this application provides a method and system for implicit feedback acquisition and incremental training based on subsequent user semantics to solve the above-mentioned technical problems. Summary of the Invention

[0007] The purpose of this invention is to provide an implicit feedback acquisition and incremental training method and system based on user subsequent semantics, in order to solve the technical problems in the prior art, such as sparse explicit feedback samples, high implicit feedback noise, underutilization of subsequent interaction semantics, and the impact of backflow samples on the quality of incremental training.

[0008] To address the aforementioned technical problems, this invention provides a method for implicit feedback acquisition and incremental training based on subsequent user semantics, comprising: Acquire dialogue fragments, which include the basic large model output text, user subsequent text, copy behavior, regeneration behavior, new session initiation behavior, time interval from output completion to user subsequent text generation, and current session cumulative rounds; Semantic classification is performed on the user's subsequent text to obtain the intent category and the first confidence level. The polarity baseline value of the intent category is multiplied by the first confidence level to form the intent tendency score. Copying, regenerating, and initiating new sessions are mapped to a comprehensive behavior score based on direction and intensity. The ratio of the time interval to the length of the text output by the basic large model is mapped to a time interval score. The cumulative number of rounds in the current session is input into a saturation growth function to form a session continuation score. The intention tendency score, behavior comprehensive score, time interval score, and conversation continuation score are weighted based on learnable weights and then input into the activation function to obtain the implicit feedback comprehensive score and the second confidence level. Based on implicit feedback comprehensive scoring, second confidence level, first threshold and second threshold, dialogue segments are selected and training labels are generated. A fine-tuning dataset is constructed. The sample selection conditions or training loss weights are adjusted based on the quantitative relationship between positive and negative samples in the fine-tuning dataset. Incremental training is performed on the base model based on the fine-tuning dataset.

[0009] In some specific embodiments, acquiring dialogue fragments further includes: Read the basic large model output completion time, user subsequent text generation time, copy trigger record, regenerate trigger record and new session start record under the same session identifier from the interactive system event log or server log; The cumulative number of rounds in the current session is calculated based on the cumulative rules following the user's initial input. The output text of the basic model, the user's subsequent text, time intervals, and records of various interaction behaviors are combined into a dialogue segment to be scored.

[0010] In some specific embodiments, performing semantic classification on subsequent user text further includes: The user's subsequent text and the basic large model's output text are input into the text intent classification model; The text intent classification model outputs the intent category among explicit approval, implicit approval, partial modification, questioning and probing, explicit denial, and irrelevant input; The predicted probability of the reading intent category is used as the first confidence level; When the predicted probability is lower than the preset confidence condition, the dialogue fragment containing the user's subsequent text will be sent to the non-training area.

[0011] In some specific embodiments, multiplying the polarity baseline value of the intent category by a first confidence level to form an intent tendency score further includes: Polarity baseline values, distributed from positive to negative, are set for explicit approval, implicit approval, partial correction, questioning and inquiry, explicit denial, and irrelevant inputs, respectively. The polarity baseline value of the selected intent category is multiplied by the first confidence level to form the intent tendency score.

[0012] In some specific embodiments, the copying behavior, regeneration behavior, and new session initiation behavior are mapped to a comprehensive behavior score by direction and intensity, further including: The copying behavior is mapped to a positive behavior score based on the proportion of copied content to the length of the basic large model output text; Based on the number of regenerated triggers and the timing of the regenerated triggers, the regenerated behavior is mapped to a negative behavior score; Weak negative behavior scores are generated based on the behavior of initiating new sessions; A comprehensive behavioral score is obtained by aggregating positive behavior scores, negative behavior scores, and weakly negative behavior scores based on preset internal weights.

[0013] In some specific embodiments, mapping the ratio of the time interval to the length of the output text of the base large model to a time interval score further includes: The relative reading time ratio is obtained by dividing the time interval between the completion of the output and the subsequent text generation by the length of the basic large model output text. The relative reading time ratio was compared with the unread threshold, the normal reading interval, and the timeout threshold, respectively. Segmented mapping is performed on the relative reading time ratios below the unread threshold, the relative reading time ratios within the normal reading range, and the relative reading time ratios exceeding the timeout threshold to obtain time interval scores.

[0014] In some specific embodiments, the current session's cumulative rounds are input into a saturation growth function to form a session continuation score, further including: Add the current session's cumulative rounds to the scaling constant to obtain the round-normalized denominator; The session continuation score is obtained by dividing the current cumulative number of rounds by the round-normalized denominator. The session continuation score is updated based on a saturation growth function as the cumulative number of rounds in the current session increases.

[0015] In some specific embodiments, the process of filtering dialogue segments and generating training labels based on implicit feedback comprehensive scoring, a second confidence level, a first threshold, and a second threshold further includes: When the implicit feedback comprehensive score is higher than the first threshold and the second confidence level meets the preset confidence conditions, the dialogue segment is marked as a positive sample; When the implicit feedback overall score is lower than the second threshold and the second confidence level meets the preset confidence conditions, the dialogue segment is marked as a negative sample; When the implicit feedback comprehensive score is between the first and second thresholds or the second confidence level does not meet the preset confidence conditions, the dialogue segment is sent to the non-training area.

[0016] In some specific embodiments, the updating of the learnable weights, the first threshold, and the second threshold includes: Anchor point samples are constructed from dialogue fragments that simultaneously contain explicit feedback records and complete implicit behavior records; Transform explicit consent records and explicit disagreement records into monitoring targets; Supervised calibration of learnable weights is performed using intention propensity score, comprehensive behavioral score, time interval score, conversation continuation score, and comprehensive implicit feedback score as calibration inputs. The first and second thresholds are adjusted based on the consistency ratio between the newly added explicit feedback and the training labels.

[0017] Based on the same concept, the present invention also provides an implicit feedback acquisition and incremental training system based on subsequent user semantics, comprising: The segment acquisition module is configured to acquire dialogue segments, which include the output text of the basic large model, the user's subsequent text, copy behavior, regeneration behavior, new session initiation behavior, the time interval from the completion of output to the generation of the user's subsequent text, and the current cumulative number of rounds in the session. The intent scoring module is configured to perform semantic classification on the user's subsequent text to obtain the intent category and the first confidence level. The polarity baseline value of the intent category is multiplied by the first confidence level to form an intent tendency score. The behavior time scoring module is configured to map copy behavior, regenerate behavior, and new session initiation behavior to a comprehensive behavior score according to direction and intensity, map the ratio of time interval to the length of the text output by the basic large model to a time interval score, and input the cumulative rounds of the current session into a saturation growth function to form a session continuation score. The fusion screening module is configured to input the intention tendency score, behavior comprehensive score, time interval score and conversation continuation score into the activation function after being weighted by learnable weights to obtain the implicit feedback comprehensive score and the second confidence level. The training control module is configured to filter dialogue segments and generate training labels based on implicit feedback comprehensive score, second confidence level, first threshold and second threshold, construct fine-tuning dataset, adjust sample selection conditions or training loss weights based on the quantity relationship of positive and negative samples in fine-tuning dataset, and perform incremental training on the base large model based on fine-tuning dataset.

[0018] Compared with existing technologies, its advantages are as follows: This invention discloses an implicit feedback collection and incremental training method and system based on user subsequent semantics. It obtains user subsequent text, copying behavior, regeneration behavior, new session initiation behavior, time interval, and current session cumulative rounds that are naturally generated after the output text of the basic large model. The above data is combined into a dialogue segment to be scored. It can form an implicit feedback collection data source without the user clicking the feedback button, reducing the limitation of explicit feedback sparsity on data backflow.

[0019] The system performs semantic classification on subsequent user text, including explicit approval, implicit approval, partial correction, questioning and follow-up, explicit denial, and irrelevant input. It also generates an intent tendency score based on intent category and first confidence level. This system can distinguish different semantic states such as continued questioning after approval, questioning and follow-up, partial correction, and irrelevant input, reducing the risk of mixing different user attitudes into the same training label.

[0020] By mapping copying, regeneration, and new session initiation behaviors to behavioral scores with direction and intensity, and aggregating these scores into a comprehensive behavioral score, the system can transform user actions such as adoption, abandonment, or re-request of the basic large model's output text into computable feedback components, enabling behavioral signals to participate in training label generation.

[0021] Dividing the time interval between the completion of the output and the subsequent text generation by the length of the output text of the basic large model yields the relative reading time ratio. By forming a time interval score through segmented mapping, the interference of the difference in the length of the output text of the basic large model on the absolute response time judgment can be reduced, making reading time information more suitable for participating in feedback judgment together with semantic intent and interactive behavior.

[0022] By inputting the cumulative number of rounds in the current session into a saturation growth function to form a session continuation score, we can express the reference value of continuous interaction within the same session for implicit feedback, while limiting the excessive influence of long session rounds on the overall score through saturation growth.

[0023] By weighting the intent tendency score, behavior comprehensive score, time interval score, and conversation continuation score with learnable weights and then inputting them into the activation function, an implicit feedback comprehensive score and a second confidence level are obtained. This can unify text semantics, behavioral operations, relative reading time, and conversation continuation information into a filterable feedback result, providing a consistent data processing basis for the division of positive samples, negative samples, and samples that are not yet trained.

[0024] By constructing anchor samples using dialogue fragments that simultaneously contain explicit feedback records and complete implicit behavior records, and using explicit agreement records and explicit disagree records as supervision targets to calibrate learnable weights, a first threshold, and a second threshold, it is possible to maintain a calibration relationship between implicit feedback scores and user-initiated feedback, and reduce the bias caused by the implicit score's own iterative training.

[0025] When constructing the fine-tuning dataset, dialogue segments are selected based on implicit feedback comprehensive score, second confidence level, first threshold and second threshold. Dialogue segments in the middle range or with insufficient confidence are sent to the non-training area. This can reduce the entry of low-confidence or unclear-biased samples into the incremental training process and reduce the impact of noisy samples on the update of the basic large model.

[0026] By adjusting the sample selection criteria or training loss weights based on the quantitative relationship between positive and negative samples in the fine-tuning dataset, the majority and minority class samples can be processed when the proportion of naturally collected samples is unbalanced, so that the incremental training process can obtain more stable sample label input. Attached Figure Description

[0027] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart illustrating some specific embodiments of the implicit feedback collection and incremental training method based on subsequent user semantics of the present invention. Figure 2 This is a schematic diagram of the structure of an implicit feedback acquisition and incremental training system based on subsequent user semantics in some specific embodiments of the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0029] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms "a," "the," and "the" as used in the embodiments of this application are also intended to include the plural forms, unless the context clearly indicates otherwise, and "multiple" generally includes at least two.

[0030] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0031] It should be understood that although the terms first, second, third, etc., may be used in the embodiments of this application, these descriptions should not be limited to these terms. These terms are only used to distinguish the descriptions. For example, first may also be referred to as second without departing from the scope of the embodiments of this application, and similarly, second may also be referred to as first.

[0032] Depending on the context, the words “if” or “suppose” as used here can be interpreted as “when” or “in response to determination” or “in response to detection.” Similarly, depending on the context, the phrases “if determination” or “if detection (of the stated condition or event)” can be interpreted as “when determination” or “in response to determination” or “when detection (of the stated condition or event)” or “in response to detection (of the stated condition or event).”

[0033] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that an article or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such an article or device. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the article or device that includes said element.

[0034] It should be noted that any symbols and / or numbers present in the specification that are not marked in the accompanying drawings are not reference numerals.

[0035] Reference Figure 1 An implicit feedback acquisition and incremental training method based on subsequent user semantics includes: S101, Obtain a dialogue fragment, the dialogue fragment including the basic large model output text, user subsequent text, copy behavior, regeneration behavior, new session initiation behavior, time interval from output completion to user subsequent text generation, and current session cumulative rounds; S102, Perform semantic classification on the user's subsequent text to obtain the intent category and the first confidence level, and multiply the polarity baseline value of the intent category with the first confidence level to form an intent tendency score; S103 maps copying behavior, regeneration behavior, and new session initiation behavior to a comprehensive behavior score based on direction and intensity; maps the ratio of time interval to the length of the text output by the basic large model to a time interval score; and inputs the cumulative rounds of the current session into a saturation growth function to form a session continuation score. S104, the intention tendency score, behavior comprehensive score, time interval score and conversation continuation score are weighted based on learnable weights and then input into the activation function to obtain the implicit feedback comprehensive score and the second confidence score; S105: Based on implicit feedback comprehensive score, second confidence level, first threshold and second threshold, dialogue segments are selected and training labels are generated. A fine-tuning dataset is constructed. The sample selection conditions or training loss weights are adjusted based on the quantity relationship between positive and negative samples in the fine-tuning dataset. Incremental training is performed on the basic large model based on the fine-tuning dataset.

[0036] To provide a clearer explanation, the steps in the embodiments of the present invention are described in detail below: S101, Obtain a dialogue fragment, the dialogue fragment including the basic large model output text, user subsequent text, copy behavior, regeneration behavior, new session initiation behavior, time interval from output completion to user subsequent text generation, and current session cumulative rounds; Furthermore, the system reads the completion time of the basic large model output, the time of subsequent text generation by the user, the copy trigger record, the regeneration trigger record, and the new session start record under the same session identifier from the event records of the interactive system or the server logs. The cumulative number of rounds in the current session is calculated based on the cumulative rules following the user's initial input. The output text of the basic model, the user's subsequent text, time intervals, and records of various interaction behaviors are combined into a dialogue segment to be scored.

[0037] In this embodiment, the basic large model is deployed on a cloud server, computing cluster, or internal enterprise server. Users interact with the basic large model in multiple rounds through web pages, applications, or integrated dialog interfaces. Upon completion of each round of basic large model output, a completion time is generated, and the output text is written to the dialog log. When a user continues to input text within the same session, a subsequent text generation time is generated, and the subsequent text is associated with the previous round's basic large model output text. Copy trigger records can be generated from front-end copy events, server-side event tracking, or clipboard operation logs, and must include at least the copy occurrence time and the length of the copied text. Regeneration trigger records can be generated by the user clicking the regeneration control, and must include at least the number of triggers and the trigger time. New session opening records can be generated by the user actively creating a new session, closing the current session and then recreating it, or switching to a new session in the session list.

[0038] The cumulative number of rounds in the current session begins counting from the user's first input in the current session. The first input and the first output of the basic model constitute the first round of interaction. The second round of interaction occurs when the user inputs again and receives another output from the basic model. Subsequent rounds are continuously accumulated according to the same session identifier. Topic switching maintains the continuous accumulation of rounds within the current session because topic switching within the same session still reflects the user's continued use, partial modification, or abandonment of previous responses. If the completion time of the basic model output is missing under the same session identifier, the completion time of the adjacent server response is used as the completion time of the basic model output. If the time of subsequent text generation by the user is missing, the time of user input submission is used as the time of subsequent text generation. Through the above processing, the dialogue fragment to be scored obtained contains text objects, behavior objects, time objects, and round objects that can be directly read in subsequent steps.

[0039] In this embodiment, detectable records can also be saved in the dialogue segments to be scored. These detectable records include a session identifier, a summary of the basic large-scale model's output text, a summary of the user's subsequent text, the completion time of the basic large-scale model's output, the generation time of the user's subsequent text, the copy ratio, the number of regeneration triggers, a new session initiation flag, the cumulative number of rounds in the current session, and a data integrity flag. These detectable records can be written to an audit directory in a time-series database or object storage, and can be read according to the session identifier via a debugging interface for verifying the input source and sample selection results of the implicit feedback comprehensive scoring.

[0040] S102, Perform semantic classification on the user's subsequent text to obtain the intent category and the first confidence level, and multiply the polarity baseline value of the intent category with the first confidence level to form an intent tendency score; Furthermore, the user's subsequent text and the basic large model's output text are combined with the input text intent classification model; The text intent classification model outputs the intent category among explicit approval, implicit approval, partial modification, questioning and probing, explicit denial, and irrelevant input; The predicted probability of the reading intent category is used as the first confidence level; When the predicted probability is lower than the preset confidence condition, the dialogue fragment containing the user's subsequent text will be sent to the non-training area.

[0041] In this embodiment, the input received by the text intent classification model can be formed by concatenating the output text of the base model and the user's subsequent text. During concatenation, a separator can be added between the output text of the base model and the user's subsequent text to allow the text intent classification model to distinguish between the preceding response and the user's subsequent expression. The text intent classification model can adopt a structure of a pre-trained language model plus a classification head. For example, a pre-trained model with text classification capabilities can be used as an encoder, and a fully connected classification head with a 6-dimensional output can be connected after the sentence vector output by the encoder. During training, sample labels are marked according to the user's attitude towards the preceding base model output text in the subsequent text: text directly expressing affirmation without further questions is marked as explicit approval; text continuing from the preceding content and asking subsequent questions is marked as implicit approval; text containing specific objects and content for correction is marked as partial correction; text only expressing doubt or follow-up questions without providing any correction content is marked as questioning and follow-up inquiry; text directly negating the preceding base model output text is marked as explicit negation; and text that has no effective feedback relationship with the preceding base model output text is marked as irrelevant input.

[0042] The inference process of the text intent classification model includes text cleaning, word segmentation or encoding, vectorization representation, classification head output, and probability normalization. After the classification head outputs the unnormalized scores for the six categories, a normalization function is used to obtain the predicted probability for each category. The category with the highest predicted probability is taken as the intent category, and this predicted probability is used as the first confidence level. If the first confidence level If the score is below a preset threshold, such as below 0.55, it indicates that the semantic attitude is not clear enough, and the dialogue fragment containing subsequent user text will be sent to the non-training zone. Dialogue fragments in the non-training zone can be retained and will enter the threshold screening process after subsequent explicit feedback, manual sampling, or more contextual information is added.

[0043] In this embodiment, local correction recognition can also be performed on the user's subsequent text. When the intent category is partial correction, the text intent classification model or the additional sequence labeling model can extract the correction object and correction content from the user's subsequent text. For example, if the user's subsequent text is "The year in the second paragraph is incorrect; it should be 2024," the correction object is "the year in the second paragraph," and the correction content is "2024." The correction object and correction content are then entered into the subsequent sample label generation process to distinguish between corrected and uncorrected segments in the output text of the basic large model.

[0044] Furthermore, multiplying the polarity baseline value of the intent category by the first confidence level to form the intent propensity score includes: Polarity baseline values, distributed from positive to negative, are set for explicit approval, implicit approval, partial correction, questioning and inquiry, explicit denial, and irrelevant inputs, respectively. The polarity baseline value of the selected intent category is multiplied by the first confidence level to form the intent tendency score.

[0045] In this embodiment, the polarity benchmark value is used to convert discrete semantic intent categories into a tendency quantity that can participate in numerical fusion. For example, the polarity benchmark values ​​for explicit approval, implicit approval, partial correction, questioning and probing, explicit denial, and irrelevant input are +0.9, +0.6, +0.2, -0.3, -0.9, and 0, respectively. If the text intent classification model outputs an intent category of implicit approval, and the first confidence level... If the value is 0.80, then the intention tends to favor the score. If the text intent classification model outputs an intent category of explicit negation, and the first confidence level is... If the value is 0.90, then the intention is biased towards the score. By multiplying the polarity baseline value by the first confidence level, semantic direction and the credibility of model judgments jointly participate in the formation of the implicit feedback comprehensive score.

[0046] In this embodiment, the polarity baseline value can be set according to the business scenario during deployment initialization, or it can be calibrated after the accumulation of explicit feedback anchor point samples. During calibration, the consistency distribution between each intent category and explicit agreement and disagreement records is statistically analyzed, and the polarity baseline value is slightly updated. For example, when "questioning and follow-up" often corresponds to users continuing to explore rather than providing negative feedback in a certain business scenario, the polarity baseline value for questioning and follow-up can be adjusted from -0.3 to -0.1; when "partial correction" often corresponds to partial errors in previous answers, the polarity baseline value for partial correction can be adjusted from +0.2 to 0 or a negative small value. The updated polarity baseline value is used in the intent tendency score calculation and participates in the intent tendency score calculation of subsequent dialogue segments.

[0047] In this embodiment, irrelevant inputs can also be set as a masked category. When the intent category is irrelevant input and the first confidence level meets the preset confidence condition, the intent tendency score is set to 0, and the second confidence level of the dialogue segment is reduced to prevent text without feedback relationship from interfering with the training labels. If the first confidence level of irrelevant input is lower than the preset confidence condition, the dialogue segment is processed according to the preset confidence condition and sent to the non-training area.

[0048] S103 maps copying behavior, regeneration behavior, and new session initiation behavior to a comprehensive behavior score based on direction and intensity; maps the ratio of time interval to the length of the text output by the basic large model to a time interval score; and inputs the cumulative rounds of the current session into a saturation growth function to form a session continuation score. Furthermore, the copying behavior is mapped to a positive behavior score based on the proportion of copied content to the length of the basic large model output text; Based on the number of regenerated triggers and the timing of the regenerated triggers, the regenerated behavior is mapped to a negative behavior score; Weak negative behavior scores are generated based on the behavior of initiating new sessions; A comprehensive behavioral score is obtained by aggregating positive behavior scores, negative behavior scores, and weakly negative behavior scores based on preset internal weights.

[0049] In this embodiment, the copying behavior indicates that the user has at least partially adopted the output text of the base model. The ratio of the copied content length to the output text length of the base model is calculated. For example, positive behavior scoring when not copied. Set to 0; When it is not higher than 20%, Take +0.3; When it is higher than 20% but not higher than 60%, Take +0.6; When it is above 60%, Take +1.0. When the output text of the same basic large model is copied two or more times consecutively, it can be... Multiply by 0.9 to reduce the impact of external user records, citations, or misoperations on satisfaction assessment.

[0050] The regeneration behavior indicates a negative bias in the user's perception of the current base model's output text. When regeneration is not triggered, the negative behavior score is... Set to 0; when regeneration is first triggered, Set the value to -0.8; when regeneration is triggered multiple times, update using a saturated cumulative method. , The lower limit is -1.0. If the user triggers a regeneration before the new output is fully generated, then... You can directly take -1.0. The new session initiation behavior is used to form a weakly negative behavior score. For example, when a user initiates a new session shortly after the output of the basic large model, Take -0.6; when no new session is opened, Set to 0. The negative intensity of the new session initiation behavior is lower than that of the regeneration behavior because a new session initiation may be due to task completion, user task switching, or interface operation habits.

[0051] Overall Behavioral Score It can be calculated using the following formula: ; in, Assess the overall behavioral score; Scoring the positive behaviors corresponding to the copying behavior; To regenerate the negative behavior score corresponding to the behavior; Score the weak negative behavior corresponding to the behavior that starts a new session; , , The internal weights for copy behavior score, regenerated behavior score, and new session initiation behavior score are respectively set to 0.4, 0.4, and 0.2, for example, respectively. After the calculation is complete, you can... Limited to Within the interval. If a record of a certain behavior is missing, the score of the missing behavior is set to 0, and a behavior missing flag is written into the detectable records; if there are copying behaviors and regenerating behaviors within the same time window, the positive and negative components are retained respectively, and the semantic score, behavior score, time score and round score are fused by the learnable weights in the subsequent weighted fusion processing.

[0052] In this embodiment, updates can also be made based on explicit feedback anchor point samples. , and When historical data shows a high proportion of copying behavior consistent with explicit consent records, This can be increased; when the proportion of regenerated behavior consistent with explicit disagreement records is high. This can be increased; when the proportion of new session initiation behavior consistent with explicit disagreement records is low, It can be reduced. Internal weight updates only change the process by which the overall behavioral score is formed.

[0053] Furthermore, the ratio of the time interval to the length of the output text of the basic large model is mapped to a time interval score, and the session continuation score is formed by the cumulative rounds of input in the current session using a saturation growth function. The relative reading time ratio is obtained by dividing the time interval between the completion of the output and the subsequent text generation by the length of the basic large model output text. The relative reading time ratio was compared with the unread threshold, the normal reading interval, and the timeout threshold, respectively. Segmented mapping is performed on the relative reading time ratios below the unread threshold, the relative reading time ratios within the normal reading range, and the relative reading time ratios exceeding the timeout threshold to obtain time interval scores; Add the current session's cumulative rounds to the scaling constant to obtain the round-normalized denominator; The session continuation score is obtained by dividing the current cumulative number of rounds by the round-normalized denominator.

[0054] In this embodiment, the time interval The basic large model outputs the duration between the completion time of the output and the time when the user subsequently generates text; the basic large model outputs the text length. Statistics can be calculated based on the number of characters or the number of tokens. Relative reading time ratio Calculate using the following formula: ; in, This is a ratio of relative reading time. The time interval between the completion of the output and the generation of subsequent text by the user; Output the text length of the basic large model.

[0055] use Replacing absolute time interval This can reduce the impact of differences in the length of the output text of the base model on the time score. For example, in Chinese interactive scenarios, the unread threshold... Take 0.04 as the lower limit of the normal reading range. Take 0.08, the upper limit of the normal reading range. Set the value to 0.5, which is the timeout threshold. Take 0.8. Time interval score. Determined according to the following segmentation method: when hour, ; when hour, ; when hour, ; when hour, ; when hour, .

[0056] in, Rate the time interval; This is a ratio of relative reading time. This is the unread threshold; This represents the lower limit of the normal reading range. This represents the upper limit of the normal reading range. This is the timeout threshold. , , and These results are derived from historical reading speed distribution, business scenario configuration, or explicit feedback anchor point sample calibration results. The time interval score is output to the weighted fusion processing for weighted fusion. If... If the output text is 0 or missing from the base model, the dialogue segment is sent to the non-training area; if... If the value is negative or exceeds the session timeout limit, the time record will be marked as abnormal and the time interval score will be set to 0.

[0057] Session continuation score Used to indicate the duration of interaction within the same session. Let... Accumulate rounds for the current session. Let be a scale constant, then: ; in, Score the duration of the conversation; Accumulate rounds for the current session; It is the half-saturation scale constant. The accumulation starts from the user's first input in the current session, and continues even when switching topics; An example value of 5 can be used, but it can also be set within the range of 5 to 8. When and hour, ;when As it continues to increase, Gradually approaching 1. Through a saturation growth function, the early rounds of interaction contribute significantly to the conversation continuation score, while the contribution gradually decreases as subsequent rounds increase, thus avoiding the infinite amplification of implicit positive feedback in long conversations.

[0058] In this embodiment, time data availability conditions can be configured separately for offline processing and online processing. During online processing... The time interval score is generated after the user's subsequent text is generated and can participate in the weighted fusion process in real time; during offline batch processing, historical logs are read in the daily backflow task for batch calculation. , and For the basic model output text where subsequent text is missing, a "no subsequent text" flag is generated after the session times out, and the corresponding dialogue fragment is sent to the non-training area or the low-confidence observation area.

[0059] S104, the intention tendency score, behavior comprehensive score, time interval score and conversation continuation score are weighted based on learnable weights and then input into the activation function to obtain the implicit feedback comprehensive score and the second confidence score; In this embodiment, implicit feedback comprehensive scoring It can be formed using the following formula: ; in, This is a comprehensive score based on implicit feedback. Score based on intentional tendency; Assess the overall behavioral score; Rate the time interval; Score the duration of the conversation; , , , These are learnable weights; For the activation function, the Sigmoid function is preferred, that is: ; in, The input value is the weighted sum of the learnable weights and the scores. Mapped to interval, A value close to 1 indicates that the overall tone of the dialogue segment is positive. A value close to 0 indicates that the overall tone of the dialogue segment is negative. A value close to 0.5 indicates that the dialogue segment lacks a clear direction.

[0060] Second confidence level Used to measure the availability of a comprehensive score. For example, It can be determined by the first confidence level. Behavioral integrity coefficient and time validity coefficient Multiplying them together, we get: ; in, This is the second confidence level; The first confidence level; The coefficient for behavioral integrity; Time validity coefficient. Behavioral integrity coefficient. The value is assigned based on the completeness of the collection of copied trigger records, regenerated trigger records, and new session start records. For example, a value of 1 is assigned when all three types of behavior records are complete, 0.8 is assigned when one type is missing, and 0.6 is assigned when two or more types are missing; Time validity coefficient. according to and Whether a value is valid, for example and Take 1 if both are valid. or In case of an anomaly, use 0.5. Output to sample selection and training label generation processing, and They jointly decide whether a dialogue segment enters the positive sample area, the negative sample area, or the area where it is not trained.

[0061] Furthermore, the weighted fusion process also includes updating the learnable weights, the first threshold, and the second threshold: Anchor point samples are constructed from dialogue fragments that simultaneously contain explicit feedback records and complete implicit behavior records; Transform explicit consent records and explicit disagreement records into monitoring targets; Supervised calibration of learnable weights is performed using intention propensity score, comprehensive behavioral score, time interval score, conversation continuation score, and comprehensive implicit feedback score as calibration inputs. The first and second thresholds are adjusted based on the consistency ratio between the newly added explicit feedback and the training labels.

[0062] In this embodiment, the explicit feedback anchor point samples are derived from dialogue fragments where the user actively clicks "agree" or "disagree," and these fragments simultaneously contain complete subsequent user text, copying behavior, regeneration behavior, new session initiation behavior, time interval, and the current session's cumulative rounds. Supervision Target It can be set to: Explicit consent record corresponding Explicitly disagreeing with the corresponding record Learnable weights to Offline supervised calibration can be performed using cross-entropy loss, and the loss function can be: ; in, This is the loss value; To provide explicit feedback on the monitoring objectives; This is a comprehensive score for implicit feedback. During training, using... , , and As input features, As a supervision objective, updates are performed through mini-batch gradient descent. to After deployment, when the cumulative number of new explicit feedback reaches a preset number, such as 500, a lightweight update can be performed during off-peak hours; when the number of new explicit feedback is lower than the preset number, the current weight is retained to avoid instability in the score caused by short-term fluctuations.

[0063] In this embodiment, the following in-depth processing can also be performed while updating the first threshold and the second threshold: Within a preset sliding window, the consistency ratio of positive sample labels with explicit consent records and the consistency ratio of negative sample labels with explicit disagreement records are statistically analyzed. When any consistency ratio is lower than the preset calibration condition, the screening threshold associated with the low consistency ratio sample category is increased, and the excluded dialogue segments are sent to the non-training area. When the consistency ratio meets the preset calibration conditions and the number of training batch samples is lower than the preset batch conditions, the screening threshold associated with the insufficient sample category is lowered.

[0064] For example, the preset sliding window can take the most recent 7 days or the most recent 5000 anchor point samples, the preset calibration condition can be a consistency ratio of not less than 0.85, and the preset batch condition can be that a single incremental training batch is not less than 1000 samples. The threshold adjustment can be done in steps of 0.02, with the first threshold limited to the range of 0.7 to 0.85 and the second threshold limited to the range of 0.15 to 0.3. This in-depth processing generates updated first and second thresholds, which are then used by the sample selection and training label generation processes.

[0065] S105: Based on implicit feedback comprehensive score, second confidence level, first threshold and second threshold, dialogue segments are selected and training labels are generated. A fine-tuning dataset is constructed. The sample selection conditions or training loss weights are adjusted based on the quantity relationship between positive and negative samples in the fine-tuning dataset. Incremental training is performed on the basic large model based on the fine-tuning dataset.

[0066] Furthermore, when the implicit feedback comprehensive score is higher than the first threshold and the second confidence level meets the preset confidence conditions, the dialogue segment is marked as a positive sample; When the implicit feedback overall score is lower than the second threshold and the second confidence level meets the preset confidence conditions, the dialogue segment is marked as a negative sample; When the implicit feedback comprehensive score is between the first and second thresholds or the second confidence level is lower than the preset confidence condition, the dialogue segment is sent to the non-training area.

[0067] In this embodiment, the first threshold is used to filter positive samples, and the second threshold is used to filter negative samples. For example, the first threshold can be set to 0.75, the second threshold can be set to 0.25, and the preset confidence condition for the second confidence level can be set to... Not less than 0.6. When Higher than 0.75 and If the value is not lower than 0.6, the dialogue segment is marked as a positive sample; when... Below 0.25 and If the value is not lower than 0.6, the dialogue segment is marked as a negative sample; when... Located between 0.25 and 0.75, or If the score is below 0.6, the dialogue segment is moved to the non-training zone. Samples in the non-training zone retain the original input, rating components, and confidence scores for subsequent explicit feedback calibration, manual sampling, or re-scoring.

[0068] Each sample in the fine-tuning dataset can contain prompt text, base model output text, user follow-up text, training labels, implicit feedback composite score, second confidence score, and sample source identifier. Positive samples can be constructed to retain the base model output text as a preferred sample, while negative samples can be constructed to reduce the probability of the base model output text being a preferred choice. Irrelevant inputs or samples with insufficient confidence are excluded from the fine-tuning dataset to avoid contaminating the training data with text that has no feedback relationship.

[0069] In this embodiment, local labeling can also be performed while the dialogue fragments are marked as positive samples: When the intent category is partial correction, the correction object and correction content are extracted from the user's subsequent text, and the correction object is matched with the text fragments in the output text of the basic large model. Generate local negative sample labels for the matched text segments, and retain positive sample labels for the text segments in the output text of the base model that were not matched by the corrected object; Local negative sample labels and positive sample labels are jointly written into the fragment-level label set of the same training sample. For example, if the base model output text contains three paragraphs, and the user subsequently points out a factual error in the second paragraph and provides a correction, then the text fragment corresponding to the second paragraph is labeled as a local negative sample, while the first and third paragraphs are retained as positive sample labels. If the correction target fails to match a text fragment in the base model output text, the dialogue fragment is sent to the non-training area or the manual review queue.

[0070] Adjust the sample selection criteria or training loss weights based on the quantitative relationship between positive and negative samples in the fine-tuning dataset, and perform incremental training on the base model based on the fine-tuning dataset.

[0071] In this embodiment, the number of positive samples in the current batch is counted after the fine-tuning dataset is formed. and the number of negative samples .like Significantly greater than ,For example If the threshold exceeds 3, the first threshold is raised and the second threshold is lowered to reduce the number of low-quality positive samples entering the training batch and expand the candidate range of negative samples; if Significantly greater than ,For example If the threshold exceeds 3, the second threshold is raised and the first threshold is lowered to reduce the number of low-quality negative samples entering the training batch and expand the candidate range of positive samples. The training loss weights can be set inversely proportional to the number of classes; for example, the positive sample loss weights... Negative sample loss weights If the number of samples in a certain category is 0, the samples are sent to the waiting buffer. Incremental training of the current batch will start once the number of samples meets the preset batch conditions.

[0072] Incremental training can employ a configurable triggering framework. Triggering conditions can include reaching a preset number of samples in the fine-tuning dataset, a preset time interval since the last incremental training, or a sustained decrease in the explicit feedback consistency ratio. Parameter updates can utilize LoRA, other efficient fine-tuning methods, or full-scale fine-tuning. When using LoRA, the core parameters of the basic large model are frozen, low-rank adaptation parameters are inserted into the attention projection layer or feedforward network layer, training objectives are constructed based on positive and negative samples, and the low-rank adaptation parameters are updated accordingly. After training, the incremental parameters, training batch identifier, sample statistics, and threshold version number are saved, ensuring that each model update can be traced back to the corresponding fine-tuning dataset.

[0073] In this embodiment, the model training process includes training data sources, training label sources, training objectives, parameter update methods, and stopping conditions. The training data source is a fine-tuned dataset constructed from sample selection and training label generation. The training label sources are positive sample labels, negative sample labels, and fragment-level label sets. The training objective can be either a preference-based learning objective or a supervised fine-tuning objective. When using a preference-based learning objective, the output text of the large model based on positive samples is used as the preference output, and the output text of the large model based on negative samples is used as the non-preference output. When using a supervised fine-tuning objective, locally corrected content can be used as the corrected target text for training. The training stopping conditions can be reaching a preset number of training epochs, the validation set loss remaining flat or increasing for two consecutive epochs, or the training time reaching a preset upper limit. After training, the incremental parameters are saved, and the incremental parameter versions are associated with the weight versions and threshold versions.

[0074] In this embodiment, the model inference process uses a base model or a large model with incremental parameters to receive user input and generate the output text of the base model. During the inference phase, only user input and the base model are used to generate the output text of the base model. Subsequent user text is used for the next round of feedback collection and incremental training. The model evaluation process can use explicit feedback anchor samples, manually sampled samples, or historical replay samples. Evaluation metrics may include the explicit feedback consistency rate, the proportion of areas not yet trained, the ratio of positive to negative samples, and the change in training batch loss. The evaluation results are used to update the learnable weights, the first threshold, the second threshold, and the sample selection criteria.

[0075] The following description, using specific examples, elaborates on this embodiment.

[0076] Suppose that in a certain round of dialogue, the basic large model outputs text length... The time interval between the completion of the output (500 characters) and the subsequent text generated by the user. The current session has a total of 120 rounds. The score is 4. The user's subsequent text is "This explanation is basically correct. By the way, let me also talk about how to handle user privacy during deployment." The user did not click "regenerate," did not start a new session, and copied 250 characters from the basic large model output text. The process involves acquiring and processing dialogue fragments, reading the basic large model output text, the user's subsequent text, the copied record, the regenerated record, and the new session record. and These are combined to form the dialogue fragment to be scored. Semantic classification processing combines the output text of the basic large model with the user's subsequent text input text to form a text intent classification model. The output intent category is implicit approval, with a first confidence level. It is 0.80.

[0077] Intent preference scores are calculated by adding 0.6 to the implicit acceptance polarity benchmark. The behavioral comprehensive score mapping process calculates a replication ratio of 250 / 500 = 0.5, representing the replication behavior score. Take +0.6; Regenerate if not clicked. Return 0; no new session opened. Take 0; when , and When the values ​​are 0.4, 0.4, and 0.2 respectively, The calculation of time interval score and conversation continuation score involves calculating the relative reading time ratio. It falls within the normal reading range of 0.08 to 0.5. Take 1; when When taking 5, .

[0078] In weighted fusion processing, for example, take , , , The weighted sum is , ,get Approximately 0.824. When the behavior record is complete and the time record is valid, and All values ​​are set to 1, second confidence level In the sample selection and training label generation process, if the first threshold is 0.75, the second threshold is 0.25, and the preset confidence condition is... If the confidence level is not lower than 0.6, the dialogue segment is marked as a positive sample and written into the fine-tuning dataset. Incremental training processes count the number of positive and negative samples in the current batch. If the ratio of positive to negative samples meets the training conditions, the positive sample is included in incremental training. If the number of positive samples is significantly excessive, the weight of negative samples is increased in the training loss, or the first threshold is raised in the next batch to filter out low-confidence positive samples.

[0079] For the purpose of simplicity, the method steps disclosed in the above embodiments are described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0080] like Figure 2 As shown, the present invention also provides an implicit feedback acquisition and incremental training system based on subsequent user semantics, comprising: The segment acquisition module is configured to acquire dialogue segments, which include the output text of the basic large model, the user's subsequent text, copy behavior, regeneration behavior, new session initiation behavior, the time interval from the completion of output to the generation of the user's subsequent text, and the current cumulative number of rounds in the session. The intent scoring module is configured to perform semantic classification on the user's subsequent text to obtain the intent category and the first confidence level. The polarity baseline value of the intent category is multiplied by the first confidence level to form an intent tendency score. The behavior time scoring module is configured to map copy behavior, regenerate behavior, and new session initiation behavior to a comprehensive behavior score according to direction and intensity, map the ratio of time interval to the length of the text output by the basic large model to a time interval score, and input the cumulative rounds of the current session into a saturation growth function to form a session continuation score. The fusion screening module is configured to input the intention tendency score, behavior comprehensive score, time interval score and conversation continuation score into the activation function after being weighted by learnable weights to obtain the implicit feedback comprehensive score and the second confidence level. The training control module is configured to filter dialogue segments and generate training labels based on implicit feedback comprehensive score, second confidence level, first threshold and second threshold, construct fine-tuning dataset, adjust sample selection conditions or training loss weights based on the quantity relationship of positive and negative samples in fine-tuning dataset, and perform incremental training on the base large model based on fine-tuning dataset.

[0081] It is worth noting that although only some basic functional modules are disclosed in the embodiments of this invention, it does not mean that the composition of this system is limited to the above-mentioned basic functional modules. On the contrary, based on the above-mentioned basic functional modules, those skilled in the art can arbitrarily add one or more functional modules in combination with existing technology to form an infinite number of embodiments or technical solutions. That is to say, this system is open rather than closed. The fact that this embodiment only discloses a few basic functional modules should not be considered as the scope of protection of this invention being limited to the disclosed basic functional modules. At the same time, for the convenience of description, the above devices are described separately according to their functions as various units and modules. Of course, in implementing this invention, the functions of each unit and module can be implemented in one or more software and / or hardware.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for implicit feedback acquisition and incremental training based on subsequent user semantics, characterized in that, include: Acquire dialogue fragments, which include the basic large model output text, user subsequent text, copy behavior, regeneration behavior, new session initiation behavior, time interval from output completion to user subsequent text generation, and current session cumulative rounds; Semantic classification is performed on the user's subsequent text to obtain the intent category and the first confidence level. The polarity baseline value of the intent category is multiplied by the first confidence level to form the intent tendency score. Copying, regenerating, and initiating new sessions are mapped to a comprehensive behavior score based on direction and intensity. The ratio of the time interval to the length of the text output by the basic large model is mapped to a time interval score. The cumulative number of rounds in the current session is input into a saturation growth function to form a session continuation score. The intention tendency score, behavior comprehensive score, time interval score, and conversation continuation score are weighted based on learnable weights and then input into the activation function to obtain the implicit feedback comprehensive score and the second confidence level. Based on implicit feedback comprehensive scoring, second confidence level, first threshold and second threshold, dialogue segments are selected and training labels are generated. A fine-tuning dataset is constructed. The sample selection conditions or training loss weights are adjusted based on the quantitative relationship between positive and negative samples in the fine-tuning dataset. Incremental training is performed on the base model based on the fine-tuning dataset.

2. The implicit feedback acquisition and incremental training method based on subsequent user semantics as described in claim 1, characterized in that, Obtain dialogue fragments, further including: Read the basic large model output completion time, user subsequent text generation time, copy trigger record, regenerate trigger record and new session start record under the same session identifier from the interactive system event log or server log; The cumulative number of rounds in the current session is calculated based on the cumulative rules following the user's initial input. The output text of the basic model, the user's subsequent text, time intervals, and records of various interaction behaviors are combined into a dialogue segment to be scored.

3. The implicit feedback acquisition and incremental training method based on subsequent user semantics as described in claim 1, characterized in that, Perform semantic classification on subsequent user text, further including: The user's subsequent text and the basic large model's output text are input into the text intent classification model; The text intent classification model outputs the intent category among explicit approval, implicit approval, partial modification, questioning and probing, explicit denial, and irrelevant input; The predicted probability of the reading intent category is used as the first confidence level; When the predicted probability is lower than the preset confidence condition, the dialogue fragment containing the user's subsequent text will be sent to the non-training area.

4. The implicit feedback acquisition and incremental training method based on subsequent user semantics according to claim 1, characterized in that, The intention propensity score is formed by multiplying the polarity baseline of the intention category by the first confidence level, and further includes: Polarity baseline values, distributed from positive to negative, are set for explicit approval, implicit approval, partial correction, questioning and inquiry, explicit denial, and irrelevant inputs, respectively. The polarity baseline value of the selected intent category is multiplied by the first confidence level to form the intent tendency score.

5. The implicit feedback acquisition and incremental training method based on subsequent user semantics according to claim 1, characterized in that, Copying, regenerating, and initiating new sessions are mapped to a comprehensive behavior score based on direction and intensity, further including: The copying behavior is mapped to a positive behavior score based on the proportion of copied content to the length of the basic large model output text; Based on the number of regenerated triggers and the timing of the regenerated triggers, the regenerated behavior is mapped to a negative behavior score; Weak negative behavior scores are generated based on the behavior of initiating new sessions; A comprehensive behavioral score is obtained by aggregating positive behavior scores, negative behavior scores, and weakly negative behavior scores based on preset internal weights.

6. The implicit feedback acquisition and incremental training method based on subsequent user semantics according to claim 1, characterized in that, The ratio of the time interval to the length of the output text of the base model is mapped to a time interval score, which further includes: The relative reading time ratio is obtained by dividing the time interval between the completion of the output and the subsequent text generation by the length of the basic large model output text. The relative reading time ratio was compared with the unread threshold, the normal reading interval, and the timeout threshold, respectively. Segmented mapping is performed on the relative reading time ratios below the unread threshold, the relative reading time ratios within the normal reading range, and the relative reading time ratios exceeding the timeout threshold to obtain time interval scores.

7. The method for implicit feedback acquisition and incremental training based on subsequent user semantics according to claim 1, characterized in that, The cumulative number of rounds in the current session is input into a saturated growth function to form a session continuation score, which further includes: Add the current session's cumulative rounds to the scaling constant to obtain the round-normalized denominator; The session continuation score is obtained by dividing the current cumulative number of rounds by the round-normalized denominator. The session continuation score is updated based on a saturation growth function as the cumulative number of rounds in the current session increases.

8. The method for implicit feedback acquisition and incremental training based on subsequent user semantics according to claim 1, characterized in that, Based on implicit feedback comprehensive scoring, second confidence level, first threshold, and second threshold, dialogue segments are selected and training labels are generated, which further includes: When the implicit feedback comprehensive score is higher than the first threshold and the second confidence level meets the preset confidence conditions, the dialogue segment is marked as a positive sample; When the implicit feedback overall score is lower than the second threshold and the second confidence level meets the preset confidence conditions, the dialogue segment is marked as a negative sample; When the implicit feedback comprehensive score is between the first and second thresholds or the second confidence level does not meet the preset confidence conditions, the dialogue segment is sent to the non-training area.

9. The method for implicit feedback acquisition and incremental training based on subsequent user semantics according to claim 1, characterized in that, The updates to the learnable weights, the first threshold, and the second threshold include: Anchor point samples are constructed from dialogue fragments that simultaneously contain explicit feedback records and complete implicit behavior records; Transform explicit consent records and explicit disagreement records into monitoring targets; Supervised calibration of learnable weights is performed using intention propensity score, comprehensive behavioral score, time interval score, conversation continuation score, and comprehensive implicit feedback score as calibration inputs. The first and second thresholds are adjusted based on the consistency ratio between the newly added explicit feedback and the training labels.

10. An implicit feedback acquisition and incremental training system based on subsequent user semantics, characterized in that, include: The segment acquisition module is configured to acquire dialogue segments, which include the output text of the basic large model, the user's subsequent text, copy behavior, regeneration behavior, new session initiation behavior, the time interval from the completion of output to the generation of the user's subsequent text, and the current cumulative number of rounds in the session. The intent scoring module is configured to perform semantic classification on the user's subsequent text to obtain the intent category and the first confidence level. The polarity baseline value of the intent category is multiplied by the first confidence level to form an intent tendency score. The behavior time scoring module is configured to map copy behavior, regenerate behavior, and new session initiation behavior to a comprehensive behavior score according to direction and intensity, map the ratio of time interval to the length of the text output by the basic large model to a time interval score, and input the cumulative rounds of the current session into a saturation growth function to form a session continuation score. The fusion screening module is configured to input the intention tendency score, behavior comprehensive score, time interval score and conversation continuation score into the activation function after being weighted by learnable weights to obtain the implicit feedback comprehensive score and the second confidence level. The training control module is configured to filter dialogue segments and generate training labels based on implicit feedback comprehensive score, second confidence level, first threshold and second threshold, construct fine-tuning dataset, adjust sample selection conditions or training loss weights based on the quantity relationship of positive and negative samples in fine-tuning dataset, and perform incremental training on the base large model based on fine-tuning dataset.

Images