A voice assistant model-oriented behavior index driven training method, system, device and medium

Abstract

The application discloses a kind of behavior index drive training methods, systems, equipment and medium for voice assistant model, it is related to voice assistant technical field, including: training data and special package, multi-stage training, raw inference output, cleanup post-processing, raw / clean double-track record, behavior index evaluation, behavior gate / promotion, high-risk sample extraction, artificial review, ablation analysis and data package rule reflux.The application combines multi-stage training, raw output and cleaning output double-track record, behavior index evaluation, stage gate, post-processing credible check, artificial review sample extraction, ablation verification and training data reflux to improve the reliability of voice assistant in real voice interaction, high-risk security blocking and multi-round task recovery scene.

Description

Technical Field

[0001] This invention relates to the field of voice assistant technology, and more specifically to a behavior-based training method, system, device, and medium for voice assistant models. Background Technology

[0002] With the application of large language models in scenarios such as voice assistants, accessibility assistance, personal assistants, intelligent customer service, medical and health Q&A, and in-vehicle voice systems, the models not only need to generate natural language answers, but also need to complete actions such as reminder settings, information confirmation, safety prompts, risk blocking, and user intent restoration in multi-turn voice interactions. For user groups such as visually impaired users who heavily rely on voice interaction, whether the model's output is concise, audible, reliable, and safe will directly affect the task completion effect and usage risks.

[0003] Current voice assistant model training and evaluation processes typically use text similarity, exact matching rate, total score, training loss, or hit rate of general safety keywords as the main evaluation criteria. These metrics can reflect a certain degree of text output quality, but they are difficult to fully reflect the task completion ability, risk control ability, and multi-turn recovery ability in real voice interaction.

[0004] Therefore, in view of the shortcomings of existing technologies, how to provide a behavior-based training method, system, device and medium for voice assistant models is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of this, the present invention provides a behavior index-driven training method, system, device and medium for voice assistant models, which combines multi-stage training, dual-track recording of raw output and cleaned output, behavior index evaluation, stage gating, post-processing credibility verification, manual review of sample extraction, ablation verification and training data feedback to improve the reliability of voice assistants in real voice interaction, high-risk security blockade and multi-round task recovery scenarios.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a behavior-based training method for voice assistant models, comprising: Construct a multi-stage training process for voice assistants, wherein the multi-stage training process includes one or more training stages; Configure corresponding training data packages and evaluation datasets for each training phase; Construct candidate models, and use these candidate models to infer the evaluation dataset to obtain the original output; Post-processing is performed on the original output to obtain the cleaned output, and both the original output and the cleaned output are saved. Behavioral metrics are calculated based on the original output, cleaned output, reference answers, and interaction context information. Based on the gating of the execution phase of the aforementioned behavioral indicators and the model promotion judgment, the model promotion judgment adopts a behavior rule priority strategy. When a candidate model passes the preset set of behavior rules, it is marked as promoted. For models that do not advance, post-processing is performed to verify their reliability and detect whether there is any masking of model failure, thus obtaining diagnostic fields. Automatically extract manually reviewed samples by combining diagnostic fields; Configure review fields for the manually reviewed sample to obtain the manual review results; Ablation analysis was conducted based on manual review results, behavioral indicators at each stage, and diagnostic fields to obtain ablation analysis results. Based on behavioral indicators, post-processing diagnosis, manual review results, and ablation analysis results, at least one of the following—failed samples, training data packages, post-processing rules, or stage gating rules—is fed back to update the training and evaluation closed loop.

[0007] Preferably, post-processing is performed to verify its reliability and detect whether there is a masking model failure, including: Determine if the original output and the cleaned output are different; Calculate the first similarity between the original output and the reference answer; Calculate the second similarity between the cleaned output and the reference answer; Detect whether the original output contains duplicates, contamination markers, leak warnings, empty outputs, trailing questions, or excessively long outputs.

[0008] Preferably, the candidate model is an autoregressive large language model based on the Transformer decoder architecture, trained using the LoRA parameter fine-tuning method; The LoRA update expression is: W'=W+(α / r)BA; Where W is the frozen base model weight matrix, A and B are low-rank trainable matrices, r is the LoRA rank, and α is the scaling factor. The training process freezes the backbone parameters of the base model, and the LoRA adapter is deployed in the attention projection layer.

[0009] Preferably, the training data package includes one or more of the following: basic behavior data package, ASR fault tolerance data package, security and trust calibration data package, repeat control data package, and cleanup regression data package.

[0010] Preferably, the behavioral indicator calculation logic includes three types: rule-based indicators, hybrid indicators, and LLMJudge-type indicators; The rule-based indicators directly form single-sample judgments based on required_actions, forbidden_actions, control_intent, risk_family, and output text rules; The hybrid index first executes deterministic rules, and then calls a model reviewer with a temperature of 0 for samples where the rules cannot determine the outcome. The LLM Judge type metric uses fixed prompts to perform pairwise comparisons or scoring.

[0011] Preferably, the applicable sample set for a certain indicator m is set as S_m, and the number of samples is N_m; I(·) is an indicator function, which takes the value 1 when the condition is met, and 0 otherwise; The formulas for calculating the pass rate of various indicators are as follows: M_pass=(1 / N_m)×ΣI(pass_i); The formulas for calculating the violation rate of various indicators are as follows: M_violation=(1 / N_m)×ΣI(violation_i).

[0012] Preferably, the behavioral metrics include one or more of the following: task completion rate, clarification recovery rate, necessary clarification accuracy, minimum clarification rate, uncertainty calibration rate, false completion claim rate, overconfidence rate, risk blocking rate, safe next step suggestion rate, repeated control compliance rate, first step hit rate, slow segmentation rate, tail suppression rate, prompt leakage rate, and stability.

[0013] Preferably, a behavior metric-driven training system for voice assistant models includes: A multi-stage training module is used to construct a multi-stage training process for voice assistants, wherein the multi-stage training process includes one or more training stages. The data package management module is used to configure the corresponding training data package and evaluation dataset for each training stage; The inference and dual-track recording module is used to construct candidate models, perform inference on the evaluation dataset based on the candidate models to obtain the original output, perform post-processing on the original output to obtain the cleaned output, and save the original output and the cleaned output. The behavior indicator calculation module is used to calculate behavior indicators based on the original output, cleaned output, reference answer, and interaction context information. The behavior gating and promotion module is used to perform stage gating and model promotion judgment based on the behavior indicators. The model promotion judgment adopts a behavior rule priority strategy. When a candidate model passes the preset behavior rule set, it is marked as promoted. The trust verification module is used to perform trust verification on the post-processing of models that have not been upgraded, to detect whether there is any masking of model failure, and to obtain diagnostic fields. The manual review sample extraction module is used to automatically extract manually reviewed samples by combining diagnostic fields. The manual review end is used to configure review fields for the manually reviewed sample and obtain the manual review result; The ablation analysis module is used to perform ablation analysis based on manual review results, behavioral indicators at each stage, and diagnostic fields to obtain ablation analysis results. The closed-loop backflow module is used to update at least one of the following based on behavioral indicators, post-processing diagnosis, manual review results, and ablation analysis results: failed samples, training data packages, post-processing rules, or stage gating rules, thereby forming a training and evaluation closed loop.

[0014] Preferably, a computer device includes a memory and a processor, wherein the memory stores a computer program, characterized in that the processor executes the computer program to implement the steps of the above-described behavior index-driven training method for a voice assistant model.

[0015] Preferably, a computer-readable storage medium stores a computer program thereon, characterized in that, when the computer program is executed by a processor, it implements the steps of the above-described behavior index-driven training method for a voice assistant model.

[0016] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a behavior-based training method, system, device, and medium for voice assistant models. Compared with existing training and evaluation methods that mainly rely on text similarity, total score, or offline loss, the present invention has the following beneficial effects: (1) The evaluation results are closer to the real voice interaction goals: the behavioral indicators are used as the main basis for advancement, so that the model evaluation shifts from text similarity to real behavioral performance such as task completion, clarification and recovery, risk blocking, trust calibration, and repetition control.

[0017] (2) Improve the task recovery capability in ASR error and multi-round confirmation scenarios: It is implemented by the ASR error recovery training stage and the corresponding behavior gate, which can be used for special evaluation of scenarios such as entity misidentification, time misidentification, and contact misidentification.

[0018] (3) Improve reliability and security blocking capabilities in high-risk scenarios: This is achieved through indicators such as calibrable trust and reliability training phase, uncertainty calibration rate, false completion claim rate, overconfidence rate and risk blocking rate.

[0019] (4) Improve the availability of responses in voice-audible interactive scenarios: This is achieved by indicators such as repetitive and slow control training phases, repetitive control compliance rate, first-step hit rate, slow segmentation rate, and tail suppression rate. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0021] Figure 1 The overall flowchart of the closed-loop training and evaluation driven by behavior indicators provided by this invention is shown.

[0022] Figure 2 This is a diagram illustrating the multi-stage training and evaluation structure provided in an embodiment of the present invention.

[0023] Figure 3 The flowchart for raw / clean dual-track trusted verification provided in the embodiments of the present invention is shown.

[0024] Figure 4 The behavioral indicator Gate and model advancement logic diagram provided for embodiments of the present invention.

[0025] Figure 5 A flowchart for manually reviewing sample extraction provided in an embodiment of the present invention.

[0026] Figure 6 This is a connection structure diagram of the training and evaluation closed-loop device module provided in an embodiment of the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] This invention discloses a behavior-based training method for voice assistant models, comprising: A multi-stage training process for voice assistants is constructed, wherein the multi-stage training process includes one or more training stages; the multi-stage training process includes at least one or more of the following: behavior-based training stage, speech recognition error recovery training stage, calibrable trust and reliability training stage, and repetitive and slow control training stage.

[0029] Configure corresponding training data packages and evaluation datasets for each training phase; Construct candidate models, and infer the evaluation dataset based on the candidate models to obtain the raw response. The original output is processed using a cleanup post-processing method to obtain a clean response output. Both the original output and the clean response output are saved. The cleanup post-processing can remove content such as leaky text, duplicate segments, contamination markers, formatting tails, or redundant follow-up questions. Behavioral metrics are calculated based on the original output, cleaned output, reference answers, and interaction context information. Based on the gating of the behavioral indicators and the model promotion judgment, the model promotion judgment adopts a behavior rule priority strategy. When a candidate model passes the preset set of behavioral rules, it is marked as promoted. The historical baseline total score, text similarity, or the total score increment relative to the historical baseline can be used as observation items or auxiliary indicators.

[0030] For models that do not advance, post-processing is performed to verify their reliability and detect whether there is any masking of model failure, thus obtaining diagnostic fields. The system automatically extracts high-value manually reviewed samples based on diagnostic fields. The extraction criteria include post-processing rewriting, low similarity between clear output and reference answer, excessively long or repetitive original output, high-risk security scenarios, ASR (Automatic Speech Recognition) error correction scenarios, repeat / control scenarios, stage-gated critical samples, and samples where there is a discrepancy between automatic scoring and human expectations.

[0031] Configure audit fields for the manually reviewed samples to obtain the manual audit results; the audit fields may include one or more of human_accept, failure_type, severity, preferred_response, and notes. The system can also generate an audit_focus field to prompt auditors to pay attention to issues such as whether post-cleanup processing concealed failures, whether the semantics deviated from the reference answer, whether risk blocking was sufficient, whether the task was completed after ASR correction, and whether slow or step-by-step restatements were followed.

[0032] Ablation analysis was conducted based on manual review results, behavioral indicators at each stage, and diagnostic fields to obtain ablation analysis results. Based on behavioral indicators, post-processing diagnosis, manual review results, and ablation analysis results, at least one of the following—failed samples, training data packages, post-processing rules, or stage gating rules—is fed back to update the training and evaluation closed loop.

[0033] like Figure 1 The diagram shown is the overall flowchart of the behavior index-driven training and evaluation closed loop according to an embodiment of the present invention. It shows the training data and special packages, multi-stage training, raw inference output, cleanup post-processing, raw / clean dual-track recording, behavior index evaluation, behavior gate / promotion, high-risk sample extraction, manual review, ablation analysis, and data package rule feedback.

[0034] The method of this invention can be executed by a server, training and evaluation platform, cloud host, edge computing device or other computer device with data processing function, and is used to perform multi-stage training, evaluation, post-processing trust verification, manual review of sample extraction, ablation analysis and closed-loop backflow on a voice assistant model.

[0035] In one embodiment of the present invention, the training and evaluation platform uses Qwen3-14B-Base as the base model and constructs candidate models using an efficient LoRA parameter fine-tuning method based on ms-swift. The platform does not perform a single unified training iteration, but rather executes the training in the following order: "Data Quality Gating—Behavioral Foundation Training—ASR Fault Tolerance and Clarification Recovery Training—Repetitive and Slow Control Training—Security and Trust Calibration Training—Cleanup Anomaly Suppression and Regression—Template and Agent Layer Blind Testing." Each stage has corresponding training data packets, evaluation datasets, and pass_rules. After all rules in the current stage pass, the platform merges the LoRA checkpoints of that stage with the base model or the merged model from the previous stage, and uses the resulting model as the initialization model for the next stage. If any stage fails, the system records rule_failures, adjusts the data packet weights or training hyperparameters, and retryes until the maximum number of attempts is reached, at which point the pipeline stops. Table 1 shows the processing procedure for each stage.

[0036] Table 1 Processing procedures at each stage

[0037] In this embodiment of the invention, a separate metric profile, calibrated_trust_fit, is reserved. Uncertainty calibration, false completion claims, and overconfidence can be handled as separate training / evaluation phases. The current main stageflow incorporates these reliability capabilities into safety_fit. Both of these organizational methods belong to the multi-stage implementation methods of this invention.

[0038] Specifically, post-processing is performed to verify its reliability and detect whether there are any masking model failures, including: Determine if the original output and the cleaned output are different; Calculate the first similarity between the original output and the reference answer; Calculate the second similarity between the cleaned output and the reference answer; Detect whether the original output contains duplicates, contamination markers, leak warnings, empty outputs, trailing questions, or excessively long outputs.

[0039] Based on the detection results, determine whether post-processing might mask model failure.

[0040] like Figure 3 The diagram shows the raw / clean dual-track trusted verification process, which includes the model's original output, cleanup processing, similarity comparison, duplicate and leak detection, and the determination of whether to cover up failures.

[0041] Specifically, the judgment, calculation, and detection process in the post-cleanup processing is as follows: (1) Dual-track recording. For each evaluation sample, both raw_response and clean_response are saved simultaneously, and cleanup_changed, raw_response_chars, clean_response_chars, raw_similarity_to_label, similarity_to_label, split, priority_score, and sample_id are recorded. cleanup_changed can be determined by whether raw_response and clean_response are different after uniform blank normalization.

[0042] (2) Length calculation. Calculate raw_len and clean_len respectively, and calculate the compression ratio R_len=raw_len / max(clean_len,1). In this embodiment of the invention, when raw_len≥120 and R_len≥2.5, mark raw_clean_length_collapse; when clean_len<16, mark clean_too_short.

[0043] (3) Prompt or role leak detection. If raw_response matches the original system prompt, role tags starting with system / assistant / user, script tags such as ".cout / .assistant / .system", or ChatML tags such as <|im_start|> / <|im_end|>, then raw_prompt_or_role_leak is marked.

[0044] (4) Repetition detection. The output is segmented by period, question mark, exclamation mark or newline. If the same sentence with a length of no less than 6 characters appears at least 3 times, it is judged as sentence-level repetition; if the 12-character n-gram after removing whitespace appears at least 4 times, it is judged as segment repetition; if the same character appears at least 10 times consecutively, it is judged as character screen flooding. Any success is marked as raw_repetition.

[0045] (5) Semantic change detection. Calculate the first similarity S_raw between raw and the reference answer, and the second similarity S_clean between clean and the reference answer. Mark low_clean_similarity_after_cleanup when cleanup_changed=true and S_clean<0.18; mark raw_clean_similarity_gap when |S_raw-S_clean|≥0.25.

[0046] (6) Task closure and abnormal content detection. If the end of raw or clean contains the pattern "Do you need me...?" or "Do you need me...?", then mark tail_question_after_action; if it contains a mobile phone number, landline number or test number without context, then mark fabricated_contact_or_number.

[0047] (7) High-value scenario weighting. When a split belongs to safety, asr, repeat or hard, and priority_score≥9 and cleanup_changed=true, mark it as high_risk_split_priority to increase its priority for manual review.

[0048] (8) Blocking Gate. If any of the following strong risk factors occur: raw_prompt_or_role_leak, raw_repetition, raw_clean_length_collapse, clean_too_short, or fabricated_contact_or_number, then blocking_gate_flagged=true, which can directly prevent the current candidate model from advancing or require a re-evaluation.

[0049] (9) High Recall Gate. In addition to the strong risk signals mentioned above, when cleanup_changed occurs simultaneously with at least one of the following: low clean similarity, tail questioning, or high priority of high-risk scenarios, high_recall_gate_flagged=true. This gate is used to recall as many failed samples as possible that may have been covered up by cleanup and send them for manual review. It is not directly equivalent to the final failure conclusion.

[0050] (10) Manual truth verification. Human reviewers fill in cleanup_masked_failure, human_accept, auto_failure_confirmed, metric_false_positive, failure_type, severity, and preferred_spoken_response. Based on this, the system calculates TP, FP, FN, TN, precision, and recall for high recall gates or blocking gates, and returns the missed samples to cleanup regression test cases.

[0051] The manually reviewed project examples were selected in stratification: 10 safety-related, 10 ASR-related, 15 repeat-related, and 15 hard-related. Candidate samples were first sorted by priority_score, cleanup_changed, lower clean similarity, and longer raw output. Then, an audit_focus was generated to prompt reviewers to pay attention to safety blocking, task completion, audibility, repetition control, false completion claims, overconfidence, and whether cleanup attempts concealed failures.

[0052] Specifically, the candidate model is an autoregressive large language model based on the Transformer decoder architecture, with the base model being Qwen3-14B-Base. This specific model is only an example and does not constitute a limitation on the brand, parameter size, or number of network layers of the base model; other autoregressive models capable of receiving multi-turn text context and generating voice assistant responses can also be used.

[0053] During training, the basic parameters are frozen, and a LoRA low-rank adapter is set in one or more of the attention projection layers q_proj, k_proj, v_proj, and o_proj, and one or more of the feedforward networks gate_proj and down_proj.

[0054] Training is performed using the LoRA parameter fine-tuning method; The LoRA update expression is: W'=W+(α / r)BA; Where W is the frozen base model weight matrix, A and B are low-rank trainable matrices, r is the LoRA rank, and α is the scaling factor.

[0055] In the configuration of this embodiment, r can be 8 or 16, α can be 16 or 32, dropout can be 0.05 or 0.10, and the learning rate can be 5×10^-6 to 1.5×10^-5.

[0056] The training and evaluation samples adopt a ChatML-like multi-turn message structure, which includes at least the sample identifier, sample group, user message, and a final-turn assistant message as a reference answer. It can also include metadata such as control_intent, expected_first_step, asr_error_type, required_clarification, risk_family, required_actions, forbidden_actions, strict_exact, require_no_extra_tail, risk_level, and historical context. The input contract for the project evaluator can also be organized into sample_id, group, user_query, reference_answer, candidate_answer, and optional tags, meta tags, evaluation, and context_messages.

[0057] (1) The training and evaluation platform reads the current stage configuration, selects the special data package, data package weight, LoRA hyperparameter and evaluation group for this stage, and generates the training recipe.

[0058] (2) The model training service performs supervised LoRA fine-tuning on the base model or the merged model from the previous stage according to the recipe, forming the candidate model and LoRA checkpoints for the current stage.

[0059] (3) The inference service encodes system instructions, multi-round interaction context, current user input and sample metadata into a token sequence, which is then decoded by the candidate model through autoregression. In the project evaluation configuration, the temperature can be set to 0 and the maximum number of newly generated tokens can be set to 256 to reduce random fluctuations.

[0060] (4) Save the text directly generated by the model and not yet cleaned or security controlled as raw_response, and record sample_id, split, reference answer, context, character length and inference state in sync.

[0061] (5) Send the raw_response to cleanup or system auxiliary security controller to delete role leakage, duplicate pollution, redundant end sentences or unfounded false completion statements, and obtain clean_response; at the same time, retain diagnostic fields such as modification reason, rule number and length before and after.

[0062] (6) The evaluator performs rule-based scoring, hybrid scoring, or model-assisted scoring based on raw_response, clean_response, reference answer, and sample metadata, respectively, and generates structured behavioral indicators.

[0063] (7) The stage controller reads pass_rules and uses comparison operations such as >=, <=, ==, !=, in or contains to judge each item; if all are satisfied, stage_passed=true, and if any is not satisfied, the specific rule_failures are output.

[0064] (8) After the stage is passed, the platform reads the last_model_checkpoint, merges the LoRA checkpoint with the corresponding basic model into a merged model that can be loaded independently, and uses the merged model as the initialization model for the next stage; if the stage fails, the platform adjusts the parameters such as the weight, rank, learning rate, and number of steps of the special data package and then tries again.

[0065] Specifically, the training data package includes one or more of the following: basic behavior data package, ASR fault tolerance data package, security and trust calibration data package, repeat control data package, and cleanup regression data package.

[0066] like Figure 4 The diagram shows the logic of the behavior metrics gate and promotion, displaying ASR metrics, trust metrics, safety metrics, repeat / control metrics, global metrics, and model advancement strategies.

[0067] Specifically, the behavioral indicator calculation logic includes three categories: rule-based indicators, hybrid indicators, and LLMJudge-type indicators; The rule-based indicators directly form single-sample judgments based on required_actions, forbidden_actions, control_intent, risk_family, and output text rules; The hybrid index first executes deterministic rules, and then calls a model reviewer with a temperature of 0 for samples where the rules cannot determine the outcome. The LLM Judge type metric uses fixed prompts to perform pairwise comparisons or scoring.

[0068] Specifically, let S_m be the applicable sample set for a certain indicator m, and let N_m be the number of samples; I(·) is an indicator function, which takes the value 1 when the condition is met, and 0 otherwise; The formulas for calculating the pass rate of various indicators are as follows: M_pass=(1 / N_m)×ΣI(pass_i); The formulas for calculating the violation rate of various indicators are as follows: M_violation=(1 / N_m)×ΣI(violation_i).

[0069] Precision is TP / (TP+FP), recall is TP / (TP+FN), and continuous rating is the arithmetic mean of single-sample ratings.

[0070] Specifically, the behavioral indicators include one or more of the following: task completion rate, clarification recovery rate, necessary clarification accuracy, minimum clarification rate, uncertainty calibration rate, false completion claim rate, overconfidence rate, risk blocking rate, safe next step suggestion rate, repeated control compliance rate, first step hit rate, slow segmentation rate, tail suppression rate, warning leakage rate, and stability. The judgment process for each indicator is shown in Table 2.

[0071] Table 2 Judgment process for each indicator

[0072] Specifically, the behavioral foundation stage requires a concise pass rate of no less than 0.88 and a tail suppression rate of no less than 0.90; the ASR stage requires a task completion rate of no less than 0.78 and a clarification recovery rate of no less than 0.75; the repetitive control stage requires a control compliance rate of no less than 0.82 and both the first-step hit rate and slow segmentation rate of no less than 0.78; and the safety stage requires a risk identification recall rate of no less than 0.90, a dangerous action blocking rate of no less than 0.92, and a safe next step suggestion rate of no less than 0.80. The above values ​​are only for the current configuration example of this project, and the actual scope of protection applied for is not limited to these values.

[0073] Specifically, the implementation process of the phase gate is as follows: (1) The stage gating of the ASR error recovery training stage may include ASR task completion rate not lower than the first threshold, clarification recovery rate not lower than the second threshold, necessary clarification accuracy not lower than the third threshold, minimum clarification rate not lower than the fourth threshold, and prompt leakage rate not higher than the fifth threshold.

[0074] (2) The stage gating of the calibrable trust and reliability training stage may include uncertainty calibration rate not lower than the sixth threshold, false completion declaration rate not higher than the seventh threshold, overconfidence rate not higher than the eighth threshold, risk blocking rate in high-risk scenarios not lower than the ninth threshold, and global stability not lower than the tenth threshold.

[0075] (3) The stage gating of the repetitive and slow control training phases may include the repetitive control compliance rate, first step hit rate, slow segmentation rate and tail inhibition rate, which respectively meet the preset thresholds.

[0076] Specifically, the implementation process of the repetitive and slow speed control effect is as follows: (1) The training samples or evaluation samples are labeled with control intents such as “repeat, slow down, only say the key points, only say the first step, split into two sentences, continue, next step, stop for a while” through control_intent, and the previous round of valid responses are saved through context_messages, and the expected first step is labeled through expected_first_step.

[0077] (2) The model does not expand the task again under repeated requests, but maintains the core facts, order and risk constraints of the previous round; deletes explanatory groundwork under the "just say the key points" request; places the target step in the first sentence under the "first step" request; and resumes from the interrupted position of the previous round under the "continue" request.

[0078] (3) For requests to "slow down" or "split into two sentences", the current project mainly implements slow-speed audibility at the text generation layer, that is, it breaks down long replies into multiple phrase semantic units, reduces list-like tone and complex clauses, and suppresses additional follow-up questions at the end. If the deployment end has a speech synthesis control interface, it can also reduce the TTS speech rate or increase the pauses between sentences according to the segmentation boundaries; this TTS control is an equivalent extended implementation method.

[0079] (4) During the evaluation, the control compliance rate, first-step hit rate, repeat fidelity rate, slow segmentation rate, tail suppression rate, and echo repeat violation rate are calculated respectively. The main project process passes this stage only when the control compliance rate is not less than 0.82, the first-step hit rate is not less than 0.78, the slow segmentation rate is not less than 0.78, and the tail suppression rate is not less than 0.90.

[0080] The technical effect is not achieved solely through changes in playback speed, but through a combined process of "control intent recognition, historical response citation, short sentence recombination, first-step priority, tail suppression, and behavioral indicator gating," enabling auditory users to more easily understand and continue completing the task.

[0081] Reduce the risk of post-processing masking model failures: Implemented by raw / clean dual-track recording and cleanup trusted verification, it can detect situations where valid content is accidentally deleted, outputs fail to be pruned, or polluted outputs are masked during post-processing.

[0082] Reduce manual review costs and improve review targeting: This is achieved through an automatic sampling mechanism for high-value manual review samples, enabling manual review to focus on samples with high risk, high uncertainty, unstable automatic evaluation, or significant post-processing changes.

[0083] Improve the interpretability, reproducibility, and auditability of training schemes: This is achieved through ablation matrix and closed-loop backflow mechanism, which can verify the contribution of different training packages, post-processing rules, and gating rules to the improvement of behavioral capabilities.

[0084] A continuous improvement mechanism is established: the closed-loop feedback module incorporates failed samples, review conclusions, post-processing rule repairs, and gating rule adjustments into the next round of training and evaluation, thereby achieving continuous iteration of the training and evaluation process.

[0085] In embodiments of the present invention, such as Figure 2 The diagram shows the multi-stage training and evaluation structure, illustrating stages such as behavior fit, ASR recovery fit, calibrated trust / reliability fit, repeat / control fit, cleanup regression, and ablation / human audit. The multi-stage training process can be executed in the order of behavior foundation, ASR error recovery, trust calibration, and repeat control. The order of stages can also be adjusted according to business risks, or multiple stages can be merged into a single comprehensive training stage.

[0086] Specifically, the current project adopts a sequential phased pipeline; alternatively, the priority of each capability can be calculated based on the risk distribution of the target business, and safety_fit can be moved forward to before the ASR or repeat phase. Alternatively, ASR, repeat, and safety-specific packages can be mixed into a comprehensive training recipe according to their weights. After training, the corresponding behavioral indicators are still calculated on each independent evaluation group. Only when all core gates are passed can the project be promoted.

[0087] In this embodiment of the invention, the training data package may use manually labeled data, automatically constructed data, online failure samples, simulated ASR error samples, historical manually reviewed samples, or a combination thereof.

[0088] Specifically, manually labeled samples are used for high-risk and boundary conditions; automated construction programs can generate misspelled, missing, or near-homophone words for entities, times, contacts, and actions; online failure logs are de-identified and converted into bad cases; during manual review, the failure_type, preferred_spoken_response, and cleanup_masked_failure fields can be routed back to ASR, repeat, security, or cleanup data packets, respectively. Before routement, unique ID, no exact overlap, and manual review gating are still performed.

[0089] In this embodiment of the invention, behavioral indicators can be calculated using rule-based calculation, model-assisted scoring, manual annotation scoring, or a combination of multiple methods.

[0090] Specifically, rules are used for highly deterministic prompts, tail-end questions, false completion statements, and length metrics; a hybrid approach of rules and LLM judges is used for semantic task completion, clarification recovery, and risk assessment; and final labels are given manually for highly controversial or high-risk samples. The fusion process can adopt a "rules-first, model-supplemented, human-decided" approach, or weights can be assigned to each score to form a comprehensive confidence level.

[0091] In this embodiment of the invention, the stage gating can use a fixed threshold, or it can use a dynamic threshold, quantile threshold, or multi-index weighted rule relative to the historical baseline.

[0092] Specifically, the current project mainly uses fixed threshold pass_rules; dynamic thresholds can also be set based on historical baseline mean and standard deviation, or the candidate model indicators can be set to preset increments of historical best values; different weights can also be set according to business risks such as security, ASR, and repeat, and it is required that no security hard indicator can be offset by other high scores.

[0093] In this embodiment of the invention, the model advancement strategy can prioritize behavioral rules or adopt a composite strategy that combines behavioral rules with total score, similarity, and manual review pass rate.

[0094] Specifically, the current configuration requires `require_pass_gate=true`, and the total score increment relative to the best baseline can be used as an auxiliary condition. The model can also first pass all behavioral hard rules, and then compare the weighted total score, similarity, LLM Judge win rate, and manual review pass rate. If the total score improves but the safety or false completion indicators fail, the model will still not be allowed to advance.

[0095] In this embodiment of the invention, the cleanup post-processing can be implemented based on rules, or based on classification models, sequence labeling models, language model judgments, or a combination of multiple strategies.

[0096] Specifically, regular expressions can be used to remove role tags, script markers, and duplicate segments; classification models can be trained to identify clue leaks, contamination, tail-ending questions, and fake numbers; sequence labeling models can be used to determine the character ranges to be deleted; or a language model can be called to rewrite the code. Regardless of the method used, the original output, cleaned output, reasons for modification, and rule version should all be retained.

[0097] In this embodiment of the invention, the cleanup trusted verification can compare the length, similarity, keywords, risky actions, warning leakage markers, duplicate segments, or semantic integrity of the raw response and the clean response.

[0098] Specifically, in addition to the existing rules for length collapse, excessively short output, duplication, leakage, poor similarity, and fictitious numbers in this embodiment, task slots, risk actions, and security suggestions can be extracted to compare whether the clean function misses key slots; or an independent review model can be used to compare the semantic completeness of the raw, clean, and reference answers. Strong risk signals enter the blocking gate, while weak risk signals enter the high-recall manual review gate.

[0099] In this embodiment of the invention, manual review of sample extraction can employ methods such as rule-based sampling, risk-stratified sampling, active learning sampling, and a combination of random sampling and high-risk sampling.

[0100] like Figure 5 The flowchart shown is for the manual review sample extraction process, illustrating the extraction logic for cleanup_changed, low similarity, raw output that is too long or repetitive, high-risk samples, ASR samples, repeat / control samples, and gate critical samples.

[0101] Specifically, the current implementation uses a stratified quota extraction method (safety / ASR / repeat / hard) to extract 50 entries; alternatively, active learning can be used, prioritizing entries based on model scoring uncertainty, automatic scorer divergence, gate critical distance, or historical false alarm rate. To prevent observing only failed samples, a certain proportion of random samples can be added as controls.

[0102] In this embodiment of the invention, ablation analysis can remove the ASR-specific package, the trust / safety-specific package, the cleanup post-processing, and the repeat / control-specific package, or it can be replaced with a control experiment with different training data sources, different gating rules, or different post-processing strategies.

[0103] Specifically, using the complete scheme A0 as a baseline, A1 removes the ASR-specific package, A2 removes the safety / trust-specific package, A3 disables cleanup or the safety controller, and A4 adds the repeat / control-specific package; all experiments maintain consistency in the base model, evaluation set, random seed, and main training budget. Changes in task completion, clarification recovery, risk blocking, false completion, duplication control, and raw / clean risk metrics are compared to determine the sources of capability improvement. Alternatively, different LoRA ranks, packet weights, gating thresholds, or post-processing versions can be used as a control.

[0104] The embodiments of the present invention can be used for voice assistants for visually impaired users, and can also be extended to intelligent customer service, in-vehicle voice, medical and health Q&A, reminder assistant, accessibility assistance system and other model systems that require highly reliable voice interaction.

[0105] Specifically, when used in visually impaired voice assistants, the focus is on limiting visually dependent words and emphasizing short announcements; when used in intelligent customer service, required_actions are replaced with work order creation, information verification, and transfer to human agents; when used in in-vehicle voice systems, distraction prevention and dangerous operation prevention are added; when used in medical and health Q&A, expressions of uncertainty and medical advice are added; and when used in reminder assistants, task status or tool return values ​​are used to verify whether the task has been truly completed. All of the above scenarios can reuse multi-stage training, dual-track recording, behavior gates, human review, and feedback structures.

[0106] In one specific embodiment of the present invention, a behavior-based training system for voice assistant models can be deployed on servers, cloud hosts, training and evaluation platforms, edge computing devices, or other electronic devices, and can communicate with model training services, model inference services, data storage services, and manual review terminals, including: A multi-stage training module is used to construct a multi-stage training process for voice assistants. The multi-stage training process includes one or more training stages. The multi-stage training module connects the data packet management module and the model inference service, and trains candidate models according to the stage configuration to form candidate models under different stages or different ablation conditions. The data package management module is used to configure the corresponding training data packages and evaluation datasets for each training stage. The data package management module manages training data packages and evaluation datasets such as basic behavior, ASR fault tolerance, security trust, repetition control, and cleanup regression, and provides data to the multi-stage training module and behavior indicator calculation module.

[0107] The inference and dual-track recording module is used to construct candidate models, perform inference on the evaluation dataset based on the candidate models, and obtain raw output; perform post-processing on the raw output to obtain clean output, and save the raw output and clean output; the inference and dual-track recording module connects the candidate model and the post-processing module, generates raw response, performs cleanup to obtain clean response, and retains raw / clean dual-track results and related context; The behavior indicator calculation module is used to calculate behavior indicators based on raw output, cleaned output, reference answer, and interaction context information. The behavior indicator calculation module connects the reasoning and dual-track recording module and calculates behavior indicators such as ASR recovery, trust / reliability, safety, repeat / control, and global stability based on raw response, clean response, reference answer, and interaction context. The behavior gating and promotion module is used to perform stage gating and model promotion judgment based on the behavior indicators. The model promotion judgment adopts a behavior rule priority strategy. When a candidate model passes the preset behavior rule set, it is marked as promoted. The behavior gating and promotion module is connected to the behavior indicator calculation module and determines whether the candidate model passes the stage gating or model promotion according to the preset behavior rule set. The credible verification module is used to perform credible verification on post-processing for models that have not been promoted, to detect whether there is any masking of model failure, and to obtain diagnostic fields. The cleanup credible verification module is located between the post-processing module and the evaluation results. It is used to identify whether cleanup has mistakenly deleted valid outputs or masked model failures, and outputs diagnostic fields such as cleanup_changed, raw_similarity_to_label, similarity_to_label, raw_response_chars, and clean_response_chars.

[0108] The manual review sample extraction module connects the dual-track recording module, the behavioral indicator calculation module, and the credibility verification module. It is used to automatically extract manual review samples by combining diagnostic fields; extract high-risk, high-uncertainty, or suspected failure samples from the evaluation results, and generate audit focus fields such as audit_focus.

[0109] The manual review terminal is connected to the manual review sample extraction module. It is used to receive review samples and enter review results such as human_accept, failure_type, severity, preferred_response, and notes. It also configures review fields for the manual review samples and obtains the manual review results. The ablation analysis module is used to conduct ablation analysis based on manual review results, behavioral indicators at each stage, and diagnostic fields to obtain ablation analysis results; it also constructs and executes control experiments to remove specific training data packets or post-processing rules and compares changes in behavioral indicators.

[0110] The closed-loop feedback module connects the manual review end, ablation analysis module, data packet management module, and gating configuration. It is used to feed back at least one of the following to the next round of training and evaluation based on behavioral indicators, post-processing diagnosis, manual review results, and ablation analysis results: failed samples, review conclusions, data corrections, training data packets, post-processing rule repairs, or stage gating rule adjustments, thus forming a training and evaluation closed loop.

[0111] like Figure 6 The diagram showing the training and evaluation closed-loop device module connection structure illustrates the data packet management, multi-stage training, inference and dual-track recording, behavior index calculation, behavior gating and promotion, cleanup credibility verification, manual review sample extraction, ablation analysis and closed-loop backflow modules, as well as their connection relationships with model training service, model inference service, data storage service and manual review terminal.

[0112] The system comprises several modules: a data package management module provides training and evaluation data; a multi-stage training module generates candidate models; an inference and dual-track recording module saves raw and cleaned outputs; a behavior indicator calculation module calculates real-world interactive behavior; a behavior gating and advancement module performs model advancement judgments; a credibility verification module identifies and handles post-processing risks; a manual review sample extraction module submits high-value samples for manual review; an ablation analysis module verifies the source of capabilities; and a closed-loop feedback module feeds back the review and analysis results to the data, rules, and gating configurations. These modules together form a closed loop from "training data input—multi-stage training—raw / clean dual-track evaluation—behavior indicator gate—manual review and ablation—data and rule feedback."

[0113] In one specific embodiment of the present invention, a computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described behavior index-driven training method for a voice assistant model.

[0114] In one specific embodiment of the present invention, a computer-readable storage medium is provided, wherein a computer program is stored thereon, characterized in that the computer program, when executed by a processor, implements the steps of the above-described behavior index-driven training method for a voice assistant model.

[0115] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0116] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A behavior-based training method for voice assistant models, characterized in that, include: Construct a multi-stage training process for voice assistants, wherein the multi-stage training process includes one or more training stages; Configure corresponding training data packages and evaluation datasets for each training phase; Construct candidate models, and use these candidate models to infer the evaluation dataset to obtain the original output; Post-processing is performed on the original output to obtain the cleaned output, and both the original output and the cleaned output are saved. Behavioral metrics are calculated based on the original output, cleaned output, reference answers, and interaction context information. Based on the gating of the execution phase of the aforementioned behavioral indicators and the model promotion judgment, the model promotion judgment adopts a behavior rule priority strategy. When a candidate model passes the preset set of behavior rules, it is marked as promoted. For models that do not advance, post-processing is performed to verify their reliability and detect whether there is any masking of model failure, thus obtaining diagnostic fields. Automatically extract manually reviewed samples by combining diagnostic fields; Configure review fields for the manually reviewed sample to obtain the manual review results; Ablation analysis was conducted based on manual review results, behavioral indicators at each stage, and diagnostic fields to obtain ablation analysis results. Based on behavioral indicators, post-processing diagnosis, manual review results, and ablation analysis results, at least one of the following—failed samples, training data packages, post-processing rules, or stage gating rules—is fed back to update the training and evaluation closed loop.

2. The behavior-based training method for voice assistant models according to claim 1, characterized in that, Post-processing is performed to verify its reliability and detect any masking model failures, including: Determine if the original output and the cleaned output are different; Calculate the first similarity between the original output and the reference answer; Calculate the second similarity between the cleaned output and the reference answer; Detect whether the original output contains duplicates, contamination markers, leak warnings, empty outputs, trailing questions, or excessively long outputs.

3. The behavior-driven training method for voice assistant models according to claim 1, characterized in that, The candidate model is an autoregressive large language model based on the Transformer decoder architecture, trained using the LoRA parameter fine-tuning method; The LoRA update expression is: W'=W+(α / r)BA; Where W is the frozen base model weight matrix, A and B are low-rank trainable matrices, r is the LoRA rank, and α is the scaling factor. The training process freezes the backbone parameters of the base model, and the LoRA adapter is deployed in the attention projection layer.

4. The behavior-based training method for voice assistant models according to claim 1, characterized in that, The training data package includes one or more of the following: basic behavior data package, ASR fault tolerance data package, security and trust calibration data package, repeat control data package, and cleanup regression data package.

5. The behavior-based training method for voice assistant models according to claim 1, characterized in that, The behavioral indicator calculation logic includes three categories: rule-based indicators, hybrid indicators, and LLM Judge-type indicators; The rule-based indicators directly form single-sample judgments based on required_actions, forbidden_actions, control_intent, risk_family, and output text rules; The hybrid index first executes deterministic rules, and then calls a model reviewer with a temperature of 0 for samples where the rules cannot determine the outcome. The LLM Judge type metric uses fixed prompts to perform pairwise comparisons or scoring.

6. The behavior-based training method for voice assistant models according to claim 5, characterized in that, Let S_m be the applicable sample set for a certain indicator m, and N_m be the number of samples; I(·) is an indicator function, which takes the value 1 when the condition is met, and 0 otherwise; The formulas for calculating the pass rates of various indicators are as follows: M_pass=(1 / N_m)×ΣI(pass_i); The formulas for calculating the violation rate of various indicators are as follows: M_violation=(1 / N_m)×ΣI(violation_i).

7. The behavior-based training method for voice assistant models according to claim 1, characterized in that, The behavioral metrics include one or more of the following: task completion rate, clarification recovery rate, necessary clarification accuracy, minimum clarification rate, uncertainty calibration rate, false completion claim rate, overconfidence rate, risk blocking rate, safe next step suggestion rate, repeated control compliance rate, first step hit rate, slow segmentation rate, tail suppression rate, alert leakage rate, and stability.

8. A behavior index-driven training system for voice assistant models, employing the behavior index-driven training method for voice assistant models as described in any one of claims 1-7, characterized in that, include: A multi-stage training module is used to construct a multi-stage training process for voice assistants, wherein the multi-stage training process includes one or more training stages. The data package management module is used to configure the corresponding training data package and evaluation dataset for each training stage; The inference and dual-track recording module is used to construct candidate models, perform inference on the evaluation dataset based on the candidate models to obtain the original output, perform post-processing on the original output to obtain the cleaned output, and save the original output and the cleaned output. The behavior indicator calculation module is used to calculate behavior indicators based on the original output, cleaned output, reference answer, and interaction context information. The behavior gating and promotion module is used to perform stage gating and model promotion judgment based on the behavior indicators. The model promotion judgment adopts a behavior rule priority strategy. When a candidate model passes the preset behavior rule set, it is marked as promoted. The trust verification module is used to perform trust verification on the post-processing of models that have not been upgraded, to detect whether there is any masking of model failure, and to obtain diagnostic fields. The manual review sample extraction module is used to automatically extract manually reviewed samples by combining diagnostic fields. The manual review end is used to configure review fields for the manually reviewed sample and obtain the manual review result; The ablation analysis module is used to perform ablation analysis based on manual review results, behavioral indicators at each stage, and diagnostic fields to obtain ablation analysis results. The closed-loop backflow module is used to update at least one of the following based on behavioral indicators, post-processing diagnosis, manual review results, and ablation analysis results: failed samples, training data packages, post-processing rules, or stage gating rules, thus forming a training and evaluation closed loop.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the behavior index-driven training method for a voice assistant model according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the behavior index-driven training method for a voice assistant model according to any one of claims 1 to 7.

Images