Context unit scoring methods and apparatuses, computing devices

Abstract

The embodiment of the specification relates to the technical field of artificial intelligence and the technical field of natural language processing, in particular to a context unit scoring method and device and a computing device, wherein the context unit scoring method comprises the following steps: acquiring a current context unit and a plurality of candidate scoring items based on a medical dialogue flow; determining a target scoring item from the plurality of candidate scoring items based on the similarity between the current context unit and the plurality of candidate scoring items; determining a context score of the current context unit based on the similarity between the current context unit and the target scoring item; determining a target scoring strategy from a preset scoring strategy based on the context score and a preset context score threshold; and scoring the current context unit based on the target scoring strategy to obtain a target scoring result of the current context unit. The invalid evaluation of low-value or repetitive dialogue content is avoided, and therefore the accuracy and scoring efficiency of the context unit scoring result are improved.

Description

Technical Field

[0001] The embodiments in this specification relate to the fields of artificial intelligence technology and natural language processing technology, and in particular to a context unit scoring method, a context unit scoring device, and a computing device. Background Technology

[0002] With the rapid development of medical informatics and artificial intelligence technologies, real-time analysis and scoring of medical dialogues based on automatic speech recognition (ASR) and natural language processing (NLP) are being increasingly widely used in clinical quality control, diagnosis and treatment assistance, and other scenarios.

[0003] Currently, scoring medical conversations mainly relies on fixed triggering mechanisms to invoke the scoring model. Some methods use fixed time periods to periodically send accumulated conversation text to the scoring model for scoring; others, in order to improve the real-time performance of the scoring, trigger the model to score immediately every time the speech recognition output is updated.

[0004] However, the aforementioned technical solutions have certain technical problems in practical applications. When facing scenarios such as streaming medical dialogues, scoring methods based on fixed time periods are prone to producing semantically incomplete fragments due to mechanically segmenting dialogue content according to time points, leading to decreased scoring accuracy. Furthermore, immediately triggering the model to score each time the speech recognition output is updated lacks the ability to reuse historical scoring results, resulting in repeated scoring of the same or similar content and wasting computational resources and costs. Therefore, there is an urgent need for a contextual unit scoring method with higher accuracy and scoring efficiency. Summary of the Invention

[0005] In view of this, embodiments of this specification provide a method for scoring contextual units. One or more embodiments of this specification also relate to a contextual unit scoring device, a computing device, a computer-readable storage medium, and a computer program product, to address the technical deficiencies existing in the prior art.

[0006] According to a first aspect of the embodiments of this specification, a context unit scoring method is provided, comprising: Obtain the current context unit and multiple candidate scoring items constructed based on the medical dialogue flow, wherein the multiple candidate scoring items are determined based on the historical scoring results of the historical context unit; The target scoring item is determined from multiple candidate scoring items based on the similarity between the current context unit and multiple candidate scoring items. The context score of the current context unit is determined based on the similarity between the current context unit and the target score item. Based on contextual scoring and preset contextual scoring thresholds, the target scoring strategy is determined from the preset scoring strategies; Based on the target scoring strategy, the current context unit is scored to obtain the target score result of the current context unit.

[0007] According to a second aspect of the embodiments of this specification, a context unit scoring device is provided, comprising: The acquisition module is configured to acquire the current context unit and multiple candidate scoring items constructed based on the medical dialogue flow, wherein the multiple candidate scoring items are determined based on the historical scoring results of the historical context unit; The target scoring item determination module is configured to determine the target scoring item from multiple candidate scoring items based on the similarity between the current context unit and multiple candidate scoring items; The context scoring determination module is configured to determine the context score of the current context unit based on the similarity between the current context unit and the target scoring item. The target scoring strategy determination module is configured to determine the target scoring strategy from the preset scoring strategies based on context scoring and preset context scoring thresholds. The scoring module is configured to score the current context unit based on the target scoring strategy and obtain the target score result of the current context unit.

[0008] According to a third aspect of the embodiments of this specification, a computing device is provided, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer program / instructions, which, when executed by the processor, implement the steps of the above-described context unit scoring method.

[0009] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores a computer program / instructions that, when executed by a processor, implement the steps of the above-described context unit scoring method.

[0010] According to a fifth aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described context unit scoring method.

[0011] One embodiment of this specification implements a context unit scoring method, comprising: acquiring a current context unit constructed based on a medical dialogue flow and multiple candidate scoring items, wherein the multiple candidate scoring items are determined based on historical scoring results of historical context units; determining a target scoring item from the multiple candidate scoring items based on the similarity between the current context unit and the multiple candidate scoring items; determining a context score for the current context unit based on the similarity between the current context unit and the target scoring item; determining a target scoring strategy from preset scoring strategies based on the context score and a preset context scoring threshold; and scoring the current context unit based on the target scoring strategy to obtain the target score result for the current context unit.

[0012] By acquiring the current context unit constructed based on medical dialogue flow and a dynamically updated set of candidate scoring items based on historical scoring results, and determining the target scoring item from the candidate scoring items based on the similarity between the current context unit and multiple candidate scoring items, and then determining the context score based on the similarity between the current context unit and the target scoring item, the context score can accurately reflect the dialogue content of the current context unit and the potential scoring value and contribution of the candidate scoring items to be scored. By dynamically selecting the target scoring strategy from the preset scoring strategies based on the context score and preset context scoring threshold, accurate scoring triggering and targeted scoring strategies are achieved. Scoring based on the target scoring strategy and obtaining the target scoring result forms a closed-loop scoring logic from value assessment to strategy adaptation. While ensuring the accuracy of the scoring results, it avoids ineffective evaluation of low-value or repetitive dialogue content, improves the utilization efficiency of scoring resources, and enables the specific scoring strategy to dynamically adjust to the actual value of the dialogue content of the current context unit, thereby improving the accuracy and efficiency of the context unit scoring results. Attached Figure Description

[0013] Figure 1 This is a flowchart of a context unit scoring method provided in one embodiment of this specification; Figure 2 This is a schematic diagram of a method for determining the current context unit provided in one embodiment of this specification; Figure 3 This is a schematic diagram illustrating a method for comparing contextual scoring with a preset contextual scoring threshold, provided in one embodiment of this specification. Figure 4 This is a schematic diagram illustrating a method for determining a scoring request by calling a large language model, as provided in one embodiment of this specification. Figure 5 This is a schematic diagram of a traffic shaping method for a scoring request provided in one embodiment of this specification; Figure 6 This is a schematic diagram of a context unit scoring system provided in one embodiment of this specification; Figure 7 This is a timing diagram illustrating a context unit scoring method provided in one embodiment of this specification; Figure 8 This is a schematic diagram of the processing flow of a context unit scoring method provided in one embodiment of this specification; Figure 9 This is a schematic diagram of the structure of a context unit scoring device provided in one embodiment of this specification; Figure 10 This is a flowchart illustrating the processing procedure of a context unit scoring method provided in one embodiment of this specification. Detailed Implementation

[0014] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0015] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0016] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0017] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in one or more embodiments of this specification are obtained through open-source datasets or public datasets that comply with their license agreements, or are obtained with full authorization from the relevant parties. Moreover, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0018] In one or more embodiments of this specification, a large model refers to a deep learning model with a large number of model parameters, typically containing hundreds of millions, tens of billions, hundreds of billions, trillions, or even tens of trillions of model parameters. A large model can also be called a foundation model. It is pre-trained using large-scale unlabeled corpora to produce a pre-trained model with hundreds of millions of parameters. Such models can adapt to a wide range of downstream tasks and have good generalization ability. Examples include Large Language Models (LLMs) and multi-modal pre-training models.

[0019] In practical applications, large models only require a small number of samples to fine-tune the pre-trained model before they can be applied to different tasks. Large models can be widely used in fields such as Natural Language Processing (NLP) and Computer Vision. Specifically, they can be applied to computer vision tasks such as Visual Question Answering (VQA), Image Captioning (IC), and Image Generation, as well as natural language processing tasks such as text-based sentiment classification, text summarization, and machine translation. The main application scenarios of large models include digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.

[0020] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0021] Automatic Speech Recognition (ASR) is a technology that converts human speech signals into corresponding text data. It can be used to transform real-time speech communication between doctors and patients into a text stream that can be analyzed and processed by computers, providing basic input for subsequent semantic understanding and scoring.

[0022] The Large Language Model (LLM) is a deep neural network model trained on a large amount of text data. It has powerful natural language understanding and generation capabilities and can be used for complex reasoning tasks such as semantic understanding of medical dialogue content, rating item coverage judgment, and evidence extraction. It is the core model for scoring in this solution.

[0023] Top-K aggregation strategy is a method that selects the top K elements from multiple candidate items for comprehensive processing. It can be used to aggregate and calculate the multiple similarity scores between the current context unit and these target scoring items when there are multiple target scoring items. For example, the final context score can be obtained by weighted summation or averaging.

[0024] The BERT encoder (Bidirectional Encoder Representations from TransformersEncoder) is a pre-trained language model based on the Transformer architecture. It can be used to convert input text into vector representations containing deep semantic information. By capturing bidirectional contextual information, it generates feature vectors that accurately reflect the semantics of the text for subsequent similarity calculations.

[0025] Term Frequency-Inverse Document Frequency (TF-IDF) is a commonly used weighting technique in information retrieval and text mining. It can be used to evaluate the importance of a word to a document by representing the text as a vector composed of word weights, providing a numerical feature representation for subsequent keyword matching and similarity calculation.

[0026] The Sentence Transformer is a semantic encoding model based on a pre-trained language model. It can be used to directly encode variable-length text sentences into dense vectors of fixed dimensions, so that sentences with similar meanings have similar positions in the vector space. It is a basic tool for achieving efficient vector similarity calculation.

[0027] Cosine similarity is a method that measures the similarity between two vectors by measuring the cosine of the angle between them in a high-dimensional space. It can be used to calculate the semantic closeness between context feature vectors and rating item feature vectors. Its value ranges from -1 to 1, and the closer the value is to 1, the more similar the two texts are semantically.

[0028] Euclidean distance is a metric that measures the straight-line distance between two points in a multidimensional space. It can be used to calculate the distance between two feature vectors in the semantic space. The smaller the distance, the smaller the difference between the two vectors, meaning that the corresponding text semantics are closer. It is one of the commonly used alternatives for vector similarity calculation.

[0029] With the rapid development of medical informatics and artificial intelligence technologies, real-time analysis and scoring of medical dialogues based on automatic speech recognition (ASR) and natural language processing (NLP) are being increasingly widely used in clinical quality control, diagnosis and treatment assistance, and other scenarios.

[0030] Currently, scoring medical conversations mainly relies on fixed triggering mechanisms to invoke the scoring model. Some methods use fixed time periods to periodically send accumulated conversation text to the scoring model for scoring; others, in order to improve the real-time performance of the scoring, trigger the model to score immediately every time the speech recognition output is updated.

[0031] However, the above-mentioned technical solutions have certain technical problems in practical applications. When facing scenarios such as streaming medical dialogues, the scoring method based on a fixed time period is prone to producing semantically incomplete fragments due to mechanically segmenting the dialogue content according to time points, resulting in a decrease in scoring accuracy. On the other hand, triggering the model to score immediately every time the speech recognition output is updated lacks the ability to reuse historical scoring results, which can lead to repeated scoring of the same or similar content, resulting in a waste of computing resources and costs.

[0032] In view of this, this specification provides a method for scoring contextual units, and also relates to a contextual unit scoring device, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0033] See Figure 1 , Figure 1 A flowchart of a context unit scoring method according to an embodiment of this specification is shown, which specifically includes the following steps.

[0034] Step 102: Obtain the current context unit and multiple candidate scoring items constructed based on the medical dialogue flow, wherein the multiple candidate scoring items are determined based on the historical scoring results of the historical context unit.

[0035] The contextual unit scoring method described in one or more embodiments of this specification can be applied to computer systems and / or applications that require content analysis and scoring of real-time data streams. Specifically, this contextual unit scoring method can be applied to medical information systems, such as clinical quality control systems running on hospital servers or cloud platforms, to monitor, analyze, and score real-time dialogues between doctors and patients, thereby assisting in treatment decisions, improving the quality of medical documents, or conducting medical quality control. Simultaneously, this contextual unit scoring method can also be applied to online consultation platforms to conduct real-time assessments of the standardization and completeness of doctor-patient communication. This contextual unit scoring method can be deployed on computing devices with data processing capabilities, such as servers, personal computers, mobile devices, or embedded devices, and implemented through corresponding software programs.

[0036] Medical dialogue streams are data streams of continuous voice or text conversations between doctors and patients in medical settings. They can be processed in real-time using technologies such as Automatic Speech Recognition (ASR) to convert speech to text, or the corresponding text information can be directly collected to provide raw input for subsequent semantic analysis and scoring. Specifically, each data unit in a medical dialogue stream typically includes a speaker identifier (e.g., doctor or patient), text content, and corresponding timestamp information.

[0037] A contextual unit is a dialogue segment with independent and complete semantics, formed by integrating and aggregating multiple text fragments in a medical dialogue stream. Contextual units can serve as the basic unit for subsequent scoring processing, ensuring that the scoring model receives logically complete contextual information. For example, a contextual unit can be a complete question-and-answer pair, where the doctor asks a question and the patient gives an answer, such as: "Doctor: What's bothering you lately? Patient: I've had a headache for three days. Doctor: Any other symptoms? Patient: My throat is also a bit sore."; or it can be a topical segment revolving around a specific condition (such as present medical history or past medical history), which may include multiple rounds of dialogue. Specifically, the construction of contextual units can be based on preset integration rules (such as punctuation, speaker switching, topic changes, etc.) to integrate fragmented recognition results into sentences, and then, based on aggregation conditions, aggregate the sentences into meaningful dialogue units.

[0038] The current context unit is the most recently constructed and ready-to-be-scored unit in the ongoing medical dialogue flow, prepared for scoring at the current moment. The current context unit is the immediate input to the scoring process, representing the latest progress of the dialogue. Specifically, it can include a complete question-and-answer pair just completed by the doctor and patient in the current dialogue, or a complete paragraph of a topic discussed by the doctor and patient in the current dialogue.

[0039] Historical context units are those context units that have been processed and scored by the system prior to the current context unit. Historical context units can be used to generate historical scoring results to dynamically update candidate scoring items, and can also provide a reference for scoring the current context unit. Specifically, historical context units can include question-and-answer pairs completed in previous dialogues, topic segments already discussed in previous dialogues, etc. For example, if the system has previously scored a dialogue about "chief complaint symptoms" (i.e., a historical context unit) and determined that the dialogue has covered the scoring item "completeness of symptom description," then the information covered by this "completeness of symptom description" item can be determined from the historical scoring results of this historical context unit, thus removing "completeness of symptom description" from the candidate scoring items. The historical scoring results of historical context units can be continuously accumulated, forming an evolving state of scoring knowledge, providing a reference for judging the value of the current context unit, thereby avoiding repeated evaluation of already scored (i.e., covered) scoring items and achieving efficient utilization of scoring resources.

[0040] Multiple candidate scoring items are a set of scoring items that are dynamically determined based on historical scoring results obtained from scoring historical context units and still need to be evaluated. These items can be used to calculate similarity with the current context unit, determining which candidate scoring items are related to the current context unit, thus deciding whether to trigger scoring and what scoring strategy to adopt. For example, a complete scoring item set in a medical dialogue scoring system may include ten items such as "Description of Present Illness," "Inquiry into Past Medical History," and "Confirmation of Allergy History." During the continuous scoring of context units, based on historical scoring results of historical context units, it can be confirmed that "Description of Present Illness" and "Inquiry into Past Medical History" have been covered, meaning they have already been questioned and scored. The remaining eight items, such as "Confirmation of Allergy History" and "Inquiry into Family History," belong to multiple candidate scoring items. Multiple candidate scoring items are a key focus in subsequent scoring processes.

[0041] In practical applications, the current context unit can be obtained by processing the medical dialogue stream.

[0042] Specifically, medical dialogue streams can be continuously received from the speech recognition module or text acquisition module, which may include multiple dialogue text fragments. By integrating multiple text dialogue fragments, relatively complete dialogue sentences in terms of grammar and semantics can be obtained. Subsequently, these dialogue sentences can be further aggregated to obtain a contextual unit output with complete semantics, i.e., the current contextual unit.

[0043] Optionally, the integration of text fragments can be based on preset boundary conditions or semantic information of the text fragments. Boundary conditions may include detecting strong punctuation (such as periods or question marks) or pauses exceeding a predetermined duration threshold; semantic information may include speaker switching, semantic topic switching, etc.

[0044] Optionally, the aggregation of dialogue statements can be performed when the dialogue statements meet preset aggregation conditions, which may include the number of characters or dialogue rounds reaching a predetermined threshold, or the detection of natural boundaries of topics, etc.

[0045] Furthermore, the current context unit can be obtained by directly reading pre-stored context units from storage systems such as local storage and cloud storage; or by receiving dynamically input or network-transmitted context units through external device interfaces or network ports.

[0046] Multiple candidate scoring items can be obtained directly from the storage system.

[0047] Specifically, the storage system can store multiple candidate scoring items determined based on historical scoring results of historical context units, and dynamically update them based on the scoring process. After scoring a historical context unit in each round, it can determine which scoring items have been covered based on the corresponding historical scoring results, and record and store them. During the scoring process of the current context unit, queries can be performed in the storage system, and covered scoring items can be removed from a predefined set of complete scoring items, leaving the remaining scoring items as multiple candidate scoring items.

[0048] Optionally, the determination of candidate scoring items can employ a sliding window mechanism to maintain recent scoring results. For example, the scoring results of the 10 most recent historical context units can be maintained, and the set of candidate scoring items can be dynamically updated based on set difference operations. Alternatively, a more complex determination mechanism can be used, such as combining the priority weights or historical scoring frequencies of each scoring item to sort or filter candidate scoring items, to ensure that the set of candidate scoring items accurately reflects the scoring dimensions that need to be evaluated in the current dialogue.

[0049] In this step, by acquiring the current context unit constructed based on the medical dialogue flow and the candidate scoring item set dynamically updated based on historical scoring results, the semantic integrity of the scoring input is ensured, avoiding misjudgments and instability caused by incomplete sentences. At the same time, by dynamically updating the candidate scoring item set through historical scoring results, the repeated evaluation of already covered scoring items is avoided, improving the utilization efficiency of scoring resources. This provides accurate input for subsequent context scoring and scoring strategy selection, enabling accurate initiation and efficient execution of the scoring process, thereby improving the accuracy and efficiency of context unit scoring results.

[0050] Step 104: Based on the similarity between the current context unit and multiple candidate scoring items, determine the target scoring item from the multiple candidate scoring items.

[0051] Similarity is the semantic relevance between a current contextual unit and a candidate scoring item. It can be used to assess the content proximity between a contextual unit and a scoring item, thereby determining whether the contextual unit is likely to contain information related to the scoring item. For example, a current contextual unit describing a patient's "headache for the past three days" may have a high similarity to a candidate scoring item whose content is "inquire about present medical history (symptom description)"; however, the similarity between this contextual unit and a candidate scoring item whose content is "inquire about family history of genetic diseases" would be low. Specifically, similarity can include keyword matching similarity, vector similarity, or a mixture of both. Specifically, similarity can be calculated in various ways, including keyword similarity, vector similarity, or hybrid similarity. Keyword similarity can be calculated based on the overlap between the keywords of the candidate scoring item and the keywords of the current context unit, such as calculating the ratio of the number of keyword hits to the total number of keywords. Vector similarity can be calculated based on the cosine similarity between the context vector and the scoring item vector. Hybrid similarity can combine multiple different similarity calculation methods, such as weighted fusion.

[0052] Target scoring items are one or more scoring items selected from multiple candidate scoring items that are most semantically relevant to the current context unit, and can be used for subsequent context scoring calculations. Specifically, target scoring items may include scoring items with the highest similarity to the current context unit, or scoring items with a similarity exceeding a preset threshold. In essence, target scoring items serve as a reference for determining context scoring and can accurately reflect the potential value of the current context unit to each candidate scoring item.

[0053] In practical applications, the similarity between the current contextual unit and multiple candidate scoring items can be determined first. This can be achieved through various methods.

[0054] One alternative approach is to use keyword matching to calculate similarity. Specifically, this can be achieved by constructing a keyword set for each candidate scoring item and segmenting the dialogue text included in the current context unit. Furthermore, similarity can be obtained by calculating the overlap between the two keyword sets, such as the ratio of the number of overlapping keywords to the total number of keywords.

[0055] Another alternative approach is to use vector similarity calculation. Specifically, a pre-trained semantic encoding model can be used to encode the dialogue text of the current context unit and the description text of each candidate rating item to obtain the corresponding feature representations. Furthermore, the cosine similarity between the feature representation of the current context unit and the feature representation of each candidate rating item can be calculated, and this cosine value can be used as the similarity between the two.

[0056] Another alternative approach is to use a hybrid similarity calculation method. Specifically, keyword matching and vector similarity calculation methods can be combined. For example, keyword matching similarity and vector similarity can be calculated separately, and then a weighted sum can be used to obtain the final similarity score. The weights can be flexibly adjusted according to the actual scoring scenario.

[0057] Once the similarity between the current context unit and multiple candidate scoring items is determined, the target scoring item can be determined from the multiple candidate scoring items based on the similarity.

[0058] One alternative approach is to employ a maximum value strategy, which involves determining the maximum value among all similarity scores and identifying the candidate scoring item corresponding to that maximum value as the target scoring item.

[0059] Another alternative approach is to employ a strategy combining threshold filtering and ranking. A similarity threshold can be set, and all candidate scoring items with similarities higher than this threshold can be used as target scoring items. Furthermore, if no candidate scoring items with similarities higher than the threshold are found, it indicates that the text scoring value of the current context unit is low, and scoring can be delayed or skipped.

[0060] Another alternative approach is to combine the attribute information of each candidate scoring item to determine the target scoring item. For example, a priority weight can be preset for each candidate scoring item. In the process of determining the target scoring item, not only should similarity be considered, but the similarity can also be multiplied by the priority weight to obtain a comprehensive score. Then, the items can be sorted and selected based on the comprehensive score, thereby ensuring that candidate scoring items that are more critical in clinical quality control can be given priority even if their similarity to the current context unit is slightly lower.

[0061] In this step, the target scoring item is determined from multiple candidate scoring items based on the similarity between the current context unit and multiple candidate scoring items. This ensures that the context scoring accurately reflects the dialogue content of the current context unit and the potential scoring value and contribution of the candidate scoring items to be scored. By determining the target scoring item based on similarity, the scoring focus is accurately located on one or more of the most relevant scoring dimensions, providing a reference for subsequent calculation of context scores and selection of appropriate scoring strategies, thereby improving the accuracy and efficiency of the context unit scoring results.

[0062] Step 106: Determine the context score of the current context unit based on the similarity between the current context unit and the target scoring item.

[0063] Contextual scoring is a quantitative metric used to measure the value of new information or the contribution of a current contextual unit to an uncovered scoring item. It can also be called information gain (IG). It determines whether a current contextual unit includes new information related to an uncovered scoring item, thus deciding whether to trigger a score for that unit. In other words, it provides a quantitative basis for the system to determine whether a current contextual unit is "worth" triggering subsequent large-scale model scoring. Specifically, contextual scoring can be approximated by calculating the similarity between the current contextual unit and the target scoring item. The higher the similarity, the more new information the contextual unit includes related to the item to be scored, and the higher its contextual score. For example, if there is only one target scoring item, the similarity corresponding to that item can be directly used as the contextual score; if there are multiple target scoring items, the similarities corresponding to multiple target scoring items can be aggregated using Top-K or other aggregation algorithms to obtain the contextual score.

[0064] Specifically, in information theory, contextual scoring can be defined as IG(D_new) = H(Score) - H(Score |D_new). A practical approximation is IG(D_new) ≈ max_{i ∈ Uncovered} sim(D_new, r_i), which calculates the similarity between each uncovered scoring item and the current context, taking the maximum value. Higher similarity indicates that the contextual unit contains more new information related to the unscored item, making it more "worthy" of triggering LLM scoring.

[0065] In practical applications, the context score of the current context unit can be determined based on the similarity between the current context unit and the target scoring item.

[0066] Specifically, there can be one or more target scoring items, and different aggregation strategies can be adopted accordingly to determine the contextual scoring.

[0067] One possible approach is to employ a maximum value strategy, which involves calculating the similarity between the current contextual unit and each target rating item to obtain a set of similarity scores, and then taking the maximum value in this set as the contextual score of the current contextual unit. In other words, the value of the current contextual unit is determined by the target rating item that best matches it; if the current contextual unit is highly correlated with any target rating item, then the current contextual unit can be considered to have a high rating value.

[0068] Another alternative approach is to employ a Top-K aggregation strategy. This involves first selecting the K target rating items (e.g., K=3) that have the highest similarity to the current context unit, and then weighting and summing or averaging these similarity scores to obtain the context score for the current context unit. This aggregation strategy can more comprehensively reflect the combined contribution of the current context unit to multiple rating dimensions.

[0069] In this step, the context score of the current context unit is determined based on the similarity between the current context unit and the target scoring item. This ensures that the context score accurately reflects the dialogue content of the current context unit and its potential scoring value and contribution to the candidate scoring items to be scored. By using the context score and a preset context score threshold, scoring strategies can be dynamically selected to achieve accurate scoring triggering and avoid ineffective evaluation of low-value or repetitive dialogue content. At the same time, through the calculation of the context score, the scoring focus is accurately located on one or more of the most relevant scoring dimensions, providing a quantitative basis for subsequent scoring strategy selection and improving the accuracy and efficiency of the context unit scoring results.

[0070] Step 108: Based on contextual scoring and preset contextual scoring thresholds, determine the target scoring strategy from the preset scoring strategies.

[0071] Preset context scoring thresholds are one or more pre-defined numerical boundaries used for comparison with context scores. They can be used to determine the value of the current context unit and serve as a basis for selecting different scoring strategies. Specifically, by comparing the context score of the current context unit with the preset context scoring thresholds, the scoring value of the context unit can be automatically classified, thereby triggering differentiated processing procedures.

[0072] Optionally, the preset context scoring threshold may include a first scoring threshold and a second scoring threshold. The first scoring threshold can be higher to identify context units with high scoring value, while the second scoring threshold can be lower to identify context units with lower value. When the current context unit's context score reaches the first scoring threshold, it indicates that the current context unit contains new information highly relevant to the uncovered scoring items, and scoring for the current context unit can be directly triggered. When the current context unit's context score is lower than the second scoring threshold, it indicates that its value is low, possibly containing meaningless content such as greetings or confirmations, and scoring for the current context unit can be skipped. When the current context unit's context score is between the two thresholds, it indicates that its scoring value is in a fuzzy range, and further judgment can be made based on other conditions. The setting of the preset context scoring threshold can be flexibly adjusted according to the actual scoring application scenario to meet the requirements of accuracy, cost, and response speed.

[0073] Preset scoring strategies are a set of different processing methods predefined by the system for scoring large models of contextual units. They can be used to select the most appropriate scoring method under different system states or contextual values ​​to balance scoring accuracy, response time, and computational cost. Preset scoring strategies can specifically include full scoring strategies, lightweight scoring strategies, queuing strategies, skipping scoring strategies, and circuit breaker scoring strategies. Specifically, preset scoring strategies are scoring execution schemes pre-set by the system based on historical experience and engineering practices. Each scoring strategy can correspond to different scoring depths, computational resource consumption, and response times.

[0074] Among them, the full scoring strategy can perform a complete evaluation of all uncovered scoring items; the lightweight scoring strategy can perform a simplified evaluation only for target scoring items corresponding to high information gain; the queuing strategy can put scoring tasks into a queue to wait for resources; the skip scoring strategy can abandon the scoring of the current context unit; and the circuit breaker scoring strategy can suspend scoring when the system is abnormal. The specific content of the preset scoring strategies can also be defined and expanded according to actual scoring needs and system architecture, such as including exploratory scoring after degradation.

[0075] The target scoring strategy is the most suitable scoring execution method for the current context unit, determined from preset scoring strategies. It guides subsequent scoring operations, transforming abstract decision logic into concrete execution instructions. This guides the system to complete the scoring of the current context unit in the most appropriate way, thereby achieving optimal resource allocation and accurate and efficient scoring. Specifically, the target scoring strategy determines the scoring depth, resource consumption, and response time based on a comparison between the current context unit's context score and a preset context scoring threshold.

[0076] Optionally, the target scoring strategy can be dynamically determined by combining the system state information of the context unit scoring system. By determining the target scoring strategy, it can be ensured that the scoring operation matches the current system state and contextual value, thereby improving scoring efficiency and system stability.

[0077] In practical applications, the target scoring strategy can be determined based on contextual scoring and preset contextual scoring thresholds by using a combination of multi-level threshold determination and auxiliary conditions.

[0078] Specifically, based on the context score and a preset context score threshold, it can be determined whether to trigger a score for the current context unit. If the context score of the current context unit reaches the preset context score threshold, the target score strategy can be determined from the preset score strategies by further combining the system status information of the context unit score system.

[0079] Optionally, the system status information may include dimensions such as system latency information, system failure information, and system load information, and may also include other status information, such as the last scoring interval and the number of currently uncovered scoring items.

[0080] Furthermore, the preset context scoring threshold can be divided into multiple numerical intervals, each directly mapping to a preset scoring strategy. For example, the context score can be divided into different intervals such as [0, 0.2], (0.2, 0.4], (0.4, 0.6], (0.6, 0.8], and (0.8, 1.0], corresponding to different scoring strategies such as skip, queue, lightweight, and complete, respectively. This allows the system to automatically select the corresponding target scoring strategy based on the context score, achieving efficient scoring scheduling.

[0081] In this step, the target scoring strategy is determined from the preset scoring strategies based on contextual scores and preset contextual scoring thresholds. This enables the system to automatically determine the target scoring strategy based on the contextual score of the current contextual unit. By comparing the contextual score with the preset contextual scoring thresholds, the value of the contextual unit is quantitatively evaluated, providing an objective basis for the dynamic selection of scoring strategies. This allows the scoring operation to adapt to the actual value of the dialogue content of the current contextual unit, improving the accuracy and efficiency of the scoring results.

[0082] Step 110: Based on the target scoring strategy, score the current context unit to obtain the target score result of the current context unit.

[0083] Scoring is the process of using a defined target scoring strategy, invoking scoring models such as Large Language Models (LLMs), to reason about the current contextual units and generate corresponding scoring results. It can be used for structured scoring of doctor-patient dialogue content, providing information such as the coverage status of scoring items, specific evidence, and scores. Specifically, scoring can be performed by invoking a pre-defined scoring model. Its execution depth and output format depend on the target scoring strategy. For example, a full scoring strategy can evaluate all uncovered scoring items, outputting complete scoring evidence and scores; a lightweight scoring strategy can simplify the evaluation only for target scoring items with high information gain and can limit the output length and fields to reduce latency and computational costs.

[0084] The target scoring result is the final score output obtained after scoring the current context unit based on the target scoring strategy. It can be used to update the coverage status of scoring items, record scoring evidence, and analyze scoring results. Specifically, it can include the set of scoring covered items, evidence fragments, scoring scores, and execution time. In particular, the target scoring result is the output data of the scoring, which can characterize the scoring quality of the current context unit and is used to measure the effectiveness and comprehensiveness of the medical dialogue. Simultaneously, the target scoring result can serve as a reference for updating candidate scoring items in subsequent scoring processes, achieving a closed loop in the scoring process.

[0085] In practical applications, once the target scoring strategy is determined, the current statement unit can be scored according to the target scoring strategy.

[0086] Specifically, scoring can be performed by calling a pre-defined scoring model, and input data for the scoring model can be constructed based on the target scoring strategy. This data may include scoring item identifiers, dialogue text of the current context unit, specific scoring prompts to guide the model in making targeted scores, and information such as limiting the complexity of the output format.

[0087] Once the input data has been constructed, it can be sent to a pre-defined scoring model (such as a large language model) for inference. The scoring model can analyze the input data based on its internal algorithms and the knowledge learned during the pre-training process, and return the original response data, which may include information such as the identifier of the scoring item covered by the current scoring, the score, the text fragment on which the scoring is based, the execution status, and the execution time.

[0088] Upon receiving response data, it can be parsed, converted into a structured target score result, and then output.

[0089] Optionally, scoring the current sentence unit can also employ a token bucket to control the processing flow of scoring requests. Specifically, a token bucket can be maintained, replenishing scoring tokens at a preset rate. Each scoring request can be configured to consume one or multiple scoring tokens determined by weight. During the actual scoring process, scoring token information can be obtained, and if sufficient tokens are available, the current context unit can be scored based on the target scoring strategy. If tokens are insufficient, the target scoring strategy can be adjusted, downgrading the strategy or skipping scoring, and the scoring request can be executed again once sufficient tokens are available again.

[0090] The token bucket's token replenishment rate can be flexibly adjusted based on the actual scoring request execution latency or queue length, thereby achieving a balance between the scoring request execution throughput and system load.

[0091] In addition, a concurrency threshold for executing rating requests can be set. If the concurrency of currently executing rating requests in parallel reaches this threshold, subsequent rating requests will need to wait or skip the rating process.

[0092] In this step, the current context unit is scored based on the target scoring strategy to obtain the target score result of the current context unit. This allows the scoring operation to adaptively select the appropriate execution method according to the scoring value of the context unit. The structured scoring result output provides an accurate data foundation for the subsequent dynamic updating of candidate scoring items and the implementation of closed-loop scoring logic, thereby improving the accuracy and efficiency of the context unit scoring results.

[0093] In this embodiment, by acquiring the current context unit constructed based on the medical dialogue flow and the set of candidate scoring items dynamically updated based on historical scoring results, a target scoring item is determined from the candidate scoring items based on the similarity between the current context unit and multiple candidate scoring items. Then, a context score is determined based on the similarity between the current context unit and the target scoring item. This ensures that the context score accurately reflects the dialogue content of the current context unit and its potential scoring value and contribution to the candidate scoring items to be scored. By dynamically selecting a target scoring strategy from preset scoring strategies based on the context score and a preset context scoring threshold, accurate scoring triggering and targeted scoring strategies are achieved. Scoring based on the target scoring strategy and obtaining the target scoring result forms a closed-loop scoring logic from value assessment to strategy adaptation. While ensuring the accuracy of the scoring results, it avoids ineffective evaluation of low-value or repetitive dialogue content, improves the utilization efficiency of scoring resources, and allows the specific scoring strategy to dynamically adjust to the actual value of the dialogue content of the current context unit, thereby improving the accuracy and efficiency of the context unit scoring results.

[0094] In one optional embodiment of this specification, obtaining the current context unit constructed based on the medical dialogue flow includes: Obtain the medical conversation stream; Speech recognition is performed on the medical dialogue stream to obtain multiple dialogue text fragments; Based on the semantic information of dialogue text fragments, multiple text fragments of the dialogue are integrated to obtain at least one dialogue statement; If at least one dialogue statement satisfies a preset aggregation condition, aggregate at least one dialogue statement to obtain the current context unit.

[0095] Speech recognition is the process of converting speech signals from medical conversations into text data. It can be used to convert the voice communication between doctors and patients into processable text input in real time, providing basic data for subsequent semantic analysis and scoring. Specifically, speech recognition is usually implemented using automatic speech recognition models that can output partial recognition results in a streaming manner, such as generating a text segment per second.

[0096] Dialogue text fragments are discretized, unintegrated text data units output by speech recognition. They can be used to construct complete dialogue sentences, specifically including single phrases, sentences, or short dialogues spoken by a doctor or patient. Specifically, dialogue text fragments typically include speaker identification, text content, and timestamps. These fragments serve as input to the semantic buffer module, allowing for further integration to obtain at least one dialogue sentence.

[0097] Semantic information refers to the semantic content and contextual relationships included in a dialogue text fragment. It can be used to determine the logical connections and semantic coherence between text fragments. Specifically, semantic information can include speaker switching, topic changes, semantic theme shifts, strong punctuation, and pauses exceeding a predetermined threshold. It can provide a basis for decision-making in the subsequent integration process, ensuring that the merged dialogue statements are grammatically and semantically fluent and complete. Semantic information can be used to determine whether text fragments can be merged into complete dialogue statements or contextual units.

[0098] Integration is the process of piecing together and combining multiple fragmented text segments into a grammatically and semantically complete sentence based on the semantic information of dialogue text fragments. It can be used to eliminate fragmentation in speech recognition output, providing high-quality sentence-level input for subsequent construction of contextual units. Specifically, the integration process can be based on various boundary conditions, such as the detection of strong punctuation, pause times exceeding a threshold, and speaker switching. When any of these conditions are met, the dialogue text fragments in the current buffer can be concatenated and output as a single dialogue sentence, ensuring the integrity of the input data for subsequent aggregation operations and avoiding scoring errors caused by semantic incompleteness.

[0099] Dialogue statements are sentences with relatively complete grammar and semantics, formed by integrating multiple dialogue text fragments, and can be used for context aggregation processing. Specifically, each dialogue statement typically corresponds to a complete expression, which can include consecutive statements from one or more speakers. For example, it could be a complete question-and-answer pair, such as "Doctor: What's bothering you lately? Patient: I've had a headache for three days." It could also be a coherent dialogue around a specific topic. Dialogue statements are the basic input for context aggregation, and their completeness and coherence determine the quality of the context unit.

[0100] Predefined aggregation conditions are a set of predefined rules or thresholds used to determine when multiple consecutive dialogue statements can be merged into a contextual unit with independent and complete semantics. These conditions control the timing and granularity of contextual unit output and can include parameters such as character count, dialogue turn number, and topic boundary. Specifically, different preset aggregation conditions can be judged individually or in combination to form a hierarchical output strategy. For example, preset aggregation conditions may include: outputting when the maximum character count or maximum turn number is reached; outputting when the target character count is reached and question-and-answer completion or topic boundary is detected; and outputting when the minimum character count is reached and the pause duration is satisfied. By reasonably setting preset aggregation conditions, the semantic integrity and scoring applicability of the current contextual unit obtained through aggregation can be ensured.

[0101] Optionally, the completion of the question-and-answer session is determined by preset rules. For example, if the patient has just finished answering and the number of dialogue rounds is greater than or equal to two, the question-and-answer session can be considered complete. Topic boundaries can be determined by keyword matching. Signal words for topic switching can be preset in the system, such as "that / then / next / also / there is / right," as well as keywords for the consultation category, such as: present medical history "this time / recently," past medical history "before / before," family history "family / parents," and personal history "smoking / drinking / allergies." When the signal words are detected or the consultation category is switched, it can be determined that the topic boundary has been reached.

[0102] Aggregation is the process of combining multiple consecutive, content-related dialogue statements into a single contextual unit with independent and complete semantics, according to preset aggregation conditions. It can be used to generate logically complete dialogue segments suitable for large language model scoring. Specifically, aggregation can be triggered by preset aggregation conditions such as dialogue turn, character count, or topic boundaries, combining consecutive dialogue statements into a semantically complete contextual unit.

[0103] In practical applications, speech recognition of medical dialogue streams can be performed based on Automatic Speech Recognition (ASR) technology. This involves using a pre-trained speech recognition model that can convert speech signals into text.

[0104] Specifically, speech recognition can employ deep learning-based acoustic and language models to segment continuous speech signals into speech frames, extract features, convert them into phoneme sequences using an acoustic model, and then predict the corresponding word sequences using a language model.

[0105] Optionally, speech recognition can be implemented using streaming processing, outputting the recognition results corresponding to the current medical dialogue in real time, avoiding waiting for complete speech input and ensuring the real-time nature of the dialogue. Specific speech recognition methods can be adapted to the medical dialogue scenario; for example, model parameters can be adjusted to accommodate the pronunciation of medical terminology, or speaker recognition technology can be used to distinguish between the doctor's and patient's speech.

[0106] When multiple dialogue text fragments are obtained, they can be integrated based on the semantic information of the dialogue text fragments to obtain at least one dialogue statement.

[0107] Specifically, a boundary detection-based buffer merging mechanism can be adopted. The system maintains a sentence buffer. Whenever a new dialogue text fragment is received, it can be appended to the sentence buffer while simultaneously performing semantic recognition to detect whether preset boundary conditions have occurred. These boundary conditions can include: detection of strong punctuation marks, detection of speaker switching, detection of continuous silence exceeding a pause threshold, etc. When any boundary condition is met, all dialogue text fragments in the current buffer can be concatenated to form a single dialogue statement, and the sentence buffer can be cleared.

[0108] Optionally, for continuous text without clear boundaries, a maximum waiting time can be set, and the current dialogue text fragments can be directly integrated when the maximum waiting time is reached, in order to prevent sentences from becoming too long.

[0109] Given at least one dialogue statement, the current statement unit can be obtained by determining whether the at least one dialogue statement meets the preset aggregation conditions, and if so, by aggregating the at least one dialogue statement.

[0110] Specifically, the system can maintain a context buffer, and when a new dialogue statement is generated, it can be appended to the context buffer and the current state information of the buffer can be updated, such as the total number of characters (or tokens) and the number of dialogue rounds.

[0111] Meanwhile, the system can continuously determine the current context buffer status information and preset aggregation conditions. The preset aggregation conditions can include multi-level threshold combinations, that is, they can include maximum aggregation conditions, ideal aggregation conditions, and minimum aggregation conditions. For example, the maximum aggregation conditions can include the maximum number of characters threshold and the maximum number of rounds threshold, the ideal aggregation conditions can include the ideal number of characters threshold and the ideal number of rounds threshold, and the minimum aggregation conditions can include the minimum number of characters threshold and the minimum number of rounds threshold, etc.

[0112] When the state information in the context buffer reaches the maximum output threshold, forced output is triggered, and the current buffer content is output as a context unit. When the ideal output threshold is reached, it can further detect whether the question and answer have been completed. When the question and answer are completed or the topic changes, the context unit is output. When the minimum condition is met, it can further detect whether a new sentence has arrived. If no new sentence arrives within a preset time, the context unit is output. If the minimum condition is not met, it continues to wait for subsequent dialogue sentences to arrive.

[0113] For example, see Figure 2 , Figure 2 This specification illustrates a schematic diagram of a current context unit determination method according to an embodiment of the present specification, as shown below. Figure 2 As shown.

[0114] The method for determining the current context unit can include two parts: integration of dialogue text fragments and aggregation of dialogue statements.

[0115] In the integration of dialogue text fragments: Receive dialogue text fragments and append them to the sentence buffer; Determine if strong punctuation marks are present or if a pause threshold is met. If so, integrate the dialogue segments; otherwise, return the received dialogue text segment. Determine if a speaker switch has occurred. If yes, integrate the dialogue segments; otherwise, return to receive the dialogue text segments. Perform dialogue fragment integration to obtain dialogue statements, and clear the sentence buffer.

[0116] In dialogue statement aggregation: Receive dialogue statements and append them to the context buffer; Statistics on statement length and dialogue rounds; Determine if the maximum output condition has been met; if so, output the current context unit. If not, determine whether the ideal output condition has been met; if so, output the current context unit. If not, determine whether the minimum output condition has been met; if so, output the current context unit. If not, wait to receive the dialogue statement.

[0117] In this embodiment, a medical dialogue stream is acquired, and speech recognition is performed on the medical dialogue stream to obtain multiple dialogue text fragments. Based on the semantic information of the dialogue text fragments, the multiple dialogue text fragments are integrated to obtain at least one dialogue statement. If at least one dialogue statement meets a preset aggregation condition, the at least one dialogue statement is aggregated to obtain the current context unit. This ensures the semantic integrity of the context unit and avoids inaccurate scoring due to incomplete sentences. By integrating based on semantic information and aggregating based on preset aggregation conditions, the context unit can accurately reflect the semantic coherence of the dialogue, providing high-quality input data for subsequent scoring. Through the hierarchical preset aggregation conditions, the semantic boundaries of different dialogue scenarios can be dynamically adapted, improving the flexibility and accuracy of context unit construction, thereby improving the accuracy and efficiency of context unit scoring.

[0118] In one optional embodiment of this specification, determining the target rating item from multiple candidate rating items based on the similarity between the current context unit and multiple candidate rating items includes: Feature extraction is performed on the current context unit and multiple candidate scoring items to obtain context feature representation and multiple scoring item feature representation; Based on contextual feature representation and multiple rating item feature representations, the similarity between the current contextual unit and each candidate rating item is determined; Based on multiple similarities, multiple candidate scoring items are ranked to obtain the ranking results of the scoring items; Based on the ranking results of the scoring items, the target scoring item is determined.

[0119] Feature extraction is the process of extracting numerical or structured information that represents the semantic content of raw text data, including current context units and multiple candidate scoring items, using specific algorithms or methods. It transforms unstructured natural language content into a feature form that computers can process, specifically including word segmentation, word embedding encoding, and key information extraction. Specifically, feature extraction can obtain word sequences by performing basic processing on the text content, such as word segmentation, stop word removal, and keyword extraction; it can also use pre-trained lightweight semantic coding models to generate high-dimensional feature vectors, such as BERT-based encoders or sentence-transformers encoding models, to perform deep encoding of the text. By performing feature extraction, the accuracy of subsequent semantic matching can be ensured, avoiding similarity calculation biases caused by insufficient feature representation.

[0120] Feature representation is a numerical or structured form obtained from text data through the feature extraction process, capable of representing the semantic content of the text. It can be used for text comparison, computation, and reasoning. Specifically, feature representation can include word segmentation sequences, word frequency encoding vectors, word embedding vectors, etc.

[0121] Contextual feature representation is a feature form that represents the overall semantic content of a current context unit after feature extraction. It can be used to compare with the feature representations of various candidate scoring items to determine similarity. Specifically, contextual feature representation can be a word segmentation sequence based on the current context unit text, or a high-dimensional vector generated by a pre-trained semantic encoding model.

[0122] The feature representation of a rating item refers to the feature form obtained after feature extraction of candidate rating items, which can characterize the semantic content of the descriptive text. It can be used to calculate the similarity with the feature representation of the current context unit. Specifically, the feature representation of a rating item can be a word segmentation sequence of the descriptive text of the rating item, or a high-dimensional vector generated by a pre-trained semantic coding model, and can have the same form as the context feature representation.

[0123] The sorting is based on the similarity calculation results between the current context unit and multiple candidate scoring items. The candidate scoring items are arranged from high to low according to the similarity value. The originally unordered set of candidate scoring items is ordered according to the degree of semantic relevance with the current context unit, which provides a clear ranking basis for subsequent target scoring item selection. It can be used to determine the target scoring item most relevant to the current context unit. Specifically, the sorting can be implemented using common algorithms such as quicksort and heapsort.

[0124] The ranking of scoring items is based on the similarity calculation between the current context unit and multiple candidate scoring items. The resulting sequence, obtained by ranking the candidate scoring items, can be used to determine the scoring item most relevant to the current context unit. Specifically, the ranking result can include not only the scoring item identifier but also metadata such as its similarity score and priority weight. Based on this ranking result, the system can employ various strategies to determine the target scoring item, such as directly selecting the top-ranked scoring item or selecting the top K scoring items.

[0125] In practical applications, feature extraction can be implemented in various ways depending on the different similarity calculation methods.

[0126] One possible approach is to employ word segmentation-based keyword extraction. Specifically, the text of the current context unit and the descriptive text of each candidate scoring item can be segmented into words, and after operations such as removing stop words, a word sequence is obtained. Furthermore, a term frequency-inverse document frequency (TF-IDF) feature can be constructed to represent the text as a vector composed of the weights of each word.

[0127] Another alternative approach is to employ vector encoding based on a pre-trained model. Specifically, semantic encoding models such as sentence-transformers can be used to input the text of the current context unit and the descriptive text of each candidate rating item into the model, obtaining the corresponding encoded vectors as feature representations.

[0128] Another option is to use hybrid feature extraction, which involves extracting keyword features and semantic vector features simultaneously and then concatenating or fusing them to form a more comprehensive feature representation.

[0129] Furthermore, the feature representations of candidate scoring items can be pre-calculated and stored when the system starts up to avoid redundant calculations; and the feature representations of current context units can be extracted in real time each time a score is needed.

[0130] Once the contextual feature representation and multiple rating item feature representations are obtained, the similarity between the current contextual unit and each candidate rating item can be calculated.

[0131] Specifically, the method for calculating similarity can depend on the specific form of the feature representation. If the feature representation is a word sequence, similarity can be determined by calculating the overlap of keywords using keyword matching methods. If the feature representation is a semantic encoding vector, measures such as cosine similarity or Euclidean distance can be used. If the feature representation includes multiple modalities, a hybrid similarity calculation can be used. For example, keyword matching similarity can be calculated first, and if the keyword matching similarity is greater than a preset threshold, the keyword similarity can be directly used as the final similarity. Otherwise, cosine similarity can be further calculated, and the two types of similarities obtained can be weighted and summed to obtain the final similarity. The specific weights can be adjusted according to the actual scenario.

[0132] Once the similarity between the current context unit and each candidate scoring item is obtained, all candidate scoring items can be sorted.

[0133] Specifically, the sorting process can employ any standard sorting algorithm, such as quicksort or mergesort, to arrange the similarity items in descending order, resulting in an ordered ranking of the scoring items. Based on this ranking, various strategies can be used to determine the target scoring item.

[0134] One alternative approach is to use a maximum value strategy, which involves directly identifying the candidate scoring item that ranks first (i.e. has the highest similarity) in the ranking results as the target scoring item.

[0135] Another alternative approach is to use a threshold truncation and Top-K strategy, which involves setting a similarity threshold and using all candidate rating items with similarity higher than the threshold as target rating items; or directly selecting the top K (e.g., the top 3) candidate rating items as target rating items.

[0136] Another option is to combine the attribute information of the scoring items themselves. For example, a priority weight can be preset for each scoring item. When sorting, the items are sorted first according to similarity, and then the scoring items with higher priority are selected when the similarity is close. Alternatively, the similarity and priority weight can be multiplied to obtain a comprehensive score, and then sorted according to the comprehensive score.

[0137] In the embodiments of this specification, by acquiring the current context unit and multiple candidate scoring items, feature extraction is performed to obtain the corresponding feature representations. Similarity is calculated based on the feature representations, the candidate scoring items are ranked, and the target scoring item is determined. This ensures that the target scoring item accurately reflects the semantic relevance between the current context unit and the candidate scoring items. By calculating and ranking based on feature representations, the determination of the target scoring item is ensured, taking into account both semantic matching and the priority of scoring items, thus improving the accuracy and rationality of target scoring item selection. Through reasonable feature extraction and ranking methods, an accurate assessment of the semantic relationship between the current context unit and the candidate scoring items is achieved, providing a reliable basis for subsequent context scoring and scoring strategy selection, thereby improving the accuracy and efficiency of context unit scoring results.

[0138] In one optional embodiment of this specification, determining the similarity between the current contextual unit and each candidate rating item based on contextual feature representation and multiple rating item feature representations includes: Based on contextual feature representation and multiple rating item feature representations, keyword matching is performed to determine the similarity between the current contextual unit and each candidate rating item; And / or, based on contextual feature representation and multiple rating item feature representations, perform vector similarity calculation to determine the similarity between the current contextual unit and each candidate rating item.

[0139] Keyword matching is a method for calculating semantic similarity based on textual overlap. It can be used to quickly assess the lexical-level association between the current context unit and candidate scoring items. Specifically, it can include calculating keyword overlap and the ratio of keyword hits to the total number of keywords. Specifically, keyword matching compares the keyword set of the context unit with the keyword set of the candidate scoring item, calculating the ratio of the number of overlapping keywords to the total number of keywords, thus obtaining a similarity score between 0 and 1. For example, for the candidate scoring item "confirm allergy history," its keyword set might include "allergy," "medication," and "penicillin." If the current context unit includes "I am allergic to penicillin," keyword matching can detect the overlap between "allergy" and "penicillin," thus obtaining a high keyword matching similarity.

[0140] Vector similarity calculation is a method that uses high-dimensional vectors based on contextual feature representations and rating item feature representations to determine the semantic relevance between the current contextual unit and candidate rating items by calculating the geometric similarity between the vectors. It can be used to measure the semantic closeness between the current contextual unit and candidate rating items, and can specifically include calculating cosine similarity, Euclidean distance, etc. Specifically, vector similarity calculation can be performed by encoding the text content of the current contextual unit and each candidate rating item into high-dimensional encoded vectors, and then calculating the cosine similarity between these encoded vectors to obtain a similarity score between 0 and 1. For example, for the candidate rating item "asking about current symptoms," its semantic vector is located in the "symptoms" region in space; if the current contextual unit is "I have a headache recently," although it does not contain the words "symptoms," its semantic vector can fall into a similar region, and thus a higher score can be obtained through vector similarity calculation (such as cosine similarity).

[0141] In practical applications, keyword matching based on contextual feature representation and rating item feature representation can be used to determine similarity, and keyword set comparison can be employed.

[0142] Specifically, a keyword set can be constructed for each candidate scoring item. This set can be generated based on the descriptive text of the scoring item through word segmentation, part-of-speech tagging, and medical domain dictionary matching. Simultaneously, the same word segmentation and keyword extraction processing is performed on the text of the current context unit to obtain its keyword set.

[0143] Furthermore, the degree of overlap between the two sets can be calculated, for example, by dividing the size of the intersection of the two sets by the size of the union; or by calculating the ratio of the number of keyword hits to the total number of keywords.

[0144] In addition, to improve the accuracy of keyword matching, different keywords can be assigned different weights. For example, highly related words such as "allergy" and "symptoms" can be given higher weights, so that the hits of these words contribute more to the similarity score.

[0145] In practical applications, vector similarity calculations can be performed based on contextual feature representations and rating item feature representations. A pre-trained semantic coding model can be used to generate vectors and calculate their spatial distance.

[0146] Specifically, the feature representations of the scoring items and the contextual features can be encoded vectors. The cosine similarity between the feature vector of the current contextual unit and the encoded vector of each candidate scoring item can be calculated. The closer the cosine value is to 1, the more consistent the directions of the two vectors in the semantic space, meaning the more similar the textual meanings. Alternatively, other metrics such as dot product or Euclidean distance can also be used.

[0147] In addition, to further improve accuracy, a coding model fine-tuned for the medical field can be adopted to make the vector space better reflect the semantic relationships between medical terminology and consultation logic.

[0148] In the embodiments of this specification, keyword matching and vector similarity calculation are performed based on contextual feature representation and multiple scoring item feature representations, enabling similarity calculation to balance speed and accuracy. Rapid keyword matching avoids complex calculations for high similarity cases, improving computational efficiency. Precise measurement through vector similarity calculation ensures accurate capture of deep semantic relationships. The combination of keyword matching and vector similarity calculation enables hybrid similarity calculation, improving its robustness and accuracy, providing a reliable basis for determining subsequent target scoring items, thereby enhancing the accuracy and efficiency of contextual unit scoring results.

[0149] In one optional embodiment of this specification, determining a target scoring strategy from preset scoring strategies based on contextual scoring and a preset contextual scoring threshold includes: When the context score reaches the preset context score threshold, the target score strategy is determined from the preset score strategies.

[0150] In practical applications, the target scoring strategy is determined based on contextual scoring and a preset contextual scoring threshold. Specifically, the preset contextual scoring threshold can be used as the scoring trigger condition, and when the contextual score reaches the preset contextual scoring threshold, scoring is triggered for the current contextual unit. Furthermore, the target scoring strategy is determined from the preset scoring strategies.

[0151] If the context score does not reach the preset context score threshold, the scoring of the current context unit can be skipped, and the current scoring process can be terminated directly. The skipped current context unit can be stored in the context unit buffer queue, and can be combined with the subsequent context units for re-scoring, or a unified re-scoring can be performed when the number of skipped context units in the buffer reaches a preset number.

[0152] In one alternative embodiment, a multi-level threshold can be used to determine whether to trigger a score for the current context unit.

[0153] Optionally, the preset context scoring threshold may include a first scoring threshold and a second scoring threshold.

[0154] The first scoring threshold is a relatively high numerical boundary among the preset context scoring thresholds. It can be used to identify contextual units that are highly relevant to the uncovered scoring items and have high scoring value. When the context score reaches or exceeds the first scoring threshold, it indicates that the current contextual unit contains new information that is highly relevant to the uncovered scoring items. In this case, scoring for the current contextual unit can be directly triggered, and a complete scoring strategy can usually be selected for comprehensive and thorough scoring.

[0155] The second scoring threshold is a lower numerical boundary among the preset context scoring thresholds. It can be used to identify context units with low value, which may include meaningless content such as greetings and confirmations. When the context score is lower than the second scoring threshold, it indicates that the current context unit contributes little to the uncovered scoring items, and its scoring can be skipped directly, thus avoiding waste of computational resources.

[0156] Secondary decision conditions are auxiliary conditions used to further determine whether to trigger scoring when the context score falls between the first and second scoring thresholds. These conditions can be used to handle context units with score values ​​in a fuzzy range. Specifically, secondary decision conditions may include auxiliary indicators such as the number of potential coverage items (i.e., the number of candidate scoring items with a similarity exceeding an intermediate threshold to the current context unit), the number of dialogue turns in the current context unit, and topic boundary detection results. When the context score is in a fuzzy range, the system can perform a secondary decision based on these secondary decision conditions. For example, if the current context score is between 0.2 and 0.6, it can further detect whether at least two candidate scoring items have a similarity exceeding 0.4, or whether the current context unit has reached four or more dialogue turns. If so, scoring can be triggered; otherwise, scoring is skipped.

[0157] In practical applications, the target scoring strategy can be determined from the preset scoring strategies based on contextual scoring and preset contextual scoring thresholds. A multi-level threshold determination method can be used.

[0158] Specifically, the preset context scoring thresholds include a first scoring threshold and a second scoring threshold, where the first scoring threshold is higher and the second scoring threshold is lower. After obtaining the context score of the current context unit, it can be compared with these two thresholds.

[0159] If the context score reaches the first scoring threshold, it indicates that the current context unit contains new information that is highly relevant to the uncovered scoring items and has high scoring value, thus triggering a score.

[0160] If the context score does not reach the second scoring threshold, it indicates that the current context unit has a low value and may contain meaningless content such as small talk or confirmation. In this case, the scoring can be skipped.

[0161] If the context score falls between the second and first scoring thresholds, it indicates that its score value is in a fuzzy range. In this case, a secondary judgment process can be initiated, combining additional auxiliary conditions for further evaluation. If the secondary judgment conditions are met, the current context unit can be considered to still have comprehensive scoring value, and scoring can be triggered; otherwise, scoring is skipped.

[0162] For example, see Figure 3 , Figure 3 This diagram illustrates a method for comparing contextual scores and preset contextual score thresholds according to an embodiment of this specification. Figure 3 As shown, the comparison between contextual scores and preset contextual score thresholds can specifically include: Receive the current context unit; Feature extraction is performed on the current context unit and multiple candidate scoring items to obtain context feature representation and multiple scoring item feature representation; Based on contextual feature representation and multiple scoring item feature representations, similarity is determined, and based on the similarity, the contextual score of the current contextual unit is determined. Determine whether the context score has reached the preset context score threshold. If so, determine the target score strategy from the preset score strategies. If not, discard or delay the score.

[0163] In the embodiments of this specification, by determining the target scoring strategy from preset scoring strategies based on contextual scoring and preset contextual scoring thresholds, the system can dynamically adjust the scoring strategy according to the value of contextual units, avoiding invalid scoring of low-value contextual units and omission of scoring of high-value contextual units. It achieves hierarchical judgment of the value of contextual units, enabling the system to trigger scoring for high-value contextual units and skip scoring for low-value contextual units. This improves the utilization efficiency of scoring resources, enhances the accuracy of scoring triggering, and allows the scoring system to more intelligently adapt to the actual value of different dialogue content, thereby improving the overall efficiency of the scoring system while ensuring the accuracy of scoring results.

[0164] In an optional embodiment of this specification, before determining the target scoring strategy from the preset scoring strategies when the context score reaches a preset context score threshold, the method further includes: Obtain the system status information of the context unit scoring system; When the context score reaches a preset context score threshold, a target scoring strategy is determined from the preset scoring strategies, including: When the context score reaches the preset context score threshold, the target score strategy is determined from the preset score strategies based on the system status information.

[0165] The context unit scoring system is an overall system architecture for executing the context unit scoring method provided in one or more embodiments of this specification. Specifically, it may include multiple functional units such as a data receiving module, a semantic buffer module, an information gain filtering module, an adaptive scheduling module, a traffic shaping module, a large model scoring execution module, and a state storage and write-back module. Specifically, the context unit scoring system can start from the input of a medical dialogue stream, construct context units through the semantic buffer, calculate and filter information gain, make adaptive scheduling decisions, control traffic shaping, execute large model scoring, and finally write back the state and output the results, forming a complete closed-loop processing flow. Specifically, the context unit scoring system can be deployed on one or more servers, implementing each functional unit in the form of software modules, and interacting with data through a network communication interface. The context unit scoring system is the technical carrier of the context unit scoring method, and its operational status information (such as latency, failure rate, load, etc.) can directly affect the selection and execution of the target scoring strategy.

[0166] System status information is a collection of quantitative indicators reflecting the current operational status of the context unit scoring system. It can be used to assess the system's health and load level, providing a basis for adaptive scheduling decisions. System status information includes dimensions such as system latency, system failure, and system load. Specifically, system latency information can include the average latency over a recent period; system failure information can include the recent scoring failure rate; and system load information can include the current queue length and concurrency usage. By determining the target scoring strategy based on system status information, the selection of the target scoring strategy can consider not only the context unit's own context scoring but also the system's current capacity and service quality, thereby achieving more comprehensive dynamic scheduling and ensuring scoring accuracy and efficiency under different system states.

[0167] Specifically, determining the target scoring strategy based on system state information can be achieved through various methods. For example, a threshold judgment method can be used, which compares the system state information with a preset threshold and selects the corresponding scoring strategy based on the comparison result. Alternatively, a weighted comprehensive method can be used, which assigns different weights to each dimension of the system state information, calculates a comprehensive score, and then selects a scoring strategy based on the comprehensive score. Another method can be used, which uses a set of preset decision rules based on system state information to sequentially judge and select the scoring strategy according to the rules.

[0168] In practical applications, system status information can include different dimensions such as system latency information, system failure information, and system load information. In addition, it can also include other information dimensions such as the last scoring interval and the number of currently uncovered items.

[0169] Specifically, system status information across different dimensions can be compared with corresponding thresholds for each dimension, and different target scoring strategies can be determined based on the thresholds reached. For example, if system latency information, system failure information, and system load information do not reach a preset latency threshold, or if the system load information does not reach a preset load threshold, it indicates that the system is currently in a very good operating state, and the complete scoring strategy can be determined as the target scoring strategy, meaning that scoring can be performed on all candidate scoring items for the current context unit. If system failure information reaches a preset failure threshold, it indicates that the current scoring service is malfunctioning and cannot perform any normal scoring operations, and the circuit breaker scoring strategy can be determined as the target scoring strategy, meaning that the scoring service for the current context unit is stopped.

[0170] For example, see Figure 4 , Figure 4 This specification illustrates a schematic diagram of a method for determining a scoring request by invoking a large language model, as provided in one embodiment. Figure 4 As shown, the method for determining the execution of the scoring request by calling the large language model can specifically include: Obtain the system status information of the context unit scoring system; The system status information is evaluated; Determine whether the large language model can be directly invoked to execute the scoring request; If so, invoke the large language model to execute the scoring request; If not, downgrade the current scoring request and re-acquire the system status information of the context unit scoring system.

[0171] In the embodiments of this specification, when the context score reaches a preset context score threshold, a target scoring strategy is determined from preset scoring strategies based on system status information. This allows the scoring operation to be dynamically adjusted according to the current health and load of the system. This avoids performing high-cost full scoring when the system is overloaded or unstable, and also avoids performing inefficient lightweight scoring or skipping scoring when the system is healthy. Thus, while ensuring the accuracy of the scoring results, the utilization efficiency of scoring resources is improved. This achieves optimized matching between the scoring strategy and the system status, enabling the scoring system to remain stable and efficient under various operating environments. At the same time, it provides an accurate system status reference for subsequent scoring processes, thereby improving the accuracy and efficiency of context unit scoring results.

[0172] In one optional embodiment of this specification, the preset scoring strategy includes a complete scoring strategy, and the system status information includes system latency information, system failure information, and system load information; Based on system status information, the target scoring strategy is determined from the preset scoring strategies, including: If the system latency information does not reach the preset latency threshold, the system failure information does not reach the preset failure threshold, and the system load information does not reach the preset load threshold, the complete scoring strategy is determined as the target scoring strategy.

[0173] The Full-Scale (FULL) scoring strategy is a comprehensive evaluation method that the system can execute when scoring large-scale models of contextual units. It is one of the preset scoring strategies and can be used to perform in-depth analysis and structured output of all uncovered candidate scoring items to obtain comprehensive scoring information. Specifically, the FULL strategy involves a complete evaluation of all uncovered candidate scoring items, outputting complete scoring criteria and scores. Its execution depth and output format are complete, and it can include detailed information such as the set of scoring covered items, evidence fragments, scores, and execution time.

[0174] System latency information is a quantitative indicator reflecting the time spent by a context-based unit scoring system in processing requests. It is one dimension of system status information and can be used to assess the system's current response speed and processing efficiency. Specifically, system latency information can include the average latency value (avg_latency) over a recent period, quantile latency (P95 latency), etc. For example, the average latency time of the last 10 scoring requests can be calculated, or a moving average can be used to smooth fluctuations. By monitoring system latency information, it is possible to determine whether the system is in a high-latency state, thereby adjusting the scoring strategy when latency is too high and avoiding performing costly full scoring when the system response is slow.

[0175] System failure information is a quantitative indicator reflecting the frequency of failed rating requests in a context-based unit rating system. It can be used to assess the system's current stability and reliability. Specifically, system failure information can include the recent rating failure rate (failure_rate) or the number of consecutive failures (consecutive_failures). For example, the percentage of failed requests out of the last 20 can be calculated, or a sliding window can be used to dynamically calculate the failure rate. By monitoring system failure information, it's possible to determine if the system is unstable, and then implement a circuit breaker rating strategy when the failure rate is too high to prevent further system deterioration and ensure overall system availability.

[0176] Specifically, a rating request failure refers to the failure to return a valid rating result after a rating request is sent to the LLM service. Typical failure scenarios may include: LM service timeout (response time exceeding a preset limit); LLM service returning error codes, such as HTTP 5xx; and LLM returning results with an abnormal format that cannot be parsed, such as JSON parsing failure. In the code implementation, the rating request execution can be wrapped in a try-except block. Any execution exception can be recorded as a failure using `scheduler.record_result(False, latency)`, and the `consecutive_failures` counter can be incremented to obtain system failure information.

[0177] System load information is a quantitative indicator reflecting the current workload of the context unit scoring system. It is one dimension of the system status information and can be used to assess the system's current resource utilization and processing capacity. Specifically, system load information can include the current queue length (queue_length) and concurrency inflight (concurrency_inflight), such as the number of context units currently waiting to be scored or the number of scoring requests being processed. System load information, along with system latency information, system failure information, and other technical features, constitutes a component of system status information. By monitoring system load information, it is possible to determine whether the system is overloaded. This allows for the selection of queuing or skipping scoring strategies when the load is too high, preventing system resource exhaustion, ensuring system stability under high load conditions, and improving system resource utilization efficiency.

[0178] A preset latency threshold is a pre-defined numerical boundary used for comparison with system latency information. It can be used to determine whether the current system latency is within an acceptable range, thereby deciding whether to execute the full scoring strategy. Specifically, the preset latency threshold can be a specific value, such as a recent latency of 10 seconds. When the system latency information does not reach this threshold, it indicates that the system response speed is fast, and the full scoring strategy can be executed; when the system latency information reaches this threshold, it indicates that the system response speed is slow, and a lightweight scoring strategy or a queuing strategy may need to be selected.

[0179] A preset failure threshold is a pre-defined numerical boundary used for comparison with system failure information. It is used to determine whether the current failure rate of the system is within an acceptable range, thereby deciding whether to execute the full scoring strategy. Specifically, the preset failure threshold can be a specific value, such as a failure rate of 0.15 or a consecutive failure count of 10. When the system failure information does not reach the threshold, it indicates that the system stability is high, and the full scoring strategy can be executed. When the system failure information exceeds the threshold, it indicates that the system stability is low, the scoring service is malfunctioning, and it cannot proceed normally; therefore, a circuit breaker scoring strategy needs to be selected.

[0180] A preset load threshold is a pre-defined numerical boundary used for comparison with system load information. It can be used to determine whether the current system load is within an acceptable range, thereby deciding whether to execute the full scoring strategy. Specifically, the preset load threshold can be a specific value, such as a queue length of 100 or a concurrency of 10. When the system load information does not reach this threshold, it indicates that the system resource usage is low, and the full scoring strategy can be executed; when the system load information reaches this threshold, it indicates that the system resource usage is high, and it may be necessary to choose to wait in a queue or skip the scoring strategy.

[0181] In practical applications, determining the full-scale scoring strategy as the target scoring strategy requires satisfying multiple system state conditions simultaneously.

[0182] Specifically, once the context score has reached the preset context score threshold, the current system status information can be further obtained, including system latency information, system failure information, and system load information.

[0183] Furthermore, the system status information is compared with the corresponding preset thresholds. If the system latency information does not reach the preset latency threshold, the system failure information does not reach the preset failure threshold, and the system load information does not reach the preset load threshold, the system can be determined to be in a healthy and idle state. In this case, the full scoring strategy can be selected as the target scoring strategy. That is, if the context unit scoring system has sufficient operating resources and a good operating environment, the full scoring strategy can be executed.

[0184] In the embodiments of this specification, by determining the complete scoring strategy as the target scoring strategy when the system latency information does not reach the preset latency threshold, the system failure information does not reach the preset failure threshold, and the system load information does not reach the preset load threshold, the system can dynamically adjust the scoring strategy according to the current health status and load conditions. This ensures that the complete scoring process is executed when system resources are sufficient, latency is controllable, and stability is high, thereby improving the comprehensiveness and accuracy of the scoring results. By making multi-dimensional judgments based on system status information, the system avoids executing high-cost complete scoring when system latency is too high, failure rate is too high, or load is too high, reducing unnecessary consumption of computing resources and improving the utilization efficiency of scoring resources. At the same time, by real-time monitoring of system status information and multi-dimensional threshold judgments, the scoring strategy can adapt to the current operating environment of the system, improving the overall stability of the scoring system and user experience, and achieving the optimal balance between scoring quality and system performance.

[0185] In one optional embodiment of this specification, the preset scoring strategy includes a lightweight scoring strategy, and the system status information includes system latency information, system failure information, and system load information. Based on system status information, the target scoring strategy is determined from the preset scoring strategies, including: If the system latency information reaches the preset latency threshold, the system failure information does not reach the preset failure threshold, and the system load information does not reach the preset load threshold, then the lightweight scoring strategy is determined as the target scoring strategy.

[0186] Lightweight scoring strategy (LIGHT) is a simplified evaluation method that can be executed when the system scores large-scale contextual units. It is one of the preset scoring strategies and can be used to simplify the evaluation of target scoring items corresponding to high information gain, thereby reducing the computational cost and response time of scoring. Specifically, lightweight scoring strategy involves simplifying the evaluation only for target scoring items, while limiting the output length and fields to reduce the large language model resources required for scoring. Its execution depth and output format are concise, and can include basic information such as scoring coverage item identifiers, brief evidence fragments, and scoring scores.

[0187] In practical applications, determining a lightweight scoring strategy requires satisfying multiple system state conditions simultaneously.

[0188] Specifically, when the context score has reached the preset context score threshold, the current system status information can be obtained, including system latency information, system failure information, and system load information.

[0189] Furthermore, the system status information is compared with corresponding preset thresholds. If the system latency information reaches the preset latency threshold, the system failure information does not reach the preset failure threshold, and the system load information does not reach the preset load threshold, it can be determined that the system is currently in a state of high latency but good load and stability. In this case, the lightweight scoring strategy can be determined as the target scoring strategy. That is, when the context unit scoring system has high latency, only the target scoring item is simplified for evaluation, and the output length and fields are limited.

[0190] In the embodiments of this specification, a lightweight scoring strategy is determined as the target scoring strategy when system latency information reaches a preset latency threshold, system failure information does not reach a preset failure threshold, and system load information does not reach a preset load threshold. This allows the system to dynamically adjust the scoring strategy based on the current health status and load conditions, ensuring that the lightweight scoring process is executed even when system latency is high but stability is good. This reduces the computational complexity and response latency of the scoring while maintaining basic scoring quality. By using multi-dimensional judgment based on system status information, high-cost full scoring is avoided when system latency is too high, reducing unnecessary consumption of computing resources and improving the utilization efficiency of scoring resources. At the same time, through real-time monitoring of system status information and multi-dimensional threshold judgment, the scoring strategy can adapt to the current operating environment of the system, improving the overall stability of the scoring system and user experience, and achieving a reasonable balance between scoring quality and system performance.

[0191] In one optional embodiment of this specification, the preset scoring strategy includes a waiting scoring strategy, and the system status information includes system latency information, system failure information, and system load information. Based on system status information, the target scoring strategy is determined from the preset scoring strategies, including: If the system delay information does not reach the preset delay threshold, the system failure information does not reach the preset failure threshold, and the system load information reaches the preset load threshold, the waiting scoring strategy is determined as the target scoring strategy.

[0192] The Queued scoring strategy is a waiting processing method that the system can execute when scoring large models of context units. It is one of the preset scoring strategies and can be used to put scoring tasks into a queue to wait for system resources to be released when the system load is high, so as to avoid system overload. Its execution method is to temporarily cache the current context unit scoring task and continue to execute it after the system load decreases. Its execution depth and output format remain intact and can include basic information such as scoring coverage item identifiers, evidence fragments, and scoring scores.

[0193] In practical applications, determining the waiting scoring strategy requires satisfying multiple system state conditions simultaneously.

[0194] Specifically, when the context score has reached the preset context score threshold, the current system status information can be obtained, including system latency information, system failure information, and system load information.

[0195] Furthermore, the system status information is compared with the corresponding preset thresholds. If the system latency information does not reach the preset latency threshold, the system failure information does not reach the preset failure threshold, and the system load information reaches the preset load threshold, it can be determined that the system is currently in a state of high load but good latency and stability. At this time, the waiting scoring strategy can be determined as the target scoring strategy. That is, when the load of the context unit scoring system is high, the scoring task is temporarily placed in the queue to wait until the system load is reduced to an acceptable range.

[0196] In the embodiments of this specification, by determining the waiting scoring strategy as the target scoring strategy when system latency information does not reach a preset latency threshold, system failure information does not reach a preset failure threshold, and system load information reaches a preset load threshold, the system can dynamically adjust the scoring strategy according to the current health status and load conditions. This ensures that the waiting scoring process is executed when the system load is high but latency and stability are good, thereby avoiding system overload while guaranteeing the complete execution of the scoring task. Through multi-dimensional judgment based on system status information, high-cost complete scoring is avoided when the system load is too high, reducing unnecessary system resource competition and waiting time, and improving the utilization efficiency of system resources. At the same time, through real-time monitoring of system status information and multi-dimensional threshold judgment, the scoring strategy can adapt to the current operating environment of the system, improving the overall stability of the scoring system and user experience, and achieving a reasonable balance between scoring quality and system performance.

[0197] In one optional embodiment of this specification, the preset scoring strategy includes a skip scoring strategy, and the system status information includes system latency information, system failure information, and system load information. Based on system status information, the target scoring strategy is determined from the preset scoring strategies, including: If the system latency information reaches the preset latency threshold, the system failure information does not reach the preset failure threshold, and the system load information reaches the preset load threshold, then the skip scoring strategy is determined as the target scoring strategy.

[0198] The Skip Scoring strategy (SKIPPED) is a skip processing method that can be executed when the system performs large-scale scoring on contextual units. It is one of the preset scoring strategies and can be used to abandon the scoring operation for the current contextual unit when system latency is too high and the load is too heavy, thus avoiding unnecessary consumption of system resources. Specifically, the Skip Scoring strategy directly skips the scoring process of the current contextual unit, does not call the large language model scoring service, has an empty execution depth and output format, and does not produce any scoring results.

[0199] In practical applications, determining the skip scoring strategy requires satisfying multiple system state conditions simultaneously.

[0200] Specifically, when the context score has reached the preset context score threshold, the current system status information can be obtained, including system latency information, system failure information, and system load information.

[0201] Furthermore, the system status information is compared with the corresponding preset thresholds. If the system latency information reaches the preset latency threshold, the system failure information does not reach the preset failure threshold, and the system load information reaches the preset load threshold, it can be determined that the system is currently in a state of excessive latency and excessive load. At this time, the skip scoring strategy can be determined as the target scoring strategy. That is, when the latency and load of the context unit scoring system are both high, the scoring of the current context unit is skipped.

[0202] Optionally, when implementing a skip scoring strategy, it is a one-time decision for a single current context unit. When the next context unit arrives, the system state information is re-evaluated for a new decision. Skipped current context units are not discarded but stored in a cache list, namely the skipped_contexts list, for example, a maximum of 50 context units can be retained.

[0203] For skipped contextual units, there are several subsequent scoring methods. One option is to aggregate the text content of the skipped contextual unit along with newly arriving dialogue text fragments or statements to form a new contextual unit, which is then re-scored and re-scheduled in the next round. Another option is to provide an external retrieval interface, namely the `get_skipped_contexts()` interface, within the contextual unit scoring system. This would allow for offline batch re-scoring of the text content within skipped contextual units after the current medical dialogue ends, enabling either delayed or offline processing.

[0204] In the embodiments of this specification, by determining the skip scoring strategy as the target scoring strategy when system latency information reaches a preset latency threshold, system failure information does not reach a preset failure threshold, and system load information reaches a preset load threshold, the system can dynamically adjust the scoring strategy according to the current health status and load conditions. This ensures that the skip scoring process is executed when system latency and load are too high, thereby avoiding unnecessary consumption of system resources, preventing system overload, and improving the utilization efficiency of system resources. Through multi-dimensional judgment based on system status information, the system avoids performing high-cost full scoring when system latency is too high and load is too heavy, reducing unnecessary system resource competition and waiting time, and improving the utilization efficiency of system resources. At the same time, through real-time monitoring of system status information and multi-dimensional threshold judgment, the scoring strategy can adapt to the current operating environment of the system, improving the overall stability of the scoring system and user experience, and achieving a reasonable balance between scoring quality and system performance.

[0205] In one optional embodiment of this specification, the preset scoring strategy includes a circuit breaker scoring strategy, and the system status information includes system latency information, system failure information, and system load information; Based on system status information, the target scoring strategy is determined from the preset scoring strategies, including: When the number of system failures reaches a preset failure threshold, the circuit breaker scoring strategy is determined as the target scoring strategy.

[0206] The circuit breaker scoring strategy is a pause processing method that can be executed when the system scores large-scale language model units. It is one of the preset scoring strategies and can be used to pause the scoring operation for the current context unit when the system failure information reaches a preset failure threshold, that is, when the system failure rate is too high or the continuous failure reaches the threshold, in order to prevent system overload and resource waste. Its execution method is to directly pause the scoring process without calling the large language model scoring service. Its execution depth and output format are empty, and no scoring results are generated.

[0207] In practical applications, the circuit breaker scoring strategy can be determined solely based on the system failure information dimension.

[0208] Specifically, when the context score has reached the preset context score threshold, the current system status information can be obtained, including system latency information, system failure information, and system load information.

[0209] Furthermore, the system status information is compared with the corresponding preset thresholds. When the system failure information reaches the preset failure threshold, it can be determined that the system is currently in a state of excessive latency and excessive load. At this time, the circuit breaker scoring strategy can be determined as the target scoring strategy.

[0210] In other words, the circuit breaker scoring strategy indicates that the downstream large language model service of the current context unit scoring system has experienced a persistent anomaly and is unable to provide scoring services normally. In this case, regardless of whether the context score has reached the preset context score threshold, or whether the system latency and load conditions have reached the threshold, the circuit breaker scoring strategy will be determined as the target scoring strategy.

[0211] Furthermore, after the circuit breaker scoring strategy is implemented, the context unit scoring system can enter a cooling-off period, which is a continuously preset cooling-off time T_cool. During the cooling-off period, any new scoring request will be blocked.

[0212] Afterward, the system can enter a Half-Open state, allowing a limited number of exploratory scoring requests to pass through and attempting to invoke downstream large language models to execute the scoring requests. The execution method and number of exploratory scoring requests can be preset; for example, one request can be allowed every 5 seconds, and a lightweight scoring strategy can be adopted to reduce the impact on the system.

[0213] In the half-open state, if the number of consecutive successful executions of a rating request reaches a preset recovery threshold (e.g., 5 consecutive successful executions), it can be determined that the rating request service has returned to normal, and the circuit breaker rating policy can be lifted, allowing normal rating requests to be executed according to other policies again.

[0214] In the embodiments described in this specification, when the system failure information reaches a preset failure threshold, the circuit breaker scoring strategy is determined as the target scoring strategy. This allows the system to dynamically adjust the scoring strategy based on the current failure rate, ensuring that the circuit breaker scoring process is executed when the system failure rate is too high. This prevents further system deterioration and avoids continuing to perform scoring operations when the system is abnormal, thus improving the utilization efficiency of system resources. By making multi-dimensional judgments based on system status information, the high-cost full scoring is avoided when the system failure rate is too high, reducing unnecessary system resource consumption. At the same time, through real-time monitoring of system status information and a circuit breaker recovery mechanism, the scoring strategy can adapt to the current operating environment of the system, improving the overall stability of the scoring system and user experience, and achieving a reasonable balance between scoring quality and system performance.

[0215] In an optional embodiment of this specification, before scoring the current context unit based on the target scoring strategy and obtaining the target score result for the current context unit, the method further includes: Retrieve rating token information; Based on the target scoring strategy, the current context unit is scored to obtain the target score result of the current context unit, including: When the scoring token information is sufficient, the current context unit is scored based on the target scoring strategy to obtain the target scoring result of the current context unit; If there are insufficient scoring tokens, adjust the target scoring strategy and score the current context unit based on the adjusted target scoring strategy to obtain the target scoring result of the current context unit.

[0216] Scoring token information is a quantitative indicator reflecting the availability of tokens for a scoring request. It can be used to assess the current flow control status of the system and provide a basis for decision-making regarding the execution of scoring requests. Specifically, scoring token information includes parameters such as the number of available tokens in the current token bucket, the token replenishment rate, and the token usage status, which can be used to determine whether the system allows a scoring request to be initiated. For example, scoring token information can be represented as follows: the number of scoring tokens is 5, the replenishment rate is 1 token every 5 seconds, and the current status is that tokens are sufficient.

[0217] Specifically, the rating token information can come from the token bucket mechanism. The token bucket is a traffic shaping mechanism used to control the rate of rating requests. It can limit the average rate of rating requests per unit time and allow limited bursts, thereby preventing instantaneous high concurrency from overwhelming downstream rating services. Specifically, the token bucket can replenish tokens at a preset rate. The token capacity of the token bucket limits the maximum number of burst requests. Each rating request consumes one or more tokens. If there are insufficient tokens, the rating request will be blocked or downgraded. For example, the token bucket can be configured to replenish 1 token every 5 seconds, with a capacity of 3, meaning that a maximum of 1 request is allowed every 5 seconds, but 3 requests can burst.

[0218] "Token Sufficient" is a status of the scoring token information, indicating that the number of available tokens in the current token bucket has reached or exceeded a preset threshold, allowing scoring requests to be initiated normally. Specifically, the token sufficient status usually indicates that the number of tokens is greater than or equal to 1, meaning that the current scoring request initiation rate is controllable, the system computing resources are sufficient, and scoring requests using complete or lightweight scoring strategies can be processed.

[0219] "Insufficient tokens" is a status indicating that the number of available tokens in the token bucket is below a preset threshold, or there are no available tokens to meet the token requirements of a scoring request. Therefore, the scoring request needs to be downgraded or skipped. Specifically, an insufficient token status typically means that the number of tokens is less than 1, and the system cannot execute the normal scoring strategy, requiring adjustment of the target scoring strategy. For example, if the number of tokens is 1, and the target scoring strategy requires 2 tokens, the target scoring strategy needs to be downgraded, such as to a lightweight scoring strategy, which can execute the scoring request using only 1 token. When the number of tokens is 0, it indicates that the current system cannot execute the scoring request, and the target scoring strategy can be adjusted to a skip scoring strategy.

[0220] The adjusted target scoring strategy is a new scoring strategy modified from the original target scoring strategy to address token shortage situations. It can be used to reduce the resource consumption of scoring requests when system resources are strained. Specifically, the adjusted target scoring strategy includes downgrading to a lightweight scoring strategy, waiting for scoring, or directly setting it to a skip scoring strategy.

[0221] In practical applications, obtaining rating token information can be achieved by interacting with the token bucket.

[0222] Specifically, the system can pre-maintain a token bucket instance that can replenish tokens at a fixed rate and has a token capacity.

[0223] Before determining the target scoring strategy and initiating a scoring request, the system can call the token bucket's token acquisition interface (such as try_acquire()). This interface attempts to retrieve a token from the token bucket. If there are available tokens in the token bucket, the acquisition is successful, and the interface returns a token sufficient status, such as a status indicator indicating sufficient tokens (e.g., True or 1), while the number of tokens in the bucket is decremented by 1. If there are no tokens left in the token bucket, the acquisition fails, and the interface returns a token insufficient status, such as a status indicator indicating insufficient tokens (e.g., False or 0).

[0224] If there are insufficient scoring tokens, the system needs to adjust the original target scoring strategy. The adjustment method can be flexibly selected based on the preset degradation rules and the current system status.

[0225] Specifically, a step-by-step degradation strategy can be adopted. For example, if the original target scoring strategy is a full scoring strategy, when there are insufficient tokens, it can first try to adjust it to a lightweight scoring strategy and try to obtain tokens again. If the lightweight scoring strategy still cannot be executed due to insufficient tokens, it can be further adjusted to a waiting scoring strategy, and the request can be placed in a queue to be executed after the resources are released. If the queue is also full, it can finally be adjusted to a skip scoring strategy, directly abandoning this call, and storing the context unit content in a skip list for subsequent offline processing.

[0226] Optionally, after obtaining the rating token information, the method also includes obtaining concurrency information.

[0227] Specifically, based on traffic shaping for rating requests using rating token information, further shaping can be achieved by combining concurrency limits. Concurrency limits are a mechanism used to control the number of concurrent rating requests, preventing resource exhaustion caused by processing too many requests simultaneously and ensuring stable system operation. Specifically, a maximum concurrency threshold can be set. When the current number of concurrent rating requests reaches this threshold, newly generated rating requests will be blocked or downgraded until the concurrency decreases. For example, a maximum concurrency limit of 2 means the system can process a maximum of 2 rating requests simultaneously.

[0228] Concurrency limiting, together with token bucket technology, forms a traffic shaping module. By limiting the number of concurrent requests and the request rate, it enables comprehensive control over scoring requests and avoids system overload.

[0229] Optionally, the preset rate at which the token bucket replenishes tokens can be adjusted based on system latency information in the system status information. Specifically, if the system's most recent average latency value is higher than a preset average threshold, the preset rate can be reduced; if the system's most recent average latency value is lower than the preset average threshold and the scoring request queue is empty, the preset rate can be increased.

[0230] For example, see Figure 5 , Figure 5 This specification illustrates a traffic shaping method for a rating request according to an embodiment of the present invention, as shown in the diagram. Figure 5 As shown, the traffic shaping method for this rating request can specifically include: Get the currently pending rating requests; Retrieve scoring token information from the token bucket; Determine if the scoring token information indicates sufficient tokens; If not, then enter the first waiting queue and re-acquire the scoring token information; If so, then determine whether the rating request has reached the concurrency limit; If so, it enters the second waiting queue, and the concurrency limit for the rating request is re-evaluated; If not, the large language model is invoked to execute the scoring request.

[0231] In the embodiments described in this specification, by acquiring scoring token information, the target scoring strategy is executed when tokens are sufficient and adjusted when tokens are insufficient. This allows the system to dynamically adjust the scoring strategy based on the current token status, avoiding system overload caused by executing the high-cost full scoring strategy when tokens are insufficient. Through the coordinated control of token bucket and concurrency limits, accurate traffic shaping of scoring requests is achieved, preventing system overload and resource waste, improving the stability and resource utilization of the system under high load conditions, while ensuring the basic requirements of scoring quality, and realizing performance optimization of the scoring system under resource constraints.

[0232] In an optional embodiment of this specification, after scoring the current context unit based on the target scoring strategy and obtaining the target score result for the current context unit, the method further includes: The target score is determined to be the historical score, and multiple candidate score items are updated based on the historical score.

[0233] In practical applications, after obtaining the target score result for the current context unit, the target score result can be identified as the historical score result, and multiple candidate score items can be updated based on the historical score result. Updating multiple candidate score items based on historical score results is a closed-loop process of dynamically maintaining the set of items to be scored.

[0234] Specifically, after obtaining the target score result of the current context unit, the target score result can be used as a historical score result and stored in a pre-built historical score result library, so that it can become the basis for scoring historical context units in subsequent scoring processes.

[0235] Furthermore, the target scoring results can be parsed to extract the identifiers of newly covered scoring items during the current scoring process, and the newly covered scoring items can be removed from the current set of multiple candidate scoring items to obtain an updated set of candidate scoring items.

[0236] In addition, the update operation can be combined with a sliding window mechanism. For example, only the most recent N historical rating results can be retained to dynamically calculate the candidate set, or the order of the rating items in the candidate set can be adjusted according to the priority and time decay factor of the rating items to ensure that the set of candidate rating items can reflect the dimensions that still need to be evaluated in the current dialogue in real time and accurately.

[0237] The updated candidate scoring items will serve as input for the next round of scoring, used to calculate similarity and evaluate value with new contextual units.

[0238] In the embodiments of this specification, by determining the target scoring result as the historical scoring result and dynamically updating multiple candidate scoring items based on the historical scoring result, the system can continuously maintain the set of uncovered scoring items based on the historical scoring result, avoiding repeated evaluation of covered scoring items. By extracting the identifiers of newly covered scoring items from the target scoring result and removing these scoring items from the candidate scoring item set, the accuracy and timeliness of the candidate scoring item set are ensured, avoiding repeated evaluation of the same content in subsequent scoring, thereby reducing the ineffective consumption of large language model calls, improving the utilization efficiency of scoring resources, and providing accurate input basis for subsequent context unit scoring. This achieves closed-loop management from scoring execution to candidate scoring item update, further improving the accuracy and efficiency of context unit scoring results.

[0239] Corresponding to the above method embodiments, this specification also provides an embodiment of a context unit scoring system, specifically, see [link to embodiment]. Figure 6 , Figure 6 A schematic diagram of a contextual unit scoring system provided in one embodiment of this specification is shown. Figure 6 As shown, the context unit scoring system includes: The speech recognition input module can receive the speech stream of doctor-patient dialogue and generate continuous text segment events (including speaker identifier, text, and timestamp). This module continuously outputs partial results (partial recognized text) in a "streaming" manner.

[0240] The fragment event access module can be used to standardize the text fragments output by ASR into a unified event format (e.g., {speaker, text, timestamp}) and deliver them to the subsequent processing chain; this module can be implemented as a message queue, event bus or in-process buffer queue.

[0241] The Semantic Buffer module is used to merge fragmented ASR segments into semantically complete contextual units called DialogueContext. This module consists of a sentence integration submodule and a context aggregation submodule. The sentence integration submodule can stitch together multiple fragments into a complete sentence based on strong punctuation (period, question mark, etc.), pause time threshold, speaker switching, etc. The context aggregation submodule can aggregate consecutive sentences into contextual units (such as question-and-answer pairs or topic paragraphs) that are meaningful for scoring, while satisfying minimum / ideal / maximum length and round constraints.

[0242] The information gain filtering module can be used to determine whether the current context unit can bring new information to the "uncovered scoring item set", thus deciding whether it is worthwhile to trigger the large model scoring. Information gain can be calculated using approximate methods such as "keyword matching, vector similarity, or mixed similarity", and high / low thresholds can be set for triggering or skipping.

[0243] The Adaptive Scheduler module selects and executes a corresponding scoring strategy based on system health status (latency, failure rate, queue length, concurrency, etc.) after information gain has been assessed. Preset scoring strategies include at least: Full scoring, Light scoring, Queued scoring, Skip scoring, and Circuit Breaker scoring. This module also maintains a consecutive failure / success count and implements closed-loop control for degradation and recovery.

[0244] The Flow Shaping module can be used to impose hard constraints on LLM calls to prevent transient concurrency surges. This module includes at least: The Token Bucket submodule replenishes tokens at a fixed rate, with the bucket capacity limited by bursts, and tokens must be consumed before any call can be made. The concurrency limiting submodule (Semaphore / concurrency counter) can limit the maximum number of LLM requests in transit at the same time.

[0245] When there are insufficient tokens or the concurrency is full, tasks can be queued or dropped / degraded according to a policy.

[0246] The Large Model Scoring Executor (LLM Executor) can be used to select the corresponding scoring prompt and output format based on the scheduling action. FULL: Perform a full evaluation of all uncovered rating items; LIGHT: Lightweight evaluation is performed only on candidate scoring items with high information gain, and the output length / fields are limited to reduce latency and cost.

[0247] The state storage and write-back module can be used to store and update scoring item coverage states (covered_items), evidence fragments, recent latency statistics (avg_latency), failure rate (failure_rate), queue length, etc. This module provides closed-loop input for the next round of scheduling.

[0248] External output and display module: Used to return real-time scoring results (coverage items, evidence, scores, etc.) to the upper-level system or front-end interface to provide real-time quality control prompts or record improvement suggestions.

[0249] The contextual unit scoring system provided in this specification ensures scoring accuracy through semantic buffering, controls call costs through information gain filtering, ensures system stability and controllable latency through adaptive scheduling and traffic shaping, and forms a closed-loop optimization mechanism through state write-back. It achieves a balance between the high-frequency output of streaming speech recognition and the high latency and high volatility of large language model batch processing, and achieves an engineering-feasible balance between scoring accuracy, computational cost, and operational stability, thereby improving the overall efficiency and quality of real-time scoring of medical dialogues.

[0250] Corresponding to the above method embodiments, this specification also provides a temporal description embodiment of the context unit scoring method, specifically, see [link to embodiment]. Figure 7 , Figure 7 A timing diagram illustrating a contextual unit scoring method provided in one embodiment of this specification is shown. Figure 7 As shown.

[0251] The user / client generates a voice stream and inputs it into the speech recognition module; The speech recognition input module recognizes the speech stream, generates text segments, and sends them to the semantic buffer module; The semantic buffer module receives text fragments, integrates and aggregates them, generates contextual units, and sends them to the information gain module. The information gain module receives context units, performs information gain calculations, generates a scoring request, and sends it to the scheduling and shaping module. The scheduling and shaping module receives scoring requests, and based on the traffic shaping strategy, determines whether to execute the scoring request and sends it to the large language model service. The large language model service receives and executes scoring requests, generates scoring results, and returns them to the scheduling and shaping module. The scheduling and shaping module receives the scoring results and sends them back to the user / client.

[0252] Corresponding to the above-described embodiment of the context unit scoring system, this specification also provides an embodiment of a specific processing flow for the context unit scoring method within the context unit scoring system. Specifically, see [link to embodiment]. Figure 8 , Figure 8 A schematic diagram illustrating the processing flow of a contextual unit scoring method according to an embodiment of this specification is shown. Figure 8 As shown, this contextual unit scoring method can specifically include: Receive text fragment events output by the automatic speech recognition module.

[0253] The text fragment is written into the sentence buffer of the semantic buffer module.

[0254] The sentence buffer is integrated based on at least one boundary condition to obtain a complete sentence.

[0255] Complete sentences are written into the context buffer of the semantic buffer module and continuously aggregated to form context units.

[0256] Output the context unit when the context output conditions are met.

[0257] Perform information gain calculation on the output context unit.

[0258] Determine whether the information gain has reached the information gain threshold. If so, trigger scoring; otherwise, skip scoring.

[0259] Read system status information and select the target scoring strategy.

[0260] When the target scoring strategy is a full scoring strategy or a lightweight scoring strategy, rate and concurrency control are implemented: Perform large language model scoring and return the scoring results.

[0261] In the embodiments of this specification, by receiving text fragment events output by the automatic speech recognition module, integrating them into complete sentences based on boundary conditions, and aggregating them into contextual units, the semantic integrity of the contextual units is ensured, avoiding inaccurate scoring due to incomplete sentences. By performing information gain calculation and determining whether to trigger scoring based on the information gain threshold, automatic filtering of low-value or repetitive dialogue content is achieved, avoiding invalid evaluation of covered scoring items. By reading system status information to dynamically select the target scoring strategy and implementing rate and concurrency control under the complete scoring strategy or the lightweight scoring strategy, the scoring system can adaptively adjust the scoring depth and resource consumption according to the current system latency, failure rate, and load. This achieves a dual improvement in scoring accuracy and efficiency, reducing the computational cost and resource consumption of large language model calls while ensuring the quality of medical dialogue analysis. This enables the system to maintain stable operation under high load scenarios and provides more timely and accurate scoring support for clinical quality control and diagnosis and treatment assistance, improving the overall performance of the real-time analysis and scoring system for medical dialogue.

[0262] Corresponding to the above method embodiments, this specification also provides embodiments of a context unit scoring device. Figure 9 A schematic diagram of a context unit scoring device according to one embodiment of this specification is shown. Figure 9 As shown, the device includes: The acquisition module 902 is configured to acquire the current context unit and multiple candidate scoring items constructed based on the medical dialogue flow, wherein the multiple candidate scoring items are determined based on the historical scoring results of the historical context unit; The target scoring item determination module 904 is configured to determine the target scoring item from multiple candidate scoring items based on the similarity between the current context unit and multiple candidate scoring items; The context scoring determination module 906 is configured to determine the context score of the current context unit based on the similarity between the current context unit and the target scoring item. The target scoring strategy determination module 908 is configured to determine the target scoring strategy from the preset scoring strategies based on context scoring and preset context scoring thresholds. The scoring module 910 is configured to score the current context unit based on the target scoring strategy and obtain the target scoring result of the current context unit.

[0263] The acquisition module is a component in the context unit scoring device used to receive and process medical dialogue stream data. It can be used to acquire raw dialogue data from a speech recognition system or an external data source. Specifically, the acquisition module may include a processing unit that continuously receives the medical dialogue stream, acquiring multiple dialogue text fragments from the medical dialogue stream to provide basic input data for subsequent context unit construction.

[0264] The target scoring item determination module is a component in the context unit scoring device used to determine the target scoring item most relevant to the current context unit from a set of candidate scoring items. It can be used to determine the target scoring item from multiple candidate scoring items based on the similarity between the current context unit and these items, providing a reference for context scoring calculation. Specifically, the target scoring item determination module may include a processing unit that performs feature extraction, similarity calculation, and ranking of the current context unit and multiple candidate scoring items, used to determine one or more scoring items most semantically relevant to the current context unit.

[0265] The context scoring module is a component in the context unit scoring device used to calculate the added information value of the current context unit to the uncovered scoring items. It can be used to determine the context score of the current context unit based on the similarity between the current context unit and the target scoring item, providing a quantitative basis for scoring strategy selection. Specifically, the context scoring module may include a processing unit that calculates and aggregates the similarity between the current context unit and the target scoring item to determine the value of the context score.

[0266] The target scoring strategy determination module is a component in the context unit scoring device used to determine a target scoring strategy from preset scoring strategies based on the context score and a preset context scoring threshold. It can be used to determine the target scoring strategy based on the context score and the preset context scoring threshold, providing execution instructions to the scoring module. Specifically, the target scoring strategy determination module may include a processing unit that compares the context score and the preset context scoring threshold, and makes a decision by combining system state information, to determine the appropriate scoring execution method for the current context unit.

[0267] The scoring module is a component in the context unit scoring device used to invoke a large language model to infer the current context unit and generate a scoring result. It can be used to score the current context unit based on a target scoring strategy, obtain the target scoring result for the current context unit, and provide output data for subsequent scoring processes. Specifically, the scoring module may include a processing unit that parses the target scoring strategy, constructs input data, invokes the scoring model for inference, and parses the response data to generate a structured scoring result.

[0268] Optionally, the acquisition module 902 is further configured to: acquire a medical dialogue stream; perform speech recognition on the medical dialogue stream to obtain multiple dialogue text fragments; integrate the multiple dialogue text fragments based on the semantic information of the dialogue text fragments to obtain at least one dialogue statement; and aggregate at least one dialogue statement to obtain the current context unit if at least one dialogue statement meets a preset aggregation condition.

[0269] Optionally, the target rating item determination module 904 is further configured to: extract features from the current context unit and multiple candidate rating items to obtain context feature representations and multiple rating item feature representations; determine the similarity between the current context unit and each candidate rating item based on the context feature representations and multiple rating item feature representations; sort the multiple candidate rating items based on the multiple similarities to obtain a rating item sorting result; and determine the target rating item based on the rating item sorting result.

[0270] Optionally, the target rating item determination module 904 is further configured to: perform keyword matching based on contextual feature representation and multiple rating item feature representations to determine the similarity between the current contextual unit and each candidate rating item; and / or, perform vector similarity calculation based on contextual feature representation and multiple rating item feature representations to determine the similarity between the current contextual unit and each candidate rating item.

[0271] Optionally, the target scoring strategy determination module 908 is further configured to: determine a target scoring strategy from preset scoring strategies based on context scoring and preset context scoring thresholds, including: determining the target scoring strategy from preset scoring strategies when the context scoring reaches the preset context scoring threshold.

[0272] Optionally, the acquisition module 902 is further configured to: acquire the system status information of the context unit scoring system; the target scoring strategy determination module 908 is further configured to: determine the target scoring strategy from the preset scoring strategies based on the system status information when the context score reaches the preset context score threshold.

[0273] Optionally, the preset scoring strategy includes a complete scoring strategy, and the system status information includes system latency information, system failure information, and system load information; the target scoring strategy determination module 908 is further configured to: determine the complete scoring strategy as the target scoring strategy when the system latency information does not reach the preset latency threshold, the system failure information does not reach the preset failure threshold, and the system load information does not reach the preset load threshold.

[0274] Optionally, the preset scoring strategy includes a complete scoring strategy, and the system status information includes system latency information, system failure information, and system load information; the target scoring strategy determination module 908 is further configured to: determine the lightweight scoring strategy as the target scoring strategy when the system latency information reaches a preset latency threshold, the system failure information does not reach a preset failure threshold, and the system load information does not reach a preset load threshold.

[0275] Optionally, the preset scoring strategy includes a complete scoring strategy, and the system status information includes system latency information, system failure information, and system load information; the target scoring strategy determination module 908 is further configured to: determine the waiting scoring strategy as the target scoring strategy when the system latency information does not reach the preset latency threshold, the system failure information does not reach the preset failure threshold, and the system load information reaches the preset load threshold.

[0276] Optionally, the preset scoring strategy includes a complete scoring strategy, and the system status information includes system latency information, system failure information, and system load information; the target scoring strategy determination module 908 is further configured to: determine the skip scoring strategy as the target scoring strategy when the system latency information reaches a preset latency threshold, the system failure information does not reach a preset failure threshold, and the system load information reaches a preset load threshold.

[0277] Optionally, the preset scoring strategy includes a complete scoring strategy, and the system status information includes system latency information, system failure information, and system load information; the target scoring strategy determination module 908 is further configured to: determine the circuit breaker scoring strategy as the target scoring strategy when the system failure information reaches a preset failure threshold.

[0278] Optionally, the acquisition module 902 is further configured to: acquire scoring token information; the scoring module 910 is further configured to: when the scoring token information is sufficient, score the current context unit based on the target scoring strategy to obtain the target scoring result of the current context unit; when the scoring token information is insufficient, adjust the target scoring strategy, and score the current context unit based on the adjusted target scoring strategy to obtain the target scoring result of the current context unit.

[0279] Optionally, the device also includes an update module configured to: determine the target score result as a historical score result, and update multiple candidate score items based on the historical score result.

[0280] The context unit scoring device provided in the embodiments of this specification constructs a complete closed-loop system from input processing to scoring output through the collaborative work of the acquisition module, the target scoring item determination module, the context scoring determination module, the target scoring strategy determination module, and the scoring module. This avoids ineffective evaluation of low-value or repetitive dialogue content, reduces the cost and resource consumption of calling large language models, and ensures the scoring quality of high-value context units through dynamic strategy selection, thereby improving the accuracy, efficiency, and system stability of context unit scoring as a whole.

[0281] The above is an illustrative scheme of a context unit scoring device according to this embodiment. It should be noted that the technical solution of this context unit scoring device and the technical solution of the above-described context unit scoring method belong to the same concept. For details not described in detail in the technical solution of the context unit scoring device, please refer to the description of the technical solution of the above-described context unit scoring method.

[0282] Figure 10 A structural block diagram of a computing device 1000 according to one embodiment of this specification is shown. The components of the computing device 1000 include, but are not limited to, a memory 1010 and a processor 1020. The processor 1020 is connected to the memory 1010 via a bus 1030, and a database 1050 is used to store data.

[0283] The computing device 1000 also includes an access device 1040, which enables the computing device 1000 to communicate via one or more networks 1060. Examples of these networks include a Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device 1040 may include one or more of any type of wired or wireless network interface (e.g., a network interface controller (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0284] In one embodiment of this specification, the above-described components of the computing device 1000 and Figure 10 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 10 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0285] The computing device 1000 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 1000 can also be a mobile or stationary server.

[0286] The processor 1020 is used to execute the following computer program / instructions, which, when executed by the processor, implement the steps of the above-mentioned context unit scoring method.

[0287] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the above-described context unit scoring method belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the above-described context unit scoring method.

[0288] An embodiment of this specification also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the above-described context unit scoring method.

[0289] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solution of the context unit scoring method described above. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the context unit scoring method described above.

[0290] An embodiment of this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described context unit scoring method.

[0291] The above is an illustrative example of a computer program according to this embodiment. It should be noted that the technical solution of this computer program belongs to the same concept as the technical solution of the aforementioned contextual unit scoring method. Details not described in detail in the computer program's technical solution can be found in the description of the technical solution of the aforementioned contextual unit scoring method.

[0292] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0293] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0294] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0295] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0296] The preferred embodiments disclosed above are merely illustrative of this specification. Optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A method for scoring contextual units, characterized in that, include: Obtain the current context unit and multiple candidate scoring items constructed based on the medical dialogue flow, wherein the multiple candidate scoring items are determined based on the historical scoring results of the historical context unit; Based on the similarity between the current context unit and the plurality of candidate rating items, the target rating item is determined from the plurality of candidate rating items; The context score of the current context unit is determined based on the similarity between the current context unit and the target scoring item. Based on the context score and the preset context score threshold, a target score strategy is determined from the preset score strategies; Based on the target scoring strategy, the current context unit is scored to obtain the target scoring result of the current context unit.

2. The method according to claim 1, characterized in that, The acquisition of the current context unit constructed based on the medical dialogue stream includes: Obtain the medical dialogue stream; Speech recognition is performed on the medical dialogue stream to obtain multiple dialogue text fragments; Based on the semantic information of the dialogue text fragments, the multiple dialogue text fragments are integrated to obtain at least one dialogue statement; If at least one dialogue statement satisfies a preset aggregation condition, the at least one dialogue statement is aggregated to obtain the current context unit.

3. The method according to claim 1, characterized in that, The step of determining the target rating item from the plurality of candidate rating items based on the similarity between the current context unit and the plurality of candidate rating items includes: Feature extraction is performed on the current context unit and the multiple candidate scoring items to obtain context feature representations and multiple scoring item feature representations; Based on the contextual feature representation and the multiple rating item feature representations, the similarity between the current contextual unit and each candidate rating item is determined; Based on multiple similarities, the multiple candidate scoring items are sorted to obtain the scoring item ranking result; Based on the ranking results of the scoring items, the target scoring item is determined.

4. The method according to claim 3, characterized in that, The step of determining the similarity between the current context unit and each candidate rating item based on the context feature representation and the multiple rating item feature representations includes: Based on the contextual feature representation and the multiple rating item feature representations, keyword matching is performed to determine the similarity between the current contextual unit and each candidate rating item; And / or, based on the context feature representation and the multiple rating item feature representations, perform vector similarity calculation to determine the similarity between the current context unit and each candidate rating item.

5. The method according to any one of claims 1-4, characterized in that, The step of determining the target scoring strategy from the preset scoring strategies based on the context score and the preset context score threshold includes: If the context score reaches the preset context score threshold, a target score strategy is determined from the preset score strategies.

6. The method according to claim 5, characterized in that, Before determining the target scoring strategy from the preset scoring strategies when the context score reaches the preset context score threshold, the method further includes: Obtain the system status information of the context unit scoring system; The step of determining a target scoring strategy from preset scoring strategies when the context score reaches the preset context score threshold includes: When the context score reaches the preset context score threshold, a target score strategy is determined from the preset score strategies based on the system status information.

7. The method according to claim 6, characterized in that, The preset scoring strategy includes a complete scoring strategy, and the system status information includes system latency information, system failure information, and system load information; The step of determining the target scoring strategy from the preset scoring strategies based on the system state information includes: If the system latency information does not reach the preset latency threshold, the system failure information does not reach the preset failure threshold, and the system load information does not reach the preset load threshold, then the complete scoring strategy is determined to be the target scoring strategy.

8. The method according to claim 6, characterized in that, The preset scoring strategy includes a lightweight scoring strategy, and the system status information includes system latency information, system failure information, and system load information. The step of determining the target scoring strategy from the preset scoring strategies based on the system state information includes: If the system latency information reaches a preset latency threshold, the system failure information does not reach a preset failure threshold, and the system load information does not reach a preset load threshold, then the lightweight scoring strategy is determined to be the target scoring strategy.

9. The method according to claim 6, characterized in that, The preset scoring strategy includes a waiting scoring strategy, and the system status information includes system delay information, system failure information, and system load information. The step of determining the target scoring strategy from the preset scoring strategies based on the system state information includes: If the system delay information does not reach the preset delay threshold, the system failure information does not reach the preset failure threshold, and the system load information reaches the preset load threshold, then the waiting scoring strategy is determined to be the target scoring strategy.

10. The method according to claim 6, characterized in that, The preset scoring strategy includes a skip scoring strategy, and the system status information includes system latency information, system failure information, and system load information. The step of determining the target scoring strategy from the preset scoring strategies based on the system state information includes: If the system delay information reaches a preset delay threshold, the system failure information does not reach a preset failure threshold, and the system load information reaches a preset load threshold, then the skip scoring strategy is determined to be the target scoring strategy.

11. The method according to claim 6, characterized in that, The preset scoring strategy includes a circuit breaker scoring strategy, and the system status information includes system latency information, system failure information, and system load information. The step of determining the target scoring strategy from the preset scoring strategies based on the system state information includes: If the system failure information reaches a preset failure threshold, the circuit breaker scoring strategy is determined to be the target scoring strategy.

12. The method according to claim 1, characterized in that, Before scoring the current context unit based on the target scoring strategy to obtain the target score result of the current context unit, the method further includes: Retrieve rating token information; The step of scoring the current context unit based on the target scoring strategy to obtain the target scoring result of the current context unit includes: When the scoring token information is in a token-sufficient state, the current context unit is scored based on the target scoring strategy to obtain the target scoring result of the current context unit; If the scoring token information indicates insufficient tokens, the target scoring strategy is adjusted, and the current context unit is scored based on the adjusted target scoring strategy to obtain the target scoring result of the current context unit.

13. The method according to claim 1, characterized in that, After scoring the current context unit based on the target scoring strategy to obtain the target score result of the current context unit, the method further includes: The target score result is determined to be a historical score result, and the multiple candidate score items are updated based on the historical score result.

14. A contextual unit scoring device, characterized in that, include: The acquisition module is configured to acquire the current context unit and multiple candidate scoring items constructed based on the medical dialogue flow, wherein the multiple candidate scoring items are determined based on the historical scoring results of the historical context unit; The target rating item determination module is configured to determine the target rating item from the plurality of candidate rating items based on the similarity between the current context unit and the plurality of candidate rating items; The context scoring determination module is configured to determine the context score of the current context unit based on the similarity between the current context unit and the target scoring item; The target scoring strategy determination module is configured to determine a target scoring strategy from a preset scoring strategy based on the context score and a preset context score threshold. The scoring module is configured to score the current context unit based on the target scoring strategy, and obtain the target scoring result of the current context unit.

15. A computing device, characterized in that, include: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the context unit scoring method according to any one of claims 1 to 13.

16. A computer-readable storage medium, characterized in that, It stores a computer program / instruction that, when executed by a processor, implements the steps of the context unit scoring method according to any one of claims 1 to 13.

17. A computer program product, characterized in that, Includes a computer program / instruction that, when executed by a processor, implements the steps of the context unit scoring method according to any one of claims 1 to 13.

Images