Multi-dimensional real-time doctor-patient communication evaluation analysis method and system

Abstract

The invention provides a multi-dimensional real-time doctor-patient communication evaluation analysis method and system, and the method comprises the steps: collecting the real-time dialogue content between a doctor and an AI simulation patient, and carrying out the processing of the real-time dialogue content, and forming a structured dialogue event sequence; dividing the dialogue event sequence into a plurality of semantic event units, inputting the semantic event units into an intention recognition model, and extracting a semantic tag sequence; constructing a semantic intention trajectory based on the semantic tag sequence, matching the semantic intention trajectory with a predefined dimension scoring model, and generating a three-dimensional preliminary scoring result of treatment universality, polite normalization and humanistic care normalization; obtaining a dynamic weight distribution parameter, carrying out weighted fusion on the dynamic weight distribution parameter and the preliminary scoring result to generate a comprehensive scoring result, and judging whether the comprehensive scoring result is lower than a preset threshold value or not; and if yes, generating a dynamic feedback signal. According to the method, the comprehensive accuracy and situation adaptability of doctor-patient communication evaluation can be improved, and the effect of doctor-patient communication training is improved.

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, and in particular to a multi-dimensional real-time doctor-patient communication assessment and analysis method and system. Background Technology

[0002] Doctor-patient communication is a crucial aspect of healthcare, especially for medical students newly entering clinical practice or inexperienced clinicians. Effective communication with patients is a skill that urgently needs improvement. Traditional doctor-patient communication training relies heavily on real-life case simulations and instructor guidance, often using audio or video recordings of doctor-patient dialogues, followed by manual evaluation by the instructor. This method fails to provide real-time feedback during the communication process, making it difficult for doctors to adjust their communication strategies promptly, resulting in delayed training effectiveness and an inability to effectively correct immediate problems. Furthermore, traditional instructor evaluations often focus on the standardization of communication techniques or the use of polite language, lacking a comprehensive consideration of multiple dimensions such as "the breadth of treatment" and "the standardization of humanistic care." For example, some systems only assess whether doctors use polite titles, failing to consider whether they have adequately inquired about the patient's condition and demonstrated empathy and care within the context of the clinical setting.

[0003] Therefore, it is necessary to improve traditional doctor-patient communication training methods to overcome the shortcomings of existing technologies. Summary of the Invention

[0004] To overcome the problems existing in related technologies, one of the objectives of this invention is to provide a multi-dimensional real-time doctor-patient communication assessment and analysis method. This method can dynamically and in real-time evaluate doctor-patient communication methods in multiple dimensions, thereby improving the overall accuracy and situational adaptability of doctor-patient communication assessment and enhancing the effectiveness of doctor-patient communication training.

[0005] A multi-dimensional real-time doctor-patient communication assessment and analysis method includes: Collect real-time dialogue between doctors and AI-simulated patients, and process the collected content to form a structured sequence of dialogue events; The dialogue event sequence is divided into multiple semantic event units. Each semantic event unit is input into the intent recognition model to extract semantic label sequences including medical information, polite language, and expressions of humanistic care. A semantic intent trajectory is constructed based on the semantic label sequence. The semantic intent trajectory is matched with a predefined dimensional scoring model to generate preliminary scoring results in three dimensions: treatment breadth, politeness norms, and humanistic care norms. Obtain dynamic weight allocation parameters, weight and fuse the dynamic weight allocation parameters with the preliminary scoring results to generate a comprehensive scoring result, and determine whether it is lower than the preset threshold. If so, a dynamic feedback signal is generated.

[0006] In a preferred embodiment of the present invention, obtaining the dynamic weight allocation parameters includes: Contextual complexity analysis is performed on the semantic event units of the current dialogue to identify the linguistic structural complexity of the dialogue content; Emotional fluctuation modeling is performed on the voice emotion recognition results of AI-simulated patients. The patient's emotional fluctuation index is calculated based on the emotional polarity change rate of the time series to quantify the dynamic change amplitude of the patient's emotions in the current conversation. Based on the knowledge graph of the importance of treatment topics, the importance of diagnosis and treatment keywords in the current semantic event is scored to obtain the importance factor of treatment topics, which is used to reflect the criticality of the current dialogue content in the diagnosis and treatment process. The context complexity index, patient emotional fluctuation index, and treatment topic importance factor are used as input parameters and fed into a reinforcement learning-based dynamic weight adjustment model. The initial weight allocation vector of the three-dimensional score is generated through a multi-factor fusion decision algorithm. The initial weight allocation vector is normalized and biased by combining historical dialogue weight adjustment records to obtain the final dynamic weight allocation parameters for the three-dimensional scores.

[0007] In a preferred embodiment of the present invention, the step of collecting real-time dialogue content between the doctor and the AI-simulated patient, and processing the collected content to form a structured dialogue event sequence, includes: The system collects real-time dialogue content between doctors and AI-simulated patients, including voice signals and text transcription data, and records dialogue timestamp information. The speech signal is denoised and processed for speech recognition to generate a text sequence, which is then semantically aligned with the original text data to form a structured dialogue event sequence.

[0008] In a preferred embodiment of the present invention, dividing the dialogue event sequence into multiple semantic event units includes: The dialogue was initially segmented using a sliding window segmentation algorithm based on semantic coherence metrics; The semantic similarity between adjacent segments is calculated using the BERT semantic embedding model to determine the segmentation point; The semantic event units are length normalized and encapsulated into a structured data format.

[0009] In a preferred embodiment of the present invention, the intent recognition model includes: The semantic event units are preprocessed to generate standardized semantic input text; Generate high-dimensional semantic vector sequences using a pre-trained language model; Temporal semantic features are extracted using a Bi-GRU network, and global semantic representations are generated by combining an attention mechanism. The semantic label sequence is output through a fully connected classification layer.

[0010] In a preferred embodiment of the present invention, the step of constructing a semantic intent trajectory based on a semantic tag sequence includes: Constructing a semantic intent temporal evolution graph based on a sliding window; Density clustering algorithm is used to analyze the distribution density of intent tags; Use linear regression and change point detection techniques to identify trends in intent; A dynamic distribution map of the focus of attention is generated by combining the context offset judgment rule base.

[0011] In a preferred embodiment of the present invention, the step of matching the semantic intent trajectory with a predefined dimensional scoring model to generate preliminary scoring results in three dimensions—the breadth of treatment, politeness normativity, and humanistic care normativity—includes: Obtain the semantic intent trajectory generated from the doctor-patient dialogue sequence. The semantic intent trajectory contains the distribution information of three types of intent labels in multiple semantic event units: diagnosis and treatment information, polite language, and expressions of humanistic care. Based on the semantic intent trajectory, calculate the intent density distribution matrix of the three types of intent tags in each semantic event unit; The intent density distribution matrix is ​​matched with a predefined dimensional scoring model, and preliminary scoring results for the three dimensions of treatment breadth, politeness norms, and humanistic care norms are generated through the preset mapping function in the dimensional scoring model.

[0012] In a preferred embodiment of the present invention, generating the dynamic feedback signal includes: Corrective suggestions are generated by matching quantitative indicators of scoring deviation with a feedback strategy rule base. Generate visual chart data based on a scoring trend analysis model; The output is sent to the doctor's terminal interface through the human-computer interaction interface.

[0013] The second objective of this invention is to provide a multi-dimensional real-time doctor-patient communication assessment and analysis system, characterized in that it is used to implement the multi-dimensional real-time doctor-patient communication assessment and analysis method described above; The system includes: The data acquisition module is used to acquire voice and text dialogue data; A speech recognition and alignment module, wherein the speech recognition and alignment module is used to generate a structured dialogue event sequence; A semantic event segmentation module, which is used to segment semantic event units; An intent recognition module is used to extract semantic label sequences; A semantic trajectory construction module is used to construct semantic intent trajectories and analyze intent density and trends; The dimension scoring module is used to generate preliminary scores for the breadth of treatment, politeness norms, and humanistic care norms. A weight adjustment module, which is used to generate dynamic weight allocation parameters; The scoring fusion and feedback module is used to generate a comprehensive score and dynamic feedback signal.

[0014] The beneficial effects of this invention are as follows: This invention provides a multi-dimensional real-time doctor-patient communication assessment and analysis method. The method includes: collecting real-time dialogue content between a doctor and an AI-simulated patient, processing the collected content to form a structured dialogue event sequence; dividing the dialogue event sequence into multiple semantic event units, inputting each semantic event unit into an intent recognition model, and extracting semantic label sequences including medical information, polite language, and expressions of humanistic care; constructing a semantic intent trajectory based on the semantic label sequences, matching the semantic intent trajectory with a predefined dimensional scoring model to generate preliminary scoring results for three dimensions: treatment breadth, politeness standardization, and humanistic care standardization; obtaining dynamic weight allocation parameters, weighting and fusing the dynamic weight allocation parameters with the preliminary scoring results to generate a comprehensive scoring result, and determining whether it is below a preset threshold; if so, generating a dynamic feedback signal. In practical use, this method can generate real-time feedback during the interaction between the doctor and the AI-simulated patient. Through multi-dimensional preliminary scoring and dynamic weight adjustment, trained doctors can understand their shortcomings in patient communication in real time, changing the traditional static scoring model. This method can also dynamically adjust the weights of each dimension by obtaining dynamic weight allocation parameters. This effectively solves the problems of distorted and insensitive scoring under fixed weights in different dialogue scenarios, making the scoring results more realistic and accurate in reflecting the doctor's actual communication level. This immediate and precise training method can effectively accelerate the doctor's learning curve and improve the efficiency and effectiveness of communication training.

[0015] This application also provides a system for implementing the above-mentioned multi-dimensional real-time doctor-patient communication assessment and analysis method. The system can simulate patients in real time with the help of AI, evaluate the doctor-patient communication structure in real time, and dynamically adjust the weight parameters during the evaluation process to adjust the comprehensive score results, so that the score results are closer to the actual situation, thereby helping to improve the training effect of doctors. Attached Figure Description

[0016] Figure 1 This is a flowchart of a multi-dimensional real-time doctor-patient communication assessment and analysis method provided in an embodiment of the present invention. Detailed Implementation

[0017] Preferred embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0018] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms “a,” “the,” and “the” used in this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0019] It should be understood that although the terms "first," "second," "third," etc., may be used in this invention to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this invention, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Thus, features defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0020] Doctor-patient communication is a crucial aspect of healthcare, especially for medical students newly entering clinical practice or inexperienced clinicians. Effective communication with patients is a skill that urgently needs improvement. Traditional doctor-patient communication training relies heavily on real-life case simulations and instructor guidance, often using audio or video recordings of doctor-patient dialogues, followed by manual evaluation by the instructor. This method fails to provide real-time feedback during the communication process, making it difficult for doctors to adjust their communication strategies promptly, resulting in delayed training effectiveness and an inability to effectively correct immediate problems. Furthermore, traditional instructor evaluations often focus on the standardization of communication techniques or the use of polite language, lacking a comprehensive consideration of multiple dimensions such as "the breadth of treatment" and "the standardization of humanistic care." For example, some systems only assess whether doctors use polite titles, failing to consider whether they have adequately inquired about the patient's condition and demonstrated empathy and care within the context of the clinical setting.

[0021] Based on this, this application provides a multi-dimensional real-time doctor-patient communication assessment and analysis method.

[0022] Example The embodiments provided are as follows: Figure 1As shown, this application provides a multi-dimensional real-time doctor-patient communication assessment and analysis method, including the following steps: S100: Collect real-time dialogue content between doctors and AI-simulated patients, and process the collected content to form a structured dialogue event sequence. S200. Divide the dialogue event sequence into multiple semantic event units, input each semantic event unit into the intent recognition model, and extract semantic label sequences including medical information, polite language, and expressions of humanistic care. S300. Construct a semantic intent trajectory based on the semantic label sequence, match the semantic intent trajectory with a predefined dimensional scoring model, and generate preliminary scoring results for three dimensions: treatment breadth, politeness norms, and humanistic care norms. S400. Obtain the dynamic weight allocation parameters, weight and fuse the dynamic weight allocation parameters with the preliminary scoring results to generate a comprehensive scoring result, and determine whether it is lower than the preset threshold. S500, if so, generates a dynamic feedback signal.

[0023] Specifically, the implementation process of this method is as follows: The system collects real-time conversations between doctors and AI-simulated patients, including voice signals and text transcription data, and records conversation timestamps simultaneously.

[0024] This step collects real-time dialogue content between doctors and AI-simulated patients, including voice signals and text transcription data, and records dialogue timestamp information simultaneously, providing basic data input for subsequent semantic event segmentation, intent recognition, and multi-dimensional scoring.

[0025] Multi-channel data acquisition was performed on the interaction process between doctors and AI-simulated patients to obtain speech signals and corresponding text transcription data in order to construct a structured set of original dialogue inputs.

[0026] The acquired speech signals and text transcription data are timestamped based on a time synchronization protocol to ensure consistency between the two in the time dimension and obtain a synchronized dialogue data stream.

[0027] A speech activity detection algorithm is used to detect endpoints of speech signals in synchronized dialogue data streams, identify the start and end boundaries of speech, and thus divide them into independent speech segment units.

[0028] Semantic boundary alignment is performed on the transcribed text based on speech segment units to generate corresponding text semantic units, forming a semantically consistent set of dialogue segments. Alignment processing based on a time synchronization protocol ensures that each speech sentence and its corresponding text are perfectly matched on the timeline. This step eliminates contextual confusion caused by audio-text asynchrony, allowing subsequent semantic event segmentation, intent density calculation, and context offset recognition to be based on real-time relationships, greatly improving the accuracy of the system's "context-aware" capabilities.

[0029] By integrating a set of semantically consistent dialogue fragments with their corresponding timestamps, a structured dialogue event sequence with timestamps is generated, which serves as the input basis for subsequent context segmentation and intent recognition.

[0030] By simultaneously acquiring speech signals and text transcription data, the system obtained complementary bimodal information. Speech contains paralinguistic information (such as intonation, pauses, and emotion) that text cannot cover, while text provides precise semantic content. This multi-channel data acquisition ensures the integrity and richness of the raw information, laying an irreplaceable data foundation for subsequent in-depth semantic and sentiment analysis.

[0031] The resulting "time-stamped structured dialogue event sequence" not only contains information about "who said what and when," but more importantly, it naturally records the evolution of the dialogue. This temporal structure is the direct input for constructing the "semantic intent trajectory" (i.e., a graph of intent density changing over time). This allows the method to leap from static "dialogue content" analysis to dynamic "communication process" analysis. Only based on this temporal and structured data can the system calculate the changing trends of intent and identify the shift in the doctor's focus, thereby achieving true dynamic evaluation, rather than a static and general assessment of the entire dialogue.

[0032] The acquired speech signals are denoised and processed for speech recognition to generate a text sequence, which is then semantically aligned with the original text data to form a structured dialogue event sequence.

[0033] This step performs noise reduction and speech recognition processing on the acquired speech signals to generate a text sequence, and then semantically aligns it with the original text data to form a structured dialogue event sequence. It plays a crucial role in the overall technical solution by transforming raw speech data into analyzable text data, providing structured input for subsequent intent recognition and semantic trajectory modeling. Specifically, it includes: The acquired raw audio signal is subjected to wavelet transform-based noise reduction processing to remove environmental background noise and non-speech interference components, and a clean speech signal after noise reduction is obtained.

[0034] Based on the denoised clean speech signal, a speech-to-text transcription process is performed using a speech recognition model based on a deep recurrent neural network (RNN-Transducer) to generate a preliminary text sequence.

[0035] The initial text sequence is subjected to contextual semantic correction processing based on the BERT language model to correct misrecognition caused by homophones or ambiguous context during speech recognition, thereby obtaining a semantically enhanced text sequence.

[0036] Specifically, BERT's core task during the pre-training phase is cloze test (Masked Language Model, MLM). During training, it randomly "masks" some words in a sentence (turning them into "[MASK]"), and then asks the model to predict what the masked word is based on all the words to its left and right.

[0037] In the application of this method, the text transcribed by speech recognition (ASR) may contain errors, such as misrecognizing "he suffers from gastritis" as "he suffers from pharyngitis".

[0038] The system inputs the erroneous sentence into a BERT model fine-tuned with medical text. By analyzing the context of the entire sentence ("he", "has", "a disease"), BERT calculates that the probability of the word "gastritis" appearing in the current position is much higher than the coined word "fear inflammation". Therefore, it can automatically correct the text to the correct "gastritis".

[0039] This correction is not based on simple dictionary matching, but on a deep understanding of the overall semantics, thus enabling it to handle more complex ambiguities.

[0040] Based on the original text data and the semantically enhanced text sequence, the Dynamic Time Warping (DTW) algorithm is used to perform semantic alignment processing to eliminate the time offset difference between speech and text, and obtain a time-aligned text sequence.

[0041] In real-time conversations between doctors and AI-simulated patients, the system uses an Automatic Speech Recognition (ASR) engine to convert the doctor's speech into a preliminary text sequence. However, due to the time required for ASR processing, network latency, and the buffering mechanism of the ASR engine itself, a non-linear time offset occurs between the recognized text stream and the original, strictly timestamped text transcription data (i.e., the AI ​​patient's questions and the pre-set doctor's microphone sequence). This distortion prevents the system from associating ASR-recognized keywords (such as "penicillin allergy") with precise moments in the conversation stream, thus compromising the accuracy of subsequent semantic event segmentation and intent trajectory modeling.

[0042] This step performs a fine alignment between the text sequence with time distortion generated by ASR and the original text sequence with high-precision timestamps, assigns correct timestamps to each word or phrase in the ASR text, and thus generates a "time-aligned text sequence".

[0043] The specific process is as follows: The original text data sequence. Each element of this sequence is a word or a short phrase, and each element is accompanied by high-precision start and end timestamps obtained from the interaction log. For example: R[1]={"text":"Hello","start_time":1000,"end_time":1200} R[2]={"text":"Excuse me","start_time":1205,"end_time":1350} R[3]={"text":"Where","start_time":1355,"end_time":1450} ...... The preliminary text sequence generated after ASR recognition. The words in this sequence are distorted and distorted in time, and may only have rough time information for the entire sentence, or no reliable time information at all.

[0044] T[1]="You" T[2]="Good" T[3]="Please" T[4]="Ask" T[5]="Where" Convert each text element in the two sequences into a comparable numerical feature. In this embodiment, word vector technology is used.

[0045] Using a pre-trained word embedding model (such as the initial layer of Word2Vec or BERT), map each word to a vector in a high-dimensional space (for example, a 300-dimensional vector).

[0046] For short phrases, the average value of all word vectors within the sentence can be calculated as the feature vector of the sentence.

[0047] Calculate the distance matrix: Construct a matrix D of size len(T)xlen(R). Each element D[i,j] in the matrix represents the "distance" between the i-th word in the test sequence and the j-th word in the reference sequence.

[0048] The calculation of distance usually uses cosine distance or Euclidean distance. For example, cosine distance: The smaller the distance, the more similar the two words are semantically.

[0049] Then the DTW algorithm is executed to find the optimal curved path.

[0050] The goal of the DTW algorithm is to find a warping path W = w1, w2, ..., wk that runs through the distance matrix D, where wk = (i, j), such that the sum of the distances between all points on this path is minimized.

[0051] This path must start from (1,1) and end at (len(T),len(R)).

[0052] The path must be monotonically increasing, meaning it cannot go back.

[0053] The path is continuous.

[0054] Dynamic programming solution: Fill a cumulative cost matrix C using dynamic programming, where C[i,j] represents the minimum cumulative distance from (1,1) to (i,j).

[0055] The recursive formula is usually: C[i,j]=D[i,j]+min(C[i-1,j],C[i,j-1],C[i-1,j-1]) Starting from C[len(T),len(R)], backtrack backwards to find the path that minimizes the cumulative cost, which is the optimal curved path W.

[0056] Each point (i,j) on the curved path W explicitly indicates that "the i-th word in the test sequence T should be aligned with the j-th word in the reference sequence R".

[0057] Based on this mapping relationship, the start and end timestamps of the j-th word in the reference sequence R are "assigned" to the i-th word in the test sequence T.

[0058] For example, if the path contains (5,3), that is, T[5] (“where”) is aligned with R[3] (“where”), then T[5] is assigned start_time=1355 and end_time=1450.

[0059] Traverse the entire curved path, assign a precise timestamp to each word in the ASR test sequence T, and finally generate a time-aligned text sequence.

[0060] Perform structured encapsulation processing based on dialogue role labels on the time-aligned text sequence and the original text data to generate a structured dialogue event sequence with timestamps, speaker identifiers and text content.

[0061] The aligned dialogue event sequence is divided into multiple semantic event units, each of which contains a continuous dialogue segment and a corresponding time interval.

[0062] This step divides the aligned dialogue event sequence into multiple semantic event units. Each semantic event unit contains a continuous dialogue segment and a corresponding time interval to support fine-grained processing of subsequent intent recognition and semantic trajectory modeling, thereby improving context awareness and evaluation accuracy. The specific implementation steps are as follows: Based on the aligned structured dialogue event sequence, a sliding window segmentation algorithm based on semantic coherence measurement is used to initially segment continuous dialogue segments to obtain candidate semantic event boundaries.

[0063] Semantic consistency is verified for the boundaries of candidate semantic events. The semantic similarity between adjacent segments is calculated using the BERT semantic embedding model. If the similarity is lower than a set threshold, it is taken as the formal segmentation point of the semantic event.

[0064] Based on the segmentation points, the dialogue event sequence is divided into semantic event units, generating multiple semantically independent dialogue segment units, and each unit is labeled with a start and end timestamp.

[0065] Each semantic event unit is length normalized, and a dynamic time warping algorithm is used to align the dialogue content within the unit to eliminate the impact of speech rate differences on subsequent intent recognition.

[0066] The normalized semantic event units are encapsulated into a structured data format, including text content, time interval, speaker identifier, and semantic embedding vector, for use as input to the subsequent intent recognition model.

[0067] Each semantic event unit is input into the intent recognition model to extract the corresponding semantic label sequence. The semantic labels include three core intent labels: medical information, polite language, and expressions of humanistic care.

[0068] This step inputs each semantic event unit into the intent recognition model to extract the corresponding semantic label sequence. The semantic labels include three core intent labels: medical information, polite language, and expressions of humanistic care. This achieves a semantically structured expression of the dialogue content, providing basic semantic recognition support for subsequent semantic intent trajectory construction and dimensional scoring. The implementation process of this step is as follows: Preprocessing operations are performed on the text content in the segmented semantic event units, including stop word removal, standardization of expressions, and part-of-speech tagging, to obtain standardized semantic input text.

[0069] Pre-trained domain-adaptive language models are used to embed and represent normalized semantic input text, generating high-dimensional semantic vector sequences to capture the contextual semantic relationships between words in semantic event units.

[0070] The high-dimensional semantic vector sequence is input into a multi-layer bidirectional gated recurrent unit (Bi-GRU) to extract temporal semantic features from the semantic event unit in order to obtain a context-aware semantic feature sequence.

[0071] The attention mechanism is used to perform weighted fusion of context-aware semantic feature sequences to generate a global semantic representation vector of semantic event units, thereby strengthening the expression weight of key intent information.

[0072] The global semantic representation vector is input into the fully connected classification layer, and the classification decision is performed by combining the Softmax activation function. The semantic label sequence corresponding to the semantic event unit is output. The semantic labels include three core intent labels: diagnosis and treatment information, polite language, and humanistic care expression.

[0073] Based on semantic tag sequences, semantic intent trajectories are constructed, and the distribution density and changing trend of each intent tag in each semantic event are calculated to identify context shifts and changes in the focus of attention expressed by doctors.

[0074] This step constructs a semantic intent trajectory based on the semantic tag sequence, calculates the distribution density and changing trend of each intent tag in each semantic event, and identifies contextual shifts and changes in the doctor's focus, thereby providing context-aware evidence for subsequent dimensional scoring models. The specific steps are as follows: The input semantic label sequence is processed by time series modeling, and a time evolution map of semantic intent trajectory is constructed based on the sliding window algorithm to obtain the time series distribution characteristics of semantic intent.

[0075] Based on the time-series distribution characteristics of semantic intent trajectories, a density clustering algorithm is used to perform clustering analysis on the frequency of occurrence of each intent tag in different time windows in order to identify the density distribution pattern of each intent in the dialogue process.

[0076] Based on the density distribution pattern, the distribution density index of each intent tag in the semantic event unit is calculated, including the frequency of intent occurrence, the proportion of duration, and the rate of change of intensity, in order to quantify the concentration and persistence of intent expression.

[0077] A trend detection algorithm is applied to the distribution density index, and linear regression and change point detection techniques are used to identify the rising, falling or stable trend of intent density over time, so as to obtain the characteristics of intent change trend.

[0078] Based on the trend characteristics of intent change and the pre-set rule base for determining context shift, the system identifies whether doctors have context shifts in their conversations, such as a general shift in treatment, a lack of politeness norms, or insufficient expression of humanistic care, and generates context shift identification signals.

[0079] By combining context offset identification signals and intent density distribution, an attention mechanism model is used to calculate the changes in the attention weight of doctors to each intent dimension during the dialogue process, generating a dynamic distribution map of attention center.

[0080] The semantic intent trajectory is matched with a predefined dimensional scoring model to generate preliminary scoring results for three dimensions: treatment breadth, politeness norms, and humanistic care norms.

[0081] This step, based on the mapping relationship between semantic intent trajectories and dimensional scoring models, generates preliminary scoring results for three dimensions: therapeutic breadth, politeness norms, and humanistic care norms. This provides the foundational input for subsequent dynamic weight adjustments and comprehensive score fusion, and is a core step in achieving multi-dimensional real-time communication assessment. The specific details of this step are as follows: The semantic intent trajectory data is processed to calculate intent density, so as to obtain the intent density distribution matrix of diagnosis and treatment information, polite language and humanistic care expression in each semantic event, which is used as the input feature vector of the dimensional scoring model.

[0082] Based on the intent density distribution matrix, a predefined dimensional scoring mapping function is used to perform treatment breadth scoring on the diagnosis and treatment information density to generate preliminary score values ​​for the treatment breadth dimension and output the treatment breadth score vector.

[0083] Based on the intent density distribution matrix, a predefined dimension scoring mapping function is used to perform politeness normativity scoring on politeness use density to generate preliminary scores for politeness normativity dimensions and output politeness normativity score vectors.

[0084] Based on the intent density distribution matrix, a predefined dimension scoring mapping function is used to perform humanistic care expression density calculation on humanistic care normative scoring to generate preliminary score values ​​for humanistic care normative dimensions and output humanistic care normative score vector.

[0085] The treatment breadth rating vector, politeness normativity rating vector, and humanistic care normativity rating vector are dimensionally aligned and standardized to generate a unified set of preliminary three-dimensional rating results, which serve as input rating data for the adaptive weight adjustment model.

[0086] Based on the contextual complexity of the current dialogue, the patient's emotional fluctuation index, and the importance factor of the treatment topic, the data are input into the adaptive weight adjustment model to generate dynamic weight allocation parameters for the three-dimensional score.

[0087] This step, based on the contextual complexity of the current dialogue, the patient's emotional fluctuation index, and the importance factor of the treatment topic, inputs these factors into an adaptive weight adjustment model to generate dynamic weight allocation parameters for the three-dimensional score. Its purpose is to achieve real-time dynamic adjustment of the weights of the scoring dimensions through a context-aware mechanism, thereby improving the accuracy and context adaptability of the multi-dimensional score. Specifically, this step is implemented as follows: Contextual complexity analysis is performed on the semantic event units of the current dialogue, and a syntactic complexity index is calculated based on the syntactic structure complexity evaluation model to identify the linguistic structural complexity of the dialogue content.

[0088] Emotional fluctuation modeling is performed on the voice emotion recognition results of AI-simulated patients. The patient's emotional fluctuation index is calculated based on the emotional polarity change rate of the time series to quantify the dynamic change amplitude of the patient's emotions in the current conversation.

[0089] Based on the knowledge graph of the importance of treatment topics, the importance of diagnosis and treatment keywords in the current semantic event is scored to obtain the importance factor of treatment topics, which is used to reflect the criticality of the current dialogue content in the diagnosis and treatment process.

[0090] The context complexity index, patient emotional fluctuation index, and treatment topic importance factor are used as input parameters and fed into a reinforcement learning-based dynamic weight adjustment model. The initial weight allocation vector of the three-dimensional score is generated through a multi-factor fusion decision algorithm.

[0091] The initial weight allocation vector is normalized and biased by combining historical dialogue weight adjustment records to obtain the final dynamic weight allocation parameters for the three-dimensional scores.

[0092] The dynamic weight allocation parameters are weighted and fused with the preliminary scoring results to generate a comprehensive scoring result, and it is determined whether the comprehensive score is lower than a preset threshold.

[0093] This step involves weighting and fusing the dynamic weight allocation parameters with the preliminary scoring results to generate a comprehensive score. It then determines whether this score falls below a preset threshold, providing a trigger for the subsequent feedback control module and ensuring the real-time nature and accuracy of the evaluation results. The specific steps are as follows: The preliminary scores for the three dimensions of treatment breadth, courtesy and norms, and humanistic care norms were normalized to eliminate the influence of different scoring scales on the comprehensive score and obtain a standardized dimensional score vector.

[0094] Based on contextual complexity, patient emotional fluctuation index, and importance factor of treatment topic, a dynamic weight allocation model is executed to calculate the weight allocation coefficients of the three-dimensional scores in the current dialogue context, and generate a dynamic weight vector.

[0095] The standardized dimensional score vector and the dynamic weight vector are multiplied by a dot product to achieve weighted fusion of multidimensional scores and generate a weighted comprehensive score value.

[0096] A threshold judgment algorithm is applied to the weighted comprehensive score to determine whether it is lower than the preset feedback trigger threshold, and a binary feedback trigger signal is generated.

[0097] Based on the weighted comprehensive score and the feedback trigger signal, a structured score output data packet is generated. The data packet includes scores for each dimension, dynamic weight coefficients, comprehensive score, and feedback trigger status, which is used to drive the feedback control module to execute the next operation.

[0098] If the overall score is determined to be lower than a preset threshold, a dynamic feedback signal is generated. The feedback signal includes prompts, corrective suggestions, or a visual chart of the scoring trend.

[0099] This step, during the interaction between the doctor and the AI-simulated patient, determines whether to generate dynamic feedback signals based on multi-dimensional scoring results. This allows for real-time intervention and guidance of the quality of doctor-patient communication, thereby improving the doctor's communication skills and overall diagnostic and treatment competence. The specific steps are as follows: Based on the weighted comprehensive score results of three dimensions—the breadth of treatment, the standardization of politeness, and the standardization of humanistic care—a scoring threshold determination algorithm is executed to classify the communication quality level of the current dialogue segment and obtain a communication quality status identifier.

[0100] If the communication quality status identifier indicates that the overall score is lower than the preset threshold, the difference vector between the current score and the benchmark score is calculated based on the score deviation calculation module to generate a score deviation quantification index.

[0101] The scoring deviation quantification index is matched with the feedback strategy rule base, and the feedback signal generation engine is invoked to generate a text feedback signal containing specific correction suggestions. The feedback suggestions are generated based on semantic matching between historical high-scoring dialogue patterns and the current semantic intent trajectory.

[0102] The rating trend analysis model is used to fit the rating change curve of the current dialogue to generate a rating trend visualization chart. The chart data includes the rating change slope, local extreme points, and trend prediction range.

[0103] By integrating text feedback signals with rating trend visualization charts, and outputting the data to the doctor's terminal interface through the human-computer interaction interface module, dynamic prompts and visual guidance for the doctor's communication behavior can be achieved.

[0104] Finally, the scoring results, weighting parameters, feedback signals, and corresponding semantic event information can be stored in the database for subsequent model optimization and evaluation criterion updates.

[0105] This step stores the scoring results, weight allocation parameters, feedback signals, and corresponding semantic event information in the database for subsequent model optimization and evaluation criterion updates, ensuring the system has continuous learning and dynamic adaptation capabilities. The specific steps are as follows: The generated comprehensive scoring results, dynamic weight allocation parameters, feedback signal content, and associated semantic event information are structurally encapsulated to form a standardized data record format.

[0106] Based on a pre-defined data classification strategy, structured data records are labeled with multi-dimensional tags according to evaluation dimensions, dialogue session IDs, timestamps, and doctor identifiers to achieve efficient data retrieval and classification management.

[0107] The labeled data records are written to the evaluation record table and feedback log table in the distributed time series database through the database transaction interface to ensure the integrity and consistency of the evaluation data.

[0108] Data quality verification is performed on the evaluation records written to the database. Abnormal records are detected based on integrity constraint rules and data consistency verification algorithms to generate data cleaning and repair suggestions.

[0109] Based on historical evaluation records and feedback data, an incremental model training dataset is constructed and updated to the model training repository through a data version control mechanism to support the continuous optimization of the multidimensional evaluation module and the adaptive updating of the scoring criteria.

[0110] Example 2 This embodiment provides a multi-dimensional real-time doctor-patient communication assessment and analysis system, which is used to implement the multi-dimensional real-time doctor-patient communication assessment and analysis method described above. The system includes: The data acquisition module is used to acquire voice and text dialogue data; A speech recognition and alignment module, wherein the speech recognition and alignment module is used to generate a structured dialogue event sequence; A semantic event segmentation module, which is used to segment semantic event units; An intent recognition module is used to extract semantic label sequences; A semantic trajectory construction module is used to construct semantic intent trajectories and analyze intent density and trends; The dimension scoring module is used to generate preliminary scores for the breadth of treatment, politeness norms, and humanistic care norms. A weight adjustment module, which is used to generate dynamic weight allocation parameters; The scoring fusion and feedback module is used to generate a comprehensive score and dynamic feedback signal.

[0111] Specifically, the data acquisition module includes a high-fidelity microphone array, an audio codec, and a data interface for communicating with the AI ​​simulated patient platform. This module is used to capture the real-time speech signals of the dialogue between the doctor and the AI ​​simulated patient and convert them into a digital audio stream. It simultaneously receives interaction logs from the AI ​​simulated patient platform and obtains the corresponding text transcription data. It runs a time synchronization service, assigning a unified and accurate timestamp to each piece of speech and text data based on the Network Time Protocol (NTP). It can also output a multimodal raw data stream with synchronized timestamps.

[0112] The speech recognition and alignment module includes a speech recognition engine that integrates an ASR model based on an RNN-Transducer or similar advanced architecture to convert the input speech signal into a preliminary text sequence.

[0113] It also includes a semantic corrector and a dynamic time warping aligner. Specifically, the semantic corrector has a built-in BERT language model fine-tuned for medical texts. It performs contextual understanding and error correction on the initial text output by ASR, corrects homophones and recognition errors, generates a semantically enhanced text sequence, executes the DTW algorithm, and performs fine-grained time alignment between the semantically enhanced text sequence and the original text transcription data to eliminate non-linear time shifts.

[0114] The semantic event segmentation module includes a semantic coherence analyzer: using a pre-trained semantic embedding model (such as Sentence-BERT) to calculate the semantic similarity between adjacent sentences or segments in a dialogue sequence.

[0115] It also includes a boundary decision maker, which uses a sliding window technique to traverse the structured dialogue event sequence. When the semantic similarity between adjacent segments within the window is lower than a preset threshold, the boundary of the semantic event is determined.

[0116] The dimension scoring module includes the following components: Treatment Breadth Rating Analyzer: Receives a density index of intent to provide treatment information and outputs a preliminary score for that dimension using predefined rules or a regression model. Politeness Normativity Rating Analyzer: Receives a density index of intent to use polite language and outputs a preliminary score for the corresponding dimension. Humanistic Care Normativity Rating Analyzer: Receives a density index of intent to express humanistic care and outputs a preliminary score for the corresponding dimension.

[0117] The weight adjustment module incorporates a dynamic weight adjustment model trained using reinforcement learning. This model receives three contextual factors: a context complexity analysis unit, a sentiment fluctuation analysis unit, and a topic importance analysis unit. It then uses a neural network to perform multi-factor fusion decision-making, generating initial weight allocation vectors for the three-dimensional scores. The context complexity analysis unit calculates the syntactic complexity and term density of the current dialogue. The emotion fluctuation analysis unit calculates the emotion fluctuation index based on the AI ​​patient's speech and text. The topic importance analysis unit queries the medical knowledge graph to obtain the importance factor of the current diagnosis and treatment topic.

[0118] The scoring fusion and feedback module includes a fusion unit, a decision maker, and a feedback generator. The fusion unit weights and fuses the preliminary score output by the dimension scoring module with the dynamic weight parameters output by the weight adjustment module to calculate a comprehensive score. The decision unit compares the comprehensive score with a preset feedback trigger threshold and generates a binarized trigger signal.

[0119] If the trigger signal is true, the feedback generator will retrieve the corresponding text prompts and correction suggestions from the feedback strategy rule base based on the scoring deviation, and generate a scoring trend visualization chart.

[0120] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of this application. It should also be understood that, for ease of description, the dimensions of the various parts shown in the drawings are not drawn to actual scale. Techniques, methods, and devices known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and devices should be considered part of the specification. In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters in the following drawings denote similar items; therefore, once an item is defined in one drawing, it need not be further discussed in subsequent drawings. In the description of this application, it should be understood that the orientation or positional relationship indicated by directional terms such as "front, back, up, down, left, right", "horizontal, vertical, horizontal" and "top, bottom" is usually based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing this application and simplifying the description. Unless otherwise stated, these directional terms do not indicate or imply that the device or element referred to must have a specific orientation or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on the scope of protection of this application; the directional terms "inner" and "outer" refer to the inner and outer contours relative to the outline of each component itself.

[0121] For ease of description, spatial relative terms such as "above," "on top of," "on the upper surface of," "above," etc., are used herein to describe the spatial positional relationship of a device or feature as shown in the figures to other devices or features. It should be understood that spatial relative terms are intended to encompass different orientations in use or operation beyond the orientation of the device as described in the figures. For example, if the device in the figures were inverted, a device described as "above" or "on top of" other devices or structures would subsequently be positioned as "below" or "under" other devices or structures. Thus, the exemplary term "above" can include both "above" and "below." The device may also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatial relative descriptions used herein will be interpreted accordingly.

[0122] Furthermore, it should be noted that the use of terms such as "first" and "second" to define components is merely for the purpose of distinguishing the corresponding components. Unless otherwise stated, these terms have no special meaning and therefore should not be construed as limiting the scope of protection of this application. The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-dimensional real-time doctor-patient communication evaluation and analysis method, characterized in that, The method comprises the following steps: Collecting real-time conversation content between the doctor and the AI simulated patient, processing the collected content, and forming a structured conversation event sequence; Divide the conversation event sequence into multiple semantic event units, input each semantic event unit into an intent recognition model, and extract a semantic tag sequence including diagnosis and treatment information, polite language, and expression of humanistic care; Based on the semantic tag sequence, construct a semantic intent trajectory, match the semantic intent trajectory with a pre-defined dimension score model, and generate preliminary score results of three dimensions of treatment comprehensiveness, polite normativity, and humanistic care normativity; Obtain dynamic weight allocation parameters, weight and integrate the dynamic weight allocation parameters and the preliminary score results to generate a comprehensive score result, and determine whether it is lower than a pre-set threshold; If yes, generate a dynamic feedback signal.

2. The multi-dimensional real-time doctor-patient communication evaluation and analysis method according to claim 1, wherein: The dynamic weight allocation parameters are obtained by: Performing context complexity analysis on the context semantic event unit of the current conversation to identify the language structure complexity of the conversation content; Performing emotion fluctuation modeling on the voice emotion recognition result of the AI simulated patient, calculating the patient emotion fluctuation index based on the time series emotion polarity change rate, and quantifying the dynamic change amplitude of the patient emotion in the current conversation; Based on the treatment topic importance knowledge graph, the importance score of the diagnosis and treatment keywords in the current semantic event is calculated to obtain the treatment topic importance factor, which reflects the key degree of the current conversation content in the diagnosis and treatment process; Input the context complexity index, patient emotion fluctuation index, and treatment topic importance factor into the dynamic weight adjustment model based on reinforcement learning, and generate an initial weight allocation vector for three-dimensional scoring through a multi-factor fusion decision algorithm; Perform normalization processing on the initial weight allocation vector, and combine the historical conversation weight adjustment record for bias correction to obtain the final three-dimensional score dynamic weight allocation parameters.

3. The multi-dimensional real-time doctor-patient communication evaluation and analysis method according to claim 1, wherein: The method comprises the following steps: Collecting real-time conversation content between the doctor and the AI simulated patient, processing the collected content, and forming a structured conversation event sequence, including: Collecting real-time conversation content between the doctor and the AI simulated patient, the real-time conversation content including voice signal and text transcription data, and recording conversation timestamp information; Performing noise reduction and speech recognition processing on the voice signal to generate a text sequence, and performing semantic alignment with the original text data to form a structured conversation event sequence.

4. The multi-dimensional real-time doctor-patient communication evaluation and analysis method according to claim 1, wherein: The conversation event sequence is divided into multiple semantic event units, including: Using a sliding window segmentation algorithm based on semantic coherence measurement to preliminarily segment the conversation; Using a BERT semantic embedding model to calculate the semantic similarity of adjacent segments to determine the segmentation point; Performing length normalization processing on the semantic event units and packaging them into a structured data format.

5. The multi-dimensional real-time doctor-patient communication evaluation and analysis method according to any one of claims 1-4, wherein: The intention recognition model comprises: text preprocessing is performed on the semantic event units to generate normalized semantic input text; a pre-trained language model is used to generate a high-dimensional semantic vector sequence; time sequence semantic features are extracted through a Bi-GRU network, and global semantic representation is generated in combination with an attention mechanism; a semantic label sequence is output through a fully connected classification layer.

6. The multi-dimensional real-time doctor-patient communication evaluation and analysis method according to any one of claims 1-4, characterized in that: the semantic intention trajectory is constructed based on the semantic label sequence, comprising a semantic intention time evolution graph is constructed based on a sliding window; a density clustering algorithm is used to analyze the distribution density of the intention labels; linear regression and change point detection technology are used to identify intention change trends; a context shift judgment rule base is used to generate a focus center dynamic distribution graph.

7. The multi-dimensional real-time doctor-patient communication evaluation and analysis method according to claim 6, characterized in that: the semantic intention trajectory is matched with a pre-defined dimension score model to generate preliminary score results of three dimensions of treatment comprehensiveness, politeness standardization and humanistic care standardization, comprising: a semantic intention trajectory is generated from the doctor-patient dialogue sequence, and the semantic intention trajectory contains distribution information of three types of intention labels of diagnosis and treatment information, polite language and humanistic care expression in multiple semantic event units; based on the semantic intention trajectory, an intention density distribution matrix of the three types of intention labels in each semantic event unit is calculated; the intention density distribution matrix is matched with a pre-defined dimension score model, and preliminary score results of three dimensions of treatment comprehensiveness, politeness standardization and humanistic care standardization are respectively generated through a mapping function preset in the dimension score model.

8. The multi-dimensional real-time doctor-patient communication evaluation and analysis method according to any one of claims 1-4, characterized in that: the dynamic feedback signal is generated, comprising: a correction suggestion is generated by matching a score deviation quantification index with a feedback strategy rule base; visual chart data is generated based on a score trend analysis model; output to a doctor terminal interface through a human-computer interaction interface.

9. A multi-dimensional real-time doctor-patient communication evaluation and analysis system, characterized in that: for implementing the multi-dimensional real-time doctor-patient communication evaluation and analysis method according to any one of claims 1-8; the system comprises: a data acquisition module, which is used to acquire voice and text dialogue data; a speech recognition and alignment module, which is used to generate a structured dialogue event sequence; a semantic event division module, which is used to divide semantic event units; an intention recognition module, which is used to extract a semantic label sequence; a semantic trajectory construction module, which is used to construct a semantic intention trajectory and analyze intention density and trends; a dimension score module, which is used to generate preliminary scores of treatment comprehensiveness, politeness standardization and humanistic care standardization; a weight adjustment module, which is used to generate dynamic weight allocation parameters; a score fusion and feedback module, which is used to generate a comprehensive score and a dynamic feedback signal.

Images