Psychiatric auxiliary medical record generation and quantitative evaluation method based on large language model reasoning and multi-modal fusion

Abstract

The present application relates to a kind of psychiatric auxiliary medical record generation and quantitative evaluation method based on large language model inference and multi-modal fusion, comprising: mental motor evaluation based on the acoustic characteristics of current patient;Disease extraction based on standard sentence base vector projection, filter effective pathological statement set;Based on probability entropy value and semantic coherence, the thought form disorder that current patient exists is judged hierarchically;Risk weighted classification based on semantic distance: calculate the semantic distance between patient conversation context and preset suicide intent anchor vector, carry out suicide risk classification evaluation;Mental motor evaluation conclusion, effective pathological statement set, thought form abnormality feature evaluation result, suicide risk classification result are summarized, and it is written in standard electronic medical record auxiliary evaluation template, form final quantitative evaluation report.The present application also relates to a corresponding system, device, processor and its storage medium, with the advantages of evaluation objectification, process automation, application scenario.

Description

Technical Field

[0001] This invention relates to the field of smart healthcare technology, and more particularly to the field of natural language processing. Specifically, it relates to a method, system, device, processor, and computer-readable storage medium for generating and quantitatively evaluating psychiatric auxiliary medical records based on large language model reasoning and multimodal fusion. Background Technology

[0002] Unlike most clinical departments, psychiatric disorders currently lack imaging or biochemical indicators that can serve as the gold standard for diagnosis. Clinical judgment relies heavily on in-depth analysis and professional assessment of information such as the patient's complaints, emotional responses, thought processes, and behavioral manifestations during doctor-patient conversations. This highly subjective interview-based assessment model places extremely high demands on the professionalism of diagnostic tools. Currently, the industry has explored using an "ASR (Automatic Speech-to-Text) + general large-scale model" approach to assist in psychiatric assessment and medical record generation. However, in actual clinical application, this technical approach has revealed the following technical shortcomings: 1. Loss of nonverbal information: In psychiatric consultations, the patient's speech rate (e.g., increased speech rate in mania), pauses (e.g., slowed response in depression), and intonation changes (e.g., flat tone in emotional apathy) are core diagnostic criteria. Current methods first convert speech to plain text using ASR, an process that irreversibly discards all of the aforementioned acoustic information, resulting in the loss of key dimensions in subsequent analysis.

[0003] 2. Lack of objective quantitative indicators: The current system can only generate qualitative descriptions (such as "patient is depressed") and cannot provide calculable objective quantitative indicators (such as "emotional fluctuation variance = 0.82"). This makes it difficult for doctors to accurately compare the changes in the same patient's condition at different times through data (such as assessing the degree of improvement of "emotional apathy"), and cannot meet the clinical needs for objective and quantitative disease progression tracking.

[0004] 3. The Challenge of Identifying Thought Pattern Disorders: One of the core manifestations of schizophrenia is thought pattern disorder, such as "illogical reasoning" and "speech mixing." General large-scale models, in order to generate fluent text, employ a decoding strategy with a "fluency bias," tending to automatically patch logical breaks or semantic incoherence. This not only fails to identify pathological features but also "flattens" them, directly contradicting the diagnostic goals.

[0005] 4. Lack of contextual understanding in risk assessment: Risk assessment for suicide, violence, etc., is a crucial part of psychiatric consultation. Existing keyword-matching solutions cannot effectively distinguish between semantically drastically different contexts such as "I want to die" and "I'm so tired," causing high-risk signals to be drowned out by everyday expressions or generating a large number of false alarms, making them difficult to apply in clinical practice.

[0006] In summary, the existing "speech-to-text + general large model" technical solution has fundamental defects and cannot meet the core needs of psychiatric clinicians for in-depth analysis and objective quantification of doctor-patient dialogues. There is an urgent need for a technical solution that can integrate multimodal information, provide objective quantitative indicators, and possess professional diagnostic logic in psychiatry. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method, system, device, processor and computer-readable storage medium for generating and quantitatively evaluating psychiatric auxiliary medical records based on large language model reasoning and multimodal fusion.

[0008] To achieve the above objectives, the present invention provides a method, system, device, processor, and computer-readable storage medium for generating and quantitatively evaluating psychiatric auxiliary medical records based on large language model reasoning and multimodal fusion, as follows: This method for generating and quantitatively evaluating psychiatric auxiliary medical records based on large language model reasoning and multimodal fusion is characterized by the following features: Step S1: Perform a psychomotor assessment based on the current acoustic characteristics of the patient to identify whether there is psychomotor retardation or psychomotor agitation; Step S2, Symptom Extraction Based on Basis Vector Projection: By constructing standard sentence baseline vectors for depression, anxiety, and hallucination, the similarity between the semantic vector of the patient's statement and the baseline vector is calculated, and a set of effective pathological statements is screened through threshold gating, thereby generating semantic reference material for the current patient's present medical history. Step S3, Quantification of thought disorder based on probability entropy: The probability entropy value and semantic coherence of the patient's conversation are predicted using a large language model, and a thought dispersion index quantification model is constructed. By monitoring fluctuation characteristics, the abnormal characteristics of the patient's thought patterns are objectively identified and graded. Step S4, Risk-weighted classification assessment based on semantic distance: By calculating the semantic distance between the patient's dialogue context and the preset suicide intention anchor vector, the current suicide risk of the patient is classified and potentially assessed. Step S5: Summarize the psychomotor assessment conclusions, the set of valid pathological statements, the assessment results of abnormal thought patterns, and the suicide risk classification results. After quantitative assessment, write them into the standard electronic medical record auxiliary assessment template to form the final quantitative assessment reference report.

[0009] Preferably, step S1 specifically includes: Step S11: Collect the dialogue between the patient and the doctor, perform speech activity detection on the patient's audio, and use voiceprint clustering to distinguish the doctor's speech segments S. dr and patient's voice segment S pt ; Step S12, calculate the patient's response latency: Define the latency ΔT of the i-th round of dialogue. i The moment the doctor ends speaking. end dr The moment t begins speaking to the patient start pt Difference: ΔT i =t start pt (i)-t end dr (i) Step S13: Calculate the patient's speech rate variation coefficient throughout the entire consultation process as follows: Where, σ spm μ is the standard deviation of the number of syllables per minute. spm The average number of syllables per minute; The judgment logic for the psychomotor assessment is set as follows: Psychomotor agitation: μ spm Significantly higher than the norm and CV rate Higher than the first preset threshold; Psychomotor retardation: μ spm Significantly lower than the norm and CV rate It does not exceed the second preset threshold.

[0010] Preferably, step S2 specifically includes: Step S21, constructing a standard pathological space: Several standard sentences describing psychiatric symptoms are pre-selected quantitatively from the official unified disease classification, and the embedding vectors of the corresponding standard sentences are obtained through a large language model. The mean of these vectors is then used to construct a baseline depression vector V. dep Anxiety baseline vector V anx Illusion reference vector V hallu ; Step S22: Convert the patient's speech into sentences, for each sentence u of the patient t Obtain its semantic vector E(u) t And calculate its cosine similarity projection on each reference vector in the following manner: in, For depression cosine similarity, For anxiety cosine similarity, Cosine similarity to hallucination; Step S23: Dynamically and adaptively set a noise gating threshold θ based on the obtained text length and colloquialism. When any cosine similarity is greater than θ, the patient's dialogue is identified as a valid pathological statement. Step S24: Arrange all valid pathological statements in chronological order as semantic reference material for the current patient's present medical history, fill them into the present medical history reference column, and indicate the relevant characteristics of the corresponding pathological type.

[0011] Preferably, step S3 specifically includes: Step S31, read the preceding text of the patient's dialogue. Input a large language model and obtain the probability distribution for generating the next token x. ; Step S32: Based on the probability distribution results obtained above, calculate the corresponding local Shannon entropy. : Where V represents the vocabulary size, i.e., the total number of tokens; Step S33: Calculate the similarity between the currently generated word vector and the preceding context vector. : in, This represents the semantic vector of the words and phrases generated by the patient during the dialogue at the current time t. The overall semantic vector representing the context preceding the patient's dialogue; Step S34, based on the local Shannon entropy and similarity Constructing a thought dispersion index : in, and All of these are preset weighting coefficients, which will be adaptively adjusted based on the semantic projection results in step S2. Step S35: Determine the patient's potential abnormal thought patterns according to the following assessment criteria: If the TDI curve shows a sustained high-level oscillation, it suggests a loosening / fractured mindset. If the TDI curve shows periodic jumps, it suggests that the person has a tendency to fly through the mind.

[0012] Preferably, step S4 specifically includes: Step S41: Based on psychiatric diagnostic and treatment standards and a suicide risk assessment scale, a preset anchor vector representing suicidal intent is established. ; Step S42, when the patient's conversation contains a set of sensitive words The sensitive word w in k When this occurs, the following risk detection will be triggered: Step S421, Context Window Extraction: Extract the sensitive word w k Context windows of the k tokens before and after , where k will dynamically expand or contract based on sentence length or semantic segmentation; Step S422, Semantic distance calculation: Calculate the context window semantic vectors With risk anchor vector A suicide The Euclidean distance D between them: If D < τ1, then a high risk of suicide is considered. If τ1≤D<τ2, then it is determined that there is a possibility of suicide risk. If D≥τ2, then there is a low risk of suicide. Where τ1 and τ2 are both preset grading thresholds.

[0013] Preferably, step S5 specifically includes: Step S51: Based on the current psychomotor assessment conclusion, effective pathological statement set, assessment results of abnormal thought patterns and suicide risk classification results of the patient, a quantitative assessment vector is formed, and the equivalent standard scale reference score corresponding to the quantitative index is generated according to the pre-established calibration mapping relationship. Step S52: The quantitative assessment results, the reference scores of the equivalent standard scale, and the corresponding original evidence fragments are written into the standard electronic medical record auxiliary assessment record, and a corresponding quantitative assessment report is generated for reference.

[0014] The psychiatric auxiliary medical record generation and quantitative assessment system based on large language model reasoning and multimodal fusion for implementing the above-described methods is characterized in that the system includes: The multimodal synchronous acquisition module simultaneously acquires the patient's speech and text streams and establishes a millisecond-level timestamp mapping. The acoustic pathology feature analysis module extracts features of response latency, speech rate variation coefficient, and fundamental frequency energy from the patient's speech stream based on signal processing algorithms, and performs corresponding psychomotor assessments. The semantic vector projection module uses a large language model to extract the embedding vectors of the patient's statements, calculates the similarity between them and the preset baseline vectors, and filters out the set of valid pathological statements. The thought entropy monitoring module uses a large language model to predict the probability distribution entropy of the next token, and combines the similarity between the word vectors generated in the current patient dialogue and the vectors of the preceding context to measure the coherence of their thinking, thereby identifying and classifying the abnormal characteristics of the patient's thought patterns. The context-based risk weighting module calculates the true threat level of risk words in patient conversations based on semantic distance, thereby conducting a graded potential assessment of suicide risk. The structured medical record generation engine is connected to the above modules to quantify the acoustic features, pathological projection scores, thought entropy values, and potential suicide risk levels obtained from calculation, evaluation, and / or grading, and fill them into the standard electronic medical record auxiliary assessment template to form the final quantitative assessment reference report.

[0015] This psychiatric auxiliary medical record generation and quantitative assessment device based on large language model reasoning and multimodal fusion is characterized by the following features: A processor is configured to execute computer-executable instructions; The memory stores one or more computer-executable instructions, which, when executed by the processor, implement the steps of the above-described method for generating and quantitatively evaluating psychiatric auxiliary medical records based on large language model reasoning and multimodal fusion.

[0016] The processor for generating and quantitatively evaluating psychiatric auxiliary medical records based on large language model reasoning and multimodal fusion is characterized in that the processor is configured to execute computer-executable instructions, which, when executed by the processor, implement the steps of the aforementioned method for generating and quantitatively evaluating psychiatric auxiliary medical records based on large language model reasoning and multimodal fusion.

[0017] The computer-readable storage medium is characterized in that it stores a computer program that can be executed by a processor to implement the steps of the above-described method for generating and quantitatively evaluating psychiatric auxiliary medical records based on large language model reasoning and multimodal fusion.

[0018] The method, system, device, processor, and computer-readable storage medium for generating and quantitatively evaluating psychiatric auxiliary medical records based on large language model reasoning and multimodal fusion of the present invention have the following significant advantages compared with the prior art: I. By fusing multimodal features, the deficiency of non-linguistic information loss is overcome. This application integrates the acoustic features of speech with the semantic features of text, overcoming the limitations of existing pure text solutions in information acquisition. By simultaneously capturing acoustic signals such as speech rate, pauses, and intonation, as well as semantic signals such as language logic and content, it can more completely reconstruct the patient's psychomotor state and significantly reduce the risk of missed diagnoses due to missing information dimensions.

[0019] Second, it has achieved objective quantification and precise classification of mental symptoms. Unlike existing technologies that can only output qualitative descriptions, this application introduces computational techniques such as basis vector projection, probability entropy, and semantic distance to mathematically model abstract psychopathological phenomena. For example, it can achieve objective quantitative grading and accurate identification of core clinical indicators such as psychomotor agitation or retardation, thought disorder, and suicide risk, thus solving the pain point of psychiatric diagnosis, which has long relied on subjective experience and lacked objective measurement standards.

[0020] III. Achieving efficient and standardized medical record generation through end-to-end intelligent processes. This application automates the entire process from assessment and symptom extraction to consultation analysis and medical record generation. By internalizing the professional diagnostic logic of psychiatry into the system, it can automatically transform multimodal analysis results into well-structured and standardized electronic medical records without human intervention. This not only frees doctors from heavy paperwork but also ensures the standardization and homogenization of medical records from the source.

[0021] IV. Based on a large language model, a highly practical clinical support system has been constructed. This application deeply integrates large language models with psychiatric clinical scenarios, forming a non-invasive, multi-scenario adaptable technical solution. It can be embedded in outpatient systems to assist in real-time consultations, used for inpatient course monitoring and efficacy evaluation, and provide structured, computable data support for early screening and clinical research, demonstrating significant clinical application value and promising prospects for widespread use. Attached Figure Description

[0022] Figure 1 The flowchart shows the psychiatric auxiliary medical record generation and quantitative assessment method based on large language model reasoning and multimodal fusion of the present invention. Detailed Implementation

[0023] To more clearly describe the technical content of the present invention, the following description is provided in conjunction with specific embodiments.

[0024] Before describing the embodiments of the present invention in detail, it should be noted that, in the following, the terms “comprising,” “including,” or any other variations are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0025] The implementation process of this technical solution will be further explained below with reference to specific embodiments: like Figure 1 As shown in the embodiments of this application, a method for generating and quantitatively assessing psychiatric auxiliary medical records based on large language model reasoning and multimodal fusion is provided, which is achieved by constructing a multimodal signal fusion and AI reasoning framework. This method extracts the patient's speech acoustic features and performs cross-modal fusion with the reasoning features of the dialogue text from the large language model. It constructs a digital twin model for psychiatric assessment from four dimensions: psychomotor function, pathological statements, thought patterns, and suicide risk. Based on this model, automated and quantitative assessment of the severity, symptoms, and risk level of mental illness patients can be achieved, and standardized medical records can be generated accordingly, providing objective data support for clinical consultation, treatment plan development, and prognosis monitoring.

[0026] The method for generating and quantitatively evaluating psychiatric auxiliary medical records based on large language model reasoning and multimodal fusion includes the following steps: Step S1 involves conducting a psychomotor assessment based on acoustic characteristics to identify the presence of psychomotor retardation or psychomotor agitation. This can be used to help determine the risk of major depressive disorder (often accompanied by psychomotor retardation) or mania (often accompanied by psychomotor agitation).

[0027] In a preferred embodiment of the present invention, step S1 specifically includes: Step S11: Collect the dialogue between the patient and the doctor, perform voice activity detection (VAD) on the audio, and use voiceprint clustering to distinguish the doctor's speech segments S. dr and patient's voice segment S pt ; Step S12: Calculate the response latency, and define the latency ΔT for the i-th round of dialogue. i The moment the doctor ends speaking. end dr The moment t begins speaking to the patient start pt Difference: ΔT i =t start pt (i)-t end dr (i) Step S13: Calculate the coefficient of variation (CV) of speech rate, calculate the number of syllables per minute (SPM) of the patient, and calculate its coefficient of variation (CV) throughout the entire consultation process. Where, σ spm μ is the standard deviation of the number of syllables per minute. spm The average number of syllables per minute. In a preferred embodiment of the present invention, the psychomotor assessment and determination logic is as follows: Psychomotor agitation: if μ spm Significantly higher than the norm (higher than the upper limit of the healthy population / norm, e.g., higher than the normal mean +1.5SD or +2SD) and CV rate If the speech pattern is higher than the first preset threshold, it meets the typical speech characteristics of a manic / hypomanic episode and belongs to the [Pressured Speech] feature label, which should be entered into the [Mental Examination - Speech] column of the medical record. Psychomotor retardation: if μ spm Significantly lower than the norm (lower than the healthy / normative lower limit, e.g., lower than the normal mean -1.5 SD or -2 SD) and CV rate If the value is normal or slightly low (not exceeding the second preset threshold), it meets the typical speech characteristics of a major depressive episode and belongs to the [speech retardation] characteristic label, which should be entered into the [Mental Examination - Speech] column of the medical record.

[0028] It is important to emphasize that in the context of psychiatry / psychometrics, a "norm" is simply a standard reference frame used for comparison. It refers to the score distribution of a representative standardized sample (i.e., the "norm group") on a specific assessment tool. In this approach, a multidimensional population norm array is used, which is stratified according to "age-gender-region" and includes norm intervals such as speech rate and latency to enhance rigor.

[0029] In one specific embodiment of the present invention, exemplary, it pertains to audio consultations: Doctor: "How have you been feeling lately?" → S dr Patient: "It's...not very good." →S pt The doctor finished speaking in 12.5 seconds, and the patient began speaking in 16.3 seconds. (Calculation...) Seconds. Common in depressed patients: ΔT is significantly larger; common in manic patients: ΔT is very small, almost instantaneous.

[0030] In one specific embodiment of the present invention, exemplarily, the average speaking speed of a healthy person... Syllables per minute, standard for healthy individuals Commonly used thresholds: ±1.5 SD; Threshold for agitation is ≥150 + 1.5 × 20 = 180, threshold for sluggishness is ≤150 - 1.5 × 20 = 120; First preset threshold (agitation): CV > 0.3, second preset threshold (sluggishness): CV ≤ 0.25. The patient's entire dialogue... Syllables per minute (far > 180) ,determination: Significantly higher than the norm. Elevated → Tag: Impulsive speech → Enter in medical record: Mental examination - Speech: Impulsive speech, psychomotor agitation.

[0031] Step S2 involves symptom extraction based on basis vector projection. This involves constructing baseline vectors for standard sentences related to depression, anxiety, and hallucinations, calculating the similarity between the semantic vector of the patient's statements and the baseline vectors, and then filtering the set of valid pathological statements using a threshold gating system. Specifically, this includes the following steps: Step S21, constructing a standard pathological space Optionally, 500 standard sentences describing depression (anxiety, hallucination) from the ICD-10 (International Classification of Diseases) are pre-selected, and the embedding vectors of the standard sentences are obtained through a large language model. The average value is then used to obtain the reference vectors. V dep (Depression baseline vector) V anx (Anxiety baseline vector) V hallu (Illusion Baseline Vector) Step S22, Real-time Semantic Projection: Convert the patient's speech into sentences, for each sentence u of the patient t Obtain its semantic vector Calculate its cosine similarity projection onto the reference vector: Step S23, noise gating: Set a projection threshold θ (e.g., 0.65). When any similarity is greater than θ, the patient's dialogue is considered a valid pathological statement. In practical applications, the projection threshold θ will be dynamically adjusted according to the length of the input text and the degree of colloquialism. For example, short sentences will be lowered to prevent missed reports, and long narratives will be raised to suppress noise, so as to enhance robustness.

[0032] Step S24, temporal aggregation: Arrange all valid pathological statements in the entire dialogue in chronological order, fill them into the present illness history (HPI) reference column as reference material, and indicate the relevant characteristics with corresponding pathological types (such as: depression, anxiety, hallucination).

[0033] It is important to emphasize that, since the embedding space itself has semantic generalization capabilities—colloquial, euphemistic, and dialectal expressions that are semantically close to the ICD standard sentences can be correctly projected—the aforementioned baseline vector construction space essentially adopts a hierarchical structure of "standard definition layer + population / colloquial / dialect generalization layer." This allows for accurate projection even when targeting patients' colloquial expressions, dialects, euphemistic expressions, or special subjects such as children and the elderly.

[0034] In one specific embodiment of the present invention, exemplarily, the patient statements described in the following table are used: Based on the patient's statements above, the calculation results for step S2 are as follows: The selected valid pathological statements include: The present illness reference material (HPI) generated based on the above valid pathological statements is as follows: Depression-related material: I haven't been able to feel happy lately, and I don't want to do anything; Anxiety-related material: I always feel anxious and worried that something might happen to my family members; Hallucination-related material: I often hear people cursing me behind my back, but when I turn around, there's no one there; Depression-related material: I can't sleep at night, and even if I do fall asleep, I wake up early.

[0035] Step S3: Quantify thought disorder based on probability entropy value. Use a large language model to predict the probability entropy value and semantic coherence of the patient's conversation, and construct a thought dispersion index (TDI) quantitative model. By monitoring the fluctuation characteristics of this index (such as continuous high-level oscillation or periodic jumps), objectively identify and classify the potential thought disorder problems such as loosening, fragmentation or flight of thought that the patient may have.

[0036] Since the logic of normal people is predictable, while the logic of patients with thought disorders is chaotic, when using a large language model to predict the next word in a patient's conversation, a probability distribution is output. If the patient's speech is logically coherent, the model's predicted perplexity and entropy are low; if the patient uses word salad, the model becomes confused, and the predicted entropy value increases sharply.

[0037] Specifically, the following steps are included: Step S31: Obtain the probability distribution of the large language model generating the next token, and combine it with the context of the patient's dialogue. Input a large language model and obtain the probability distribution for generating the next token x. .

[0038] Step S32, calculate the local Shannon entropy: Where V is the vocabulary size (total number of tokens). To generate the i-th Token x i The probability distribution. The higher the score, the more confused the model.

[0039] Step S33: Calculate semantic coherence by measuring the similarity between the currently generated word / phrase and the preceding context vector. : Here, cos() is used to calculate the cosine similarity. This represents the semantic vector of the words spoken by the patient at the current time t. This represents the overall semantic vector of the preceding text in the patient's dialogue (or the overall semantic vector of the preceding text after condensation).

[0040] Step S34, construct the Thought Disorder Index (TDI): in, and As a preset weighting coefficient, this part of the coefficients will be feedforward adjusted based on the preliminary symptom tendency of the upstream semantic projection. and and in - The TDI (Thought Pattern Disorder) classification is further refined on the feature plane. A higher TDI value indicates greater uncertainty in the model's predictions and a more significant semantic deviation, meaning that the abnormal characteristics of thought patterns are more prominent.

[0041] Step S35: Determine the possible abnormal characteristics of the patient's thought patterns. The criteria for this determination are as follows: If the TDI curve shows sustained high-level oscillations, it suggests a loosening / fractured mindset.

[0042] If the TDI curve shows periodic jumps, it suggests that the person has a tendency to fly through the mind.

[0043] In a preferred embodiment of the present invention, the TDI is normalized and numerically converted to the range of 0-10, and written into the medical record, such as: "Logical Reasoning Score: 4.2 / 10 (Severe risk of impairment)". For example, the upper limit of normal TDI is determined using data from healthy individuals, the patient's TDI is scaled proportionally to 0-10 to obtain a score, and the median of the entire dialogue is taken as the final logical reasoning score.

[0044] In one specific embodiment of the present invention, for example, the preceding text of the dialogue segment is "After I got up this morning, I first brushed my teeth, and then prepared to eat…", and the current word is "breakfast". The following probability distribution is calculated: Using the Large Language Model (LLM), the probability of predicting the next token is concentrated on reasonable options: P(“Breakfast”) = 0.7, P(“Coffee”) = 0.2, P(“Lunch”) = 0.08, P(“Dinner”) = 0.02; Calculate the local Shannon entropy: H(t) = -(0.7log20.7 + 0.2log20.2 + 0.08log20.08 + 0.02log20.02) ≈ 1.233; Calculate semantic coherence The phrase "breakfast" is semantically highly correlated with the preceding phrases "getting up and brushing teeth," with a cosine similarity of 0.85. The calculated Thought Dispersion Index (TDI) is calculated as 0.6 × 1.23 + 0.4 × (1 - 0.85) ≈ 0.80. The results indicate that the TDI curve remains stable within the 0.7-1.0 range, with a median TDI score of 0.9 for the entire dialogue. Normalized to the 0-10 range, this results in a score of 8.5 / 10 (normal).

[0045] In one specific embodiment of the present invention, for example, the preceding text of a dialogue segment is "The sky is blue, the puppy is running, there are apples on the table, car wheels...", and the current word is "moon". The following probability distribution is calculated: Since LLM cannot predict the logical direction, the probabilities are uniformly distributed: P(“round”) = 0.25, P(“moon”) = 0.25, P(“hair”) = 0.25, P(“running”) = 0.25; Calculate the local Shannon entropy: H(t) = -4 × (0.25log20.25) = 2.0; Calculate semantic coherence The word "moon" is semantically disjointed from the preceding phrases "sky, puppy, table, car," with a cosine similarity of 0.3; the computational thinking discrete index... =0.6×2.0+0.4×(1-0.3)=1.48. The results show that the TDI curve continues to oscillate at a high level in the range of 1.3-2.0, and the median TDI score for the entire conversation is 1.8, which is normalized to 4.2 / 10 (there is a severe risk of impairment).

[0046] It should be noted that in the example above, the selected... =0.6, =0.4. The historical calibration method for this group's baseline coefficient is as follows: based on the dialogue data of 100 patients with thought disorder and 100 healthy controls, a grid search was performed with the objective of maximizing the binary AUC of the two groups' TDI curves, and the maximum AUC obtained was 0.93. Therefore, This is the default implementation method of this solution in the above embodiments, and is not arbitrarily specified. In practical applications, the selection of this coefficient will be adaptively adjusted according to the semantic projection results in step S2, as detailed below: Step S2 first outputs semantic projection scores for depression, anxiety, hallucination, etc.: Normalize the projected scores to obtain the relative weights of each symptom tendency: in, For smoothing coefficients, This indicates the corresponding symptom category. The system adjusts the value based on the pre-defined symptom category. Bounded adjustments are made around the default coefficient: Final adoption: Each correction factor and its upper and lower bounds are still calibrated with maximizing the validation set AUC or minimizing the error with the clinical reference score. When none of the semantic projection scores exceed the effective pathological statement threshold, or when there is no clear dominant symptom tendency, the system will directly regress to the default coefficients. , .

[0047] In particular, for the selection of large language models, in low computing power / high concurrency scenarios, the distillation of small models can be used to calculate the probability entropy per token, thereby effectively reducing overhead.

[0048] Step S4 involves risk-weighted grading based on semantic distance. This involves calculating the semantic distance between the patient's dialogue context and the preset suicidal intent anchor vector to assess suicide risk. This step does not train for identifying suicide keywords; instead, it calculates the semantic distance between the patient's dialogue context and the preset suicidal intent anchor vector, thus addressing the issue of keyword false positives. Specifically, it includes the following steps: Step S41, Define risk anchor points: Pre-define an anchor point vector A representing a definite suicidal intent. suicide .

[0049] Anchor vector A suicide It is formed by the aggregation of semantic vectors of sentences expressing high-lethal suicidal ideation as clearly defined in psychiatric diagnostic criteria and suicide risk assessment scales (such as SAD PERSONS and C-SSRS).

[0050] As examples, sentences expressing highly lethal suicidal ideation include, but are not limited to: I have a clear suicide plan and intend to carry it out soon; I want to end my life; I feel that life is meaningless and I plan to commit suicide; I have prepared the tools for suicide; I have persistent suicidal thoughts and specific plans to carry them out... The above sentence is encoded using the BERT model, and the anchor vector is obtained through mean aggregation. For ease of calculation and display, the high-dimensional vector is simplified to 10 dimensions, resulting in the final anchor vector A. suicide =[0.82,0.79,0.85,0.81,0.78,0.83,0.80,0.84,0.77,0.82].

[0051] Step S42, trigger detection when a set of sensitive words appears in the patient's conversation. (Sensitive words such as "death", "end", "suicide", "don't want to live", "pass away", "conclusion") k When, calculation is triggered: Step S421, extract the context window and extract the word w. k The context window C of the k (e.g., k=5) tokens before and after. window In practical applications, the risk ranging window It will dynamically expand and contract based on sentence length and semantics, for example: short sentences Expanding to cross-sentences and long complex sentences This enhances the ability to detect suicidal / self-harm intentions from a distance or across sentences.

[0052] Step S422, semantic distance calculation, calculate context window C window semantic vector E(C) window ) and risk anchor vector A suicide Euclidean distance D: Step S423, the risk classification criteria are set as follows: If D < τ1, then a high risk of suicide is considered. If τ1≤D<τ2, then it is determined that there is a possibility of suicide risk. If D≥τ2, then there is a low risk of suicide. Wherein, τ1 and τ2 are preset grading thresholds. The risk grading results are written into the patient's electronic medical record auxiliary assessment record.

[0053] This step is used to conduct a graded auxiliary assessment of potential suicide risk. Even if a patient says "die," if the context is "I'm laughing so hard," the semantic encoding vector distance is [not specified]. Extremely far away will be judged as having a low risk of suicide, thus avoiding false alarms for keywords.

[0054] In one specific embodiment of the present invention, for example, the patient's original dialogue is: "Lately, I feel like there's no hope in living, and sometimes I think about ending it all." The detection is triggered based on the sensitive word "end". Extracting Context Window: with w k = Centered on "End", extract the context windows of 5 tokens before and after it, and the complete window content is C window =“Sometimes I think about ending it all, I've been feeling that way lately”; Calculate semantic distance: for C window Semantic encoding yields a simplified 10-dimensional vector: E(C) window = [0.52, 0.55, 0.58, 0.54, 0.56, 0.53, 0.57, 0.55, 0.54, 0.56]; Calculation and A suicide The Euclidean distance is: D≈0.626; τ1 is 0.3, τ2 is 0.8, 0.3≤D<0.8, indicating a possible risk of suicide.

[0055] Step S5 involves summarizing the psychomotor assessment conclusions, the set of valid pathological statements, the assessment results of abnormal thought patterns, and the suicide risk classification results. After quantitative evaluation, these findings are entered into the standard electronic medical record auxiliary assessment template to form the final quantitative assessment reference report. This includes the following steps: Step S51: The large language model is used to summarize the current patient's psychomotor assessment conclusion, effective pathological statement set, abnormal thought pattern assessment results, and suicide risk classification results. The summarized results are then quantified by constructing AI quantitative indicators to form a quantitative assessment vector. The AI ​​quantitative indicators are then calibrated and mapped with standard scales (such as PANSS-P2 / HAMD / PHQ-9) to obtain the corresponding equivalent standard scale reference scores. Step S52: Combine the AI ​​quantitative assessment indicators and the equivalent standard scale reference scores into the standard electronic medical record auxiliary assessment record, and generate a corresponding quantitative assessment report for reference.

[0056] As a preferred embodiment of the present invention, regarding the quantification of results for relevant indicators mentioned in this technical solution, in practical applications, as an example, this solution can define relevant indicators according to the following three levels: 1. Raw TDI Take the median of the original TDI for the entire dialogue: 2. 0–10 Standardized Anomaly Score and Logical Thinking Ability Rating Based on the reference distribution formed by healthy controls and samples with abnormal thought patterns, a monotonic calibration function was used. Convert the original TDI to the 0–10 range: in, The higher the value, the greater the degree of abnormality. A higher value indicates better logical thinking. These two values ​​are opposite in nature and should not be used interchangeably.

[0057] 3. Equivalent PANSS-P2 reference score Based on the previously calibrated mapping function, the system converts the standardized TDI into an equivalent reference score for PANSS-P2:

[0058] in, This indicates the mapping relationship established based on the previous calibration results.

[0059] For example, consider an illustrative patient in an intermediate state. The patient states: "Lately, I've been feeling a bit confused. I think about work, then home, and then I think about not sleeping well at night, but I can still answer questions." In a representative context window, assuming the model's prediction entropy is 1.60 and semantic coherence is 0.60, then:

[0060] After taking the median of the entire conversation, the following example is obtained: Illustrative interpolation based on the reference distribution between existing normal examples and obviously abnormal examples yields... The score is approximately 3.2, and the logical reasoning score is approximately 6.8. These values ​​are only used to illustrate the calculation path of intermediate states and are not considered as new clinical test results. Calculation examples are shown in the table below: The three examples above are quantitative calculations of patient dialogues, respectively demonstrating a normal reference state, a significantly abnormal state, and an intermediate transitional state. It is evident that the higher the standardized TDI, the lower the logical reasoning score.

[0061] In a specific embodiment of the present invention, as an example, when the thought form disorder score is TDI = 6.82, which is equivalent to PANSS-P2 ≈ 5, it represents a risk of "moderate to severe" thought form disorder. This result is an example of scale mapping and does not correspond to the three patient dialogue cases mentioned above. Therefore, it is not necessary to supplement the local entropy, semantic coherence, or original TDI in the patient dialogue form.

[0062] Regarding the quantitative mapping relationship between various indicators, as an example, this scheme also includes: In the initial sample of 120 desensitization clinic dialogues, the correlation coefficient between TDI and PANSS-P2 was [value missing]. The correlation coefficient with the total TLC score is All of them ; In the baseline coefficient calibration, the TDI binary classification AUC of 100 thought disorder samples and 100 healthy controls was 0.93; In a test of 1,500 medical consultation statements containing high-risk sensitive words, the risk semantic ranging method reduced the false positive rate to 3.2% while maintaining a high-risk intent recall rate of no less than 98.5%.

[0063] The above results correspond to scale consistency, discrimination of abnormal thinking patterns, and risk context disambiguation ability, respectively, which can jointly reflect the quantitative accuracy of this scheme from multiple different perspectives.

[0064] This application also provides a psychiatric auxiliary medical record generation and quantitative assessment system based on large language model reasoning and multimodal fusion, wherein the system includes: The multimodal synchronous acquisition module simultaneously acquires the patient's speech and text streams and establishes a millisecond-level timestamp mapping. The acoustic pathology feature analysis module extracts response latency, speech rate variation coefficient, and fundamental frequency energy features from the patient's speech stream based on signal processing algorithms to conduct psychomotor assessment. The semantic vector projection module uses a large language model to extract the embedding vectors of the patient's statements, calculates their similarity with the preset baseline vectors, and filters the set of valid pathological statements. The Entropy Monitoring module uses a large language model to predict the probability distribution entropy of the next token, quantifies the coherence of thought, and identifies and grades the abnormal characteristics of the patient's thought patterns. The context-based risk weighting module calculates the true threat level of risk words based on semantic distance and performs a graded potential assessment of suicide risk. The structured medical record generation engine quantifies the acoustic features, pathological projection scores, thought entropy values, and suicide risk levels obtained from the above calculations, assessments, or classifications, and fills them into the standard electronic medical record auxiliary assessment template to form the final quantitative assessment reference report.

[0065] This psychiatric auxiliary medical record generation and quantitative assessment device based on large language model reasoning and multimodal fusion includes: A processor is configured to execute computer-executable instructions; The memory stores one or more computer-executable instructions, which, when executed by the processor, implement the steps of the above-described method for generating and quantitatively evaluating psychiatric auxiliary medical records based on large language model reasoning and multimodal fusion.

[0066] The processor for generating and quantitatively evaluating psychiatric auxiliary medical records based on large language model reasoning and multimodal fusion is configured to execute computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the aforementioned method for generating and quantitatively evaluating psychiatric auxiliary medical records based on large language model reasoning and multimodal fusion are implemented.

[0067] The computer-readable storage medium is characterized in that it stores a computer program that can be executed by a processor to implement the steps of the above-described method for generating and quantitatively evaluating psychiatric auxiliary medical records based on large language model reasoning and multimodal fusion.

[0068] It is important to emphasize that the electronic medical record-assisted assessment records and quantitative assessment reports generated by this technical solution are ultimately intended to provide medical personnel with quantitative information on mental state, risk warnings, and primary evidence. They cannot be used to determine a patient's disease diagnosis or treatment plan independently. The patient's final clinical diagnosis and treatment decisions will be made by a qualified clinician based on a complete medical history, face-to-face consultation, physical examination, formal scales, and other examination results.

[0069] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0070] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution device.

[0071] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0072] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0073] In the description of this specification, references to terms such as "an embodiment," "some embodiments," "example," "specific example," or "embodiment," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0074] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

[0075] This invention employs a method, system, device, processor, and computer-readable storage medium for generating and quantitatively assessing psychiatric auxiliary medical records based on large language model reasoning and multimodal fusion. Without altering the parameters of the large language model, it utilizes probability distributions and semantic vectors from the model reasoning process, combined with speech acoustic features, to perform multi-dimensional quantitative analysis and risk grading of psychiatric doctor-patient dialogues, generating structured electronic medical record-assisted assessment records. It offers advantages such as objective assessment, automated processes, and scenario-based application.

[0076] In this specification, the invention has been described with reference to specific embodiments thereof. However, it will be apparent that various modifications and variations can be made without departing from the spirit and scope of the invention. Therefore, the specification and drawings should be considered illustrative rather than restrictive.

Claims

1. A method for generating and quantitatively evaluating psychiatric auxiliary medical records based on large language model reasoning and multimodal fusion, characterized in that, The method includes: Step S1: Perform a psychomotor assessment based on the current acoustic characteristics of the patient to identify whether there is psychomotor retardation or psychomotor agitation; Step S2, Symptom Extraction Based on Basis Vector Projection: By constructing standard sentence baseline vectors for depression, anxiety, and hallucination, the similarity between the semantic vector of the patient's statement and the baseline vector is calculated, and a set of effective pathological statements is screened through threshold gating, thereby generating semantic reference material for the current patient's present medical history. Step S3, Quantification of thought disorder based on probability entropy: The probability entropy value and semantic coherence of the patient's conversation are predicted using a large language model, and a thought dispersion index quantification model is constructed. By monitoring fluctuation characteristics, the abnormal characteristics of the patient's thought patterns are objectively identified and graded. Step S4, Risk-weighted classification assessment based on semantic distance: By calculating the semantic distance between the patient's dialogue context and the preset suicide intention anchor vector, the current suicide risk of the patient is classified and potentially assessed. Step S5: Summarize the psychomotor assessment conclusions, the set of valid pathological statements, the assessment results of abnormal thought patterns, and the suicide risk classification results. After quantitative assessment, write them into the standard electronic medical record auxiliary assessment template to form the final quantitative assessment reference report.

2. The method for generating and quantitatively evaluating psychiatric auxiliary medical records based on large language model reasoning and multimodal fusion according to claim 1, characterized in that, Step S1 specifically includes: Step S11: Collect the dialogue between the patient and the doctor, perform speech activity detection on the patient's audio, and use voiceprint clustering to distinguish the doctor's speech segments S. dr and patient's voice segment S pt ; Step S12, calculate the patient's response latency: Define the latency ΔT of the i-th round of dialogue. i The moment the doctor ends speaking. end dr The moment t begins speaking to the patient start pt Difference: ΔT i =t start pt (i)-t end dr (i) Step S13: Calculate the patient's speech rate variation coefficient throughout the entire consultation process as follows: Where, σ spm μ is the standard deviation of the number of syllables per minute. spm The average number of syllables per minute; The judgment logic for the psychomotor assessment is set as follows: Psychomotor agitation: μ spm Significantly higher than the norm and CV rate Higher than the first preset threshold; Psychomotor retardation: μ spm Significantly lower than the norm and CV rate It does not exceed the second preset threshold.

3. The method for generating and quantitatively evaluating psychiatric auxiliary medical records based on large language model reasoning and multimodal fusion according to claim 1, characterized in that, Step S2 specifically includes: Step S21, constructing a standard pathological space: Several standard sentences describing psychiatric symptoms are pre-selected quantitatively from the official unified disease classification, and the embedding vectors of the corresponding standard sentences are obtained through a large language model. The mean of these vectors is then used to construct a baseline depression vector V. dep Anxiety baseline vector V anx Illusion reference vector V hallu ; Step S22: Convert the patient's speech into sentences, for each sentence u of the patient t Obtain its semantic vector E(u) t And calculate its cosine similarity projection on each reference vector in the following manner: in, For depression cosine similarity, For anxiety cosine similarity, Cosine similarity to hallucination; Step S23: Dynamically and adaptively set a noise gating threshold θ based on the obtained text length and colloquialism. When any cosine similarity is greater than θ, the patient's dialogue is identified as a valid pathological statement. Step S24: Arrange all valid pathological statements in chronological order as semantic reference material for the current patient's present medical history, fill them into the present medical history reference column, and indicate the relevant characteristics of the corresponding pathological type.

4. The method for generating and quantitatively evaluating psychiatric auxiliary medical records based on large language model reasoning and multimodal fusion according to claim 3, characterized in that, Step S3 specifically includes: Step S31, read the preceding text of the patient's dialogue. Input a large language model and obtain the probability distribution for generating the next token x. ; Step S32: Based on the probability distribution results obtained above, calculate the corresponding local Shannon entropy. : Where V represents the vocabulary size, i.e., the total number of tokens; Step S33: Calculate the similarity between the currently generated word vector and the preceding context vector. : in, This represents the semantic vector of the words and phrases generated by the patient during the dialogue at the current time t. The overall semantic vector representing the context preceding the patient's dialogue; Step S34, based on the local Shannon entropy and similarity Constructing a thought dispersion index : in, and All of these are preset weighting coefficients, which will be adaptively adjusted based on the semantic projection results in step S2. Step S35: Determine the patient's potential abnormal thought patterns according to the following assessment criteria: If the TDI curve shows a sustained high-level oscillation, it suggests a loosening / fractured mindset. If the TDI curve shows periodic jumps, it suggests that the person has a tendency to fly through the mind.

5. The method for generating and quantitatively evaluating psychiatric auxiliary medical records based on large language model reasoning and multimodal fusion according to claim 1, characterized in that, Step S4 specifically includes: Step S41: Based on psychiatric diagnostic and treatment standards and a suicide risk assessment scale, a preset anchor vector representing suicidal intent is established. ; Step S42, when the patient's conversation contains a set of sensitive words The sensitive word w in k When this occurs, the following risk detection will be triggered: Step S421, Context Window Extraction: Extract the sensitive word w k Context windows of the k tokens before and after , where k will dynamically expand or contract based on sentence length or semantic segmentation; Step S422, Semantic distance calculation: Calculate the context window semantic vectors Risk anchor vector The Euclidean distance D between them: If D < τ1, then a high risk of suicide is considered. If τ1≤D<τ2, then it is determined that there is a possibility of suicide risk. If D≥τ2, then it is determined that there is a low risk of suicide. Where τ1 and τ2 are both preset grading thresholds.

6. The method for generating and quantitatively evaluating psychiatric auxiliary medical records based on large language model reasoning and multimodal fusion according to claim 1, characterized in that, Step S5 specifically includes: Step S51: Based on the current psychomotor assessment conclusion, effective pathological statement set, assessment results of abnormal thought patterns and suicide risk classification results of the patient, a quantitative assessment vector is formed, and the equivalent standard scale reference score corresponding to the quantitative index is generated according to the pre-established calibration mapping relationship. Step S52: The quantitative assessment results, the reference scores of the equivalent standard scale, and the corresponding original evidence fragments are written into the standard electronic medical record auxiliary assessment record, and a corresponding quantitative assessment report is generated for reference.

7. A psychiatric auxiliary medical record generation and quantitative assessment system based on large language model reasoning and multimodal fusion for implementing the method of any one of claims 1 to 6, characterized in that, The system includes: The multimodal synchronous acquisition module simultaneously acquires the patient's speech and text streams and establishes a millisecond-level timestamp mapping. The acoustic pathology feature analysis module extracts features of response latency, speech rate variation coefficient, and fundamental frequency energy from the patient's speech stream based on signal processing algorithms, and performs corresponding psychomotor assessments. The semantic vector projection module uses a large language model to extract the embedding vectors of the patient's statements, calculates the similarity between them and the preset baseline vectors, and filters out the set of valid pathological statements. The thought entropy monitoring module uses a large language model to predict the probability distribution entropy of the next token, and combines the similarity between the word vectors generated in the current patient dialogue and the vectors of the preceding context to measure the coherence of their thinking, thereby identifying and classifying the abnormal characteristics of the patient's thought patterns. The context-based risk weighting module calculates the true threat level of risk words in patient conversations based on semantic distance, thereby conducting a graded potential assessment of suicide risk. The structured medical record generation engine is connected to the above modules to quantify the acoustic features, pathological projection scores, thought entropy values, and potential suicide risk levels obtained from calculation, evaluation, and / or grading, and fill them into the standard electronic medical record auxiliary assessment template to form the final quantitative assessment reference report.

8. A device for generating and quantitatively evaluating psychiatric auxiliary medical records based on large language model reasoning and multimodal fusion, characterized in that, The device includes: A processor is configured to execute computer-executable instructions; The memory stores one or more computer-executable instructions, which, when executed by the processor, implement the steps of the method for generating and quantitatively evaluating psychiatric auxiliary medical records based on large language model reasoning and multimodal fusion as described in any one of claims 1 to 6.

9. A processor for generating and quantitatively evaluating psychiatric auxiliary medical records based on large language model reasoning and multimodal fusion, characterized in that, The processor is configured to execute computer-executable instructions, which, when executed by the processor, implement the steps of the method for generating and quantitatively evaluating psychiatric auxiliary medical records based on large language model reasoning and multimodal fusion as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, It stores a computer program that can be executed by a processor to implement the steps of the method for generating and quantitatively evaluating psychiatric auxiliary medical records based on large language model reasoning and multimodal fusion as described in any one of claims 1 to 6.

Images