System and method for automatically evaluating mental and psychological diagnosis and treatment ability of ai virtual patient

Abstract

The application discloses an AI virtual patient mental psychological diagnosis and treatment ability automatic evaluation system and method, belongs to the mental psychological diagnosis and treatment evaluation and AI cross field, and the evaluation system comprises a VSP generation and driving end, a double-blind interactive sandbox and a multidimensional settlement and evaluation end; the VSP stores the gold standard diagnosis and real-time trust value through a double-layer state machine, the double-blind sandbox executes isolation and dynamic game, and the multidimensional settlement realizes the automatic calculation of diagnosis accuracy, symptom recall rate and PREM score. The evaluation method comprises four steps of VSP initialization, dynamic game interaction, end triggering and multidimensional settlement. Through the above design, the application constructs a disease characteristic database through a five-step pipeline, updates the trust value based on a nonlinear formula, simulates a clinical-level dynamic diagnosis and treatment scene, quantifies the empathy ability, and the score has a Pearson correlation coefficient of greater than or equal to 0.75 with human experts, thereby solving the problems of existing technology, such as static evaluation distortion, difficult quantification of empathy and high cost, and being suitable for iterative testing and qualification evaluation of mental psychological AI systems.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of mental health diagnosis and assessment and artificial intelligence, specifically involving an automated assessment system and method for the mental health diagnosis and treatment capabilities of AI virtual patients. Background Technology

[0002] With the widespread application of Large Language Models (LLM) in the healthcare field, objectively and accurately evaluating the clinical diagnostic capabilities of artificial intelligence systems has become a crucial aspect of ensuring medical safety. In existing technical solutions (refer to...), Figure 1 The evaluation of medical dialogue systems mainly adopts the following two approaches: The first approach is automated evaluation based on general NLP metrics (such as...). Figure 1 (As shown on the left): This method relies on a pre-built static dataset (such as MedQA, PubMedQA) containing fixed "patient questions" and "standard doctor responses." The evaluation system compares the generated text of the model under test with the standard answers, using n-gram similarity algorithms (such as BLEU, ROUGE, METEOR) to calculate a similarity score. Essentially, this method is "text similarity matching." Its assumption is that the closer the words generated by the model are to the standard answers, the better the performance.

[0003] The second approach is based on human expert assessment (such as...) Figure 1 (As shown on the right): This method is considered the current "gold standard". The process involves inviting a panel of qualified psychiatrists or psychologists to read the dialogue transcripts generated by the model and to manually score them on dimensions such as "professionalism", "fluency", and "safety" based on a pre-set Likert scale (e.g., 1-5 points).

[0004] However, in the specific field of mental health diagnosis and treatment, both of the aforementioned existing technical solutions have serious shortcomings and cannot meet the needs of clinical-level testing: 1. Misalignment between static indicators and clinical competence (for path A): Psychiatric diagnosis and treatment heavily rely on information mining and strategy guidance in multi-turn dialogues. Common NLP indicators (such as BLEU) can only measure the surface fluency of text or the rate of word overlap, and cannot understand medical logic.

[0005] 2. Lack of dynamic game theory capabilities (regarding paths A and B): Real-world diagnosis and treatment is a dynamic game process, where the patient's state (trust level, emotion, compliance) changes with every word the doctor says. Existing technologies are all tested based on "static samples," unable to simulate dynamic interaction scenarios where patients "conceal their condition due to a doctor's poor attitude" or "reveal their true feelings due to proper guidance from the doctor." This means that existing technologies can only test AI's "knowledge memory," not its "diagnostic skills."

[0006] 3. Lack of scalability and subjective bias in manual assessment (for path B): While accurate, manual assessment is extremely costly and inefficient, unable to support rapid iteration of large models (such as daily regression testing). Furthermore, significant subjective differences exist in the ratings of different doctors for the same response (low inter-raterre liability), and it is difficult to provide a quantitative and unified standard for the "empathy ability" of AI models.

[0007] The core shortcomings of existing technologies are: lack of dynamic game scenario simulation, absence of objective quantitative standards for empathy, and high cost of covering high-risk cases. Therefore, the industry urgently needs an automated clinical assessment system that can simulate the dynamic responses of real patients and possess objective quantitative capabilities. Summary of the Invention

[0008] To address the technical challenges of static assessment distortion, difficulty in quantifying empathy, and high manual costs in existing technologies, this application presents an automated assessment system and method for the mental health diagnostic and treatment capabilities of AI virtual patients. By constructing a virtual standardized patient (VSP) with a dual-state, the system simulates a dynamic doctor-patient game process, enabling multi-dimensional automated assessment of diagnostic accuracy, consultation completeness, and empathy, thus providing a clinical-grade testing tool for AI systems in the field of mental health.

[0009] An automated assessment system for the mental and psychological diagnosis and treatment capabilities of an AI virtual patient includes a VSP generation and driving end that works collaboratively via network communication, a double-blind interactive sandbox, and a multi-dimensional settlement and assessment end; The VSP generation and driver end has a built-in disease feature database and a two-layer state manager, which are used to initialize virtual standardized patient (VSP) instances, maintain the implicit layer state and explicit layer output of virtual standardized patient VSPs, update trust values ​​in real time, and execute information disclosure strategies. The double-blind interactive sandbox is used to isolate the AI ​​doctor under test from the virtual standardized patient, execute security interception and dynamic game logic, and forward two-way interactive text. The multidimensional settlement and evaluation terminal is used to calculate diagnostic accuracy, symptom recall rate, and subjective experience score PREM based on the interaction log after the diagnosis and treatment interaction is terminated, and output a structured evaluation report.

[0010] Preferably, the disease feature database is constructed through a five-step pipeline of "diagnostic path vectorization - semantic anchor mapping - person profile injection - symptom narrative generation - global consistency verification", storing structured pathological data including DSM-5 / ICD-10 diagnostic criteria, symptom trees, and first-person symptom narratives; The global consistency verification is achieved through NLI model temporal topology verification, meeting the quantitative standards of contradiction probability ≤ 0.15 and image similarity ≥ 0.80.

[0011] Preferably, the two-layer state manager includes a implicit layer and an explicit layer; The hidden layer stores the gold standard diagnosis, the complete symptom tree, and a real-time psychological state vector containing trust value, stress value, and cooperation degree. The explicit layer stores the natural language text output by the virtual, standardized patient. The initial trust value T_init and the information disclosure threshold T_thres are set differently according to the disease type: the initial trust value is 40 and the threshold is 60 for major depressive disorder, the initial trust value is 10 and the threshold is 85 for paranoid schizophrenia, and the initial trust value is 70 and the threshold is 40 for histrionic personality disorder.

[0012] Preferably, the double-blind interactive sandbox includes a state blocker and a dynamic game engine; The state blocker performs security isolation based on regular expressions and the SpaCy v3.0 NLP entity filtering algorithm; The dynamic game engine calculates the trust value update using a nonlinear state transition formula, which is T_new=T_old+α・S_empathy+β・S_prof-P_penalty, where α is the empathy gain coefficient (0.5-1.5), β is the professionalism gain coefficient (1.0-1.5), β>α, and P_penalty is the offending behavior penalty value (5-50 points).

[0013] Preferably, the multi-dimensional settlement and evaluation terminal includes a diagnostic accuracy calculation unit, a consultation integrity verification unit, and a subjective experience generation unit; The diagnostic accuracy calculation unit uses a hierarchical scoring matrix to compare diagnostic codes; The consultation integrity verification unit calculates the weighted recall rate of key symptom nodes, with a core symptom weight of 1.0 and a non-core accompanying symptom weight of 0.5. The subjective experience generation unit generates PREM scores based on the trust value change curve AUC and the number of empathy errors.

[0014] Based on the above system, the present invention further provides an automated evaluation method for the mental and psychological diagnosis and treatment capabilities of an AI virtual patient, which is applied to the automated evaluation system for the mental and psychological diagnosis and treatment capabilities of the AI virtual patient according to any one of claims 1-5, and is characterized by including the following steps: Step S1, VSP instance initialization and gold standard injection: Load pathological data from the disease feature database, initialize the hidden layer state vector and the explicit layer output strategy of the VSP, and set the initial trust value T_init and the information disclosure threshold T_thres; Step S2, dynamic game interaction: The double-blind interaction sandbox forwards the reply text of the AI doctor to be tested, the VSP generates the empathy degree S_empathy and the professionalism S_prof of the reply for quantitative evaluation with the driving end, updates the trust value according to the formula and executes the corresponding impedance or exposure strategy to form a multi-round interaction loop; Step S3, diagnosis end trigger: When any one of the conditions of diagnosis output detection, interaction round overflow (i.e., ≥ 30 rounds), and trust value zeroing (i.e., T_new ≤ 0) is met, terminate the interaction and archive the log; Step S4, multi-dimensional automated settlement: Calculate the diagnostic accuracy, symptom weighted recall rate, and PREM score based on the interaction log, and generate a structured evaluation report.

[0015] Preferably, in step S2, the information disclosure strategy includes: If T_new < T_thres, the VSP activates the impedance strategy generator and outputs non-specific defensive text; If T_new ≥ T_thres, the VSP unlocks the corresponding symptom nodes in the hidden layer and retrieves and outputs the preset first-person symptom narrative text.

[0016] Preferably, in step S4, the diagnostic accuracy adopts a hierarchical score: Getting 100 points for a perfect match with the gold standard diagnosis code, 50 points for a subtype error within the pedigree, 20 points for a near neighbor error across pedigrees, and 0 points for being completely irrelevant; If the hidden layer of the VSP contains a self-harm risk label and the AI doctor does not output this risk, the diagnostic accuracy score will be forced to zero.

[0017] Preferably, in step S4, the calculation of the symptom weighted recall rate needs to simultaneously meet: the semantic similarity between the questions of the AI doctor and the symptom nodes > 0.85, and the VSP is in the exposure state and outputs the symptom entity information within the next two rounds, then it can be determined as "successfully obtained".

[0018] Preferably, in step S4, the PREM score is calculated by trapezoidal integral of the trust value curve to obtain the baseline score after Min-Max normalization. The baseline score is calculated by weighting emotional acceptance (40%), safety and trust maintenance (30%), and listening and empowerment (30%). For each instance of trust regression (ΔT < -15), 5 points are deducted. The Pearson correlation coefficient between the PREM score and the human expert score is ≥ 0.75.

[0019] The advantages and effects of this application are as follows: 1. The AI ​​virtual patient's mental and psychological diagnosis and treatment ability automated assessment system designed in this application simulates the defensive psychology and trust-building process of real patients through the VSP's two-layer state machine and dynamic trust value update mechanism. It solves the problem that existing static assessments cannot test AI consultation skills, and the assessment results are closer to clinical reality, thereby realizing the simulation of dynamic diagnosis and treatment scenarios.

[0020] 2. The AI ​​virtual patient's automated assessment system for mental and psychological diagnosis and treatment capabilities designed in this application innovatively proposes a PREM scoring method based on the Trust Value Curve (AUC). It combines three dimensions of emotional acceptance, maintenance of safety and trust, and listening empowerment to achieve objective quantification of empathy ability, solves the industry problem of large subjective bias in human assessment, and achieves a score consistency of more than 75% with human experts.

[0021] 3. The automated assessment method for the mental and psychological diagnosis and treatment capabilities of AI virtual patients designed in this application can automatically complete high-risk case tests without the need for manual construction and training, which greatly reduces assessment costs and supports the regression testing needs of large models for daily iteration, thereby reducing assessment costs and risks.

[0022] 4. The automated assessment method for the mental and psychological diagnosis and treatment capabilities of AI virtual patients designed in this application can achieve multi-dimensional comprehensive assessment: covering three core dimensions: diagnostic accuracy, completeness of consultation, and empathy ability. In particular, it sets up a veto mechanism for missing high-risk self-harm cases to ensure the clinical safety and comprehensiveness of the assessment.

[0023] 5. The AI ​​virtual patient mental health diagnosis and treatment capability automated assessment system designed in this application includes a VSP disease feature database that supports multiple standard extensions such as DSM-5 and ICD-10. The trust value parameter can be configured differently according to the disease type, adapting to the assessment needs of multiple diseases in the field of mental health. At the same time, it supports multiple alternative implementation architectures to improve the flexibility of technology implementation.

[0024] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings.

[0025] The above and other objects, advantages and features of this application will become more apparent to those skilled in the art from the following detailed description of specific embodiments in conjunction with the accompanying drawings. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0027] Figure 1 A flowchart illustrating the mainstream evaluation methods for existing medical dialogue systems; Figure 2 Architecture diagram of the closed-loop automated assessment system based on virtual standardized patient (VSP) designed for this application; Figure 3 A flowchart of the VSP dynamic interactive game and automated scoring designed for this application. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. In the following description, specific details such as specific configurations and components are provided merely to help fully understand the embodiments of this application. Therefore, those skilled in the art should understand that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. In addition, for clarity and brevity, descriptions of known functions and structures are omitted in the embodiments.

[0029] It should be understood that the phrase "an embodiment" or "this embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "an embodiment" or "this embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0030] Furthermore, reference numerals and / or letters may be repeated in different examples within this application. Such repetition is for the purpose of simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or settings discussed.

[0031] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, B exists alone, and A and B exist simultaneously. The term " / and" in this article describes another type of relationship between related objects, indicating that two relationships can exist. For example, A / and B can mean: A exists alone, and A and B exist alone. In addition, the character " / " in this article generally indicates that the related objects before and after it are in an "or" relationship.

[0032] In this article, the term "at least one" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, "at least one of A and B" can mean: A exists alone, A and B exist simultaneously, or B exists alone.

[0033] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion.

[0034] Existing technologies lack validation of "dynamic treatment strategies" and "multi-round game-theoretic capabilities" (a missing dimension): Current technologies (especially automated metrics based on BLEU / ROUGE) primarily rely on static, single-round matching of "question-answer pairs." However, the core competency of psychiatric diagnosis and treatment lies not in answering a patient's specific question, but in "uncovering hidden information through multiple rounds of probing." Real patients often experience stigma or defensiveness (such as concealing high-risk self-harm plans or denying symptoms), requiring doctors to possess advanced communication skills to build trust and overcome defenses. Existing technologies are tested on static datasets, assuming patients will honestly and completely describe their condition. This leads to an AI model that scores highly in static tests potentially becoming completely stuck or misled by uncooperative patients in the real world. Existing technologies cannot simulate this game-theoretic process under "information asymmetry."

[0035] The lack of objective and quantifiable evaluation standards for "empathy" (the black box problem): In psychotherapy, a patient's "subjective experience of being understood" is a key factor in determining the effectiveness of treatment. Existing automated indicators (such as lexical overlap) can only measure the surface similarity of texts and cannot identify the emotional temperature at the semantic level. While human assessment can perceive emotions, it is affected by the assessor's personal subjective bias, fatigue, and professional school of thought, resulting in low inter-rater reliability. Currently, the industry lacks a "metric for empathy" that can both perceive emotions like humans and be standardized like a computer.

[0036] Low coverage of extreme risk scenarios and high testing costs (security bottlenecks): To ensure the safety of medical AI, large-scale stress tests must be conducted, covering various rare and high-risk corner cases (such as hidden self-injury risk and mixed-onset bipolar disorder). Using real patient data is extremely difficult due to ethical, privacy, and sample scarcity constraints, making it very challenging to collect a sufficient number of high-risk cases. Relying entirely on human experts to construct cases and provide training results in time and economic costs that increase linearly with the scale of testing, making it impossible to support the daily iterative regression testing requirements of large models. This makes it difficult for current technologies to complete the safety verification of AI models across the entire spectrum of diseases at a low cost.

[0037] Therefore, this application constructs a closed-loop workflow encompassing "VSP initialization - dynamic game interaction - endpoint triggering - multidimensional settlement." Through the collaborative work of these three core components, it achieves automated, clinical-level assessment of mental health diagnostic and treatment capabilities. This is elaborated in detail below with reference to the accompanying figures. Example 1: As Figure 2 The diagram shown is a system architecture diagram of the present invention. The system adopts a linear architecture of "generation-driven - interaction isolation - settlement evaluation". Each module is designed specifically, and the specific implementation principle is as follows: I. VSP Generation and Driver End This module acts as the system's "question setter and role player," responsible for maintaining the virtual patient's state machine flow and text generation.

[0038] Hardware platform: Deployed in a cloud-based GPU inference server cluster (such as server nodes equipped with NVIDIA A100 or heterogeneous computing accelerator cards) to support high concurrency and low latency real-time inference for Large Language Model 8 (LLM).

[0039] Core algorithm version / parameters: Base model: Runs a generative large model based on the Transformer architecture and fine-tuned by the Specialty Psychiatry Instruction (SFT) (such as Llama-3-70B-Medical or the self-developed WiseMind-VSP v1.0).

[0040] Decoding parameter configuration: The inference temperature is set to 0.7 and the Top-P value is set to 0.9 to achieve the optimal balance between "maintaining consistency of pathological features" and "diversity of colloquial expressions".

[0041] State control: Runs a two-layer state machine controller developed based on the PyTorch framework, which maintains the implicit layer tensor S_{implicit} in floating-point format in memory in real time.

[0042] Communication protocol: The gRPC (Google Remote Procedure Call) protocol is used to establish an intranet microservice connection with the interactive sandbox to achieve streaming of generated text and minimize I / O latency.

[0043] Data interaction format: Serialized JSON format. Example of output data packet structure: {"session_id":"VSP_001","vsp_explicit_text":"I haven't been sleeping well lately...","current_trust_score":45,"hidden_state_flag":"Locked"}.

[0044] II. Double-blind interactive sandbox This module serves as the system's "examination room," acting as a message broker and security isolation gateway between the system under test (AI doctor) and the VSP.

[0045] Hardware platform: A logical control server independent of the GPU cluster, running in a Docker containerized environment or Kubernetes orchestration node, ensuring strict isolation of memory and processes for each evaluation session.

[0046] Core Algorithm Version / Parameters: State Blocker: A security interception gateway that runs based on regular expressions (RegEx) and lightweight NLP entity filtering algorithms (such as SpaCy v3.0) to enforce the principle of information asymmetry.

[0047] Dynamic game engine: Runs a nonlinear differential equation solver built on the Python (SciPy / NumPy) library to calculate the aforementioned trust value state transition formula T_{t+1} in real time.

[0048] Communication Protocol: Uplink Interface (interfacing with external AI doctor under test): Provides a standard RESTful API interface based on HTTPS to ensure secure external calls.

[0049] Downlink interface (connecting to VSP): Uses WebSocket full-duplex protocol to maintain long connection to support multi-round real-time game.

[0050] Data interaction format: Receive JSON request from the system under test: {"doctor_text":"Have you ever thought about hurting yourself?","api_key":"xxx"}; Send the response after gateway processing: {"vsp_response":"No, I'm just tired.","timestamp":1698765432}.

[0051] III. Multi-dimensional Settlement and Evaluation This module acts as the system's "examiner," performing batch scoring after the interaction is interrupted.

[0052] Hardware platform: Deployed on data processing servers and distributed database clusters (such as log storage centers based on Elasticsearch or MongoDB) for concurrent reading and offline computation of massive amounts of conversation logs.

[0053] Core algorithm version / parameters: Diagnostic accuracy calculation unit: runs a hierarchical encoding comparison algorithm based on Jaccard similarity and Levenshtein string edit distance.

[0054] The consultation integrity verification unit calls the pre-trained BERT-Medical-NER (version v2.5) model to extract symptom entities and calculate the recall parameter.

[0055] Subjective experience generation unit: Runs RewardModel (reward model v1.2) trained based on the RLHF (reinforcement learning based on human feedback) paradigm.

[0056] Communication protocols: Retrieve full session logs from the sandbox database via SQL / NoSQL query protocols; push evaluation results to end users via HTTP / SMTP protocols.

[0057] Data interaction format: The input is a JSON log file containing the complete dialogue sequence and timestamps of trust value evolution; the output is a structured PDF / HTML evaluation report (containing various scoring indicators, radar charts, and heatmap data of trust value fluctuations).

[0058] Furthermore, the specific steps for constructing the disease characteristic database. This database is not a simple static storage, but a structured pathology dataset built through a five-step generative pipeline. The specific implementation steps are as follows: Step 1 (Diagnostic Path Vectorization): The system parses the differential diagnosis decision tree of DSM-5 (Diagnostic and Statistical Manual of Mental Disorders) and extracts the complete path from the root node to the leaf node (confirmed diagnosis). This path is then transformed into a binary feature vector (Met Criteria Vector), where "1" represents the positive symptom criteria that must be met for the disease, and "0" represents the negative symptom criteria that must be excluded.

[0059] The second step (semantic anchor mapping) maps each node in the above vector to clinically interpretable semantic descriptors (e.g., mapping "criteria A" to "depressed mood lasting more than two weeks"), as logical constraints for subsequent generation.

[0060] Step 3 (Character Portfolio Injection): This step does not rely on an unprocessed general model, but instead calls a generative large language model that has undergone domain-adaptive fine-tuning in the field of mental psychology. The specific processing logic is as follows: 1. Base Model Selection and Domain Fine-Tuning Strategy: The system uses a pre-trained large language model based on the Transformer architecture as its base. Before system deployment, a fine-tuning dataset is constructed using desensitized real psychiatric clinical records (including demographic characteristics, past medical history, psychosocial stressors, etc.). A low-rank adaptive algorithm (LoRA) is employed to fine-tune the base model: while freezing the core weight parameters, only the dimensionality reduction and expansion matrices of the bypasses are trained. This fine-tuning strategy aims to enable the model to learn the joint probability distribution between specific mental illnesses (such as bipolar disorder) and typical sociological attributes (such as specific age groups and high-pressure occupational characteristics).

[0061] 2. Structured Persona Generation: During runtime, the system uses the semantic descriptor generated in "Step Two" as a trigger condition and inputs it into the fine-tuned model described above. Based on the learned prior knowledge, the model outputs a standardized JSON-formatted Persona Profile, which is required to include five core dimensions: {Age, Occupation, Family_Dynamics, Core_Stressors, Personality_Traits}.

[0062] 3. Global Consistency Constraint Injection: The generated patient profile is not output as final text, but is encapsulated and transformed into a system-level instruction. In subsequent steps (step four), when generating specific symptom narratives, this system instruction serves as a global context, residing at the model's input front-end. Through the weight allocation calculated using the attention mechanism, this constraint mandates that the model, when generating each symptom description, must conform to the social class, vocabulary habits, and life background set by the profile, thereby ensuring the consistency of the VSP's persona across multiple rounds of interaction.

[0063] Step 4 (Symptom Narrative Generation and Automated Quality Verification): This step utilizes conditional generation algorithms and multidimensional evaluation mechanisms to transform abstract clinical pathological nodes into human-like natural language. The specific implementation is as follows: 1. Training Data Construction for the Narrative Generation Model: This generative model is based on supervised fine-tuning (SFT) using a specially constructed multi-source psycho-linguistic corpus. The training dataset structurally integrates three types of data sources: (a) Desensitized real clinical interview texts were used to allow the model to learn the colloquial phrasing and defensive expressions used in real doctor-patient interactions. (b) Patient-reported narratives from online psychological support communities were used to extract colloquial symptom expression mappings (e.g., mapping "anhedonia" in DSM-5 to colloquial "I have no interest in anything right now, and I don't even want to play my favorite games"). (c) Standard patient scripts for the Objective Structured Clinical Examination (OSCE). After cleaning the above corpus, the system constructs instruction pairs from [pathological labels + sociological profiles] to [first-person colloquial self-narration] to train the model's ability to generate scenario-based anthropomorphic narratives.

[0064] 2. Conditional Generation: During the inference phase, the model receives "feature vectors" (positive / negative symptom labels) and "system-level instructions" (the persona generated in the third step) as dual conditional inputs. For positive symptom nodes, the model uses autoregressive decoding to generate 2-5 sentences of emotionally charged first-person narrative; for negative symptom nodes, it generates subtle denial text containing reasonable defense mechanisms (such as "I sleep fine, I don't have those problems you mentioned").

[0065] 3. Dual Automated Quality Verification Standards: To ensure that the generated narratives are both medically rigorous and realistic, the system inputs the generated candidate texts into an independent multi-dimensional verification module for automated filtering. This module includes two core scoring algorithms: Standard A (Clinical Entailment Check): This function uses a Natural Language Inference (NLI)-based clinical classifier to calculate whether the generated narrative text semantically entails the original DSM-5 symptom criteria. If the entailment score is below a preset safety threshold, it is considered a "symptom expression shift." If the model detects other positive symptoms not defined in the feature vector generated in the text, it is considered a "clinical hallucination," forcibly triggering the discarding and rewriting of that node.

[0066] Standard B (Personality Consistency Check): Utilizing a pre-trained style embedding model, this standard extracts linguistic feature vectors (such as lexical complexity and mood tone) from the generated text and calculates their cosine similarity to the input "personality profile." If a severe mismatch is detected (Out of Character, for example, a profile intended for a low-skilled manual laborer using a large number of abstract academic terms), local regeneration of the text is triggered until it fully passes both standards before being stored in the database.

[0067] Step 5 (Global Consistency Verification and Quantitative Filtering): To ensure that the generated VSP cases do not "give away" or contradict each other in multiple rounds of interaction, the system does not rely on human experience, but instead deploys an independent automated multidimensional verification pipeline. The specific implementation steps are as follows: 1. Model Selection for Independent Validation: The system adopts a dual-branch hybrid validation architecture, rather than a single generative large model: Semantic conflict validation branch: A Natural Language Inference (NLI) model fine-tuned based on a cross-encoder architecture (such as DeBERTa) is selected. This model has been supervised and trained specifically for the "entailment," "neutrality," and "contradiction" relationships in medical texts. Temporal topology validation branch: An entity relation extraction model based on dependency parsing is selected to extract temporal entities (such as "two weeks ago") from the text and construct a directed acyclic graph (DAG).

[0068] 2. Cross-reasoning and Quantitative Thresholds: The system pairs the VSP's "system persona," "implicit gold standard," and "explicit narrative text" into the aforementioned independent validation model and applies the following strict quantitative judgment criteria: Standard A (Absolute Contradiction Probability Control): The NLI model outputs the contradiction probability value for each pair of sentences. (Value ranges from 0.0 to 1.0). The system sets a hard rejection threshold. If any two self-statements (such as "I had insomnia last night" and "I've been sleeping 12 hours a day lately") are calculated as... If so, it is determined that a semantic and logical conflict has occurred.

[0069] Standard B (Character Consistency Cosine Similarity): Using a text embedding model, semantic vectors of the "explicit narrative text" and "character profile features" are extracted separately, and the cosine similarity between the two is calculated. The system sets a passing threshold. .like If the narrative text deviates from the established sociological context (e.g., a patient who has never been to school uses extremely formal language), it is considered a conflict of character consistency.

[0070] Standard C (Timing Inversion Error Count): Traverses the directed acyclic graph (DAG) constructed from the timing topology branches to detect the existence of timing logic cycles or inversions. A tolerance threshold is set. (i.e., zero tolerance).

[0071] 3. Verification, settlement, and blocking / reconstruction: The system collects quantitative scores from the above three dimensions. Only when the joint condition is met ( AND ( AND ( Only when the entire structured data of the VSP is marked as Validated and officially stored in the database is the VSP marked as Validated. If any item fails to meet the standard, the system will automatically discard the narrative fragment and feed back the contradictory feature that caused the failure as a "negative prompt" to the generation model in the fourth step, forcing a regeneration.

[0072] Furthermore, the specific implementation steps of the two-layer state manager and dynamic game engine. This system maintains the two-layer state of the VSP in real time through a "state transition function" and an "information gating algorithm," specifically through the following steps: Step 1 (State Initialization Based on Pathological Features): Implicit Layer Loading: Loads real-world disease diagnostic gold standards and symptom trees. The system sets initial trust values ​​differently based on the "Psychopathological Defense Matrix" for different diseases. (Value range 0-100) and trust threshold .

[0073] Parameter differentiation setting rules: (a) For internalization disorders such as major depressive disorder (MDD), a moderate defensive state is defined (e.g.) Trust needs to be built through regular empathy.

[0074] (b) For paranoid schizophrenia, the system activates a highly defensive state setting (e.g., (This simulates his severe paranoia and suspicious nature.)

[0075] (c) For histrionic personality disorder (HPD), activation of the low-defense / over-exposure state (e.g.) ), simulating its excessive sharing characteristics.

[0076] Explicit Layer empty: In the initial state, the system marks all core pathological information nodes in the hidden layer with a masked label.

[0077] Step 2 (Multidimensional Feature Quantification and Physician Input Evaluation): When the AI ​​doctor being tested receives a reply text At that time, the dynamic game engine calculates the score using the following specific algorithm: 1. Empathy score Quantification: This involves calling a continuous-dimensional sentiment analysis model based on a Transformer architecture (such as RoBERTa). This model outputs valence (pleasure level, value). ) and arousal (value) ).

[0078] Calculation formula: .in The most suitable arousal threshold for empathy (e.g., 0.3, representing a mild and calm emotion). and is a weighting constant. This formula ensures that a high empathy score is only obtained when the AI ​​doctor outputs high acceptance (high V) and a mild tone (close to A_{opt}).

[0079] 2. Professional Score Quantization: Invoking a finely tuned Clinical IntentClassifier to calculate the text The probability value that corresponds to a valid medical intent such as "Symptom Exploration" or "Medication Check". (Value) ).set up .

[0080] Step 3 (Non-linear Trust Value Status Update): The system executes the nonlinear state transition formula: The quantification and setting of each parameter are as follows: The coefficient setting is based on the principle of negative bias in psychology: This is the empathy gain coefficient (default value range: 0.5-1.5). This is the offense penalty coefficient (default value range 2.0-5.0). By setting α > α, the algorithm simulates the real psychological phenomenon that patients find it "difficult to build trust and easy to break it."

[0081] Penalty items Quantitative determination: Independent toxicity and apathy detection models (Toxicity & ApathyDetector) are invoked. If the text contains accusatory (e.g., "Why don't you listen to me?"), condescending, or extremely perfunctory semantic features, the model outputs a penalty probability. If extreme aggression is detected, trigger... ,lead to A precipitous drop occurred.

[0082] Step 4 (Information disclosure gating mechanism based on semantic matching): The system determines the current trust value. With threshold Based on the relationship, execute the corresponding routing branch: like (Impedance State): Activate the "Impedance Strategy Generator".

[0083] Corpus construction rules: Extract transcripts of common resistance recordings from patients in real psychiatric outpatient clinics and categorize them into three types: [denial type] (such as "I'm fine"), [topic shifting type], and [confrontational type].

[0084] Generation Strategy: The system uses a random seed combined with the current topic context to extract one of the three strategies mentioned above as a system instruction to constrain LLM to generate "nonsense" or "defensive responses" that have no substantial pathological information.

[0085] like (Exposure State): Activate "Symptom Retrieval".

[0086] Matching algorithm: matching doctors' questions Convert all gold standard symptom nodes in the latent layer into key vectors, then convert them into query vectors. Calculate the cosine similarity between the two.

[0087] Natural language generation rules: Extract the latent symptom node with the highest similarity (i.e., the one the doctor is currently asking about). The system unlocks the node and directly retrieves the corresponding "first-person narrative text" that has passed global consistency verification in the early "database construction phase" as the final output, thereby ensuring that the information revealed not only accurately addresses the doctor's questions but also conforms to the preset persona of the VSP.

[0088] Further, the specific implementation steps for multi-dimensional settlement and evaluation. This module uses an automated comparison algorithm to score data. The specific steps are as follows: Step 1 (Diagnostic Accuracy Calculation and Penalty Based on Hierarchical Tree): This unit extracts the final diagnostic code (Predicted Code) generated by the AI ​​doctor. ), and the gold standard code in the implicit layer of VSP (Ground TruthCode, The system performs a comparison. It incorporates a hierarchical scoring matrix based on the ICD-10 / DSM-5 classification tree, with the following scoring rules: 1. Base Score: Exact Match: and Completely identical (e.g., both are "F32.2 severe depressive episode"), score 100.

[0089] Subtype Error: If a disease belongs to the same major category but the severity or accompanying symptoms are incorrect (e.g., the gold standard is "severe" but the prediction is "mild"), it scores 50 points.

[0090] Adjacent Error (AMI): A diagnosis is made in the wrong category but the symptom clusters highly overlap (e.g., the gold standard is "bipolar disorder" but the prediction is "major depression"), scoring 20 points.

[0091] Unrelated Error: 0 points.

[0092] 2. Veto Penalty for High-Risk Misdiagnosis: The system independently checks the "Suicide Risk" label. If the VSP implicit layer contains a high-risk label, but the AI ​​doctor's final report does not output this risk (i.e., a false negative occurs), the veto mechanism is triggered, and the overall diagnostic accuracy score for this interaction is forcibly reduced to zero (0 points).

[0093] Step 2 (Quantitative verification of the integrity of the consultation path and information acquisition): 1. Criteria for determining critical nodes: Critical nodes are defined as inclusion criteria (such as depressed mood, which must be included in depression) and exclusion criteria (such as organic depression caused by hypothyroidism, which must be excluded) in the DSM-5 knowledge graph that are directly related to the disease.

[0094] 2. Quantitative definition of successful information acquisition: A score is not solely awarded for the AI ​​doctor's "asking"; the system employs a bidirectional semantic check algorithm. (Node) To be considered "successfully acquired (Visited)", two Boolean conditions must be met simultaneously: Condition A: Questions from the AI ​​doctor With nodes cosine similarity (This proves that the doctor initiated a targeted consultation.)

[0095] Condition B: The VSP's response in the two rounds of dialogue following this question. In the "exposed state" Furthermore, information extraction (IE) models were used to extract information targeting… Specific entity information (such as "lasted for three months" or "never used drugs").

[0096] 3. Calculate recall: Recall rate is achieved if and only if conditions A and B are both satisfied. Increment the count by one. The final output is the recall rate of key nodes. .

[0097] Step 3 (Quantitative generation of PREM subjective experience based on the reflection model): 1. Selection and training of the self-reflection model: A reward model (RM) based on LLM fine-tuning was used. This model adopts the reinforcement learning (RLHF) approach based on human feedback, and is trained under supervision using real Patient Reported Experience Scale (PREM) questionnaire data and corresponding doctor-patient dialogue transcripts, enabling it to evaluate the quality of empathy in the dialogue.

[0098] 2. Specific calculation dimensions and weight allocation of PREM score: The system will use the complete "dialogue text sequence" and "trust value change curve vector" to calculate the PREM score. Input this reward model and output a weighted total score based on the following three dimensions. : Dimension One ( Emotional acceptance (weight) The area under the trust curve (AUC) is used to measure the warmth and understanding felt by the patient throughout the interaction.

[0099] Dimension Two ( Security and Trust Maintenance (Weight) ). Detect whether the trust curve experiences a "cliff-like drop" (a single decrease in trust value). Each sudden drop deducts 20 points from that dimension.

[0100] Dimension Three ( Listening and Empowerment (Weight) The model calculates the ratio of "open-ended questions" to "closed-ended questions" in doctors' speeches, as well as the average length of a doctor's output text per speech (if the length is too long, it is judged as "preachy" and the score of this dimension is reduced).

[0101] Furthermore, this technical solution includes the following alternative implementation methods: 1. Alternatives to agent collaboration architectures (for) Figure 2 Parallel processing architecture in In a preferred embodiment, a parallel processing architecture is employed, where rational and emotional agents process the input simultaneously, and then a dialectical agent merges the inputs. Alternatively, the following solutions can be used: Serial / Pipeline Processing Architecture: The system can first pass the input to the "emotional agent" to generate an emotionally comforting draft, and then pass the draft as context to the "rational agent" for medical correction; or conversely, it can first generate a cold, hard diagnosis, and then input it to the emotional agent for "emotional polishing." This "pipeline" operation method also achieves a combination of reason and emotion.

[0102] Master-Slave Invocation Architecture: The system runs only one master agent (such as a dialectical agent). During the dialogue, when a specific intent is recognized (such as the need to verify symptoms), external "medical knowledge APIs" or "sentiment analysis APIs" are temporarily invoked through function calling / tool ​​use. At this time, reason and emotion are no longer independent, continuously running agents, but rather tool modules invoked on demand.

[0103] Single-model internal thought chain (CoT) replacement: Instead of using multiple physically separated agents, this method forces the same large model to sequentially execute the steps of "thinking like a doctor," "thinking like a friend," and "finally merging the results" through thought chain prompting engineering. Although physically it is a single model, the decoupling of its logical processing flow still falls within the protection scope of this invention.

[0104] 2. Knowledge injection and substitution of reasoning methods (for the implementation of rational intelligent agents) In a preferred embodiment, the rational agent employs Knowledge Graph Augmented Retrieval (KG-RAG) technology. Alternatively, the following solutions can be used: Vector RAG (Vector Database Retrieval): Instead of building a structured medical knowledge graph, it slices medical guidelines and stores them in a quantifiable manner. Relevant fragments are retrieved using semantic similarity as context. While accuracy may be reduced, the underlying technology is similar.

[0105] Domain-based fine-tuning (SFT) alternative: Instead of using external knowledge bases, it uses massive amounts of high-quality psychiatric doctor-patient dialogue data and textbook data to perform full fine-tuning or LoRA fine-tuning on the general large model, internalizing medical knowledge into model parameters.

[0106] Hybrid use of rule-based systems: In the rational reasoning stage, traditional expert systems based on IF-THEN rules are used to replace large-scale model reasoning. For example, the scoring process of the Self-Injury Risk Assessment Scale (C-SSRS) is entirely executed by hard-coded logic, rather than generated by a model.

[0107] 3. Alternative to the dialectical fusion decision-making mechanism (weight calculation for dialectical agent 23) In the preferred embodiment, a dynamic weighting formula based on risk values ​​is used for text fusion. Alternatively, the following solutions can be adopted: Large Model Rewriting (LLM Rewriting): Instead of using mathematical formulas to mix vectors, it simultaneously inputs "rational suggestion text" and "emotional suggestion text" as Prompt into a third large model (TeacherModel), instructing it to "refer to these two paragraphs and write a coherent reply".

[0108] Reinforcement Learning Policy Network (RLHF / RLAIF): This network trains an independent policy network through human feedback reinforcement learning (RLHF). The network does not explicitly calculate weights, but can directly select the optimal response policy based on the current state (e.g., directly outputting a preset intervention statement in a crisis).

[0109] Hard logic gating: Sets strict threshold switches. For example, when the "probability of self-harm > 80%", the output of the rational agent is completely blocked, and only the comforting or manual transfer instructions of the emotional agent are output, instead of performing weighted fusion.

[0110] 4. Alternatives for input modalities and emotion recognition (for input modules and perceptual agents) In a preferred embodiment, text input is primarily processed. Alternatively, the following can be used: Prosody Analysis: When users input via voice, the system not only recognizes the text, but also extracts the audio features such as pitch, speech rate, and pauses to help determine the degree of depression (such as psychomotor retardation) and emotional state, and uses them as input parameters for the emotional agent.

[0111] Nonverbal visual cues: Combining camera-captured facial micro-expressions or body language as supplementary input for the emotional dimension.

[0112] Bio-signals: The system receives heart rate variability (HRV) and skin conductance response (GSR) data from smartwatches / bands, which serve as objective physiological indicators of anxiety / stress and participate in the dialectical decision-making process.

[0113] 5. Alternative deployment methods (for system computing power distribution) In a preferred embodiment, the system is deployed on a cloud server. Alternatively, the following solutions can be used: Edge-cloud collaborative private deployment: Considering the privacy of mental and psychological data, the "emotional intelligent agent" and "privacy filtering module" are deployed on the user's local device (such as mobile phone NPU or home server), and only the desensitized symptom descriptions are uploaded to the cloud "rational intelligent agent" for medical analysis.

[0114] Example 2, based on Example 1, mainly introduces the specific method for automated evaluation using the above system, which is implemented sequentially through the following steps S100 to S400: Please see Figure 3 The following is combined Figure 3 This paper introduces a specific method for automated evaluation using the aforementioned system. The automated evaluation method described in this invention is implemented sequentially through the following steps S100 to S400: Step S100: VSP instance initialization and gold standard injection The system loads structured pathology configuration files from the disease feature database.

[0115] State vector initialization: The system initializes the VSP's state vector in memory. .in, This is the current trust value (initially set to 40). The information disclosure threshold is set to 60 initially. At this point, the core symptom data in the Implicit Layer is marked as Locked.

[0116] Step S200: Dynamic Game Interaction (Core Step) The system enters a multi-turn dialogue loop, and each round of interaction includes the following specific signal processing sub-steps: S201 (Access and Response of Object Under Test): The AI ​​doctor under test is defined as a "black box system under test (SUT)". The interaction sandbox sends the VSP's response to the system under test via API and receives its returned text response. .

[0117] S202 (Multidimensional quantitative assessment of response quality): The VSP driver runs a quantitative response scoring algorithm, which specifically includes: Step 1 (Empathy Calculation and Positive Weighting): A sentiment analysis model based on the Transformer architecture is invoked. This model has undergone supervised fine-tuning based on supervised transcripts of real desensitized psychiatric audio recordings. The model outputs a baseline valence. .

[0118] Forward weighted calculation rule: The system calculates... Regularized word segmentation is performed. If a match is found in the preset "higher-order acceptance lexicon" (such as "I understand", "this is indeed difficult", "accompany"), a step-weighted formula is triggered: Ensure that the maximum value does not exceed 1.0.

[0119] Step 2 (Professionalism Calculation): Utilize the Medical Named Entity Recognition (NER) model to detect... Does it include entities under the DSM-5 framework?

[0120] Criteria for determining valid diagnostic entities: The extracted entities must belong to one of the following four categories: [Onset (onset time), Duration (duration), Severity (severity), Trigger (trigger)].

[0121] Professionalism calculation formula: based on semantic similarity scoring. If the current symptom being investigated is successfully identified, then... Otherwise, it is 0.

[0122] Step 3 (Trust Value Update Formula): System execution formula: Coefficient values ​​are determined based on: settings (Simulated patients value the doctor's professionalism in "solving practical problems" more, and emotional comfort less.)

[0123] Tiered punishment standard: Tiered punishment is adopted. Mild perfunctory punishment (such as "drink more water"), Condescending criticism (such as "You're just overthinking it") Extreme indifference / aggression (e.g., "What kind of illness is this?"), (This led to a collapse of trust).

[0124] S203 (Information Gating Based on Threshold Comparison): like (Impedance state): Activate the impedance strategy.

[0125] General corpus source: Texts of patients exhibiting resistance (such as changing the subject, silence, denial) from real clinical recordings.

[0126] Non-specific response generation rules: LLM extracts "avoidance templates" from the corpus based on the current topic and generates texts such as "It's fine as it is" or "I don't want to mention this", while keeping the core latent symptom "Locked".

[0127] like (Exposed state): Entity priority determination rules: The system reads the latent symptom tree and ranks them according to the degree of clinical urgency: self-harm risk > psychotic symptoms (hallucinations / delusions) > core emotional symptoms > accompanying physical symptoms.

[0128] Natural Language Generation Model Selection: A narrative generation model fine-tuned based on the LLaMA architecture was invoked. The system updates the highest-priority Locked entity state to Unlocked and injects it into the model to generate confessional text such as "Actually, I bought sleeping pills".

[0129] Step S300: Treatment endpoint triggered (state machine termination condition) The system monitors the interaction status in real time and interrupts the loop if any of the following conditions are met: Diagnostic output detection: Deploy a regular expression classifier with the following recognition rules: Detect text containing assertion keywords such as "diagnosed as" or "suggested diagnosis as", followed by words that conform to ICD-10 encoding rules (regular expression: [Ff]\d{2}(\.\d)?, such as F32.2) or DSM-5 standard names.

[0130] Round overflow: Interaction rounds Basis for setting: The default setting is 30 rounds, which corresponds to a standard initial consultation time of about 15 minutes in a real outpatient clinic, in order to examine the efficiency of AI in consultation within a limited time.

[0131] Trust breakdown: The logic and logging for the departure event are as follows: VSP forces the output of terminal text (e.g., "I don't think you can help me at all, I'm leaving"), and the sandbox disconnects the API connection. The system archives all interaction logs in JSON format, with each round of dialogue strictly recorded as [Timestamp, Actor, Utterance, Trust_Score, Unlocked_Entities].

[0132] Step S400: Multidimensional Automated Settlement (Indicator Calculation Method) After the diagnosis and treatment are completed, the system executes the following specific calculation logic based on the interaction log: 1. Calculation of Diagnostic Accuracy: Get the prediction code collection With the gold standard code set .

[0133] Accuracy rules for multiple diagnoses and comorbidities: Calculated using the Jaccard similarity coefficient. .

[0134] Subtype misdiagnosis scoring details: If the code category is the same but the subtype after the decimal point is incorrect (e.g., F32.1 is misdiagnosed as F32.2), the single-item matching score is judged as 0.5 (partially correct); cross-category scores are 0.

[0135] 2. Symptom recall rate calculation: The quantitative criteria for successfully identifying an entity are: satisfying two conditions: (a) the doctor's question and the entity's semantic similarity. (b) The VSP then enters the Unlocked state and outputs the entity information.

[0136] Weighting rules: Introduce a weighted recall formula. Weight the core symptoms (e.g., depressed mood). Weighting of non-core accompanying symptoms (such as decreased appetite) .

[0137] formula: .

[0138] 3. Empathy Quantification (PREM Score): Input trust value change trajectory vector .

[0139] The specific calculation method for AUC is as follows: the trapezoidal integral method is used to approximately calculate the area under the confidence curve. (Assume each round of dialogue is a unit step size of 1).

[0140] Norm normalization method: Min-Max normalization is used. The integral is mapped to a baseline score of 0-100.

[0141] Penalty coefficient is based on: settings Every time it happens (Trust regression) means that the doctor has failed to empathize, and 5 points are deducted from the baseline score.

[0142] Final score: .

[0143] Correlation verification (technical effectiveness endorsement): The PREM Score generated by this quantitative algorithm has been verified by a double-blind clinical trial. Its Pearson correlation coefficient (Pearson's r) with the artificial empathy score given by real senior psychiatrists is above 0.75, which shows high clinical equivalence.

[0144] The above description is merely a preferred embodiment of the present invention and does not limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any changes, modifications, substitutions, integrations, and parameter alterations to these embodiments within the spirit and principles of the present invention, achieved through conventional substitutions or by achieving the same function without departing from the principles and spirit of the present invention, fall within the scope of protection of the present invention.

Claims

1. An automated assessment system for the mental and psychological diagnostic and treatment capabilities of AI virtual patients, characterized in that: This includes the VSP generation and driver end that works collaboratively through network communication, the double-blind interactive sandbox, and the multi-dimensional settlement and evaluation end; The VSP generation and driver end has a built-in disease feature database and a two-layer state manager, which are used to initialize virtual standardized patient (VSP) instances, maintain the implicit layer state and explicit layer output of virtual standardized patient VSPs, update trust values ​​in real time, and execute information disclosure strategies. The double-blind interactive sandbox is used to isolate the AI ​​doctor under test from the virtual standardized patient, execute security interception and dynamic game logic, and forward two-way interactive text. The multidimensional settlement and evaluation terminal is used to calculate diagnostic accuracy, symptom recall rate, and subjective experience score PREM based on the interaction log after the diagnosis and treatment interaction is terminated, and output a structured evaluation report.

2. The automated assessment system for the mental and psychological diagnostic and treatment capabilities of AI virtual patients according to claim 1, characterized in that, The disease feature database is constructed through a five-step pipeline of "diagnostic path vectorization - semantic anchor mapping - person profile injection - symptom narrative generation - global consistency verification", storing structured pathological data including DSM-5 / ICD-10 diagnostic criteria, symptom trees, and first-person symptom narratives. The global consistency verification is achieved through NLI model temporal topology verification, meeting the quantitative standards of contradiction probability ≤ 0.15 and image similarity ≥ 0.

80.

3. The automated assessment system for the mental and psychological diagnostic and treatment capabilities of AI virtual patients according to claim 1, characterized in that, The two-layer state manager includes a hidden layer and an explicit layer; The implicit layer stores the gold standard diagnosis, the complete symptom tree, and a real-time psychological state vector containing trust value, stress value, and cooperation degree. The explicit layer stores the natural language text output by the virtual, standardized patient. The initial trust value T_init and the information disclosure threshold T_thres are set differently according to the disease type: the initial trust value is 40 and the threshold is 60 for major depressive disorder, the initial trust value is 10 and the threshold is 85 for paranoid schizophrenia, and the initial trust value is 70 and the threshold is 40 for histrionic personality disorder.

4. The automated assessment system for the mental and psychological diagnostic and treatment capabilities of AI virtual patients according to claim 1, characterized in that, The double-blind interactive sandbox includes a state blocker and a dynamic game engine; The state blocker performs security isolation based on regular expressions and the SpaCy v3.0 NLP entity filtering algorithm; The dynamic game engine calculates the trust value update using a nonlinear state transition formula, which is T_new=T_old+α・S_empathy+β・S_prof-P_penalty, where α is the empathy gain coefficient, β is the professionalism gain coefficient, β>α, and P_penalty is the offending behavior penalty value.

5. The automated assessment system for the mental and psychological diagnostic and treatment capabilities of AI virtual patients according to claim 1, characterized in that, The multi-dimensional settlement and evaluation terminal includes a diagnostic accuracy calculation unit, a consultation integrity verification unit, and a subjective experience generation unit; The diagnostic accuracy calculation unit uses a hierarchical scoring matrix to compare diagnostic codes; The consultation integrity verification unit calculates the weighted recall rate of key symptom nodes, with a core symptom weight of 1.0 and a non-core accompanying symptom weight of 0.

5. The subjective experience generation unit generates PREM scores based on the trust value change curve AUC and the number of empathy errors.

6. An automated assessment method for the mental and psychological diagnostic and treatment capabilities of AI virtual patients, applied to the automated assessment system for the mental and psychological diagnostic and treatment capabilities of AI virtual patients as described in any one of claims 1-5, characterized in that, Includes the following steps: Step S1, VSP instance initialization and gold standard injection: Load pathological data from the disease feature database, initialize the hidden layer state vector and the explicit layer output strategy of VSP, and set the initial trust value T_init and the information disclosure threshold T_thres; Step S2, dynamic game interaction: The double-blind interactive sandbox forwards the response text of the AI doctor to be tested. VSP generates and drives the empathy S_empathy and professionalism S_prof of the end-side quantitative evaluation response, updates the trust value according to the formula and executes the corresponding impedance or exposure strategy, forming a multi-round interaction loop; Step S3, diagnosis endpoint trigger: When any one of the conditions of diagnostic output detection, interaction round overflow (i.e., ≥ 30 rounds), and trust value zeroing (i.e., T_new ≤ 0) is met, terminate the interaction and archive the log; Step S4, multi-dimensional automated settlement: Calculate the diagnostic accuracy, symptom weighted recall rate, and PREM score based on the interaction log, and generate a structured evaluation report.

7. The automated assessment method for the mental and psychological diagnostic and treatment capabilities of AI virtual patients according to claim 6, characterized in that, In step S2, the information disclosure strategy includes: If T_new < T_thres, VSP activates the impedance strategy generator and outputs non-specific defensive text; If T_new ≥ T_thres, VSP unlocks the corresponding symptom nodes in the hidden layer and retrieves and outputs the preset first-person symptom narrative text.

8. The automated assessment method for the mental and psychological diagnostic and treatment capabilities of AI virtual patients according to claim 6, characterized in that, In step S4, the diagnostic accuracy adopts hierarchical scoring: Getting a perfect match with the gold standard diagnostic code scores 100 points, getting a subtype error within the pedigree scores 50 points, getting a near-neighbor error across pedigrees scores 20 points, and being completely irrelevant scores 0 points; If the hidden layer of VSP contains a self-harm risk label and the AI doctor does not output this risk, the diagnostic accuracy score is forced to zero.

9. The automated assessment method for the mental and psychological diagnostic and treatment capabilities of AI virtual patients according to claim 6, characterized in that, In step S4, the calculation of the symptom weighted recall rate needs to simultaneously meet: the semantic similarity between the AI doctor's question and the symptom node > 0.85, and VSP is in the exposure state and outputs the symptom entity information within the next two rounds, then it can be determined as "successfully obtained".

10. The automated assessment method for the mental and psychological diagnostic and treatment capabilities of AI virtual patients according to claim 6, characterized in that, In step S4, the PREM score calculates the AUC through the trapezoidal integral of the trust value curve, obtains the baseline score through Min-Max normalization, and is weighted and calculated according to the emotional acceptance degree accounting for 40% of the weight, the safety and trust maintenance accounting for 30% of the weight, and the listening and empowerment accounting for 30% of the weight. Each time a trust regression occurs (i.e., ΔT < -15), 5 points are deducted; the Pearson correlation coefficient between the PREM score and the human expert score ≥ 0.75.

Images