Method and system for generating a dermatological diagnosis and treatment plan

Abstract

The application discloses a skin disease diagnosis and treatment scheme generation method and a generation system. The generation method comprises the following steps: acquiring a plurality of skin lesion images and medical record text content of a patient; determining the overall skin lesion condition and pathogenesis of the patient based on the acquired plurality of skin lesion images and in combination with a skin disease representation model, wherein the skin disease representation model is constructed based on a general multi-modal large model; determining the diagnosis condition and treatment scheme of the skin disease based on the acquired plurality of skin lesion images and medical record text content, the determined overall skin lesion condition and pathogenesis, and in combination with a skin disease reasoning model, so as to generate the diagnosis and treatment scheme of the skin disease, wherein the skin disease reasoning model is constructed based on the skin disease representation model. In this way, the diagnosis and treatment process of the skin disease is simplified, and the diagnosis and treatment efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of dermatology, and more specifically, to a method and system for generating treatment plans for dermatological diseases. Background Technology

[0002] Traditional Chinese Medicine (TCM) has developed over thousands of years, forming a systematic theoretical framework and rich clinical experience. In TCM theory, skin diseases are regarded as external manifestations of imbalance of Yin and Yang and dysfunction of the internal organs. Its diagnosis emphasizes a holistic view and syndrome differentiation and treatment, requiring a comprehensive assessment of local characteristics of skin lesions (such as color, shape, texture, and distribution) and systemic symptoms to identify syndromes, and on this basis, to achieve individualized treatment "tailored to the individual".

[0003] In recent years, large-scale language models have demonstrated the potential to learn complex task relationships and structured reasoning from massive amounts of data, providing new technical pathways for mining tacit knowledge, integrating fragmented information, and generating personalized treatment plans in the field of Traditional Chinese Medicine (TCM). However, current mainstream large-scale models are prone to severe "illusion" phenomena when applied in professional medical scenarios, indicating that the scale and data resources of coarse-grained models cannot overcome their limitations, thus restricting their practical application value in TCM clinical practice.

[0004] Therefore, how to enhance the practical application value of large-scale models in the medical field has become a technical problem that needs to be solved in this field. Summary of the Invention

[0005] In view of this, this application proposes a method and system for generating treatment plans for skin diseases, in order to enhance the practical application value of large models in the medical field.

[0006] In a first aspect, this application provides a method for generating a treatment plan for skin diseases. The method includes: acquiring multiple skin lesion images and medical record text content of a patient; determining the overall skin lesion condition and pathogenesis of the patient based on the acquired multiple skin lesion images and a skin disease representation model, wherein the skin disease representation model is constructed based on a general multimodal large model; and determining the diagnosis and treatment plan for the skin disease based on the acquired multiple skin lesion images and medical record text content, the determined overall skin lesion condition and pathogenesis, and a skin disease inference model, thereby generating a treatment plan for the skin disease, wherein the skin disease inference model is constructed based on the skin disease representation model.

[0007] Optionally, the skin disease representation model is constructed based on the following: taking a single training lesion image as input, the local lesion condition corresponding to the single training lesion image as output, and using the maximum likelihood objective to train a general multimodal large model to obtain a skin recognition model; taking multiple training lesion images as input, the overall lesion condition and pathogenesis corresponding to the multiple training lesion images as output, and using the maximum likelihood objective to train the obtained skin recognition model to obtain the skin disease representation model.

[0008] Optionally, the dermatology inference model is constructed based on the following: taking multiple training lesion images, the overall lesion condition and pathogenesis corresponding to the multiple training lesion images, and the medical records of patients corresponding to the multiple training lesion images as input, and taking the diagnosis and treatment plan corresponding to the multiple training lesion images as output, the dermatology representation model is trained using the maximum likelihood objective to obtain the dermatology inference model.

[0009] Optionally, the general multimodal large model is Qwen2.5-VL-7B or LLaVA.

[0010] Secondly, this application also provides a system for generating a treatment plan for skin diseases. The system includes: an acquisition module for acquiring multiple skin lesion images and medical record text content of a patient; a skin disease characterization model for determining the overall skin lesion condition and pathogenesis of the patient based on the acquired multiple skin lesion images, wherein the skin disease characterization model is constructed based on a general multimodal large model; and a skin disease inference model for determining the diagnosis and treatment plan for the skin disease based on the acquired multiple skin lesion images and medical record text content, the determined overall skin lesion condition, and the pathogenesis, thereby generating a treatment plan for the skin disease, wherein the skin disease inference model is constructed based on the skin disease characterization model.

[0011] Optionally, the skin disease representation model is constructed based on the following: taking a single training lesion image as input, the local lesion condition corresponding to the single training lesion image as output, and using the maximum likelihood objective to train a general multimodal large model to obtain a skin recognition model; taking multiple training lesion images as input, the overall lesion condition and pathogenesis corresponding to the multiple training lesion images as output, and using the maximum likelihood objective to train the obtained skin recognition model to obtain the skin disease representation model.

[0012] Optionally, the dermatology inference model is constructed based on the following: taking multiple training lesion images, the overall lesion condition and pathogenesis corresponding to the multiple training lesion images, and the medical records of patients corresponding to the multiple training lesion images as input, and taking the diagnosis and treatment plan corresponding to the multiple training lesion images as output, the dermatology representation model is trained using the maximum likelihood objective to obtain the dermatology inference model.

[0013] Optionally, the general multimodal large model is Qwen2.5-VL-7B or LLaVA.

[0014] Thirdly, this application also provides a machine-readable storage medium storing instructions that cause a machine to perform the above-described generation method.

[0015] Fourthly, this application also provides an electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the executable instructions to implement the above-described generation method.

[0016] According to the technical solution of this application, multiple skin lesion images and medical record text content are acquired; based on the acquired multiple skin lesion images, combined with a skin disease characterization model, the overall skin lesion condition and pathogenesis of the patient are determined; based on the acquired multiple skin lesion images and medical record text content, the determined overall skin lesion condition and pathogenesis, combined with a skin disease reasoning model, the diagnosis and treatment plan for the skin disease are determined, thus realizing the diagnosis and treatment of skin diseases. Furthermore, in determining the diagnosis and treatment plan, the overall skin lesion condition and pathogenesis of the patient are determined by combining the dermatological representation model, which is constructed based on a general multimodal large model. The overall skin lesion condition and pathogenesis determined by the dermatological inference model and the dermatological representation model are then used to determine the diagnosis and treatment plan for the skin disease. The dermatological inference model is constructed based on the dermatological representation model. Instead of simply piling up model parameters, it generates a diagnosis and treatment plan through collaboration between models. This improves upon the severe hallucination problem that often occurs when large models are applied in medical scenarios, while reducing their massive consumption of data, computing power, and electricity. This allows large models to be widely used in the medical field, enhancing their practical application value. In addition, the generated diagnosis and treatment plan can provide a reference for professional doctors, assisting them in determining treatment plans more quickly and effectively for the diagnosis and treatment of skin diseases. Furthermore, when determining the diagnosis and treatment plan, it can integrate visual and textual information to achieve multimodal reasoning in diagnosis and treatment. In addition, visual information of skin lesions (including color, shape, texture, etc.) is an important basis for diagnosis, and textual information describing the patient's symptoms also plays a key role in diagnosis. As a result, the determined diagnosis and treatment plan can be more in line with the diagnosis and treatment logic, thus improving the reliability of the generated diagnosis and treatment plan.

[0017] Other features and advantages of this application will be described in detail in the following detailed description section. Attached Figure Description

[0018] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application, and the illustrative embodiments and descriptions thereof are used to explain this application. In the drawings: Figure 1 This is a flowchart of a method for generating a skin disease diagnosis and treatment plan according to a preferred embodiment of this application; Figure 2 This is a structural block diagram of a system for generating a skin disease diagnosis and treatment plan according to a preferred embodiment of this application. Detailed Implementation

[0019] The technical solution of this application will now be described in detail with reference to the accompanying drawings and embodiments.

[0020] Firstly, this application provides a method for generating a treatment plan for skin diseases. In this embodiment, the treatment plan for skin diseases can be generated based on Traditional Chinese Medicine, and there is no limitation thereto.

[0021] Figure 1 This is a flowchart illustrating a method for generating a skin disease diagnosis and treatment plan according to a preferred embodiment of this application. Figure 1 As shown, the generation method includes the following.

[0022] In step S10, multiple skin lesion images and medical record text content of the patient are acquired. The medical record text content reflects the patient's medical history and current symptoms. For example, an interactive interface can be set up to input multiple skin lesion images and medical record text content of the patient, thereby achieving the acquisition of multiple skin lesion images and medical record text content of the patient.

[0023] In step S11, based on the acquired multiple skin lesion images and combined with a dermatological characterization model, the patient's overall skin lesion condition and pathogenesis are determined. The dermatological characterization model is constructed based on a general multimodal large model. The overall skin lesion condition reflects the global status of the patient's skin lesions, providing a global description of the patient's skin lesions. For example, the overall skin lesion condition reflects global signs, lesions, dryness / moisture, distribution, color, and boundaries. In this embodiment, the general multimodal large model can be any general multimodal large model capable of simultaneously processing multiple image and text information. For example, the general multimodal large model can be a Qwen series general multimodal large model, or a general multimodal large model can be LLaVA, CliP-2, etc.

[0024] In step S12, based on the acquired multiple skin lesion images and medical record text content, the determined overall skin lesion condition and pathogenesis, and combined with the dermatology inference model, the diagnosis and treatment plan for the skin disease are determined to generate a treatment plan for the skin disease. The dermatology inference model is constructed based on the skin disease representation model. The diagnosis includes pathogenesis analysis and syndrome differentiation, and the treatment plan includes treatment principles and prescription recommendations. In this application, the pathogenesis analysis can be a holistic pathogenesis analysis combining the skin lesions and symptoms of the patient. Optionally, in this embodiment, when using Traditional Chinese Medicine (TCM) to treat the skin disease, the pathogenesis analysis includes TCM pathogenesis analysis, and the prescription recommendations include TCM formulas.

[0025] In this embodiment of the application, the skin disease representation model in this application is obtained by training a general multimodal large model.

[0026] Optionally, in embodiments of this application, the dermatological phenomenology model is constructed based on the following.

[0027] Using a single training lesion image as input and the corresponding local lesion condition as output, a general multimodal large model is trained using the maximum likelihood objective to obtain a skin recognition model. Specifically, for a single training lesion image, the local lesion condition reflects the local state of the lesion in that image, describing the lesion condition in that image and providing a fine-grained description of the local lesion lesions. Optionally, a backpropagation gradient update algorithm with enhanced fine-tuning can be used for training. It should be noted that, in this embodiment, the training lesion image is the lesion image used in the model construction stage.

[0028] In this embodiment, the skin recognition model (DERM-Rec) is responsible for simulating lesion perception, morphological recognition, and visual-semantic representation of a single image. It is responsible for identifying lesions from a single image input, performing morphological analysis, and generating visual-semantic representations. To achieve effective perception of dermatological conditions, the skin recognition model is decomposed into fine-grained recognition and semantic understanding of local lesion morphology.

[0029] The skin recognition model is a multimodal image-to-text model that learns to identify and generate localized, fine-grained physiological descriptions of patient lesions from a given single patient skin lesion image through reinforcement fine-tuning. During training, a text-image pair dataset is first constructed. Each sample in this dataset consists of a single skin lesion image paired with a corresponding fine-grained description of the lesion (i.e., the local lesion condition). Then, through reinforcement fine-tuning training, a general multimodal model is constructed as the skin recognition model. During this training phase, a single dermatological image serves as input, and the model learns to extract clinically significant lesion features from local visual patterns, thereby establishing basic lesion-level cognition. For example, available lesion images from all clinical cases are collected, and dermatologists are invited to annotate each image with detailed local lesion descriptions according to standard clinical observation dimensions. After validation and quality control, each lesion image is paired with its text description, ultimately forming a dataset containing 518 single image-text pairs. It should be noted that, in this application, the number of single image-text pairs included in the dataset can be determined according to specific circumstances, and no limitation is imposed. A general multimodal large model (e.g., DMRM-VL-7B) is used as the base model, and maximum likelihood targeting is employed to train the generation of local lesion descriptions from a single lesion image. The resulting agent is defined as a skin disease recognition model.

[0030] Using multiple training lesion images as input and the overall lesion condition and pathogenesis corresponding to these images as output, a skin recognition model is trained using the maximum likelihood objective to obtain a skin disease representation model. Optionally, a backpropagation gradient update algorithm with enhanced fine-tuning can be used for training.

[0031] The Dermatological Representation Model (DERM-Rep) integrates multi-view lesion images to form a comprehensive understanding of the patient's overall skin condition and its TCM pathogenesis, generating structured specialist descriptions and pathogenesis representations. Based on skin recognition models, the modeling objective expands from local lesion identification to the comprehensive interpretation and representation of multi-view dermatological information.

[0032] The dermatological phenomenology model is also a general multimodal large model. The difference is that it doesn't start from a general multimodal large model, but rather is a further model training based on the skin recognition model. As mentioned earlier, the skin recognition model can already generate a long, rich, fine-grained, and clinically accurate description of skin lesions based on a single image. The dermatological phenomenology model takes as input all images of a patient's entire body relevant to their visit (usually 3-10 images of skin lesions in different positions), and learns to output the patient's overall skin lesion condition and the pathogenesis description reflected in the overall skin lesion condition. Furthermore, the "structured" aspect mentioned here means that the description of the patient's overall skin lesion condition and the pathogenesis description generated by the dermatological phenomenology model are decoupled and have certain format requirements, such as: [Patient's current overall skin lesions]...[Patient's skin lesion pathogenesis]....

[0033] The process of constructing this target model for skin disease representation is similar to that of modeling a skin recognition model. It also adopts a training method of reinforcement fine-tuning to construct a set of image-text datasets. Each data sample here includes a set of skin lesion images and two formatted text descriptions.

[0034] In real-world clinical settings, patient visits typically involve multiple images of lesions acquired from different anatomical locations or perspectives, making single-image analysis insufficient to fully reflect a patient's dermatological condition. All lesion images acquired during a single clinical visit are considered a unified input set. Dermatologists then generate detailed individualized dermatological descriptions based on these image sets. Since surface manifestations alone cannot fully capture the essence of the disease, clinicians further provide corresponding TCM (Traditional Chinese Medicine) pathogenesis analysis to explain the underlying mechanisms reflected in the observed lesion patterns. This process generates an inference sample with multiple image inputs and dual outputs, including a comprehensive description of skin lesions and a pathogenesis analysis.

[0035] As described in the text, the sample construction is based on a set of skin lesion images of patients. Therefore, the clinicians mainly completed two annotation tasks. Based on a given set of patient skin lesion images, (1) they wrote a global description of the patient's skin lesion condition, which may include location, nature, dryness, distribution, color, boundary, etc., and logically linked the skin lesion conditions of different locations according to the pathological condition; (2) they wrote an analysis of the pathogenesis of the skin lesion condition. 148 high-quality samples can be constructed to train the agent to aggregate, abstract and semantically model multi-perspective dermatological information. It should be noted that in this application, the number of high-quality samples can be determined according to the specific situation, and there is no restriction on this. While inheriting the ability to identify single lesions, the agent forms a structured representation of the individual dermatological condition of the patient and the corresponding TCM pathogenesis. This agent is called the Dermatological Representation Model (DERM-Rep).

[0036] Optionally, in this embodiment, the dermatology inference model is constructed based on the following: Multiple training lesion images, the overall lesion condition and pathogenesis corresponding to the multiple training lesion images, and the patient's medical records corresponding to the multiple training lesion images are used as inputs; the diagnostic information and treatment plan corresponding to the multiple training lesion images are used as outputs; and a maximum likelihood objective is employed to train the dermatology representation model to obtain the dermatology inference model.

[0037] In this embodiment, the Dermatology Reasoning Model (DERM-Reason) combines the output of the dermatology representation model with multi-view lesion images (multiple skin lesion images), patient history and current symptoms to perform comprehensive reasoning, and finally generate an overall pathogenesis analysis, syndrome diagnosis, treatment principles and prescription suggestions.

[0038] The dermatology inference model is based on the dermatology representation model. New clinical tasks are designed, and new training samples are constructed for further training. The training tasks of the dermatology inference model include overall patient pathogenesis analysis, syndrome differentiation, treatment principle selection, and prescription suggestions (prescription generation). The input data for each training sample of the dermatology inference model includes: 1) a global description of the patient's skin lesions generated by the dermatology representation model (overall skin lesion condition); 2) a specialist pathogenesis analysis of the skin lesion reaction generated by the dermatology representation model (pathogenesis); 3) the patient's medical history and current symptoms (patient's medical record content); and 4) all skin lesion images included in the patient's current consultation. The output includes the corresponding clinical diagnosis (overall pathogenesis analysis, syndrome differentiation) and treatment tasks (treatment principles and prescription suggestions) for the above four parts.

[0039] In this application, a model is constructed by jointly modeling the above four parts to provide the final results of dermatological diagnosis and treatment plans. The joint modeling of these four parts is linked together through the real logic of clinical diagnosis and treatment. In the preliminary stage, the skin recognition model and the dermatological representation model establish lesion-level perception and patient-level representation of dermatological diseases and their pathogenesis. Based on this, a dermatological reasoning model is constructed to address advanced diagnostic and treatment decisions in real-world clinical scenarios.

[0040] Specifically, the individual dermatological descriptions and pathogenesis of patients generated by the dermatological phenomenology model serve as intermediate inference representations. These are integrated with multi-view lesion images, medical history, physical signs, and current symptoms to form a structured multimodal pathological input. The output includes overall pathogenesis analysis, syndrome differentiation, treatment principles, and prescription suggestions, and can construct a dataset containing 134 validation samples. It should be noted that in this application, the number of validation samples included in the dataset can be determined based on specific circumstances and is not limited thereto.

[0041] The dermatology reasoning model employs a two-stage reasoning-based generative strategy. The agent first generates an overall pathogenesis analysis as an intermediate high-level clinical representation, which then constrains and guides syndrome identification, treatment principles, and prescription recommendations. This design simulates the clinical reasoning path from pathogenesis analysis to treatment decision-making. The final generated agent is called the Dermatology Reasoning Model (DERM-Reason).

[0042] Alternatively, in the embodiments of this application, the general multimodal large model may be Qwen2.5-VL-7B or LLaVA.

[0043] This application provides a multi-agent collaborative diagnostic framework based on clinical practice for generating treatment plans for skin diseases, thus assisting in the diagnosis and treatment of dermatological conditions. This framework restructures complex dermatological decision-making into a phased, multimodal reasoning process and enables effective collaboration among multiple specialized agents under actual clinical constraints.

[0044] This application further proposes a task decomposition and redefinition paradigm derived from clinical practice in traditional Chinese medicine dermatology. This paradigm systematically transforms the expert-led diagnostic workflow into modeling objectives that can be processed by artificial intelligence, thereby providing a generalizable methodology for structured artificial intelligence modeling in dermatology and related medical specialties.

[0045] The technical solution provided in this application supports joint reasoning in dermatology based on visual and textual dual-modality, realizing the modeling and inference process of multimodal information fusion.

[0046] The embodiments described in this application also demonstrate, through a task-oriented sample reconstruction and phased joint fine-tuning method, that high-quality multimodal modeling and inference can still be achieved even under conditions of extremely limited labeled data and lightweight model parameters. This provides an empirical basis for constructing an artificial intelligence system that balances data efficiency and computational efficiency and can support complex clinical tasks.

[0047] Empirical studies have shown that the lightweight multimodal model trained based on the implementation method of this application outperforms the general multimodal model with hundreds of billions of parameters in challenging clinical multi-task scenarios.

[0048] The beneficial effects of the embodiments described in this application also include the following aspects: (1) The clinical annotation sample requirement is extremely low. Traditional methods usually rely on tens of thousands of multimodal annotation samples for large model training, while this application only uses a few hundred samples to achieve performance exceeding that of models with hundreds of billions of parameters (such as GPT-5, Gemini 3, etc.).

[0049] (2) The model parameter size is significantly reduced. The lightweight model built based on this application has only 7 billion parameters, while similar advanced models (such as Gemini3, GPT-5, etc.) have hundreds of billions of parameters.

[0050] (3) Deployment costs are significantly reduced. The model in this application only requires a single graphics card with a video memory capacity of 24GB to be deployed, while the model with hundreds of billions of parameters usually requires more than 1TB of video memory resources, thus significantly reducing the deployment threshold and hardware costs.

[0051] The technical solution provided in this application analyzes real-world clinical practice in dermatology and reconstructs the problem through task decomposition and redefinition, proposing a multi-agent framework, DERM-3R (including DERM-Rep and DERM-Reason, where DERM-Rep is built upon DERM-Rec). This framework employs collaborative multi-agent modeling with a lightweight multimodal large model, addressing three core issues in dermatological diagnosis: refined lesion perception and identification, multi-perspective lesion aggregation combined with specialist pathogenesis modeling, and diagnostic reasoning based on holistic syndromes. Through this design, DERM-3R effectively solves challenges such as complex multi-task modeling, scarcity of complete clinical samples, and limited computational resource deployment. This framework can be built based on the Qwen2.5-VL-7B multimodal large model and fine-tuned using LoRA on multiple (e.g., 103) real-world TCM dermatology cases. Furthermore, this framework can also be built based on other general-purpose multimodal large models.

[0052] The technical solution provided by the embodiments of this application will be evaluated below in conjunction with the assessment. It should be noted that the following assessment is an exemplary description using traditional Chinese medicine treatment for psoriasis as an example.

[0053] 1) Machine Testing The diagnostic model constructed using the embodiments of this application achieved an absolute advantage among all comparative models: its total score of 44.16 points had a variance of only 1.49, while the total scores and variances of Qwen3-VL-8B-instruct, Qwen2.5-VL-7B-instruct, GPT-5.1-instant, and Gemini-3-flash were 39.42 (1.91), 35.54 (2.06), 41.23 (2.03), and 41.17 (2.05), respectively. The model constructed using the embodiments of this application achieved the highest total score with the lowest variance, fully demonstrating its superior performance and stability. When comparing Qwen3-VL-8B and Qwen2.5-VL-8B, Qwen3-VL-8B improved its total score in the professional domain task by 10.1% (absolute score of 3.88), indicating enhanced problem-solving ability within the domain. However, due to the 7.4% decrease in variance, its problem-solving ability and stability are still limited. In comparison, the diagnostic model constructed according to the embodiments of this application shows a performance improvement of 24.3% (absolute score of 8.63) and a variance reduction of 27.5%, which fully verifies its effectiveness.

[0054] 1.1) DERM-Rep Assessment Evaluations were conducted using Gemini-3-Flash and GPT-5.2. Under the Gemini-3-Flash evaluator, DERM-Rep achieved a total score of 22.25, closely following GPT-5.1-Instant (23.17) and Gemini-3-Flash (22.8333), significantly outperforming the baseline model Qwen2.5-VL-7B (18.1667) and maintaining a competitive score with Qwen3-VL-8B (21.83), as shown in Table 1. Under the GPT-5.2 evaluator, DERM-Rep achieved a total score of 14.25, significantly surpassing Qwen2.5-VL-7B (11.75), but still slightly lower than GPT-5.1-Instant (15.88) and Qwen3-VL-8B (14.92), as shown in Table 1.

[0055] The overall evaluation of DERM-Rep shows an average total score of 18.25, while its benchmark model scores 14.96, indicating that the DERM-Rep framework brings significant performance improvements.

[0056] Dimensional analysis reveals that DERM-Rep demonstrates the most significant advantage in pathogenesis reasoning. Under the GPT-5.2 assessment criteria, DERM-Rep achieved the highest score (4.08) in pathogenesis explanation, outperforming GPT-5.1-Instant (3.96) and Gemini-3-Flash (3.46), as shown in Table 1. This indicates that by structurally integrating multi-perspective pathological information, this agent can generate more clinically consistent and professionally aligned pathogenesis explanations, especially under more stringent assessment criteria.

[0057] Table 1 1.2) DERM-Reason Assessment This model is capable of end-to-end multimodal clinical reasoning, including overall pathogenesis analysis, syndrome differentiation, treatment methods, and prescription recommendations. Table 2 summarizes the evaluation results from the three major models: Deepseek-V3.2, Gemini-3-Flash, and GPT-5.2. Among all evaluators, DERM-Reason performed exceptionally well with the highest total score, significantly outperforming the second-place GPT-5.1-Instant (17.11, 33.14, 20.68) and all other baseline models.

[0058] Table 2 2) Manual evaluation Fourteen dermatologists from nine hospitals were invited to evaluate the model. DERM-3R performed consistently well among all comparison models, achieving the best performance across all categories, with the highest overall average score and the smallest variance. Compared to Qwen3-VL-8B-instruct, Qwen2.5-VL-7B-instruct, GPT-5.1-instant, and Gemini-3-flash, it showed performance improvements of 11.4%, 23.3%, 6.3%, and 10.1%, respectively.

[0059] Table 3 DERM-3R achieved performance advantages over Qwen2.5-VL-7B in all five tasks, with increases of 23.3%, 27.5%, 31.5%, 27%, and 34.2%, respectively, while Qwen3-VL-8B achieved performance advantages over Qwen2.5-VL-7B of 10.6%, 18.8%, 14.6%, 9.9%, and 10.9%.

[0060] The above results provide strong evidence that the DERM-3R multi-agent framework can effectively construct complex dermatological reasoning systems under limited data and computational resources, outperforming general-purpose and similarly sized multimodal models.

[0061] Secondly, this application also provides a system for generating treatment plans for skin diseases.

[0062] Figure 2 This is a structural block diagram of a system for generating a skin disease diagnosis and treatment plan according to a preferred embodiment of this application. Figure 2 As shown, the generation system includes an acquisition module 10, a skin disease representation model 20, and a skin disease inference model 30. The acquisition module 10 acquires multiple skin lesion images and medical record text content from the patient. The skin disease representation model 20, based on the acquired multiple skin lesion images, determines the overall skin lesion condition and pathogenesis of the patient; the skin disease representation model 20 is constructed based on a general multimodal large model. The skin disease inference model 30, based on the acquired multiple skin lesion images and medical record text content, the determined overall skin lesion condition, and the pathogenesis, determines the diagnosis and treatment plan for the skin disease to generate a treatment plan. The skin disease inference model 30 is constructed based on the skin disease representation model 20.

[0063] Optionally, the skin disease representation model 20 is constructed based on the following: taking a single training lesion image as input and the local lesion condition corresponding to the single training lesion image as output, a general multimodal large model is trained using the maximum likelihood objective to obtain a skin recognition model; taking multiple training lesion images as input and the overall lesion condition and pathogenesis corresponding to the multiple training lesion images as output, the skin recognition model is trained using the maximum likelihood objective to obtain the skin disease representation model 20.

[0064] Optionally, the dermatology inference model 30 is constructed based on the following: taking multiple training lesion images, the overall lesion condition and pathogenesis corresponding to the multiple training lesion images, and the medical records of patients corresponding to the multiple training lesion images as inputs, and taking the diagnostic conditions and treatment plans corresponding to the multiple training lesion images as outputs, the dermatology representation model 20 is trained using the maximum likelihood objective to obtain the dermatology inference model 30.

[0065] Alternatively, the general multimodal large model is Qwen2.5-VL-7B or LLaVA.

[0066] The specific working principle and benefits of the skin disease diagnosis and treatment plan generation system provided in this application embodiment are similar to those of the skin disease diagnosis and treatment plan generation method provided in this application embodiment, and will not be repeated here.

[0067] Thirdly, this application also provides a machine-readable storage medium storing instructions that cause a machine to perform the above-described generation method.

[0068] Fourthly, this application also provides an electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the executable instructions to implement the above-described generation method.

[0069] The preferred embodiments of this application have been described in detail above. However, this application is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this application, various simple modifications can be made to the technical solution of this application, and these simple modifications all fall within the protection scope of this application.

[0070] It should also be noted that the various specific technical features described in the above embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, this application will not describe the various possible combinations separately.

[0071] Furthermore, various different implementations of this application can be combined in any way, as long as they do not violate the spirit of this application, they should also be regarded as the content disclosed in this application.

Claims

1. A method for generating a treatment plan for skin diseases, characterized in that, The generation method includes: Acquire multiple skin lesion images and medical record text content from the patient; Based on the acquired multiple skin lesion images, combined with the skin disease characterization model, the overall skin lesion condition and pathogenesis of the patient are determined. The skin disease characterization model is constructed based on a general multimodal large model. Based on the acquired multiple skin lesion images and medical record text content, the determined overall skin lesion condition and pathogenesis, and combined with the skin disease inference model, the diagnosis and treatment plan for the skin disease are determined to generate a diagnosis and treatment plan for the skin disease. The skin disease inference model is constructed based on the skin disease representation model.

2. The generation method according to claim 1, characterized in that, The skin disease phenotype model was constructed based on the following: Using a single training lesion image as input and the local lesion condition corresponding to the single training lesion image as output, a general multimodal large model is trained using the maximum likelihood objective to obtain a skin recognition model. Using multiple training skin lesion images as input, and the overall skin lesion condition and pathogenesis corresponding to the multiple training skin lesion images as output, the skin recognition model is trained using the maximum likelihood objective to obtain the skin disease characterization model.

3. The generation method according to claim 1, characterized in that, The skin disease inference model is constructed based on the following: The skin disease representation model is trained using multiple training skin lesion images, the overall skin lesion condition and pathogenesis corresponding to the multiple training skin lesion images, and the medical records of the patients corresponding to the multiple training skin lesion images as inputs, and the diagnostic conditions and treatment plans corresponding to the multiple training skin lesion images as outputs. The maximum likelihood objective is used to obtain the skin disease inference model.

4. The generation method according to any one of claims 1-3, characterized in that, The general multimodal large model is Qwen2.5-VL-7B or LLaVA.

5. A system for generating treatment plans for skin diseases, characterized in that, The generation system includes: The acquisition module is used to acquire multiple skin lesion images and medical record text content of the patient; A skin disease characterization model is used to determine the overall skin lesion status and pathogenesis of a patient based on multiple acquired skin lesion images, wherein the skin disease characterization model is constructed based on a general multimodal large model; A skin disease inference model is used to determine the diagnosis and treatment plan of skin diseases based on multiple acquired skin lesion images and medical record text content, the determined overall skin lesion condition and pathogenesis, so as to generate a diagnosis and treatment plan for skin diseases. The skin disease inference model is constructed based on the skin disease representation model.

6. The generation system according to claim 5, characterized in that, The skin disease phenotype model was constructed based on the following: Using a single training lesion image as input and the local lesion condition corresponding to the single training lesion image as output, a general multimodal large model is trained using the maximum likelihood objective to obtain a skin recognition model. Using multiple training skin lesion images as input, and the overall skin lesion condition and pathogenesis corresponding to the multiple training skin lesion images as output, the skin recognition model is trained using the maximum likelihood objective to obtain the skin disease characterization model.

7. The generation system according to claim 5, characterized in that, The skin disease inference model is constructed based on the following: The skin disease representation model is trained using multiple training skin lesion images, the overall skin lesion condition and pathogenesis corresponding to the multiple training skin lesion images, and the medical records of the patients corresponding to the multiple training skin lesion images as inputs, and the diagnostic conditions and treatment plans corresponding to the multiple training skin lesion images as outputs. The maximum likelihood objective is used to obtain the skin disease inference model.

8. The generation system according to any one of claims 5-7, characterized in that, The general multimodal large model is Qwen2.5-VL-7B or LLaVA.

9. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores instructions that cause the machine to perform the generation method according to any one of claims 1-4.

10. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the executable instructions to implement the generation method according to any one of claims 1-4.

Images