A skin disease auxiliary diagnosis method and device based on dynamic retrieval and chain reasoning and a medium

Abstract

The application discloses a kind of based on dynamic retrieval and chain reasoning's auxiliary diagnosis method, device and medium of dermatosis.It is described as follows:The method includes: obtaining skin disease image and user description text;First visual language model is input to target sample, and medical entity feature is extracted and primary diagnosis hypothesis is generated;According to the clinical diagnosis sequence of mechanism inference from feature perception, generate reasoning node sequence;Based on current node and reasoning context, construct retrieval query, retrieve reference information in skin disease knowledge base and integrate with reasoning node, form image-knowledge-reasoning chain triple knowledge structure;The integration result of the reasoning node sequence and the triple knowledge structure is input into second language model, and auxiliary diagnosis report containing diagnosis reasoning process and diagnosis conclusion is generated and output.No skin disease image-diagnosis report pairing training data can improve the explainability and reliability of zero sample diagnosis.The application also provides the corresponding diagnostic device and computer readable medium.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence, intelligent analysis of medical images, and medical information processing, and in particular to a method, device, and computer-readable storage medium for auxiliary diagnosis of skin diseases based on visual language models and large language models, dynamic knowledge retrieval, and structured chain reasoning. Background Technology

[0002] Artificial intelligence has been widely applied in medical image analysis and assisted diagnosis. Traditional dermatological image analysis mainly relies on image recognition algorithms such as deep convolutional neural networks. While these methods can extract basic visual features and identify different types of skin diseases, their diagnostic results lack interpretability, struggle to generate coherent medical diagnostic reports, and typically require large amounts of labeled data, limiting their application in scenarios with few or no samples.

[0003] In recent years, multimodal deep learning methods combining vision and language, especially visual-language models, have become the mainstream development direction in the field of medical image-assisted diagnosis. These models possess powerful multimodal perception and text generation capabilities. Furthermore, to address issues such as "illusion" and "logical gaps" in large models within medical scenarios, existing technologies have proposed retrieval-augmented generation (RAG) methods that combine domain knowledge bases. These methods assist the model in reasoning by retrieving relevant medical literature or databases.

[0004] However, the closest auxiliary diagnostic system to this invention still has the following significant shortcomings in actual clinical applications: Unclear reasoning logic and poor interpretability: Existing methods often employ a single or unstructured chain of thought (COT) when making diagnostic inferences. This reasoning lacks clear constraints from clinical diagnostic logic. It fails to effectively integrate multi-dimensional information (such as skin anatomy and pathological mechanisms), resulting in a lack of traceability and reliable explanatory basis in the diagnostic process.

[0005] Knowledge retrieval lacks dynamism and task awareness: Most existing RAG solutions employ static or superficial semantic retrieval. That is, the construction of retrieval queries cannot be adjusted in real time according to the task of the current reasoning node in the diagnostic process and the completed reasoning context, resulting in the retrieved knowledge often lacking specificity and temporal consistency, making it difficult to support complex multi-step clinical reasoning.

[0006] Insufficient multimodal verification mechanisms: Existing technologies struggle to organically integrate and effectively cross-verify image evidence, knowledge base information, and reasoning conclusions during the reasoning process. In particular, they lack mechanisms for secondary visual verification of images based on the reasoning state, making it difficult to meet the needs of complex tasks such as differential diagnosis.

[0007] Therefore, there is an urgent need for a skin disease diagnostic technology solution that can integrate multimodal perception, task-aware dynamic knowledge retrieval, and structured chain reasoning mechanisms to improve the accuracy, reliability, and clinical interpretability of diagnosis. Summary of the Invention

[0008] The purpose of this invention is to provide a method, device, and computer-readable storage medium for auxiliary diagnosis of skin diseases based on dynamic retrieval and chain reasoning, so as to overcome the shortcomings of the prior art in terms of diagnostic accuracy, interpretability, and adaptability to complex clinical procedures.

[0009] To achieve the above objectives, the first aspect of this invention proposes a method for auxiliary diagnosis of skin diseases based on dynamic retrieval and chained reasoning, the method comprising: Obtain target samples, which include skin disease images and their corresponding user description text; The target sample is input into the first visual language model to extract the medical entity features of skin diseases and generate initial diagnostic hypotheses; Based on the aforementioned features and diagnostic hypotheses, following the clinical diagnostic sequence from feature perception to mechanism inference, a three-way parallel thinking chain is constructed in the differential diagnosis stage, including skin lesion observation, skin anatomy, and pathological mechanisms. This chain sequentially generates multiple interrelated reasoning nodes, forming a sequence of dependent reasoning nodes. A retrieval query statement is generated based on the current inference node and the completed inference context in the inference node sequence. Relevant reference information is retrieved and obtained from the pre-built dermatology knowledge base. The reference information is integrated with the corresponding inference node to construct a three-element knowledge structure of dermatology image-knowledge-inference chain for auxiliary diagnosis. The inference node sequence and the ternary knowledge structure are input into the second language big model to integrate the inference information and generate a target skin disease auxiliary diagnostic report containing the diagnostic inference process and diagnostic conclusion. Output an auxiliary diagnostic report of the target skin disease image.

[0010] In some embodiments, the inference node sequence includes multiple inference nodes arranged in the order of the clinical diagnosis process, and the inference node sequence includes at least feature extraction nodes, secondary visual verification nodes, differential diagnosis nodes, and knowledge retrieval nodes.

[0011] In some embodiments, multiple thought chains are constructed during the differential diagnosis stage, including a skin lesion observation reasoning chain, a skin anatomy reasoning chain, and a pathological mechanism reasoning chain, to analyze candidate diseases from different clinical dimensions.

[0012] In some embodiments, the dynamic knowledge retrieval dynamically generates retrieval query statements based on different task types and inference node content, and adjusts the retrieval scope and focus as the inference process progresses.

[0013] In some embodiments, the method further includes generating visual verification guidance information based on the reasoning result, and performing secondary visual verification on the dermatology image based on the guidance information to calibrate the visual features identified during the reasoning process.

[0014] A second aspect of the present invention provides a skin disease auxiliary diagnostic device based on dynamic retrieval and chained reasoning, the device comprising: The acquisition module is used to acquire target samples, including skin disease images and their corresponding user description text; The chain reasoning module is used to call the visual recognition module and the language reasoning module according to the chain reasoning rules based on the characteristics of the skin disease medical entity to obtain the reasoning node sequence; The retrieval module is used to call the language reasoning module to generate retrieval query statements and retrieve reference information related to the diagnostic task from the dermatology medical knowledge base; The visual recognition module is used to extract features from the skin disease image to obtain the medical entity features of the skin disease and the initial diagnostic hypothesis, and to perform secondary recognition on the image based on the guidance information generated by the language reasoning module to obtain differential diagnostic information. The language reasoning module is used to generate retrieval query statements for diagnostic tasks, generate guidance information to guide the visual recognition module to perform secondary recognition, and integrate reasoning information from the reasoning node sequence to generate a skin disease auxiliary diagnostic report.

[0015] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is used to implement any of the above-described auxiliary diagnostic methods for skin diseases.

[0016] This invention overcomes the shortcomings of existing technologies, such as lack of dynamic adjustment and interpretability in the reasoning process, by introducing a dynamic retrieval mechanism and a chain-like reasoning structure. By combining the reasoning capabilities of a domain knowledge base and a general large model, it can improve the zero-shot performance of the model in assisted diagnosis of skin diseases without additional training. Furthermore, this invention effectively integrates image features, knowledge base information, and the dependencies between reasoning nodes through the construction of a ternary knowledge structure, enhancing the accuracy and interpretability of diagnosis, and making it more adaptable to handle more complex diagnostic tasks. Attached Figure Description

[0017] Figure 1 This is a flowchart of the skin disease auxiliary diagnosis method based on dynamic retrieval and chain reasoning provided in the embodiments of the present invention; Figure 2 This is a reasoning process diagram of the skin disease auxiliary diagnosis method based on dynamic retrieval and chain reasoning provided in the embodiments of the present invention; Figure 3 This is an experimental result diagram of the skin disease auxiliary diagnosis method based on dynamic retrieval and chain reasoning provided in the embodiments of the present invention; Figure 4 This is a functional module diagram of the skin disease auxiliary diagnostic device based on dynamic retrieval and chain reasoning provided in the embodiments of the present invention; Figure 5 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0018] To enable those skilled in the art to better understand the technical solutions of the present invention, the present invention will be further described below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following embodiments are only for illustrating the present invention and do not constitute a limitation on the scope of protection of the present invention.

[0019] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to limit the invention.

[0021] This invention provides a method for auxiliary diagnosis of skin diseases based on dynamic retrieval and chained reasoning, such as... Figure 1 As shown, it includes the following steps: S100. Obtain target samples, including images of skin diseases and user description text.

[0022] Obtain the target sample, denoted as ,in Images representing skin diseases This represents the user's description text.

[0023] S200. Input the target sample into the first visual language model, extract the medical entity features of skin diseases, and generate a preliminary diagnostic hypothesis.

[0024] The target sample and prompt words used for feature extraction. Input the first visual language model Extracting medical entity features of skin diseases Task Type and primary diagnostic hypotheses The feature extraction process can be formally represented as:

[0025] S300. Generate a sequence of inference nodes based on the clinical diagnosis sequence from feature perception to mechanism inference.

[0026] Clinical diagnostic logic should consider specific medical knowledge, clinical procedures, and reasoning structures. It should rely on a multi-layered reasoning framework within the field of dermatology, combining a medical knowledge base with practical clinical experience to form a reasoning process with a medical professional background. Based on this clinical diagnostic logic, the medical entity characteristics of the dermatological disease are... Task Categories and set of primary diagnostic hypotheses Input the structured thought chain module to generate a sequence containing multiple reasoning nodes. Let this sequence of reasoning nodes be denoted as . .

[0027] like Figure 2 As shown, this invention uses causal-driven reasoning based on logical constraints to achieve a flexible mechanism of "reasoning, retrieving, and verifying simultaneously," ensuring the rigor of the diagnostic path in medical logic and its adaptability to complex cases. The structured thinking chain construction process uses task category information. For global constraints, based on the medical entity characteristics of dermatology and set of primary diagnostic hypotheses As input for reasoning, a sequence of reasoning nodes with clear semantics and dependencies is generated by sequentially arranging various reasoning nodes in a structured thought chain. Among them, each reasoning node These are not simple logical judgment units, but rather nodes corresponding to specific reasoning functions. These nodes include task-aware dynamic knowledge retrieval reasoning nodes, domain knowledge-guided visual guidance information generation reasoning nodes, knowledge-guided visual verification reasoning nodes, and knowledge integration reasoning nodes. Each reasoning node is activated sequentially according to the diagnostic logic order, and the reasoning process of subsequent nodes depends on the intermediate reasoning results of the preceding nodes and the contextual information. Through the above approach, the structured thinking chain transforms the original medical image analysis process into a reasoning structure constrained by tasks, driven by knowledge, and equipped with a multi-way verification mechanism, providing a clear reasoning framework for subsequent dynamic knowledge retrieval and the construction of a ternary knowledge structure.

[0028] S400: Generate a retrieval query based on the current inference node and inference context, retrieve reference information in the dermatology knowledge base and integrate it with the inference node to form a three-element knowledge structure of image-knowledge-inference chain.

[0029] In this invention, the dynamic retrieval generates search queries based on the content of the current inference node, task type information, and the completed inference context. Unlike traditional static retrieval, the dynamic retrieval of this invention can adapt to changes in the inference process in real time, retrieving information most relevant to the task of the current inference node. This means that as the inference deepens, the system will, based on existing logical discoveries, "on demand" more targeted medical evidence from the knowledge base.

[0030] Specifically, the dynamic retrieval process is as follows: Based on the inference node sequence Combined with task category In a pre-built knowledge base for dermatology Semantic retrieval is performed within the inference chain. For the first... Nodes Construct search query :

[0031] Perform a search to obtain reference information. :

[0032] in This represents the semantic similarity function.

[0033] The reference information With corresponding reasoning nodes Medical physical characteristics of skin diseases Constructing a three-element knowledge structure for skin diseases: image-knowledge-reasoning chain.

[0034] The complete ternary knowledge sequence is represented as:

[0035] S500: Input the integration result of the reasoning node sequence and the ternary knowledge structure into the second language model to generate a structured diagnostic report containing the diagnostic reasoning process and diagnostic conclusion.

[0036] For the inference node sequence And the aforementioned dermatology image-knowledge-reasoning chain ternary knowledge structure Knowledge integration is performed to generate an auxiliary diagnostic report for the target dermatology, including the diagnostic reasoning process and diagnostic conclusions. The knowledge integration process can be represented as follows:

[0037] Among them, the diagnostic report Include: Reasoning process:

[0038] Diagnostic conclusion:

[0039] S600, Output target skin disease auxiliary diagnostic report The final output format is:

[0040] This method, through the organic combination of dynamic retrieval mechanism and chain reasoning, realizes knowledge-driven interpretable auxiliary diagnosis of skin diseases, effectively improving the accuracy and reliability of auxiliary diagnosis.

[0041] Figure 3 This paper presents quantitative comparative experimental results between the proposed method and existing baseline methods on two publicly available dermatology diagnostic datasets (SD-198 and SkinCap). The experimental evaluation employed multiple metrics, including BLEU-1, ROUGE-L, BERTScore, and diagnostic accuracy (Acc).

[0042] The comparison methods include three types of models: General Visual Language Models (VLMs): Qwen2.5-VL-7B-Instruct and InternVL3-8B; Medical Visual Language Models (VLMs): LLAVA-med (2023), HuaTuoGPT-Vision-7B (2024), LingShu-7B (2025), and the Skinvl series of multimodal large models for dermatological diseases; Finally, there is the method presented in this paper (Ours); Our proposed method achieves state-of-the-art performance on the SD-198 dataset, improving diagnostic accuracy from 0.294 to 0.645 compared to the base model Qwen2.5-VL-7B-Instruct, representing a 119.4% improvement. On the SkinCap dataset, our method also achieves BLEU-1 (0.246) and ROUGE-L (0.204) scores.

[0043] Figure 4 This invention illustrates the functional module architecture of a skin disease auxiliary diagnostic device based on dynamic retrieval and chained reasoning, which includes the following core modules: 2001 Acquisition Module The acquisition module is responsible for receiving and preprocessing input data. Its functions include: acquiring images of skin diseases. and its corresponding user description text target samples ; 2004 Visual Recognition Module The visual recognition module is responsible for processing skin disease images and providing structured visual evidence for subsequent reasoning; the visual recognition module is integrated with the first large visual language model. The visual encoder component and the multilayer perceptron component are closely related, and their main functions are: The visual recognition module first utilizes a pre-trained visual encoder. Input skin disease images Processing is performed to extract basic image features. This visual encoder is typically based on the Transformer architecture, which transforms an image into a series of visual tag embeddings.

[0044] The feature extraction process can be described as follows:

[0045] in, , indicates inclusion There are 1 visual marker, each marker having a dimension of 1. The characteristic matrix of .

[0046] In obtaining basic visual features Then, the visual recognition module, through the feature projection layer and cross-modal attention mechanism within the model, combined with the language decoder, will... Transformed into entity features with medical semantics These characteristics It contains key information related to skin diseases, such as: Skin lesion morphology (e.g., papules, plaques, vesicles, etc.); Colors (such as red, brown, white, etc.); Texture (such as rough, smooth, flaky, etc.).

[0047] A key function of the visual recognition module is based on the language reasoning module (i.e., the language decoder). Guiding information generated in the inference chain For images Perform secondary identification.

[0048] The process of extracting medical entity features relies on the interaction of visual and textual prompts, which can be abstractly represented as:

[0049] in, The input task description or prompts guide the model to focus on medical-related entities.

[0050] When chained reasoning requires differential diagnosis or verification of local details, guiding information It will indicate the image regions or features that the model needs to focus on.

[0051] The secondary identification process aims to: Local detail verification: confirming the details of specific skin lesions; Differential diagnosis exclusion: For a candidate disease, look for key visual evidence that supports or excludes the disease; secondary identification involves examining images... and guidance information Simultaneously input To achieve and output differential diagnosis results :

[0052] Among them, the results The results are fed back to the language reasoning module in the form of structured text, which is used for updating nodes in the reasoning chain and confirming the final diagnosis.

[0053] 2005 Language Reasoning Module The language reasoning module is the core reasoning component of this diagnostic device, and it employs an independent, pre-trained large-scale language model. It is responsible for integrating and reasoning the sequence information of inference nodes, generating retrieval statements, and outputting the final structured auxiliary diagnostic report.

[0054] The language reasoning module receives the following input: medical entity features extracted by the visual recognition module. and primary diagnostic hypothesis User description text A three-element knowledge structure of skin disease images-knowledge-reasoning chain constructed by the retrieval module and the chained reasoning module. .

[0055] The language reasoning module performs step-by-step reasoning based on a sequence of reasoning nodes. The reasoning process can be represented as follows:

[0056] in, This represents a ternary structure that incorporates information from the previous node, serving as the conditional input for the current inference.

[0057] A core function of the language reasoning module is to enable bidirectional interaction with the visual recognition module. When the reasoning process requires verification of local details or differential diagnosis, it is responsible for generating guidance information based on the current reasoning state. This guides the visual recognition module to perform secondary visual verification. (Guidance information) The generation is based on a comprehensive judgment of the current inference node, the retrieved reference information, and the visual features to be verified. Guiding information. The generation process can be formalized as follows:

[0058] in: This indicates the reference information obtained from the current dynamic retrieval; Extracted medical entity features; The current reasoning history context; After completing the inference node sequence and knowledge structure After its construction, the language generation module performs reasoning information integration. It brings together all visual, knowledge, and reasoning information, through... Its text generation capabilities ultimately output a structured diagnostic report. :

[0059] in It is the final prompt information that guides the generation of a complete report format (including reasoning process, diagnostic conclusions, disease summary, etc.).

[0060] 2002 Chain Reasoning Module The chained reasoning module 2002 is responsible for constructing, executing, and managing the sequence of reasoning nodes. .

[0061] Function: Medical entity features extracted based on the visual recognition module and task type By calling the language reasoning module 2005, a sequence of reasoning nodes is constructed. At the same time, this module manages the dependencies and logical constraints between inference nodes.

[0062] Node structure: Each inference node Includes: a description of the current reasoning step. , and intermediate conclusions .

[0063] Formal Reasoning: The logical drive of the reasoning module is implemented by the language reasoning module, and its process can be represented as follows: =

[0064] in, This is reference information returned by the retrieval module. It is the history of previous reasoning.

[0065] 2003 Search Module The retrieval module 2003 is responsible for retrieving information from a knowledge base in the field of dermatology. Dynamic knowledge retrieval with task awareness is performed.

[0066] Query Construction: The module receives the current inference node output by the chained inference module. and in combination with task categories Information such as these are used to construct query statements for knowledge base retrieval. :

[0067] Semantic retrieval: The module is located in the knowledge base. Perform semantic retrieval in the middle to obtain Related reference information The retrieval process is formally represented as:

[0068] Wherein, the similarity Cosine similarity is typically used for calculation:

[0069] The This is a function that converts text into vectors for pre-trained embedding models, such as BERT or similar architectures.

[0070] The device achieves the organic integration of visual understanding, knowledge retrieval, and chain reasoning through modular design, supporting the generation of interpretable auxiliary diagnostic reports for skin diseases.

[0071] like Figure 5The diagram shown is a hardware structure schematic of an electronic device provided in an embodiment of the present invention. The electronic device includes a memory 2101, which can be implemented in the form of read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 2101 can store the operating system and other applications. When the technical solution provided in the embodiment of the present invention is implemented through software or firmware, the relevant program code is stored in the memory 2101, including skin disease diagnosis model parameters, visual language large model weights, domain knowledge base data, and chain reasoning algorithm code, etc., and is called and executed by the processor 2102 to execute the skin disease auxiliary diagnosis method based on dynamic retrieval and chain reasoning in the embodiment of the present invention. The processor 2102 can be implemented using a general-purpose CPU (Central Processing Unit), GPU (Graphics Processing Unit), microprocessor, Application Specific Integrated Circuit (ASIC), or one or more integrated circuits, etc., to execute relevant programs to realize the technical solutions provided in the embodiments of the present invention, including the processing tasks of core functional modules such as visual feature extraction, medical entity recognition, structured thinking chain construction, dynamic knowledge retrieval, multi-dimensional knowledge integration, and diagnostic report generation; The input / output interface 2103 is used to realize information input and output, specifically including the input acquisition of skin disease images, the reception of task description text, and the output display of information such as diagnostic reports and reasoning process descriptions; The communication interface 2104 is used to enable communication and interaction between this device and other devices. Communication can be achieved via wired means (such as USB, Ethernet, serial port, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, 5G, etc.). It is used for data exchange and collaborative work with external systems such as medical information systems (HIS / PACS), image acquisition equipment, and remote diagnostic platforms. Bus 2105 transmits information between various components of the device (e.g., memory 2101, processor 2102, input / output interface 2103, and communication interface 2104), and uses a standard bus architecture to achieve high-speed data transmission; This invention also provides an electronic device, which includes a processor and a memory. The memory stores a computer program, and when the processor executes the computer program, it implements all the steps of the above-described skin disease diagnosis method.

[0072] Furthermore, embodiments of the present invention also provide a computer-readable storage medium storing one or more computer programs thereon, which can be executed by one or more processors to implement the above-described skin disease diagnosis method based on dynamic retrieval and chained reasoning.

[0073] It should be noted that the various embodiments of the present invention can be combined with each other. Where there is no conflict, the technical features of the present invention can be arbitrarily combined or equivalently substituted, all of which should fall within the protection scope of the present invention. Those skilled in the art should understand that, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the present invention are equally applicable to similar technical problems. Furthermore, modifications to the technical solutions described in the above embodiments, or equivalent substitutions of some of the technical features, do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for auxiliary diagnosis of skin diseases based on dynamic retrieval and chained reasoning, characterized in that, Includes the following steps: S100. Obtain target samples, wherein the target samples include skin disease images and their corresponding user description text; S200. Input the target sample into the first visual language model, extract medical entity features and generate a preliminary diagnostic hypothesis; S300. Generate a sequence of inference nodes based on the clinical diagnostic sequence from feature perception to mechanism inference; S400. Construct a retrieval query based on the current node and the reasoning context, dynamically retrieve reference information from the dermatology knowledge base and integrate it with the reasoning node to form a three-element knowledge structure of image-knowledge-reasoning chain; S500. Input the integration result of the reasoning node sequence and the ternary knowledge structure into the second language model to generate a structured diagnostic report containing the diagnostic reasoning process and diagnostic conclusion; S600. Outputs a medical auxiliary diagnostic report of the target skin image.

2. The method according to claim 1, characterized in that, The inference node sequence includes: Task-aware dynamic retrieval: The visual recognition module identifies the task type of the current node, and based on the task type, the language reasoning module generates a retrieval query statement optimized for that task type; the retrieval query statement is then used to retrieve relevant reference information from the dermatology knowledge base; Three-chain parallel reasoning: The language reasoning module constructs three parallel multi-path thought chains; Visual verification guidance generation: The language reasoning module is used to generate guidance information to guide the visual model to perform secondary verification; Secondary visual verification: The guidance information is input into the visual recognition module, and the three inference chains execute secondary visual verification in parallel; Inference Node Sequence Integration: The language inference module is used to perform consistency analysis on the intermediate inference results, consistency verification results and secondary verification results of the three inference chains, to examine the logical conflicts or support relationships between the chains, and to output a target structured auxiliary diagnostic report.

3. The method according to claim 1, characterized in that, The inference node sequence is dynamically adjusted according to the task type, which includes: Multiple-choice task, used for reasoning based on multiple candidate diagnostic options; Open question-answering tasks are used to generate open-ended diagnostic conclusions; Differential diagnosis is a task used to rule out other similar diseases and arrive at the most probable diagnosis.

4. The three-chain parallel reasoning mechanism according to claim 2, wherein the three chains include: The skin lesion observation chain, skin anatomy chain, and pathological mechanism chain are characterized in that, after each chain outputs intermediate reasoning results in parallel, the intermediate reasoning results are checked for consistency through a language reasoning module to detect logical conflicts or supporting relationships, thereby ensuring the comprehensiveness of the reasoning logic.

5. The method according to claim 1, characterized in that, The dynamic knowledge retrieval includes: Based on the content of the current inference node, the task type information, and the completed inference context, the language inference module generates a query statement for knowledge base retrieval. Based on the query statement, a semantic search is performed in the dermatology knowledge base to obtain reference information related to the current inference node; Furthermore, the search scope and focus are adjusted based on the context of the inference node to ensure the consistency and relevance of the search results with the current inference sequence; Based on the retrieval results, the input information of the inference node is updated and combined with the completed inference context to construct a three-element knowledge structure of skin disease image-knowledge-inference chain for auxiliary diagnosis.

6. The method according to claim 1, characterized in that, The dermatology image-knowledge-reasoning chain ternary knowledge structure is constructed in the following way: For the first inference node sequence Each inference node Compare it with the skin disease image Medical entity characteristics and the reference information retrieved by this node Bind and construct a triple. .

7. The method according to claim 1, characterized in that, The integration of the reference information with the current inference node is achieved in the following way: Specific prompt templates are built to generate specialized text prompts based on the task type and context information of the current inference node. The language inference module injects reference information into the inference process in a structured form, thereby enhancing the relevance and accuracy of the inference.

8. The method according to claim 1, characterized in that, The integration of the reasoning information includes: Construct prompt words for the diagnostic task; aggregate intermediate inference results given by the visual recognition module to obtain comprehensive inference information including lesion features, lesion location, candidate diseases and their inference chains; The comprehensive reasoning information and the ternary knowledge structure are input into the language reasoning module to construct an integrated reasoning prompt for the diagnosis of skin diseases; Based on the reasoning integration prompts, a target dermatological auxiliary diagnostic report is generated and output, which includes the diagnostic reasoning process, disease diagnosis, and treatment suggestions.

9. A skin disease auxiliary diagnostic device based on dynamic retrieval and chain reasoning, characterized in that, include: The acquisition module is used to acquire target samples, including skin disease images and their corresponding user description text; The chain-based reasoning module is used to call the visual recognition module and the language reasoning module according to the reasoning rules based on the medical entity characteristics of the skin disease to obtain a sequence of reasoning nodes. The retrieval module is used to call the language reasoning module based on the inference node sequence to generate a knowledge base query statement, and to retrieve reference information related to the diagnostic task from the dermatology medical knowledge base; The visual recognition module is used to extract features from the skin disease images to obtain medical entity features of skin diseases and a set of candidate diseases, and to perform secondary recognition on the images based on the guidance information generated by the language reasoning module to obtain differential diagnostic information. The language reasoning module is used to generate knowledge retrieval statements for skin disease diagnosis tasks, generate guidance information to guide the visual recognition module to perform secondary recognition, and integrate reasoning information to generate an auxiliary diagnostic report for the target skin disease.

10. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of claims 1-8.

Images