Clinical examination content generation method and system based on multi-modal technology

Abstract

The invention relates to the field of medical teaching, and particularly discloses a clinical examination content generation method and system based on a multi-modal technology, and the method comprises a multi-modal data collection module which is used for collecting clinical medicine related information from a plurality of data sources; the data processing and classifying module is used for processing the clinical medicine related information acquired by the multi-modal data acquisition module and extracting effective medical knowledge and clinical situation data; the data embedding module is used for embedding the extracted data into the text embedding model and converting different types of data in the extracted data into uniform vector representation convenient for processing and training of the text embedding model; the retrieval content generation module is used for performing retrieval in the storage unit embedded into the text model by utilizing a vector retrieval algorithm according to examination requirements to obtain question setting reference data, and generating examination questions in combination with a cue word project; and the content optimization and verification module is used for constructing a real-time feedback mechanism for constructing the text embedding model.

Description

Technical Field

[0001] The present invention relates to the field of medical education technology, and in particular to a method and system for generating clinical examination content based on multimodal technology. Background Art

[0002] With the continuous development of medical education, clinical examinations play a vital role in medical teaching and clinical talent training. Traditional clinical examination content generation mainly relies on experts to manually write test questions, which has problems such as low test question generation efficiency, repeated content, and difficulty in comprehensively evaluating the comprehensive ability of candidates. In addition, with the rapid advancement of medical knowledge and the increasing complexity of clinical cases, a single data source and manual question-setting methods can no longer meet the growing demand for clinical examinations.

[0003] At present, the existing methods for generating clinical examination content usually rely on manual writing and computer-aided design. They usually extract key information such as medical history, symptoms, and diagnosis results from the clinical case library to construct examination questions. The examination content is presented in the form of text description, picture display or video playback, and candidates answer according to the given situation. However, this method is not only cumbersome and time-consuming, but also easily affected by subjective factors, resulting in inconsistent quality of examination content and difficulty in timely updating.

[0004] In summary, the existing methods of automatically generating examination content in medical education rely on manual writing and computer-aided design. The process of constructing examination content is cumbersome and time-consuming, and is affected by human subjective factors, resulting in different emphasis and coverage of knowledge points in the examination content construction process. The constructed examination content is not comprehensive enough, and it is difficult to update the generation logic of the examination content. Summary of the invention

[0005] The purpose of the present invention is to provide a clinical examination content generation method and system based on multimodal technology to solve the technical problems in the prior art, that is, since the existing method of automatically generating examination content in medical education relies on manual writing and computer-aided design, the process of constructing the examination content is cumbersome and time-consuming, and is affected by human subjective factors, resulting in different emphases and coverage of knowledge points in the examination content construction process, the constructed examination content is not comprehensive enough, and it is difficult to update the generation logic of the examination content.

[0006] In order to solve the above technical problems, the present invention specifically provides the following technical solutions: The method for generating clinical examination content based on multimodal technology includes the following steps: Step 100, collecting clinical medicine related information from multiple data sources, wherein the clinical medicine related information includes medical texts, clinical case data, medical images, patient voice and video records; Step 200: Use a multimodal large model to clean, denoise and classify the collected clinical medicine related information to extract effective medical knowledge and clinical context data; Step 300: embed the extracted effective medical knowledge and clinical context data into the multimodal large model using a text embedding model, and convert different types of data in the extracted effective medical knowledge and clinical context data into a unified vector representation that is convenient for processing and training of the multimodal large model; Step 400: According to preset test requirements, a vector retrieval algorithm is used to retrieve question reference data in a storage unit of a text embedding model, and the integrated question reference data is combined with a prompt word project as input to a multimodal large model, and the test content is generated by the multimodal large model.

[0007] As a preferred solution of the present invention, irrelevant or duplicate information in the collected clinical medical related information is removed by cleaning and denoising, and missing data is corrected, and then the data is labeled and classified according to the standards and specifications in the medical field: It includes labeling the image data in the clinical medicine related information into different lesion types and classifying the voice data into different clinical dialogue scenes, labeling the medical entities and relationships related to the clinical dialogue scenes, and statistically summarizing medical keywords.

[0008] As a preferred solution of the present invention, the clinical dialogue scenario includes typical diagnosis and treatment links such as patient complaint collection, doctor's medical examination, diagnosis discussion, treatment plan communication and follow-up education; The marked medical entities include diseases, symptoms, examinations, drugs, and limbs during surgery; and the logic between the marked medical entities is described through causal relationships, treatment associations, and examination indication relationships; Medical keywords are obtained based on a standard terminology library and word frequency analysis statistics, and the medical keywords include symptom keywords, examination keywords and drug keywords.

[0009] As a preferred solution of the present invention, the method for constructing the text embedding model includes: The 4-bit quantized Qwen large language model is used as the underlying pre-trained language model of the text embedding model; The contrastive learning mechanism of SimCSE is connected to the underlying pre-trained language model to generate positive samples by applying random perturbations to the same text to simulate different representations of the same text; Using other randomly selected texts as negative samples, the underlying pre-trained language model is optimized through a contrastive loss function of contrastive learning, so that the learning of the underlying pre-trained language model has discriminative semantic representation.

[0010] As a preferred solution of the present invention, in the selection of negative samples, dynamic mining of hard negative samples of samples that are semantically related to positive samples but have different contents is combined, specifically including: Using the initially trained semantic embedding model, the semantic similarity between each sample in the corpus and the target positive sample is calculated, and samples with semantic similarity to the target sample but not completely consistent in content are selected as hard negative samples; and the hard negative sample library is updated by regularly recalculating the semantic similarity.

[0011] As a preferred solution of the present invention, the hard negative sample is integrated into the loss function of contrastive learning, and the loss function is optimized. as follows: ; in, is the current sample,feature vector; is a positive sample, is a hard negative sample, is a random negative sample, Indicates that hard negative samples are more important, is the temperature parameter; , indicating the sample and The similarity measure between them is cosine similarity.

[0012] As a preferred solution of the present invention, according to the preset examination requirements, including question type design, clinical examination standards and clinical ability assessment requirements, the question reference data related to the target requirements, including medical knowledge data, case instance data and imaging data, are quickly retrieved through a vector retrieval algorithm; The retrieved reference data for questions are integrated and classified into different modalities, including text descriptions, medical images, and voice records, and clinical scenarios are constructed. Combined with the prompt word engineering, the data of different modalities are fused into a unified text format, and the fused content is used as the input of the multimodal big model. The multimodal big model generates the test content through the set question instructions and templates.

[0013] As a preferred solution of the present invention, a real-time feedback mechanism for constructing a text embedding model includes: Recognize the same text format after fusion, and automatically correct the unclear semantics or logical errors identified; Perform consistency checks on data of different modalities to ensure semantic matching between data contents of different modalities; Conduct correlation analysis between the generated test content and the preset test requirements, including first using authoritative medical outlines to build a disease diagnosis and treatment relationship network, combining the model to extract key features of the test requirements, and then through semantic analysis and cognitive level classification to determine whether the generated content is closely related to the test objectives; The coverage rate of the test points of the generated test content in the medical examination standards is determined, specifically including the establishment of a core test point database based on the medical standard terminology library, matching knowledge points through intelligent coding and dynamic weight calculation, and automatically generating supplementary content if the coverage rate is less than the set threshold.

[0014] This embodiment provides a clinical examination content generation system based on multimodal technology for implementing the clinical examination content generation method based on multimodal technology, including: A multimodal data acquisition module, wherein the multimodal data acquisition module collects clinical medicine related information from multiple data sources, wherein the clinical medicine related information includes medical text, clinical case data, medical images, patient voice and video records; A data processing and classification module, which is used to clean, denoise and classify the clinical medicine related information collected by the multimodal data collection module, and extract effective medical knowledge and clinical context data; A data embedding module, which is used to embed the extracted effective medical knowledge and clinical context data into the text embedding model, and at the same time convert different types of data in the extracted effective medical knowledge and clinical context data into a unified vector representation that is convenient for multimodal large model processing and training; A retrieval content generation module, wherein the retrieval content generation module is used to retrieve question reference data in a storage unit of a multimodal large model using a vector retrieval algorithm according to preset test requirements, and input the integrated question reference data into the multimodal large model in combination with a prompt word project, so that the multimodal large model generates test questions; A content optimization and verification module is used to construct a real-time feedback mechanism for building a text embedding model. The implementation feedback mechanism includes: A recognition module, which recognizes the same format of the fused text and automatically corrects the unclear semantics or logical errors found; The checking module is used to check the consistency of different modal data to ensure semantic matching between the contents of different modal data; The correlation module is used to analyze the correlation between the generated test content and the preset test requirements and to determine the test point coverage of the generated test content in the medical test standards.

[0015] As a preferred solution of the present invention, it also includes an expert review and calibration module, which is used to construct a port for manually reviewing the generated test content, and feed back the reviewed content to the data embedding module after text analysis.

[0016] Compared with the prior art, the present invention has the following beneficial effects: The clinical examination content generation method of the present invention is based on multimodal technology. By collecting, processing, classifying, training and converting multimodal data such as medical text, image, voice, etc. into the same text vector expression, and being able to automatically retrieve and integrate according to preset examination requirements, it can realize automated and structured examination content generation.

[0017] The method provided by the present invention can improve the generation efficiency and quality of examination content, simulate complex clinical scenarios as much as possible, and provide a more scientific and accurate examination content generation solution for medical education. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0019] Figure 1 It is an overall flow chart of the objective structured clinical examination content generation method and system based on multimodal technology provided in an embodiment of the present invention.

[0020] Figure 2 It is a schematic diagram of the framework of the objective structured clinical examination content generation method and system based on multimodal technology provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] like Figure 1 As shown, the present invention provides a method and system for generating clinical examination content based on multimodal technology, comprising the following steps: Step 100, collecting clinical medicine related information from multiple data sources, wherein the clinical medicine related information includes medical texts, clinical case data, medical images, patient voice and video records; Step 200: Clean, denoise and classify the collected clinical medicine related information to extract effective medical knowledge and clinical context data; Step 300: embed the extracted effective medical knowledge and clinical context data into the multimodal large model using a text embedding model, and convert different types of data in the extracted effective medical knowledge and clinical context data into a unified vector representation that is convenient for processing and training of the multimodal large model; Step 400: According to preset test requirements, a vector retrieval algorithm is used to retrieve question reference data in a storage unit of a text embedding model, and the integrated question reference data is input into a multimodal large model in combination with a prompt word project, and the test content is generated by the multimodal large model.

[0023] The irrelevant or duplicate information in the collected clinical medical related information is removed by cleaning and denoising, and the missing data is corrected. Then the data is labeled and classified according to the standards and specifications in the medical field: It includes labeling the image data in the clinical medicine related information into different lesion types and classifying the voice data into different clinical dialogue scenes, labeling the medical entities and relationships related to the clinical dialogue scenes, and statistically summarizing medical keywords.

[0024] Specifically, according to the standards and specifications in the medical field, multimodal data are labeled and classified, imaging data are labeled into different lesion types, voice data are classified into different clinical dialogue scenarios, and relevant medical entities and relationships are labeled, and medical keywords are statistically summarized.

[0025] In a specific embodiment, the clinical dialogue scenario covers typical diagnosis and treatment links such as collection of patient complaints, doctor's interview and physical examination, diagnosis discussion, communication of treatment plans and follow-up education.

[0026] Medical entity annotations include types such as diseases, symptoms, examinations, drugs, and anatomical parts, and describe the logic between entities through relationships such as causality, treatment associations, and examination indications.

[0027] Medical keywords are based on standard terminology and word frequency analysis statistics, such as symptom keywords such as chest pain, examination keywords such as electrocardiogram, and drug keywords such as aspirin, and are combined with multimodal data annotations such as image lesion types and voice dialogue scenarios to achieve unified data vectorization. Ultimately, the multimodal large model is supported to generate test questions that fit real clinical scenarios, such as designing differential diagnosis questions by combining CT images with interview texts.

[0028] The method for constructing the text embedding model includes: The 4-bit quantized Qwen large language model is used as the underlying pre-trained language model of the text embedding model; The contrastive learning mechanism of SimCSE is connected to the underlying pre-trained language model to generate positive samples by applying random perturbations to the same text to simulate different representations of the same text; Using other randomly selected texts as negative samples, the underlying pre-trained language model is optimized through a contrastive loss function of contrastive learning, so that the learning of the underlying pre-trained language model has discriminative semantic representation.

[0029] Unlike traditional text embedding models such as BERT-based SimCSE, which usually adopt full-precision pre-training models, this method innovatively uses the 4-bit quantized Qwen large language model as the underlying architecture, significantly reducing storage and computing costs. At the same time, through the improved SimCSE contrastive learning mechanism, random perturbations are applied to the same text to generate positive samples, and random text is used as negative samples to compensate for the semantic loss that may be caused by quantization, thereby enhancing the robustness to medical text noise.

[0030] In addition, combined with the pre-training advantages of the Qwen model on medical corpus, we further introduced difficult negative sample mining strategies such as sampling negative examples from similar disease descriptions, so that the model can perform better in clinical terminology distinction such as "hepatitis" and "cirrhosis" and multimodal alignment such as imaging reports and medical consultation text tasks.

[0031] Actual tests show that this method significantly outperforms traditional solutions in terms of medical text similarity matching accuracy while maintaining efficient reasoning.

[0032] In the negative sample selection, we combine dynamic mining of hard negative samples that are semantically related to positive samples but have different contents, including: Using the initially trained semantic embedding model, the semantic similarity between each sample in the corpus and the target positive sample is calculated, and samples with semantic similarity to the target sample but not completely consistent in content are selected as hard negative samples; and the hard negative sample library is updated by regularly recalculating the semantic similarity.

[0033] Furthermore, hard negative samples are incorporated into the loss function of contrastive learning of the underlying pre-trained language model to optimize the objective function as follows: ; in, is the current sample,feature vector; is a positive sample, is a hard negative sample, is a random negative sample, Indicates that hard negative samples are more important, is the temperature parameter; , indicating the sample and The similarity measure between them is cosine similarity.

[0034] According to the preset examination requirements, including question type design, clinical examination standards and clinical ability assessment requirements, vector retrieval algorithms are used to quickly retrieve question reference data related to the target requirements, including medical knowledge data, case instance data and imaging data. For example, electrocardiogram data, patient complaint text and first aid recordings are extracted from the chest pain center case library.

[0035] The retrieved reference data for questions are integrated and classified into different modalities, including text descriptions, medical images, and voice records, and clinical scenarios are constructed. Combined with the prompt word engineering, the data of different modalities are fused into a unified text format, and the fused content is used as the input of the text embedding model. The text embedding model generates the test content through the set question instructions and templates.

[0036] Build a real-time feedback mechanism for text embedding models, including: Recognize the same text format after fusion, and automatically correct the unclear semantics or logical errors identified; Perform consistency checks on data of different modalities to ensure semantic matching between data contents of different modalities; The generated test content is analyzed for correlation with the preset test requirements. The system double-checks the relationship between the medical knowledge graph and cognitive level matching: first, the disease diagnosis and treatment relationship network is constructed using the authoritative medical outline, and the key features of the test requirements are extracted by combining the model. Then, semantic analysis and cognitive level classification are performed to ensure that the generated content is closely related to the test objectives. The coverage rate of the test points in the generated test content in the medical examination standards is determined, and a core test point database is established based on the medical standard terminology library. The knowledge points are matched through intelligent coding and dynamic weight calculation. If the coverage rate is less than 85%, supplementary content is automatically generated, combined with sampling review by medical experts, to ultimately ensure that the test questions meet the syllabus requirements and comprehensively cover key knowledge.

[0037] This embodiment provides a clinical examination content generation system based on multimodal technology for implementing the clinical examination content generation method based on multimodal technology, including: A multimodal data acquisition module, wherein the multimodal data acquisition module collects clinical medicine related information from multiple data sources, wherein the clinical medicine related information includes medical text, clinical case data, medical images, patient voice and video records; A data processing and classification module, which is used to clean, denoise and classify the clinical medicine related information collected by the multimodal data collection module, and extract effective medical knowledge and clinical context data; A data embedding module, which is used to embed the extracted effective medical knowledge and clinical context data into a text embedding model, and at the same time convert different types of data in the extracted effective medical knowledge and clinical context data into a unified vector representation that is convenient for multimodal large model processing and training, that is, the data embedding module is specifically a text embedding model; A search content generation module, wherein the search content generation module is used to retrieve question reference data in a storage unit of a text embedding model using a vector search algorithm according to preset test requirements, and generate test questions by combining the integrated question reference data with a prompt word project; A content optimization and verification module is used to construct a real-time feedback mechanism for building a text embedding model. The implementation feedback mechanism includes: A recognition module, which recognizes the same format of the fused text and automatically corrects the unclear semantics or logical errors found; The checking module is used to perform consistency checks on data of different modalities to ensure semantic matching between the contents of data of different modalities; to perform consistency checks on multimodal content text, images, and audio to ensure semantic matching between the contents of different modalities and avoid distortion of test questions due to inconsistent information.

[0038] The correlation module is used to analyze the correlation between the generated test content and the preset test requirements and to determine the test point coverage of the generated test content in the medical test standards.

[0039] Conduct relevance analysis on the exam content to ensure that it meets the theme and requirements of the target exam and covers the key knowledge points and clinical scenarios in the medical exam standards.

[0040] It also includes an expert review and calibration module, which is used to build a port for manually reviewing the generated test content, and feed back the reviewed content to the data embedding module after text analysis.

[0041] The expert review and standard calibration module combines the feedback from medical experts and industry standards to conduct expert review and calibration of the generated test content. It should be noted that the expert review and standard calibration specifically includes establishing an expert review and feedback mechanism, inviting medical experts to conduct manual review to ensure that the quality of the questions meets the requirements; a set of standardized calibration processes can be implemented to ensure that the generated test content strictly complies with the current professional knowledge specifications, industry standards and examination requirements in the medical field.

[0042] The expert review and calibration module is used to ensure the accuracy and professionalism of the generated clinical examination content, and mainly includes four aspects: manual review port, review standard setting, text analysis feedback and continuous optimization.

[0043] This module provides an interactive interface that allows medical experts to review and modify test content, and set review standards based on industry standards to ensure consistency.

[0044] The reviewed content undergoes text analysis to extract key medical terms and perform consistency checks, and feedback information is generated and sent back to the data embedding module to optimize the training data.

[0045] In addition, this module combines expert feedback and system self-learning mechanisms to continuously optimize the content generation model and improve its adaptability and accuracy to clinical examination needs.

[0046] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.

Claims

1. A method for generating clinical examination content based on multimodal technology, characterized in that: The steps include: Step 100, collecting clinical medicine related information from multiple data sources, wherein the clinical medicine related information includes medical texts, clinical case data, medical images, patient voice and video records; Step 200: Use a multimodal large model to clean, denoise and classify the collected clinical medicine related information to extract effective medical knowledge and clinical context data; Step 300: embed the extracted effective medical knowledge and clinical context data into the multimodal large model using a text embedding model, and convert different types of data in the extracted effective medical knowledge and clinical context data into a unified vector representation that is convenient for processing and training of the multimodal large model; Step 400: According to preset test requirements, a vector retrieval algorithm is used to retrieve question reference data in a storage unit of a text embedding model, and the integrated question reference data is combined with a prompt word project as input to a multimodal large model, and the test content is generated by the multimodal large model.

2. The method for generating clinical examination content based on multimodal technology according to claim 1, characterized in that: The irrelevant or duplicate information in the collected clinical medical related information is removed by cleaning and denoising, and the missing data is corrected. Then the data is labeled and classified according to the standards and specifications in the medical field: It includes labeling the image data in the clinical medicine related information into different lesion types and classifying the voice data into different clinical dialogue scenes, labeling the medical entities and relationships related to the clinical dialogue scenes, and statistically summarizing medical keywords.

3. The method for generating clinical examination content based on multimodal technology according to claim 2, characterized in that: The clinical dialogue scenarios include typical diagnosis and treatment links such as patient complaint collection, doctor's interview and physical examination, diagnosis discussion, treatment plan communication and follow-up education; The marked medical entities include diseases, symptoms, examinations, drugs, and limbs during surgery; and the logic between the marked medical entities is described through causal relationships, treatment associations, and examination indication relationships; Medical keywords are obtained based on a standard terminology library and word frequency analysis statistics, and the medical keywords include symptom keywords, examination keywords and drug keywords.

4. The method for generating clinical examination content based on multimodal technology according to claim 1, characterized in that: The method for constructing the text embedding model includes: The 4-bit quantized Qwen large language model is used as the underlying pre-trained language model of the text embedding model; The contrastive learning mechanism of SimCSE is connected to the underlying pre-trained language model to generate positive samples by applying random perturbations to the same text to simulate different representations of the same text; Using other randomly selected texts as negative samples, the underlying pre-trained language model is optimized through a contrastive loss function of contrastive learning, so that the learning of the underlying pre-trained language model has discriminative semantic representation.

5. The method for generating clinical examination content based on multimodal technology according to claim 4, characterized in that: In the negative sample selection, we combine dynamic mining of hard negative samples that are semantically related to positive samples but have different contents, including: Using the initially trained semantic embedding model, the semantic similarity between each sample in the corpus and the target positive sample is calculated, and samples with semantic similarity to the target sample but not completely consistent in content are selected as hard negative samples; and the hard negative sample library is updated by regularly recalculating the semantic similarity.

6. The method for generating clinical examination content based on multimodal technology according to claim 5, characterized in that: The hard negative samples are integrated into the loss function of contrastive learning, and the loss function is optimized. as follows: ； in, is the current sample,feature vector; is a positive sample, is a hard negative sample, is a random negative sample, Indicates that hard negative samples are more important, is the temperature parameter; , indicating the sample and The similarity measure between them is cosine similarity.

7. The method for generating clinical examination content based on multimodal technology according to claim 1, characterized in that: According to the preset examination requirements, including question type design, clinical examination standards and clinical ability assessment requirements, the vector retrieval algorithm is used to quickly retrieve the reference data related to the target requirements, including medical knowledge data, case instance data and imaging data; The retrieved reference data for questions are integrated and classified into different modalities, including text descriptions, medical images, and voice records, and clinical scenarios are constructed. Combined with the prompt word engineering, the data of different modalities are fused into a unified text format, and the fused content is used as the input of the multimodal big model. The multimodal big model generates the test content through the set question instructions and templates.

8. The method for generating clinical examination content based on multimodal technology according to claim 1, characterized in that: Build a real-time feedback mechanism for text embedding models, including: Recognize the same text format after fusion, and automatically correct the unclear semantics or logical errors identified; Perform consistency checks on data of different modalities to ensure semantic matching between data contents of different modalities; Conduct correlation analysis between the generated test content and the preset test requirements, including first using authoritative medical outlines to build a disease diagnosis and treatment relationship network, combining the model to extract key features of the test requirements, and then through semantic analysis and cognitive level classification to determine whether the generated content is closely related to the test objectives; The coverage rate of the test points of the generated test content in the medical examination standards is determined, specifically including the establishment of a core test point database based on the medical standard terminology library, matching knowledge points through intelligent coding and dynamic weight calculation, and automatically generating supplementary content if the coverage rate is less than the set threshold.

9. A clinical examination content generation system based on multimodal technology for implementing the clinical examination content generation method based on multimodal technology as described in claims 1 and 8, characterized in that: Included in the modal model embed: A multimodal data acquisition module, wherein the multimodal data acquisition module collects clinical medicine related information from multiple data sources, wherein the clinical medicine related information includes medical text, clinical case data, medical images, patient voice and video records; A data processing and classification module, which is used to clean, denoise and classify the clinical medicine related information collected by the multimodal data collection module, and extract effective medical knowledge and clinical context data; A data embedding module, which is used to extract effective medical knowledge and clinical context data, embed them into the multimodal large model through a text embedding model, and convert different types of data in the extracted effective medical knowledge and clinical context data into a unified vector representation that is convenient for processing and training of the multimodal large model; A retrieval content generation module, wherein the retrieval content generation module is used to retrieve question reference data in a storage unit of a text embedding model using a vector retrieval algorithm according to preset test requirements, and input the integrated question reference data into a multimodal large model in combination with a prompt word project, so that the multimodal large model generates test questions; Content optimization and verification module, the content optimization and verification module is used to build a real-time feedback mechanism for the text embedding model, the implementation feedback mechanism includes: A recognition module, which recognizes the same format of the fused text and automatically corrects the unclear semantics or logical errors found; The checking module is used to check the consistency of different modal data to ensure semantic matching between the contents of different modal data; The correlation module is used to analyze the correlation between the generated test content and the preset test requirements and to determine the test point coverage of the generated test content in the medical test standards.

10. The clinical examination content generation system based on multimodal technology according to claim 9, characterized in that: It also includes an expert review and calibration module, which is used to build a port for manually reviewing the generated test content, and feed back the reviewed content to the data embedding module after text analysis.

Images