263 results about "Medical report" patented technology

The decision process of a first machine learning (ML) model may be explained based on a second ML model implemented on an apparatus. The apparatus may obtain a prediction about an image made based on the first ML model. The apparatus may further determine visual concepts associated with the image that may have been used by the first ML model to make the prediction, and determine respective contributions of the visual concepts to the prediction made by the first ML model. The apparatus may then generate, based on the second ML model, a textual description that explains the respective contributions of the visual concepts to the prediction made by the first ML model. The second ML model may determine respective image features associated with the visual concepts, map the determined image features to corresponding text features, and generate the textual description based at least on the text features.

The invention relates to the technical field of medical health, in particular to an intelligent analysis system based on multi-modal medical data, which comprises a multi-modal feature extraction module used for extracting local and global features from medical images and text reports; the cross-modal alignment module is used for aligning potential representations of image and text modals through a variational auto-encoder and a decoupling characterization method; and the cross-modal interaction and fusion module is used for carrying out interaction and fusion on the modal features through a multi-head attention mechanism and a semantic guidance mechanism to generate shared multi-modal representation. According to the method, complementary information of medical images and report texts can be effectively integrated, meanwhile, the integrity of information in modals is ensured, meanwhile, through design and optimization of a plurality of innovative modules, the framework shows excellent performance in cross-modals of medical report analysis, and the method is suitable for popularization and application. An efficient and robust solution is provided for complex medical image and text report analysis.

A neural network system is trained to generate textual reports (that is, medical reports, such as radiology reports) from one or more medical images, by fine-tuning a pre-trained neural network system (a visual language model, "VLM") operative, upon receiving an input comprising at least one image and a textual input, to generate a value indicative of a predicted likelihood of one or more candidate text continuations of the textual input. The fine-tuning of the neural network system is performed to reduce the value of a cost function which includes a first prediction cost term based on a first training database including first training datasets of at least one medical image and an associated text report, the first training datasets corresponding to first individuals. The first prediction cost term further includes a cost value for each individual, inversely dependent on a likelihood value of the associated textual report, conditioned on the at least one medical image, and created by the neural network system.

The invention discloses a three-dimensional medical image visual language model pre-training method for reconstructing a hybrid strategy, which belongs to the technical field of medical image calculation and comprises the steps of constructing a medical image text pair data set, a language text mask reconstruction strategy, a visual image mask reconstruction strategy, a semantic perception fusion strategy and multi-task joint learning. According to the method, the large language model is finely adjusted, the diagnosis and attribute information in the medical report is extracted by using the finely-adjusted large language model, the efficient prompt is generated, and the large language model has relatively high generalization ability, so that the cost of manual annotation is greatly saved; according to the semantic perception fusion strategy, the text feature obtained by the text encoder and the image feature obtained by the image encoder are combined to obtain the new text feature, so that the text perceives the diagnosis and attribute information of the image in advance, the alignment of the image and the text in the embedding space is further optimized, and the pre-training efficiency is improved.

The invention relates to an intelligent cervical cancer screening method based on a multi-modal large language model, which comprises the following steps: data collection and processing: collecting and processing colposcope images and cervical cancer related medical reports, and pre-training a model for extracting key image and text features and providing input for the multi-modal model to enhance the accuracy and efficiency of cervical cancer screening; setting a multi-modal fusion model; fusing the image features and the text features; model training and tuning: taking the colposcope image and the corresponding text data as input, carrying out joint training on the multi-modal model, and finding an optimal configuration; and model evaluation and verification: carrying out performance evaluation on the model, analyzing the performance of the model on different categories and carrying out clinical verification. According to the method, colposcope images can be efficiently processed and analyzed, cervical cancer screening and diagnosis can be processed, a multi-modal large language model of a medical report can be generated, prognosis prediction, disease progress prediction and risk assessment can be carried out, and doctors can be helped to make more effective diagnosis and treatment plans.

Various systems and methods are provided for generating and editing of medical reports and records using artificial intelligence (AI) assistance. An intelligent medical reporting tool may edit medical materials such as electronic medical records, medical reports, and treatment plans. The intelligent medical reporting tool can provide edit recommendations and suggested additions to medical materials to a user of the tool. Edit recommendations and suggested additions may be related to contradictions, incompleteness, clarity, and clinical guidelines.

The described technology is generally directed towards automated and dynamic creation of a consolidated summary of medical information pertaining to a patient. Rather than a caregiver having to trawl through numerous reports and medical information, utilizing consolidated reports enables efficient review of medical information to enable expedited determination and treatment of the patient. In an example sequence of operation, a collection of patient / medical reports can be individually summarized to form a collection of summarized versions of the original reports. The summarized reports can be further collectively summarized to generate a consolidated summary. Hence, the consolidated summary is a summary of the initial collection of patient / medical records. A consolidated summary can comprise patient information, medical history, medical images, pertinent medical research, etc. A consolidated summary can be presented on an interactive display, with the respective initial patient / medical reports accessible via selectable hyperlinks presented in the consolidated summary.

The invention discloses a medical report generation method, a model training method, equipment and a medium, and the model training method comprises the steps: constructing a medical report generation model framework which comprises a global semantic collaborative multi-modal enhancement module, a visual encoder, a text encoder, a medical insight analyzer and an LLM decoder; wherein the global semantic collaborative multi-modal enhancement module respectively enhances a medical image and a medical report by utilizing a selected image enhancement strategy and a text enhancement strategy, and the medical insight analyzer comprises a fine-grained structure learning device and a global context guide learning device which are connected in sequence so as to enhance the cross-modal alignment capability; and performing intelligent collaborative optimization by taking a strategy set formed by an image enhancement strategy and a text enhancement strategy and architecture configuration parameters of the medical insight analyzer as optimization targets to obtain an optimal medical report generation model. The medical report generation performance can be effectively improved.

A generation method and generation apparatus of a medical report are provided. In the method, the writing style is analyzed from multiple historical texts, where the writing style includes multiple common words in the historical text and the contextual relationships that connect those common words; the medical data is converted into draft text that conforms to the template text, where the template text is a report that conforms to a preset style; and by using the draft text and writing style as input data of the language model, an output report that conforms to the writing style is generated, where the language model selects sentences that conform to the writing style.

The invention discloses a method for automatically generating a medical report by using multi-modal feature fusion Mamba. The method comprises the following steps: inputting an original medical image into a visual feature extraction module to output visual features; inputting the original medical report into a text feature extraction module, and outputting text features; respectively inputting the visual features and the text features into an SSM-Attention visual text feature integration module, and outputting new visual features and text features; inputting the new visual features into a Transform encoder, enhancing the visual features, and extracting advanced visual features; and inputting the text features and the enhanced advanced visual features into a multi-modal fusion Mama module to output fusion features, and inputting the fusion features into a Transform decoder to automatically generate a medical report. According to the invention, the SSM-Attention and the Multi-modal Mama can be independently developed and improved, newly developed technical modules can be conveniently integrated, and the method is not only limited to fusion of medical image and text features, but also suitable for fusion of any multi-modal features.

The invention discloses a medical image diagnosis report generation method and device, equipment, a medium and a product. According to the method, image features of a radiology image are extracted, a medical knowledge graph is encoded as priori knowledge to help identify abnormal areas in the image, and correlation among some disease expressions is considered through modeling of disease keywords. Integrating the extracted image features and the medical knowledge graph features by using a graph attention mechanism to obtain visual fusion features; and inputting the visual fusion features into a pre-trained target prediction model to finally obtain predicted continuous semantic space vectors, and further obtaining a generated radiology medical report through mapping of discrete words and the continuous semantic space vectors, so that medical image features and text disease keyword representations can be accurately aligned. And the accuracy of diagnosis report generation is further improved.

The invention discloses a processing method for structural information extraction and privacy protection of a medical report, which relates to the field of medical report processing and can efficiently and accurately extract text information in a table by integrating advanced image preprocessing, dynamic layout analysis, OCR (optical character recognition) technologies and a dual-channel privacy detection model. Establishing a coordinate mapping relation between the image description paragraph and the corresponding image; the generated JSON structure containing the text, table and image-text mapping relation ensures the integrity and accuracy of the data; the problems of data dislocation, context splitting, privacy disclosure and the like when the medical document with complex typesetting is processed in the prior art are solved; besides, an improved Transform framework is constructed on the basis of a Qwen model, a logical reasoning chain is injected, meanwhile, a case reading large model is obtained by integrating clinical diagnosis rules and multi-modal contrast learning strategy training, the model is utilized to generate a high-quality structured abstract, and the abstract generation accuracy is greatly improved.

The invention discloses a chest radiology medical report generation method based on a knowledge graph, and the method comprises the steps: obtaining a to-be-processed chest radiology image, inputting the to-be-processed chest radiology image into a trained medical report generation network, and obtaining a chest radiology medical report. The training process of the network comprises the following steps: acquiring a chest radiology medical report set; constructing a chest radiology knowledge graph; extracting text features, image features and knowledge features from the chest radiology medical report, inputting the text features, the image features and the knowledge features into a double-path knowledge enhancement gating module, aligning the text features and the image features, aligning the image features and the knowledge features, and calculating alignment loss; fusing the aligned features to obtain enhanced fusion features; inputting the enhanced fusion features into a classifier, and calculating classification loss; the fusion features are input into a decoder, a chest radiology medical report is generated, and generation loss is calculated; and network parameters are optimized through loss back propagation. According to the invention, the medical report generation quality and accuracy can be obviously improved.

Disclosed herein are systems, media, and methods for medical report generation or evaluation. Natural language processing, computer vision, large language models, or other software may be used to facilitate accurate, efficient evaluation of contemporary or historical medical reports. The present disclosure may provide particular utility in patient health diagnostics, prognostics, treatment, or monitoring.

The present invention provides a computer-implemented, machine learning method for generating safe text. A first portion of a trainable prompt is generated using negative influential features and positive influential features of a predicted condition. A second portion of the trainable prompt is trained to steer a pre-trained large language model (PLLM) to generate the safe text using at least the first portion of the trainable prompt. The method has applications including, but not limited to, use cases in medicine (e.g., digital medicine, personalized healthcare, AI-assisted drug or vaccine development, diagnosis or treatment, disease prediction, etc.), and cyber security.

The invention discloses a method for automatically generating a medical image diagnosis report based on a graph convolutional network, and the method achieves the extraction of specific pathological information from a medical image and the generation of an accurate diagnosis report through the combination of medical image processing and text generation technologies. Guiding image feature extraction by using a disease label, and performing targeted extraction and discretization processing on pathological information in a medical image through a GCN (Graph Convolutional Network); besides, the designed cross-modal alignment module can effectively connect the medical image, the diagnosis report and the disease label, the semantic consistency between the image and the text is enhanced, and the extra workload is reduced through the pre-constructed relation matrix. According to the method, the quality and efficiency of medical report generation are remarkably improved, a new technical path is provided for automatic generation of the medical image diagnosis report, and clinical practicability and calculation efficiency are considered at the same time.

The invention relates to the technical field of medical report generation, in particular to an intelligent anesthesia record sheet automatic generation system. According to the system, patient information is automatically recognized through the OCR unit, and real-time collection and structured storage of physical sign data, medication records and surgical operation are achieved in combination with a medical equipment interface and a voice recognition technology; time sequence data analysis and multi-source data fusion modeling are adopted to construct a medical event causal relationship graph so as to improve the accuracy of anomaly recognition and patient sign evaluation; based on a time axis construction unit and a structured template engine, an anesthesia record list is automatically analyzed and filled, and the manual input workload is greatly reduced; and finally, in combination with block chain evidence storage and a multi-level encryption technology, the security and non-tampering property of the anesthesia record list are ensured, the reliability of medical data management is improved, the recording, monitoring and storage efficiency of the anesthesia process is comprehensively optimized, and the medical quality and the patient safety are improved.

Medical report data (e.g., a paragraph of text that is dictated by a radiologist as they review a medical image) is automatically parsed and associated with an appropriate section (e.g., Lungs, Mediastinum, Heart, Bones, etc.) of a medical report. Each report section of a template report may be associated with character strings that are used to associate portions of the medical report data with a corresponding report section. Natural Language Processing may then be applied to each report section (and any associated text from the medical report data) to determine whether the section includes a “finding” or “no finding.” The medical report may then be used to train an AI model to emulate the analyzed medical report data.

An apparatus for the generation of a medical report is disclosed. The apparatus includes at least processor and a memory communicatively connected to the processor. The memory instructs the processor to receive a user query. The memory instructs the processor to receive a user profile comprising a plurality of medical tests. The memory instructs the processor to generate testing data as a function of the plurality of medical tests using an encoder. The memory instructs the processor to generate textual data that is representative of the testing data using a querying transformer model (Q-former). The memory instructs the processor to generate a medical report as a function of the user query and the textual data using a report large language model (LLM).

The embodiment of the invention discloses an electronic medical record generation method and device, electronic equipment and a storage medium. The method comprises the following steps: in response to an electronic medical record generation instruction for a skin inquiry service, obtaining interaction content between a provider and an acquirer of the skin inquiry service and a deep learning model for realizing electronic medical record generation; acquiring skin data of the skin lesion part inquired by the acquirer; wherein the skin data comprises at least one of a skin image and a skin medical report; and generating an electronic medical record corresponding to the skin inquiry service according to the deep learning model, the interaction content and the skin data. According to the technical scheme provided by the embodiment of the invention, on the basis of the interactive content, the skin data capable of providing main auxiliary diagnosis information for a doctor is combined, so that the deep learning model can be utilized to process the multi-modal information (namely, the interactive content and the skin data) to generate the electronic medical record; therefore, the generation quality of the electronic medical record for the skin inquiry service is improved.

The invention discloses a cross-modal gating fusion radiology report generation method combined with clinical semantic enhancement, and relates to the technical field of medical report generation. A semantic enhancement decoding unit is used for performing deep fusion on image sequence representation, historical text embedding and clinical semantic representation to obtain final enhancement decoding representation of multi-modal semantic enhancement; and performing report generation on the final enhanced decoding representation output by the semantic enhanced decoding unit to obtain a predicted radiology report. The predicted radiology report generated by the radiology report generation method is closer to a radiology report sample text in the aspects of sentence structure and coherence.

The invention discloses a similarity report prompt cross-modal enhancement medical report automatic generation method, which comprises the following steps: an image feature extraction module extracts multi-scale visual features, and a text feature coding module extracts text features; the medical image in the training stage has a corresponding report; in the testing stage, similar cases are selected for medical images used for testing through a dynamic prompt generation module, text features of text reports corresponding to the similar cases are subjected to weighted fusion and output to obtain dynamic prompt vectors, and the dynamic prompt vectors are used for replacing the corresponding reports; the cross-modal feature fusion module dynamically fuses the image features and the text features, and outputs to obtain a fusion feature vector; the fusion feature vector and an unfinished report generated by autoregression are input into the CMN together to obtain enhanced features; and the report generation module outputs a natural language-based diagnostic report text for the enhanced feature processing. According to the method, a dynamic prompt mechanism and cross-modal attention are adopted, irrelevant noise interference is reduced, and misdiagnosis risks are reduced.

The invention discloses a medical report generation method and device, a storage medium and electronic equipment, and the method comprises the steps: obtaining patient information, inputting the patient information into a report generation model, obtaining diagnosis features, determining historical diagnosis information related to the patient information in a historical diagnosis record as associated information according to the diagnosis features, and generating a medical report according to the associated information. And determining prompt information, inputting the prompt information and the diagnosis features into a report generation model, and generating a medical report. According to the method, a doctor only needs to input the patient information into the report generation model, the medical report generated by the model can be obtained, the doctor does not need to manually write the medical report, and the determination efficiency of the medical report is improved.

An apparatus for diagnosing a medical condition including at least a processor and a memory, the memory containing instructions configuring the at least a processor to receive a medical image that is the result of a medical imaging procedure and train a medical image classifier and a medical report classifier. Training the classifiers includes receiving training data, including a plurality of prior medical images and a plurality of prior medical reports, training the medical image classifier and the medical report classifier, and optimizing both classifiers using a common loss function. The memory further containing instructions configuring the at least a processor to generate a label for the medical image, including inputting the medical image into the medical image classifier and receiving the label as output from the medical image classifier.

The invention discloses a multi-task large language model training method oriented to the field of biomedicine, belongs to the field of large language model training, aims to solve the problem of realizing multi-task processing by a single large language model, and has the technical key points that a training set of an instruction data set for medical intelligent question and answer tasks is constructed; constructing a training set of a second instruction data set for the medical report generation task; constructing a training set of a third instruction data set for the biomedical information extraction task; and training the large language model by using the training set. The method has the effect that accurate and efficient processing of multiple tasks of the single large language model can be realized.

The embodiment of the invention relates to a method and a device for generating a medical report based on evidence-based medicine. The method comprises the following steps: acquiring patient information and an original clinical problem; inputting the patient information, the original clinical question and the first cue word into a target model to generate a structured clinical question; the target model is a medical field large language model; retrieving a medical literature library according to the clinical problem to obtain a plurality of medical literatures; inputting the target medical literature and the second cue word into a target model, so that the target model performs bias risk assessment on the target literature, and a structured initial assessment result of the target literature is generated; inputting the clinical problem, each medical literature, the initial evaluation result of each medical literature and the third cue word into a target model, and enabling the target model to carry out thinking chain gradual reasoning based on a GRADE evaluation process in the third cue word to generate a medical report; the medical report includes medical suggestions for clinical issues, and recommendation intensities and evidence quality levels of the medical suggestions.

A computer-implemented method for creating medical reports, in particular physician's letters, comprises the acquisition of patient information from various sources of electronic health records, with the latter being available in natural-language sampling points. Medical units are determined by computer-aided pattern recognition and classified according to their event type. Event groups that have temporal, thematic or treatment-related relationships are formed. Relationship information is derived and stored in a structured data representation. In addition, these data are enriched with an external medical knowledge base and converted into a format that can be processed by a generative language model in order to generate the continuous text for the medical report.

The invention discloses a medical report interpretation method and a related device, which can be used in the field of artificial intelligence, and the method comprises the steps: firstly, obtaining a medical report; the medical report comprises at least one of text information, image information and table information; then, text information in the medical report is recognized through an optical character recognition technology, and a text recognition result is obtained; identifying the medical report based on the text identification result through a multi-modal large model to obtain a first identification result; the multi-modal large model carries out semantic understanding based on the first recognition result and the target prompt word, and report interpretation content is generated; the report interpretation content comprises a report abstract and personalized suggestions. Therefore, the optical character recognition technology and the multi-mode large model are combined, the defect that medical report solving and interpretation are difficult to accurately conduct depending on single-mode data is overcome, and a rapid, accurate and comprehensive medical report interpretation scheme is provided.

The invention relates to the technical field of medical natural language processing, in particular to a medical report automatic analysis and structured generation method based on a multi-modal vision-language model. The method comprises the following steps: S1, acquiring a target medical image and an initial text report corresponding to the target medical image, and performing feature extraction and preprocessing to obtain a multi-level visual feature sequence and a text feature sequence; s2, obtaining at least one focus-description cluster corresponding to the independent clinical semantic unit according to the multi-level visual feature sequence and the text feature sequence; s3, for each focus-description cluster, the fuzzy medical description in the focus-description cluster is converted into structured attribute information with a deterministic state label; and S4, based on all the focus-description clusters and the corresponding structured attribute information, carrying out content assembly and generation under logic constraints, and outputting a structured medical report with a confidence identifier. According to the invention, deep understanding and high-reliability structured output of the medical report are realized.

The invention relates to the technical field of medical report generation, and discloses a medical image automatic annotation and report generation system which comprises a preprocessing module, a hierarchical feature extraction module, a semantic annotation module, a guide generation module, a diagnosis reasoning module and a man-machine cooperation interface. According to the method, through standardized preprocessing, hierarchical feature extraction and automatic generation of the annotation and the report, the processing efficiency is greatly improved, and the manual load is reduced; secondly, on the basis of hierarchical feature extraction and semantic and visual embedding alignment, feature support and semantic mapping precision of annotation and reporting are enhanced, and result accuracy and cross-scene consistency are improved; meanwhile, multi-modal alignment processing and multi-dimensional information fusion enhance the adaptability of the system to different image types, generate a structured report conforming to the clinical process and improve the clinical applicability; in addition, a man-machine cooperation feedback mechanism forms a parameter optimization closed loop, so that the system dynamically adapts to new scenes and specifications, and continuous improvement of generalization ability is realized.