Semantic-based medical image report template generation system

Abstract

The invention relates to the technical field of medical image processing, and discloses a semantic-based medical image report template generation system, which comprises an image input module used for inputting a medical image of a patient, an image acquisition module used for acquiring a picture of the medical image, and an image processing module used for processing the picture of the medical image. The lesion area recognition module is used for recognizing one or more lesion areas in a medical image, the feature extractor is used for extracting lesion feature information in the medical image, and the lesion area recognition module recognizes lesion parts on the medical image through the arrangement of the lesion area module. The lesion area recognition module adopts an image segmentation algorithm and is used for splitting and positioning a plurality of lesion areas in the medical image, lesion position images are extracted through a feature extractor after recognition, and a report is generated conveniently, so that lesions of a plurality of parts in the medical image are analyzed more accurately.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and more specifically to a semantic-based medical image report template generation system. Background Art

[0002] The medical imaging report template is a standardized document format used to record and present the results and descriptions of medical imaging examinations, such as CT and MRI. It includes information such as the date and time of the examination, patient information, imaging findings, diagnosis results, comprehensive evaluation, and doctor's signature.

[0003] In the related art, it is proposed that under normal circumstances, a doctor analyzes the medical image, gives the analysis result of the image, and generates a medical image report based on the analysis result and the medical image. However, the process of determining the analysis result takes a long time. For example, the existing publication number CN116864064A is a method, device and storage medium for generating a medical image report, which obtains a medical image of a patient; determines an image of a region of interest in the medical image based on the medical image; determines the image of the medical image based on the image of the region of interest; obtains the image analysis result in a language form; and generates a medical image report based on the image analysis result.

[0004] Although the above-mentioned existing technical solutions can achieve the effect of automatically generating medical imaging reports by automatically analyzing medical images, they still have the following defects: although this method can view and analyze the lesion sites in the medical images, if there are lesions in multiple sites, there will be blurred edges, resulting in inaccurate analysis results and affecting the judgment of the patient. Summary of the invention

[0005] In order to overcome the above-mentioned defects of the prior art, the present invention provides a semantic-based medical imaging report template generation system to solve the problems existing in the above-mentioned background technology.

[0006] The present invention provides the following technical solution: a semantic-based medical imaging report template generation system, comprising an image input module, wherein the image input module is used to input a patient's medical image; An image acquisition module, wherein the image acquisition module is used to acquire medical image images; A lesion region recognition module, the lesion region recognition module is used to recognize one or more lesion regions in a medical image; A feature extractor, wherein the feature extractor is used to extract lesion feature information in a medical image; An image confirmation module, which is used to determine the authenticity of the lesion area, that is, whether the lesion actually occurs; MongoDB database, wherein the MongoDB database is used to store the pre-trained model; A natural language generation module and a text generation module, wherein the natural language generation module and the text generation module are used to convert the analyzed information into understandable text information; An output module is used to output information.

[0007] Furthermore, the medical imaging report template generation system further comprises the following steps: Step S101, obtaining medical images of a patient; Step S102, automatically identifying multiple lesion areas through an image segmentation algorithm; Step S103, evaluating the accuracy of the image segmentation result by fusing multimodal information; Step S104, if it is analyzed that there is no corresponding pre-trained model in the lesion area and symptom database of the image, the image is stored; Step S105, if the results of the medical imaging can be analyzed according to the pre-trained model in the MongoDB database, a medical imaging report is generated through natural language processing (NLP) and text generation models. Step S101 can be connected to the hospital computer system through the image input module, so that medical images can be obtained quickly and safely. These reports can include lesion descriptions, clinical recommendations, diagnostic results, etc. to provide comprehensive medical evaluation information.

[0008] Furthermore, the image input module is electrically connected to the image acquisition module, the image acquisition module is electrically connected to the lesion area identification module, the lesion area identification module is electrically connected to the image confirmation module, the feature extractor is electrically connected to the lesion area identification module and the image confirmation module, the image confirmation module is connected to the MongoDB database data through a database interface, the image confirmation module is electrically connected to the natural language processing module and the text generation module, and the natural language processing module and the text generation module are electrically connected to the output module.

[0009] Furthermore, the lesion area identification module adopts an image segmentation algorithm, which is used to split and locate multiple lesion areas in the medical image, and is suitable for use in conjunction with the feature extractor. The feature extractor is used to extract the characteristics of the lesion position. The lesion area identification module identifies the lesion site on the medical image. The lesion area identification module adopts an image segmentation algorithm, which is used to split and locate multiple lesion areas in the medical image. After identification, the lesion position image is extracted by the feature extractor to facilitate the generation of a report, thereby making the analysis of lesions in multiple parts of the medical image more accurate.

[0010] Furthermore, the image confirmation module is used to determine the extracted feature information and compare it with the pre-trained model in the MongoDB database. The pre-trained model can be stored in the MongoDB database in advance. The image confirmation module can also be used to confirm the authenticity of the features to reduce the risk of misjudgment.

[0011] Furthermore, when a lesion is found, its information is input into the natural language processing model, and a medical imaging report is generated by the trained text generation model.

[0012] Furthermore, the step 105 further includes the following steps: Step S1051, identifying the lesion area in the medical image by a lesion area identification module; Step S1052, extracting the features of the main lesion location by a feature extractor; Step S1053, then the image confirmation module compares with the MongoDB database to find the corresponding pre-trained image; Step S1050: If no suitable pre-trained image is found, the pre-trained image classification model can be used to extract feature vectors of new medical images, ensuring that the image has been processed into a format suitable for model input, and extracting feature vectors from it and storing them in the MongoDB database; Step S1054, after confirmation, the result is explained in natural language through the natural language processing module and the text generation module; Step S1055, output the text summary and image description together. If no pre-trained model that can be used is found in the MongoDB database, the feature extractor can extract the features of the medical image being analyzed and then store them in the MongoDB database, so that the system has the ability to automatically learn and learn new medical image data, so as to facilitate a larger range of analysis of medical reports.

[0013] Furthermore, the output module is used to receive the medical image processing results sent by the natural language processing module and the text generation module, and output a medical image report according to the image processing results.

[0014] Furthermore, the feature extractor includes a neural network model to facilitate learning of the system.

[0015] Furthermore, the image confirmation module is also used to determine the number of lesion areas on the medical image.

[0016] Compared with the prior art, the present invention has the following beneficial effects: The present application sets up a lesion area module, and the lesion area recognition module identifies the lesion site on the medical image. The lesion area recognition module adopts an image segmentation algorithm to split and locate multiple lesion areas in the medical image. After identification, the lesion position image is extracted through a feature extractor to facilitate the generation of a report, thereby making the analysis of lesions in multiple parts of the medical image more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a schematic diagram of the use process of the present invention.

[0018] Figure 2 Schematic diagram of the generation system of the present invention.

[0019] Figure 3 It is a schematic diagram of the steps of the present invention.

[0020] Figure 4 It is a schematic diagram of the medical imaging report in the present invention. DETAILED DESCRIPTION

[0021] The technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the present invention. In addition, the forms of the various structures recorded in the following embodiments are merely illustrative. The semantic-based medical imaging report template generation system involved in the present invention is not limited to the various structures recorded in the following embodiments. All other implementations obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0022] See also Figure 1 and Figure 2 As shown, the present invention provides a semantic-based medical imaging report template generation system, including an image input module, the image input module is used to input the medical image of the patient; An image acquisition module, which is used to acquire medical image images; A lesion region recognition module, which is used to recognize one or more lesion regions in a medical image; Feature extractor, which is used to extract lesion feature information from medical images; An image confirmation module is used to determine the authenticity of the lesion area, that is, whether the lesion actually occurs; MongoDB database, which is used to store pre-trained models; A natural language generation module and a text generation module, the natural language generation module and the text generation module are used to convert the analyzed information into understandable text information; Output module: The output module is used to output information.

[0023] The image input module is convenient for connecting to the hospital's computer system to obtain the patient's medical images. The input medical images are obtained through the image acquisition module, and the lesion area recognition module recognizes multiple or one lesion features on the medical image. The feature extractor extracts the lesion site, and the features are used to judge the symptoms, degree and range of the lesion. The image confirmation module confirms whether the lesion actually occurs at the location, thereby improving the accuracy of medical image analysis. When confirming, the image confirmation module will call the MongoDB database pre-trained model, which is a medical image model of different modalities, to judge the medical image being analyzed, and then the natural language generation module and the text generation module generate a report, which is then output by the output module.

[0024] See also Figure 1 and Figure 2 As shown, the medical imaging report template generation system also includes the following steps: Step S101, obtaining medical images of a patient; Step S102, automatically identifying multiple lesion areas through an image segmentation algorithm; Step S103, evaluating the accuracy of the image segmentation result by fusing multimodal information; Step S104, if it is analyzed that there is no corresponding pre-trained model in the lesion area and symptom database of the image, the image is stored; Step S105: If the results of the medical imaging can be analyzed according to the pre-trained model in the MongoDB database, a medical imaging report is generated through natural language processing (NLP) and a text generation model.

[0025] First, the system will obtain the patient's medical images, which can be connected to the hospital's computer system through the image input module to facilitate real-time acquisition of medical imaging data; Then, the image is feature-recognized, and the lesion site on the medical image is identified through the lesion area identification module. The lesion area identification module adopts an image segmentation algorithm to split and locate multiple lesion areas in the medical image. After identification, the lesion location image is extracted through the feature extractor to facilitate the generation of the report, thereby making the analysis of lesions in multiple parts of the medical image more accurate; Then, by comparing with the pre-trained model in the MongoDB database, the lesion condition at the lesion location is analyzed. If there is no pre-trained model for the medical image in the MongoDB database, the features extracted by the feature extractor are imported into the MongoDB database for storage to become a new pre-trained model, so that the system has learning ability and can analyze medical images more accurately. After the analysis is completed, the medical imaging report is generated by natural language processing (NLP) and text generation models to generate simple and easy-to-understand reports, which can not only save doctors' time, but also make the reports simple and easy to understand, making it easier to analyze the patient's condition.

[0026] See also Figure 2 As shown, the image input module is electrically connected to the image acquisition module, the image acquisition module is electrically connected to the lesion area recognition module, the lesion area recognition module is electrically connected to the image confirmation module, the feature extractor is electrically connected to the lesion area recognition module and the image confirmation module, the image confirmation module is connected to the MongoDB database data through the database interface, the image confirmation module is electrically connected to the natural language processing module and the text generation module, and the natural language processing module and the text generation module are electrically connected to the output module.

[0027] See also Figure 1 , Figure 2 , Figure 3 and Figure 4 As shown, the lesion area recognition module adopts an image segmentation algorithm, which is used to split and locate multiple lesion areas in medical images and is suitable for use with a feature extractor. The feature extractor is used to extract the features of the lesion location. The image confirmation module is used to determine the extracted feature information and compare it with the pre-trained model in the MongoDB database. The pre-trained model can be stored in the MongoDB database in advance. When the lesion site is found, its information is input into the natural language processing model, and a medical image report is generated through the trained text generation model. The output module is used to receive the medical image processing results sent by the natural language processing module and the text generation module, and output the medical image report according to the image processing results. The feature extractor includes a neural network model. The image confirmation module is also used to determine the number of lesion areas on the medical image.

[0028] It should be noted that the feature extractor includes a neural network model, which facilitates the system to learn, thereby storing new features extracted by the feature extractor in the MongoDB database.

[0029] Among them, the MongoDB database is very flexible for storing image and text data. It can store them as documents, key-value pairs or binary files, which facilitates the analysis of medical images.

[0030] After extracting the pre-trained model from the MongoDB database and extracting the image feature vector, it is compared with the feature vectors of other medical images to find the lesion area. The image confirmation module also includes an image matching algorithm, which uses a comparison method to analyze the medical image, thereby analyzing the cause and scope of the lesion in the medical image.

[0031] See also Figure 1and Figure 4 As shown, step 105 also includes the following steps: Step S1051, identifying the lesion area in the medical image by a lesion area identification module; Step S1052, extracting the features of the main lesion location by a feature extractor; Step S1053, then the image confirmation module compares with the MongoDB database to find the corresponding pre-trained image; Step S1050: If no suitable pre-trained image is found, the pre-trained image classification model can be used to extract feature vectors of new medical images, ensuring that the image has been processed into a format suitable for model input, and extracting feature vectors from it and storing them in the MongoDB database; Step S1054, after confirmation, the result is explained in natural language through the natural language processing module and the text generation module; Step S1055: output the text summary and the image description together.

[0032] If no pre-trained model is found in the MongoDB database, the feature extractor can extract the features of the medical image being analyzed and store them in the MongoDB database, so that the system has the ability to learn automatically. The following is part of the code for storing feature data in the MongoDB database. #New medical image feature data obtained by the feature extractor (this data does not exist in the MongoDB database) new_image_id = "12345" image_features = [0.1, 0.2, 0.3, 0.4] # Example feature vector #Save feature data to MongoDB database data = { "_id": new_image_id, "features": image_features } collection.insert_one(data).

[0033] The process of the present invention when in use is as follows: the system will obtain the patient's medical image, which can be connected to the hospital's computer system through the image input module, so as to facilitate the real-time acquisition of medical image data; Then, the image is feature-recognized, and the lesion site on the medical image is identified through the lesion area identification module. The lesion area identification module adopts an image segmentation algorithm to split and locate multiple lesion areas in the medical image. After identification, the lesion location image is extracted through the feature extractor to facilitate the generation of the report, thereby making the analysis of lesions in multiple parts of the medical image more accurate; Then, by comparing with the pre-trained model in the MongoDB database, the lesion condition at the lesion location is analyzed. If there is no pre-trained model for the medical image in the MongoDB database, the features extracted by the feature extractor are imported into the MongoDB database for storage to become a new pre-trained model, so that the system has learning ability and can analyze medical images more accurately. After the analysis is completed, the medical imaging report is generated by natural language processing (NLP) and text generation models to generate simple and easy-to-understand reports, which can not only save doctors' time, but also make the reports simple and easy to understand, making it easier to analyze the patient's condition.

[0034] Any of the steps described above may be stored as computer instructions or programs in an unlimited computer memory, and may be called and recognized by an unlimited computer processor to implement any step in the embodiments of the present application, without any unnecessary restrictions.

[0035] The above is only a specific implementation of the present invention, and any other improvements made based on the concept of the present invention are considered to be within the protection scope of the present invention.

Claims

1. A semantic-based medical imaging report template generation system, characterized in that: include: An image input module, wherein the image input module is used to input medical images of patients; An image acquisition module, wherein the image acquisition module is used to acquire medical image images; A lesion region recognition module, the lesion region recognition module is used to recognize one or more lesion regions in a medical image; A feature extractor, wherein the feature extractor is used to extract lesion feature information in a medical image; An image confirmation module, which is used to determine the authenticity of the lesion area, that is, whether the lesion actually occurs; MongoDB database, wherein the MongoDB database is used to store the pre-trained model; A natural language generation module and a text generation module, wherein the natural language generation module and the text generation module are used to convert the analyzed information into understandable text information; An output module is used to output information.

2. The semantic-based medical imaging report template generation system according to claim 1, characterized in that: The medical imaging report template generation system further comprises the following steps: Step S101, obtaining medical images of a patient; Step S102, automatically identifying multiple lesion areas through an image segmentation algorithm; Step S103, evaluating the accuracy of the image segmentation result by fusing multimodal information; Step S104, if it is analyzed that there is no corresponding pre-trained model in the lesion area and symptom database of the image, the image is stored; Step S105: If the results of the medical imaging can be analyzed according to the pre-trained model in the MongoDB database, a medical imaging report is generated through natural language processing (NLP) and a text generation model.

3. The semantic-based medical imaging report template generation system according to claim 1, characterized in that: The image input module is electrically connected to the image acquisition module, the image acquisition module is electrically connected to the lesion area recognition module, the lesion area recognition module is electrically connected to the image confirmation module, the feature extractor is electrically connected to the lesion area recognition module and the image confirmation module, the image confirmation module is connected to the MongoDB database data through a database interface, the image confirmation module is electrically connected to the natural language processing module and the text generation module, and the natural language processing module and the text generation module are electrically connected to the output module.

4. The semantic-based medical imaging report template generation system according to claim 1, characterized in that: The lesion region identification module adopts an image segmentation algorithm, which is used to split and locate multiple lesion regions in a medical image and is suitable for use with the feature extractor, which is used to extract features of lesion locations.

5. The semantic-based medical imaging report template generation system according to claim 1, characterized in that: The image confirmation module is used to determine the extracted feature information and compare it with the pre-trained model in the MongoDB database. The pre-trained model can be stored in the MongoDB database in advance.

6. The semantic-based medical imaging report template generation system according to claim 1, characterized in that: When a lesion is found, its information is input into the natural language processing model, and a medical imaging report is generated through the trained text generation model.

7. The semantic-based medical imaging report template generation system according to claim 2, characterized in that: The step 105 further comprises the following steps: Step S1051, identifying the lesion area in the medical image by a lesion area identification module; Step S1052, extracting the features of the main lesion location by a feature extractor; Step S1053, then the image confirmation module compares with the MongoDB database to find the corresponding pre-trained image; Step S1050: If no suitable pre-trained image is found, the pre-trained image classification model can be used to extract feature vectors of new medical images, ensuring that the image has been processed into a format suitable for model input, and extracting feature vectors from it and storing them in the MongoDB database; Step S1054, after confirmation, the result is explained in natural language through the natural language processing module and the text generation module; Step S1055: output the text summary and the image description together.

8. The semantic-based medical imaging report template generation system according to claim 1, characterized in that: The output module is used to receive the medical image processing results sent by the natural language processing module and the text generation module, and output a medical image report according to the image processing results.

9. The semantic-based medical imaging report template generation system according to claim 1, characterized in that: The feature extractor includes a neural network model.

10. The semantic-based medical imaging report template generation system according to claim 1, characterized in that: The image confirmation module is also used to determine the number of lesion areas on the medical image.

Images