Medical image processing system and method for report generation

Abstract

The present invention discloses a medical image processing system and method for report generation, which obtains a medical image to be analyzed; extracts the shallow features, middle features and deep features of the medical image to be analyzed to obtain a medical image shallow feature map, a medical image middle feature map and a medical image deep feature map; fuses and enhances the medical image shallow feature map, the medical image middle feature map and the medical image deep feature map to obtain an optimized autocorrelation enhanced medical image feature matrix; and generates a report text based on the optimized autocorrelation enhanced medical image feature matrix. In this way, it can provide valuable reference for doctors' diagnosis and decision-making, reduce the time doctors spend on report writing, and enable them to focus more on clinical work and patient care.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent medical image processing, and in particular to a medical image processing system and method for report generation. Background Art

[0002] Medical imaging is the process of obtaining images of the internal structure and function of the human body using various imaging techniques such as X-rays, computed tomography, magnetic resonance imaging, ultrasound, and nuclear medicine. These images can provide doctors and medical professionals with a detailed understanding of disease, injury, and abnormalities.

[0003] Medical images play an important role in clinical medicine. They can help doctors diagnose diseases at an early stage, assess the severity and progression of diseases, develop treatment plans, guide surgical operations, and monitor the effectiveness of treatment.

[0004] Generating reports based on medical images has several important purposes. However, there are still some challenges when generating reports. Traditional medical image analysis methods usually require a lot of time for manual writing.

[0005] Therefore, an optimized medical image processing method for report generation is desired. Summary of the invention

[0006] The embodiment of the present invention provides a medical image processing system and method for report generation, which obtains a medical image to be analyzed; extracts the shallow features, middle features and deep features of the medical image to be analyzed to obtain a medical image shallow feature map, a medical image middle feature map and a medical image deep feature map; fuses and enhances the medical image shallow feature map, the medical image middle feature map and the medical image deep feature map to obtain an optimized autocorrelation enhanced medical image feature matrix; and generates a report text based on the optimized autocorrelation enhanced medical image feature matrix. In this way, it can provide valuable reference for doctors' diagnosis and decision-making, reduce the time doctors spend on report writing, and enable them to focus more on clinical work and patient care.

[0007] The embodiment of the present invention further provides a medical image processing method for report generation, which includes:

[0008] Obtain medical images to be analyzed;

[0009] Extracting shallow features, middle features and deep features of the medical image to be analyzed to obtain a medical image shallow feature map, a medical image middle feature map and a medical image deep feature map;

[0010] fusing and enhancing the shallow feature map of the medical image, the middle feature map of the medical image, and the deep feature map of the medical image to obtain an optimized autocorrelation enhanced medical image feature matrix; and

[0011] Generate a report text based on the optimized autocorrelation enhanced medical image feature matrix.

[0012] An embodiment of the present invention further provides a medical image processing system for report generation, which includes:

[0013] A medical image acquisition module, used to acquire medical images to be analyzed;

[0014] A feature map extraction module, used to extract shallow features, middle features and deep features of the medical image to be analyzed to obtain a medical image shallow feature map, a medical image middle feature map and a medical image deep feature map;

[0015] A fusion and enhancement module, used for fusing and enhancing the shallow feature map of the medical image, the middle feature map of the medical image, and the deep feature map of the medical image to obtain an optimized autocorrelation enhanced medical image feature matrix; and

[0016] A report text generation module is used to generate a report text based on the optimized autocorrelation enhanced medical image feature matrix. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0018] Figure 1 The present invention provides a flowchart of a medical image processing method for report generation in an embodiment of the present invention.

[0019] Figure 2 A schematic diagram of a system architecture of a medical image processing method for report generation provided in an embodiment of the present invention.

[0020] Figure 3 A block diagram of a medical image processing system for report generation provided in an embodiment of the present invention.

[0021] Figure 4 This is an application scenario diagram of a medical image processing method for report generation provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0022] To make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0023] Unless otherwise specified, all technical and scientific terms used in the embodiments of the present application have the same meaning as those commonly understood by those skilled in the art of the present application. The terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit the scope of this application.

[0024] In the embodiments of the present application, it should be noted that, unless otherwise specified and limited, the term "connection" should be understood in a broad sense. For example, it can be an electrical connection or a connection between two components. It can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to the specific circumstances.

[0025] It should be noted that the terms "first\second\third" involved in the embodiments of the present application are only used to distinguish similar objects, and do not represent a specific order for the objects. It is understandable that the specific order or sequence of "first\second\third" can be interchanged where permitted. It should be understood that the objects distinguished by "first\second\third" can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein.

[0026] Medical imaging is an important medical diagnostic tool that uses different imaging techniques to obtain images of the internal structure and function of the human body. These images can help doctors and medical professionals better understand diseases, injuries and abnormalities, thereby making accurate diagnoses and treatment plans.

[0027] X-rays are one of the most common medical imaging techniques. They use X-rays to pass through the human body and generate images by detecting the degree of absorption of the rays. X-rays can be used to examine bones, lungs, and certain soft tissue structures. CT scans use X-rays and computer technology to generate detailed cross-sectional images, which can provide more detailed images than traditional X-rays and help detect tumors, vascular diseases, fractures, etc. Magnetic resonance imaging uses strong magnetic fields and harmless radio waves to generate high-resolution images, which can provide detailed anatomical structure and tissue information. It has high accuracy for examining the brain, spine, joints, internal organs, etc. Ultrasound uses the echoes of high-frequency sound waves to generate images. It is a non-invasive imaging technology that can be used to examine the fetus, abdomen, heart, blood vessels, etc. Ultrasound is very useful for observing the dynamic changes of organs and fluids in real time. Nuclear medicine uses radioactive tracers to observe the function of internal organs and tissues in the human body. For example, single photon emission computed tomography (SPECT) and positron emission computed tomography (PET) can be used to detect cancer, cardiovascular disease, and neurological diseases.

[0028] Medical imaging plays an important role in clinical medicine, providing doctors with very valuable information for early disease diagnosis, assessing the severity and progression of the disease, formulating treatment plans, guiding surgical operations, and monitoring the effectiveness of treatment. Medical imaging technology can help doctors detect abnormalities in the early stages of disease development. For example, lung nodules can be detected through X-rays or CT scans, and MRI can help detect brain tumors early. Early diagnosis helps to take treatment measures early and improve the success rate of treatment. Medical imaging can help doctors assess the severity and progression of the disease. For example, MRI can provide detailed images of joint structures to help assess the extent of arthritis. By performing regular imaging examinations, doctors can monitor changes in the disease and adjust treatment plans.

[0029] Medical imaging can provide an important basis for doctors to formulate treatment plans. For example, in cancer treatment, CT, MRI and PET scans can help determine the location, size and spread of tumors, thereby determining treatment methods and plans such as surgery, radiotherapy, and chemotherapy. Medical imaging is increasingly used in surgery. For example, using CT or MRI images for three-dimensional reconstruction can help doctors plan the surgical path and determine the scope of surgery before surgery. During surgery, doctors can use real-time X-ray or ultrasound images to guide surgical operations and improve the accuracy and safety of surgery. Medical imaging can be used to monitor the effectiveness of treatment. For example, by comparing images before and after treatment, the degree of tumor reduction can be evaluated and whether the treatment is effective can be determined, which helps doctors adjust treatment plans in a timely manner and provide better treatment results.

[0030] In one embodiment of the present invention, Figure 1The present invention provides a flowchart of a medical image processing method for report generation in an embodiment of the present invention. Figure 2 FIG. 1 is a schematic diagram of a system architecture of a medical image processing method for report generation provided in an embodiment of the present invention. Figure 1 and Figure 2 As shown, the medical image processing method for report generation according to an embodiment of the present invention includes: 110, obtaining a medical image to be analyzed; 120, extracting shallow features, middle features and deep features of the medical image to be analyzed to obtain a medical image shallow feature map, a medical image middle feature map and a medical image deep feature map; 130, fusing and enhancing the medical image shallow feature map, the medical image middle feature map and the medical image deep feature map to obtain an optimized autocorrelation enhanced medical image feature matrix; and, 140, generating a report text based on the optimized autocorrelation enhanced medical image feature matrix.

[0031] The step 110 obtains the medical images to be analyzed. These images may be images generated by various medical imaging devices (such as X-ray, CT, MRI, etc.). By obtaining the medical images to be analyzed, necessary data can be provided for subsequent analysis and diagnosis.

[0032] The step 120 extracts shallow features, mid-level features, and deep features from the medical image to be analyzed by using deep learning and computer vision technology. The features may be representations of low-level visual features (such as edges, textures, etc.) to high-level semantic features (such as organs, lesions, etc.) of the image. Extracting features at different levels helps to capture different levels of information in the medical image and provide more comprehensive data for subsequent analysis and diagnosis.

[0033] In step 130, the extracted medical image feature maps are fused and enhanced to obtain an optimized autocorrelation enhanced medical image feature matrix. Fusion and enhancement can be achieved through different methods and techniques, such as feature fusion, feature weighting, feature enhancement, etc. The purpose of this step is to comprehensively utilize feature information at different levels to improve the expressiveness and discrimination of features, thereby obtaining a more representative and discriminative feature matrix.

[0034] In step 140, based on the optimized autocorrelation enhanced medical image feature matrix, natural language processing and machine learning techniques can be applied to generate corresponding report text. The report text can include descriptions of medical images, diagnostic results, lesion assessments, treatment recommendations, etc. By automatically generating report text, work efficiency can be improved, the workload of doctors can be reduced, and the consistency and accuracy of reports can be ensured.

[0035] By acquiring the medical images to be analyzed, extracting features, fusing and enhancing features, and generating report text based on the optimized features, it is possible to achieve automated analysis and diagnosis of medical images. These steps can provide necessary data, capture information at different levels, improve the expressiveness and differentiation of features, and improve work efficiency and report accuracy, which helps improve the results of medical image analysis and diagnosis, and provide doctors with more accurate and faster diagnosis results.

[0036] Generating reports based on medical images has several important purposes. Specifically, first, reports can record the results and observations of medical images so that doctors and other medical professionals can review and understand them. Second, reports can provide descriptions and analysis of the condition, including information such as the location, size, morphology, and density of the lesion, to help doctors make accurate diagnoses and develop treatment plans. In addition, reports can also serve as a delivery and communication tool for medical imaging results, enabling doctors, patients, and other medical team members to share and discuss the condition.

[0037] However, there are still some challenges in generating reports. Traditional medical image analysis methods usually require a lot of time for manual writing. Therefore, an optimized medical image processing method for report generation is expected.

[0038] In response to the above technical problems, the technical concept of this application is to extract and analyze the lesion region of interest of the medical image to be analyzed, and generate a report text based on this, providing valuable reference for doctors' diagnosis and decision-making. At the same time, it reduces the time doctors spend on report writing, allowing them to focus more on clinical work and patient care.

[0039] Based on this, in the technical solution of the present application, the medical image to be analyzed is first obtained, wherein the medical image to be analyzed can come from various imaging technologies, such as X-ray, computed tomography (CT), magnetic resonance imaging (MRI), ultrasound and nuclear medicine.

[0040] Then, the medical image to be analyzed is subjected to noise reduction processing to obtain a noise-reduced medical image to be analyzed. Here, considering that medical images are usually affected by noise, such as electronic noise in imaging equipment, scattering noise in gamma rays, etc., these noises will reduce the quality and clarity of the image, making details difficult to discern, and may even cover up or distort important features in the image, such as lesions. Noise reduction processing can reduce the impact of noise and make important features in the image more obvious and recognizable.

[0041] Specifically, filters based on statistical principles can be used to perform statistical analysis on pixels in an image to reduce noise, such as mean filters and median filters. The mean filter smoothes the image by calculating the average value of neighborhood pixels, while the median filter removes noise by calculating the median of neighborhood pixels. The partial differential equation (PDE) method can also be used for denoising. Among them, the partial differential equation (PDE) method reduces noise by describing the grayscale changes and texture information in the image. For example, Total Variation Regularization is a commonly used PDE-based denoising method that smoothes the image by minimizing the total variation of the image.

[0042] In a specific embodiment of the present application, shallow features, middle features and deep features of the medical image to be analyzed are extracted to obtain a shallow feature map of the medical image, a middle feature map of the medical image and a deep feature map of the medical image, including: performing denoising on the medical image to be analyzed to obtain a denoised medical image to be analyzed; and passing the denoised medical image to be analyzed through a medical image feature extractor based on a pyramid network to obtain the shallow feature map of the medical image, the middle feature map of the medical image and the deep feature map of the medical image.

[0043] Next, the denoised medical image to be analyzed is passed through a pyramid network-based medical image feature extractor to obtain a medical image shallow feature map, a medical image middle feature map, and a medical image deep feature map; and a feature map cascade fusion module is used to fuse the medical image shallow feature map, the medical image middle feature map, and the medical image deep feature map to obtain a medical image multi-scale feature map. It should be understood that medical images usually contain rich hierarchical structures and diverse features. Shallow features mainly include low-level image features, such as edges, textures, etc.; middle-level features have higher-level semantic information, such as shape, structure, etc.; and deep features can capture more abstract and advanced semantic feature representations. By extracting and fusing multiple layers of features, the model can have a more comprehensive and diverse feature expression capability.

[0044] In a specific embodiment of the present application, the denoised medical image to be analyzed is passed through a medical image feature extractor based on a pyramid network to obtain the shallow feature map of the medical image, the middle feature map of the medical image and the deep feature map of the medical image, including: passing the denoised medical image to be analyzed through a first convolution module of the medical image feature extractor based on a pyramid network to obtain the shallow feature map of the medical image; passing the denoised medical image to be analyzed through a second convolution module of the medical image feature extractor based on a pyramid network to obtain the middle feature map of the medical image; and passing the denoised medical image to be analyzed through a third convolution module of the medical image feature extractor based on a pyramid network to obtain the deep feature map of the medical image.

[0045] In a specific embodiment of the present application, the shallow feature map of the medical image, the middle feature map of the medical image and the deep feature map of the medical image are fused and enhanced to obtain an optimized autocorrelation enhanced medical image feature matrix, including: using a feature map cascade fusion module to fuse the shallow feature map of the medical image, the middle feature map of the medical image and the deep feature map of the medical image to obtain a multi-scale feature map of the medical image; performing feature autocorrelation association enhancement on the multi-scale feature map of the medical image to obtain an autocorrelation enhanced multi-scale feature map of the medical image; performing dimensionality reduction processing on the autocorrelation enhanced multi-scale feature map of the medical image to obtain an autocorrelation enhanced medical image feature matrix; and performing feature distribution optimization on the autocorrelation enhanced medical image feature matrix to obtain the optimized autocorrelation enhanced medical image feature matrix.

[0046] Furthermore, a feature map cascade fusion module is used to fuse the shallow feature map of the medical image, the middle feature map of the medical image and the deep feature map of the medical image to obtain a multi-scale feature map of the medical image, including: performing an upsampling and convolution module based on a first upsampling factor on the deep feature map of the medical image to obtain a first upsampling deep convolutional coding feature map; performing an upsampling and convolution module based on a second upsampling factor on the middle feature map of the medical image to obtain a first upsampling middle convolutional coding feature map; performing an element-by-element multiplication operation on the first upsampling deep convolutional coding feature map, the first upsampling middle convolutional coding feature map and the shallow feature map of the medical image to obtain a shallow-middle-deep interactive feature map; performing an upsampling and convolution module based on a second upsampling factor on the deep feature map of the medical image to obtain a shallow-middle-deep interactive feature map. The invention relates to an upsampling and convolution module of the sampling factor to obtain a second upsampled deep convolution coding feature map; the medical image middle-layer feature map and the second upsampled deep convolution coding feature map are element-by-element multiplied to obtain a medium-deep interaction feature map; the second upsampled deep convolution coding feature map and the medium-deep interaction feature map are spliced ​​to obtain a medium-deep splicing interaction feature map; the medium-deep splicing interaction feature map is subjected to convolution and upsampling to obtain an adjusted medium-deep splicing interaction feature map, and the adjusted medium-deep splicing interaction feature map has the same size as the shallow-medium-deep interaction feature map; and the shallow-medium-deep interaction feature map and the adjusted medium-deep splicing interaction feature map are spliced ​​to obtain the medical image multi-scale feature map.

[0047] It should be understood that although the medical image multi-scale feature map is obtained by the deep feature interaction of the medical image shallow feature map, the medical image middle-level feature map and the medical image deep feature map, it expresses the multi-scale features of the medical image about the lesion information. However, the image semantic information expressed at different spatial positions in the medical image is different from the degree of lesion feature distribution information. In other words, the eigenvalues ​​at various positions in the feature map have different importance for forming complete target feature information. Therefore, in the technical solution of the present application, the medical image multi-scale feature map is subjected to feature autocorrelation association enhancement to obtain an autocorrelation enhanced medical image multi-scale feature map. Wherein, the medical image multi-scale feature map is subjected to feature autocorrelation association enhancement to obtain an autocorrelation enhanced medical image multi-scale feature map by processing the medical image multi-scale feature map through a feature autocorrelation association enhancement module to obtain the autocorrelation enhanced medical image multi-scale feature map.

[0048] The autocorrelation enhancement module constructs a relationship matrix by constructing the similarity between each feature value in the multi-scale feature map of the medical image, and uses the relationship matrix to aggregate complete information about the characteristics of the target lesion. In this way, the model is helped to better understand the relationship between different areas in the image and highlight the feature distribution of important areas.

[0049] In a specific embodiment of the present application, the multi-scale feature map of the medical image is subjected to feature autocorrelation enhancement to obtain an autocorrelation enhanced multi-scale feature map of the medical image, including: passing the multi-scale feature map of the medical image through the first convolution layer of the feature autocorrelation enhancement module to obtain a first feature map; passing the first feature map through the second convolution layer of the feature autocorrelation enhancement module to obtain a second feature map; expanding each feature matrix of the second feature map along the channel dimension into feature vectors to obtain a sequence of first feature vectors; calculating the cosine similarity between any two first feature vectors in the sequence of the first feature vectors to obtain a cosine similarity feature map; passing the cosine similarity feature map through the Softmmax function The method comprises the steps of: normalizing the first deconvolution layer of the feature autocorrelation enhancement module and performing a normalization process on the first deconvolution layer of the feature autocorrelation enhancement module to obtain a first fusion feature map; normalizing the first deconvolution feature map to obtain a normalized cosine similarity feature map; multiplying the normalized cosine similarity feature map and the cosine similarity feature map by position points to obtain a similarity mapping optimization feature map; passing the similarity mapping optimization feature map through the first deconvolution layer of the feature autocorrelation association enhancement module to obtain a first deconvolution feature map; calculating the element-by-element sum of the first deconvolution feature map and the first feature map to obtain a first fusion feature map; passing the first fusion feature map through the second deconvolution layer of the feature autocorrelation association enhancement module to obtain a second deconvolution feature map; calculating the element-by-element sum of the second deconvolution feature map and the medical image multi-scale feature map to obtain the autocorrelation enhanced medical image multi-scale feature map.

[0050] Subsequently, global mean pooling is performed on the multi-scale feature map of the autocorrelation enhanced medical image along the channel dimension to obtain an autocorrelation enhanced medical image feature matrix.

[0051] In a specific embodiment of the present application, the autocorrelation enhanced medical image multi-scale feature map is subjected to dimensionality reduction processing to obtain an autocorrelation enhanced medical image feature matrix, including: performing global mean pooling along the channel dimension on the autocorrelation enhanced medical image multi-scale feature map to obtain the autocorrelation enhanced medical image feature matrix.

[0052] In the technical solution of the present application, the shallow feature map of the medical image, the middle feature map of the medical image and the deep feature map of the medical image respectively express the image semantic features of different scales and different depths of the medical image to be analyzed after denoising based on the pyramid network. In this way, when using the feature map cascade fusion module to fuse the shallow feature map of the medical image, the middle feature map of the medical image and the deep feature map of the medical image, and performing feature autocorrelation association enhancement on the obtained multi-scale feature map of the medical image, the correlation of the channel dimension can be constrained based on the cross-scale and cross-depth image semantic feature representation of each feature matrix, so that the channel distribution of the autocorrelation enhanced medical image multi-scale feature map also follows the cross-scale and cross-depth image semantic feature distribution. Therefore, after the global mean pooling of the autocorrelation enhanced medical image multi-scale feature map is performed along the channel dimension, the obtained autocorrelation enhanced medical image feature matrix will have a cross-scale and cross-depth image semantic feature distribution.

[0053] Therefore, considering the differences between the image semantic feature distributions across scale and depth dimensions of the autocorrelation enhanced medical image feature matrix, the distribution of image semantic features related to the dimensional subset of the autocorrelation enhanced medical image feature matrix will become sparse, resulting in poor convergence of the probability density distribution of the regression probability of each eigenvalue of the autocorrelation enhanced medical image feature matrix when the autocorrelation enhanced medical image feature matrix is ​​subjected to pixel-by-pixel class probability regression mapping through the Softmax function, thereby affecting the accuracy of the pixel-by-pixel classification results obtained by the Softmax function.

[0054] Therefore, preferably, each eigenvalue of the autocorrelation enhanced medical image feature matrix is ​​optimized, which is specifically expressed as: each eigenvalue of the autocorrelation enhanced medical image feature matrix is ​​optimized by the following optimization formula to obtain the optimized autocorrelation enhanced medical image feature matrix; wherein the optimization formula is: ;in, and is the autocorrelation enhanced medical imaging feature matrix No. and eigenvalues, and is the autocorrelation enhanced medical imaging feature matrix The global feature mean of is the first eigenvalues, Represents calculation of the natural exponential function raised to a value.

[0055] Specifically, for the autocorrelation enhanced medical image feature matrix The local probability density mismatch of the probability density distribution in the probability space caused by the sparse distribution in the high-dimensional feature space is simulated by regularizing the global self-consistent class coding to enhance the medical image feature matrix through autocorrelation. The global self-consistent relationship between the encoding behavior of the high-dimensional features in the probability space is used to adjust the error landscape of the feature manifold in the high-dimensional open space domain to achieve the autocorrelation enhanced medical image feature matrix The high-dimensional features of the self-consistent matching class encoding of the explicit probability space embedding, thereby improving the autocorrelation enhanced medical image feature matrix The convergence of the probability density distribution of the regression probability is improved, and the accuracy of the pixel-by-pixel classification results obtained by the Softmax function is improved.

[0056] Then, the optimized autocorrelation enhanced medical image feature matrix is ​​classified pixel by pixel using the Softmax function to obtain the lesion region of interest extraction result. Among them, the Softmax function outputs a probability value, which indicates the confidence that each pixel position belongs to a different category. This probability output can provide more fine-grained information, and can be more than just a simple binary classification result. By setting an appropriate threshold, pixels can be divided into regions of interest and non-regions of interest according to the probability value, thereby obtaining a more accurate lesion extraction result. Here, the lesion region of interest extraction result can be regarded as a lesion region of interest image.

[0057] Then, a geometric analysis is performed on the lesion region of interest extraction result to obtain an analysis result, wherein the analysis result includes lesion size, lesion morphology and lesion density; and a report text is generated based on the lesion region of interest extraction result and the analysis result.

[0058] Among them, geometric morphology analysis is an analytical method used to study the shape and structure of an object. It involves quantitative analysis of various measures and features of geometric shapes to obtain a description and understanding of the shape. In other words, geometric morphology analysis can quantify and describe the shape of an object from different perspectives.

[0059] Specifically, in an embodiment of the present application, based on the lesion region of interest extraction result and the analysis result, the process of generating a report text includes: first, the lesion region of interest result extracted from the medical image is matched with the original image, and the region of interest can be visualized on the original image by color mapping or superposition, so as to intuitively show the position and shape of the lesion. Then, according to the extracted lesion region of interest and the analysis result, the corresponding report text is generated using natural language processing technology. The template for generating the report can be designed according to actual needs, including the structure of the report, the organization of paragraphs and sentences. Then, the analysis results are inserted into the report text in a structured manner, for example, the characteristic parameters of the lesion are presented in the form of a table or list.

[0060] In a specific embodiment of the present application, a report text is generated based on the optimized autocorrelation enhanced medical image feature matrix, including: using a Softmax function to perform pixel-by-pixel classification on the optimized autocorrelation enhanced medical image feature matrix to obtain a lesion region of interest extraction result; performing geometric morphological analysis on the lesion region of interest extraction result to obtain an analysis result, the analysis result including lesion size, lesion morphology and lesion density; and, based on the lesion region of interest extraction result and the analysis result, a report text is generated.

[0061] In summary, the medical image processing method for report generation based on the embodiment of the present invention is explained, which extracts and analyzes the lesion region of interest of the medical image to be analyzed, and generates a report text based on this, providing a valuable reference for the doctor's diagnosis and decision-making. At the same time, it reduces the time doctors spend on report writing, allowing them to focus more on clinical work and patient care.

[0062] Figure 3 FIG. 1 is a block diagram of a medical image processing system for report generation provided in an embodiment of the present invention. Figure 3 As shown, the medical image processing system 200 for report generation includes: a medical image acquisition module 210, used to acquire the medical image to be analyzed; a feature map extraction module 220, used to extract the shallow features, middle features and deep features of the medical image to be analyzed to obtain a medical image shallow feature map, a medical image middle feature map and a medical image deep feature map; a fusion and enhancement module 230, used to fuse and enhance the medical image shallow feature map, the medical image middle feature map and the medical image deep feature map to obtain an optimized autocorrelation enhanced medical image feature matrix; and a report text generation module 240, used to generate a report text based on the optimized autocorrelation enhanced medical image feature matrix.

[0063] Those skilled in the art will appreciate that the specific operations of each step in the medical image processing system for generating a report have been described above with reference to Figure 1 to Figure 2 The description of the medical image processing method for report generation has been described in detail, and therefore, its repeated description will be omitted.

[0064] As described above, the medical image processing system 200 for report generation according to an embodiment of the present invention can be implemented in various terminal devices, such as a server for medical image processing for report generation, etc. In one example, the medical image processing system 200 for report generation according to an embodiment of the present invention can be integrated into a terminal device as a software module and / or a hardware module. For example, the medical image processing system 200 for report generation can be a software module in the operating system of the terminal device, or can be an application developed for the terminal device; of course, the medical image processing system 200 for report generation can also be one of the many hardware modules of the terminal device.

[0065] Alternatively, in another example, the medical image processing system 200 for report generation and the terminal device may also be separate devices, and the medical image processing system 200 for report generation may be connected to the terminal device via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0066] Figure 4 FIG. 1 is an application scenario diagram of a medical image processing method for report generation provided in an embodiment of the present invention. Figure 4 As shown, in this application scenario, first, obtain the medical image to be analyzed (for example, Figure 4 Then, the acquired medical image to be analyzed is input to a server (for example, Figure 4 In S) as shown in , the server is capable of processing the medical image to be analyzed based on a medical image processing algorithm for report generation to generate a report text.

[0067] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A medical image processing method for report generation, characterized in that: include: Obtain medical images to be analyzed; Extracting shallow features, middle features and deep features of the medical image to be analyzed to obtain a medical image shallow feature map, a medical image middle feature map and a medical image deep feature map; The shallow feature map of the medical image, the middle feature map of the medical image and the deep feature map of the medical image are fused and enhanced to obtain an optimized autocorrelation enhanced medical image feature matrix; and Generate a report text based on the optimized autocorrelation enhanced medical image feature matrix; The shallow feature map of the medical image, the middle feature map of the medical image and the deep feature map of the medical image are fused and enhanced to obtain an optimized autocorrelation enhanced medical image feature matrix, including: Using a feature map cascade fusion module to fuse the shallow feature map of the medical image, the middle feature map of the medical image, and the deep feature map of the medical image to obtain a multi-scale feature map of the medical image; Performing feature autocorrelation enhancement on the medical image multi-scale feature map to obtain an autocorrelation enhanced medical image multi-scale feature map; Performing dimensionality reduction processing on the autocorrelation enhanced medical image multi-scale feature map to obtain an autocorrelation enhanced medical image feature matrix; and Performing feature distribution optimization on the autocorrelation enhanced medical image feature matrix to obtain the optimized autocorrelation enhanced medical image feature matrix; The step of performing feature autocorrelation enhancement on the multi-scale feature map of the medical image to obtain the autocorrelation enhanced multi-scale feature map of the medical image includes: Passing the first convolutional layer of the feature autocorrelation enhancement module on the multi-scale feature map of the medical image to obtain a first feature map; Passing the first feature map through the second convolutional layer of the feature autocorrelation association enhancement module to obtain a second feature map; Expanding each feature matrix of the second feature map along the channel dimension into feature vectors to obtain a sequence of first feature vectors; Calculating the cosine similarity between any two first feature vectors in the sequence of the first feature vectors to obtain a cosine similarity feature graph; The cosine similarity feature map is normalized by a Softmax function to obtain a normalized cosine similarity feature map; Multiplying the normalized cosine similarity feature map and the cosine similarity feature map by position points to obtain a similarity mapping optimization feature map; Passing the similarity map optimization feature map through the first deconvolution layer of the feature autocorrelation association enhancement module to obtain a first deconvolution feature map; Calculating an element-by-element sum of the first deconvolution feature map and the first feature map to obtain a first fused feature map; Passing the first fused feature map through the second deconvolution layer of the feature autocorrelation association enhancement module to obtain a second deconvolution feature map; Calculating the element-by-element sum of the second deconvolution feature map and the medical image multi-scale feature map to obtain the autocorrelation enhanced medical image multi-scale feature map; The optimization of each eigenvalue of the autocorrelation enhanced medical image feature matrix is ​​specifically expressed as follows: the optimization of each eigenvalue of the autocorrelation enhanced medical image feature matrix is ​​performed using the following optimization formula to obtain the optimized autocorrelation enhanced medical image feature matrix; wherein the optimization formula is: ;in, and is the autocorrelation enhanced medical imaging feature matrix No. and eigenvalues, and is the autocorrelation enhanced medical imaging feature matrix The global feature mean of is the first eigenvalues, Represents calculation of the natural exponential function value raised to the power of value.

2. The medical image processing method for report generation according to claim 1, characterized in that: Extracting shallow features, middle features and deep features of the medical image to be analyzed to obtain a medical image shallow feature map, a medical image middle feature map and a medical image deep feature map, including: Performing noise reduction processing on the medical image to be analyzed to obtain a noise-reduced medical image to be analyzed; and The denoised medical image to be analyzed is passed through a medical image feature extractor based on a pyramid network to obtain a shallow feature map of the medical image, a middle feature map of the medical image, and a deep feature map of the medical image.

3. The medical image processing method for report generation according to claim 2, characterized in that: The denoised medical image to be analyzed is passed through a medical image feature extractor based on a pyramid network to obtain a shallow feature map of the medical image, a middle feature map of the medical image, and a deep feature map of the medical image, including: Passing the denoised medical image to be analyzed through the first convolution module of the pyramid network-based medical image feature extractor to obtain the shallow feature map of the medical image; Passing the denoised medical image to be analyzed through the second convolution module of the pyramid network-based medical image feature extractor to obtain the medical image mid-level feature map; and The denoised medical image to be analyzed is passed through the third convolution module of the pyramid network-based medical image feature extractor to obtain the deep feature map of the medical image.

4. The medical image processing method for report generation according to claim 3, characterized in that: Using a feature map cascade fusion module to fuse the shallow feature map of the medical image, the middle feature map of the medical image, and the deep feature map of the medical image to obtain a multi-scale feature map of the medical image, including: The medical image deep feature map is subjected to an upsampling and convolution module based on a first upsampling factor to obtain a first upsampled deep convolutional coding feature map; The medical image middle-layer feature map is subjected to an upsampling and convolution module based on a second upsampling factor to obtain a first up-sampled middle-layer convolutional coding feature map; Performing an element-by-element multiplication operation on the first upsampled deep convolutional coding feature map, the first upsampled middle convolutional coding feature map, and the medical image shallow feature map to obtain a shallow-middle-deep interactive feature map; The medical image deep feature map is subjected to an upsampling and convolution module based on a second upsampling factor to obtain a second upsampled deep convolutional coding feature map; Performing an element-by-element multiplication operation on the medical image middle-layer feature map and the second up-sampled deep convolutional coding feature map to obtain a middle-deep interactive feature map; Splicing the second up-sampled deep convolutional coding feature map and the mid-depth interaction feature map to obtain a mid-depth spliced ​​interaction feature map; Performing convolution and upsampling on the middle-deep splicing interaction feature map to obtain an adjusted middle-deep splicing interaction feature map, wherein the adjusted middle-deep splicing interaction feature map has the same size as the shallow-middle-deep interaction feature map; and The shallow-medium-depth interaction feature map and the adjusted medium-depth splicing interaction feature map are spliced ​​to obtain the medical image multi-scale feature map.

5. The medical image processing method for report generation according to claim 4, characterized in that: Performing dimensionality reduction processing on the autocorrelation enhanced medical image multi-scale feature map to obtain an autocorrelation enhanced medical image feature matrix, including: The autocorrelation enhanced medical image multi-scale feature map is globally mean pooled along the channel dimension to obtain the autocorrelation enhanced medical image feature matrix.

6. The medical image processing method for report generation according to claim 5, characterized in that: Based on the optimized autocorrelation enhanced medical image feature matrix, a report text is generated, including: Using a Softmax function to classify the optimized autocorrelation enhanced medical image feature matrix pixel by pixel to obtain a lesion region of interest extraction result; Performing geometric analysis on the lesion region of interest extraction result to obtain an analysis result, wherein the analysis result includes lesion size, lesion morphology and lesion density; and A report text is generated based on the lesion region of interest extraction result and the analysis result.

7. A medical image processing system for report generation, characterized in that: include: A medical image acquisition module, used to acquire medical images to be analyzed; A feature map extraction module, used to extract shallow features, middle features and deep features of the medical image to be analyzed to obtain a medical image shallow feature map, a medical image middle feature map and a medical image deep feature map; A fusion and enhancement module, used for fusing and enhancing the shallow feature map of the medical image, the middle feature map of the medical image and the deep feature map of the medical image to obtain an optimized autocorrelation enhanced medical image feature matrix; and A report text generation module, used for generating a report text based on the optimized autocorrelation enhanced medical image feature matrix; The shallow feature map of the medical image, the middle feature map of the medical image and the deep feature map of the medical image are fused and enhanced to obtain an optimized autocorrelation enhanced medical image feature matrix, including: Using a feature map cascade fusion module to fuse the shallow feature map of the medical image, the middle feature map of the medical image, and the deep feature map of the medical image to obtain a multi-scale feature map of the medical image; Performing feature autocorrelation enhancement on the medical image multi-scale feature map to obtain an autocorrelation enhanced medical image multi-scale feature map; Performing dimensionality reduction processing on the autocorrelation enhanced medical image multi-scale feature map to obtain an autocorrelation enhanced medical image feature matrix; and Performing feature distribution optimization on the autocorrelation enhanced medical image feature matrix to obtain the optimized autocorrelation enhanced medical image feature matrix; The step of performing feature autocorrelation enhancement on the multi-scale feature map of the medical image to obtain the autocorrelation enhanced multi-scale feature map of the medical image includes: Passing the first convolutional layer of the feature autocorrelation enhancement module on the multi-scale feature map of the medical image to obtain a first feature map; Passing the first feature map through the second convolutional layer of the feature autocorrelation association enhancement module to obtain a second feature map; Expanding each feature matrix of the second feature map along the channel dimension into feature vectors to obtain a sequence of first feature vectors; Calculating the cosine similarity between any two first feature vectors in the sequence of the first feature vectors to obtain a cosine similarity feature graph; The cosine similarity feature map is normalized by a Softmax function to obtain a normalized cosine similarity feature map; Multiplying the normalized cosine similarity feature map and the cosine similarity feature map by position points to obtain a similarity mapping optimization feature map; Passing the similarity map optimization feature map through the first deconvolution layer of the feature autocorrelation association enhancement module to obtain a first deconvolution feature map; Calculating an element-by-element sum of the first deconvolution feature map and the first feature map to obtain a first fused feature map; Passing the first fused feature map through the second deconvolution layer of the feature autocorrelation association enhancement module to obtain a second deconvolution feature map; Calculating the element-by-element sum of the second deconvolution feature map and the medical image multi-scale feature map to obtain the autocorrelation enhanced medical image multi-scale feature map; The optimization of each eigenvalue of the autocorrelation enhanced medical image feature matrix is ​​specifically expressed as follows: the optimization of each eigenvalue of the autocorrelation enhanced medical image feature matrix is ​​performed using the following optimization formula to obtain the optimized autocorrelation enhanced medical image feature matrix; wherein the optimization formula is: ;in, and is the autocorrelation enhanced medical imaging feature matrix No. and eigenvalues, and is the autocorrelation enhanced medical imaging feature matrix The global feature mean of is the first eigenvalues, Represents calculation of the natural exponential function value raised to the power of value.

8. The medical image processing system for report generation according to claim 7, characterized in that: The feature map extraction module comprises: a noise reduction processing unit, configured to perform noise reduction processing on the medical image to be analyzed to obtain a noise-reduced medical image to be analyzed; and The feature extraction unit is used to pass the denoised medical image to be analyzed through a medical image feature extractor based on a pyramid network to obtain the shallow feature map of the medical image, the middle feature map of the medical image and the deep feature map of the medical image.

Images