A medical image processing method and image processing system

Abstract

The present invention relates to a medical image processing method and an image processing system, and relates to the technical field of medical image processing, wherein the method comprises: acquiring multiple medical images of a patient's lesion site; determining a reference medical image from the multiple medical images based on relevant information of the patient's lesion site; performing multi-modal spatial position registration on the multiple medical images based on the reference medical image, and generating multiple medical images after multi-modal spatial position registration; determining a fusion weight corresponding to each medical image based on relevant information of the patient's lesion site; and fusing the multiple medical images after multi-modal spatial position registration based on the fusion weight corresponding to each medical image, and generating a fused medical image, which has the advantage of improving the fusion quality of multi-modal medical images.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a medical image processing method and an image processing system. Background Art

[0002] At present, various modalities of medical imaging equipment have been widely used in clinical diagnosis and medical research. The imaging technologies involved in these multi-modal medical imaging equipment mainly include positron emission computed tomography (PET), computed tomography (CT), magnetic resonance imaging (MR), ultrasound imaging (US) and digital radiography (DR). Among them, CT images, MR images and US images can provide morphological and structural information of the imaging site; PET images can provide metabolic and functional information of the imaging site. Unfortunately, any imaging method of any modality has its own defects. For example, CT scanning has a good imaging effect on tissues with large density differences, and has high spatial and temporal resolution, but poor soft tissue contrast; MR scanning has a high contrast for soft tissue, but the imaging results are usually difficult to quantify; PET scanning images have high sensitivity and specificity, but poor spatial resolution. Medical image data of different modalities can obtain different morphological parameters, metabolic parameters, apparent diffusion coefficient, Young's modulus and other parameters of the imaged tissue. Fusion of medical image information of different modalities can achieve information complementarity and differential verification, and effectively realize early and accurate diagnosis.

[0003] At present, instructions are sent to medical imaging devices of different modalities at the same time to make the medical imaging devices of different modalities collect data at the same time, so as to align the medical images corresponding to the data collected by the medical imaging devices of different modalities in time. However, due to the different time systems used by medical imaging devices of different modalities and the differences in the image acquisition process, the alignment accuracy is low, which in turn causes the fusion processing effect to be poor, resulting in low reliability of the fusion processing results of medical images of different modalities.

[0004] Therefore, it is necessary to provide a medical image processing method and an image processing system for improving the fusion quality of multimodal medical images. Summary of the invention

[0005] The present invention provides a medical image processing method, comprising: acquiring multiple medical images of a patient's lesion site; determining a reference medical image from the multiple medical images based on relevant information of the patient's lesion site; performing multi-modal spatial position alignment on the multiple medical images based on the reference medical image to generate multiple medical images after multi-modal spatial position alignment; determining a fusion weight corresponding to each of the medical images based on the relevant information of the patient's lesion site; and fusing the multiple medical images after multi-modal spatial position alignment based on the fusion weight corresponding to each of the medical images to generate a fused medical image.

[0006] Furthermore, based on the relevant information of the patient's lesion site, a reference medical image is determined from the multiple medical images, including: based on the relevant information of the patient's lesion site, at least one candidate medical image is determined from the multiple medical images; for each of the candidate medical images, a registration cost parameter corresponding to the candidate medical image is calculated; based on the registration cost parameter corresponding to each of the candidate medical images, a reference medical image is determined from the multiple medical images.

[0007] Furthermore, based on the relevant information of the patient's lesion site, at least one candidate medical image is determined from the multiple medical images, including: obtaining relevant information of the lesion site of multiple sample patients and multiple medical images; for each of the sample patients, determining a reference medical image from the multiple medical images of the lesion site of the sample patient; clustering the multiple sample patients based on the relevant information of the lesion site of the multiple sample patients by a clustering algorithm to determine multiple sample clusters; based on the relevant information of the patient's lesion site, determining at least one similar sample cluster from the multiple sample clusters; based on the relevant information of the patient's lesion site, determining multiple similar sample patients from the at least one similar sample cluster; based on the relevant information of the patient's lesion site, determining at least one candidate medical image from the multiple medical images.

[0008] Furthermore, calculating the registration cost parameters corresponding to the candidate medical image includes: determining a reference area of ​​the candidate medical image; extracting a target area from each medical image of the patient's lesion site based on the reference area of ​​the candidate medical image; and calculating the registration cost parameters corresponding to the candidate medical image based on the grayscale information of the reference area of ​​the candidate medical image and the grayscale information of the target area corresponding to each of the medical images.

[0009] Furthermore, based on the grayscale information of the reference area of ​​the candidate medical image and the grayscale information of the target area corresponding to each of the medical images, the registration cost parameters corresponding to the candidate medical image are calculated, including: determining the number of sampling pixels for the grayscale information of the reference area of ​​the candidate medical image; determining multiple pixel sampling schemes based on the number of sampling pixels and the reference area of ​​the candidate medical image; for each pixel sampling scheme, based on the pixel sampling scheme, sampling multiple pixels from the reference area of ​​the candidate medical image and sampling multiple pixels from the target area corresponding to each of the medical images, and determining the grayscale differentiation parameters corresponding to the pixel sampling scheme based on the grayscale information of the multiple pixels sampled from the reference area of ​​the candidate medical image and the grayscale information of the multiple pixels sampled from the target area corresponding to each of the medical images; and calculating the registration cost parameters corresponding to the candidate medical image based on the grayscale differentiation parameters corresponding to each pixel sampling scheme.

[0010] Furthermore, based on the grayscale information of multiple pixels sampled from the reference area of ​​the candidate medical image and the grayscale information of multiple pixels sampled from the target area corresponding to each of the medical images, the grayscale differentiation parameters corresponding to the pixel sampling scheme are determined, including: based on the pixel sampling scheme, sampling multiple pixels from the reference area of ​​the candidate medical image to generate a reference grayscale matrix, wherein the row vectors of the reference grayscale matrix include the grayscale values ​​of the pixels sampled from the reference area of ​​the candidate medical image and the grayscale features of adjacent pixels; based on the pixel sampling scheme, sampling multiple pixels from the target area corresponding to each of the medical images to generate a floating grayscale matrix corresponding to each of the medical images, wherein the row vectors of the floating grayscale matrix include the grayscale values ​​of the pixels sampled from the target area corresponding to the medical image and the grayscale features of adjacent pixels; based on the reference grayscale matrix and the floating grayscale matrix corresponding to each of the medical images, the grayscale differentiation parameters corresponding to the pixel sampling scheme are determined.

[0011] Furthermore, based on the reference medical image, multimodal spatial position alignment is performed on the multiple medical images to generate multiple medical images after multimodal spatial position alignment, including: through a genetic algorithm, based on the grayscale information of the reference medical image and the grayscale information of the multiple medical images, multimodal spatial position alignment is performed on the multiple medical images to generate multiple medical images after multimodal spatial position alignment.

[0012] Furthermore, based on the relevant information of the patient's lesion site, the fusion weight corresponding to each of the medical images is determined, including: obtaining the fused medical images of the lesion sites of the multiple sample patients and the fusion weight corresponding to each of the medical images; based on the fusion weight corresponding to each of the medical images of the multiple similar sample patients, determining multiple fusion weight schemes, each of the fusion weight schemes including the fusion weight corresponding to each of the medical images.

[0013] Furthermore, based on the fusion weight corresponding to each of the medical images, the multiple medical images after the multimodal spatial position alignment are fused to generate a fused medical image, including: for each of the fusion weight schemes, the multiple medical images after the multimodal spatial position alignment are fused based on the fusion weight scheme through a multimodal fusion model to generate an initial fused medical image; and the fused medical image is generated based on the initial fused medical image corresponding to each of the fusion weight schemes through a secondary fusion model.

[0014] The present invention provides a medical image processing system, which is used for the above-mentioned medical image processing method, including: an image acquisition module, which is used to acquire multiple medical images of a patient's lesion site from multiple medical imaging devices; an image processing module, which is used to determine a reference medical image from the multiple medical images based on the relevant information of the patient's lesion site, and is also used to perform multi-modal spatial position alignment on the multiple medical images based on the reference medical image to generate multiple medical images after multi-modal spatial position alignment, and is also used to determine the fusion weight corresponding to each of the medical images based on the relevant information of the patient's lesion site, and is also used to fuse the multiple medical images after multi-modal spatial position alignment based on the fusion weight corresponding to each of the medical images to generate a fused medical image.

[0015] Compared with the prior art, the medical image processing method and image processing system provided in this specification have at least the following beneficial effects:

[0016] 1. By fusing images from multiple imaging modalities (such as CT, MRI, PET, etc.), the advantages of images from different modalities can be combined to provide more comprehensive and richer anatomical structure and functional information. This helps doctors to more accurately determine the nature, boundaries and growth status of lesions. A single imaging modality may be limited by its imaging principle and resolution, resulting in the inability to clearly display certain lesion features. Multimodal fusion images can make up for this deficiency and reduce the possibility of misdiagnosis and missed diagnosis. Based on the fused images, doctors can develop more personalized treatment plans. For example, in tumor treatment, combining images from different modalities can more accurately determine the location, size and shape of the tumor, thereby selecting the most appropriate surgical path or radiotherapy plan;

[0017] 2. By considering the specific information of the patient's lesion site, the most suitable candidate image as the registration benchmark is screened out from a variety of medical images, and the final reference image is further determined by calculating the registration cost parameters. This process ensures the quality of the reference image, thereby improving the accuracy of subsequent registration. By calculating the registration cost parameters such as grayscale differentiation parameters, the similarities and differences between different images can be more accurately evaluated, providing an optimization basis for the registration algorithm and further improving the accuracy of registration. Through the clustering algorithm and the selection of similar sample patients, the number of candidate images that need to be evaluated is reduced, thereby speeding up the processing speed. By comparing multiple pixel sampling schemes, the optimal sampling scheme can be quickly found, reducing unnecessary calculations and improving processing efficiency. Based on the data of sample patients, similar sample clusters are determined through a clustering algorithm, so that the method can adapt to patients with different lesions and enhance the robustness of the method. Provide high-quality fused images: Based on accurate registration results, high-quality fused images can be generated to provide doctors with more comprehensive and accurate lesion information, which helps doctors make more accurate diagnoses and formulate more reasonable treatment plans;

[0018] 3. Genetic algorithm is an optimization algorithm that simulates natural selection and genetic mechanisms and has global search capabilities. It can search for the optimal solution in the entire solution space, thereby avoiding falling into the local optimum and improving the accuracy of registration. Genetic algorithms can adaptively adjust the search direction and step size during the evolution process, continuously optimize the quality of the solution according to the feedback of the fitness function, and further improve the accuracy of registration. Genetic algorithms have implicit parallelism and can process multiple solutions at the same time, thereby accelerating the registration process. Medical image registration often faces complex anatomical structures and changeable imaging conditions. Genetic algorithms can handle these complex situations and improve the robustness of registration. Through genetic operations such as crossover and mutation, genetic algorithms can jump out of local extreme value areas and find the global optimal solution, thereby enhancing the robustness of registration and generating more comprehensive and accurate medical images;

[0019] 4. By considering the specific information of the patient's lesion site and determining personalized fusion weights for each medical image, it can more accurately reflect the importance and contribution of the image in the fusion process, thereby improving the quality of the fused image. The selection of multiple fusion weight schemes and the application of secondary fusion models can further optimize the fusion effect, reduce information loss and redundancy, and make the fused image clearer and more accurate. The fused medical image integrates information from multiple imaging modalities, can provide more comprehensive and accurate lesion information, and help doctors make more accurate diagnosis and treatment decisions. By optimizing the fusion weights, the contrast between the lesion area and the surrounding tissue can be enhanced, making it easier for doctors to observe and identify lesion features. The algorithm automatically determines the fusion weights and performs image fusion, which reduces the impact of manual intervention and subjective judgment, and improves the intelligence and automation level of the processing process. Automated processing can significantly improve the speed and efficiency of image fusion and meet the needs of rapid clinical diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] This specification will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:

[0021] Figure 1 is a flow chart of a medical image processing method shown in one embodiment of the present application;

[0022] Figure 2 is a flowchart of determining at least one candidate medical image shown in an embodiment of the present application;

[0023] Figure 3 It is a module diagram of an image processing system shown in one embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following briefly introduces the drawings required for describing the embodiments.

[0025] Figure 1 is a flow chart of a medical image processing method shown in one embodiment of the present application. Figure 1 As shown, a medical image processing method may include the following steps.

[0026] Step 110, obtaining a plurality of medical images of the patient's lesion site.

[0027] Specifically, a variety of medical images of the patient's lesion site can be obtained from a variety of medical imaging devices, wherein the various medical imaging devices may at least include positron emission tomography (PET), computed tomography (CT), magnetic resonance imaging (MR), ultrasound imaging (US), and digital radiography (DR), etc.

[0028] Step 120: determining a reference medical image from a plurality of medical images based on relevant information of the patient's lesion site.

[0029] Specifically include:

[0030] Based on the relevant information of the patient's lesion site, determining at least one candidate medical image from a plurality of medical images;

[0031] For each candidate medical image, calculating a registration cost parameter corresponding to the candidate medical image;

[0032] Based on the registration cost parameter corresponding to each candidate medical image, a reference medical image is determined from a plurality of medical images. For example, a candidate medical image with the smallest registration cost parameter may be used as a reference medical image.

[0033] Figure 2 is a flowchart of determining at least one candidate medical image shown in an embodiment of the present application, such as Figure 2 As shown, preferably, based on the relevant information of the patient's lesion site, at least one candidate medical image is determined from a plurality of medical images, including:

[0034] Obtain relevant information on the lesion sites of multiple sample patients and multiple medical images;

[0035] For each sample patient, a reference medical image is determined from a plurality of medical images of the lesion site of the sample patient. For example, the reference medical image may be determined from a plurality of medical images of the lesion site of the sample patient manually (e.g., by an expert, etc.), or may be determined from a plurality of medical images of the lesion site of the sample patient based on the quality of the medical images by a machine learning model.

[0036] Clustering the multiple sample patients based on the relevant information of the lesion sites of the multiple sample patients by using a clustering algorithm to determine multiple sample clusters;

[0037] Based on the relevant information of the patient's lesion site, determining at least one similar sample cluster from the multiple sample clusters;

[0038] Based on the relevant information of the lesion site of the patient, a plurality of similar sample patients are determined from at least one similar sample cluster;

[0039] At least one candidate medical image is determined from the plurality of medical images based on a plurality of reference medical images of similar sample patients.

[0040] Specifically, relevant information about the lesion site may include: the specific location of the lesion site, the type of lesion (such as tumor, inflammation, bleeding, etc.), the stage of the lesion (such as early, middle, late) or the severity (such as mild, moderate, severe), and other clinical information of the patient (for example, the patient's age, gender, medical history, symptoms and other clinical information).

[0041] The similarity of the relevant information of any two sample patients can be calculated based on the relevant information of the lesion sites of the two sample patients through a similarity calculation model, wherein the similarity calculation model can be a Convolutional Neural Network (CNN) model.

[0042] The K-means clustering algorithm can be used to cluster multiple sample patients according to the similarity of relevant information between any two sample patients to determine multiple sample clusters.

[0043] The similarity calculation model can be used to calculate the similarity of relevant information between the sample patient corresponding to the cluster center of the sample cluster and the current patient based on the relevant information of the lesion site of the sample patient corresponding to the cluster center of the sample cluster and the relevant information of the lesion site of the patient, and the sample clusters whose relevant information similarity is greater than the first relevant information similarity threshold are taken as similar sample clusters.

[0044] The similarity calculation model can be used to calculate the similarity of relevant information between the sample patients included in the similar sample cluster and the current patient based on the relevant information of the lesion site of the sample patients included in the similar sample cluster and the relevant information of the lesion site of the patient, and the sample patients whose relevant information similarity is greater than a second relevant information similarity threshold are regarded as similar sample patients.

[0045] For example, the reference medical image of similar sample patient 1 is a CT image, and the reference medical image of similar sample patient 2 is an MR image, and the candidate medical images of the current patient include the CT image and the MR image.

[0046] Preferably, calculating the registration cost parameter corresponding to the candidate medical image includes:

[0047] determining a reference region of a candidate medical image;

[0048] Extracting a target region from each medical image of the patient's lesion site based on the reference region of the candidate medical image, wherein the target region may be a region of the patient's lesion site;

[0049] Based on the grayscale information of the reference area of ​​the candidate medical image and the grayscale information of the target area corresponding to each medical image, the registration cost parameter corresponding to the candidate medical image is calculated.

[0050] Specifically, for each medical image, a corresponding region extraction model may be established, and a target region may be extracted from the medical image through the corresponding region extraction model, wherein the region extraction model may be a Convolutional Neural Networks (CNN) model.

[0051] Preferably, based on the grayscale information of the reference area of ​​the candidate medical image and the grayscale information of the target area corresponding to each medical image, calculating the registration cost parameter corresponding to the candidate medical image includes:

[0052] For the grayscale information of the reference area of ​​the candidate medical image, determining the number of sampling pixels;

[0053] Determining a plurality of pixel sampling schemes based on the number of sampled pixels and a reference region of the candidate medical image;

[0054] For each pixel sampling scheme, based on the pixel sampling scheme, a plurality of pixels are sampled from a reference area of ​​the candidate medical image, and a plurality of pixels are sampled from a target area corresponding to each medical image, and a grayscale differentiation parameter corresponding to the pixel sampling scheme is determined based on grayscale information of the plurality of pixels sampled from the reference area of ​​the candidate medical image and grayscale information of the plurality of pixels sampled from the target area corresponding to each medical image, wherein the target area corresponding to the medical image may be an area in the medical image corresponding to a pixel position of the reference area of ​​the candidate medical image;

[0055] Based on the grayscale differentiation parameters corresponding to each pixel sampling scheme, the registration cost parameters corresponding to the candidate medical image are calculated.

[0056] Specifically, the reference area of ​​the candidate medical image may be divided into multiple sub-areas, and for each sub-area, the grayscale value standard deviation of the pixels in the sub-area is calculated, and the number of sampling pixels is determined based on the grayscale value standard deviation of the pixels in each sub-area.

[0057] The number of sampled pixels can be determined based on the following formula:

[0058]

[0059] in, is the number of sampled pixels, is the preset number of sampling pixels, is the total number of sub-regions, is the standard deviation of the grayscale values ​​of the pixels in the mth sub-region, is the preset parameter, [ ] is the rounding operation.

[0060] The positions of at least some of the sampled pixels may be different in different pixel sampling schemes.

[0061] The registration cost parameter corresponding to the candidate medical image can be calculated based on the following formula:

[0062]

[0063] in, is the registration cost parameter corresponding to the i-th candidate medical image, is the weight corresponding to the jth pixel sampling scheme under the i-th candidate medical image, is the grayscale differentiation parameter corresponding to the jth pixel sampling scheme under the i-th candidate medical image, is the total number of pixel sampling schemes under the i-th candidate medical image.

[0064] Preferably, based on the grayscale information of multiple pixels sampled from the reference area of ​​the candidate medical image and the grayscale information of multiple pixels sampled from the target area corresponding to each medical image, determining the grayscale differentiation parameter corresponding to the pixel sampling scheme includes:

[0065] Based on a pixel sampling scheme, a plurality of pixels are sampled from a reference region of the candidate medical image to generate a reference grayscale matrix, wherein a row vector of the reference grayscale matrix includes grayscale values ​​of pixels sampled from the reference region of the candidate medical image and grayscale features of adjacent pixels;

[0066] Based on the pixel sampling scheme, a plurality of pixels are sampled from the target area corresponding to each medical image to generate a floating grayscale matrix corresponding to each medical image, wherein the row vector of the floating grayscale matrix includes the grayscale value of the pixel sampled from the target area corresponding to the medical image and the grayscale features of the adjacent pixels;

[0067] Based on a reference grayscale matrix and a floating grayscale matrix corresponding to each medical image, a grayscale differentiation parameter corresponding to a pixel sampling scheme is determined.

[0068] Specifically, for each sampled pixel of the reference area of ​​the candidate medical image, pixels whose pixel distance to the pixel is less than a pixel distance threshold (for example, 2 pixels) can be regarded as neighboring pixels of the pixel, and the grayscale features of the neighboring pixels corresponding to the pixel can be calculated based on the grayscale value of the pixel and the grayscale values ​​of the neighboring pixels of the pixel.

[0069] The adjacent pixel grayscale features can be calculated based on the following formula:

[0070]

[0071] in, is the grayscale feature of the adjacent pixels of the e-th sampled pixel in the reference area of ​​the i-th candidate medical image, is the gray value of the fth neighboring pixel of the eth sampled pixel in the reference area of ​​the i-th candidate medical image, is the gray value of the e-th sampled pixel in the reference area of ​​the i-th candidate medical image, is the total number of adjacent pixels of the e-th sampled pixel in the reference area of ​​the i-th candidate medical image.

[0072] The grayscale differentiation parameter corresponding to the pixel sampling scheme can be calculated based on the following formula:

[0073]

[0074] in, is the total number of various medical images, is the gray value of the e-th sampled pixel in the target area of ​​the h-th medical image, is the grayscale feature of the adjacent pixels of the e-th sampled pixel in the target area of ​​the h-th medical image, is the total number of pixels sampled.

[0075] Step 130 , based on the reference medical image, perform multi-modal spatial position registration on the multiple medical images to generate multiple medical images after multi-modal spatial position registration.

[0076] Specifically include:

[0077] Through a genetic algorithm, based on the grayscale information of a reference medical image and the grayscale information of multiple medical images, multi-modal spatial position alignment is performed on multiple medical images to generate multiple medical images after multi-modal spatial position alignment.

[0078] Specifically, a variety of medical images after multi-modal spatial position registration can be generated based on the following process:

[0079] S11. Define the fitness function:

[0080] The fitness function is a criterion for evaluating the quality of each potential solution (i.e., image registration parameters) in the genetic algorithm. In multimodal medical image registration, the fitness function is usually based on image similarity metrics, such as mutual information (MI), normalized cross-correlation (NCC), or normalized mutual information (NMI). These metrics can evaluate the degree of similarity between two images under given transformation parameters.

[0081] S12. Initialize the population:

[0082] The population is a set of initial solutions in the genetic algorithm, each of which represents a set of possible image registration parameters (such as translation, rotation, scaling, etc.). These parameters can be randomly generated or initialized based on some prior knowledge.

[0083] S13. Select operation:

[0084] In the selection operation, according to the evaluation results of the fitness function, excellent individuals are selected from the current population as parents of the next generation. Selection strategies include roulette selection, tournament selection, etc.

[0085] S14, cross operation:

[0086] Crossover (or hybridization) is the main way to generate new solutions in genetic algorithms. It generates new offspring individuals by exchanging some genes (i.e., registration parameters) of two or more parent individuals. The design of the crossover operation should take into account the characteristics of medical image registration to ensure that the generated parameters are physically meaningful (e.g., the rotation angle should be limited to a reasonable range).

[0087] S15, mutation operation:

[0088] The mutation operation randomly changes certain genes in individuals with a certain probability to increase the diversity of the population and avoid premature convergence. In image registration, mutation can be a small random adjustment to the registration parameters.

[0089] S16, Iteration and Optimization:

[0090] Repeat the selection, crossover and mutation operations to form a new population and re-evaluate the fitness of each individual. This process will continue to iterate until a certain stopping condition is met (such as reaching the maximum number of iterations, the fitness function value no longer significantly improves, etc.).

[0091] S17, generate the registered image:

[0092] The individual with the highest fitness in the final population (i.e., the optimal registration parameter) is used to perform spatial transformation on multiple medical images other than the benchmark medical image to generate a registered image.

[0093] Step 140: Determine the fusion weight corresponding to each medical image based on the relevant information of the patient's lesion site.

[0094] Specifically include:

[0095] Obtaining fused medical images of lesion sites of multiple sample patients and fusion weights corresponding to each medical image;

[0096] Based on the fusion weights corresponding to each medical image of multiple similar sample patients, multiple fusion weight schemes are determined, and each fusion weight scheme includes a fusion weight corresponding to each medical image.

[0097] For example, the fusion weight corresponding to each medical image of a similar sample patient can be used as a fusion weight scheme. For another example, the fusion weight corresponding to each medical image of multiple similar sample patients can be weighted summed as a fusion weight scheme.

[0098] Step 150 , based on the fusion weight corresponding to each medical image, the multiple medical images after multi-modal spatial position registration are fused to generate a fused medical image.

[0099] Specifically include:

[0100] For each fusion weight scheme, a plurality of medical images after multi-modal spatial position registration are fused based on the fusion weight scheme through a multi-modal fusion model to generate an initial fused medical image, wherein the multi-modal fusion model may be a Generative Adversarial Networks (GAN) model;

[0101] A fused medical image is generated based on the initial fused medical image corresponding to each fusion weight scheme through a secondary fusion model, wherein the secondary fusion model can be a Generative Adversarial Networks (GAN) model.

[0102] Figure 3 is a module diagram of an image processing system shown in an embodiment of the present application, such as Figure 3 As shown, an image processing system may include an image acquisition module and an image processing module.

[0103] The image acquisition module can be used to acquire a variety of medical images of the patient's lesion site from a variety of medical imaging devices.

[0104] The image processing module can be used to determine a reference medical image from a plurality of medical images based on relevant information about the patient's lesion site, and to perform multimodal spatial position registration on a plurality of medical images based on the reference medical image to generate a plurality of medical images after multimodal spatial position registration. It can also be used to determine the fusion weight corresponding to each medical image based on relevant information about the patient's lesion site, and to fuse a plurality of medical images after multimodal spatial position registration based on the fusion weight corresponding to each medical image to generate a fused medical image.

[0105] An image processing system may be used to execute an image processing method. For more description of an image processing system, please refer to the relevant description of an image processing method, which will not be repeated here.

[0106] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, as an example and not a limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.

Claims

1. A medical image processing method, characterized in that: include: Acquire multiple medical images of the patient's lesion site from multiple medical imaging devices; Determining a reference medical image from the plurality of medical images based on the relevant information of the patient's lesion site; Based on the reference medical image, performing multi-modal spatial position registration on the multiple medical images to generate multiple medical images after multi-modal spatial position registration; Based on the relevant information of the patient's lesion site, determining a fusion weight corresponding to each of the medical images; Based on the fusion weight corresponding to each of the medical images, the multiple medical images after the multi-modal spatial position registration are fused to generate a fused medical image; Based on the relevant information of the patient's lesion site, determining a reference medical image from the multiple medical images includes: Based on the relevant information of the patient's lesion site, determining at least one candidate medical image from the multiple medical images; For each of the candidate medical images, calculating a registration cost parameter corresponding to the candidate medical image; Determining a reference medical image from the plurality of medical images based on a registration cost parameter corresponding to each of the candidate medical images; Based on the relevant information of the patient's lesion site, determining at least one candidate medical image from the multiple medical images, including: Obtain relevant information about the lesion sites of multiple sample patients and multiple medical images; For each of the sample patients, determining a reference medical image from a plurality of medical images of the lesion site of the sample patient; Clustering the multiple sample patients based on the relevant information of the lesion sites of the multiple sample patients by a clustering algorithm to determine multiple sample clusters; Based on the relevant information of the lesion site of the patient, determining at least one similar sample cluster from the multiple sample clusters; Based on the relevant information of the patient's lesion site, determining a plurality of similar sample patients from the at least one similar sample cluster; At least one candidate medical image is determined from the multiple medical images based on the multiple reference medical images of the similar sample patients.

2. A medical image processing method according to claim 1, characterized in that: Calculating the registration cost parameter corresponding to the candidate medical image includes: determining a reference region of the candidate medical image; Extracting a target region from each medical image of the patient's lesion site based on the reference region of the candidate medical image; Based on the grayscale information of the reference area of ​​the candidate medical image and the grayscale information of the target area corresponding to each of the medical images, the registration cost parameters corresponding to the candidate medical images are calculated.

3. A medical image processing method according to claim 2, characterized in that: Calculating the registration cost parameters corresponding to the candidate medical images based on the grayscale information of the reference area of ​​the candidate medical images and the grayscale information of the target area corresponding to each of the medical images includes: For the grayscale information of the reference area of ​​the candidate medical image, determining the number of sampling pixels; Determining a plurality of pixel sampling schemes based on the number of sampling pixels and a reference area of ​​the candidate medical image; For each pixel sampling scheme, based on the pixel sampling scheme, a plurality of pixels are sampled from a reference area of ​​the candidate medical image, and a plurality of pixels are sampled from a target area corresponding to each of the medical images, and a grayscale differentiation parameter corresponding to the pixel sampling scheme is determined based on grayscale information of the plurality of pixels sampled from the reference area of ​​the candidate medical image and grayscale information of the plurality of pixels sampled from the target area corresponding to each of the medical images; Based on the grayscale differentiation parameters corresponding to each pixel sampling scheme, the registration cost parameters corresponding to the candidate medical image are calculated.

4. A medical image processing method according to claim 3, characterized in that: Determining a grayscale differentiation parameter corresponding to the pixel sampling scheme based on grayscale information of a plurality of pixels sampled from a reference region of the candidate medical image and grayscale information of a plurality of pixels sampled from a target region corresponding to each of the medical images includes: Based on the pixel sampling scheme, a plurality of pixels are sampled from the reference region of the candidate medical image to generate a reference grayscale matrix, wherein a row vector of the reference grayscale matrix includes grayscale values ​​of pixels sampled from the reference region of the candidate medical image and grayscale features of adjacent pixels; Based on the pixel sampling scheme, a plurality of pixels are sampled from the target area corresponding to each of the medical images to generate a floating grayscale matrix corresponding to each of the medical images, wherein a row vector of the floating grayscale matrix includes grayscale values ​​of pixels sampled from the target area corresponding to the medical image and grayscale features of adjacent pixels; Based on the reference grayscale matrix and the floating grayscale matrix corresponding to each of the medical images, the grayscale differentiation parameters corresponding to the pixel sampling scheme are determined.

5. A medical image processing method according to any one of claims 1 to 4, characterized in that: Based on the reference medical image, performing multi-modal spatial position registration on the multiple medical images to generate multiple medical images after multi-modal spatial position registration, including: Through a genetic algorithm, based on the grayscale information of the reference medical image and the grayscale information of the multiple medical images, multi-modal spatial position alignment is performed on the multiple medical images to generate multiple medical images after multi-modal spatial position alignment.

6. A medical image processing method according to claim 1, characterized in that: Based on the relevant information of the patient's lesion site, determining the fusion weight corresponding to each of the medical images, including: Acquire fused medical images of the lesion sites of the multiple sample patients and a fusion weight corresponding to each of the medical images; Based on the fusion weights corresponding to each of the medical images of the multiple similar sample patients, multiple fusion weight schemes are determined, each fusion weight scheme including a fusion weight corresponding to each of the medical images.

7. A medical image processing method according to claim 6, characterized in that: Based on the fusion weight corresponding to each of the medical images, the multiple medical images after the multi-modal spatial position registration are fused to generate a fused medical image, including: For each of the fusion weight schemes, a plurality of medical images after the multi-modal spatial position registration are fused based on the fusion weight scheme through a multi-modal fusion model to generate an initial fused medical image; The fused medical image is generated by a secondary fusion model based on the initial fused medical image corresponding to each fusion weight scheme.

8. A medical image processing system, characterized in that: A medical image processing method for executing any one of claims 1 to 7, comprising: An image acquisition module, used for acquiring a variety of medical images of a patient's lesion site from a variety of medical imaging devices; The image processing module is used to determine a reference medical image from the multiple medical images based on the relevant information of the patient's lesion site, and is also used to perform multi-modal spatial position alignment on the multiple medical images based on the reference medical image to generate multiple medical images after multi-modal spatial position alignment. It is also used to determine the fusion weight corresponding to each of the medical images based on the relevant information of the patient's lesion site, and is also used to fuse the multiple medical images after multi-modal spatial position alignment based on the fusion weight corresponding to each of the medical images to generate a fused medical image.

Images