A method and system for fusing three-dimensional real-life medical images and path diagrams

Abstract

The present invention provides a method and system for fusing three-dimensional real-life medical images and path graphs, and relates to the field of three-dimensional image fusion technology. It collects multiple types of medical images in a target area, performs grayscale processing and classifies them according to preset thresholds to obtain an image structure feature information set, collects a path graph in the target area, performs grayscale processing and threshold classification processing to obtain a path structure feature information set, divides and compares it, and determines whether it meets the preset structure feature error threshold. If it does not meet the threshold, the corresponding structure feature information is deleted, and the modified structure feature information is generated based on the interpolation method. The three-dimensional image fusion is constructed and rendered to obtain a fused medical image. The present invention solves the technical problem that medical imaging equipment in the prior art can only collect and present two-dimensional plane information, resulting in poor medical image judgment effect, and realizes the fusion of three-dimensional real-life medical images and path graphs, achieving the technical effect of improving the medical image judgment effect.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional image fusion, and in particular to a method and system for fusing three-dimensional real-scene medical images with a path map. Background Art

[0002] Medical imaging refers to the technology and process of obtaining images of internal tissues of the human body or a part of the human body in a non-invasive manner for medical treatment or medical research. It includes the following two relatively independent research directions: medical imaging system and medical image processing. The former refers to the process of image formation, including the study of imaging mechanism, imaging equipment, imaging system analysis and other issues; the latter refers to further processing of the acquired images, the purpose of which is to restore the original unclear image, to highlight certain feature information in the image, or to classify the image pattern, etc. Medical imaging provides great help for clinical diagnosis and treatment guidance.

[0003] However, in the prior art, medical imaging equipment can only collect and present two-dimensional plane information, so doctors need to adjust the viewing angle to obtain more angle and size information, resulting in poor medical image judgment results. Summary of the invention

[0004] The embodiments of the present application provide a method and system for fusing three-dimensional real-life medical images with a path map, which is used to solve the technical problem that medical imaging equipment in the prior art can only collect and present two-dimensional plane information, so that doctors need to adjust the viewing angle to obtain more angle and size information, resulting in poor medical image judgment results.

[0005] In view of the above problems, an embodiment of the present application provides a method and system for fusing three-dimensional real-life medical images with a path map.

[0006] In a first aspect, an embodiment of the present application provides a method for fusing a three-dimensional real-life medical image with a path map, the method comprising: collecting multiple types of medical images within a target area to obtain a medical image set; graying the medical image set, and classifying the grayed medical image set according to a preset threshold to obtain an image structure feature information set; inputting the image structure feature information set into a three-dimensional medical image analysis model to obtain a three-dimensional image structure feature information set; collecting a path map within the target area, and performing graying and threshold classification processing to obtain a path structure feature information set of the path map; dividing and comparing the data in the three-dimensional image structure feature information set and the path structure feature information set to determine whether a preset structure feature error threshold is met to obtain multiple groups of determination results; if the determination result is no, deleting the corresponding structure feature information, and generating modified structure feature information within the structure feature error threshold based on an interpolation method to obtain a corrected three-dimensional image structure feature information set and a corrected path structure feature information set; performing three-dimensional image fusion construction and rendering based on the corrected three-dimensional image structure feature information set and the corrected path structure feature information set to obtain a fused medical image.

[0007] In a second aspect, an embodiment of the present application provides a three-dimensional real-life medical image and path map fusion system, the system comprising: a medical image acquisition module, the medical image acquisition module being used to acquire multiple types of medical images within a target area to obtain a medical image set; a grayscale processing module, the grayscale processing module being used to grayscale the medical image set, and classify the grayscaled medical image set according to a preset threshold to obtain an image structure feature information set; a model analysis module, the model analysis module being used to input the image structure feature information set into a three-dimensional medical image analysis model to obtain a three-dimensional image structure feature information set; a path map acquisition module, the path map acquisition module being used to acquire a path map within the target area, and performing grayscale and threshold classification processing to obtain the path map. A path structure feature information set of a graph; a data division module, the data division module is used to divide and compare the data in the three-dimensional image structure feature information set and the path structure feature information set, judge whether the preset structure feature error threshold is met, and obtain multiple groups of judgment results; an information correction module, the information correction module is used to delete the corresponding structure feature information when the judgment result is no, and generate modified structure feature information within the structure feature error threshold based on the interpolation method, to obtain a corrected three-dimensional image structure feature information set and a corrected path structure feature information set; a fused medical image acquisition module, the fused medical image acquisition module is used to perform three-dimensional image fusion construction and rendering based on the corrected three-dimensional image structure feature information set and the corrected path structure feature information set, to obtain a fused medical image.

[0008] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0009] The embodiment of the present application provides a method for fusing a three-dimensional real-life medical image with a path map, which relates to the field of three-dimensional image fusion technology. It collects multiple types of medical images in a target area, obtains a medical image set, performs grayscale processing, and classifies according to a preset threshold value to obtain an image structure feature information set, obtains a three-dimensional image structure feature information set, collects a path map in the target area, performs grayscale and threshold classification processing, obtains a path structure feature information set of the path map, divides and compares, determines whether a preset structure feature error threshold is met, obtains multiple groups of judgment results, and deletes the corresponding structure feature information if the judgment result is no. Based on the interpolation method, the modified structure feature information is generated to obtain a corrected three-dimensional image structure feature information set and a corrected path structure feature information set, performs three-dimensional image fusion construction and rendering, and obtains a fused medical image. The method solves the technical problem that the medical imaging equipment in the prior art can only collect and present two-dimensional plane information, so that the doctor needs to adjust the viewing angle to obtain more angle size information, resulting in poor judgment effect on the medical image, realizes the fusion of the three-dimensional real-life medical image and the path map, thereby accurately displaying the three-dimensional stereo image, and then allowing the doctor to obtain more comprehensive and accurate organ structure morphology information, so as to achieve the technical effect of improving the judgment effect of the medical image.

[0010] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A schematic diagram of a method for fusing a three-dimensional real-life medical image with a path map is provided for an embodiment of the present application;

[0012] Figure 2 A schematic diagram of the process of obtaining a medical image set in a method for fusing a three-dimensional real-life medical image with a path map is provided for an embodiment of the present application;

[0013] Figure 3 A schematic diagram of the process of obtaining an image structure feature information set and a background information set in a method for fusing a three-dimensional real-life medical image with a path map is provided for an embodiment of the present application;

[0014] Figure 4 A schematic diagram of the structure of a three-dimensional real-life medical image and path map fusion system is provided for an embodiment of the present application.

[0015] Explanation of the accompanying drawings: medical image acquisition module 10, grayscale processing module 20, model analysis module 30, path map acquisition module 40, data division module 50, information correction module 60, fused medical image acquisition module 70. DETAILED DESCRIPTION

[0016] The embodiment of the present application provides a method for fusing three-dimensional real-life medical images with a path map, so as to solve the technical problem that the prior art can only make judgments through a single medical image, resulting in poor medical image judgment results.

[0017] Embodiment 1

[0018] like Figure 1 As shown, the embodiment of the present application provides a method for fusing a three-dimensional real-life medical image with a path map, the method comprising:

[0019] Step S100: collecting multiple types of medical images in the target area to obtain a medical image set;

[0020] Furthermore, if Figure 2 As shown, step S100 of the present application also includes:

[0021] Step S110: collecting X-ray images, CT images and MRI images within the target area;

[0022] Step S120: Obtain the medical image set based on the X-ray image, CT image and MRI image.

[0023] Specifically, the target area is an area that requires medical image acquisition, such as a surgical area, and X-ray images, CT images, and MRI images are acquired for the target area. Both X-ray images and CT images are completed by X-rays. X-ray images can obtain perspective images, CT images can obtain cross-sectional images, and MRI images are magnetic resonance imaging examinations that can obtain cross-sectional, sagittal, and coronal images.

[0024] X-rays can image the structure of human tissues based on two points. One is the basic characteristics of X-rays, namely penetrability, absorbability, fluorescence effect, and photosensitivity effect. The other is the inherent density and thickness differences of human tissue structures. When X-rays pass through tissues of different densities and thicknesses in the human body, they are absorbed by the tissues to different degrees, resulting in different amounts of X-rays reaching the radiation receiving device, thus forming a black-and-white contrast image. The higher the density of the material, the more X-rays it absorbs. Based on the different degrees of X-ray absorption, biological tissues are divided into three categories: high-density tissue, medium-density tissue, and low-density tissue. In addition, the thicker the biological tissue, the less X-rays it can pass through, and the whiter it will be. When the tissue structure is diseased, its density and thickness change, and the black-and-white-gray contrast in the X-ray image changes.

[0025] By collecting X-ray images, CT images and MRI images, a set of medical images of the target area is obtained, which enables the grasp of multi-angle and multi-level two-dimensional images of the target area, laying the foundation for subsequent three-dimensional medical image analysis.

[0026] Step S200: gray-scale the medical image set, and classify the gray-scaled medical image set according to a preset threshold to obtain an image structure feature information set;

[0027] Furthermore, if Figure 3 As shown, step S200 of the present application also includes:

[0028] Step S210: grayscale the medical images in the medical image set to obtain a grayscale image set;

[0029] Step S220: acquiring the preset threshold value according to the grayscale medical image of the same area of ​​the target area within the historical time;

[0030] Step S230: using the preset threshold to classify the medical images in the grayscale image set to obtain the image structure feature information set and the background information set.

[0031] Specifically, grayscale processing is the process of converting a color image into a grayscale image. The color information is usually expressed in the RGB model, which is also called the additive color mixing model. It is a method of mixing colors by superimposing the three colors of R (red), G (green) and B (blue). It is especially suitable for luminous objects such as displays. Grayscale processing is the process of making the R, G, and B components of a color image equal, that is, R = G = B. There are mainly component method, maximum value method, average value method, weighted average method, etc. For example, the average value method is used, that is, the brightness of the three components in the color image is averaged to obtain a gray value. The calculation formula is:

[0032] Gray(i,j)=(R(i,j)+G(i,j)+B(i,j) / 3

[0033] Among them, Gray(i,j) is the gray value obtained, R(i,j), G(i,j), and B(j,j) are the three-component brightness of R(red), G(green), and B(blue) in the color image. After calculation, the color at this time is represented by the gray color, and the image obtained is a gray image. The color value of each pixel on the gray image is also called grayscale, which refers to the color depth of the point in the black and white image, and generally ranges from 0 to 255. Therefore, for the computer, the range of a pixel in the gray image is only 0-255. When R=G=B=255, the gray value reaches the highest and is displayed as white, otherwise it is displayed as black. The color medical image is converted into a gray image through grayscale processing to obtain a gray image set.

[0034] Acquire multiple grayscale medical images collected in the target area within a historical time, wherein the historical time is the time when the target area medical images were collected multiple times in the past, and the multiple grayscale medical images are multiple medical images of different types and angles of the same area, such as perspective images, cross-sectional images, sagittal and coronal images of the vascular tissue structure after angiography, and the main body of the multiple grayscale medical images is the vascular tissue structure. According to the vascular tissue structure image, the grayscale value of the vascular tissue structure is low, and it is displayed as black in the image, and the background grayscale value is high, and it is displayed as white in the image. According to the distribution of the vascular tissue structure of the multiple grayscale medical images in the image, a preset threshold is set. For example, the preset threshold is set to 200, and the grayscale value greater than or equal to 200 in the grayscale image is determined as a background pixel, and the grayscale value less than 200 is determined as a structural pixel.

[0035] The preset threshold is used to divide and classify the medical images in the grayscale image set, and the coordinate set of pixels with grayscale values ​​greater than or equal to the preset threshold, that is, the grayscale value is greater than or equal to 200, is obtained as the background information set; the coordinate set of pixels with grayscale values ​​less than the preset threshold, that is, the grayscale value is less than 200, is obtained as the image structure feature information set.

[0036] Step S300: inputting the image structure feature information set into a three-dimensional medical image analysis model to obtain a three-dimensional image structure feature information set;

[0037] Furthermore, step S300 of the present application also includes:

[0038] Step S310: Acquire a plurality of sample image structure feature information sets of the same area of ​​the target area;

[0039] Step S320: constructing a three-dimensional medical image based on the plurality of sample image structure feature information sets to obtain a plurality of sample three-dimensional image structure feature information sets;

[0040] Step S330: using the plurality of sample image structure feature information sets and the plurality of sample three-dimensional image structure feature information sets as construction data to construct the three-dimensional medical image analysis model;

[0041] Step S340: inputting the image structure feature information set into the three-dimensional medical image analysis model to obtain the three-dimensional image structure feature information set.

[0042] Specifically, multiple sample image structure feature information sets within the target area are obtained, wherein the multiple sample image structure feature information sets include coordinate sets of perspective images, cross-sectional images, sagittal plane images, and coronal plane images. A three-dimensional rectangular coordinate system is constructed, in which there are three mutually perpendicular axes: the x-axis and the y-axis are mutually perpendicular, and the plane formed is parallel to the front of the target area, and the z-axis is perpendicular to the front of the target area. The sample image structure feature information set corresponding to the perspective image, that is, the coordinate set, is input into the x-axis and the y-axis to obtain a plane in the three-dimensional medical image. The cross-sectional image is a cross-sectional image of the body obtained along the xz plane, the sagittal image is a cross-sectional image of the body obtained along the yz plane, and the coronal image is a cross-sectional image of the body obtained along the xy plane. The coordinate set corresponding to the cross-sectional image is input into the x-axis and z-axis, the coordinate set corresponding to the sagittal image is input into the y-axis and z-axis, and the coordinate set corresponding to the coronal image is input into the x-axis and y-axis to complete the construction of the three-dimensional medical image. The coordinate set of each pixel point in the three-dimensional medical image is the three-dimensional image structure feature information set, and multiple sample three-dimensional image structure feature information sets are obtained.

[0043] Based on the BP neural network, the three-dimensional medical image analysis model is constructed, and according to the multiple sample image structure feature information sets and the multiple sample three-dimensional image structure feature information sets and their mapping relationships, the training set, the verification set and the test set are divided and labeled, and the three-dimensional medical image analysis model is trained, verified and tested to obtain the three-dimensional medical image analysis model whose accuracy meets the preset requirements, wherein the input data of the three-dimensional medical image analysis model is the image structure feature information set, and the output data is the three-dimensional image structure feature information set.

[0044] The image structure feature information set is input into the three-dimensional medical image analysis model, the image structure feature information set is matched with a sample image structure feature information set, and according to a mapping relationship between the multiple sample image structure feature information sets and the multiple sample three-dimensional image structure feature information sets, a three-dimensional image structure feature information set corresponding to the image structure feature information set is obtained.

[0045] Furthermore, step S330 of the present application also includes:

[0046] Step S331: annotating and dividing the plurality of sample image structure feature information sets and the plurality of sample 3D image structure feature information sets to obtain a training set, a validation set and a test set;

[0047] Step S332: constructing a network structure of the three-dimensional medical image analysis model based on a BP neural network, wherein the input data of the three-dimensional medical image analysis model is an image structure feature information set, and the output data is a three-dimensional image structure feature information set;

[0048] Step S333: using the training set to perform iterative supervised training on the three-dimensional medical image analysis model, and updating model parameters by gradient descent method until the three-dimensional medical image analysis model converges or the accuracy reaches a preset requirement;

[0049] Step S334: using the verification set and the test set to verify and test the three-dimensional medical image analysis model, to obtain the three-dimensional medical image analysis model whose accuracy meets the preset requirements.

[0050] Specifically, BP neural network is a multi-layer feedforward neural network trained according to the error back propagation algorithm. It does not need to determine the mathematical equation of the mapping relationship between input and output in advance. It only learns certain rules through its own training to obtain the result closest to the expected output value when the input value is given. The core of BP neural network to realize its function is BP algorithm. Its basic idea is gradient descent method, which uses gradient search technology to minimize the mean square error between the actual output value and the expected output value of the network.

[0051] A plurality of sample image structure feature information sets and a plurality of sample three-dimensional image structure feature information sets are used as construction data, and the construction data are randomly divided into a training set, a validation set and a test set according to a certain ratio, and the divided training set, validation set and test set are labeled. Exemplarily, the preferred ratio of the present application is 8:1:1, wherein the training set is used to train the model, that is, to determine the learning parameters such as the weight and bias of the model; the validation set is used to verify each model using the validation set data after multiple models are trained in the training set, and to record the model accuracy, and then select the best model and its corresponding parameters; the test set is used only once, that is, it is used to evaluate the final model after the training is completed. It does not participate in the process of learning parameters or the process of hyperparameter selection, but is only used for model evaluation.

[0052] Based on machine learning, supervised training of the model is performed through the training set, the obtained model is verified through the validation set, the trained model is evaluated through the test set, and the model parameters are updated through the gradient descent method, wherein the gradient descent method is an iterative method for solving unconstrained optimization problems, each time moving a certain step length along the negative gradient direction of the objective function until a local minimum or a global minimum is reached. The parameters are continuously adjusted until the accuracy meets the preset requirements. For example, the preset requirement is set to an accuracy of 95%. When the model accuracy meets the preset requirements, that is, the accuracy reaches 95%, it is determined that the model meets the use requirements, and the three-dimensional medical image analysis model is output.

[0053] Step S400: collecting the path map in the target area, and performing grayscale conversion and threshold classification processing to obtain a path structure feature information set of the path map;

[0054] Specifically, a 3D-DSA examination is used (DSA stands for Digital Subtraction Angiograph y, which is a means of vascular development. For example, this technology is used in interventional examinations and is widely used in cerebral blood vessels, coronary blood vessels, etc. It can avoid the influence of bones and organs and more intuitively judge the shape and changes of blood vessels.) The blood vessels are imaged by injecting contrast agents, and the vascular path map in the target area is obtained. The path map is grayed and threshold classified. The method used is exactly the same as the above step S200, which will not be repeated here.

[0055] Step S500: dividing and comparing the data in the three-dimensional image structure feature information set and the path structure feature information set to determine whether a preset structure feature error threshold is met, and obtaining multiple groups of determination results;

[0056] Furthermore, step S500 of the present application includes:

[0057] Step S510: dividing the data in the 3D image structure feature information set and the path structure feature information set according to the same division distance to obtain a plurality of intervals, each interval including a corresponding 3D image structure feature information subset and a path structure feature information subset;

[0058] Step S520: setting and obtaining the preset structural feature error threshold according to the three-dimensional medical images of the same area of ​​the target area within the historical time;

[0059] Step S530: According to the multiple intervals, the three-dimensional image structure feature information subset and the path structure feature information subset in each interval are judged one by one whether they meet the preset structure feature error threshold, and the multiple groups of judgment results are obtained.

[0060] Specifically, the division distance is set according to the size of the three-dimensional image, and the coordinates in the three-dimensional image structure feature information set and the path structure feature information set are equally divided by the division distance. Exemplarily, 10% of the number of pixels of the three-dimensional image on the three coordinate axes is set as the division distance, and the number of pixels of the three-dimensional image on the x-axis, y-axis, and z-axis are set to U, V, and W respectively. Then, after the division, each interval is 10% U, 10% V, and 10% W, and each interval corresponds to the corresponding three-dimensional image structure feature information subset and path structure feature information subset.

[0061] Similarly, a preset structural feature error threshold is set according to the size of the three-dimensional medical image. Exemplarily, it is set to 0.1% of the number of pixels of the three-dimensional medical image on the three coordinate axes. For example, if the size of the three-dimensional medical image is 5000*6000*7000, the preset structural feature error threshold is set to 5000*0.1%, 6000*0.1%, and 7000*0.1% on the x-axis, y-axis, and z-axis, respectively. The preset structural feature error threshold is used to judge the errors of the three-dimensional image structural feature information subset and the path structural feature information subset in each interval.

[0062] The three-dimensional image structure feature information subset and the path structure feature information subset in the same interval are judged, that is, the coordinates of the x-axis, y-axis, and z-axis of the three-dimensional image in the same interval are compared with the coordinates of the x-axis, y-axis, and z-axis of the path. When the coordinates of the x-axis, y-axis, and z-axis respectively meet the preset structure feature error thresholds on the corresponding axes, it is judged that the preset structure feature error threshold is met, and the satisfaction indicates that the errors between the two are within an acceptable range and can be ignored; when at least one of the coordinates of the x-axis, y-axis, and z-axis does not meet the preset structure feature error threshold on the corresponding axis, it is judged that the preset structure feature error threshold is not met, and the failure to meet the preset structure feature error threshold indicates that the errors between the two are large and cannot be fused, and further processing is required. In this way, the three-dimensional image structure feature information subset and the path structure feature information subset in each interval are judged one by one to obtain the multiple groups of judgment results, including the results that meet the preset structure feature error threshold and the results that do not meet the preset structure feature error threshold.

[0063] Step S600: if the judgment result is no, the corresponding structural feature information is deleted, and within the structural feature error threshold, based on the interpolation method, the modified structural feature information is generated to obtain a corrected 3D image structural feature information set and a corrected path structural feature information set;

[0064] Furthermore, step S600 of the present application includes:

[0065] Step S610: if the judgment result is no, the 3D image structure feature information subset and the path structure feature information subset in the corresponding interval are deleted to form a blank interval;

[0066] Step S620: Perform linear interpolation of the structural feature information based on the three-dimensional image structural feature information subset and the path structural feature information subset in the previous interval and the next interval of the blank interval to generate modified structural feature information, and obtain the corrected three-dimensional image structural feature information set and the corrected path structural feature information set.

[0067] Specifically, a result that does not meet a preset structural feature error threshold is obtained. If it does not meet the threshold, it means that the errors between the three-dimensional image structural feature information subset and the path structural feature information subset in the interval are large and cannot be fused. The three-dimensional image structural feature information subset and the path structural feature information subset in the interval are deleted and used as a blank interval.

[0068] Linear interpolation refers to the interpolation method in which the interpolation function is a first-order polynomial. The interpolation error at the interpolation node is zero. The geometric meaning of linear interpolation is to use a straight line passing through point A and point B to approximate the original function. Linear interpolation can be used to approximately replace the original function, and can also be used to calculate values ​​that are not in the table during the table lookup process.

[0069] For example, the coordinates of the previous interval are set to A(x0,y0,z0), and the coordinates of the next interval are set to B(x1,y1,z1). Based on the known coordinates A(x0,y0,z0) and B(x1,y1,z1), we need to get [x0,x1],

[0070] The value of C(x,y,z) at a certain position in the interval [y0,y1] and [z0,z1] on the straight line is (x-x0) / (y-y0) / (z-z0)=(x1-x0) / (y1-y0) / (z1-z0). According to the formula, when any two of x, y, and z are determined, the value of the other item can be obtained, and C(x,y,z) can be obtained.

[0071] For obtaining any two of x, y, and z, for example, obtain x and y, and convert the three-dimensional calculation into a plane calculation, that is, in the xy plane, the coordinates of the previous interval are known to be A1(x0, y0), and the coordinates of the next interval are B1(x1, y1). To obtain the value of C1(x, y) at a certain position in the interval [x0, x1] and [y0, y1] on the straight line, we can get (x-x0) / (y-y0)=(x1-x0) / (y1-y0). For any x, the value of y can be obtained by calculation, and conversely, for any y, the value of x can also be obtained by calculation, so as to obtain C1(x, y).

[0072] The obtained C(x, y, z) is used as the modified structural feature information, and the modified structural feature information is used as the corrected three-dimensional image structural feature information set and the corrected path structural feature information set.

[0073] Step S700: Based on the corrected three-dimensional image structure feature information set and the corrected path structure feature information set, three-dimensional image fusion construction and rendering are performed to obtain a fused medical image.

[0074] Specifically, the coordinate set in the corrected three-dimensional image structure feature information set and the coordinate set in the corrected path structure feature information set are input into a three-dimensional rectangular coordinate system, and three-dimensional image fusion construction and rendering are performed to obtain a fused medical image, thereby realizing the fusion of three-dimensional real-life medical images and path maps, thereby accurately displaying three-dimensional stereo images, allowing doctors to obtain more comprehensive and accurate organ structure and morphological information, thereby achieving the technical effect of improving the judgment effect of medical images.

[0075] In summary, the method and system for fusing 3D real-life medical images and pathmaps provided in the embodiments of the present application have the following technical effects:

[0076] Collect multiple types of medical images in the target area, obtain a medical image set, perform grayscale processing, and classify according to a preset threshold to obtain an image structure feature information set, obtain a three-dimensional image structure feature information set, collect a path map in the target area, and perform grayscale and threshold classification processing to obtain a path structure feature information set of the path map, divide and compare, judge whether it meets the preset structure feature error threshold, obtain multiple groups of judgment results, and in the case of a negative judgment result, delete the corresponding structure feature information, generate modified structure feature information based on the interpolation method, obtain a corrected three-dimensional image structure feature information set and a corrected path structure feature information set, perform three-dimensional image fusion construction and rendering, and obtain a fused medical image. The invention solves the technical problem that the medical imaging equipment in the prior art can only collect and present two-dimensional plane information, so that the doctor needs to adjust the viewing angle to obtain more angle size information, resulting in poor judgment effect on the medical image, realizes the fusion of three-dimensional real-life medical images and path maps, thereby accurately displaying three-dimensional stereo images, and then allowing doctors to obtain more comprehensive and accurate organ structure morphology information, achieving the technical effect of improving the judgment effect of medical images.

[0077] Embodiment 2

[0078] Based on the same inventive concept as the method for fusing a three-dimensional real-life medical image with a path map in the aforementioned embodiment, Figure 4 As shown, the present application provides a three-dimensional real-life medical image and path map fusion system, the system comprising:

[0079] A medical image acquisition module 10, wherein the medical image acquisition module 10 is used to acquire multiple types of medical images in a target area to obtain a medical image set;

[0080] A grayscale processing module 20, the grayscale processing module 20 is used to perform grayscale processing on the medical image set, and classify the grayscale processed medical image set according to a preset threshold value to obtain an image structure feature information set;

[0081] A model analysis module 30, wherein the model analysis module 30 is used to input the image structure feature information set into a three-dimensional medical image analysis model to obtain a three-dimensional image structure feature information set;

[0082] A path map acquisition module 40, the path map acquisition module 40 is used to collect the path map in the target area, and perform grayscale and threshold classification processing to obtain a path structure feature information set of the path map;

[0083] A data division module 50, the data division module 50 is used to divide and compare the data in the three-dimensional image structure feature information set and the path structure feature information set, determine whether a preset structure feature error threshold is met, and obtain multiple groups of determination results;

[0084] The information correction module 60 is used to delete the corresponding structural feature information when the judgment result is negative, and generate modified structural feature information based on the interpolation method within the structural feature error threshold to obtain a corrected three-dimensional image structural feature information set and a corrected path structural feature information set;

[0085] The fused medical image acquisition module 70 is used to perform three-dimensional image fusion construction and rendering based on the corrected three-dimensional image structure feature information set and the corrected path structure feature information set to obtain a fused medical image.

[0086] Furthermore, the system also includes:

[0087] An image acquisition module, used to collect X-ray images, CT images and MRI images within the target area;

[0088] The image set acquisition module is used to obtain the medical image set based on the X-ray image, CT image and MRI image.

[0089] Furthermore, the system also includes:

[0090] An image processing module, used for performing grayscale processing on the medical images in the medical image set to obtain a grayscale image set;

[0091] A preset threshold value acquisition module, used to acquire the preset threshold value according to the grayscale medical image of the same area of ​​the target area within a historical time;

[0092] The classification module is used to classify the medical images in the grayscale image set using the preset threshold value to obtain the image structure feature information set and the background information set.

[0093] Furthermore, the system also includes:

[0094] A sample acquisition module, used to acquire a plurality of sample image structure feature information sets of the same area of ​​the target area;

[0095] A three-dimensional medical image construction module, used to construct a three-dimensional medical image based on the plurality of sample image structure feature information sets, and obtain a plurality of sample three-dimensional image structure feature information sets;

[0096] A construction data acquisition module is used to construct the three-dimensional medical image analysis model by using the plurality of sample image structure feature information sets and the plurality of sample three-dimensional image structure feature information sets as construction data;

[0097] The structural feature information set acquisition module is used to input the image structural feature information set into the three-dimensional medical image analysis model to obtain the three-dimensional image structural feature information set.

[0098] Furthermore, the system also includes:

[0099] A data labeling and division module, used to label and divide the plurality of sample image structure feature information sets and the plurality of sample three-dimensional image structure feature information sets to obtain a training set, a validation set and a test set;

[0100] A network structure construction module, used to construct the network structure of the three-dimensional medical image analysis model based on the BP neural network, the input data of the three-dimensional medical image analysis model is the image structure feature information set, and the output data is the three-dimensional image structure feature information set;

[0101] A model training module, used to use the training set to perform iterative supervised training on the three-dimensional medical image analysis model, and update model parameters by gradient descent method until the three-dimensional medical image analysis model converges or the accuracy reaches a preset requirement;

[0102] The analysis model acquisition module is used to verify and test the three-dimensional medical image analysis model using the verification set and the test set to obtain the three-dimensional medical image analysis model whose accuracy meets the preset requirements.

[0103] Furthermore, the system also includes:

[0104] An interval acquisition module, used to divide the data in the three-dimensional image structure feature information set and the path structure feature information set according to the same division distance to obtain multiple intervals, each interval including a corresponding three-dimensional image structure feature information subset and a path structure feature information subset;

[0105] An error threshold acquisition module, used for setting and obtaining the preset structural feature error threshold according to the three-dimensional medical images of the same area of ​​the target area in the historical time;

[0106] The information subset judgment module is used to judge whether the three-dimensional image structure feature information subset and the path structure feature information subset in each interval meet the preset structure feature error threshold according to the multiple intervals, and obtain the multiple groups of judgment results.

[0107] Furthermore, the system also includes:

[0108] An information subset deletion module is used to delete the three-dimensional image structure feature information subset and the path structure feature information subset in the corresponding interval as a blank interval when the judgment result is negative;

[0109] The structural feature information generation module is used to perform linear interpolation of the structural feature information based on the three-dimensional image structural feature information subset and the path structural feature information subset in the previous interval and the next interval of the blank interval, generate modified structural feature information, and obtain the corrected three-dimensional image structural feature information set and the corrected path structural feature information set.

[0110] Through the above detailed description of a method for fusing a three-dimensional real-scene medical image with a path map, those skilled in the art can clearly understand a method and system for fusing a three-dimensional real-scene medical image with a path map in this embodiment. As for the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0111] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for fusing three-dimensional real-life medical images with a path map, characterized in that: The method comprises: Collect multiple types of medical images in the target area to obtain a medical image set; grayscale the medical image set, and classify the grayscaled medical image set according to a preset threshold to obtain an image structure feature information set; Inputting the image structure feature information set into a three-dimensional medical image analysis model to obtain a three-dimensional image structure feature information set; Collecting a path map within the target area, and performing grayscale conversion and threshold classification processing to obtain a path structure feature information set of the path map; Dividing and comparing the data in the three-dimensional image structure feature information set and the path structure feature information set to determine whether a preset structure feature error threshold is met, and obtaining multiple groups of determination results; If the judgment result is no, the corresponding structural feature information is deleted, and within the structural feature error threshold, based on the interpolation method, the modified structural feature information is generated to obtain a corrected three-dimensional image structural feature information set and a corrected path structural feature information set; Based on the corrected three-dimensional image structure feature information set and the corrected path structure feature information set, three-dimensional image fusion construction and rendering are performed to obtain a fused medical image.

2. The method according to claim 1, characterized in that Collect multiple types of medical images in the target area to obtain a medical image collection, including: Acquiring X-ray images, CT images and MRI images within the target area; The medical image set is obtained based on the X-ray image, the CT image and the MRI image.

3. The method according to claim 1, characterized in that Grayscale processing is performed on the medical image set, and the grayscale processed medical image set is classified according to a preset threshold value to obtain an image structure feature information set, including: Performing grayscale processing on the medical images in the medical image set to obtain a grayscale image set; Obtaining the preset threshold value based on the grayscale medical image of the same area of ​​the target area in the historical time; The preset threshold is used to classify the medical images in the grayscale image set to obtain the image structure feature information set and the background information set.

4. The method according to claim 1, characterized in that: Inputting the image structure feature information set into the three-dimensional medical image analysis model to obtain the three-dimensional image structure feature information set includes: Acquire a plurality of sample image structure feature information sets of the same area of ​​the target area; Based on the multiple sample image structure feature information sets, construct a three-dimensional medical image to obtain multiple sample three-dimensional image structure feature information sets; Using the plurality of sample image structure feature information sets and the plurality of sample three-dimensional image structure feature information sets as construction data, constructing the three-dimensional medical image analysis model; The image structure feature information set is input into the three-dimensional medical image analysis model to obtain the three-dimensional image structure feature information set.

5. The method according to claim 4, characterized in that Using the plurality of sample image structure feature information sets and the plurality of sample three-dimensional image structure feature information sets as construction data to construct the three-dimensional medical image analysis model, including: Performing data annotation and division on the plurality of sample image structure feature information sets and the plurality of sample three-dimensional image structure feature information sets to obtain a training set, a validation set, and a test set; Based on the BP neural network, a network structure of the three-dimensional medical image analysis model is constructed, wherein the input data of the three-dimensional medical image analysis model is an image structure feature information set, and the output data is a three-dimensional image structure feature information set; Using the training set, performing iterative supervised training on the three-dimensional medical image analysis model, and updating model parameters by gradient descent method until the three-dimensional medical image analysis model converges or the accuracy reaches a preset requirement; The three-dimensional medical image analysis model is verified and tested using the verification set and the test set to obtain the three-dimensional medical image analysis model whose accuracy meets preset requirements.

6. The method according to claim 1, characterized in that The data in the three-dimensional image structure feature information set and the path structure feature information set are divided and compared to determine whether a preset structure feature error threshold is met, and multiple groups of determination results are obtained, including: According to the same division distance, the data in the three-dimensional image structure feature information set and the path structure feature information set are divided to obtain a plurality of intervals, each interval including a corresponding three-dimensional image structure feature information subset and a path structure feature information subset; According to the three-dimensional medical images of the same area of ​​the target area in the historical time, setting the error threshold of obtaining the preset structural feature; According to the multiple intervals, the three-dimensional image structure feature information subset and the path structure feature information subset in each interval are judged one by one whether they meet the preset structure feature error threshold, so as to obtain the multiple groups of judgment results.

7. The method according to claim 6, characterized in that If the judgment result is no, the corresponding structural feature information is deleted, and within the structural feature error threshold, the modified structural feature information is generated based on the interpolation method, including: If the judgment result is negative, the three-dimensional image structure feature information subset and the path structure feature information subset in the corresponding interval are deleted to form a blank interval; According to the three-dimensional image structure feature information subset and the path structure feature information subset in the previous interval and the next interval of the blank interval, linear interpolation of the structure feature information is performed to generate modified structure feature information, and the corrected three-dimensional image structure feature information set and the corrected path structure feature information set are obtained.

8. A three-dimensional real-life medical image and path map fusion system, characterized in that: The system comprises: A medical image acquisition module, wherein the medical image acquisition module is used to acquire multiple types of medical images in a target area to obtain a medical image set; A grayscale processing module, the grayscale processing module is used to perform grayscale processing on the medical image set, and classify the grayscale processed medical image set according to a preset threshold value to obtain an image structure feature information set; A model analysis module, wherein the model analysis module is used to input the image structure feature information set into a three-dimensional medical image analysis model to obtain a three-dimensional image structure feature information set; A path map acquisition module, the path map acquisition module is used to collect the path map in the target area, and perform grayscale and threshold classification processing to obtain a path structure feature information set of the path map; A data division module, the data division module is used to divide and compare the data in the three-dimensional image structure feature information set and the path structure feature information set, determine whether a preset structure feature error threshold is met, and obtain multiple groups of determination results; An information correction module, wherein the information correction module is used to delete the corresponding structural feature information when the judgment result is negative, and generate modified structural feature information within the structural feature error threshold based on an interpolation method to obtain a corrected three-dimensional image structural feature information set and a corrected path structural feature information set; A fused medical image acquisition module is used to perform three-dimensional image fusion construction and rendering based on the corrected three-dimensional image structure feature information set and the corrected path structure feature information set to obtain a fused medical image.

Images