A cerebral hemorrhage medical image fusion method and system

Abstract

The application relates to a cerebral hemorrhage medical image fusion method and system. The method utilizes the feature that the brain structure has a certain symmetry, equally divides the brain structures in a target brain image and a reference brain image into two subimages, and when a first feature point and a second feature point are matched, a third feature point and a fourth feature point in another subimage can be used to assist the first feature point and the second feature point in matching. Moreover, the matching process of any pair of feature points is that a plurality of neighbor feature points of one feature point are mapped into the space of another feature point, then the mapping difference values are calculated, and then under the condition that the two difference values are less than a set value, it is judged that the first feature point and the second feature point are matched successfully. The method can improve the matching accuracy of the feature points and finally improve the image fusion effect.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of computer vision, and particularly relate to a brain hemorrhage medical image fusion method and system. BACKGROUND

[0002] Fusing medical images of different domains (such as CT images collected by different models of devices) helps to improve the accuracy of diagnosis. However, images collected by different models of devices can have differences (such as different configuration parameters, different shooting angles leading to feature mutations), so how to improve the effect of medical image fusion is the focus of current research.

[0003] The feature point matching process is the basis for realizing medical image fusion, that is, feature point matching is the process of aligning images of the same object obtained under different conditions, and its essence is to find a set of spatial geometric transformation relationships, so that all feature points on one image can be obtained through the transformation of another image. At present, feature point matching mainly relies on similarity matching, that is, mainly using the Euclidean distance to calculate the similarity between a set of feature points to be matched, but ignoring the problem that the similarity calculation is not accurate due to the differences between images of different domains caused by the differences in collection devices, thereby reducing the matching accuracy. SUMMARY

[0004] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0005] The main purpose of the embodiments of the present application is to propose a brain hemorrhage medical image fusion method and system, which can improve the effect of image fusion.

[0006] To achieve the above purpose, a first aspect of the embodiments of the present application provides a brain hemorrhage medical image fusion method, the method comprising:

[0007] Obtaining a target brain image and a reference brain image of a target patient, and according to the symmetry of the brain structure, equally dividing the target brain image into two target sub-images and equally dividing the reference brain image into two reference sub-images; wherein the target brain image and the reference brain image are brain hemorrhage medical images of different domains;

[0008] Extracting a plurality of feature points from the target brain image and the reference brain image respectively, and matching the plurality of feature points in the target brain image and the plurality of feature points in the reference brain image; wherein the matching process between a first feature point in the target brain image and a second feature point in the reference brain image comprises:

[0009] determining a third feature point and a fourth feature point; wherein the third feature point is a feature point which is in a different target sub-image from the first feature point and is symmetric to the first feature point in a brain structure; and the fourth feature point is a feature point which is in a different reference sub-image from the second feature point and is symmetric to the second feature point in a brain structure;

[0010] selecting a plurality of neighbor feature points corresponding to the first feature point to the fourth feature point respectively;

[0011] calculating a first mapping difference between the plurality of neighbor feature points corresponding to the first feature point and the plurality of neighbor feature points corresponding to the second feature point, and calculating a second mapping difference between the plurality of neighbor feature points corresponding to the third feature point and the plurality of neighbor feature points corresponding to the fourth feature point;

[0012] based on a weighted sum of the first mapping difference and the second mapping difference, and in a case where the sum is less than a first threshold value, determining that the first feature point and the second feature point are matched successfully;

[0013] fusing the target brain image and the reference brain image of the target patient according to all the first feature points and the second feature points which are matched successfully.

[0014] The brain hemorrhage medical image fusion method provided by the present application has at least the following beneficial effects:

[0015] Since different domain images come from different acquisition devices, there may be differences between different domain images for the same target, which causes the matching process in fusion to be inaccurate. The present application takes advantage of the symmetry of brain structure, and equally divides the brain structure in the target brain image and the reference brain image into two sub-images. Then, when the first feature point and the second feature point are matched, considering that the third feature point symmetrical to the first feature point on the brain structure has strong similarity with the first feature point, the fourth feature point symmetrical to the second feature point on the brain structure has strong similarity with the second feature point, and the domain distribution characteristics between them are invariant, the matching result of the symmetrical third feature point and the symmetrical fourth feature point can be used to assist the matching of the first feature point and the second feature point, thereby improving the accuracy of the matching of the first feature point and the second feature point, and ultimately improving the effect of image fusion. In the present application, when a group of feature points are matched, a plurality of neighbor feature points in the domain space where one feature point is located are mapped into the domain space where another feature point is located, then the mapping difference between the plurality of neighbor feature points is calculated, and then based on the mapping difference of the group of feature points and the assistance of the mapping difference of the symmetrical group of feature points, it is judged whether the two difference values are less than a set value, and in the case of less than the set value, it is judged that the first feature point and the second feature point are matched successfully. The mapping difference between the neighbors near the feature points can be used for stability judgment, which can improve the accuracy of the matching of the first feature point and the second feature point, and ultimately improve the effect of image fusion.

[0016] In some embodiments, the extracting a plurality of feature points from the target brain image and the reference brain image respectively comprises:

[0017] extracting a plurality of initial pixel points as corner points from the target brain image and the reference brain image respectively;

[0018] calculating HOG descriptors of the initial pixel points, and extracting the plurality of feature points from the plurality of initial pixel points based on the HOG descriptors; wherein the HOG descriptor comprises position features and contour features of the initial pixel points, and the contour features are gradient representations based on the position features.

[0019] In some embodiments, the process of extracting a plurality of initial pixel points as corner points from the target brain image comprises:

[0020] extracting all corner points from the target brain image;

[0021] dividing the whole region of the target brain image into four sub-regions with the geometric center of the target brain image as the center, and calculating the number of corner points in each sub-region;

[0022] In a case where the number of corner points in the sub-region is greater than a second threshold, the sub-region is further divided into four sub-regions with the geometric center in the sub-region as the center; in a case where the number of corner points in the sub-region is less than the second threshold, a corner point with the highest Harris value is selected from the corner points in the sub-region as the initial pixel point;

[0023] By analogy, until the sub-region stops being divided, and the initial pixel point corresponding to each sub-region is obtained;

[0024] The process of extracting a plurality of initial pixel points as corner points from the reference brain image comprises:

[0025] All corner points are extracted from the reference brain image;

[0026] The overall region of the reference brain image is divided into four sub-regions with the geometric center of the reference brain image as the center, and the number of corner points in each sub-region is calculated;

[0027] In a case where the number of corner points in the sub-region is greater than a third threshold, the sub-region is further divided into four sub-regions with the geometric center in the sub-region as the center; in a case where the number of corner points in the sub-region is less than the second threshold, a corner point with the highest Harris value is selected from the corner points in the sub-region as the initial pixel point;

[0028] By analogy, until the sub-region stops being divided, and the initial pixel point corresponding to each sub-region is obtained.

[0029] In some embodiments, the calculation of the first mapping difference value between the plurality of neighbor feature points corresponding to the first feature point and the plurality of neighbor feature points corresponding to the second feature point comprises:

[0030] From the HOG descriptor, a first contour feature corresponding to the first feature point and a second contour feature corresponding to the second feature point are determined;

[0031] According to the first contour feature and the second contour feature, an intermediate feature between the first feature point and the second feature point is calculated;

[0032] According to the mapped intermediate feature, the plurality of neighbor feature points corresponding to the first feature point are mapped into the neighborhood of the plurality of neighbor feature points corresponding to the second feature point;

[0033] In the neighborhood of the plurality of neighbor feature points corresponding to the second feature point, each pair of the neighbor feature points between the first feature point and the second feature point is determined, and a difference value between each pair of the neighbor feature points is calculated;

[0034] According to the difference values between pairs of the neighbor feature points between the first feature point and the second feature point, the first mapping difference value is calculated.

[0035] In some embodiments, the calculating the first mapping difference value according to the difference values between pairs of the neighbor feature points between the first feature point and the second feature point comprises:

[0036] An exponential function of a pair of the neighbor feature points is constructed with base and exponent being a product of a hyperparameter and the difference value;

[0037] The exponential functions of pairs of the neighbor feature points are summed to obtain the first mapping difference value.

[0038] In some embodiments, the calculating the first mapping difference value between the neighbor feature points corresponding to the third feature point and the neighbor feature points corresponding to the fourth feature point comprises:

[0039] From the HOG descriptor, a third contour feature corresponding to the third feature point and a fourth contour feature corresponding to the fourth feature point are determined;

[0040] According to the third contour feature and the fourth contour feature, an intermediate feature between the third feature point and the fourth feature point is calculated;

[0041] According to the mapping intermediate feature, the neighbor feature points corresponding to the third feature point are mapped into a neighborhood of the neighbor feature points corresponding to the fourth feature point;

[0042] In the neighborhood of the neighbor feature points corresponding to the fourth feature point, each pair of the neighbor feature points between the third feature point and the fourth feature point is determined, and a difference value between each pair of the neighbor feature points is calculated;

[0043] According to the difference values between pairs of the neighbor feature points between the third feature point and the fourth feature point, the second mapping difference value is calculated.

[0044] In some embodiments, the calculating the second mapping difference value according to the difference values between pairs of the neighbor feature points between the third feature point and the fourth feature point comprises:

[0045] An exponential function of a pair of the neighbor feature points is constructed with base and exponent being a product of a hyperparameter and the difference value;

[0046] The exponential functions of pairs of the neighbor feature points are summed to obtain the second mapping difference value.

[0047] ​​To achieve the above object, a second aspect of the embodiment of the present application provides a cerebral hemorrhage medical image fusion system, the system comprises:

[0048] An image acquisition module is configured to acquire a target brain image and a reference brain image of a target patient, divide a brain structure in the target brain image into two parts to obtain a first target sub-image and a second target sub-image, and divide a brain structure in the reference brain image into two parts to obtain a first reference sub-image and a second reference sub-image; wherein the target brain image and the reference brain image are cerebral hemorrhage medical images in different domains;

[0049] A feature point processing module is configured to extract a plurality of feature points from the target brain image and the reference brain image respectively, and match the plurality of feature points in the target brain image with the plurality of feature points in the reference brain image; wherein a matching process between a first feature point in the target brain image and a second feature point in the reference brain image comprises:

[0050] determining a third feature point and a fourth feature point; wherein the third feature point is a feature point that is in a different target brain image from the first feature point and is symmetrical to the first feature point, and the fourth feature point is a feature point that is in a different reference brain image from the second feature point and is symmetrical to the second feature point;

[0051] selecting K neighbor feature points corresponding to the first feature point, the second feature point, the third feature point and the fourth feature point respectively;

[0052] calculating a first mapping difference value between the K neighbor feature points corresponding to the first feature point and the K neighbor feature points corresponding to the second feature point, and calculating a second mapping difference value between the K neighbor feature points corresponding to the third feature point and the K neighbor feature points corresponding to the fourth feature point;

[0053] based on weighted summation between the first mapping difference value and the second mapping difference value, and in the case that the summation result is less than a first threshold value, determining that the first feature point and the second feature point are matched successfully;

[0054] An image fusion module is configured to fuse the target brain image and the reference brain image of the target patient according to all the first feature points and the second feature points that are matched successfully.

[0055] To achieve the above object, a third aspect of the embodiment of the present application provides an electronic device, comprising: at least one control processor and a memory connected with the at least one control processor in communication; the memory stores instructions executable by the at least one control processor, and the instructions are executed by the at least one control processor to enable the at least one control processor to execute the brain hemorrhage medical image fusion method described above.

[0056] To achieve the above object, a fourth aspect of the embodiment of the present application provides a computer readable storage medium, which stores computer executable instructions, and the computer executable instructions are used to make a computer execute the brain hemorrhage medical image fusion method described above.

[0057] It can be understood that the beneficial effects of the second aspect to the fourth aspect and the related technology compared with the beneficial effects of the first aspect and the related technology compared with the first aspect are the same, which can be referred to the related description in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or related technical description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0059] Figure 1 is a schematic diagram of the embodiment of the brain CT image provided by the present application;

[0060] Figure 2 is a flowchart of the embodiment of the brain hemorrhage medical image fusion method provided by the present application;

[0061] Figure 3 is a structural schematic diagram of the embodiment of the brain hemorrhage medical image fusion system provided by the present application;

[0062] Figure 4 is a schematic diagram of the embodiment of the electronic device provided by the present application. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical scheme and advantages of the present application more clear, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0064] In order to solve the technical defects, such as Figure 2 An embodiment of the present application provides a brain hemorrhage medical image fusion method, which comprises:

[0065] In step S110, the target brain image and the reference brain image of a target patient are obtained, and the target brain image is equally divided into two target sub-images according to the symmetry of the brain structure, and the reference brain image is equally divided into two reference sub-images.

[0066] The target patient refers to a patient suffering from cerebral hemorrhage.

[0067] The target brain image and the reference brain image are different domain images, for example, the reference brain image and the target brain image are CT images collected by different imaging devices.

[0068] The purpose of the embodiment is to find feature points matched between the reference brain image and the target brain image, and then to realize the fusion between the reference brain image and the target brain image of different domains by using the matched feature points, so as to facilitate the subsequent medical diagnosis or disease treatment of the patient by the doctor.

[0069] As Figure 1 Since the brain structure has a certain symmetry, the brain structure in the image can be divided into two parts according to the central axis (set according to the doctor's experience), that is, the reference brain image or the target brain image is divided into two sub-images.

[0070] When the feature points in the first target sub-image of the target brain image are matched with the feature points in the first reference sub-image of the reference brain image, the feature points in the second target sub-image of the target brain image and the feature points in the second reference sub-image of the reference brain image can be found, and the matched feature points are used to assist the matching process of the feature points to improve the matching accuracy of the feature points.

[0071] In step S120, a plurality of feature points are extracted from the target brain image and the reference brain image, and the plurality of feature points in the target brain image and the plurality of feature points in the reference brain image are matched.

[0072] Suppose the first feature point is a feature point in the target brain image, and the second feature point is a feature point in the reference brain image. Then, the matching process between the first feature point in the target brain image and the second feature point in the reference brain image includes:

[0073] (1) determining a third feature point and a fourth feature point.

[0074] The third feature point is a feature point that is in a different target sub-image from the first feature point and is symmetrical to the first feature point on the brain structure. Assuming that the first feature point is located in the first target sub-image of the target brain image, the third feature point is located in the second target sub-image of the target brain image.

[0075] The fourth feature point is a feature point which is in a different reference sub-image from the second feature point and is symmetrical to the second feature point on the brain structure. As the first feature point and the third feature point are consistent, no further description is given here.

[0076] (2) A plurality of neighbor feature points corresponding to the first feature point, the second feature point, the third feature point and the fourth feature point are selected respectively.

[0077] Among them, there are many ways to select neighbor feature points in the neighborhood, such as Euclidean distance calculation, etc. For details, see the introduction of the subsequent embodiments.

[0078] (3) The first mapping difference between the plurality of neighbor feature points corresponding to the first feature point and the plurality of neighbor feature points corresponding to the second feature point is calculated, and the second mapping difference between the plurality of neighbor feature points corresponding to the third feature point and the plurality of neighbor feature points corresponding to the fourth feature point is calculated.

[0079] The plurality of neighbor feature points corresponding to the first feature point in one domain image are mapped to another domain, so as to calculate the difference between the plurality of neighbor feature points corresponding to the first feature point and the second feature point in one domain space (for details, see the subsequent embodiments), and then judge whether the first feature point and the second feature point are similar based on the size of the difference. Since the distribution of neighbor feature points in each domain space has domain correlation, for example, the difference between a pair of neighbor feature points is small, which means that the pair of feature points are similar, and it can be considered that the two are matched.

[0080] Due to the symmetry of the brain structure, the third feature point and the fourth feature point are also processed in a similar manner, which will not be described here.

[0081] (4) The first mapping difference and the second mapping difference are weighted and summed, and in the case that the sum is less than the first threshold, it is judged that the first feature point and the second feature point are matched successfully. Here, a lower weight, such as 0.3, can be set for the second mapping difference, and a higher weight, such as 0.7, can be set for the first mapping difference. The first threshold can be set based on experience.

[0082] Step S130, according to all the first feature points and the second feature points matched successfully, the target brain image of the target patient and the reference brain image are fused.

[0083] It should be noted that after finding the matching feature points in the two domains, the matching feature points can be used to realize the fusion between the images. Since the fusion here is common knowledge to those skilled in the art, and the method does not involve subsequent fusion process, no further description is given here.

[0084] The method has at least the following beneficial effects:

[0085] Since different domain images come from different acquisition devices, there may be differences between different domain images for the same target, which causes the matching process in fusion to be inaccurate. The present application takes advantage of the symmetry of brain structure, and equally divides the brain structure in the target brain image and the reference brain image into two sub-images. Then, when the first feature point and the second feature point are matched, considering that the third feature point symmetrical to the first feature point on the brain structure has strong similarity with the first feature point, the fourth feature point symmetrical to the second feature point on the brain structure has strong similarity with the second feature point, and the domain distribution characteristics between them are invariant, the matching result of the symmetrical third feature point and the symmetrical fourth feature point can be used to assist the matching of the first feature point and the second feature point, thereby improving the accuracy of the matching of the first feature point and the second feature point, and ultimately improving the effect of image fusion.

[0086] In the present application, when matching a group of feature points, a plurality of neighbor feature points in the domain space where one feature point is located are mapped into the domain space where another feature point is located, and then the mapping difference between the plurality of neighbor feature points is calculated. Based on the mapping difference of the group of feature points and the auxiliary of the mapping difference of the symmetrical group of feature points, it is judged whether the two differences are less than a set value, and in the case of less than the set value, it is judged that the first feature point and the second feature point are matched successfully. The stability judgment by the mapping difference between the neighbors near the feature points can improve the accuracy of the matching of the first feature point and the second feature point, and ultimately improve the effect of image fusion.

[0087] Further, the step S120 of extracting a plurality of feature points from the target brain image and the reference brain image respectively includes steps S210 to S220.

[0088] Step S210, a plurality of initial pixel points as corner points are extracted from the target brain image and the reference brain image respectively.

[0089] In the present embodiment, the corner point detection method can be used to judge whether a pixel point in the target brain image and the reference brain image is a corner point. If the pixel point is a corner point, the pixel point as a corner point is taken as an initial pixel point.

[0090] Step S220, the HOG descriptor of the initial pixel point is calculated, and a plurality of feature points are extracted from the plurality of initial pixel points based on the HOG descriptor.

[0091] The HOG descriptor includes the position feature and the contour feature of the initial pixel point, and the contour feature is represented based on the gradient of the position feature. The HOG describes the contour of the target position by counting the gradient direction histogram of the local sub-image of the target position, and the essence is the statistics of the edge gradient information.

[0092] Further, the process of extracting the plurality of initial pixel points as corner points from the target brain image in step S210 includes the following steps S2110 to S2140.

[0093] Step S2110, extract all corner points from the target brain image;

[0094] Step S2120, divide the whole region of the target brain image into four sub-regions with the geometric center of the target brain image as the center, and calculate the number of corner points in each sub-region;

[0095] Step S2130, if the number of corner points in the sub-region is greater than the second threshold, further divide the sub-region into four sub-regions with the geometric center of the sub-region as the center; if the number of corner points in the sub-region is less than the second threshold, select the corner point with the highest Harris value from the corner points in the sub-region as the initial pixel point;

[0096] Step S2140, by analogy, until the sub-region stops dividing, and the initial pixel point corresponding to each sub-region is obtained.

[0097] The process of extracting the plurality of initial pixel points as corner points from the reference brain image in step S210 includes the following steps S2150 to S2180.

[0098] Step S2150, extract all corner points from the reference brain image;

[0099] Step S2160, divide the whole region of the reference brain image into four sub-regions with the geometric center of the reference brain image as the center, and calculate the number of corner points in each sub-region;

[0100] Step S2170, if the number of corner points in the sub-region is greater than the third threshold, further divide the sub-region into four sub-regions with the geometric center of the sub-region as the center; if the number of corner points in the sub-region is less than the second threshold, select the corner point with the highest Harris value from the corner points in the sub-region as the initial pixel point;

[0101] Step S2180, by analogy, until the sub-region stops dividing, and the initial pixel point corresponding to each sub-region is obtained.

[0102] Taking steps S2110 to S2140 as an example, in step S2120, the geometric center refers to the centroid position of the image region, which can be specifically determined by calculating the mean value of the image boundary coordinates, and is used as a reference point for dividing the region. The division into four sub-regions refers to dividing the original region into four equal parts according to the horizontal and vertical center lines, for example, dividing the image into four regions of upper left, upper right, lower left, and lower right. The second threshold (which can be based on experience) refers to a preset upper limit of the number of initial pixel points of the region, which is used to judge whether to continue to subdivide the region. The selection of the initial pixel points is performed by comparing the Harris corner point response values of each point in the region, and the pixel point corresponding to the maximum value is retained as the initial pixel point.

[0103] In this embodiment, the selection range is dynamically adjusted by recursively dividing the region. When a certain sub-region contains too many corner points, it indicates that the region may have dense or redundant features, and needs to be further subdivided to improve the selection accuracy. When the number of corner points in a sub-region is less than a threshold, the point with the highest Harris value in the region is directly selected as the representative. For example, after the initial image is divided into four sub-regions, if one of the sub-regions has more than a certain threshold (for example, 8) of corner points, the sub-region is again divided into four equal parts, and this process is repeated until the number of corner points in all sub-regions does not exceed the threshold. Finally, only one corner point with the highest Harris value is retained in each sub-region as the initial pixel point.

[0104] The traditional method usually performs global selection on all initial pixel points, resulting in redundant features not being effectively removed. However, the present method controls the selection granularity by dynamically dividing the region, retains the most representative feature points in each sub-region, and reduces interference in the subsequent matching process.

[0105] In this embodiment, only the feature point with the highest Harris value in each sub-image is selected in each sub-region, and other feature points are deleted, so that the feature points that can better reflect the information of the entire image and are uniformly distributed are finally selected from a large number of feature points.

[0106] Further, the step S120 of calculating the first mapping difference between the plurality of neighbor feature points corresponding to the first feature point and the plurality of neighbor feature points corresponding to the second feature point includes the following steps S310 to S350:

[0107] Step S310, determining a first contour feature corresponding to the first feature point and a second contour feature corresponding to the second feature point from the HOG descriptor;

[0108] Step S320, calculating an intermediate feature between the first feature point and the second feature point according to the first contour feature and the second contour feature;

[0109] Step S330, according to the intermediate feature, mapping the plurality of neighbor feature points corresponding to the first feature point to the plurality of neighbor feature points corresponding to the second feature point in the neighborhood of the plurality of neighbor feature points corresponding to the second feature point;

[0110] Step S340, in the neighborhood of the plurality of neighbor feature points corresponding to the second feature point, determining each pair of neighbor feature points between the first feature point and the second feature point, and calculating the difference value between each pair of neighbor feature points;

[0111] Step S350, according to the difference value of the plurality of pairs of neighbor feature points between the first feature point and the second feature point, calculating the first mapping difference value.

[0112] Another important application point of the present application, the embodiment in step S310, selecting the contour feature description of the first feature point and the contour feature description of the second feature point in the HOG descriptor, and then using the contour feature of the first feature point and the contour feature of the second feature point in the different domain image to construct the intermediate feature, measuring the contour transformation relationship between the first feature point and the second feature point in the local space centered by each other through the intermediate feature, mapping the feature points in one image domain to another image domain space according to the contour transformation relationship, which can greatly reduce the mapping error after mapping and improve the matching accuracy.

[0113] In step S320, the intermediate feature between the first feature point and the second feature point can be obtained by multiplying the inverse matrix of the first contour feature of the first feature point and the second contour feature of the second feature point.

[0114] Then in step S330, the intermediate feature is used as a tool for mapping feature points between domain images, and the neighbor feature points of the first feature point are mapped into the space corresponding to the second feature point according to the intermediate feature.

[0115] In step S340, the neighborhood of the second feature point refers to a circular region constructed with the second feature point as the center and a preset radius, and the feature points in the neighborhood are the neighbor feature points of the second feature point.

[0116] The core of the difference value between each pair of neighbor feature points calculated here is:

[0117] In a mapping space, the difference value between the difference of the first feature point and its corresponding neighbor feature points and the difference of the second feature point and its corresponding neighbor feature points is determined. For example, assuming that the first feature point and the second feature point are and , the kth pair of neighbor feature points is and , and the difference value is , is the intermediate feature, then the calculation process of the difference value between and includes:

[0118] .

[0119] ；

[0120] ；

[0121] In step S350, the first mapping difference value is calculated based on the first mapping difference value calculated by the plurality of neighbor feature points of the first feature point and the plurality of neighbor feature points of the second feature point in the same neighborhood.

[0122] In step S350, the first mapping difference value is calculated according to the difference values between the plurality of pairs of neighbor feature points between the first feature point and the second feature point, including the following steps S3510 to S3520:

[0123] Step S3510, constructing an exponential function of a pair of neighbor feature points with base and exponent being the product of the hyperparameter and the difference value;

[0124] Step S3520, summing the exponential functions of the plurality of pairs of neighbor feature points to obtain the first mapping difference value.

[0125] For example: ;

[0126] wherein, is the difference value between the first pair of neighbor feature points (K is the total number of neighbor feature points), is the hyperparameter, is the exponential function, is the first mapping difference value.

[0127] In this embodiment, the present embodiment finds a plurality of neighbor feature points corresponding to a pair of feature points, and then maps the feature points of different domain images into the space of the same domain image according to the intermediate feature points. If the difference values between these neighbor feature points are small (a threshold value can be set for judgment), it is considered that the two first feature points and the second feature point are relatively similar, and the two feature points are considered to be matched successfully; if are large, it is considered that they are not matched.

[0128] Further, the calculation of the first mapping difference value between the plurality of neighbor feature points corresponding to the third feature point and the plurality of neighbor feature points corresponding to the fourth feature point in step S120 includes the following steps S410 to S450:

[0129] ​Step S410, determining a third contour feature corresponding to the third feature point and a fourth contour feature corresponding to the fourth feature point from the HOG descriptor;

[0130] Step S420, calculating an intermediate feature between the third feature point and the fourth feature point according to the third contour feature and the fourth contour feature.

[0131] Step S430, mapping a plurality of neighbor feature points corresponding to the third feature point into a neighborhood of a plurality of neighbor feature points corresponding to the fourth feature point according to the intermediate feature.

[0132] Step S440, determining each pair of neighbor feature points between the third feature point and the fourth feature point in the neighborhood of the plurality of neighbor feature points corresponding to the fourth feature point, and calculating a difference value between each pair of neighbor feature points.

[0133] Step S450, calculating a second mapping difference value according to the difference values of the plurality of pairs of neighbor feature points between the third feature point and the fourth feature point.

[0134] In step S450, the second mapping difference value is calculated according to the difference values of the plurality of pairs of neighbor feature points between the third feature point and the fourth feature point, including steps S4510 to S4520 as follows:

[0135] Step S4510, constructing an exponential function of a pair of neighbor feature points with the base of and the exponent of the product of the hyperparameter and the difference value;

[0136] Step S4520, summing the exponential functions of the plurality of pairs of neighbor feature points to obtain the second mapping difference value

[0137] The present embodiment is similar to the above-described embodiments, and will not be described in detail here.

[0138] Further, the selecting of the plurality of neighbor feature points corresponding to the first feature point to the fourth feature point in step S120 includes steps S510 to S520 as follows:

[0139] Step S510, calculating a plurality of feature points with similar HOG descriptor similarity to the first target feature point as the neighbor feature points of the first target feature point in the neighborhood range of the first target feature point.

[0140] The first target feature point is the first feature point or the third feature point.

[0141] Based on the above embodiment, taking the first target feature point as the first feature point as an example, the neighborhood range of the first feature point is preset, and then based on the similarity of the HOG descriptor, the similar multiple feature points are found based on the similarity, as the multiple neighbor feature points of the first feature point. These similar neighbor feature points have higher feature similarity with the first feature point. Using the matching result of the multiple neighbor feature points as the matching result of the first feature point can improve the stability of feature point matching.

[0142] In this embodiment, the corresponding neighbor feature points are selected according to the HOG descriptor. The neighbor feature points similar to the feature points can be stably found through the descriptor.

[0143] Step S520, from the neighborhood range of the second target feature point, find multiple feature points similar to the HOG descriptors of the multiple neighbor feature points of the first target feature point, as the multiple neighbor feature points of the second target feature point.

[0144] Wherein, when the first target feature point is the first feature point, the second target feature point is the second feature point, when the first target feature point is the third feature point, the second target feature point is the fourth feature point. Wherein, the neighborhood range of the second target feature point is consistent with the first target feature point, which is to take the corresponding feature as the center of the circle, and draw a circle with the set radius (the radius can be the same). The circle is the neighborhood range of the second target feature point. Then, since the second target feature point is in the same domain image as the first target feature point, it is a symmetrical feature point. The second target feature point is used to assist the matching process of the first target feature point. Therefore, the multiple neighbor feature points corresponding to the second target feature point are based on the HOG descriptor. From the neighborhood corresponding to the second target feature point, multiple neighbor feature points similar to the first target feature point are found to enhance the importance of the matching of the second target feature point, and finally to improve the matching accuracy of the auxiliary first target feature point.

[0145] Referring to Figure 3 , one embodiment provides a cerebral hemorrhage medical image fusion system, the system comprises:

[0146] The image acquisition module 1100 is configured to acquire a target brain image and a reference brain image of a target patient, and divide the target brain image into two target sub-images and divide the reference brain image into two reference sub-images according to the symmetry of the brain structure; wherein, the target brain image and the reference brain image are cerebral hemorrhage medical images of different domains;

[0147] The feature point processing module 1200 is configured to extract a plurality of feature points from the target brain image and the reference brain image respectively, and match the plurality of feature points in the target brain image with the plurality of feature points in the reference brain image; wherein the matching process between the first feature point in the target brain image and the second feature point in the reference brain image comprises:

[0148] determining a third feature point and a fourth feature point; wherein the third feature point is a feature point that is in a different target sub-image from the first feature point and is symmetrical to the first feature point in the brain structure; and the fourth feature point is a feature point that is in a different reference sub-image from the second feature point and is symmetrical to the second feature point in the brain structure;

[0149] selecting a plurality of neighbor feature points corresponding to the first feature point to the fourth feature point respectively;

[0150] calculating a first mapping difference between the plurality of neighbor feature points corresponding to the first feature point and the plurality of neighbor feature points corresponding to the second feature point, and calculating a second mapping difference between the plurality of neighbor feature points corresponding to the third feature point and the plurality of neighbor feature points corresponding to the fourth feature point;

[0151] based on a weighted sum of the first mapping difference and the second mapping difference, and in a case where the sum result is less than a first threshold value, determining that the first feature point and the second feature point are matched successfully;

[0152] The image fusion module 1300 is configured to fuse the target brain image of the target patient and the reference brain image according to all the first feature points and the second feature points that are matched successfully.

[0153] It should be noted that the brain hemorrhage medical image fusion system provided in the embodiment and the brain hemorrhage medical image fusion method described above are based on the same inventive concept, and therefore the related content of the brain hemorrhage medical image fusion method described above is also applicable to the content of the brain hemorrhage medical image fusion system, and therefore, the details are not repeated here.

[0154] As Figure 4 embodiments of the present disclosure also provide an electronic device, and the electronic device comprises:

[0155] at least one memory;

[0156] at least one processor;

[0157] at least one program;

[0158] The program is stored in the memory, and the processor executes the at least one program to implement the brain hemorrhage medical image fusion method described above.

[0159] The electronic device can be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a vehicle-mounted computer, etc.

[0160] The electronic device of the embodiment of the present application is described in detail below.

[0161] The processor 1600 can be implemented in a general-purpose central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute related programs to implement the technical solutions provided by the embodiment of the present application.

[0162] The memory 1700 can be implemented in a read only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1700 can store an operating system and other application programs. When the technical solutions provided by the embodiment of the present application are implemented by software or firmware, the related program codes are stored in the memory 1700 and are called and executed by the processor 1600 to implement the brain hemorrhage medical image fusion method of the embodiment of the present application.

[0163] The input / output interface 1800 is configured to realize information input and output.

[0164] The communication interface 1900 is configured to realize the communication interaction between the device and other devices. The communication can be realized by a wired manner (for example, a USB, a network cable, etc.) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, etc.).

[0165] The bus 2000 is configured to transmit information between various components (for example, the processor 1600, the memory 1700, the input / output interface 1800, and the communication interface 1900) of the device.

[0166] The processor 1600, the memory 1700, the input / output interface 1800, and the communication interface 1900 are connected to each other through the bus 2000 to realize the communication connection between them in the device.

[0167] The embodiment of the present application further provides a storage medium, which is a computer readable storage medium and stores computer executable instructions. The computer executable instructions are configured to make a computer execute the brain hemorrhage medical image fusion method described above.

[0168] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. In addition, the memory can include a high-speed random access memory and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory disposed remotely with respect to the processor, which can be connected to the processor through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and the like.

[0169] The embodiments described in the present application are for more clearly illustrating the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0170] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and can include more or fewer steps than shown in the figures, or some steps or different steps.

[0171] The above-described device embodiments are only schematic, and units described as separate components can or can not be physically separate, i.e., can be located in one place or distributed over multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0172] Those skilled in the art can understand that all or some of the steps in the above disclosed method, the functional modules / units in the system and the device can be implemented as software, firmware, hardware and their appropriate combinations.

[0173] The terms "first", "second", "third", "fourth" and the like used in the description of the present application and the above-described drawings (if any) are used to distinguish similar objects, and do not necessarily have to describe a particular order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to the process, method, product or device.

[0174] It should be understood that, in the application, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" is used to describe the association relationship of the associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases of only A, only B and A and B existing at the same time, wherein A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after it. "At least one of the following" or similar expressions means any combination of these items, including any combination of single item or multiple items. For example, at least one of a, b or c can represent a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b and c can be single or multiple.

[0175] In several embodiments provided in the application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed mutual units can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0176] The units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on multiple network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment scheme.

[0177] In addition, the functional units in each embodiment of the application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0178] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions for causing an electronic device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.

[0179] The above is a specific description of the preferred implementation of the embodiments of the present application, but the embodiments of the present application are not limited to the above implementation. Those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the embodiments of the present application, and these equivalent modifications or replacements are all included in the scope defined by the claims of the embodiments of the present application.

Claims

1. A cerebral hemorrhage medical image fusion method, characterized by, The method comprises: obtaining a target brain image and a reference brain image of a target patient, and dividing the target brain image into two target sub-images and the reference brain image into two reference sub-images according to the symmetry of brain structure; wherein the target brain image and the reference brain image are cerebral hemorrhage medical images in different domains; extracting a plurality of feature points from the target brain image and the reference brain image respectively, and matching the plurality of feature points in the target brain image with the plurality of feature points in the reference brain image; wherein the matching process between a first feature point in the target brain image and a second feature point in the reference brain image comprises: determining a third feature point and a fourth feature point; wherein the third feature point is a feature point that is in a different target sub-image from the first feature point and is symmetric to the first feature point in brain structure, and the fourth feature point is a feature point that is in a different reference sub-image from the second feature point and is symmetric to the second feature point in brain structure; selecting a plurality of neighbor feature points corresponding to the first feature point to the fourth feature point respectively; calculating a first mapping difference value between the plurality of neighbor feature points corresponding to the first feature point and the plurality of neighbor feature points corresponding to the second feature point, and a second mapping difference value between the plurality of neighbor feature points corresponding to the third feature point and the plurality of neighbor feature points corresponding to the fourth feature point; judging that the first feature point and the second feature point are matched successfully based on the weighted sum of the first mapping difference value and the second mapping difference value, and in the case that the sum result is less than a first threshold value; fusing the target brain image and the reference brain image of the target patient according to all the first feature points and the second feature points that are matched successfully.

2. The cerebral hemorrhage medical image fusion method of claim 1, characterized by, The process of extracting a plurality of feature points from the target brain image and the reference brain image respectively comprises: extracting a plurality of initial pixel points as corner points from the target brain image and the reference brain image respectively; calculating a HOG descriptor of the initial pixel points, and extracting the plurality of feature points from the plurality of initial pixel points based on the HOG descriptor; wherein the HOG descriptor comprises a position feature and a contour feature of the initial pixel point, and the contour feature is a gradient representation based on the position feature.

3. The cerebral hemorrhage medical image fusion method of claim 2, characterized by, The process of extracting a plurality of initial pixel points as corner points from the target brain image comprises: extracting all corner points from the target brain image; dividing the whole region of the target brain image into four sub-regions with the geometric center of the target brain image as the center, and calculating the number of corner points in each sub-region; in the case that the number of corner points in a sub-region is greater than a second threshold value, dividing the sub-region into four sub-regions again with the geometric center of the sub-region as the center; in the case that the number of corner points in a sub-region is less than a second threshold value, selecting the corner point with the highest Harris value from the corner points in the sub-region as an initial pixel point; In succession, until the sub-region stops being divided, and the initial pixel point corresponding to each sub-region is obtained; The process of extracting a plurality of initial pixel points as corner points from the reference brain image comprises: Extracting all corner points from the reference brain image; Dividing the whole region of the reference brain image into four sub-regions with the geometric center of the reference brain image as the center, and calculating the number of corner points in each sub-region; In the case where the number of corner points in the sub-region is greater than the third threshold, the sub-region is further divided into four sub-regions with the geometric center of the sub-region as the center; in the case where the number of corner points in the sub-region is less than the second threshold, the initial pixel point is selected from the corner points in the sub-region, and the initial pixel point is the corner point with the highest Harris value; In succession, until the sub-region stops being divided, and the initial pixel point corresponding to each sub-region is obtained.

4. The cerebral hemorrhage medical image fusion method of claim 2, characterized by, The calculation of the first mapping difference value between the plurality of neighbor feature points corresponding to the first feature point and the plurality of neighbor feature points corresponding to the second feature point comprises: Determining a first contour feature corresponding to the first feature point and a second contour feature corresponding to the second feature point from the HOG descriptor; Calculating an intermediate feature between the first feature point and the second feature point according to the first contour feature and the second contour feature; Mapping the plurality of neighbor feature points corresponding to the first feature point into the neighborhood of the plurality of neighbor feature points corresponding to the second feature point according to the intermediate feature; Determining each pair of the neighbor feature points between the first feature point and the second feature point in the neighborhood of the plurality of neighbor feature points corresponding to the second feature point, and calculating the difference value between each pair of the neighbor feature points. Calculating the first mapping difference value according to the difference values of the plurality of pairs of the neighbor feature points between the first feature point and the second feature point.

5. The cerebral hemorrhage medical image fusion method of claim 4, wherein, The calculation of the first mapping difference value according to the difference values of the plurality of pairs of the neighbor feature points between the first feature point and the second feature point comprises: by Using the hyperparameter and the difference as the base, construct an exponential function for a pair of neighboring feature points; Summing the exponential functions of the plurality of pairs of the neighbor feature points to obtain the first mapping difference value.

6. The cerebral hemorrhage medical image fusion method of claim 2, wherein, The calculation of the first mapping difference value between the plurality of neighbor feature points corresponding to the third feature point and the plurality of neighbor feature points corresponding to the fourth feature point comprises: Determining a third contour feature corresponding to the third feature point and a fourth contour feature corresponding to the fourth feature point from the HOG descriptor; Calculating an intermediate feature between the third feature point and the fourth feature point according to the third contour feature and the fourth contour feature; Mapping the plurality of neighbor feature points corresponding to the third feature point into the neighborhood of the plurality of neighbor feature points corresponding to the fourth feature point according to the intermediate feature; Determining each pair of the neighbor feature points between the third feature point and the fourth feature point in the neighborhood of the plurality of neighbor feature points corresponding to the fourth feature point, and calculating the difference value between each pair of the neighbor feature points. According to the difference values of the pairs of neighbor feature points between the third feature point and the fourth feature point, the second mapping difference value is calculated.

7. The cerebral hemorrhage medical image fusion method of claim 6, characterized in that, The calculation of the second mapping difference value according to the difference values of the pairs of neighbor feature points between the third feature point and the fourth feature point comprises: by Using the hyperparameter and the difference as the base, construct an exponential function for a pair of neighboring feature points; The second mapping difference value is obtained by summing the exponential functions of the pairs of neighbor feature points.

8. The cerebral hemorrhage medical image fusion method of claim 2, wherein, The selecting of the pairs of neighbor feature points corresponding to the first feature point to the fourth feature point comprises: In the neighborhood range of a first target feature point, a plurality of feature points with similar HOG descriptors to the first target feature point are calculated as the plurality of neighbor feature points of the first target feature point; wherein the first target feature point is the first feature point or the third feature point. From the neighborhood range of a second target feature point, a plurality of feature points similar to the plurality of neighbor feature points of the first target feature point in the HOG descriptor are found as the plurality of neighbor feature points of the second target feature point; wherein when the first target feature point is the first feature point, the second target feature point is the second feature point, and when the first target feature point is the third feature point, the second target feature point is the fourth feature point.

9. A cerebral hemorrhage medical image fusion system characterized by comprising: The system comprises: An image acquisition module is configured to acquire a target brain image and a reference brain image of a target patient, and divide the target brain image into two target sub-images and divide the reference brain image into two reference sub-images according to the symmetry of brain structures; wherein the target brain image and the reference brain image are cerebral hemorrhage medical images of different domains; A feature point processing module is configured to extract a plurality of feature points from the target brain image and the reference brain image respectively, and match the plurality of feature points in the target brain image with the plurality of feature points in the reference brain image; wherein the matching process between a first feature point in the target brain image and a second feature point in the reference brain image comprises: determining a third feature point and a fourth feature point; wherein the third feature point is a feature point that is in a different target sub-image from the first feature point and is symmetrical to the first feature point in brain structure, and the fourth feature point is a feature point that is in a different reference sub-image from the second feature point and is symmetrical to the second feature point in brain structure; selecting a plurality of neighbor feature points corresponding to the first feature point to the fourth feature point respectively; calculating a first mapping difference value between the plurality of neighbor feature points corresponding to the first feature point and the plurality of neighbor feature points corresponding to the second feature point, and calculating a second mapping difference value between the plurality of neighbor feature points corresponding to the third feature point and the plurality of neighbor feature points corresponding to the fourth feature point; judging that the first feature point and the second feature point are matched successfully based on the weighted sum of the first mapping difference value and the second mapping difference value, and in the case that the sum result is less than a first threshold value. An image fusion module is configured to fuse a target brain image of the target patient and a reference brain image according to all the matched first feature points and the matched second feature points.

10. An electronic device, comprising: The method comprises: at least one control processor and a memory connected in communication with the at least one control processor; the memory stores instructions executable by the at least one control processor, and the instructions are executed by the at least one control processor to enable the at least one control processor to perform the medical image fusion method for brain hemorrhage according to any one of claims 1 to 8.

Images