Cerebral hemorrhage medical image fusion method and system

Abstract

The invention relates to a cerebral hemorrhage medical image fusion method and system.The cerebral hemorrhage medical image fusion method utilizes the characteristic that a brain structure has certain symmetry, the brain structure in a target brain image and a brain structure in a reference brain image are equally divided into two sub-images, and when a first feature point is matched with a second feature point, the first feature point is matched with the second feature point; a third feature point and a fourth feature point in the other sub-image can be utilized to assist the first feature point and the second feature point in matching; and in the matching process of any pair of feature points, a plurality of neighbor feature points of one feature point are mapped into the space of the other feature point, then the mapping difference value between the neighbor feature points is calculated, and then when the two difference values are smaller than a set value, it is judged that the first feature point and the second feature point are successfully matched, and then the matching of the first feature point and the second feature point is completed. According to the method, the feature point matching accuracy can be improved, and the image fusion effect is finally improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer vision technology, and in particular to a method and system for fusion of medical images of cerebral hemorrhage. Background Art

[0002] Fusion of medical images from different domains (such as CT images acquired by different models of equipment) can help improve diagnostic accuracy. However, images acquired by different models of equipment may differ (e.g., due to differences in configuration parameters and camera angles, resulting in sudden changes in features). Therefore, improving the effectiveness of medical image fusion is a current research focus.

[0003] The feature point matching process is the foundation of medical image fusion. Feature point matching is the process of aligning images of the same object acquired under different conditions. Its essence is to find a set of spatial geometric transformation relationships such that all feature points in one image can be obtained through the transformation of the other image. Currently, feature point matching relies primarily on similarity matching, primarily using Euclidean distance to calculate the similarity between a set of feature points to be matched. However, this method ignores the problem of inaccurate similarity calculations between images in different domains due to differences in acquisition equipment, which in turn reduces matching accuracy. Summary of the Invention

[0004] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0005] The main purpose of the embodiments of the present application is to propose a method and system for fusion of medical images of cerebral hemorrhage, which can improve the effect of image fusion.

[0006] To achieve the above-mentioned objectives, a first aspect of an embodiment of the present application provides a method for fusion of medical images of cerebral hemorrhage, the method comprising: Acquire a target brain image and a reference brain image of a target patient, and divide the target brain image into two target sub-images and the reference brain image into two reference sub-images based on the symmetry of brain structure; wherein the target brain image and the reference brain image are medical images of cerebral hemorrhage in different domains; Extracting a plurality of feature points from the target brain image and the reference brain image, respectively, and matching the plurality of feature points in the target brain image with the plurality of feature points in the reference brain image; wherein the process of matching a first feature point in the target brain image with a second feature point in the reference brain image comprises: Determining a third feature point and a fourth feature point; wherein the third feature point is located in a different target sub-image than the first feature point and is symmetrical with the first feature point in terms of brain structure; and the fourth feature point is located in a different reference sub-image than the second feature point and is symmetrical with the second feature point in terms of brain structure; Selecting a plurality of neighboring feature points corresponding to each of the first feature point to the fourth feature point; Calculating first mapping differences between a plurality of neighbor feature points corresponding to the first feature point and a plurality of neighbor feature points corresponding to the second feature point, and calculating second mapping differences between a plurality of neighbor feature points corresponding to the third feature point and a plurality of neighbor feature points corresponding to the fourth feature point; Based on a weighted sum of the first mapping difference and the second mapping difference, and when the sum result is less than a first threshold, determining that the first feature point and the second feature point are successfully matched; The target brain image and the reference brain image of the target patient are fused based on all successfully matched first feature points and second feature points.

[0007] The present application provides a method for fusion of medical images of cerebral hemorrhage, which has at least the following beneficial effects: Since images in different domains come from different acquisition devices, there may be differences between images in different domains for the same target, resulting in an inaccurate matching process in the fusion. The present application utilizes the characteristic that the brain structure has a certain symmetry, and divides the brain structure in the target brain image and the reference brain image into two equal sub-images; then, when the first feature point and the second feature point are matched, considering that the third feature point symmetrical to the first feature point in the brain structure has a strong similarity, and the second feature point symmetrical to the fourth feature point in the brain structure has a strong similarity, and there is domain distribution feature invariance between each other, the matching results of the symmetrical third feature point and the symmetrical fourth feature point can be used to assist the first feature point and the second feature point. Two feature points are matched, thereby improving the accuracy of matching the first feature point and the second feature point, and ultimately improving the effect of image fusion; when calculating the matching of a group of feature points, the present application maps multiple neighbor feature points in the domain space where one feature point is located to the domain space where another feature point is located, and then calculates the mapping difference between the multiple neighbor feature points, and then based on the mapping difference of a group of feature points, and with the assistance of the mapping difference of a symmetrical group of feature points, judges whether the size of the two differences is less than the set value, and if it is less than the set value, judges that the first feature point and the second feature point are matched successfully, and uses the mapping difference between the neighbors near the feature points for stability judgment, which can improve the accuracy of matching the first feature point and the second feature point, and ultimately improve the effect of image fusion.

[0008] In some embodiments, extracting a plurality of feature points from the target brain image and the reference brain image respectively includes: extracting a plurality of initial pixel points as corner points from the target brain image and the reference brain image respectively; Calculate the HOG descriptor of the initial pixel point, and extract the multiple feature points from the multiple initial pixel points based on the HOG descriptor; wherein the HOG descriptor includes the position feature and contour feature of the initial pixel point, and the contour feature is a gradient representation based on the position feature.

[0009] In some embodiments, the process of extracting a plurality of initial pixel points as corner points from the target brain image includes: Extracting all corner points from the target brain image; Dividing the entire target brain image into four sub-regions with the geometric center of the target brain image as the center, and calculating the number of corner points in each sub-region; When the number of corner points in a subregion is greater than a second threshold, the subregion is divided into four subregions with the geometric center of the subregion as the center; when the number of corner points in a subregion is less than the second threshold, the corner point with the highest Harris value is selected from the corner points in the subregion as the initial pixel point; And so on, until the sub-regions stop being divided and the initial pixel points corresponding to each sub-region are obtained; The process of extracting a plurality of initial pixel points as corner points from the reference brain image includes: Extracting all corner points from the reference brain image; Dividing the entire area of the reference brain image into four sub-areas with the geometric center of the reference brain image as the center, and calculating the number of corner points in each sub-area; When the number of corner points in a subregion is greater than the third threshold, the subregion is divided into four subregions with the geometric center of the subregion as the center. When the number of corner points in a subregion is less than the second threshold, the corner point with the highest Harris value is selected from the corner points in the subregion as the initial pixel point. This process continues in this way until the sub-regions are divided and the initial pixel points corresponding to each sub-region are obtained.

[0010] In some embodiments, calculating first mapping differences between a plurality of neighboring feature points corresponding to the first feature point and a plurality of neighboring feature points corresponding to the second feature point includes: Determine, from the HOG descriptor, a first contour feature corresponding to the first feature point and a second contour feature corresponding to the second feature point; Calculating an intermediate feature between the first feature point and the second feature point according to the first contour feature and the second contour feature; Mapping, according to the mapping intermediate feature, a plurality of neighbor feature points corresponding to the first feature point to a neighborhood where a plurality of neighbor feature points corresponding to the second feature point are located; In a neighborhood of a plurality of neighbor feature points corresponding to the second feature point, determining each pair of neighbor feature points between the first feature point and the second feature point, and calculating a difference between each pair of neighbor feature points; The first mapping difference is calculated according to the difference between the first feature point and the second feature point in a plurality of pairs of neighbor feature points.

[0011] In some embodiments, calculating the first mapping difference value based on the difference values of multiple pairs of neighbor feature points between the first feature point and the second feature point includes: by As the base, and with the product of the hyperparameter and the difference as the exponent, construct an exponential function of a pair of the neighbor feature points; The exponential functions of multiple pairs of the neighbor feature points are summed to obtain the first mapping difference.

[0012] In some embodiments, the calculating of first mapping differences between a plurality of neighbor feature points corresponding to the third feature point and a plurality of neighbor feature points corresponding to the fourth feature point includes: Determining, from the HOG descriptor, a third contour feature corresponding to the third feature point and a fourth contour feature corresponding to the fourth feature point; Calculating an intermediate feature between the third feature point and the fourth feature point according to the third contour feature and the fourth contour feature; Mapping the plurality of neighbor feature points corresponding to the third feature point to a neighborhood where the plurality of neighbor feature points corresponding to the fourth feature point are located according to the mapping intermediate feature; In a neighborhood of a plurality of neighbor feature points corresponding to the fourth feature point, determining each pair of neighbor feature points between the third feature point and the fourth feature point, and calculating a difference between each pair of neighbor feature points; The second mapping difference is calculated according to the difference between the plurality of pairs of neighbor feature points between the third feature point and the fourth feature point.

[0013] In some embodiments, calculating the second mapping difference according to the difference between the plurality of pairs of neighbor feature points between the third feature point and the fourth feature point includes: by As the base, and with the product of the hyperparameter and the difference as the exponent, construct an exponential function of a pair of the neighbor feature points; The exponential functions of multiple pairs of the neighbor feature points are summed to obtain the second mapping difference.

[0014] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a medical image fusion system for cerebral hemorrhage, the system comprising: an image acquisition module, configured to acquire a target brain image and a reference brain image of a target patient, divide the brain structure in the target brain image into two equal parts to obtain a first target sub-image and a second target sub-image, and divide the brain structure in the reference brain image into two equal parts to obtain a first reference sub-image and a second reference sub-image; wherein the target brain image and the reference brain image are medical images of cerebral hemorrhage in different domains; a feature point processing module, configured to extract a plurality of feature points from the target brain image and the reference brain image, respectively, and match the plurality of feature points in the target brain image with the plurality of feature points in the reference brain image; wherein the process of matching a first feature point in the target brain image with a second feature point in the reference brain image comprises: Determining a third feature point and a fourth feature point; wherein the third feature point is a feature point that is located in a different target brain image than the first feature point and is symmetrical to the first feature point; and the fourth feature point is a feature point that is located in a different reference brain image than the second feature point and is symmetrical to the second feature point; Select K neighboring feature points corresponding to each of the first feature point, the second feature point, the third feature point, and the fourth feature point; Calculating first mapping differences between the K neighbor feature points corresponding to the first feature point and the K neighbor feature points corresponding to the second feature point, and calculating second mapping differences between the K neighbor feature points corresponding to the third feature point and the K neighbor feature points corresponding to the fourth feature point; Based on a weighted sum of the first mapping difference and the second mapping difference, and when the sum result is less than a first threshold, determining that the first feature point and the second feature point are successfully matched; An image fusion module is used to fuse the target brain image and the reference brain image of the target patient based on all the first feature points and the second feature points that are successfully matched.

[0015] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application provides an electronic device, comprising: at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute the above-mentioned cerebral hemorrhage medical image fusion method.

[0016] To achieve the above-mentioned purpose, the fourth aspect of the embodiments of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the above-mentioned cerebral hemorrhage medical image fusion method.

[0017] It can be understood that the beneficial effects of the second to fourth aspects compared with the relevant technologies are the same as the beneficial effects of the first aspect compared with the relevant technologies. Please refer to the relevant description in the first aspect and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 is a schematic diagram of an embodiment of the present application providing a brain CT image; Figure 2 This is a flowchart of an embodiment of the medical image fusion method for cerebral hemorrhage provided by the present application; Figure 3 This is a schematic structural diagram of an embodiment of a medical image fusion system for cerebral hemorrhage provided by the present application; Figure 4 is a schematic diagram of an embodiment of an electronic device provided by the present application. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0021] To address technical deficiencies such as Figure 2 One embodiment of the present application provides a method for fusion of medical images of cerebral hemorrhage, the method comprising: Step S110 , obtaining a target brain image and a reference brain image of a target patient, and dividing the target brain image into two target sub-images and the reference brain image into two reference sub-images according to the symmetry of the brain structure.

[0022] The target patients are those suffering from cerebral hemorrhage.

[0023] The target brain image and the reference brain image are images in different domains. For example, the reference brain image and the target brain image are CT images acquired by different imaging devices.

[0024] The purpose of this embodiment is to find the feature points that match the reference brain image and the target brain image, and then use these successfully matched feature points to achieve the fusion of the reference brain image and the target brain image, two medical images in different domains, to facilitate subsequent medical diagnosis or disease treatment of the patient by doctors.

[0025] like Figure 1 ,Since the brain structure has a certain symmetry, the brain structure in the image can be divided into two parts according to the central axis (set according to the doctor's experience), that is, the reference brain image or the target brain image can be divided into two sub-images.

[0026] When the feature points in the first target sub-image of the target brain image are matched with the feature points in the first reference sub-image of the reference brain image, the feature points in the second target sub-image of the target brain image and the feature points in the second reference sub-image of the reference brain image can be found, and the found pair of feature points are used to assist the matching process of the pair of feature points to improve the matching accuracy of the pair of feature points.

[0027] Step S120, extracting a plurality of feature points from the target brain image and the reference brain image respectively, and matching the plurality of feature points in the target brain image with the plurality of feature points in the reference brain image; Assume that the first feature point is a feature point in the target brain image and the second feature point is a feature point in the reference brain image. Then, the process of matching the first feature point in the target brain image with the second feature point in the reference brain image includes: (1) Determine the third and fourth feature points.

[0028] The third feature point is located in a different target sub-image from the first feature point and is symmetrical with the first feature point in terms of brain structure. Assuming that the first feature point is located in the first target sub-image of the target brain image, the third feature point is located in the second target sub-image of the target brain image; The fourth feature point is located in a different reference sub-image than the second feature point and is symmetrical with the second feature point in terms of brain structure. If the first and third feature points are consistent, they will not be described in detail here.

[0029] (2) Select multiple neighboring feature points corresponding to the first feature point, the second feature point, the third feature point, and the fourth feature point.

[0030] There are many ways to select neighbor feature points within a neighborhood, such as Euclidean distance calculation, etc. See the introduction of the subsequent embodiments for details.

[0031] (3) Calculating first mapping differences between a plurality of neighboring feature points corresponding to the first feature point and a plurality of neighboring feature points corresponding to the second feature point, and calculating second mapping differences between a plurality of neighboring feature points corresponding to the third feature point and a plurality of neighboring feature points corresponding to the fourth feature point.

[0032] Multiple neighbor feature points corresponding to the first feature point in a domain image are mapped to another domain to calculate the difference between the multiple neighbor feature points corresponding to the first feature point and the second feature point in one domain space (see subsequent embodiments for details), and then judge whether the first feature point and the second feature point are similar based on the size of the difference. Since the distribution of neighbor feature points in their respective domain spaces has domain correlation, for example, if the difference between a pair of multiple neighbor feature points is small, it means that the pair of feature points are similar and can be considered to match.

[0033] Due to the symmetry of the brain structure, the third and fourth feature points are processed in a similar manner and will not be described in detail here.

[0034] (4) Based on the weighted sum of the first mapping difference and the second mapping difference, if the sum is less than a first threshold, the first feature point and the second feature point are determined to be matched successfully. Here, a lower weight, such as 0.3, can be set for the second mapping difference, and a higher weight, such as 0.7, can be set for the first mapping difference. The first threshold can be set based on experience.

[0035] In step S130 , the target brain image of the target patient and the reference brain image are fused based on all successfully matched first feature points and second feature points.

[0036] It should be noted that after finding the matching feature points in the two domains, the images can be fused based on the matching feature points. Since the fusion here is common knowledge among those skilled in the art and this method does not involve the subsequent fusion process, it will not be described in detail here.

[0037] This method has at least the following beneficial effects: Because images in different domains come from different acquisition devices, there may be differences between images in different domains for the same target, resulting in an inaccurate matching process in the fusion. The present application utilizes the characteristic that the brain structure has a certain symmetry, and divides the brain structure in the target brain image and the reference brain image into two equal sub-images; then, when matching the first feature point and the second feature point, considering that the third feature point symmetrical to the first feature point in the brain structure has a strong similarity with the first feature point, and the second feature point symmetrical to the fourth feature point in the brain structure has a strong similarity with the fourth feature point, and that there is domain distribution feature invariance between them, the matching results of the symmetrical third feature point and the symmetrical fourth feature point can be used to assist the matching of the first feature point and the second feature point, thereby improving the accuracy of the matching of the first feature point and the second feature point, and ultimately improving the effect of image fusion; When calculating the matching of a group of feature points, the present application maps multiple neighbor feature points in the domain space where one feature point is located to the domain space where another feature point is located, and then calculates the mapping differences between the multiple neighbor feature points. Based on the mapping differences of a group of feature points, and with the assistance of the mapping differences of a symmetrical group of feature points, it is judged whether the sizes of the two differences are less than the set value. If they are less than the set value, it is judged that the first feature point and the second feature point are successfully matched. The mapping differences between the neighbors near the feature points are used to perform stability judgment, which can improve the accuracy of the matching of the first feature point and the second feature point, and ultimately improve the effect of image fusion.

[0038] Furthermore, the step S120 of extracting a plurality of feature points from the target brain image and the reference brain image includes steps S210 to S220: In step S210 , a plurality of initial pixel points serving as corner points are extracted from the target brain image and the reference brain image respectively.

[0039] In this embodiment, a corner point detection method can be used to determine whether a pixel point in the target brain image and the reference brain image is a corner point. If the pixel point is a corner point, the pixel point of the corner point is used as the initial pixel point.

[0040] Step S220 , calculating the HOG descriptor of the initial pixel point, and extracting multiple feature points from the multiple initial pixel points based on the HOG descriptor.

[0041] The HOG descriptor includes the positional features and contour features of the initial pixel. The contour features are based on the gradient representation of the positional features. HOG describes the contour of the target location by statistically analyzing the gradient direction histogram of the local sub-image at the target location. Essentially, it is a statistical analysis of edge gradient information.

[0042] Furthermore, the process of extracting a plurality of initial pixel points as corner points from the target brain image in step S210 includes the following steps S2110 to S2140: Step S2110, extracting all corner points from the target brain image; Step S2120 , dividing the entire area of the target brain image into four sub-areas with the geometric center of the target brain image as the center, and calculating the number of corner points in each sub-area; Step S2130: If the number of corner points in the sub-region is greater than the second threshold, the sub-region is divided into four sub-regions with the geometric center of the sub-region as the center. If the number of corner points in the sub-region is less than the second threshold, the corner point with the highest Harris value is selected from the corner points in the sub-region as the initial pixel point. Step S2140, and so on, until the sub-region division stops and the initial pixel point corresponding to each sub-region is obtained.

[0043] The process of extracting a plurality of initial pixel points as corner points from the reference brain image in step S210 includes the following steps S2150 to S2180: Step S2150, extracting all corner points from the reference brain image; Step S2160 , dividing the entire area of the reference brain image into four sub-areas with the geometric center of the reference brain image as the center, and calculating the number of corner points in each sub-area; Step S2170: If the number of corner points in the sub-region is greater than the third threshold, the sub-region is divided into four sub-regions with the geometric center of the sub-region as the center. If the number of corner points in the sub-region is less than the second threshold, the corner point with the highest Harris value is selected from the corner points in the sub-region as the initial pixel point. Step S2180, and so on, until the sub-region division stops and the initial pixel point corresponding to each sub-region is obtained.

[0044] Taking steps S2110 to S2140 as an example, in step S2120, the geometric center refers to the centroid position of the image area, which can be determined by calculating the mean of the image boundary coordinates and used as the reference point for dividing the area. Dividing into four sub-areas means dividing the original area into four parts of equal area according to the horizontal and vertical center lines, for example, dividing the image into four areas of upper left, upper right, lower left, and lower right. The second threshold (which can be based on experience) refers to the preset upper limit of the number of initial pixels in the area, which is used to determine whether to continue to subdivide the area. The initial pixel points are screened by comparing the Harris corner point response values of each point in the area, and retaining the pixel point corresponding to the maximum value as the initial pixel point.

[0045] In this embodiment, the screening range is dynamically adjusted by recursively dividing the regions. When a subregion contains too many corner points, it indicates that the region may have dense or redundant features and requires further subdivision to improve screening accuracy. When the number of corner points in a subregion is below a threshold, the point with the highest Harris value in the region is directly selected as the representative. For example, after the initial image is divided into four subregions, if one of the subregions has more corner points than a set threshold (for example, 8), the subregion is further divided into four equal parts until the number of corner points in all subregions does not exceed the threshold. Ultimately, only one corner point with the highest Harris value is retained in each subregion as the initial pixel point.

[0046] Traditional methods typically perform a global screening of all initial pixels, resulting in ineffective removal of redundant features. This method, however, controls the granularity of screening through dynamic region partitioning, retaining the most representative feature points within each subregion and reducing interference in subsequent matching processes.

[0047] In this embodiment, only the feature points with the highest Harris value in each sub-image are selected in each sub-region, and other feature points are deleted. Finally, feature points that can better reflect the information of the entire image and are evenly distributed are screened out from a large number of feature points.

[0048] Furthermore, the calculation of the first mapping difference between the plurality of neighboring feature points corresponding to the first feature point and the plurality of neighboring feature points corresponding to the second feature point in step S120 includes the following steps S310 to S350: Step S310, determining a first contour feature corresponding to a first feature point and a second contour feature corresponding to a second feature point from the HOG descriptor; Step S320, calculating an intermediate feature between the first feature point and the second feature point based on the first contour feature and the second contour feature; Step S330, mapping the plurality of neighbor feature points corresponding to the first feature point to the neighborhood of the plurality of neighbor feature points corresponding to the second feature point according to the intermediate feature; Step S340, determining each pair of neighbor feature points between the first feature point and the second feature point in the neighborhood of the plurality of neighbor feature points corresponding to the second feature point, and calculating the difference between each pair of neighbor feature points; Step S350 : calculating a first mapping difference value based on the difference values of multiple pairs of neighboring feature points between the first feature point and the second feature point.

[0049] Another important invention point of the present application is that in step S310, this embodiment selects the contour feature description of the first feature point and the contour feature description of the second feature point in the HOG descriptor, and then uses the contour features of the first feature point and the contour features of the second feature point in different domain images to construct an intermediate feature. The intermediate feature is used to measure the contour transformation relationship between the first feature point and the second feature point that is invariant in the local space centered on each other. According to this contour transformation relationship, the feature points in one image domain are mapped to another image domain space, which can greatly reduce the mapping error after mapping and improve the matching accuracy.

[0050] In step S320 , an intermediate feature between the first feature point and the second feature point may be obtained by multiplying the first contour feature of the first feature point and the inverse matrix of the second contour feature of the second feature point.

[0051] Then in step S330, the intermediate features are used as a tool for mapping feature points between domain images, and the neighbor feature points of the first feature point are mapped to the space corresponding to the second feature point according to the intermediate features.

[0052] In step S340 , the neighborhood of the second feature point refers to a circular area constructed with the second feature point as the center and a preset radius, and the feature points in the neighborhood are neighbor feature points of the second feature point.

[0053] The core of the difference between each pair of neighbor feature points calculated here is: In a mapped space, determine the difference between the first feature point and its corresponding neighbor feature point, and the difference between the second feature point and its corresponding neighbor feature point. For example, assuming that the first feature point and the second feature point are and , the kth pair of neighbor feature points is and , assuming the difference is , For intermediate features, the calculation process of the difference includes: .

[0054] ； ； In step S350 , a first mapping difference is calculated based on multiple neighboring feature points of the first feature point and multiple neighboring feature points of the second feature point in the same neighborhood.

[0055] Calculating the first mapping difference in step S350 based on the differences between the first feature point and the second feature point in multiple pairs of neighboring feature points includes the following steps S3510 to S3520: Step S3510, As the base, the exponential function of a pair of neighbor feature points is constructed with the product between the hyperparameter and the difference as the exponent; Step S3520: sum the exponential functions of multiple pairs of neighbor feature points to obtain a first mapping difference.

[0056] For example: ; in, For the The difference between neighbor feature points (K is the total number of neighbor feature points), is a hyperparameter, is an exponential function, is the first mapping difference.

[0057] In this embodiment, the embodiment finds multiple neighboring feature points corresponding to a pair of feature points, and then maps the feature points of different domain images to the space of the same domain image based on the intermediate features. If the difference between these neighboring feature points If the first and second feature points are small (a threshold can be set for judgment), then the two first and second feature points are considered similar and the two feature points are considered to be matched successfully; if If the value is larger, then the two are considered mismatched.

[0058] Furthermore, the calculation of the first mapping differences between the plurality of neighboring feature points corresponding to the third feature point and the plurality of neighboring feature points corresponding to the fourth feature point in step S120 includes the following steps S410 to S450: Step S410, determining a third contour feature corresponding to the third feature point and a fourth contour feature corresponding to the fourth feature point from the HOG descriptor; Step S420, calculating an intermediate feature between the third feature point and the fourth feature point based on the third contour feature and the fourth contour feature; Step S430, mapping the plurality of neighbor feature points corresponding to the third feature point to the neighborhood of the plurality of neighbor feature points corresponding to the fourth feature point according to the intermediate feature; Step S440, determining each pair of neighbor feature points between the third feature point and the fourth feature point in the neighborhood of the plurality of neighbor feature points corresponding to the fourth feature point, and calculating the difference between each pair of neighbor feature points; Step S450 : calculating a second mapping difference value based on the difference values of multiple pairs of neighboring feature points between the third feature point and the fourth feature point.

[0059] Calculating the second mapping difference in step S450 based on the differences between multiple pairs of neighboring feature points between the third feature point and the fourth feature point includes the following steps S4510 to S4520: Step S4510, As the base, the exponential function of a pair of neighbor feature points is constructed with the product between the hyperparameter and the difference as the exponent; Step S4520: sum the exponential functions of multiple pairs of neighbor feature points to obtain the second mapping difference This embodiment is similar to the above embodiment and will not be described in detail here.

[0060] Furthermore, the step S120 of selecting a plurality of neighboring feature points corresponding to the first feature point to the fourth feature point includes the following steps S510 to S520: In step S510 , a plurality of feature points having similar HOG descriptor similarities to the first target feature point are calculated within a neighborhood of the first target feature point as a plurality of neighbor feature points of the first target feature point.

[0061] Wherein, the first target feature point is the first feature point, or the third feature point; Based on the above embodiment, taking the first target feature point as the first feature point as an example, the neighborhood range of the first feature point is preset, and then based on the similarity of the HOG descriptor, multiple similar feature points are found based on the similarity as multiple neighbor feature points of the first feature point. These similar neighbor feature points have a high feature similarity with the first feature point. Using the matching results of multiple neighbor feature points as the matching results of the first feature point can improve the stability of feature point matching.

[0062] In this embodiment, the corresponding neighbor feature points are selected based on the HOG descriptor, which can stably find neighbor feature points similar to the feature point.

[0063] Step S520 : Find multiple feature points with similar HOG descriptors to multiple neighboring feature points of the first target feature point from within the neighborhood of the second target feature point, and use them as multiple neighboring feature points of the second target feature point.

[0064] Among them, when the first target feature point is the first feature point, the second target feature point is the second feature point, and when the first target feature point is the third feature point, the second target feature point is the fourth feature point. Among them, the neighborhood range of the second target feature point is consistent with that of the first target feature point. Both use the corresponding feature as the center of the circle and set the radius (the radius can be the same). The circle is the neighborhood range of the second target feature point. Then, since the second target feature point is a feature point symmetrical to the first target feature point in the same domain image, and the second target feature point is used to assist the matching process of the first target feature point, the multiple neighbor feature points corresponding to the second target feature point are also based on the HOG descriptor. Multiple neighbor feature points similar to the first target feature point are found from the neighborhood corresponding to the second target feature point to enhance the importance of the matching of the second target feature point, and finally to improve the matching accuracy of the first target feature point.

[0065] Reference Figure 3 In one embodiment, a medical image fusion system for cerebral hemorrhage is provided, the system comprising: The image acquisition module 1100 is used to acquire a target brain image and a reference brain image of a target patient. Based on the symmetry of the brain structure, the target brain image is divided into two target sub-images, and the reference brain image is divided into two reference sub-images. The target brain image and the reference brain image are medical images of cerebral hemorrhage in different domains. The feature point processing module 1200 is configured to extract multiple feature points from the target brain image and the reference brain image, respectively, and match the multiple feature points in the target brain image with the multiple feature points in the reference brain image. The matching process between a first feature point in the target brain image and a second feature point in the reference brain image includes: Determine a third feature point and a fourth feature point; wherein the third feature point is a feature point that is located in a different target sub-image than the first feature point and is symmetrical with the first feature point in terms of brain structure; and the fourth feature point is a feature point that is located in a different reference sub-image than the second feature point and is symmetrical with the second feature point in terms of brain structure; Selecting a plurality of neighboring feature points corresponding to the first feature point to the fourth feature point; Calculating first mapping differences between a plurality of neighbor feature points corresponding to the first feature point and a plurality of neighbor feature points corresponding to the second feature point, and calculating second mapping differences between a plurality of neighbor feature points corresponding to the third feature point and a plurality of neighbor feature points corresponding to the fourth feature point; Based on a weighted sum of the first mapping difference and the second mapping difference, and when the sum result is less than a first threshold, determining that the first feature point and the second feature point are successfully matched; The image fusion module 1300 is configured to fuse the target brain image of the target patient and the reference brain image based on all successfully matched first feature points and second feature points.

[0066] It should be noted that the cerebral hemorrhage medical image fusion system provided in this embodiment and the above-mentioned cerebral hemorrhage medical image fusion method are based on the same inventive concept. Therefore, the relevant content of the above-mentioned cerebral hemorrhage medical image fusion method is also applicable to the content of the cerebral hemorrhage medical image fusion system. Therefore, it will not be repeated here.

[0067] like Figure 4 , an embodiment of the present application further provides an electronic device, the electronic device comprising: at least one memory; at least one processor; at least one program; The programs are stored in the memory, and the processor executes at least one program to implement the above-mentioned cerebral hemorrhage medical image fusion method implemented in the present disclosure.

[0068] The electronic device may be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a car computer, etc.

[0069] The electronic device according to the embodiment of the present application is described in detail below.

[0070] The processor 1600 may be implemented as a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute relevant programs to implement the technical solutions provided in the embodiments of the present application. The memory 1700 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1700 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1700 and is called by the processor 1600 to execute the intracerebral hemorrhage medical image fusion method of the embodiments of this application.

[0071] Input / output interface 1800, used for information input and output; Communication interface 1900, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.); The bus 2000 transmits information between various components of the device (e.g., the processor 1600 , the memory 1700 , the input / output interface 1800 , and the communication interface 1900 ); The processor 1600 , the memory 1700 , the input / output interface 1800 , and the communication interface 1900 are connected to each other in communication within the device via the bus 2000 .

[0072] An embodiment of the present application further provides a storage medium, which is a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the above-mentioned cerebral hemorrhage medical image fusion method.

[0073] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and their combinations.

[0074] The embodiments described in this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0075] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or may combine certain steps, or may include different steps.

[0076] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0077] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0078] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0079] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0080] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or pairs of components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0081] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0082] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0083] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling an electronic device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0084] The above is a specific description of the preferred implementation of the embodiments of the present application, but the embodiments of the present application are not limited to the above-mentioned implementation methods. Technical personnel familiar with the art can also make various equivalent modifications or substitutions without violating the spirit of the embodiments of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the embodiments of the present application.

Claims

1. A medical image fusion method for cerebral hemorrhage, characterized in that: The method comprises: Acquire a target brain image and a reference brain image of a target patient, and divide the target brain image into two target sub-images and the reference brain image into two reference sub-images based on the symmetry of brain structure; wherein the target brain image and the reference brain image are medical images of cerebral hemorrhage in different domains; Extracting a plurality of feature points from the target brain image and the reference brain image, respectively, and matching the plurality of feature points in the target brain image with the plurality of feature points in the reference brain image; wherein the process of matching a first feature point in the target brain image with a second feature point in the reference brain image comprises: Determining a third feature point and a fourth feature point; wherein the third feature point is located in a different target sub-image than the first feature point and is symmetrical with the first feature point in terms of brain structure; and the fourth feature point is located in a different reference sub-image than the second feature point and is symmetrical with the second feature point in terms of brain structure; Selecting a plurality of neighboring feature points corresponding to each of the first feature point to the fourth feature point; Calculating first mapping differences between a plurality of neighbor feature points corresponding to the first feature point and a plurality of neighbor feature points corresponding to the second feature point, and calculating second mapping differences between a plurality of neighbor feature points corresponding to the third feature point and a plurality of neighbor feature points corresponding to the fourth feature point; Based on a weighted sum of the first mapping difference and the second mapping difference, and when the sum result is less than a first threshold, determining that the first feature point and the second feature point are successfully matched; The target brain image and the reference brain image of the target patient are fused based on all successfully matched first feature points and second feature points.

2. The cerebral hemorrhage medical image fusion method according to claim 1, characterized in that: The extracting a plurality of feature points from the target brain image and the reference brain image respectively includes: extracting a plurality of initial pixel points as corner points from the target brain image and the reference brain image respectively; Calculate the HOG descriptor of the initial pixel point, and extract the multiple feature points from the multiple initial pixel points based on the HOG descriptor; wherein the HOG descriptor includes the position feature and contour feature of the initial pixel point, and the contour feature is a gradient representation based on the position feature.

3. The cerebral hemorrhage medical image fusion method according to claim 2, characterized in that: The process of extracting a plurality of initial pixel points as corner points from the target brain image includes: Extracting all corner points from the target brain image; Dividing the entire target brain image into four sub-regions with the geometric center of the target brain image as the center, and calculating the number of corner points in each sub-region; When the number of corner points in a subregion is greater than a second threshold, the subregion is divided into four subregions with the geometric center of the subregion as the center; when the number of corner points in a subregion is less than the second threshold, the corner point with the highest Harris value is selected from the corner points in the subregion as the initial pixel point; And so on, until the sub-regions stop being divided and the initial pixel points corresponding to each sub-region are obtained; The process of extracting a plurality of initial pixel points as corner points from the reference brain image includes: Extracting all corner points from the reference brain image; Dividing the entire area of the reference brain image into four sub-areas with the geometric center of the reference brain image as the center, and calculating the number of corner points in each sub-area; When the number of corner points in a subregion is greater than the third threshold, the subregion is divided into four subregions with the geometric center of the subregion as the center. When the number of corner points in a subregion is less than the second threshold, the corner point with the highest Harris value is selected from the corner points in the subregion as the initial pixel point. This process continues in this way until the sub-regions are divided and the initial pixel points corresponding to each sub-region are obtained.

4. The cerebral hemorrhage medical image fusion method according to claim 2, characterized in that: The calculating a first mapping difference between a plurality of neighbor feature points corresponding to the first feature point and a plurality of neighbor feature points corresponding to the second feature point includes: Determine, from the HOG descriptor, a first contour feature corresponding to the first feature point and a second contour feature corresponding to the second feature point; Calculating an intermediate feature between the first feature point and the second feature point according to the first contour feature and the second contour feature; Mapping, based on the intermediate features, a plurality of neighbor feature points corresponding to the first feature point to a neighborhood where a plurality of neighbor feature points corresponding to the second feature point are located; In a neighborhood of a plurality of neighbor feature points corresponding to the second feature point, determining each pair of neighbor feature points between the first feature point and the second feature point, and calculating a difference between each pair of neighbor feature points; The first mapping difference is calculated according to the difference between the first feature point and the second feature point in a plurality of pairs of neighbor feature points.

5. The cerebral hemorrhage medical image fusion method according to claim 4, characterized in that: The calculating the first mapping difference according to the difference between the plurality of pairs of neighbor feature points between the first feature point and the second feature point includes: by As the base, and with the product of the hyperparameter and the difference as the exponent, construct an exponential function of a pair of the neighbor feature points; The exponential functions of multiple pairs of the neighbor feature points are summed to obtain the first mapping difference.

6. The cerebral hemorrhage medical image fusion method according to claim 2, characterized in that: The calculating of first mapping differences between a plurality of neighbor feature points corresponding to the third feature point and a plurality of neighbor feature points corresponding to the fourth feature point includes: Determining, from the HOG descriptor, a third contour feature corresponding to the third feature point and a fourth contour feature corresponding to the fourth feature point; Calculating an intermediate feature between the third feature point and the fourth feature point according to the third contour feature and the fourth contour feature; Mapping, based on the intermediate features, a plurality of neighbor feature points corresponding to the third feature point to a neighborhood where a plurality of neighbor feature points corresponding to the fourth feature point are located; In a neighborhood of a plurality of neighbor feature points corresponding to the fourth feature point, determining each pair of neighbor feature points between the third feature point and the fourth feature point, and calculating a difference between each pair of neighbor feature points; The second mapping difference is calculated according to the difference between the plurality of pairs of neighbor feature points between the third feature point and the fourth feature point.

7. The cerebral hemorrhage medical image fusion method according to claim 6, characterized in that: The calculating the second mapping difference according to the difference between the plurality of pairs of neighbor feature points between the third feature point and the fourth feature point includes: by As the base, and with the product of the hyperparameter and the difference as the exponent, construct an exponential function of a pair of the neighbor feature points; The exponential functions of multiple pairs of the neighbor feature points are summed to obtain the second mapping difference.

8. The cerebral hemorrhage medical image fusion method according to claim 2, characterized in that: The selecting of a plurality of neighboring feature points corresponding to each of the first feature point to the fourth feature point includes: Within the neighborhood of the first target feature point, multiple feature points having similarities with the HOG descriptor between the first target feature point and the first target feature point are calculated as multiple neighbor feature points of the first target feature point; wherein the first target feature point is the first feature point or the third feature point; From the neighborhood of the second target feature point, find multiple feature points that are similar to the HOG descriptors of the multiple neighbor feature points of the first target feature point, as the multiple neighbor feature points of the second target feature point; wherein, when the first target feature point is the first feature point, the second target feature point is the second feature point, and when the first target feature point is the third feature point, the second target feature point is the fourth feature point.

9. A medical image fusion system for cerebral hemorrhage, characterized in that: The system comprises: an image acquisition module, configured to acquire a target brain image and a reference brain image of a target patient, and to divide the target brain image into two target sub-images and the reference brain image into two reference sub-images based on the symmetry of the brain structure; wherein the target brain image and the reference brain image are medical images of cerebral hemorrhage in different domains; a feature point processing module, configured to extract a plurality of feature points from the target brain image and the reference brain image, respectively, and match the plurality of feature points in the target brain image with the plurality of feature points in the reference brain image; wherein the process of matching a first feature point in the target brain image with a second feature point in the reference brain image comprises: Determining a third feature point and a fourth feature point; wherein the third feature point is located in a different target sub-image than the first feature point and is symmetrical with the first feature point in terms of brain structure; and the fourth feature point is located in a different reference sub-image than the second feature point and is symmetrical with the second feature point in terms of brain structure; Selecting a plurality of neighboring feature points corresponding to each of the first feature point to the fourth feature point; Calculating first mapping differences between a plurality of neighbor feature points corresponding to the first feature point and a plurality of neighbor feature points corresponding to the second feature point, and calculating second mapping differences between a plurality of neighbor feature points corresponding to the third feature point and a plurality of neighbor feature points corresponding to the fourth feature point; Based on a weighted sum of the first mapping difference and the second mapping difference, and when the sum result is less than a first threshold, determining that the first feature point and the second feature point are successfully matched; An image fusion module is used to fuse the target brain image and the reference brain image of the target patient based on all the first feature points and the second feature points that are successfully matched.

10. An electronic device, characterized in that: include: at least one control processor and a memory for communicatively coupling with the at least one control processor; The memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor to enable the at least one control processor to execute the cerebral hemorrhage medical image fusion method according to any one of claims 1 to 8.

Images