Liver and gall duct calculus positioning analysis method and system based on image processing

Abstract

The invention discloses a hepatic duct calculus positioning analysis method and system based on image processing, relates to the technical field of image data analysis, and comprises the steps of carrying out artifact and noise removal processing on a multi-modal medical image, reducing the influence of interference factors and providing a reliable basis for subsequent image feature extraction and analysis. The foreground area represents a hepatic duct area, the additional foreground area represents an organ area near the hepatic duct area, and the condition of calculus can also be reflected. The image features of all modes are integrated to determine the direct content and indirect content of the hepatic duct calculus, and the position information and related information of the hepatic duct calculus are speculated by combining the direct content and indirect content of the hepatic duct calculus; therefore, the condition and the position of the hepatic duct calculus are comprehensively reflected and analyzed through direct reflection and indirect reflection of the calculus on the image, the accuracy and the adaptability of hepatic duct calculus positioning analysis are improved, the reference reliability of doctors is guaranteed, and treatment and reference of the doctors are helped from the side.

Description

Technical Field

[0001] The present invention relates to the technical field of image data analysis, and particularly to a method and system for positioning and analyzing hepatolithiasis based on image processing. Background Art

[0002] In the clinical diagnosis of hepatolithiasis, traditional human eye recognition methods highly rely on doctors' experience and capabilities. Due to differences in the professional levels, visual sensitivities, and fatigue degrees of different doctors, the accuracy of stone positioning fluctuates significantly. For example, inexperienced doctors may overlook tiny stones or misjudge bile duct stenosis as stones, while senior doctors, although able to identify complex cases, are prone to visual fatigue after long-term film reading, increasing the risk of missed diagnosis. In addition, the complexity of the hepatobiliary duct structure and individual differences (such as bile duct branch variations and uneven stone densities) further exacerbate the subjectivity of human eye recognition. Such human errors may lead to deviations in treatment plans (such as improper selection of surgical paths) or an increased risk of postoperative recurrence. Image processing technology can significantly reduce subjectivity and provide standardized and reproducible stone positioning results through automated feature extraction and quantitative analysis (such as bile duct diameter measurement and stone density mapping), thereby assisting doctors in diagnosis.

[0003] In the prior art, the standards for human eye analysis of hepatolithiasis positioning are inconsistent and there are many errors, resulting in poor accuracy and adaptability of hepatolithiasis positioning analysis, which is not conducive to the treatment of hepatolithiasis.

[0004] Therefore, how to improve the accuracy and adaptability of hepatolithiasis positioning analysis is a technical problem to be solved at present. Summary of the Invention

[0005] The purpose of the present invention is to solve the problem of poor accuracy and adaptability in the positioning and analysis of hepatolithiasis in the prior art, and to propose a method for positioning and analyzing hepatolithiasis based on image processing. The method includes:

[0006] Obtain the basic information of the patient and multi-modal medical images, remove artifacts and noise from the multi-modal medical images, and divide the foreground area and background area in the medical images;

[0007] Based on the foreground area and the background area, confirm an additional foreground area in the background area, match and calibrate the foreground area and the additional foreground area of the multi-modal medical images, and extract the image features under the foreground area and the additional foreground area of each modal medical image;

[0008] Integrate the image features of all modalities to determine the direct content and indirect content of the hepatolithiasis;

[0009] Based on the direct and indirect content of hepatolithiasis, infer the location information and related information of hepatolithiasis, so as to assist doctors in referring to the analysis of hepatolithiasis.

[0010] In some embodiments of the present application, the removal of artifacts and noise from multi-modal medical images includes,

[0011] Using the SIFT or SURF algorithm to extract high-contrast feature points in the medical image, and matching the feature points of adjacent frames of medical images through the k-NN or FLANN algorithm;

[0012] Determine the signal-to-noise ratio of the medical image, obtain a RANSAC iteration number according to the signal-to-noise ratio, screen the matching pairs through the RANSAC iteration number, perform rigid transformation estimation, and define the transformation matrix;

[0013] Perform medical image alignment and interpolation, and perform iterative optimization to remove motion artifacts in the medical image;

[0014] Count the complex description parameters and resolution parameters in the medical image, integrate the complex description parameters to generate complex indexes, integrate the resolution parameters to generate resolution indexes, and determine the block size and search window size according to the complex indexes and resolution indexes respectively;

[0015] Based on the block size and search window size, perform block matching, weight calculation, filtering and iterative processing to complete the noise removal in the medical image.

[0016] In some embodiments of the present application, the foreground area and background area in the medical image are divided, including,

[0017] Perform feature point detection and seed point screening on the medical image, adjust the gray threshold according to the local mean and standard deviation of the gray level, and formulate a growth rule through the gray threshold and gradient direction consistency;

[0018] Perform topological constraints to divide the foreground area and background area in the medical image. The foreground area represents the hepatobiliary duct area, and the background area represents the image area in the medical image except for the hepatobiliary duct area.

[0019] In some embodiments of the present application, an additional foreground area is identified in the background area based on the foreground area and background area, including,

[0020] Select a proximity distance threshold according to the basic information of the patient, use U-Net to segment the organs near the hepatobiliary duct area, calculate the distance between each organ and the hepatobiliary duct area, and compare the distance and the proximity distance threshold to generate a candidate mask;

[0021] Extract texture features from the regions within the candidate mask, and use a preset classifier to classify the positions of the candidate masks to generate a verification mask. Take the regions of the verification mask as additional foreground regions.

[0022] In some embodiments of the present application, matching and calibrating the foreground regions and additional foreground regions of multi-modal medical images includes:

[0023] Calculate the joint entropy between the foreground regions and additional foreground regions in medical images of different modalities, initialize the transformation parameters, and iteratively optimize the joint entropy until convergence.

[0024] In some embodiments of the present application, the image features include one or more of morphological features, texture features, intensity features, and spatial features.

[0025] In some embodiments of the present application, integrating the image features of all modalities to determine the direct content and indirect content of hepatolithiasis includes:

[0026] Extract the direct content and indirect content of hepatolithiasis based on morphological features, texture features, intensity features, and spatial features. The direct content represents the content of the stones themselves, and the indirect content includes bile duct changes and the content of adjacent organ regions.

[0027] In some embodiments of the present application, inferring the location information and related information of hepatolithiasis by combining the direct content and indirect content of hepatolithiasis includes:

[0028] Determine all stone points based on the direct content of hepatolithiasis, analyze each direct content and each indirect content, generate the confidence levels corresponding to each direct content and each indirect content respectively, and integrate all the confidence levels to evaluate the confidence level of each stone point, thereby inferring the location information and related information of hepatolithiasis.

[0029] Correspondingly, the present application also provides a hepatolithiasis localization and analysis system based on image processing, including:

[0030] The first module is used to obtain the basic information of the patient and multi-modal medical images, remove artifacts and noise from the multi-modal medical images, and divide the foreground regions and background regions in the medical images;

[0031] The second module is used to identify additional foreground regions in the background region based on the foreground region and the background region, match and calibrate the foreground region and the additional foreground region of the multi-modal medical images, and extract the image features under the foreground region and the additional foreground region of each modality of medical images;

[0032] The third module is used to integrate the image features of all modalities to determine the direct content and indirect content of hepatolithiasis;

[0033] The fourth module is used to infer the location information and related information of hepatolithiasis by combining the direct content and indirect content of hepatolithiasis, so as to assist doctors in referring to the analysis of hepatolithiasis.

[0034] The beneficial effects of the present invention are as follows:

[0035] 1. Remove artifacts and noise from multi-modal medical images, reduce the influence of interference factors, and provide a reliable basis for subsequent image feature extraction and analysis. Identify additional foreground regions in the background region based on the foreground region and the background region. The foreground region represents the hepatobiliary duct region, and the additional foreground region represents the organ region near the hepatobiliary duct region, which can also reflect the situation of stones, thereby improving the reliability of hepatolithiasis localization analysis.

[0036] 2. Integrate the image features of all modalities to determine the direct content and indirect content of hepatolithiasis, and infer the location information and related information of hepatolithiasis by combining the direct content and indirect content of hepatolithiasis. Thus, comprehensively reflect and analyze the condition and location of hepatolithiasis from the direct manifestation and indirect manifestation of the stones on the image, improve the accuracy and adaptability of hepatolithiasis localization analysis, ensure the reliability of doctors' reference, and help doctors' treatment and reference indirectly. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a schematic flowchart of a method for hepatolithiasis localization analysis based on image processing proposed by the present invention;

[0038] Figure 2 It is a schematic structural diagram of a system for hepatolithiasis localization analysis based on image processing proposed by the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0039] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments.

[0040] Refer to Figure 1 , a method for hepatolithiasis localization analysis based on image processing includes the following steps:

[0041] Step S101, obtain the basic information of the patient and multi-modal medical images, remove artifacts and noise from the multi-modal medical images, and divide the foreground region and the background region in the medical images.

[0042] In this embodiment, basic information such as the patient's age, gender, medical history (such as cholecystitis, cholelithiasis), symptoms (such as abdominal pain, jaundice), and laboratory test results (such as liver function indicators, bilirubin levels) are collected to assist subsequent analysis. The acquisition of multi-modal medical images includes imaging types: CT (density information), MRI (soft tissue contrast), and ultrasound (real-time dynamics). Based on the rigid registration motion compensation technology, multiple frames of images are aligned. Three-dimensional non-local means filtering (NLM) is used to preserve edge details. The definition of hepatolithiasis is that hepatolithiasis refers to stones occurring in the intrahepatic biliary system, which may involve bile ducts at all levels in the liver, including bile canaliculi, interlobular bile ducts, segmental and lobar bile ducts in the liver, and may even involve the extrahepatic bile duct. The formation of these stones is usually related to biliary tract infections, cholestasis, biliary parasites, biliary tract anatomical variations, malnutrition, and environmental factors, etc. Hepatolithiasis may cause biliary tract obstruction, cholangitis, liver abscess, and long-term existence may also lead to serious complications such as biliary cirrhosis and cholangiocarcinoma.

[0043] In some embodiments of the present application, the removal of artifacts and noise from multi-modal medical images includes,

[0044] Using the SIFT or SURF algorithm to extract high-contrast feature points in the medical image, and matching the feature points of adjacent frames of medical images through the k-NN or FLANN algorithm;

[0045] Determine the signal-to-noise ratio of the medical image, obtain a RANSAC iteration number according to the signal-to-noise ratio, screen the matching pairs through the RANSAC iteration number, perform rigid transformation estimation, and define the transformation matrix;

[0046] Perform medical image alignment and interpolation, and perform iterative optimization to remove motion artifacts in the medical image;

[0047] Statistically obtain the complex description parameters and resolution parameters in the medical image, integrate the complex description parameters to generate complex indicators, integrate the resolution parameters to generate resolution indicators, and determine the block size and search window size according to the complex indicators and resolution indicators respectively;

[0048] Based on the block size and search window size, perform block matching, weight calculation, filtering, and iterative processing to complete the noise removal in the medical image.

[0049] In this embodiment, the reasons for motion artifacts are:

[0050] The patient's voluntary movements (such as breathing, heartbeat) cause misalignment of multiple frames of images.

[0051] Too long scanning time: increases the probability of motion, especially in abdominal scans.

[0052] Device synchronization problems: such as the desynchronization between CT and the respiratory gating device.

[0053] In this embodiment, SIFT (Scale Invariant Feature Transform) or SURF (Speeded Robust Feature) algorithm is used to extract high contrast feature points (such as liver edge, spine, ribs) in the image. The feature points of adjacent frames are matched by nearest neighbor search (k-NN) or FLANN (Approximate Nearest Neighbor) algorithm. Screening matching pairs: Use RANSAC (Random Sampling Consistency) algorithm to eliminate mismatched points. The number of RANSAC iterations, the reliability of eliminating mismatched points, and the number of iterations is increased when the signal-to-noise ratio is low. Assuming that the motion is a rigid transformation (translation, rotation, scaling), the transformation matrix is defined. The transformation matrix is solved by least squares method or singular value decomposition (SVD). The transformation matrix is applied to the coordinates of the target frame to generate an aligned image. For non-integer coordinate points, bilinear interpolation or cubic spline interpolation is used to calculate pixel values. Start with low-resolution images and gradually transition to high resolution to reduce the amount of calculation. Rough alignment is performed based on affine transformation (translation, rotation, scaling). Use mutual information (MI) or normalized cross correlation (NCC) as the optimization target to iteratively update the transformation matrix.

[0054] In this embodiment, a fixed-size block (e.g., 7×7×7) is extracted around the target pixel, and a search range (e.g., 21×21×21) is defined in the image to find similar blocks. The Euclidean distance is used to calculate the similarity between blocks and filter similar blocks. The weight is calculated based on the block similarity, the target pixel is filtered, and the image is filtered multiple times (usually 1-3 times) to gradually suppress noise.

[0055] In this embodiment, the block size is used to balance local details and global similarity. The complexity of the image affects the size of the block, and the search window size is used to control the number of similar blocks (affecting the amount of calculation). High complexity: contains a large number of small structures (such as the pulmonary vascular network and the liver bile duct tree). Low complexity: the structure is uniform or single (such as uniform adipose tissue, cerebrospinal fluid). Complex description parameters (edge density gradient, texture features, tissue contrast, etc.) and resolution parameters (spatial resolution and contrast resolution), spatial resolution: the number of pixels per unit size (such as the 512×512 matrix of CT). Contrast resolution: the ability to distinguish small grayscale differences (such as soft tissue contrast of MRI). Integrate the complex description parameters to generate a complex index (ComplexityIndex), and the calculation formula is as follows:

[0056]

[0057] Among them, C is the complexity index, n is the number of complex description parameters, α i is the combined weight of the i-th complex description parameter, Z i is the size of the i-th complex description parameter, max(α i Z i ) is αi Z i the maximum value in, min(α i Z i ), and min(α i Z i is the minimum value in, and k1 is the first constant. It represents the correction of the sum of complex description parameters of the average value determined by the maximum and minimum values to the average value of the sum type (1 / n - 1, and the average value should be a little larger), which can better reflect the complexity.

[0058] In some embodiments of the present application, dividing the foreground region and the background region in the medical image includes:

[0059] Performing feature point detection and seed point screening on the medical image, adjusting the gray threshold according to the local mean and standard deviation of the gray level, and formulating a growth rule through the gray threshold and the gradient direction consistency;

[0060] Performing topological constraints to divide the foreground region and the background region in the medical image, where the foreground region represents the hepatobiliary duct region, and the background region represents the image region in the medical image except for the hepatobiliary duct region.

[0061] In this embodiment, the process of dividing the foreground region and the background region is as follows:

[0062] Feature point detection:

[0063] Using the Hessian matrix to detect the tubular structure, and setting the response threshold to 0.1.

[0064] Performing non-maximum suppression on the detected points to retain the local maximum value.

[0065] Seed point screening:

[0066] Screening candidate points according to the gray value range (such as CT 50 - 80 HU, MRI T2 > 200).

[0067] Distinguishing foreground and background seed points through K-means clustering (K = 2).

[0068] Growth criterion:

[0069] Gray level similarity: The gray level difference between adjacent pixels is less than the threshold (such as CT 10 HU, MRI 30), and the threshold is adjusted according to the local mean and standard deviation. The formula is as follows:

[0070] T(x) = μ(x) + kσ(x);

[0071] Among them, T(x) is the dynamic gray threshold, representing the gray threshold at image position x, μ(x) is the local mean, representing the average value of the gray values of all pixels within the local window centered at position x, reflecting the average brightness of the local area. σ(x) is the local standard deviation, representing the degree of dispersion of the gray values within the local window (i.e., the degree of brightness change), reflecting the contrast or texture complexity of the local area. k is the adjustment coefficient, controlling the contribution degree of the local standard deviation to the threshold. The larger k is, the more sensitive the threshold is to the change of local contrast; the smaller k is, the closer the threshold is to the local mean.

[0072] Gradient direction consistency: The difference in gradient directions between adjacent pixels is less than the threshold (such as 20°).

[0073] Topological constraint:

[0074] Use morphological closing operation (structural element 3×3×3) to fill small holes.

[0075] Remove isolated noise through connected component analysis (area threshold > 50 voxels).

[0076] Step S102, based on the foreground region and the background region, identify the additional foreground region in the background region, match and calibrate the foreground region and the additional foreground region of the multimodal medical image, and extract the image features under the foreground region and the additional foreground region of each modality medical image.

[0077] In this embodiment, the additional foreground region is the organ near the hepatobiliary duct region. The positions and shapes of adjacent organs (such as the gallbladder, pancreas, duodenum) provide references for the localization of hepatobiliary duct stones and the situation of the stones. For example, changes in the position and shape of the gallbladder may indicate gallbladder stones or cholecystitis, which in turn affect the analysis of extrahepatic bile duct stones. Changes in the bile duct (dilation, wall thickening, stenosis) are closely related to the situation of the stones, and the manifestations of these changes on medical images can reflect the presence, location, size of the stones and their effects on the bile duct.

[0078] 1. Relationship between bile duct dilation and stones

[0079] Reason:

[0080] The stone blocks the bile duct, resulting in obstruction of bile drainage, increased pressure in the bile duct, and causing bile duct dilation.

[0081] Whether it is intrahepatic bile duct stones or extrahepatic bile duct stones, they may cause dilation of the corresponding bile ducts.

[0082] Imaging manifestation:

[0083] The diameter of the bile duct widens: On ultrasound, CT or MRI images, it can be seen that the diameter of the bile duct is significantly larger than the normal range.

[0084] Intraluminal bile shadow: The dilated bile duct is filled with low-density (CT) or isodense (MRI) bile shadow.

[0085] Upstream bile duct dilation: The upstream bile duct at the site of stone obstruction is more significantly dilated, forming the "soft vine sign" (the bile duct is dilated in a beaded pattern).

[0086] Analysis of stone conditions:

[0087] Stone location: Stones may be present downstream of the most significantly dilated bile duct.

[0088] Stone size: The larger the stone, the more severe the obstruction and the more obvious the bile duct dilation.

[0089] Degree of obstruction: When there is complete obstruction, the bile duct dilation is more obvious; when there is partial obstruction, the degree of dilation may be less.

[0090] 2. Relationship between bile duct wall thickening and stones

[0091] Reasons:

[0092] Long-term existing stones stimulate the bile duct wall, causing inflammatory reactions and fibrous tissue hyperplasia, resulting in bile duct wall thickening.

[0093] Repeated friction of stones against the bile duct wall may also cause bile duct wall damage and thickening.

[0094] Imaging manifestations:

[0095] Uneven thickening: The thickness of the bile duct wall is uneven, and it may be thicker locally.

[0096] Circular thickening: The bile duct wall shows circular thickening, and the lumen becomes narrower.

[0097] Enhancement manifestation: On enhanced CT or MRI, the thickened bile duct wall may show mild enhancement.

[0098] Analysis of stone conditions:

[0099] Chronic stone stimulation: Bile duct wall thickening indicates that the stones have been present for a long time and may be chronic stones.

[0100] Degree of inflammation: The more obvious the thickening, the more severe the inflammatory reaction and the greater the impact of the stones on the bile duct.

[0101] Risk of malignant lesions: Long-term bile duct wall thickening requires vigilance for the possibility of malignant lesions (such as bile duct cancer), and further evaluation is needed in combination with other examinations (such as tumor markers, pathological biopsy).

[0102] 3. Relationship between bile duct stricture and stones

[0103] Reasons:

[0104] Long-term obstruction of calculi leads to fibrosis and scar formation of the bile duct wall, causing bile duct stenosis.

[0105] The calculi compress the bile duct wall, resulting in local ischemia and necrosis, and further causing stenosis.

[0106] Imaging findings:

[0107] Lumen narrowing or interruption: On the image, the lumen of the bile duct is significantly narrowed or even interrupted.

[0108] Upstream bile duct dilation: The upstream bile duct of the stenosis site is significantly dilated, forming a "cup mouth sign" (the end of the bile duct suddenly becomes thinner).

[0109] Calculus shadow: The calculus shadow may be directly shown at the stenosis site, or the calculus is located downstream of the stenosis.

[0110] Analysis of calculus conditions:

[0111] Calculus size and location: Larger calculi are more likely to cause bile duct stenosis, and the stenosis site is usually upstream of the calculus.

[0112] Obstruction time: Stenosis indicates that the calculus has been present for a long time, and it may be a chronic obstruction.

[0113] Complication risk: Bile duct stenosis may lead to cholestasis, cholangitis, and even biliary cirrhosis.

[0114] Based on comprehensive analysis, the downstream of bile duct dilation, the area with thickened bile duct wall, and the site of bile duct stenosis may all be the locations where the calculi are located. Combining multi-modal imaging (such as ultrasound, CT, MRI) can improve the accuracy of calculus localization. The degree of bile duct dilation and the severity of bile duct stenosis can indirectly reflect the size and number of calculi. Larger calculi or multiple calculi are more likely to cause obvious changes in the bile duct. Thickening of the bile duct wall and bile duct stenosis indicate that the calculus has been present for a long time and may be a chronic lesion. Acute calculus obstruction usually presents as bile duct dilation, but the thickening and stenosis of the bile duct wall are not obvious.

[0115] In some embodiments of the present application, an additional foreground area is identified in the background area based on the foreground area and the background area, including

[0116] selecting a proximity distance threshold according to the basic information of the patient, using U-Net to segment the organs near the hepatobiliary duct area, calculating the distances between each organ and the hepatobiliary duct area, and comparing the distances with the proximity distance threshold to generate a candidate mask;

[0117] extracting texture features from the areas within the candidate mask, and using a preset classifier to classify the positions of the candidate mask to generate a verification mask, and taking the area of the verification mask as the additional foreground area.

[0118] In this embodiment, the spatial constraint of the anatomical correlation method is combined with the texture analysis ability of the feature matching method to improve the screening accuracy of additional foreground regions (such as the gallbladder and liver parenchyma). Anatomical correlation method: Quickly locate the organs adjacent to the hepatobiliary duct through spatial distance. Feature matching method: Use texture features to further verify the authenticity of the candidate region. Joint strategy: First, narrow the search range through the anatomical correlation method, and then accurately classify using the feature matching method.

[0119] Organ segmentation:

[0120] Use U-Net to segment organs such as the gallbladder and liver parenchyma, and the Dice coefficient needs to be >0.85.

[0121] Distance calculation:

[0122] Calculate the Euclidean distance from each organ voxel to the surface of the hepatobiliary duct.

[0123] Threshold setting: Set it considering some basic information of the patient (body fat, height, etc.).

[0124] Generate candidate mask:

[0125] Retain the organ regions with a distance < distance threshold to generate a preliminary screening mask (the region determined based on the distance).

[0126] Verification by feature matching method

[0127] Feature extraction:

[0128] Extract texture features from the regions of the candidate mask:

[0129] GLCM: Window 11×11, calculate energy, contrast, and correlation.

[0130] LBP: Radius 3 pixels, number of points 8.

[0131] Classifier training:

[0132] Use SVM (RBF kernel, γ = 0.01) to train the classification model.

[0133] Training data: Labeled regions such as the gallbladder, liver parenchyma, and blood vessels.

[0134] Region classification:

[0135] Classify the voxels within the candidate mask to generate a verification mask.

[0136] In some embodiments of the present application, the foreground regions and additional foreground regions of multi-modal medical images are matched and calibrated, including,

[0137] Calculate the joint entropy of the foreground regions and additional foreground regions in different-modal medical images, initialize the transformation parameters, and iteratively optimize the joint entropy until convergence.

[0138] In this embodiment, the rigid registration process:

[0139] Reference selection: Taking the liver contour as the reference (because the liver has obvious and stable features in abdominal images).

[0140] Algorithm: Mutual Information Maximization (MI).

[0141] Principle: By maximizing the joint entropy of two images, the optimal spatial transformation (rotation, translation) is found.

[0142] Advantages: Insensitive to gray-scale differences between modalities (such as the HU value of CT and the signal intensity of MRI).

[0143] Implementation steps:

[0144] Extract the liver contour (threshold segmentation + morphological operations can be used).

[0145] Initialize the transformation parameters (such as translation vector, rotation matrix).

[0146] Iteratively optimize MI until convergence.

[0147] In some embodiments of the present application, the image features include one or more of morphological features, texture features, intensity features, and spatial features.

[0148] In this embodiment, the foreground region features include but are not limited to the following:

[0149] Texture features:

[0150] Gray-level Co-occurrence Matrix (GLCM):

[0151] Extract features such as energy, contrast, correlation, and entropy.

[0152] Parameters: Window 11×11 pixels, directions 0°, 45°, 90°, 135°, gray levels 32.

[0153] Local Binary Pattern (LBP):

[0154] Radius 3 pixels, number of points 8, calculate histogram features.

[0155] Morphological features:

[0156] Area: Number of pixels in the stone area.

[0157] Perimeter: Length of the stone boundary.

[0158] Circularity: Perimeter² / (4π × Area) (stones are mostly approximately circular).

[0159] Intensity features:

[0160] Average HU value: The average CT value in the stone area (typical value > 100 HU).

[0161] Maximum HU value: The highest density value inside the stone.

[0162] Extraction of additional foreground region features

[0163] Gallbladder wall features:

[0164] Thickness: Measure the average thickness of the gallbladder wall (normal ≤ 3 mm).

[0165] Texture abnormality: Entropy value of LBP histogram (the entropy value increases during inflammation).

[0166] Liver parenchyma features:

[0167] Density standard deviation: Standard deviation of CT values (normal ≤ 15 HU, increases when the stone spreads).

[0168] GLCM contrast: Reflects density inhomogeneity.

[0169] Spatial relationship features:

[0170] Distance from the stone to the gallbladder: Euclidean distance (the distance decreases when the stone causes cholecystitis).

[0171] Distance from the stone to the liver parenchyma: Assess the risk of stone spread.

[0172] In step S103, integrate the image features of all modalities to determine the direct content and indirect content of hepatolithiasis.

[0173] In some embodiments of the present application, integrate the image features of all modalities to determine the direct content and indirect content of hepatolithiasis, including,

[0174] Extract the direct content and indirect content of hepatolithiasis according to morphological features, texture features, intensity features, and spatial features. The direct content represents the content of the stone itself, and the indirect content includes bile duct changes and the content of adjacent organ regions.

[0175] In this embodiment, analyze all the image features and convert them into the direct manifestation and indirect manifestation of the stone. Direct content: The features of the stone itself (such as position, size, density). Indirect content: Pathological changes caused by the stone (such as thickening of the gallbladder wall, bile duct changes (such as degree of dilation), and the content of adjacent organ regions (such as abnormal liver parenchyma density).

[0176] It can be achieved by fusing features:

[0177] Early fusion: Directly extract combined features after registering multimodal images (such as the combination of HU values and signal intensities of CT + MRI).

[0178] Late fusion: After extracting the features of each modality separately, model fusion is performed (such as weighted summation, multi-layer perceptron).

[0179] Associate features with pathology through statistical models or machine learning.

[0180] Step S104: Combine the direct and indirect content of hepatolithiasis to infer the location information and related information of hepatolithiasis, so as to assist doctors in referring to the analysis of hepatolithiasis.

[0181] In this embodiment, all stone points are identified according to the direct content (such as the stone density threshold), or all possible existing stone points can be identified according to the image grayscale, where it may be a stone (i.e., the location), or it may be other errors. Analyze each direct content and indirect content to infer the probability or possibility of each stone point, and describe it with a comprehensive confidence level.

[0182] In some embodiments of the present application, the location information and related information of hepatolithiasis are inferred by combining the direct and indirect content of hepatolithiasis, including

[0183] Determine all stone points according to the direct content of hepatolithiasis, analyze each direct content and each indirect content, generate the confidence level corresponding to each direct content and each indirect content, and integrate all confidence levels to evaluate the confidence level of each stone point, so as to infer the location information and related information of hepatolithiasis.

[0184] In this embodiment, analyze each direct content (such as the stone size) and indirect content (such as bile duct dilation) to generate the corresponding confidence level. Infer and output the stone position coordinates, confidence level, type (such as calcified stone) and risk level (such as high recurrence risk). The specific implementation formula for integrating all confidence levels to evaluate the confidence level of each stone point (Confidence Level of Stone Points) is:

[0185]

[0186] Among them, is the confidence level of the j1-th stone point, m1 is the number of direct contents of the j1-th stone point, m2 is the number of indirect contents of the j1-th stone point, are the combination weights of the j2-th direct content and the j3-th indirect content respectively, are the confidence levels of the j2-th direct content and the j3-th indirect content of the j1-th stone point respectively, is the second constant of the j1-th stone point, It represents the correction of the confidence synthesis of indirect content to the confidence synthesis of direct content, and the second constant is used to balance the magnitude of the correction function.

[0187] Correspondingly, the present application also provides a hepatolithiasis localization and analysis system based on image processing, as Figure 2 shown, including,

[0188] The first module is used to obtain the basic information of the patient and multi-modal medical images, remove artifacts and noise from the multi-modal medical images, and divide the foreground region and background region in the medical images;

[0189] The second module is used to confirm additional foreground regions in the background region based on the foreground region and the background region, match and calibrate the foreground region and the additional foreground regions of the multi-modal medical images, and extract the image features under the foreground region and the additional foreground regions of each modal medical image;

[0190] The third module is used to integrate the image features of all modalities to determine the direct content and indirect content of hepatolithiasis;

[0191] The fourth module is used to combine the direct content and indirect content of hepatolithiasis to infer the location information and related information of hepatolithiasis, so as to assist doctors in referring for hepatolithiasis analysis.

[0192] The beneficial effects of the present invention are as follows:

[0193] 1. Remove artifacts and noise from multi-modal medical images, reduce the influence of interference factors, and provide a reliable basis for subsequent image feature extraction and analysis. Confirm additional foreground regions in the background region based on the foreground region and the background region. The foreground region represents the hepatobiliary duct region, and the additional foreground region represents the organ region near the hepatobiliary duct region, which can also reflect the situation of stones, thereby improving the reliability of hepatolithiasis localization and analysis.

[0194] 2. Integrate the image features of all modalities to determine the direct content and indirect content of hepatolithiasis, and combine the direct content and indirect content of hepatolithiasis to infer the location information and related information of hepatolithiasis. Thus, comprehensively reflect and analyze the condition and location of hepatolithiasis from the direct manifestation and indirect manifestation of the stones on the image, improve the accuracy and adaptability of hepatolithiasis localization and analysis, ensure the reliability of doctors' reference, and assist doctors' treatment and reference from the side.

[0195] Through the description of the above embodiments, those skilled in the art can clearly understand that the present invention can be implemented by hardware or by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various implementation scenarios of the present invention.

[0196] Those skilled in the art can understand that the drawings are only schematic diagrams of a preferred implementation scenario, and the modules or processes in the drawings are not necessarily essential for implementing the present invention.

[0197] Those skilled in the art can understand that the modules in the system in the implementation scenario can be distributed in the system of the implementation scenario according to the description of the implementation scenario, or can be correspondingly changed and located in one or more systems different from the present implementation scenario. The modules in the above implementation scenario can be combined into one module, or can be further split into multiple sub-modules.

[0198] As mentioned above, the above are only the preferred specific implementation manners of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, making equivalent replacements or changes, should be covered by the protection scope of the present invention.

Claims

1. A method for positioning and analyzing hepatolithiasis based on image processing, characterized in that, including obtaining the basic information and multi-modal medical images of the patient, removing artifacts and noise from the multi-modal medical images, and dividing the foreground region and background region in the medical images; identifying an additional foreground region in the background region based on the foreground region and background region, matching and calibrating the foreground region and the additional foreground region of the multi-modal medical images, and extracting the image features under the foreground region and the additional foreground region of each modal medical image; integrating the image features of all modalities to determine the direct content and indirect content of hepatolithiasis; speculating the location information and related information of hepatolithiasis by combining the direct content and indirect content of hepatolithiasis, so as to assist doctors in referring to the analysis of hepatolithiasis.

2. The method for positioning and analyzing hepatolithiasis based on image processing according to claim 1, wherein removing artifacts and noise from the multi-modal medical images including using the SIFT or SURF algorithm to extract high-contrast feature points in the medical images, and matching the feature points of adjacent frames of medical images through the k-NN or FLANN algorithm; determining the signal-to-noise ratio of the medical images, obtaining a RANSAC iteration number according to the signal-to-noise ratio, screening the matching pairs through the RANSAC iteration number, performing rigid transformation estimation, and defining the transformation matrix; performing medical image alignment and interpolation, and performing iterative optimization to remove motion artifacts in the medical images; counting the complex description parameters and resolution parameters in the medical images, integrating the complex description parameters to generate complex indicators, integrating the resolution parameters to generate resolution indicators, and determining the block size and search window size according to the complex indicators and resolution indicators respectively; implementing block matching, weight calculation, filtering and iterative processing based on the block size and search window size to complete the noise removal in the medical images.

3. The method for positioning and analyzing hepatolithiasis based on image processing according to claim 1, wherein dividing the foreground region and background region in the medical images, including performing feature point detection and seed point screening on the medical images, adjusting the gray threshold according to the local mean and standard deviation of the gray level, and formulating a growth rule through the gray threshold and gradient direction consistency; performing topological constraints to divide the foreground region and background region in the medical images, where the foreground region represents the hepatobiliary duct region, and the background region represents the image region in the medical images other than the hepatobiliary duct region.

4. The method for positioning and analyzing hepatolithiasis based on image processing according to claim 3, wherein identifying an additional foreground region in the background region based on the foreground region and background region, including selecting a proximity distance threshold according to the basic information of the patient, using U-Net to segment the organs near the hepatobiliary duct region, calculating the distance between each organ and the hepatobiliary duct region, and comparing the distance with the proximity distance threshold to generate a candidate mask; extracting texture features from the regions within the candidate mask, and using a preset classifier to classify the positions of the candidate mask to generate a verification mask, and taking the regions of the verification mask as the additional foreground region.

5. The method for positioning and analyzing hepatolithiasis based on image processing according to claim 1, wherein matching and calibrating the foreground region and the additional foreground region of the multi-modal medical images including calculating the joint entropy of the foreground region and the additional foreground region between different modal medical images, initializing the transformation parameters, and iteratively optimizing the joint entropy until convergence.

6. The method for positioning and analyzing hepatolithiasis based on image processing according to claim 1, characterized in that, The image features include one or more of morphological features, texture features, intensity features and spatial features.

7. The method for positioning and analyzing hepatolithiasis based on image processing according to claim 6, wherein, Integrate the image features of all modalities to determine the direct and indirect content of hepatolithiasis, including Extract the direct and indirect content of hepatolithiasis according to morphological features, texture features, intensity features, and spatial features. The direct content represents the content of the stone itself, and the indirect content includes bile duct changes and the content of adjacent organ regions.

8. The method for positioning and analyzing hepatolithiasis based on image processing according to claim 7, wherein, Combine the direct and indirect content of hepatolithiasis to infer the location information and related information of hepatolithiasis, including Determine all stone points according to the direct content of hepatolithiasis, analyze each direct content and each indirect content, generate the confidence level corresponding to each direct content and each indirect content respectively, and integrate all confidence levels to evaluate the confidence level of each stone point, so as to infer the location information and related information of hepatolithiasis.

9. A hepatolithiasis positioning and analysis system based on image processing, characterized in that, Including The first module is used to obtain the basic information of the patient and multi-modal medical images, perform artifact and noise removal processing on the multi-modal medical images, and divide the foreground region and background region in the medical images; The second module is used to identify additional foreground regions in the background region based on the foreground region and background region, match and calibrate the foreground region and additional foreground regions of the multi-modal medical images, and extract the image features under the foreground region and additional foreground regions of each modality medical image; The third module is used to integrate the image features of all modalities to determine the direct and indirect content of hepatolithiasis; The fourth module is used to combine the direct and indirect content of hepatolithiasis to infer the location information and related information of hepatolithiasis, so as to assist doctors in referring to the analysis of hepatolithiasis.

Images