Image processing-based hepatobiliary stone positioning analysis method and system

Abstract

The application discloses a liver and bile duct stone positioning analysis method and system based on image processing, relates to the technical field of image data analysis, and comprises the following steps: removing artifacts and noise from multi-modal medical images, reducing the influence of interference factors, and providing a reliable basis for subsequent image feature extraction and analysis. The foreground region represents the liver and bile duct region, and the additional foreground region represents the organ region near the liver and bile duct region, which can also reflect the condition of the stone. The image features of all modes are integrated to determine the direct content and indirect content of the liver and bile duct stone, and the position information and related information of the liver and bile duct stone are inferred by combining the direct content and indirect content of the liver and bile duct stone, so that the direct and indirect manifestations of the stone on the image are comprehensively reflected and analyzed to reflect and analyze the condition and position of the liver and bile duct stone, the accuracy and adaptability of liver and bile duct stone positioning analysis are improved, the reliability of the doctor's reference is ensured, and the treatment and reference of the doctor are indirectly helped.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image data analysis, in particular to a liver and bile duct stone positioning analysis method and system based on image processing. BACKGROUND

[0002] In the clinical diagnosis of liver and bile duct stones, the traditional human eye recognition method is highly dependent on the experience and ability of doctors. Due to the differences in professional level, visual sensitivity and fatigue level of different doctors, the accuracy of stone positioning fluctuates significantly. For example, inexperienced doctors may ignore small stones or misjudge bile duct stenosis as stones, while experienced doctors can identify complex cases, but long-time reading of films can cause visual fatigue and increase the risk of missed diagnosis. In addition, the complexity of liver and bile duct structure and individual differences (such as bile duct branch variation, uneven stone density) further exacerbate the subjectivity of human eye recognition. Such human error can lead to deviation of treatment plan (such as improper selection of surgical path) or increase the risk of postoperative recurrence. Image processing technology can significantly reduce subjectivity by automatic feature extraction and quantitative analysis (such as bile duct diameter measurement, stone density mapping), providing standardized and repeatable stone positioning results, thereby assisting doctors in diagnosis.

[0003] In the prior art, the standards for human eye analysis of liver and bile duct stone positioning are not uniform, and there are many errors, which makes the accuracy and adaptability of liver and bile duct stone positioning analysis poor, which is not conducive to the treatment of liver and bile duct stones.

[0004] Therefore, how to improve the accuracy and adaptability of liver and bile duct stone positioning analysis is a technical problem to be solved at present. SUMMARY

[0005] The purpose of the present application is to solve the problem of poor accuracy and adaptability of liver and bile duct stone positioning analysis in the prior art, and to provide a liver and bile duct stone positioning analysis method based on image processing. The method comprises,

[0006] Obtaining the basic information of the patient and the multi-modal medical image, removing the artifacts and noise of the multi-modal medical image, and dividing the foreground region and the background region in the medical image;

[0007] Confirming the additional foreground region in the background region based on the foreground region and the background region, matching and calibrating the foreground region and the additional foreground region of the multi-modal medical image, and extracting the image features under the foreground region and the additional foreground region of each modal medical image;

[0008] Integrating all modal image features to determine the direct content and indirect content of liver and bile duct stones;

[0009] The position information and related information of the hepatobiliary duct stones are inferred based on the direct content and the indirect content of the hepatobiliary duct stones, so as to assist doctors in analyzing the hepatobiliary duct stones.

[0010] In some embodiments of the present application, the removal of artifacts and noise in multi-modal medical images includes,

[0011] High-contrast feature points in the medical images are extracted using SIFT or SURF algorithms, and the feature points of adjacent frame medical images are matched using k-NN or FLANN algorithms.

[0012] The signal-to-noise ratio of the medical images is determined, and a RANSAC iteration number is obtained according to the signal-to-noise ratio, the matching pairs are filtered through the RANSAC iteration number, the rigid transformation estimation is performed, and the transformation matrix is defined.

[0013] Medical image alignment and interpolation are performed, and iterative optimization is performed to remove motion artifacts in the medical images.

[0014] Complex description parameters and resolution parameters in the medical images are counted, complex indicators are generated by integrating the complex description parameters, and resolution indicators are generated by integrating the resolution parameters, and the block size and the search window size are determined according to the complex indicators and the resolution indicators, respectively.

[0015] Based on the block size and the search window size, block matching, weight calculation, filtering and iteration processing are realized to complete the noise removal in the medical images.

[0016] In some embodiments of the present application, the foreground region and the background region in the medical images are divided, including,

[0017] Feature point detection and seed point filtering are performed on the medical images, the gray threshold is adjusted according to the local mean and standard deviation of the gray value, and the growth rule is formulated according to the consistency of the gray threshold and the gradient direction.

[0018] Topological constraints are performed to divide the foreground region and the background region in the medical images, the foreground region represents the hepatobiliary duct region, and the background region represents the image region in the medical images except the hepatobiliary duct region.

[0019] In some embodiments of the present application, the additional foreground region is confirmed in the background region based on the foreground region and the background region, including,

[0020] A proximity distance threshold is selected according to the basic information of the patient, organs near the hepatobiliary duct region are segmented using U-Net, distances between the organs and the hepatobiliary duct region are calculated, and candidate masks are generated by comparing the distances and the proximity distance threshold.

[0021] extracting texture features in the region within the candidate mask, and using a preset classifier to perform position classification on the candidate mask to generate a verification mask, and taking the region of the verification mask as an additional foreground region.

[0022] In some embodiments of the present application, the foreground region and the additional foreground region of the multi-modal medical image are matched and calibrated, including,

[0023] The joint entropy of the foreground region and the additional foreground region between different modal medical images is calculated, the transformation parameter is initialized, and the joint entropy is iteratively optimized until convergence.

[0024] In some embodiments of the present application, the image features include one or more of morphological features, texture features, intensity features, and spatial features.

[0025] In some embodiments of the present application, all modal image features are integrated to determine the direct content and indirect content of the hepatobiliary stones, including,

[0026] The direct content and indirect content of the hepatobiliary stones are extracted according to the morphological features, texture features, intensity features, and spatial features, the direct content represents the content of the stones themselves, and the indirect content includes the changes of the bile duct and the content of the adjacent organ region.

[0027] In some embodiments of the present application, the position information and related information of the hepatobiliary stones are inferred in combination with the direct content and indirect content of the hepatobiliary stones, including,

[0028] All stone points are determined according to the direct content of the hepatobiliary stones, each direct content and each indirect content is analyzed, the confidence of each direct content and each indirect content is generated, all confidence is integrated to evaluate the confidence of each stone point, and thus the position information and related information of the hepatobiliary stones are inferred.

[0029] Correspondingly, the present application also provides a hepatobiliary stone positioning analysis system based on image processing, including,

[0030] The first module is used for acquiring the basic information of a patient and multi-modal medical images, removing artifacts and noise from the multi-modal medical images, and dividing the foreground region and the background region in the medical images;

[0031] The second module is used for confirming an additional foreground region in the background region based on the foreground region and the background region, matching and calibrating the foreground region and the additional foreground region of the multi-modal medical images, and extracting image features under the foreground region and the additional foreground region of each modal medical image;

[0032] The third module is used for integrating all modal image features to determine the direct content and indirect content of the hepatobiliary stones.

[0033] A fourth module is configured to infer the position information and related information of the hepatobiliary calculi based on the direct content and indirect content of the hepatobiliary calculi, so as to assist doctors in hepatobiliary calculi analysis.

[0034] The present application has the following advantages:

[0035] 1. The multi-modal medical image is subjected to the removal of artifacts and noise, the influence of interference factors is reduced, and a reliable basis is provided for subsequent image feature extraction and analysis. The additional foreground region is confirmed in the background region based on the foreground region and the background region, the foreground region represents the hepatobiliary region, the additional foreground region represents the organ region near the hepatobiliary region, and the condition of the calculi can also be reflected, so that the reliability of hepatobiliary calculi positioning analysis is improved.

[0036] 2. The image features of all modes are integrated to determine the direct content and indirect content of the hepatobiliary calculi, and the position information and related information of the hepatobiliary calculi are inferred based on the direct content and indirect content of the hepatobiliary calculi, so that the direct and indirect manifestations of the calculi on the image are comprehensively reflected and analyzed to reflect and analyze the condition and position of the hepatobiliary calculi, the accuracy and adaptability of hepatobiliary calculi positioning analysis are improved, the reliability of the doctor's reference is ensured, and the treatment and reference of the doctor are indirectly helped. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 A flowchart of the hepatobiliary calculi positioning analysis method based on image processing is provided.

[0038] Figure 2 A structure diagram of the hepatobiliary calculi positioning analysis system based on image processing is provided. DETAILED DESCRIPTION

[0039] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all.

[0040] Reference Figure 1 The hepatobiliary calculi positioning analysis method based on image processing comprises the following steps:

[0041] In step S101, the basic information and multi-modal medical images of the patient are acquired, the multi-modal medical images are subjected to the removal of artifacts and noise, and the foreground region and background region in the medical images are divided.

[0042] In this embodiment, the basic information of the patient is collected, such as age, gender, medical history (such as cholecystitis, cholelithiasis), symptoms (such as abdominal pain, jaundice), laboratory test results (such as liver function indicators, bilirubin levels), etc. to help subsequent analysis. Multi-modal medical image acquisition includes image types: CT (density information), MRI (soft tissue contrast), ultrasound (real-time dynamic). Motion compensation technology based on rigid registration aligns multiple frames of images. Three-dimensional non-local mean filtering (NLM) preserves edge details. Definition of hepatolithiasis: hepatolithiasis refers to stones occurring in the intrahepatic biliary system, which can involve all levels of intrahepatic bile ducts, including capillary bile ducts, interlobular bile ducts, hepatic segment and lobar bile ducts, and even extrahepatic bile ducts. The formation of these stones is usually related to biliary infection, bile stasis, biliary parasites, biliary anatomical variations, malnutrition, and environmental factors. Hepatolithiasis can cause biliary obstruction, cholangitis, liver abscess, and long-term presence can also cause serious complications such as cholestatic cirrhosis and cholangiocarcinoma.

[0043] In some embodiments of the present application, the multi-modal medical image is subjected to artifact and noise removal processing, including,

[0044] High-contrast feature points in the medical image are extracted using SIFT or SURF algorithms, and the feature points of adjacent frame medical images are matched through k-NN or FLANN algorithms;

[0045] The signal-to-noise ratio of the medical image is determined, and a RANSAC iteration number is obtained according to the signal-to-noise ratio. The matching pairs are filtered through the RANSAC iteration number, the rigid transformation is estimated, and the transformation matrix is defined;

[0046] Medical image alignment and interpolation are performed, and iterative optimization is performed to remove motion artifacts in the medical image;

[0047] The complexity description parameters and resolution parameters in the medical image are counted, the complexity description parameters are integrated to generate a complexity index, the resolution parameters are integrated to generate a resolution index, and the block size and search window size are determined according to the complexity index and resolution index, respectively;

[0048] Based on the block size and search window size, block matching, weight calculation, filtering and iteration processing are realized to complete the noise removal in the medical image.

[0049] In this embodiment, the causes of motion artifacts are:

[0050] Patient voluntary movement (such as breathing, heartbeat) causes multiple image misplacement.

[0051] Long scanning time: increases the probability of motion, especially in abdominal scanning.

[0052] Device synchronization problem: such as CT and respiratory gating device are not synchronized.

[0053] In this embodiment, SIFT (Scale-Invariant Feature Transform) or SURF (Speeded-Up Robust Features) algorithm is used to extract high-contrast feature points (e.g. liver edge, spine, rib) in the image. Feature points of adjacent frames are matched by k-NN (k-Nearest Neighbor) or FLANN (Fast Library for Approximate Nearest Neighbors) algorithm. Matching pairs are filtered: RANSAC (RANdom SAmple Consensus) algorithm is used to remove mismatched points. Iteration number of RANSAC, reliability of removing mismatched points, and signal-to-noise ratio are low, and the iteration number is increased. Assuming that the motion is a rigid transformation (translation, rotation, scaling), a transformation matrix is defined. The transformation matrix is solved by least squares or singular value decomposition (SVD). The transformation matrix is applied to the coordinates of the target frame to generate the aligned image. For non-integer coordinate points, bilinear interpolation or cubic spline interpolation is used to calculate the pixel value. Registration starts from low-resolution images, gradually transitions to high-resolution, reducing the amount of calculation. Affine transformation (translation, rotation, scaling) is used for coarse alignment. Mutual information (MI) or normalized cross correlation (NCC) is used as the optimization objective to iteratively update the transformation matrix.

[0054] In this embodiment, a fixed-size block (e.g. 7x7x7) is extracted around the target pixel, and a search range (e.g. 21x21x21) is defined in the image to find similar blocks. Euclidean distance is used to calculate the similarity between blocks, and similar blocks are filtered. The target pixel is filtered according to the block similarity weight, and the image is filtered multiple times (usually 1-3 times) to gradually suppress noise.

[0055] In this embodiment, the block size is used to balance local details and global similarity, and the complexity of the image affects the size of the block. The search window size is used to control the number of similar blocks (affecting the amount of calculation). High complexity: contains a large number of small structures (e.g. lung blood vessel network, liver bile duct tree). Low complexity: uniform or single structure (e.g. uniform fat tissue, cerebrospinal fluid). Complexity description parameters (edge density gradient, texture features, tissue contrast, etc.) and resolution parameters (spatial resolution and contrast resolution) are used to generate a complexity index (Complexity Index). Spatial resolution: number of pixels per unit size (e.g. CT's 512x512 matrix). Contrast resolution: ability to distinguish small gray scale differences (e.g. MRI's soft tissue contrast). The complexity index is calculated as follows:

[0056]

[0057] where C is the complexity index, n is the number of complexity description parameters, α i is the combined weight of the i-th complexity description parameter, Z i is the size of the i-th complexity description parameter, max(α i Z i ) is the αi Z i the maximum value in α i Z i the minimum value in α i Z i k1 is a first constant, The correction of the average value determined by the maximum value and the minimum value to the sum of the average value (1 / n-1, and the average value is a little larger) of the complexity description parameter can better reflect the complexity.

[0058] In some embodiments of the present application, the foreground region and the background region in the medical image are divided, including,

[0059] The feature point detection and the seed point screening are performed on the medical image, the gray threshold is adjusted according to the local mean value and the standard deviation of the gray value, and the growth rule is formulated through the consistency of the gray threshold and the gradient direction.

[0060] The topological constraint is performed, and the foreground region and the background region in the medical image are divided, the foreground region represents the liver and bile duct region, and the background region represents the image region in the medical image except the liver and bile duct region.

[0061] In the present embodiment, the process of dividing the foreground region and the background region is as follows:

[0062] Feature point detection:

[0063] The tubular structure is detected using the Hessian matrix, and the response threshold is set to 0.1.

[0064] The non-maximum suppression is performed on the detected points, and the local maximum value is reserved.

[0065] Seed point screening:

[0066] The candidate points are screened according to the gray value range (such as CT 50-80HU, MRI T2>200).

[0067] The foreground and background seed points are distinguished by K-means clustering (K=2).

[0068] Growth criteria:

[0069] Gray similarity: the gray difference between adjacent pixels is less than a threshold value (such as CT 10HU, MRI 30), the threshold value is adjusted according to the local mean value and the standard deviation, and the formula is as follows,

[0070] T(x)=μ(x)+kσ(x);

[0071] Where T(x) is the dynamic gray threshold, representing the gray threshold at image position x, μ(x) is the local mean, representing the average of all pixel gray values within a local window centered at position x, reflecting the average brightness of the local region. σ(x) is the local standard deviation, representing the degree of dispersion of gray values within the local window (i.e. the degree of brightness variation), reflecting the contrast or texture complexity of the local region. k is the adjustment coefficient, controlling the degree of contribution of the local standard deviation to the threshold. The larger k is, the more sensitive the threshold is to the local contrast variation; the smaller k is, the closer the threshold is to the local mean.

[0072] Gradient direction consistency: the difference in gradient direction between adjacent pixels is less than a threshold (e.g. 20°).

[0073] Topological constraints:

[0074] Fill small cavities using morphological closing operation (structure element 3x3x3).

[0075] Remove isolated noise by connected component analysis (area threshold > 50 voxels).

[0076] In step S102, the additional foreground region is confirmed in the background region based on the foreground region and the background region, and the foreground region and the additional foreground region of the multi-modal medical image are matched and calibrated to extract image features under the foreground region and the additional foreground region of each modal medical image.

[0077] In this embodiment, the additional foreground region is an organ near the liver and bile duct region, and the position and shape of the adjacent organs (such as gallbladder, pancreas, duodenum) provide a reference for the positioning of the liver and bile duct stones and the condition of the stones. For example, changes in the position and shape of the gallbladder may indicate gallbladder stones or cholecystitis, which in turn affects the analysis of stones in the extrahepatic bile duct. Changes in the bile duct (dilation, wall thickening, stenosis) are closely related to the condition of the stones, and the manifestations of these changes on medical images can reflect the presence, location, size of the stones and their impact on the bile duct.

[0078] 1. Relationship between bile duct dilation and stones

[0079] Reasons:

[0080] Stones block the bile duct, causing bile to be blocked, and the pressure in the bile duct to rise, causing the bile duct to dilate.

[0081] Both intrahepatic bile duct stones and extrahepatic bile duct stones can cause dilation of the bile duct at the corresponding site.

[0082] Image manifestations:

[0083] Increased bile duct diameter: On ultrasound, CT or MRI images, the bile duct diameter can be seen to be significantly larger than the normal range.

[0084] Intraductal bile shadow: Dilated bile ducts filled with low-density (CT) or isodense (MRI) bile shadow.

[0085] Upper bile duct dilation: The upper bile ducts upstream of the obstruction site are more dilated, forming a "soft vine sign" (the bile ducts are beaded).

[0086] Analysis of stone situation:

[0087] Stone location: The most dilated downstream bile ducts may have stones.

[0088] Stone size: The larger the stone, the more severe the obstruction, and the more obvious the bile duct dilation.

[0089] Obstruction degree: When completely obstructed, the bile ducts are more dilated; when partially obstructed, the degree of dilation may be lighter.

[0090] 2. Relationship between bile duct wall thickening and stones

[0091] Reasons:

[0092] Long-term stimulation of the bile duct wall by stones causes inflammatory reactions and fibrous tissue proliferation, leading to bile duct wall thickening.

[0093] Repeated friction of the bile duct wall by stones may also cause bile duct wall damage and thickening.

[0094] Imaging findings:

[0095] Uneven thickening: The bile duct wall is uneven in thickness, and may be thicker in some areas.

[0096] Annular thickening: The bile duct wall is annularly thickened, and the lumen is narrowed.

[0097] Enhancement: On enhanced CT or MRI, the thickened bile duct wall may show mild enhancement.

[0098] Analysis of stone situation:

[0099] Chronic stone stimulation: Bile duct wall thickening suggests that the stones have been present for a long time, possibly chronic stones.

[0100] Inflammation degree: The more obvious the thickening, the more severe the inflammatory reaction, and the greater the impact of the stones on the bile duct.

[0101] Risk of malignant lesions: Long-term bile duct wall thickening should be alert to the possibility of malignant lesions (such as bile duct cancer), and further evaluation should be made in combination with other examinations (such as tumor markers, biopsy).

[0102] 3. Relationship between bile duct stenosis and stones

[0103] Reasons:

[0104] Chronic obstruction by the stone leads to fibrosis and scarring of the bile duct wall, causing bile duct stenosis.

[0105] The stone compresses the bile duct wall, leading to ischemia and necrosis, further causing stenosis.

[0106] Imaging findings:

[0107] Narrowing or interruption of the lumen: On the image, the bile duct lumen is significantly narrowed or even interrupted.

[0108] Upstream bile duct dilation: The upstream bile duct at the stenosis site is significantly dilated, forming a "cup mouth sign" (sudden narrowing of the bile duct at the end).

[0109] Stone shadow: The stenosis site may directly show the stone shadow, or the stone is located downstream of the stenosis.

[0110] Analysis of the stone situation:

[0111] Stone size and location: Large stones are more likely to cause bile duct stenosis, and the stenosis site is usually located upstream of the stone.

[0112] Obstruction time: Stenosis suggests that the stone has been present for a long time, possibly chronic obstruction.

[0113] Complications risk: Bile duct stenosis can lead to bile stasis, cholangitis, and even biliary cirrhosis.

[0114] In summary, the downstream of the bile duct dilation, the area of bile duct wall thickening, and the site of bile duct stenosis are all possible locations of the stone. Combining multi-modal imaging (such as ultrasound, CT, MRI) can improve the accuracy of stone localization. The degree of bile duct dilation and the severity of bile duct stenosis can indirectly reflect the size and number of stones. Large stones or multiple stones are more likely to cause significant changes in the bile duct. Bile duct wall thickening and bile duct stenosis suggest that the stone has been present for a long time, possibly chronic disease. Acute stone obstruction usually presents as bile duct dilation, but bile duct wall thickening and stenosis are not obvious.

[0115] In some embodiments of the present application, the additional foreground region is identified in the background region based on the foreground region and the background region, comprising,

[0116] Select a proximity distance threshold based on the basic information of the patient, use U-Net to segment the organs near the hepatobiliary region, calculate the distance between each organ and the hepatobiliary region, and compare the distance with the proximity distance threshold to generate a candidate mask;

[0117] Extract texture features in the region within the candidate mask, and use a pre-set classifier to classify the position of the candidate mask to generate a verification mask, and use the region of the verification mask as an additional foreground region.

[0118] In this embodiment, the spatial constraints of the anatomical correlation method and the texture analysis capability of the feature matching method are combined to improve the screening accuracy of additional foreground regions (such as the gallbladder and liver parenchyma). Anatomical correlation method: quickly locate organs adjacent to the liver and bile duct through spatial distance. Feature matching method: use texture features to further verify the authenticity of the candidate region. Joint strategy: first narrow the search range through the anatomical correlation method, and then accurately classify using the feature matching method.

[0119] Organ segmentation:

[0120] Using U-Net to segment organs such as the gallbladder and liver parenchyma, the Dice coefficient needs to be >0.85.

[0121] Distance calculation:

[0122] Calculate the Euclidean distance of each organ voxel to the surface of the liver and bile duct.

[0123] Threshold setting: consider some basic information of the patient (body fat, height, etc.) to set.

[0124] Generate candidate mask:

[0125] Retain organ regions with a distance < distance threshold to generate a preliminary screening mask (region determined based on distance).

[0126] Feature matching method verification

[0127] Feature extraction:

[0128] Extract texture features from the regions of the candidate mask:

[0129] GLCM: window size 11x11, calculate energy, contrast, and correlation.

[0130] LBP: radius 3 pixels, point count 8.

[0131] Classifier training:

[0132] Train the classification model using SVM (RBF kernel, gamma = 0.01).

[0133] Training data: labeled gallbladder, liver parenchyma, blood vessels, and other regions.

[0134] Region classification:

[0135] Classify the voxels within the candidate mask to generate a verification mask.

[0136] In some embodiments of the present application, the foreground regions and additional foreground regions of multi-modal medical images are matched and calibrated, including,

[0137] Calculate the joint entropy of the foreground regions and additional foreground regions in different modal medical images, initialize the transformation parameters, and iteratively optimize the joint entropy until convergence.

[0138] In this embodiment, the rigid registration process is:

[0139] Reference selection: Liver contour as reference (liver is prominent and stable in abdominal images).

[0140] Algorithm: Mutual Information (MI) maximization.

[0141] Principle: Find the optimal spatial transformation (rotation, translation) by maximizing the joint entropy of two images.

[0142] Advantage: Insensitive to inter-modality intensity differences (e.g. HU values in CT vs. signal intensity in MRI).

[0143] Implementation steps:

[0144] Extract liver contour (thresholding + morphological operations can be used).

[0145] Initialize transformation parameters (e.g. translation vector, rotation matrix).

[0146] Iteratively optimize MI until convergence.

[0147] In some embodiments of the application, the image features include one or more of morphological features, texture features, intensity features, and spatial features.

[0148] In this embodiment, the foreground region features include but are not limited to the following:

[0149] Texture features:

[0150] Grey Level Co-occurrence Matrix (GLCM):

[0151] Extract features such as energy, contrast, correlation, entropy, etc.

[0152] Parameters: Window 11x11 pixels, directions 0°, 45°, 90°, 135°, gray levels 32.

[0153] Local Binary Pattern (LBP):

[0154] Radius 3 pixels, point number 8, calculate histogram features.

[0155] Morphological features:

[0156] Area: Number of pixels in the stone region.

[0157] Perimeter: Length of the stone boundary.

[0158] Circularity: Perimeter 24π x Area (stones are mostly circular-like).

[0159] Intensity features:

[0160] Mean HU: Average CT value of stone region (typical value > 100HU).

[0161] Max HU: Highest density value inside the stone.

[0162] Additional foreground region feature extraction

[0163] Gallbladder wall feature:

[0164] Thickness: Measure the average thickness of gallbladder wall (normal < 3mm).

[0165] Texture abnormality: LBP histogram entropy value (entropy value increases when inflammation occurs).

[0166] Liver parenchyma feature:

[0167] Density standard deviation: CT value standard deviation (normal < 15HU, increases when stone spreads).

[0168] GLCM contrast: Reflects density inhomogeneity.

[0169] Spatial relationship feature:

[0170] Stone to gallbladder distance: Euclidean distance (distance decreases when stone causes cholecystitis).

[0171] Stone to liver parenchyma distance: Assess the risk of stone spread.

[0172] Step S103, integrate the image features of all modalities to determine the direct and indirect content of hepatobiliary calculi.

[0173] In some embodiments of the present application, the direct and indirect content of hepatobiliary calculi is determined by integrating the image features of all modalities, including,

[0174] According to the morphological features, texture features, intensity features and spatial features, the direct and indirect content of hepatobiliary calculi is extracted, the direct content represents the content of the stone itself, and the indirect content includes changes in the bile duct and the content of the adjacent organ region.

[0175] In this embodiment, all image features are analyzed and converted into direct and indirect manifestations of the stone, the direct content: stone itself features (such as location, size, density). Indirect content: pathological changes caused by the stone (such as gallbladder wall thickening, bile duct changes (such as dilation degree) and adjacent organ region content (such as liver parenchyma density abnormalities).

[0176] Fusion features can be achieved:

[0177] Early fusion: Extract joint features directly after registering multi-modal images (such as combination of HU value and signal intensity of CT+MRI).

[0178] Late fusion: after extracting features of each modality, fuse them through a model (e.g. weighted sum, multi-layer perceptron).

[0179] Correlate features with pathology through statistical models or machine learning.

[0180] In step S104, the direct content and the indirect content of the hepatobiliary stones are combined to infer the location information and related information of the hepatobiliary stones, thereby assisting the doctor in analyzing the hepatobiliary stones.

[0181] In this embodiment, all stone points are identified according to the direct content (e.g. stone density threshold), or all possible stone points can be identified according to the image grayscale, which may be stones (i.e. locations) or other errors. The probability or likelihood of each stone point is analyzed according to each direct content and indirect content, and is described by a comprehensive confidence level.

[0182] In some embodiments of the present application, the direct content and the indirect content of the hepatobiliary stones are combined to infer the location information and related information of the hepatobiliary stones, including,

[0183] All stone points are determined according to the direct content of the hepatobiliary stones, each direct content and each indirect content are analyzed, a corresponding confidence level of each direct content and each indirect content is generated, all confidence levels are integrated to evaluate the confidence level of each stone point, and the location information and related information of the hepatobiliary stones are inferred.

[0184] In this embodiment, each direct content (e.g. stone size) and indirect content (e.g. bile duct dilation) is analyzed to generate a corresponding confidence level. The output of the inference includes the stone location coordinates, the confidence level, the type (e.g. calcified stone), and the risk level (e.g. high recurrence risk). The specific implementation formula for integrating all confidence levels to evaluate the confidence level of each stone point (Confidence Level of Stone Points) is as follows:

[0185]

[0186] wherein, is the confidence level of the j1thstone point, m1is the number of direct contents of the j1thstone point, m2is the number of indirect contents of the j1thstone point, are the combination weights of the j2thdirect content and the j3r indirect content, respectively, are the confidence levels of the j2thdirect content and the j3r indirect content of the j1thstone point, respectively, is the second constant of the j1thstone point, The second constant is used to balance the size of the correction function.

[0187] Correspondingly, the application also provides a liver and bile duct stone positioning analysis system based on image processing, as shown in the accompanying drawings, Figure 2 as shown in the accompanying drawings,

[0188] The first module is used for acquiring basic information of a patient and multi-modal medical images, removing artifacts and noises from the multi-modal medical images, and dividing foreground regions and background regions in the medical images.

[0189] The second module is used for confirming an additional foreground region in the background region based on the foreground regions and the background regions, matching and calibrating the foreground regions and the additional foreground regions of the multi-modal medical images, and extracting image features under the foreground regions and the additional foreground regions of each modality of medical images.

[0190] The third module is used for integrating the image features of all modalities to determine direct content and indirect content of the liver and bile duct stones.

[0191] The fourth module is used for inferring position information and related information of the liver and bile duct stones based on the direct content and the indirect content of the liver and bile duct stones, so as to assist doctors in analyzing the liver and bile duct stones.

[0192] The application has the following beneficial effects:

[0193] 1. The multi-modal medical images are subjected to artifact and noise removal processing, the influence of interference factors is reduced, and a reliable basis is provided for subsequent image feature extraction and analysis. The additional foreground region is confirmed in the background region based on the foreground regions and the background regions. The foreground regions represent the liver and bile duct regions, and the additional foreground regions represent organ regions near the liver and bile duct regions, which can also reflect the condition of the stones, thereby improving the reliability of the liver and bile duct stone positioning analysis.

[0194] 2. The image features of all modalities are integrated to determine the direct content and the indirect content of the liver and bile duct stones. The position information and related information of the liver and bile duct stones are inferred based on the direct content and the indirect content of the liver and bile duct stones. The direct and indirect manifestations of the stones on the image are comprehensively reflected and analyzed to reflect and analyze the condition and position of the liver and bile duct stones, thereby improving the accuracy and adaptability of the liver and bile duct stone positioning analysis, ensuring the reliability of the doctor's reference, and helping the doctor's treatment and reference from the side.

[0195] Those skilled in the art can clearly understand the present application through the above description of the embodiments that the present application can be implemented by hardware or by means of software and necessary universal hardware platform. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.) and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments of the present application.

[0196] Those skilled in the art can understand that the drawings are only schematic diagrams of a preferred embodiment, and the modules or flows in the drawings are not necessarily required for implementing the present application.

[0197] Those skilled in the art can understand that the modules in the system in the embodiments can be distributed in the system according to the description of the embodiments, or can be changed to be located in one or more systems different from the embodiments. The modules in the above embodiments can be combined into one module, or can be further split into a plurality of sub-modules.

[0198] The above description is only a preferred embodiment of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can make equivalent replacements or changes to the technical solutions and the inventive concept of the present application within the technical scope disclosed by the present application, which should be covered by the protection scope of the present application.

Claims

1. A method for locating and analyzing hepatobiliary stones based on image processing, characterized by, The application relates to a method for analyzing hepatobiliary calculi based on multi-modal medical images. The method comprises the following steps: acquiring basic information of a patient and multi-modal medical images, removing artifacts and noises of the multi-modal medical images, and dividing foreground regions and background regions in the medical images; Based on the foreground regions and the background regions, an additional foreground region is confirmed in the background region, the foreground regions and the additional foreground regions of the multi-modal medical images are matched and calibrated, and image features of the foreground regions and the additional foreground regions of each modality medical image are extracted; The image features of all modalities are integrated to determine direct content and indirect content of hepatobiliary calculi; The direct content and the indirect content of the hepatobiliary calculi are combined to infer position information and related information of the hepatobiliary calculi, thereby assisting doctors in analyzing the hepatobiliary calculi as a reference; The method comprises the following steps: According to the basic information of the patient, a proximity distance threshold is selected, organs near the hepatobiliary region are segmented by using a U-Net, distances between the organs and the hepatobiliary region are calculated, and the distances are compared with the proximity distance threshold to generate a candidate mask; Texture features are extracted in a region in the candidate mask, a preset classifier is used for position classification of the candidate mask, a verification mask is generated, and a region of the verification mask is used as the additional foreground region; The image features of all modalities are integrated to determine direct content and indirect content of hepatobiliary calculi, The direct content and the indirect content of the hepatobiliary calculi are combined to infer position information and related information of the hepatobiliary calculi, thereby assisting doctors in analyzing the hepatobiliary calculi as a reference; The method comprises the following steps:

2. The image processing-based hepatobiliary stone localization analysis method according to claim 1, characterized by, High-contrast feature points in the medical images are extracted by using a SIFT or SURF algorithm, and feature points of adjacent frame medical images are matched by using a k-NN or FLANN algorithm; A signal-to-noise ratio of the medical images is determined, a RANSAC iteration number is obtained according to the signal-to-noise ratio, matching pairs are screened through the RANSAC iteration number, rigid transformation estimation is performed, and a transformation matrix is defined; Medical image alignment and interpolation are performed, and iteration optimization is performed to remove motion artifacts in the medical images; Complex description parameters and resolution parameters in the medical images are counted, complex indicators are generated by integrating the complex description parameters, resolution indicators are generated by integrating the resolution parameters, and block sizes and search window sizes are determined according to the complex indicators and the resolution indicators respectively; Block matching, weight calculation, filtering and iteration processing are realized based on the block sizes and the search window sizes, so that noise removal in the medical images is completed. The method comprises the following steps: Feature point detection and seed point screening are performed on the medical images, a gray threshold is adjusted according to a local mean value and a standard deviation of the gray, and a growth rule is formulated according to the gray threshold and gradient direction consistency; 3. The image processing-based hepatobiliary stone localization analysis method according to claim 1, characterized by, Topological constraints are performed to divide the foreground regions and the background regions in the medical images, the foreground regions represent the hepatobiliary region, and the background regions represent image regions in the medical images except the hepatobiliary region. The method comprises the following steps: ​ 4. The image processing-based hepatobiliary stone localization analysis method according to claim 1, characterized by, ​ ​ The joint entropy of the foreground region and the additional foreground region in different modal medical images is calculated, the transformation parameter is initialized, and the joint entropy is iteratively optimized until convergence.

5. The image processing-based hepatobiliary stone localization analysis method according to claim 1, characterized by, The image features include one or more of morphological features, texture features, intensity features, and spatial features.

6. The image processing-based hepatobiliary stone localization analysis method according to claim 1, wherein, The location information and related information of the hepatobiliary stones are inferred by combining the direct content and the indirect content of the hepatobiliary stones, including, All stone points are determined according to the direct content of the hepatobiliary stones, each direct content and each indirect content is analyzed, the confidence of each direct content and each indirect content is generated, the confidence of each stone point is evaluated by integrating all confidences, and thus the location information and related information of the hepatobiliary stones are inferred.

7. An image processing-based biliary stone localization analysis system, characterized by, The system for implementing the image processing-based hepatobiliary stone positioning analysis method according to any one of claims 1-6 comprises, A first module is configured to acquire basic information of a patient and multi-modal medical images, remove artifacts and noise from the multi-modal medical images, and divide foreground regions and background regions in the medical images; A second module is configured to confirm additional foreground regions in the background regions based on the foreground regions and the background regions, match and calibrate the foreground regions and the additional foreground regions of the multi-modal medical images, and extract image features under the foreground regions and the additional foreground regions of each modal medical image; A third module is configured to integrate the image features of all modalities to determine direct content and indirect content of the hepatobiliary stones; A fourth module is configured to infer the location information and related information of the hepatobiliary stones by combining the direct content and the indirect content of the hepatobiliary stones, thereby assisting doctors in analyzing the hepatobiliary stones as a reference.

Images