Multi-frame image fatty liver content detection method and system based on motion suppression

Abstract

The invention discloses a multi-frame image fatty liver content detection method and system based on motion suppression, and belongs to the technical field of medical image processing.The method comprises the steps that an interested area is selected in a liver area of a patient, and processing time is set; under B-mode ultrasonic scanning, obtaining a liver image in real time; generating a liver parenchyma probability weight distribution diagram for each frame of image; inter-frame motion estimation is carried out, and image frames with motion amplitudes within a preset threshold value are screened out; performing nonlinear parameter scanning to obtain a corresponding nonlinear parameter matrix; converting each element in each nonlinear parameter matrix into a liver fat content value to obtain a corresponding liver fat content fraction matrix; and performing fusion filtering processing on the plurality of liver fat content fractional matrixes by applying a Kalman filtering algorithm to obtain a final liver fat content quantitative value. According to the method, noninvasive, accurate and quantitative liver fat content detection can be realized; the system can be integrated into ultrasonic diagnosis equipment and is suitable for hospitals, physical examination centers and other scenes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image processing, and more particularly to a multi-frame image fatty liver content detection method and system based on motion suppression. BACKGROUND

[0002] Metabolic Associated Fatty Liver Disease (MAFLD) is a liver disease closely related to insulin resistance and metabolic dysfunction, covering a disease spectrum from simple steatosis (MAFLD) to Metabolic Associated Steatohepatitis (MASH). With the prevalence of obesity and metabolic syndrome worldwide, MAFLD has become the main cause of chronic liver disease, and some patients may progress to liver fibrosis, cirrhosis and even liver cancer. Therefore, early, non-invasive and accurate liver fat content quantitative detection technology has important clinical value.

[0003] Currently, the diagnosis and evaluation of liver fat content mainly rely on two methods of liver biopsy and imaging examination. Liver biopsy, as the gold standard for liver fat content detection, directly assesses the degree of steatosis, inflammation and fibrosis through histological examination, but has problems such as trauma, sampling error, complication risk, and is difficult to use for dynamic monitoring. Imaging examination includes routine ultrasound examination, CT / MRI and CAP (controlled attenuation parameter) examination. Routine ultrasound qualitatively judges the severity of fatty liver by the strength of liver echo, but cannot be quantified and is greatly influenced by operator experience. MRI's PDFF (proton density fat fraction) is currently the most accurate imaging quantitative method, but its single detection cost is high and is strongly dependent on equipment, making it difficult to popularize. CT can also qualitatively detect fat content (such as liver-spleen CT value ratio), but CT contains radiation and cannot be used as a long-term screening method. Finally, the CAP technology based on ultrasound platform can qualitatively detect the light, medium and heavy grades of fatty liver, but still cannot achieve quantitative detection.

[0004] The relationship between the acoustic pressure of ultrasound waves propagating in biological tissue and the medium density is approximately described by Taylor's technique as follows:

[0005]

[0006] In the formula, p represents the acoustic pressure, i.e. the pressure change produced by the propagation of ultrasound waves in the medium.

[0007] represents the change in medium density, i.e. the density fluctuation caused by acoustic disturbance.

[0008] represents the initial density of the medium, i.e. the original density of the medium before the sound wave propagates.

[0009] A represents the coefficient of the first order term, reflecting the linear relationship between the sound pressure and the density change, and is related to the linear acoustic characteristics of the medium.

[0010] B represents the coefficient of the second order term, characterizing the strength of the nonlinear effect in the sound propagation process. The ratio B / A of the coefficient of the first order term A to the coefficient of the second order term B is defined as the nonlinear parameter of the medium, which is a key indicator for quantifying the nonlinear characteristics of the tissue.

[0011] C represents the coefficient of the third order term, corresponding to higher order nonlinear effects. In practical applications, the influence of the third and higher order terms is usually small, and is often ignored to simplify the model.

[0012] L represents the fourth and higher order terms.

[0013] The ratio B / A of the coefficient of the second order term to the coefficient of the first order term in the above formula is defined as the nonlinear parameter of the medium.

[0014] Table 1 below shows the nonlinear parameter values of different tissues in the human body. As can be seen from Table 1, the nonlinear parameter values of healthy liver and liver in disease state are quite different, especially the nonlinear parameter values of healthy liver and fat are quite different, which are 6.8±0.5 and 9±0.5 respectively.

[0015] Table 1 Nonlinear parameter values of different tissues in the human body

[0016]

[0017] The classical nonlinear parameter measurement method uses thermodynamic method to measure, which uses the change of wave speed with sound pressure to estimate the size of nonlinear parameter under isothermal condition. Although this method is relatively accurate, the experimental device and method are very complex, and it is not suitable for clinical diagnosis.

[0018] Patent document with publication number CN104757999A discloses a nonlinear imaging method based on ultrasonic fundamental wave and harmonic wave, but the image generated by this method does not directly participate in the calculation of nonlinearity, but generates an image associated with the nonlinear parameter; the patent with publication number CN107970042A generates nonlinear parameter image using fundamental wave and harmonic wave under high and low voltage. Using this technology, imaging the specified ROI region of liver will generate a nonlinear parameter pseudo-color image. The deeper the color of the region, the larger the nonlinear parameter, which represents high liver fat content. On the contrary, the smaller the nonlinear parameter represents low liver fat content. Therefore, this method can only obtain a qualitative result of liver fat content.

[0019] Therefore, the method for generating a nonlinear parameter image based on a fundamental wave and a harmonic wave in the related art can only realize qualitative analysis and cannot quantitatively output a fat content value, and is easily disturbed by motion, artifacts, noise and non-liver tissues. SUMMARY

[0020] The present application aims to solve the problem that non-invasive, accurate and quantitative liver fat content detection cannot be realized in the prior art, and proposes a multi-frame image fat liver content detection method and system based on motion suppression.

[0021] In order to achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0022] In a first aspect, the present application provides a multi-frame image fat liver content detection method based on motion suppression, comprising the following steps:

[0023] Select a region of interest in the liver region of a patient and set a processing time; under B-mode ultrasonic scanning, real-time liver images are acquired;

[0024] Generate a liver parenchyma probability weight distribution map for each image frame;

[0025] Perform inter-frame motion estimation to screen out image frames with a motion amplitude within a preset threshold;

[0026] Perform nonlinear parameter scanning on the screened image frames to obtain corresponding nonlinear parameter matrices;

[0027] Each nonlinear parameter value in each nonlinear parameter matrix is converted into a liver fat content score value, respectively, to obtain a corresponding liver fat content score matrix;

[0028] Apply a Kalman filtering algorithm to perform fusion filtering processing on a plurality of liver fat content score matrices to obtain a final liver fat content quantitative value.

[0029] Further, the liver parenchyma probability weight distribution map is generated by a deep learning semantic segmentation model; the model adopts a U-Net network structure, and only the pixels of the liver parenchyma region are labeled during training, and the non-liver parenchyma region includes blood vessels, shadows, calcifications and other non-liver tissues; the probability value of each pixel belonging to the liver parenchyma is output.

[0030] Further, the inter-frame motion estimation is performed to screen out image frames with a motion amplitude within a preset threshold; which comprises:

[0031] The first frame collected in the image sequence is set as a reference frame, and the frames collected thereafter are sequentially set as current frames;

[0032] The region of interest in the reference frame and the current frame is respectively divided into a plurality of image sub-blocks with a size of MxN pixels;

[0033] In the current frame, for each target image sub-block in the reference frame, a block matching calculation is performed in a corresponding search area to find an optimal matching sub-block;

[0034] According to the position difference between the target image sub-block and the optimal matching sub-block, the relative motion amplitude of the current frame and the reference frame is calculated;

[0035] The motion amplitude is compared with a distance threshold value, and if the motion amplitude is less than or equal to the distance threshold value, the current frame is retained for subsequent nonlinear parameter calculation; otherwise, the current frame is discarded.

[0036] Further, the matching criterion used in the block matching calculation is the sum of absolute differences, the square error or the normalized cross-correlation function.

[0037] Further, the screened image frames are subjected to nonlinear parameter scanning to obtain corresponding nonlinear parameter matrices; including:

[0038] Four groups of echo signals of high-voltage fundamental wave, high-voltage harmonic wave, low-voltage fundamental wave and low-voltage harmonic wave are collected;

[0039] The signals are subjected to median filtering;

[0040] Nonlinear parameter equations are constructed and solved to obtain corresponding nonlinear parameter matrices.

[0041] Further, each nonlinear parameter value in each nonlinear parameter matrix is converted into a liver fat content score value to obtain a corresponding liver fat content score matrix; including:

[0042] The nonlinear parameter value of ideal liver parenchyma is The nonlinear parameter value of fat is :

[0043] According to the linear relationship between the nonlinear parameter value and the fat content of a specified region of the liver, a linear relationship model is constructed as follows:

[0044]

[0045] f is the fat content corresponding to each nonlinear parameter value, is the nonlinear parameter value corresponding to liver parenchyma without fat, is the nonlinear parameter value corresponding to fat; P is the nonlinear parameter value corresponding to a certain pixel, i.e. the gray value;

[0046] According to the linear relationship model, each element in each nonlinear parameter matrix is converted into a liver fat content score value between 0 and 1 to obtain a liver fat content score matrix.

[0047] Further, a Kalman filtering algorithm is applied to fuse and filter the plurality of liver fat content score matrices to obtain a final liver fat content quantitative value; comprising:

[0048] establishing a state equation and an observation equation;

[0049] initializing a state vector and an error covariance matrix;

[0050] performing a prediction and update step, and adjusting the measurement noise covariance in combination with the liver parenchyma probability weight;

[0051] outputting the mean value of the optimal fat content score matrix value as the final liver fat content.

[0052] In a second aspect, the embodiments of the present application also provide a multi-frame image-based fat liver content detection system based on motion suppression, which applies the method of any one of the first aspect, and the system comprises:

[0053] an ultrasonic imaging module, configured to select a region of interest in a liver region of a patient and set a processing time; and acquire liver images in real time under B-mode ultrasonic scanning;

[0054] a liver parenchyma probability weight calculation module, configured to generate a liver parenchyma probability weight distribution map for each frame of image;

[0055] a motion estimation module, configured to perform inter-frame motion estimation and screen out image frames with a motion amplitude within a preset threshold;

[0056] a nonlinear parameter calculation module, configured to perform nonlinear parameter scanning on the screened image frames to obtain corresponding nonlinear parameter matrices;

[0057] a fat content conversion module, configured to convert each nonlinear parameter value in each nonlinear parameter matrix into a liver fat content score value to obtain a corresponding liver fat content score matrix;

[0058] a Kalman filtering module, configured to fuse and filter the plurality of liver fat content score matrices by using a Kalman filtering algorithm to obtain a final liver fat content quantitative value.

[0059] According to the technical solutions described above, compared with the prior art, the present application has the following technical effects:

[0060] The present application predicts liver fat content based on multi-frame ultrasonic images, weakens non-liver tissue interference by using liver parenchyma probability distribution, and introduces Kalman filtering to suppress noise and abnormal values. The present application can be integrated into an ultrasonic diagnostic device, and is suitable for hospitals, medical centers and other scenes, and provides a quantitative tool for fat liver screening and follow-up, and has strong stability and more reliable results. BRIEF DESCRIPTION OF DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings described below are only a part of the embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor based on the provided drawings.

[0062] Figure 1 The flow chart of the method for detecting the fat liver content based on motion suppression provided by the present application is provided.

[0063] Figure 2 The principle diagram of the fat liver content detection provided by the present application is provided.

[0064] Figure 3 The segmentation result schematic diagram of the liver image under the U_Net network model provided by the present application is provided.

[0065] Figure 4 The flow chart of the inter-frame motion estimation calculation provided by the present application is provided.

[0066] Figure 5 The ROI region schematic diagram of the liver image provided by the present application is provided.

[0067] Figure 6 The flow chart of the non-linear parameter matrix provided by the present application is provided.

[0068] Figure 7 The non-linear parameter A / B matrix schematic diagram provided by the present application is provided.

[0069] Figure 8 The liver fat content fraction matrix schematic diagram provided by the present application is provided.

[0070] Figure 9 The system block diagram of the fat liver content detection system based on motion suppression provided by the present application is provided. DETAILED DESCRIPTION

[0071] The technical solutions in the embodiments of the present application will be described clearly and completely below with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all the other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0072] The embodiment of the present application discloses a kind of multi-frame image fat liver content detection method based on motion suppression, it is based on the physical characteristics that nonlinear parameter of healthy liver and fat is different, a kind of liver fat content quantitative measurement method is proposed.The method can effectively shield the problem of too large data fluctuation due to breathing, probe shaking in liver scanning detection, while considering that nonlinear parameter in actual measurement process, due to liver contains blood vessels, non-liver tissue or noise in addition to liver parenchyma, fat, so there is a certain error between the value of nonlinear parameter and theoretical value, in addition to this situation, patient can not keep completely still in the process of detection, or the weak traction of heart leads to the difference of front and back frame measurement results, so based on the above problems, the present application proposes a kind of liver fat content detection method based on multi-frame, kalman filtering algorithm, compared with previous technology, it is more stable and the result is more reliable.

[0073] As Figure 1 The embodiment of the present application discloses a kind of multi-frame image fat liver content detection method based on motion suppression, it includes the following steps:

[0074] S1, select the region of interest in the liver region of patient and set processing time;Under B mode ultrasonic scanning, liver image is acquired in real time;

[0075] S2, generate liver parenchyma probability weight distribution map for each frame image;

[0076] S3, interframe motion estimation is carried out, and the image frame with motion amplitude in preset threshold is screened out;

[0077] S4, nonlinear parameter scanning is carried out to the screened image frame, and corresponding nonlinear parameter matrix is obtained;

[0078] S5, each nonlinear parameter value in each nonlinear parameter matrix is converted into liver fat content score value respectively, and corresponding liver fat content score matrix is obtained;

[0079] S6, kalman filtering algorithm is applied to fuse filtering processing to multiple liver fat content score matrix, and finally liver fat content quantitative value is obtained.

[0080] The method can realize non-invasive, accurate and quantitative liver fat content detection;Robustness is improved by multi-frame fusion and motion estimation;Liver parenchyma probability weighting helps to improve measurement accuracy;Finally, kalman filtering is used to suppress noise and abnormal value influence.

[0081] The technical solutions of the present application will be described in detail as follows:

[0082] 1, refer to Figure 2As shown in the total flow chart of the target liver detection, first, select a suitable region of interest (ROI) in the patient's liver position and set the processing time, the purpose of the processing time is to set a certain time range to collect enough information, then start the ultrasound B_mode mode, so that the subsequent steps of tissue probability weight calculation and frame and frame motion estimation are carried out.

[0083] 2, while B_mode scanning, generate a liver parenchyma probability weight distribution map for the current frame image of the scan, which can be a deep learning based segmentation method or other traditional image processing algorithm, if it is a deep learning method, a semantic segmentation model can be used to realize it, and the specific semantic segmentation model can select the U_Net network model, the gold standard in model training only marks the liver parenchyma part, and other blood vessels, shadows, calcification areas and other non-liver tissues are marked as non-liver tissues, so that the output in the forward inference stage after the model training is completed is the probability distribution of the liver parenchyma. For areas with high probability values, such as greater than 0.9, they can be considered as liver parenchyma, and for areas with probability values less than a specified threshold, such as less than threshold 0.6, they can be considered as shadow areas behind blood vessels, and for areas with probability values around 0.1, they can be considered as blood vessel areas or other parts that are obviously not liver parenchyma. The output probability map is retained for subsequent Kalman filtering.

[0084] Reference Figure 3 As shown in the segmentation result of the liver image under the U_Net network model, Figure 3 The left half of the image in the middle is the original ultrasound B_mode image, and the right half is the segmentation result based on the U_Net network model. The green area is the segmented liver parenchyma, and the red area is the segmented blood vessels. The probability threshold of the segmentation output is 0.5, that is, areas with a probability greater than 0.5 in the segmentation output probability map are considered to be liver parenchyma, and areas with a probability less than 0.5 are considered to be background.

[0085] After obtaining the segmentation probability map, the liver segmentation probability map is retained, called tissue probability weight map W, each element of the weight map is between 0 and 1, where the closer to 1 indicates a greater probability of liver parenchyma, and vice versa. The significance of retaining the tissue probability weight is to filter out areas with low tissue probability weight in the subsequent Kalman filtering process.

[0086] 3, motion estimation: used to judge the motion amplitude between frames, because the fat in the liver is not uniformly distributed, if the motion amplitude of the region between the front and back frames is too large, the fat content inside it may also be different and unpredictable, so the embodiment of the present application wants to avoid this situation, so the motion estimation is used to judge the motion amplitude between frames.

[0087] Motion estimation includes the following steps 1) to 5):

[0088] 1) Set the first frame captured in the image sequence as the reference frame, and set the subsequent frames as the current frames in sequence;

[0089] 2) Divide the regions of interest in the reference frame and the current frame into multiple image sub-blocks of size M×N pixels;

[0090] 3) In the current frame, for the target image sub-block in the reference frame, perform block matching calculation within the corresponding search area to find the optimal matching sub-block;

[0091] 4) Calculate the relative motion amplitude between the current frame and the reference frame based on the positional difference between the target image sub-block and the optimal matching sub-block;

[0092] 5) Compare the motion amplitude with the distance threshold. If the motion amplitude is less than or equal to the distance threshold, retain the current frame for subsequent nonlinear parameter calculation; otherwise, discard the current frame.

[0093] In practical implementation, a block motion estimation method is adopted, referring to, for example... Figure 4 As shown, the ROI region of the B-mode ultrasound image is first divided into M×N sub-blocks. Then, a concept is introduced: the image with the earlier number in the image sequence is called the previous frame, and the image with the next higher number is called the current frame (meaning the image just acquired). A block is then selected from the current frame as the target block for a motion search. Block matching criteria can include sum of absolute difference (SAD), mean square error (MSE), and normalized cross correlation function (NCCF). These methods offer similar accuracy for block matching, but their computational complexity varies significantly, with SAD requiring the least computation. The calculation method for SAD is as follows:

[0094] Formula (1)

[0095] In the above formula (1), This indicates the target image patch selected in the first frame. The image block at the (i, j) position of the current frame. The smaller the image block matching degree value calculated by the SAD-based method, the greater the similarity between the blocks. MxN SAD values are calculated by the above formula, and then a minimum SAD value is selected and compared with an absolute difference and threshold (for example, 0.2). If it is less than the absolute difference and threshold, the relative motion amount between the two frames of images can be obtained according to the position of the current frame.

[0096] The block matching method can also use image square error and cross-correlation, whose formulas are shown in (2) and (3) respectively:

[0097] Formula (2)

[0098] Formula (3)

[0099] In the above normalized cross-correlation formula, denotes the selected target image block of the first frame image, denotes the image block at the (i, j) position of the current frame. Unlike the SAD standard, the greater the normalized cross-correlation coefficient value, the greater the correlation. Therefore, when judging the mutual motion between frames, the maximum value of the MxN normalized cross-correlation coefficients needs to be found and compared with a normalized cross-correlation threshold (for example, 0.7). When it is greater than the normalized cross-correlation threshold, it is considered that the image block of the subsequent frame is formed by the motion of the target image block of the previous frame. Finally, the position of the found image block is used to judge the motion amount of the image.

[0100] In the process of judging the motion amplitude, because the block matching algorithm finds the motion relationship between the first frame image and the subsequent frame image, that is, the number of pixels deviating from the first frame, the number of pixels is compared with the distance threshold. When it is greater than the preset distance threshold, the frame is discarded. When it is less than the distance threshold, it is considered that the motion deviation of the current frame is within the expected range and is retained.

[0101] For example, the similarity calculation is performed on the first frame target image block and the subsequent MxN image blocks, which will obtain MxN similarity values. If it is the SAD method, the minimum value is selected and compared with the absolute difference and threshold. If it is less than the absolute difference and threshold, it is considered that the target image block and the subsequent frame are successfully matched. Then the position index (the position of the pixel, which is ) of the SAD minimum value is compared with the position of the target image block (also ) to calculate the position. The calculation can be performed according to the formula: The distance is used to judge the relative motion amount of the subsequent frame and the first frame, and a distance threshold is set. When the distance calculated by the above formula is greater than the distance threshold, it is considered that the relative motion amplitude is too large and is discarded.

[0102] 4. Perform nonlinear parameter scanning on the ROI regions in the filtered image frames to obtain a nonlinear parameter matrix. The region used for nonlinear parameter scanning can be an inscribed curvilinear trapezoid of the aforementioned ROI, such as... Figure 5 The green curved trapezoidal ROI region in the middle is enclosed by a white rectangle, which is the rectangular ROI region for motion estimation in step 3. Since the nonlinear parameter calculation is based on the IQ envelope signal of the RF radio frequency signal, the curved trapezoidal ROI region of B_mode mapped to the IQ envelope is exactly a regular rectangular region. After selecting the curved trapezoid within the ROI, the corresponding region of the tissue probability weight W from step 2 is extracted. Because the ROI is a curved trapezoid, the tissue probability weight of the rectangular region is obtained by performing an inverse scan transformation based on the parameters of the ultrasound system scan transformation. Used for subsequent Kalman filter calculations.

[0103] For frames where the normalized cross-correlation coefficient between the first and subsequent frames is greater than the normalized cross-correlation threshold and meets the distance threshold, a nonlinear scan is performed on the curved trapezoidal shape within the ROI to obtain a nonlinear parameter matrix. Simultaneously, image cross-correlation is calculated between the first frame and the next frame, following the same process. The calculated cross-correlation coefficient is compared with the normalized cross-correlation threshold and the distance threshold. If the conditions are met, a nonlinear parameter scan is initiated to obtain the nonlinear parameter matrix based on the ROI of that frame. This process continues until a predetermined time is reached, at which point a series of nonlinear parameter matrices are obtained.

[0104] like Figure 6 The diagram shows the algorithm flowchart for obtaining the nonlinear parameter matrix of the curved trapezoidal shape within each frame of the ROI under nonlinear parameter scanning. First, four sets of echo signals under high and low voltage conditions are obtained for each frame image, namely the high voltage fundamental signal, high voltage harmonic signal, low voltage fundamental signal, and low voltage harmonic signal. Then, median filtering is performed on each set of signals to eliminate noise interference. Finally, the nonlinear parameter equation is constructed, and the form of the nonlinear parameter equation is as follows:

[0105] Formula (4)

[0106] The calculation method for b in the above formula is as follows:

[0107] Formula (5)

[0108] x is defined as follows:

[0109] Formula (6)

[0110] In formula (6) , The value of is mainly affected by the probe emission frequency, and can be obtained by pre-estimation through experiments, represents a value related to the nonlinear parameter, specifically , represents a normalized voltage, specifically the ratio of the input high and low voltages.

[0111] The above nonlinear equation can be solved by numerical analysis method, the initial value can be generated by Gaussian distribution, the mean value of Gaussian distribution is the mean value of the theoretical nonlinear parameters of liver and fat, the variance is not too large, 4 is more appropriate, because the nonlinear parameter value of liver parenchyma is about 6, and the nonlinear parameter value of fat is about 10, the mean value of the two is about 8, so the variance of 4 can make the generated data mostly distributed in the interval (6, 10). After setting the initial value, a numerical analysis method can be selected to solve the parameter x, and numerical analysis method can select Newton method, BFGS method, etc., to obtain the nonlinear parameter matrix A / B.

[0112] 5、The above process is completed to obtain the nonlinear parameter matrix A / B, which has a matrix form as shown in Figure 7 , Figure 7 Each pixel in the nonlinear matrix of is corresponding to a plurality of liver and fat cells, according to related research, each pixel corresponds to a liver cell area with a size of 15*15, assuming that the nonlinear parameter value of the lipid droplet is 9±0.5, and the nonlinear parameter value of the liver parenchyma is 6.8±0.5, if the specified target area liver fat content ratio and the nonlinear parameter value are proportional, then the numerical range of the nonlinear parameter matrix is 7.9±0.5.

[0113] In the present application, it is assumed that the liver fat content and the size of the nonlinear parameter in the specified ROI region are in a linear relationship. The linear relationship model is constructed as follows:

[0114]

[0115] f is the fat content corresponding to each nonlinear parameter value, is the nonlinear parameter value corresponding to the liver parenchyma without fat, is the nonlinear parameter value corresponding to fat; P is the nonlinear parameter value corresponding to a certain pixel, i.e. the gray value;

[0116] According to the linear relationship model, each element in the above nonlinear parameter matrix is converted into a liver fat content value, which is between 0 and 1. The fat content matrix after conversion is as shown in Figure 8It is shown that: in the actual liver fat content detection process, due to the movement of the patient's body, the traction of the heart to the liver, the shaking of the ultrasonic probe or the difference of the ROI position, all of which will cause the measured fat content to deviate from the actual value, in addition, due to the difference between the ideal nonlinear body film and the complex composition of the human liver containing different kinds of substances, the fat content measured by the nonlinear parameter value in the specified ROI of the liver will have some errors. Based on the above analysis, the Kalman filtering algorithm based on multiple image frames is proposed to obtain stable and reliable liver fat content value.

[0117] 6、Kalman filtering as a recursive Bayesian estimation method can obtain better estimation by combining priori and observation data in the presence of dynamic process and noise. In the case of introducing ultrasonic multi-frame time series data, a dynamic model of fat content changing slowly with time is established, which can effectively improve the stability and accuracy of detection.

[0118] The first step of Kalman filtering is to establish a state equation, as shown in formula (7):

[0119] Formula (7)

[0120] In the above formula (7), represents noise obeys a multi-dimensional Gaussian distribution with a mean of 0 and a covariance matrix Q, and represents the fat fraction vector at the current time and the previous time, which can be obtained by flattening the liver fat content fraction matrix obtained in step 5, and A is the process transition matrix, because the fat content corresponding to the front and back frames is consistent within a short time, so A is an identity matrix , represents process noise, and the process noise obeys a Gaussian distribution with a mean of 0 and a covariance matrix , because each pixel is independent and does not affect each other, so the covariance matrix of the process noise is a diagonal matrix, and here is a diagonal matrix. In actual application, the variance of the process noise should be very small, because within a very small time period and the position of the ROI is almost unchanged, the observed fat content fraction will not change in principle, so the noise variance of the state equation can be set to be small, which can be 0.01.

[0121] The observation equation is as shown in formula (8):

[0122] Formula (8)

[0123] In the above formula (8), is the nonlinear parameter vector, H represents the measurement transfer matrix, here , and represent the nonlinear parameter values of liver parenchyma and fat in ideal state respectively, both are scalar, is the unit matrix, is the fat content fraction vector of the current frame, is the measurement noise, indicates obeys the multi-dimensional Gaussian distribution with mean 0 and covariance matrix R, the source of the measurement noise can be the nonlinear noise effect of other hardware of the system rather than the liver site, the influence of other substances within the liver ROI, the inherent error of the nonlinear parameter calculation method, etc.

[0124] Kalman filtering can smooth out errors, noise and outliers caused by various reasons, but the specific distribution of the error needs to be known in advance, here it is assumed that the value of the process error covariance Q is a diagonal matrix with all elements being 0.01, and the measurement error covariance R is also a diagonal matrix with the same elements, the value of the diagonal matrix can be obtained by experiment, in the case where the fat content of the liver in a certain size of tissue region is a constant value, the nonlinear parameter matrix of the tissue is determined, in the ideal case the nonlinear parameter matrix should be a constant matrix, but there is noise in the measurement process, so the value of the nonlinear parameter matrix is certainly different, then the variance of the nonlinear parameter matrix is calculated and recorded, the nonlinear parameter matrix is repeatedly calculated and the variance is recorded in different fat content regions, and finally the mean of all recorded variances is taken as the variance in the process of determining the fat content by the nonlinear parameter.

[0125] After establishing the state equation of the above Kalman filter, the error matrix, that is, the actual error between the true value and the predicted value, is initialized, which can be randomly set, for example, a diagonal matrix with the same dimension as the nonlinear parameter matrix is set, and the size of the diagonal matrix is set to 0.1, the fat content fraction vector can be directly derived from the nonlinear parameter matrix of the first frame, for example, the fat content fraction vector is obtained by directly flattening the fat content fraction matrix of the 5th step. Kalman filtering is divided into two steps of prediction and update, the prediction is as follows:

[0126] Equation (9)

[0127] is the posterior estimate value of the last frame, if it is based on the prediction of the first frame, the value is the initial fat content fraction value calculated according to the nonlinear parameter matrix of the first frame, A is the unit matrix I here, indicating that the fat content of the previous and subsequent frames is consistent, and the obtained is the prior fat content derived according to the state equation. The prior error covariance representing the prediction uncertainty is:

[0128] Equation (10)

[0129] In the above equation, A is a state transition matrix, which is an identity matrix I here, and represents a prior error covariance matrix based on the k-1 time and a posterior error covariance matrix at the k-1 time, if it is the first frame, then is the initial error matrix described above, and Q is a covariance matrix of a process model.

[0130] The update step is:

[0131] Equation (11)

[0132] is the optimal estimation value of the fat content fraction matrix, is a nonlinear parameter matrix observed at the current time, is a prior estimation value of the liver fat content (that is, an initial value or an optimal estimation value of the previous step), that is, a value calculated by Equation (9), and H is a measurement state matrix. K is a Kalman gain, and its calculation equation is described in Equation (12):

[0133] Equation (12)

[0134] In the above equation, is a prior error covariance matrix calculated by Equation (10), and H is a measurement transition matrix, represents its transpose matrix, and R is a covariance matrix of a measurement equation, but the matrix is changed in the present application, and the changed equation is as follows:

[0135] Equation (13)

[0136] In Equation (13), is the tissue probability weight probability map corresponding to the ROI inner tangent trapezoid calculated in step 4 of the above nonlinear parameter, is a very small parameter, and the main purpose is to ensure that the denominator of the above equation is not 0. After weighting by Equation (13), when the probability of whether a region is liver parenchyma is small, the covariance of the corresponding region in R will be increased, thereby reducing the influence weight of the prediction value of the nonlinear parameter on the final result.

[0137] The update equation of the posterior error covariance matrix is:

[0138] Equation (14)

[0139] In the above equation, I is an identity matrix, and K is a Kalman gain, is the a priori error covariance matrix.

[0140] After the update step is completed, the posterior error covariance matrix and the optimal estimation value of the update step are used as the prediction value of a new round. The average value is the final value of the liver fat content score. This method can effectively reduce the influence of noise on the measurement result. Compared with directly obtaining the average value of the liver fat content of multiple frames, this method can effectively suppress the influence of non-liver parenchymal regions on the final result and abnormal values caused by various factors.

[0141] Based on the same inventive concept, the application also provides a multi-frame image fat liver content detection system based on motion suppression. Since the principle of the problem solved by the system is similar to the aforementioned multi-frame image fat liver content detection method based on motion suppression, the implementation of the system can be referred to the implementation of the aforementioned method, and the repeated parts will not be described again. Referring to Figure 9 The system comprises:

[0142] An ultrasonic imaging module is configured to select a region of interest in a liver region of a patient and set a processing time, and acquire liver images in real time under B-mode ultrasonic scanning. The standardized positioning and data acquisition of the target region are realized, high-quality basic image data is provided for subsequent processing, and the standardization and repeatability of detection are ensured.

[0143] A liver parenchyma probability weight calculation module is configured to generate a liver parenchyma probability weight distribution map for each frame of image. The interference of non-target tissues on the detection result is effectively reduced, the key weight basis is provided for subsequent Kalman filtering, and the measurement accuracy is improved.

[0144] A motion estimation module is configured to perform inter-frame motion estimation and screen out image frames with a motion amplitude within a preset threshold. Motion artifacts caused by breathing, probe shaking or heart beating are fundamentally suppressed, the spatial consistency of the multi-frame data used for analysis is ensured, and a solid foundation is laid for accurate fusion.

[0145] A nonlinear parameter calculation module is configured to perform nonlinear parameter scanning on the screened image frames to obtain a nonlinear parameter matrix.

[0146] A fat content conversion module is configured to convert each nonlinear parameter value in the nonlinear parameter matrix into a liver fat content score value to obtain a liver fat content score matrix. The module realizes the mapping from physical parameters to the fat percentage content that can be directly understood clinically, and generates an intuitive fat distribution score matrix.

[0147] A Kalman filtering module applies a Kalman filtering algorithm to perform fusion filtering processing on the liver fat content score matrix to obtain a final liver fat content quantitative value. The aforementioned modules generate a plurality of fat content score matrices, liver parenchyma probability weights, and motion estimation information, which are fused to obtain an optimal estimation value through prediction and update iteration.

[0148] The system realizes non-invasive, accurate and stable liver fat content quantitative detection by integrating the innovative multi-frame fusion and anti-motion interference algorithm into an actual medical device system, and solves the core pain points of traditional clinical methods, such as large trauma, high cost, and inability to quantify or unstable results.

[0149] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the related parts can be referred to the method part.

[0150] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting fat liver content based on motion suppression of multi-frame images, characterized in that, The method comprises the following steps: selecting a region of interest in a liver region of a patient and setting a processing time; acquiring a liver image in real time under B-mode ultrasonic scanning; generating a liver parenchyma probability weight distribution map for each frame of image; performing inter-frame motion estimation to screen out image frames with a motion amplitude within a preset threshold; performing nonlinear parameter scanning on the screened image frames to obtain corresponding nonlinear parameter matrices; converting each nonlinear parameter value in each of the nonlinear parameter matrices into a liver fat content score value to obtain a corresponding liver fat content score matrix; applying a Kalman filtering algorithm to perform fusion filtering processing on a plurality of the liver fat content score matrices to obtain a final liver fat content quantitative value.

2. The method according to claim 1, wherein, The liver parenchyma probability weight distribution map is generated by a deep learning semantic segmentation model; the model adopts a U-Net network structure, and only the pixels of the liver parenchyma region are labeled during training, and the non-liver parenchyma region includes blood vessels, shadows, calcifications and other non-liver tissues; and the probability value of each pixel belonging to the liver parenchyma is output.

3. The method of claim 1, wherein the method is based on motion suppression. The inter-frame motion estimation includes: setting the first collected frame in the image sequence as a reference frame, and setting the frames collected subsequently as current frames in sequence; dividing the region of interest in the reference frame and the current frame into a plurality of image sub-blocks with a size of MxN pixels respectively; in the current frame, performing block matching calculation in a corresponding search area for a target image sub-block in the reference frame to find an optimal matching sub-block; calculating the relative motion amplitude of the current frame and the reference frame according to the position difference between the target image sub-block and the optimal matching sub-block; comparing the motion amplitude with a distance threshold value, if the motion amplitude is less than or equal to the distance threshold value, the current frame is retained for subsequent nonlinear parameter calculation; otherwise, the current frame is discarded.

4. The method of claim 3, wherein the method is based on motion suppression. The matching criterion used in the block matching calculation is an absolute difference sum, a square error or a normalized cross-correlation function.

5. The method of claim 1, wherein the method is based on motion suppression. The nonlinear parameter scanning on the screened image frames includes: collecting four groups of echo signals of high-voltage fundamental wave, high-voltage harmonic wave, low-voltage fundamental wave and low-voltage harmonic wave; performing median filtering on the signals; constructing and solving a nonlinear parameter equation to obtain a corresponding nonlinear parameter matrix.

6. The method of claim 1, wherein the method is based on motion suppression. The conversion of each nonlinear parameter value in each of the nonlinear parameter matrices into a liver fat content score value includes: The nonlinear parameter value of the ideal liver parenchyma is acquired as ; the nonlinear parameter value of the fat : constructing a linear relationship model according to the linear relationship between the nonlinear parameter value and the fat content of the specified liver region as follows: ； f is the fat content corresponding to each nonlinear parameter value, f is the fat content corresponding to each nonlinear parameter value, f is the fat content corresponding to each nonlinear parameter value, according to the linear relationship model, converting each element in each nonlinear parameter matrix into a liver fat content score value between 0 and 1 to obtain a liver fat content score matrix.

7. The method of claim 1, wherein the method is based on motion suppression. The fusion filtering processing on a plurality of the liver fat content score matrices by the Kalman filtering algorithm includes: establishing a state equation and an observation equation; initializing a state vector and an error covariance matrix; performing a prediction and an update step, and combining liver parenchyma probability weight adjustment with measurement noise covariance. The mean value of the output optimal fat content score matrix value is taken as the final liver fat content.

8. A motion-restrained multi-frame image-based fat content detection system, comprising: The system comprises the method of any one of claims 1-7. An ultrasound imaging module is configured to select a region of interest in a liver region of a patient and set a processing time, and acquire liver images in real time under B-mode ultrasound scanning; A liver parenchyma probability weight calculation module is configured to generate a liver parenchyma probability weight distribution map for each frame of image; A motion estimation module is configured to perform inter-frame motion estimation and screen out image frames with a motion amplitude within a preset threshold; A non-linear parameter calculation module is configured to perform non-linear parameter scanning on the screened image frames to obtain corresponding non-linear parameter matrices; A fat content conversion module is configured to convert each non-linear parameter value in each non-linear parameter matrix into a liver fat content score value to obtain a corresponding liver fat content score matrix; A Kalman filtering module is configured to perform fusion filtering processing on multiple liver fat content score matrices by using a Kalman filtering algorithm to obtain a final liver fat content quantitative value.

Images