Method and device for calculating liver-kidney ratio of ultrasonic image

Abstract

The invention provides a method and device for calculating the liver-kidney ratio of an ultrasonic image, and relates to the technical field of ultrasonic image analysis, and the method comprises the steps: obtaining a target ultrasonic image, carrying out the recognition and segmentation of a liver region and a kidney region in the target ultrasonic image through a pre-constructed cascading convolutional neural network frame, and obtaining the liver-kidney ratio of the target ultrasonic image; obtaining a target liver mask and a target kidney mask; based on predefined ROI (Region of Interest) parameters, filtering the target ultrasonic image, taking pixel points in a filtering result, which are positioned in the target liver mask, as liver ROI selection results, and taking pixel points in the filtering result, which are positioned in the target kidney mask, as kidney ROI selection results; and based on a pixel pair selection result, estimating the liver-kidney ratio of the target ultrasonic image through a statistical method. Through the method provided by the invention, quantitative estimation of the liver-kidney ratio is realized, and errors caused by manual operation of technicians are effectively avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of ultrasonic image analysis, and in particular to a method and device for calculating the liver-kidney ratio of an ultrasonic image. Background Art

[0002] Non-alcoholic fatty liver disease (NAFLD) is defined as a condition characterized by liver cells containing more than 5% fat. If untreated, it significantly increases the risk of developing cirrhosis and even liver cancer. Liver biopsy is a common clinical diagnostic method for fatty liver disease, but due to its invasive nature and high diagnostic cost, it is not accepted by most patients. Non-invasive methods for diagnosing fatty liver disease include magnetic resonance imaging (MRI) and ultrasound imaging (US). US, with its low cost, lack of radiation, and rapidity and convenience, has become the preferred method for diagnosing liver disease. In the early stages of ultrasound imaging technology, diagnostic results based on liver ultrasound images relied on subjective interpretation by medical personnel, which required a high level of knowledge and experience. To overcome these limitations, automated and semi-automated diagnostic technologies have been developed in the field of ultrasound medicine.

[0003] The hepatorenal index (HI) is considered an effective indicator in the diagnosis of fatty liver disease. It is defined as the ratio of the average brightness of the liver parenchyma and renal cortex in liver ultrasound images. When the liver fat content is high, the liver brightness in the ultrasound grayscale image will be significantly higher than the kidney brightness, and this fact can be used to effectively diagnose fatty liver disease. Current medical research shows that the HI is highly correlated with liver fat content. Diagnosing fatty liver by calculating the HI has been proven to be a stable and highly reproducible method. However, manual measurement was often required in the early stages. In clinical practice, doctors usually need to select a region of interest (ROI) in the liver ultrasound image and calculate the average brightness to obtain the HI, which can lead to large errors due to manual operation by technicians.

[0004] How to realize the automatic calculation of the liver-kidney ratio in ultrasound images and effectively avoid the errors caused by manual operation of technicians is a technical problem that needs to be solved at present. Summary of the Invention

[0005] The present invention provides a method and device for calculating the liver-kidney ratio of an ultrasonic image, so as to solve the defects in the prior art.

[0006] The present invention provides a method for calculating the liver-kidney ratio of an ultrasound image, comprising the following steps: Acquire a target ultrasound image, and identify and segment the liver and kidney regions in the target ultrasound image using a pre-built cascade convolutional neural network framework to obtain a target liver mask and a target kidney mask; wherein the cascade convolutional neural network framework includes two convolutional neural network-based joint recognition models for respectively performing coarse segmentation and fine segmentation on the target ultrasound image; Based on predefined region of interest (ROI) parameters, filtering the target ultrasound image, using the pixel points in the filtering result that are located within the target liver mask as the liver ROI selection result, and using the pixel points in the filtering result that are located within the target kidney mask as the kidney ROI selection result; Based on the pixel pair selection result, the liver-kidney ratio of the target ultrasound image is estimated by a statistical method.

[0007] According to a method for calculating the liver-kidney ratio of an ultrasound image provided by the present invention, the two joint recognition models based on convolutional neural networks are respectively: a first joint recognition model and a second joint recognition model, the first joint recognition model is used to perform coarse segmentation on the target ultrasound image, and the second joint recognition model is used to perform fine segmentation on the output result of the first joint recognition model; The method of identifying and segmenting the liver region and the kidney region in the target ultrasound image by using a pre-built cascade convolutional neural network framework to obtain a target liver mask and a target kidney mask includes: Inputting the target ultrasound image into the first joint recognition model to obtain a coarsely segmented liver mask and a coarsely segmented kidney mask; The coarsely segmented liver mask and the coarsely segmented kidney mask are input into the second joint recognition model to obtain the target liver mask and the target kidney mask.

[0008] According to a method for calculating the liver-kidney ratio of an ultrasound image provided by the present invention, inputting the target ultrasound image into the first joint recognition model to obtain a coarse segmentation liver mask and a coarse segmentation kidney mask includes: Inputting the target ultrasound image into the first joint recognition model to obtain a first liver recognition result and a first kidney recognition result; Image post-processing is performed on the first liver recognition result and the first kidney recognition result respectively to obtain a coarsely segmented liver image and a corresponding coarsely segmented liver mask, and a coarsely segmented kidney image and a corresponding coarsely segmented kidney mask.

[0009] According to a method for calculating the liver-kidney ratio of an ultrasound image provided by the present invention, inputting the coarsely segmented liver mask and the coarsely segmented kidney mask into the second joint recognition model to obtain the target liver mask and the target kidney mask includes: inputting the coarsely segmented liver image and the corresponding coarsely segmented liver mask, and the coarsely segmented kidney image and the corresponding coarsely segmented kidney mask into the second joint recognition model to obtain a second liver recognition result and a second kidney recognition result; Image post-processing is performed on the second liver recognition result and the second kidney recognition result to obtain the target liver mask and the target kidney mask.

[0010] According to a method for calculating the liver-kidney ratio of an ultrasound image provided by the present invention, filtering the target ultrasound image based on predefined region of interest (ROI) parameters includes: Performing mean filtering on the target ultrasound image using a filter kernel having the same size and shape as a predefined region of interest (ROI) to obtain an initial filtering result; The initial filtering result is subjected to edge corrosion processing by using an corrosion kernel having the same size and shape as the filtering kernel to obtain the filtering result.

[0011] According to a method for calculating the liver-kidney ratio of an ultrasound image provided by the present invention, the method of selecting the pixel points located within the target liver mask in the filtering result as the liver ROI selection result, and selecting the pixel points located within the target kidney mask in the filtering result as the kidney ROI selection result, includes: The pixel points in the filtering result that are located at the same depth within the target liver mask are used as the liver ROI selection result, and the pixel points in the filtering result that are located at the same depth within the target kidney mask are used as the kidney ROI selection result.

[0012] According to a method for calculating the liver-kidney ratio of an ultrasound image provided by the present invention, the method of estimating the liver-kidney ratio of the target ultrasound image by a statistical method based on the pixel pair selection result includes: Based on the pixel pair selection results, a first average brightness value of the liver ROI and a second average brightness value of the kidney ROI are calculated; calculating a ratio of the first average brightness value to the second average brightness value to obtain an estimated liver-kidney ratio of the target ultrasound image; Statistical analysis is performed on a preset number of estimated values ​​of the liver-kidney ratio of the target ultrasound image to obtain the liver-kidney ratio of the target ultrasound image.

[0013] The present invention also provides a device for calculating the liver-kidney ratio of an ultrasound image, comprising the following modules: a segmentation module, configured to acquire a target ultrasound image and identify and segment the liver and kidney regions in the target ultrasound image using a pre-built cascade convolutional neural network framework to obtain a target liver mask and a target kidney mask; wherein the cascade convolutional neural network framework includes two convolutional neural network-based joint recognition models for respectively performing coarse segmentation and fine segmentation on the target ultrasound image; a filtering module, configured to perform filtering processing on the target ultrasound image based on predefined region of interest (ROI) parameters, and to select the pixel points within the target liver mask in the filtering result as the liver ROI selection result, and to select the pixel points within the target kidney mask in the filtering result as the kidney ROI selection result; A calculation module is used to estimate the liver-kidney ratio of the target ultrasound image by a statistical method based on the pixel point pair selection result.

[0014] The present invention also provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for calculating the liver-kidney ratio of an ultrasound image as described above is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for calculating the liver-kidney ratio of an ultrasound image as described in any one of the above is implemented.

[0016] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the above methods for calculating the liver-kidney ratio of an ultrasound image.

[0017] The present invention provides a method and device for calculating the liver-kidney ratio in an ultrasound image. The method comprises acquiring a target ultrasound image, identifying and segmenting the liver and kidney regions in the target ultrasound image using a pre-built cascade convolutional neural network framework, and obtaining a target liver mask and a target kidney mask. The cascade convolutional neural network framework comprises: two joint recognition models based on convolutional neural networks, for respectively performing coarse and fine segmentation on the target ultrasound image; filtering the target ultrasound image based on predefined region of interest (ROI) parameters, selecting the liver ROI as the liver ROI selection result, and selecting the kidney ROI as the kidney ROI selection result; and estimating the liver-kidney ratio of the target ultrasound image using statistical methods based on the pixel pair selection results. Thus, the present invention utilizes an image filtering processing method to transform the positioning and selection of ROIs into pixel pairing and statistical analysis problems, thereby achieving quantitative estimation of the liver-kidney ratio. This effectively avoids errors caused by manual operation by technicians and provides better interpretability and repeatability in medicine. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 It is a flow chart of the method for calculating the liver-kidney ratio of ultrasound images provided by the present invention.

[0020] Figure 2 It is a flow chart of using the cascaded convolutional neural network framework provided by the present invention.

[0021] Figure 3 It is a flow chart of the liver-kidney ratio estimation method based on the filtering algorithm provided by the present invention.

[0022] Figure 4 It is a schematic diagram of multi-peak fitting of a large number of liver-kidney ratio estimation values ​​using the Gaussian mixture model (GMM) provided by the present invention.

[0023] Figure 5 This is one of the schematic diagrams of the results of binary classification diagnosis of patient samples based on the calculation method of the liver-kidney ratio of ultrasound images provided by the present invention.

[0024] Figure 6 This is the second schematic diagram of the results of binary classification diagnosis of patient samples based on the calculation method of the liver-kidney ratio of ultrasound images provided by the present invention.

[0025] Figure 7 It is a schematic structural diagram of the device for calculating the liver-kidney ratio of ultrasound images provided by the present invention.

[0026] Figure 8 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0027] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0028] The following combination Figures 1-8 The present invention describes a method and device for calculating the liver-kidney ratio of an ultrasound image.

[0029] Figure 1 FIG. 1 is a flow chart of a method for calculating the liver-kidney ratio of an ultrasound image provided by the present invention, as shown in FIG. Figure 1 As shown, the method includes the following: Step 100: Acquire a target ultrasound image, and identify and segment the liver region and kidney region in the target ultrasound image using a pre-constructed cascade convolutional neural network framework to obtain a target liver mask and a target kidney mask; wherein the cascade convolutional neural network framework includes: two joint recognition models based on convolutional neural networks, which are used to perform coarse segmentation and fine segmentation on the target ultrasound image, respectively.

[0030] It should be noted that the two joint recognition models based on convolutional neural networks are: a first joint recognition model and a second joint recognition model. The first joint recognition model is used to perform coarse segmentation of the target ultrasound image, and the second joint recognition model is used to perform fine segmentation of the output result of the first joint recognition model.

[0031] In step 100, the liver region and the kidney region in the target ultrasound image are identified and segmented using a pre-built cascade convolutional neural network framework to obtain a target liver mask and a target kidney mask, including: Step 110: Input the target ultrasound image into the first joint recognition model to obtain a coarsely segmented liver mask and a coarsely segmented kidney mask.

[0032] Step 110 specifically includes: Step 111: Input the target ultrasound image into the first joint recognition model to obtain a first liver recognition result and a first kidney recognition result.

[0033] Step 112: perform image post-processing on the first liver recognition result and the first kidney recognition result respectively to obtain a coarsely segmented liver image and a corresponding coarsely segmented liver mask, and a coarsely segmented kidney image and a corresponding coarsely segmented kidney mask.

[0034] Step 120: Input the coarsely segmented liver mask and the coarsely segmented kidney mask into the second joint recognition model to obtain the target liver mask and the target kidney mask.

[0035] Step 120 specifically includes: Step 121: Input the coarsely segmented liver image and the corresponding coarsely segmented liver mask, and the coarsely segmented kidney image and the corresponding coarsely segmented kidney mask into the second joint recognition model to obtain a second liver recognition result and a second kidney recognition result.

[0036] Step 122: perform image post-processing on the second liver recognition result and the second kidney recognition result respectively to obtain the target liver mask and the target kidney mask.

[0037] Specifically, this embodiment simultaneously identifies the liver and kidney regions in the target ultrasound image, employing a cascaded framework. The liver and kidney ultrasound images are first input into the first joint recognition model. The output liver and kidney recognition results undergo image post-processing, resulting in new images and corresponding masks. These new images are then input into the second joint recognition model. The outputs undergo the same image post-processing and are resized back to their original size to yield the final recognition result. This framework connects two identically structured joint recognition CNNs (Convolutional Neural Networks), hence its name, a two-step joint recognition framework using cascaded CNNs. The network architecture of the selected single model can be any state-of-the-art segmentation model.

[0038] This example employs a joint recognition strategy: a single CNN with a single-channel input and dual-channel output is used to simultaneously identify the liver and kidney regions. Using a single model for simultaneous liver and kidney recognition not only saves computational resources and time, but also fully utilizes the relative spatial position of the liver and kidneys in the image.

[0039] Morphological image post-processing is performed to improve mask quality: the output result is divided into connected domains, and only the largest connected domain is retained (ensuring that there is only one block); the interior of the image is filled to remove internal holes; finally, the smallest rectangular area containing the liver and kidney recognition results is cut out, and the same area is cut out in the corresponding original image. In this way, an ultrasound image with most of the valid area and the corresponding mask are obtained.

[0040] Separate joint recognition models are trained, and the new image and corresponding mask generated after image post-processing are fed into the more accurate joint recognition model. The first CNN locates the target region in the image and provides a coarse segmentation result for the liver (or kidney). The second CNN has the same structure as the first, except for an additional input channel. It performs fine segmentation of the target. The first CNN is trained using the ground-truth labels, while the second CNN is trained using rectangular cutouts of the region corresponding to the original image within the ground-truth labels.

[0041] Step 200: Filter the target ultrasound image based on predefined region of interest (ROI) parameters, and use the pixel points in the filtering result that are within the target liver mask as the liver ROI selection result, and use the pixel points in the filtering result that are within the target kidney mask as the kidney ROI selection result.

[0042] In step 200, filtering is performed on the target ultrasound image based on predefined region of interest (ROI) parameters, including: Step 210 : Perform mean filtering on the target ultrasound image using a filter kernel having the same size and shape as the predefined region of interest (ROI) to obtain an initial filtering result.

[0043] Step 220: Perform edge erosion processing on the initial filtering result using an erosion kernel having the same size and shape as the filtering kernel to obtain the filtering result.

[0044] In step 200, the pixel points in the filtering result located within the target liver mask are used as the liver ROI selection result, and the pixel points in the filtering result located within the target kidney mask are used as the kidney ROI selection result, including: Step 230: Pixel points in the filtering result that are located at the same depth within the target liver mask are used as the liver ROI selection result, and pixel points in the filtering result that are located at the same depth within the target kidney mask are used as the kidney ROI selection result.

[0045] Specifically, clinically calculating HI requires defining a fixed-size ROI on each of the liver and kidneys. The HI is then calculated by calculating the ratio of the average brightness of the liver and kidney ROIs. This embodiment employs a filtering algorithm to replace the manual ROI extraction process. A filter kernel matching the size and shape of the predefined ROIs is used to perform mean filtering on the ultrasound image. The brightness of each pixel in the filtered result is equivalent to the average brightness of the ROI centered at that point in the original image. This transforms the ROI selection and positioning process into a matter of pairing and statistically analyzing a large number of pixels in the filtered result.

[0046] To avoid counting points at the edges of the liver and kidney regions in the original image, which could result in the inclusion of non-liver / kidney regions and affect subsequent analysis accuracy, the identified liver and kidney masks are subjected to edge erosion, with the erosion kernel size and shape consistent with the filter kernel. Under the constraint of equal depth lines, pixel pairs within the liver and kidney masks (after erosion) in the filtered image are selected. Specifically, points within the eroded liver mask are selected as the liver ROI selection result, and points within the eroded kidney mask are selected as the kidney ROI selection result. These points are ensured to be at the same depth in the ultrasound image. All matching point pairs are identified, and the brightness ratio is calculated as the corresponding HI estimate. The depth constraint is added to prevent brightness deviations caused by varying depths in the ultrasound image.

[0047] Step 300: Estimate the liver-kidney ratio of the target ultrasound image using a statistical method based on the pixel pair selection result.

[0048] Step 300 specifically includes: Step 310: Based on the pixel pair selection result, a first average brightness value of the liver ROI and a second average brightness value of the kidney ROI are calculated.

[0049] Step 320: Calculate the ratio of the first average brightness value to the second average brightness value to obtain an estimated liver-kidney ratio of the target ultrasound image.

[0050] Step 330: Perform statistical analysis on a preset number of estimated values ​​of the liver-kidney ratio of the target ultrasound images to obtain the liver-kidney ratio of the target ultrasound image.

[0051] Specifically, a statistical analysis was performed on a large number of HI estimation results. Here, a Gaussian mixture model (GMM) was used to perform multimodal fitting on a large amount of data. The principle of this method is to assume that the sample data of each Gaussian class obeys a one-dimensional Gaussian distribution with the probability density shown in the following formula: Pick For the The mean and standard deviation of the Gaussian classes, For the Gaussian class weights, generating A standard one-dimensional Gaussian model, the distribution of the mixed Gaussian model satisfies the formula: Experiments were conducted with different numbers of peaks N, and the fitting result with the lowest Bayesian information content (BIC) was taken as the result of multi-peak fitting, so as to adaptively determine the number of peaks N for multi-peak fitting. Excluding the HI estimation results that were too high and too low, the middle peak (the The mean of the peaks was taken as the HI calculation result for subsequent diagnosis.

[0052] The method of this embodiment performs statistical analysis on a large amount of point pair data in the liver / kidney region, and is therefore relatively insensitive to the accuracy of the liver / kidney recognition results. Even if the region edge segmentation is inaccurate, it will not significantly affect the results.

[0053] The above is a description of the steps of the method for calculating the liver-kidney ratio of an ultrasound image provided by the present invention. From the description of the above steps, it can be seen that according to the method for calculating the liver-kidney ratio of an ultrasound image provided by the present invention, by acquiring a target ultrasound image, the liver area and the kidney area in the target ultrasound image are identified and segmented by a pre-constructed cascade convolutional neural network framework to obtain a target liver mask and a target kidney mask; wherein the cascade convolutional neural network framework includes: two joint recognition models based on convolutional neural networks, which are used to perform coarse segmentation and fine segmentation on the target ultrasound image respectively; based on predefined region of interest (ROI) parameters, the target ultrasound image is filtered, and the pixel points in the target liver mask in the filtering result are used as the liver ROI selection result, and the pixel points in the target kidney mask in the filtering result are used as the kidney ROI selection result; based on the pixel point selection result, the liver-kidney ratio of the target ultrasound image is estimated by statistical methods. It can be seen from this that the present invention converts the positioning and selection of ROI into pixel pairing and statistical analysis problems through the use of image filtering processing methods, thereby achieving quantitative estimation of the liver-kidney ratio, effectively avoiding the errors caused by manual operations of technicians, and having better interpretability and repeatability in medicine.

[0054] Based on the above embodiment, the method for calculating the liver-kidney ratio of an ultrasound image provided by an embodiment of the present invention is further described.

[0055] The liver ultrasound dataset used in this example comes from the Department of Internal Medicine, Hypertension, and Vascular Disease. This is an open-source 2D liver ultrasound dataset originally collected for the assessment of hepatic steatosis. It contains B-mode liver ultrasound images with a resolution of 434×636 (pixel size 0.373 mm×0.373 mm). A total of 55 obese patients (including 38 with fatty liver disease) participated in the liver ultrasound data collection. Ten frames were sampled from each patient's liver ultrasound video during one heartbeat, resulting in 550 ultrasound images (including 380 of fatty liver disease).

[0056] Liver biopsy results were also collected for each patient. The liver fat content determined by the biopsy was used as the gold standard for diagnosing steatosis, with patients with a liver fat content ≥5% considered to have fatty liver disease. The ratio of fatty liver to normal liver in the 55 patient samples was 38:17.

[0057] In the experiment, UNet was used as the deep learning model for the liver / kidney recognition task. Figure 2This is a flow chart of the cascaded convolutional neural network framework provided by the present invention. Figure 2 The proposed process of using the cascaded convolutional neural network framework is shown. The original ultrasound image is used as the input (Input) of the first joint recognition model for rough segmentation (RoughSegmentation). The output result is post-processed (Post-Processing) to obtain the coarse segmentation image and the corresponding coarse segmentation mask. It serves as the input (Input) of the second joint recognition model for fine segmentation (Final Segmentation) to obtain the final result (Result). Figure 3 : is a flow chart of the liver-kidney ratio estimation method based on the filtering algorithm provided by the present invention, Figure 3 The proposed HI estimation method based on filtering algorithm is presented. The target liver mask and target kidney mask are processed by mean filtering and then edge corrosion is performed. Figure 3 The eroded mask (ErodedMask) is shown. Pixel points are selected from the eroded liver and kidney masks using point pairing (Depth Constraint). Only liver-kidney point pairs at the same depth are retained to prevent brightness deviations caused by different depths in the ultrasound image. Figure 4 It is a schematic diagram of multi-peak fitting of a large number of liver-kidney ratio estimates using the Gaussian mixture model (GMM) provided by the present invention. In the figure, the HI calculation result is the class mean of the middle peak, 0.8864. Figure 5 This is one of the schematic diagrams of the results of binary classification diagnosis of patient samples based on the calculation method of the liver-kidney ratio of ultrasound images provided by the present invention, which shows the binary classification confusion matrix. Figure 6 This is the second schematic diagram of the results of binary classification diagnosis of patient samples based on the ultrasound image liver-kidney ratio calculation method provided by the present invention, which shows the ROC (Receiver Operating Characteristic Curve) curve and the corresponding area under the curve. Automated fatty liver diagnosis is performed with an HI = 0.9 threshold (samples with HI > 0.9 are considered to have fatty liver disease), and voting diagnosis is performed on 10 ultrasound image samples of each patient. The binary confusion matrix and the corresponding ROC curve are shown as follows: Figure 5 and Figure 6 As shown in Figure 2, only one misclassified sample among the 55 patient samples was found, resulting in a classification accuracy of 98.18%. In contrast, a previous study using the same dataset for fatty liver disease diagnosis achieved an accuracy of only 96.36% (including two misclassified samples). Furthermore, the area under the receiver operating characteristic (AUROC) curve (AROC) reached 0.9288.

[0058] In summary, the present invention proposes a method for automatically calculating the liver-kidney ratio (LIRR) in ultrasound images. This method leverages the relative spatial positional characteristics of the liver and kidney regions, employing a cascade framework to perform the recognition task. It also transforms the ROI selection and positioning process into a pixel pairing and statistical analysis problem, allowing for fully automatic calculation of the HI, thereby improving the accuracy of ultrasound-based fatty liver diagnosis.

[0059] The following describes a device for calculating the liver-kidney ratio of an ultrasound image provided by the present invention. The device for calculating the liver-kidney ratio of an ultrasound image described below and the method for calculating the liver-kidney ratio of an ultrasound image described above can refer to each other.

[0060] Figure 7 FIG. 1 is a schematic diagram of the structure of the device for calculating the liver-kidney ratio of an ultrasonic image provided by the present invention. Figure 7 As shown, the device for calculating the liver-kidney ratio of an ultrasound image provided by the present invention includes: A segmentation module 701 is configured to acquire a target ultrasound image and identify and segment the liver and kidney regions in the target ultrasound image using a pre-built cascade convolutional neural network framework to obtain a target liver mask and a target kidney mask. The cascade convolutional neural network framework includes two convolutional neural network-based joint recognition models for respectively performing coarse segmentation and fine segmentation on the target ultrasound image. A filtering module 702 is configured to perform filtering on the target ultrasound image based on predefined region of interest (ROI) parameters, and to select pixels within the target liver mask as liver ROI selection results, and pixels within the target kidney mask as kidney ROI selection results. The calculation module 703 is configured to estimate the liver-kidney ratio of the target ultrasound image by a statistical method based on the pixel pair selection result.

[0061] The present invention provides a device for calculating the liver-kidney ratio in an ultrasound image. The device obtains a target ultrasound image and identifies and segments the liver and kidney regions in the target ultrasound image using a pre-built cascade convolutional neural network framework to obtain a target liver mask and a target kidney mask. The cascade convolutional neural network framework includes two joint recognition models based on convolutional neural networks for respectively performing coarse and fine segmentation on the target ultrasound image. The target ultrasound image is filtered based on predefined region of interest (ROI) parameters, and pixels within the target liver mask in the filtering result are used as liver ROI selection results, and pixels within the target kidney mask in the filtering result are used as kidney ROI selection results. Based on the pixel pair selection results, the liver-kidney ratio of the target ultrasound image is estimated using a statistical method. It can be seen that the present invention converts the positioning and selection of ROIs into pixel pairing and statistical analysis problems by using an image filtering processing method, thereby achieving quantitative estimation of the liver-kidney ratio, effectively avoiding errors caused by manual operation by technicians, and having better interpretability and repeatability in medicine.

[0062] Based on the above embodiment, in this embodiment, the two joint recognition models based on convolutional neural networks are respectively: a first joint recognition model and a second joint recognition model, the first joint recognition model is used to perform coarse segmentation on the target ultrasound image, and the second joint recognition model is used to perform fine segmentation on the output result of the first joint recognition model; The segmentation module 701 is specifically configured to: Inputting the target ultrasound image into the first joint recognition model to obtain a coarsely segmented liver mask and a coarsely segmented kidney mask; The coarsely segmented liver mask and the coarsely segmented kidney mask are input into the second joint recognition model to obtain the target liver mask and the target kidney mask.

[0063] Based on the above embodiment, in this embodiment, the segmentation module 701 is specifically configured to: Inputting the target ultrasound image into the first joint recognition model to obtain a first liver recognition result and a first kidney recognition result; Image post-processing is performed on the first liver recognition result and the first kidney recognition result respectively to obtain a coarsely segmented liver image and a corresponding coarsely segmented liver mask, and a coarsely segmented kidney image and a corresponding coarsely segmented kidney mask.

[0064] Based on the above embodiment, in this embodiment, the segmentation module 701 is specifically configured to: inputting the coarsely segmented liver image and the corresponding coarsely segmented liver mask, and the coarsely segmented kidney image and the corresponding coarsely segmented kidney mask into the second joint recognition model to obtain a second liver recognition result and a second kidney recognition result; Image post-processing is performed on the second liver recognition result and the second kidney recognition result to obtain the target liver mask and the target kidney mask.

[0065] Based on the above embodiment, in this embodiment, the filtering module 702 is specifically configured to: Performing mean filtering on the target ultrasound image using a filter kernel having the same size and shape as a predefined region of interest (ROI) to obtain an initial filtering result; The initial filtering result is subjected to edge corrosion processing by using an corrosion kernel having the same size and shape as the filtering kernel to obtain the filtering result.

[0066] Based on the above embodiment, in this embodiment, the device further includes a selection module, which is specifically configured to: The pixel points in the filtering result that are located at the same depth within the target liver mask are used as the liver ROI selection result, and the pixel points in the filtering result that are located at the same depth within the target kidney mask are used as the kidney ROI selection result.

[0067] Based on the above embodiment, in this embodiment, the calculation module 703 is specifically used to: Based on the pixel pair selection results, a first average brightness value of the liver ROI and a second average brightness value of the kidney ROI are calculated; calculating a ratio of the first average brightness value to the second average brightness value to obtain an estimated liver-kidney ratio of the target ultrasound image; Statistical analysis is performed on a preset number of estimated values ​​of the liver-kidney ratio of the target ultrasound image to obtain the liver-kidney ratio of the target ultrasound image.

[0068] Figure 8 An example of a physical structure diagram of an electronic device is shown below. Figure 8 As shown, the electronic device may be a robot or other electronic device, and may include: a processor 810, a communications interface 820, a memory 830, and a communication bus 840. The processor 810, the communications interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 may call logic instructions in the memory 830 to execute a method for calculating the liver-kidney ratio of an ultrasound image, including: Acquire a target ultrasound image, and identify and segment the liver and kidney regions in the target ultrasound image using a pre-built cascade convolutional neural network framework to obtain a target liver mask and a target kidney mask; wherein the cascade convolutional neural network framework includes two convolutional neural network-based joint recognition models for respectively performing coarse segmentation and fine segmentation on the target ultrasound image; Based on predefined region of interest (ROI) parameters, filtering the target ultrasound image, using the pixel points in the filtering result that are located within the target liver mask as the liver ROI selection result, and using the pixel points in the filtering result that are located within the target kidney mask as the kidney ROI selection result; Based on the pixel pair selection result, the liver-kidney ratio of the target ultrasound image is estimated by a statistical method.

[0069] Furthermore, the logic instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0070] In another aspect, the present invention further provides a computer program product, comprising a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the methods for calculating the liver-kidney ratio in ultrasound images provided by the above methods, including: Acquire a target ultrasound image, and identify and segment the liver and kidney regions in the target ultrasound image using a pre-built cascade convolutional neural network framework to obtain a target liver mask and a target kidney mask; wherein the cascade convolutional neural network framework includes two convolutional neural network-based joint recognition models for respectively performing coarse segmentation and fine segmentation on the target ultrasound image; Based on predefined region of interest (ROI) parameters, filtering the target ultrasound image, using the pixel points in the filtering result that are located within the target liver mask as the liver ROI selection result, and using the pixel points in the filtering result that are located within the target kidney mask as the kidney ROI selection result; Based on the pixel pair selection result, the liver-kidney ratio of the target ultrasound image is estimated by a statistical method.

[0071] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for calculating the liver-kidney ratio in an ultrasound image provided by the above methods, including: Acquire a target ultrasound image, and identify and segment the liver and kidney regions in the target ultrasound image using a pre-built cascade convolutional neural network framework to obtain a target liver mask and a target kidney mask; wherein the cascade convolutional neural network framework includes two convolutional neural network-based joint recognition models for respectively performing coarse segmentation and fine segmentation on the target ultrasound image; Based on predefined region of interest (ROI) parameters, filtering the target ultrasound image, using the pixel points in the filtering result that are located within the target liver mask as the liver ROI selection result, and using the pixel points in the filtering result that are located within the target kidney mask as the kidney ROI selection result; Based on the pixel pair selection result, the liver-kidney ratio of the target ultrasound image is estimated by a statistical method.

[0072] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0073] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0074] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for calculating the liver-kidney ratio of an ultrasound image, characterized in that: include: Acquire a target ultrasound image, and identify and segment the liver and kidney regions in the target ultrasound image using a pre-built cascade convolutional neural network framework to obtain a target liver mask and a target kidney mask; wherein the cascade convolutional neural network framework includes two convolutional neural network-based joint recognition models for respectively performing coarse segmentation and fine segmentation on the target ultrasound image; Based on predefined region of interest (ROI) parameters, filtering the target ultrasound image, using the pixel points in the filtering result that are located within the target liver mask as the liver ROI selection result, and using the pixel points in the filtering result that are located within the target kidney mask as the kidney ROI selection result; Based on the pixel pair selection result, the liver-kidney ratio of the target ultrasound image is estimated by a statistical method.

2. The method for calculating the liver-kidney ratio of an ultrasound image according to claim 1, wherein: The two joint recognition models based on convolutional neural networks are respectively: a first joint recognition model and a second joint recognition model, the first joint recognition model is used to perform coarse segmentation on the target ultrasound image, and the second joint recognition model is used to perform fine segmentation on the output result of the first joint recognition model; The method of identifying and segmenting the liver region and the kidney region in the target ultrasound image by using a pre-built cascade convolutional neural network framework to obtain a target liver mask and a target kidney mask includes: Inputting the target ultrasound image into the first joint recognition model to obtain a coarsely segmented liver mask and a coarsely segmented kidney mask; The coarsely segmented liver mask and the coarsely segmented kidney mask are input into the second joint recognition model to obtain the target liver mask and the target kidney mask.

3. The method for calculating the liver-kidney ratio of an ultrasound image according to claim 2, wherein: Inputting the target ultrasound image into the first joint recognition model to obtain a coarsely segmented liver mask and a coarsely segmented kidney mask includes: Inputting the target ultrasound image into the first joint recognition model to obtain a first liver recognition result and a first kidney recognition result; Image post-processing is performed on the first liver recognition result and the first kidney recognition result respectively to obtain a coarsely segmented liver image and a corresponding coarsely segmented liver mask, and a coarsely segmented kidney image and a corresponding coarsely segmented kidney mask.

4. The method for calculating the liver-kidney ratio of an ultrasound image according to claim 3, wherein: Inputting the coarsely segmented liver mask and the coarsely segmented kidney mask into the second joint recognition model to obtain the target liver mask and the target kidney mask includes: inputting the coarsely segmented liver image and the corresponding coarsely segmented liver mask, and the coarsely segmented kidney image and the corresponding coarsely segmented kidney mask into the second joint recognition model to obtain a second liver recognition result and a second kidney recognition result; Image post-processing is performed on the second liver recognition result and the second kidney recognition result to obtain the target liver mask and the target kidney mask.

5. The method for calculating the liver-kidney ratio of an ultrasound image according to claim 1, wherein: The filtering process of the target ultrasound image based on predefined region of interest (ROI) parameters includes: Performing mean filtering on the target ultrasound image using a filter kernel having the same size and shape as a predefined region of interest (ROI) to obtain an initial filtering result; The initial filtering result is subjected to edge corrosion processing by using an corrosion kernel having the same size and shape as the filtering kernel to obtain the filtering result.

6. The method for calculating the liver-kidney ratio of an ultrasound image according to claim 5, characterized in that: The step of selecting the pixel points in the filtering result that are within the target liver mask as the liver ROI selection result, and selecting the pixel points in the filtering result that are within the target kidney mask as the kidney ROI selection result, includes: The pixel points in the filtering result that are located at the same depth within the target liver mask are used as the liver ROI selection result, and the pixel points in the filtering result that are located at the same depth within the target kidney mask are used as the kidney ROI selection result.

7. The method for calculating the liver-kidney ratio of an ultrasound image according to claim 1, wherein: The estimating the liver-kidney ratio of the target ultrasound image by a statistical method based on the pixel pair selection result includes: Based on the pixel pair selection results, a first average brightness value of the liver ROI and a second average brightness value of the kidney ROI are calculated; calculating a ratio of the first average brightness value to the second average brightness value to obtain an estimated liver-kidney ratio of the target ultrasound image; Statistical analysis is performed on a preset number of estimated values ​​of the liver-kidney ratio of the target ultrasound image to obtain the liver-kidney ratio of the target ultrasound image.

8. A device for calculating the liver-kidney ratio of an ultrasound image, characterized in that: include: a segmentation module, configured to acquire a target ultrasound image and identify and segment the liver and kidney regions in the target ultrasound image using a pre-built cascade convolutional neural network framework to obtain a target liver mask and a target kidney mask; wherein the cascade convolutional neural network framework includes two convolutional neural network-based joint recognition models for respectively performing coarse segmentation and fine segmentation on the target ultrasound image; a filtering module, configured to perform filtering processing on the target ultrasound image based on predefined region of interest (ROI) parameters, and to select the pixel points within the target liver mask in the filtering result as the liver ROI selection result, and to select the pixel points within the target kidney mask in the filtering result as the kidney ROI selection result; A calculation module is used to estimate the liver-kidney ratio of the target ultrasound image by a statistical method based on the pixel point pair selection result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for calculating the liver-kidney ratio of an ultrasound image according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for calculating the liver-kidney ratio of an ultrasound image according to any one of claims 1 to 7 is implemented.

Images