Kidney stone component analysis method and system based on B-mode ultrasound image

Abstract

The invention relates to a kidney stone component analysis method and system based on a B ultrasonic image. The kidney stone component analysis system based on the B-mode ultrasound image comprises the following steps: constructing a deep learning model; acquiring a B-mode ultrasound image, and preprocessing the B-mode ultrasound image; performing feature extraction on the preprocessed B-mode ultrasound image based on a deep learning model to obtain image features; performing feature processing on the image features to obtain processed image features; the feature processing comprises the steps of carrying out weighting processing on image features based on an attention mechanism and carrying out feature fusion processing on the image features and clinical information of the patient based on a dynamic feature fusion mechanism, and the processed image features comprise weighted image features with attention weights and fused image features; constructing a component analysis model; and inputting the processed image features into a component analysis model for analysis to obtain kidney stone components of the B ultrasonic image. According to the method, the kidney stone components in the B-ultrasonic graph can be accurately and efficiently analyzed.

Description

Technical Field

[0001] This application relates to the technical field of medical image processing, and particularly to a method and system for analyzing the composition of kidney stones based on B-ultrasound images. Background Art

[0002] Kidney stones are a common disease, and determining the composition of kidney stones is crucial for medical research. However, in the prior art, analyzing the composition of kidney stones mainly relies on laboratory chemical analysis, which is not only time-consuming but also costly. Summary of the Invention

[0003] Based on this, it is necessary to provide a method and system for analyzing the composition of kidney stones based on B-ultrasound images to address the problems in the related art.

[0004] To achieve the above objective, in a first aspect, this application provides a method for analyzing the composition of kidney stones based on B-ultrasound images, and the method for analyzing the composition of kidney stones based on B-ultrasound images includes:

[0005] Construct a deep learning model, where the deep learning model includes an adaptive convolutional neural network;

[0006] Obtain a B-ultrasound image and preprocess the B-ultrasound image;

[0007] Extract features from the preprocessed B-ultrasound image based on the deep learning model to obtain image features;

[0008] Perform feature processing on the image features to obtain processed image features; the feature processing includes: performing weighted processing on the image features based on an attention mechanism and performing feature fusion processing on the image features and the patient's clinical information based on a dynamic feature fusion mechanism, and the processed image features include weighted image features with attention weights and fusion image features;

[0009] Construct a composition analysis model, where the composition analysis model includes a multi-level classifier;

[0010] Input the processed image features into the composition analysis model for analysis to obtain the composition of the kidney stones in the B-ultrasound image.

[0011] In some of these embodiments, the constructing of the deep learning model includes:

[0012] Obtain a data set including historical B-ultrasound images, where the data set includes historical B-ultrasound images covering various kidney stone sizes, shapes, and compositions; each of the historical B-ultrasound images is labeled with the chemical composition analysis results of the kidney stones;

[0013] Construct an initial convolutional neural network including a convolutional layer and a pooling layer;

[0014] Train the initial convolutional neural network using the said data set;

[0015] Evaluate whether the performance of the trained initial convolutional neural network triggers an adjustment mechanism; if so, adjust the number of at least one of the convolutional layer and the pooling layer in the initial convolutional neural network based on a predetermined adjustment rule; and train the adjusted initial convolutional neural network using the said data set;

[0016] Repeat the above step until the performance of the trained initial convolutional neural network does not trigger the adjustment mechanism, then obtain the deep learning model.

[0017] In some embodiments, the data set further includes synthetic B-ultrasound images; obtaining a data set including historical B-ultrasound images and synthetic B-ultrasound images includes:

[0018] Obtain historical B-ultrasound images;

[0019] Construct a generative adversarial network, which includes a generator and a discriminator;

[0020] Iteratively train the generative adversarial network based on the historical B-ultrasound images until convergence, and the generator generates synthetic B-ultrasound images;

[0021] Pool the historical B-ultrasound images and the synthetic B-ultrasound images to obtain the data set.

[0022] In some embodiments, after obtaining the deep learning model, it further includes: optimizing the deep learning model based on transfer learning.

[0023] In some embodiments, the attention weight is obtained based on the following formula:

[0024] A = softmax(fatt(X))

[0025] where X is the image feature, fatt(·) is the calculation function of the attention mechanism, and the softmax function is used to normalize the weight;

[0026] The weighted image feature is obtained based on the following formula:

[0027] F weighted = A·X

[0028] where F weighted is the weighted image feature, A is the attention weight, and X is the image feature.

[0029] In some embodiments, the multi-level classifier includes a primary classifier and a fine classifier.

[0030] In some of these embodiments, the fused image features are obtained based on the following formula:

[0031] F fused =α·V + β·C

[0032] Where F fused is the fused image feature, V is the image feature, C is the patient's clinical information, α is the dynamic weight of the image feature, and β is the dynamic weight of the clinical information.

[0033] In some of these embodiments, after inputting the processed image features into the component analysis model for analysis, the size and distribution of kidney stones in the B-ultrasound image are also obtained. After obtaining the composition, size, and distribution of kidney stones in the B-ultrasound image, the following steps are further included:

[0034] Based on the analysis results, a visual analysis report is generated.

[0035] In some of these embodiments, based on the analysis results, a visual analysis report is generated, including:

[0036] Mapping the analysis results back to the B-ultrasound image and precisely locating the position and scope of the kidney stones;

[0037] Using different colors and shapes to mark the stone areas of stones with different compositions in the B-ultrasound image;

[0038] Based on the marked B-ultrasound image, and obtaining the composition, size, and distribution of kidney stones in the B-ultrasound image, the visual analysis report is generated.

[0039] In a second aspect, the present application also provides a kidney stone composition analysis system based on B-ultrasound images; the kidney stone composition analysis system based on B-ultrasound images includes:

[0040] A data acquisition module for at least acquiring B-ultrasound images;

[0041] A preprocessing module for at least preprocessing the B-ultrasound image;

[0042] A deep learning model construction module for constructing a deep learning model; the deep learning model includes an adaptive convolutional neural network, and the deep learning model is used to extract features from the preprocessed B-ultrasound image to obtain image features;

[0043] A feature processing module for performing feature processing on the image features to obtain processed image features; the feature processing includes: performing weighted processing on the image features based on an attention mechanism and performing feature fusion processing on the image features and the clinical information of the patient based on a dynamic feature fusion mechanism, and the processed image features include weighted image features with attention weights and fused image features;

[0044] A component analysis model construction module for constructing a component analysis model; the component analysis model includes a multi-level classifier, and the component analysis model is used to analyze the processed image features to obtain the kidney stone components of the B-ultrasound image.

[0045] In the above method and system for analyzing kidney stone components based on B-ultrasound images, by constructing a deep learning model, feature extraction is performed on the preprocessed B-ultrasound image based on the deep learning model to obtain image features, weighted processing is performed on the image features based on an attention mechanism, and feature fusion processing is performed on the image features and the clinical information of the patient based on a dynamic feature fusion mechanism. The processed image features include weighted image features with attention weights and fused image features. The processed image features are input into a component analysis model including a multi-level classifier for analysis to obtain the kidney stone components of the B-ultrasound image; this application can accurately and efficiently analyze the kidney stone components in B-ultrasound images and provide a non-invasive, fast, and economical diagnostic tool for doctors. Description of the Drawings

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application or in the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0047] Figure 1 It is a flowchart of a method for analyzing kidney stone components based on B-ultrasound images provided in an embodiment of the present application;

[0048] Figure 2 It is a structural block diagram of a system for analyzing kidney stone components based on B-ultrasound images provided in another embodiment of the present application;

[0049] Figure 3 It is an internal structure diagram of a computer device provided in another embodiment of the present application.

[0050] Description of the reference numerals: 10, data acquisition module; 20, preprocessing module; 30, deep learning model construction module; 40, feature processing module; 50, component analysis model construction module. Detailed Description of the Embodiments

[0051] In order to make the objectives, technical solutions, and advantages of this application more clear and understandable, the following further details this application in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely used to explain this application and are not used to limit this application.

[0052] Kidney stones are a common disease, and determining the composition of kidney stones is crucial for medical research. However, one method for analyzing the composition of kidney stones relies on laboratory chemical analysis, which is not only time-consuming but also costly. Another method for analyzing the composition of kidney stones is for doctors to analyze the composition of kidney stones through B-ultrasound images. However, kidney stones of different compositions may have some different characteristic manifestations on B-ultrasound images. Also, due to the complexity and noise interference of B-ultrasound images, there are still great challenges in the accuracy and efficiency of artificially analyzing the composition of kidney stones based solely on B-ultrasound images.

[0053] In one embodiment, please refer to Figure 1 , this application provides a method for analyzing the composition of kidney stones based on B-ultrasound images. The method for analyzing the composition of kidney stones based on B-ultrasound images includes the following steps: S10 to S60.

[0054] S10: Construct a deep learning model, and the deep learning model includes an adaptive convolutional neural network.

[0055] S20: Obtain a B-ultrasound image and preprocess the B-ultrasound image.

[0056] S30: Extract features from the preprocessed B-ultrasound image based on the deep learning model to obtain image features.

[0057] S40: Perform feature processing on the image features to obtain processed image features; the feature processing includes: performing weighted processing on the image features based on an attention mechanism and performing feature fusion processing on the image features and the patient's clinical information based on a dynamic feature fusion mechanism. The processed image features include weighted image features with attention weights and fused image features.

[0058] S50: Construct a composition analysis model, and the composition analysis model includes a multi-level classifier.

[0059] S60: Input the processed image features into the composition analysis model for analysis to obtain the composition of the kidney stones in the B-ultrasound image.

[0060] In the method for analyzing the composition of kidney stones based on B-ultrasound images of the present application, a deep learning model is constructed, and based on the deep learning model, feature extraction is performed on the preprocessed B-ultrasound images to obtain image features. The image features are weighted based on the attention mechanism, and feature fusion processing is performed on the image features and the clinical information of the patient based on the dynamic feature fusion mechanism. The processed image features include weighted image features with attention weights and fused image features. The processed image features are input into a composition analysis model including a multi-level classifier for analysis to obtain the composition of the kidney stones in the B-ultrasound image. The present application can accurately and efficiently analyze the composition of kidney stones in B-ultrasound images, providing a non-invasive, fast, and economical diagnostic tool for doctors.

[0061] In step S10, refer to Figure 1 step S10 of, construct a deep learning model, the deep learning model includes an adaptive convolutional neural network.

[0062] As an example, in step S10, constructing the deep learning model may include the following steps: S101~S105.

[0063] S101: Obtain a data set including historical B-ultrasound images, the data set includes historical B-ultrasound images covering various kidney stone sizes, shapes, and compositions; each of the historical B-ultrasound images is labeled with the chemical composition analysis results of the kidney stones.

[0064] S102: Construct an initial convolutional neural network including a convolutional layer and a pooling layer.

[0065] S103: Use the data set to train the initial convolutional neural network.

[0066] S104: Evaluate whether the performance of the trained initial convolutional neural network triggers an adjustment mechanism; if so, adjust the number of at least one of the convolutional layer and the pooling layer in the initial convolutional neural network based on a predetermined adjustment rule; and use the data set to train the adjusted initial convolutional neural network.

[0067] S105: Repeat the previous step until the performance of the trained initial convolutional neural network does not trigger the adjustment mechanism, then obtain the deep learning model.

[0068] As an example, in step S101, obtaining a data set including historical B-ultrasound images may include the steps of data collection and preprocessing.

[0069] As an example, the data collection step may specifically include the following: collecting a wide range of historical B-ultrasound images through a data collection module (such as a data acquisition module), where the historical B-ultrasound images cover the sizes, shapes, and compositions of various kidney stones of a large number of different types; before data collection, each historical B-ultrasound image already has professional and accurate annotations to mark and display the chemical composition analysis results of the kidney stones, so as to ensure the accuracy of the composition information of the kidney stones annotated in the historical B-ultrasound images.

[0070] As an example, the preprocessing step may specifically include the following: Denoising: Filtering techniques can be used to remove random noise in the historical B-ultrasound images to retain the key features of the kidney stones in the historical B-ultrasound images; Contrast enhancement: Adjust the contrast of the historical B-ultrasound images so that the contrast between the kidney stone area and the surrounding tissues in the historical B-ultrasound images is more obvious for subsequent identification; Standardization: Convert all the processed historical B-ultrasound images into the same size and pixel value range to ensure that the data format input into the subsequent deep learning model is consistent. Through the image preprocessing step, the quality of the historical B-ultrasound images can be improved, enabling the subsequent deep learning model to more accurately identify and analyze the B-ultrasound images.

[0071] Specifically, in step S101, efforts are made to collect and organize a dataset of B-ultrasound images containing a rich variety of kidney stone compositions. Each historical B-ultrasound image has been carefully annotated to record the specific composition information of the kidney stones. A series of preprocessing operations are performed on the collected historical B-ultrasound images, including denoising to eliminate irrelevant noise in the historical B-ultrasound images, contrast enhancement to highlight the stone features, and standardization to unify the data format. The above preprocessing steps greatly improve the recognizability of the images, laying a solid foundation for subsequent composition analysis.

[0072] As an example, steps S102 to S105 are steps for constructing and training an initial convolutional neural network to obtain a deep learning model.

[0073] As an example, to improve the accuracy of a deep learning model in identifying the composition of kidney stones, an adaptive convolutional neural network (CNN) can be designed; the adaptive CNN can dynamically adjust the depth and width of its network layers according to the complexity of the input historical B-ultrasound images to meet the learning requirements of different image features. Specifically, the adaptive CNN can achieve adaptive adjustment in the following ways: during the training of the constructed initial CNN using the dataset, the number of convolutional layers and pooling layers can be automatically increased or decreased according to the difficulty of extracting image features to achieve the optimal feature extraction effect. In the adaptive adjustment of the adaptive CNN, the network structure can be determined whether to be adjusted by monitoring the performance metrics of the adaptive CNN in the current training stage, such as the change in the loss function or the accuracy of the validation set.

[0074] In a specific example, the following steps can be included: initialize an initial CNN with basic convolutional layers and pooling layers; use the dataset to train the initial CNN; after each training epoch, evaluate the performance of the trained initial CNN on the validation set; if the performance reaches the trigger adjustment mechanism, trigger the adjustment mechanism; increase or decrease at least one of the convolutional layers and pooling layers according to a predetermined rule; for example, if the loss function has not decreased significantly for several consecutive epochs, add one convolutional layer; if the loss function value drops too fast, it may be overfitting, then reduce one convolutional layer.

[0075] Specifically, an index I(t) can be set to measure the performance of the trained initial CNN (for example, the loss function value or the validation set error rate), then the adjustment mechanism can be expressed as:

[0076] If I(t) > θ and I(t) has not decreased significantly for n consecutive training epochs, add one convolutional layer;

[0077] If I(t) < θ' and I(t) has not decreased significantly for m consecutive training epochs, add one convolutional layer;

[0078] where t represents the current training epoch, θ and θ' are preset performance thresholds, and n and m are predetermined numbers of training epochs.

[0079] It should be noted that θ and θ' here need to be preset according to the specific task and dataset; in addition, the specific operation of adding or reducing layers can be achieved by modifying the weight matrix and bias terms of the initial CNN while keeping the parameters of other layers unchanged.

[0080] Step S10 is one of the core points of this application. By setting an adaptive convolutional neural network as the deep learning model, the adaptive convolutional neural network can intelligently adjust its internal structure according to the features of the B-ultrasound image, including but not limited to adjusting the number of convolutional layers and pooling layers to optimize the feature extraction process.

[0081] As an example, to improve the performance of the adaptive convolutional neural network, some improvement measures can be adopted.

[0082] In one example, the performance of the adaptive convolutional neural network can be improved by an enhanced data synthesis technique. In this example, the data set also includes synthetic B-ultrasound images; obtaining a data set including historical B-ultrasound images and synthetic B-ultrasound images can include the following steps: obtaining historical B-ultrasound images; constructing a generative adversarial network (GAN), the generative adversarial network including a generator and a discriminator; iteratively training the generative adversarial network based on the historical B-ultrasound images until convergence, the generator generating synthetic B-ultrasound images; pooling the historical B-ultrasound images and the synthetic B-ultrasound images to obtain the data set. Generating the synthetic B-ultrasound images using the generative adversarial network can increase the diversity and quantity of training data to improve the robustness and generalization ability of the deep learning model.

[0083] Specifically, an embodiment of the process of generating the synthetic B-ultrasound images by the enhanced data synthesis technique through the generative adversarial network can include the following steps: initializing a generator G and a discriminator D; training the generator G using the historical B-ultrasound images to generate as realistic B-ultrasound images as possible, while training the discriminator D using the historical B-ultrasound images to distinguish between real historical B-ultrasound images and generated synthetic B-ultrasound images; during the training process, the generator G tries to generate synthetic B-ultrasound images that the discriminator D can judge as real, while the discriminator G tries not to be deceived by the generator G; through iterative training until the generator G can generate high-quality synthetic B-ultrasound images and the discriminator D cannot distinguish between the generated synthetic B-ultrasound images and real historical B-ultrasound images.

[0084] As an example, the training process of the generative adversarial network can be summarized into the following two optimization problems:

[0085] The optimization objective of the generator is to minimize the following loss function:

[0086]

[0087] where z is a random noise sampled from the prior noise distribution p z (z), G(z) is the image generated by the generator, D(.) is the probability that the discriminator outputs a real image, and E appears in the expression of this loss function, representing the error or loss of the generator.

[0088] The optimization objective of the discriminator is to minimize the following loss function:

[0089]

[0090] where x is the real historical B-mode ultrasound image, pdata(x) is the distribution of the real historical B-mode ultrasound image, z is the random noise sampled from the prior noise distribution p z (z), G(z) is the image generated by the generator, D(·) is the probability that the discriminator outputs a real image, and E appears in the expression of this loss function, representing the error or loss of the discriminator.

[0091] In a specific example, it may include the following steps: using a certain number of real historical B-mode ultrasound images as training data; designing a generative adversarial network GAN for the generator G and the discriminator D, where the generator G usually includes multiple deconvolution layers and the discriminator D usually includes multiple convolution layers; training the generative adversarial network GAN until convergence, so that the generator G can generate realistic synthetic B-mode ultrasound images; using the generated synthetic B-mode ultrasound images and the real historical B-mode ultrasound images together as the dataset for training the deep learning model to improve the robustness and generalization ability of the deep learning model.

[0092] Through the enhanced data synthesis technology in the above example, the generative adversarial network helps to synthesize new training data, increasing the diversity and quantity of the dataset, thus contributing to improving the performance of the deep learning model.

[0093] In another example, the deep learning model can also be optimized based on transfer learning. Through transfer learning technology, existing image recognition models (such as ResNet or Inception) can be utilized to improve the training effect and accuracy of the deep learning model.

[0094] Transfer learning is a technique that utilizes a model pre-trained on a large dataset to improve the performance of a specific task; a specific example of transfer learning in the B-ultrasound image recognition task can include the following: Select a pre-trained image recognition model, such as ResNet or Inception, which are usually trained on large-scale image datasets; Remove the top layer (usually the fully connected layer) of the pre-trained image recognition model, which is designed for the original task (such as ImageNet classification); Add a new fully connected layer, the number of output nodes of which corresponds to the number of class identifications of the current task (for example, if the task is to identify different lesion types in B-ultrasound images, the number of output nodes is equal to the number of lesion types); Connect the new fully connected layer to the rest of the pre-trained image recognition model to form a new model; Fine-tune the new model using a small number of labeled historical B-ultrasound images, that is, continue training on the new dataset, usually using a smaller learning rate to avoid destroying the feature extraction ability of the pre-trained model.

[0095] Specifically, the fine-tuning process of transfer learning can be described by the optimization problem of the following steps:

[0096] First, define a loss function L to measure the difference between the model prediction and the true label. A common loss function is the cross-entropy loss:

[0097]

[0098] where n is the number of samples, yi is the true label of the i-th sample, is the predicted probability of the i-th sample.

[0099] The optimization goal in the fine-tuning process is to minimize the loss on the new dataset:

[0100]

[0101] where x is the input historical B-ultrasound image, θ is the model parameter, f(x;θ) represents the prediction of the deep learning model for the input x, and y is the true label.

[0102] A specific example of the fine-tuning process can include the following: Collect a certain number of annotated historical B-ultrasound images as the fine-tuning dataset; Remove the top layer from the pre-trained Resnet or Inception model and add a new fully connected layer; Use the cross-entropy loss as the loss function and select a suitable optimizer (such as SGD or Adam); Fine-tune the model using a smaller learning rate until the performance of the model on the B-ultrasound image dataset no longer improves; Evaluate the accuracy and efficiency of the fine-tuned model in the B-ultrasound image recognition task.

[0103] Through transfer learning, the rich features learned by a pre-trained model can be utilized, thereby improving the training effect and accuracy of the model on specific tasks.

[0104] In step S20, refer to Figure 1 step S20 in [[ID=]], and obtain a B-ultrasound image and preprocess the B-ultrasound image.

[0105] As an example, the B-ultrasound image in this step can be obtained in the same way as the historical B-ultrasound image obtained in step S10; however, it should be noted that there is no annotation information in the B-ultrasound image in this step, but only the original B-ultrasound image obtained by the data acquisition module. For example, the B-ultrasound image obtained by the data acquisition module from a B-ultrasound device. Similarly, the B-ultrasound image in this step can be preprocessed in the same way as the historical B-ultrasound image preprocessing in step S10; that is, in this step, the B-ultrasound image is preprocessed such as denoising, enhancing contrast, and normalizing.

[0106] In step S30, refer to Figure 1 step S30 in [[ID=]], and perform feature extraction on the preprocessed B-ultrasound image based on the deep learning model to obtain image features.

[0107] As an example, the preprocessed B-ultrasound image in step S20 can be input into the deep learning model for feature extraction to obtain the image features.

[0108] In step S40, refer to Figure 1 step S40 in [[ID=]], and perform feature processing on the image features to obtain processed image features; the feature processing includes: performing weighted processing on the image features based on an attention mechanism and performing feature fusion processing on the image features and the patient's clinical information based on a dynamic feature fusion mechanism. The processed image features include weighted image features with attention weights and fused image features.

[0109] As an example, through a weight assignment mechanism, the attention mechanism enables subsequent component analysis models to automatically identify and focus on the stone area when processing images, ignoring irrelevant background information, thereby improving the accuracy of classification.

[0110] As an example, a specific example of weighting the image features based on the attention mechanism may include the following steps: Define an attention module that learns the importance weights of each pixel or feature on the B-mode ultrasound image; Input the B-mode ultrasound image into the attention module to generate an attention weight map, which can represent the importance of each part of the B-mode ultrasound image for kidney stone recognition; Use the attention weight map to weight the image features to obtain weighted image features, enabling the component analysis model to focus on the stone area.

[0111] As an example, the attention weights can be obtained based on the following formula:

[0112] A = softmax(fatt(X))

[0113] Where X is the image feature, fatt(·) is the calculation function of the attention mechanism, and the softmax function is used to normalize the weights.

[0114] As an example, the weighted image features can be obtained based on the following formula:

[0115] F weighted = A·X

[0116] Where Fweighted is the weighted image feature, A is the attention weight, and X is the image feature.

[0117] As an example, dynamic feature fusion is an advanced feature processing technology that allows the model (such as the component analysis model) to dynamically learn and combine information from different sources during training. A specific example may include the following: Extract image features from the B-mode ultrasound image, and at the same time collect the patient's clinical information, such as age, gender, and medical history, etc.; Convert the image features and clinical information into numerical representations that can be processed by the component analysis model; Design a feature fusion module that can dynamically adjust the weights between the image features and clinical information during the training process of the component analysis model to learn the best fusion method.

[0118] As an example, the clinical information can be obtained through a data acquisition module.

[0119] As an example, the fused image features can be obtained based on the following formula:

[0120] F fused = α·V + β·C

[0121] Where F fused is the fused image feature, V is the image feature, C is the patient's clinical information, α is the dynamic weight of the image feature, and β is the dynamic weight of the clinical information.

[0122] As an example, the learning of dynamic weights can be achieved in the following way:

[0123] α,β=MLP(V,C)

[0124] where MLP is a multi-layer perceptron that can learn weights α and β based on image features V and clinical information C.

[0125] The above dynamic fusion method allows the model to learn the feature fusion strategy according to the importance of the data itself, rather than relying on pre-set rules or parameters, thereby improving the accuracy of the model in identifying stone composition analysis.

[0126] In step S50, refer to Figure 1 step S50 of, and construct a composition analysis model, where the composition analysis model includes a multi-level classifier.

[0127] As an example, the constructed composition analysis model may include a primary classifier and a detailed classifier; the primary classification area quickly distinguishes whether there are kidney stones in the B-ultrasound image. For B-ultrasound images with kidney stones, the detailed classifier is used to further identify the specific composition of the kidney stones. Specifically, the role of the primary classifier is to quickly screen the B-ultrasound image to avoid unnecessary subsequent processing of images without kidney stones. The detailed classifier is responsible for precisely classifying the composition of the kidney stones, such as distinguishing calcium stones and urate stones, etc.

[0128] As an example, the purpose of the primary classifier is to quickly identify and exclude those B-ultrasound images without kidney stones to reduce the computational amount and time of subsequent processing. The primary classifier can be a simple binary classification model, and its output result can be "with kidney stones" or "without kidney stones". The formula corresponding to the primary classifier can be:

[0129]

[0130] where x is the input image feature, fprimary(x) is the output score of the primary classifier, and τ is the score threshold.

[0131] As an example, the detailed classifier further precisely classifies the B-ultrasound images identified by the primary classifier as "with kidney stones", for example, distinguishing calcium stones and urate stones, etc. The detailed classifier is usually a multi-classification model, and its output is the specific type of kidney stones. The formula corresponding to the detailed classifier is described as:

[0132] C detail (x)=argkmaxf detail (x)k

[0133] where C detail (x) is the output category of the detailed classifier, fdetail (x)k is the output score of the fine classifier for the k-th category.

[0134] In a specific example, the following content may be included:

[0135] Train the primary classifier: The historical B-ultrasound images labeled as "containing kidney stones" and "not containing kidney stones" can be used to train the primary classifier, and the primary classifier should be simple and fast enough for preliminary screening; here, the input data for training the primary classifier can be the historical image features obtained by extracting features from the historical B-ultrasound images in the previous step, or directly the historical B-ultrasound images.

[0136] Train the fine classifier: Use the historical B-ultrasound images labeled with different types of kidney stones to train the fine classifier, and the input data of the fine classifier can be the historical image features obtained by extracting features from the historical B-ultrasound images with kidney stones in the previous step.

[0137] Apply the primary classifier: The primary classifier performs a primary classification on the input B-ultrasound image and only passes the B-ultrasound images classified as "containing kidney stones" to the fine classifier; at this time, the input data of the primary classifier can be the image features obtained in the previous step.

[0138] Apply the fine classifier: Perform an accurate classification on the B-ultrasound images screened by the primary classifier and output the specific component types of the kidney stones.

[0139] Another specific example may include the following content: Collect data: Obtain a dataset containing historical B-ultrasound images and the clinical information of the corresponding patients; Preprocessing and feature extraction: Preprocess the historical B-ultrasound images and extract features from the preprocessed historical B-ultrasound images to obtain historical image features; Normalize the clinical information; Feature fusion: Use a trainable feature fusion module to combine the historical image features and the clinical information; Train the model: Train a multi-level classifier based on the data obtained previously; Evaluate the performance: Evaluate the performance of the multi-level classifier on the test set, and dynamic feature fusion can improve the classification accuracy.

[0140] In step S60, please refer to Figure 1 step S60 in, input the processed image features into the composition analysis model for analysis to obtain the kidney stone composition of the B-ultrasound image.

[0141] Suppose there is a preprocessed B-ultrasound image, and the primary classifier determines whether there are kidney stones in the B-ultrasound image; then, the deep learning model extracts the deep features of the B-ultrasound image to obtain image features; the attention mechanism then focuses on the stone area and dynamically adjusts the importance of the features; finally, combining the clinical data of the patient, the feature fusion module generates the final image feature representation (fused image features); the fine classifier identifies the specific components of the kidney stones in the B-ultrasound image based on the final image features. The above process not only improves the recognition accuracy but also increases the interpretability of the model.

[0142] As an example, in step S60, after inputting the processed image features into the composition analysis model for analysis, the size and distribution of the kidney stones in the B-ultrasound image are also obtained. After obtaining the composition of the kidney stones in the B-ultrasound image, the size and distribution of the kidney stones, that is, after step S60, the following steps may also be included:

[0143] S70: Generate a visual analysis report based on the analysis results.

[0144] As an example, generating a visual analysis report based on the analysis results, that is, step S70 may include the following steps: S701~S703.

[0145] S701: Map the analysis results back to the B-ultrasound image and accurately locate the position and scope of the kidney stones.

[0146] S702: Use different colors and shapes to mark the stone areas of kidney stones with different components in the B-ultrasound image.

[0147] S703: Generate the visual analysis report based on the marked B-ultrasound image, the composition of the kidney stones in the B-ultrasound image, the size and distribution of the kidney stones.

[0148] As an example, in step S701, regarding feature mapping and positioning, the following may be included: First, map the analysis results back to the original B-ultrasound image and accurately locate the position and scope of the kidney stones; this step can be achieved by calculating the maximum response value of the features of the kidney stones in the B-ultrasound image.

[0149] Specifically, the following formula can be used to determine the position of the kidney stones in the B-ultrasound image :

[0150]

[0151] where represents the feature score of the kidney stone at position .

[0152] As an example, in step S702, which involves visualization processing, the following content may be included: Visualize the positioning information obtained in step S701. Different stone regions of different component stones can be marked with different colors and shapes, so that the position and composition of kidney stones can be clearly identified at a glance.

[0153] Specifically, visualization can be achieved by assigning a color value C(i, j) to each pixel I(i, j) in the stone region R. The specific formula is as follows:

[0154]

[0155] where C stone represents the stone region, and C background represents the color of the background.

[0156] As an example, in step S703, which involves generating a visual analysis report, the following content may be included: Integrate the B-ultrasound image after visualization processing and related analysis data (for example, the composition of the kidney stone obtained from the B-ultrasound image) into a detailed visual analysis report. The visual analysis report not only includes the size, shape, and composition information of the kidney stone, but also can provide a description of the relationship between the kidney stone and the surrounding tissues.

[0157] On the basis that the composition of the kidney stone in the B-ultrasound image has been identified and the position of the kidney stone has been located in step S60, in the visualization processing stage, the stone region can be marked red and the background region remains gray. In the generated visual analysis report, in addition to including the marked B-ultrasound image, a table can also be attached, listing in detail the size, composition, and position information of the kidney stone.

[0158] The method for analyzing the composition of kidney stones based on B-ultrasound images in this application can effectively identify and classify the composition of kidney stones through a deep learning model and image processing methods. It has the characteristics of non-invasive, fast, and accurate, and can be widely used in clinical diagnosis.

[0159] It should be understood that although Figure 1 the steps in the flowchart are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise clearly stated in this article, there is no strict order limit for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 at least a part of the steps in Figure 1 may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps.

[0160] In another embodiment, refer to Figure 2 , the present application also provides a kidney stone composition analysis system based on B-ultrasound images. The kidney stone composition analysis system based on B-ultrasound images may include: a data acquisition module 10, a preprocessing module 20, a deep learning model construction module 30, a feature processing module 40, and a composition analysis model construction module 50. Among them, the data acquisition module 10 is used to acquire at least B-ultrasound images; the preprocessing module 20 is used to preprocess at least the B-ultrasound images; the deep learning model construction module 30 is used to construct a deep learning model. The deep learning model 30 includes an adaptive convolutional neural network, and the deep learning model 30 is used to extract features from the preprocessed B-ultrasound images to obtain image features; the feature processing module 40 is used to perform feature processing on the image features to obtain processed image features. The feature processing includes: performing weighted processing on the image features based on an attention mechanism and performing feature fusion processing on the image features and the patient's clinical information based on a dynamic feature fusion mechanism. The processed image features include weighted image features with attention weights and fused image features; the composition analysis model construction module 50 is used to construct a composition analysis model. The composition analysis model 50 includes a multi-level classifier, and the composition analysis model 50 is used to analyze the processed image features to obtain the kidney stone composition of the B-ultrasound image.

[0161] In the kidney stone composition analysis system based on B-ultrasound images of the present application, by setting the data acquisition module 10, the preprocessing module 20, the deep learning model construction module 30, the feature processing module 40, and the composition analysis model construction module 50, it is possible to accurately and efficiently analyze the kidney stone composition in the B-ultrasound image, providing a non-invasive, fast, and economical diagnostic tool for doctors.

[0162] As an example, the kidney stone composition analysis system based on B-ultrasound images of the present application can be used to execute the Figure 1 and related embodiments of the kidney stone composition analysis method based on B-ultrasound images.

[0163] In yet another embodiment, the present application also provides a computer device, which may be a server, and its internal structure diagram may be as Figure 3As shown. The computer device includes a processor, a memory, a network interface, and a database connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as historical work data and real-time work data. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, it is used to implement as Figure 1 the method for analyzing the composition of kidney stones based on B-ultrasound images and its embodiments described in

[0164] In another embodiment, the present application further provides a computer device, which can be a terminal, and its internal structure diagram can be as Figure 3 shown. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, it is used to implement as Figure 1 the method for analyzing the composition of kidney stones based on B-ultrasound images and its embodiments described in. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covered on the display screen, or a button, a trackball, or a touchpad provided on the computer device housing, or an external keyboard, touchpad, or mouse, etc.

[0165] Those skilled in the art can understand that Figure 3 the structure shown in

[0166] is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements. Figure 1 the method for analyzing the composition of kidney stones based on B-ultrasound images and its embodiments described in

[0167] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features of the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0168] The above embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed. However, it should not be construed as a limitation on the scope of the patent application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several deformations and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

Claims

1. A method for analyzing kidney stone components based on B-ultrasound images, characterized in that: include: Constructing a deep learning model, wherein the deep learning model includes an adaptive convolutional neural network; Acquire a B-ultrasound image and preprocess the B-ultrasound image; Performing feature extraction on the preprocessed B-ultrasound image based on the deep learning model to obtain image features; Performing feature processing on the image features to obtain processed image features; The feature processing includes: performing weighted processing on the image features based on an attention mechanism and performing feature fusion processing on the image features and the patient's clinical information based on a dynamic feature fusion mechanism, wherein the processed image features include weighted image features with attention weights and fused image features; Constructing a component analysis model, wherein the component analysis model includes a multi-level classifier; The processed image features are input into the component analysis model for analysis to obtain the kidney stone components of the B-ultrasound image.

2. The method according to claim 1, characterized in that The construction of the deep learning model includes: Acquire a data set including historical B-ultrasound images, wherein the data set includes historical B-ultrasound images covering various sizes, shapes and compositions of kidney stones; each of the historical B-ultrasound images is annotated with a chemical composition analysis result of the kidney stones; Construct an initial convolutional neural network including convolutional layers and pooling layers; Using the data set to train the initial convolutional neural network; Evaluate whether the performance of the trained initial convolutional neural network triggers an adjustment mechanism; if so, adjust the number of at least one of the convolutional layers and the pooling layers in the initial convolutional neural network based on a predetermined adjustment rule; and train the adjusted initial convolutional neural network using the data set; Repeat the previous step until the performance of the trained initial convolutional neural network does not trigger the adjustment mechanism, thereby obtaining the deep learning model.

3. The method according to claim 2, characterized in that The data set also includes a synthesized B-ultrasound image; obtaining a data set including historical B-ultrasound images and synthesized B-ultrasound images, including: Obtain historical B-ultrasound images; Constructing a generative adversarial network, wherein the generative adversarial network includes a generator and a discriminator; Iteratively training the generative adversarial network based on the historical B-ultrasound image until convergence, and the generator generates a synthetic B-ultrasound image; The historical B-ultrasound images and the synthetic B-ultrasound images are collected to obtain the data set.

4. The method according to claim 2, characterized in that: After obtaining the deep learning model, it also includes: optimizing the deep learning model based on transfer learning.

5. The method according to claim 1, characterized in that The attention weight is obtained based on the following formula: A=softmax(fatt(X)) Among them, X is the image feature, fatt(·) is the calculation function of the attention mechanism, and the softmax function is used to normalize the weight; The weighted image features are obtained based on the following formula: F weighted =A·X Among them, F weighted is the weighted image feature, A is the attention weight, and X is the image feature.

6. The method according to claim 1, characterized in that The fused image features are obtained based on the following formula: F fused =α·V+β·C Among them, F fused is the fusion image feature, V is the image feature, C is the patient's clinical information, α is the dynamic weight of the image feature, and β is the dynamic weight of the clinical information.

7. The method according to claim 1, characterized in that The multi-level classifier includes a primary classifier and a detailed classifier.

8. The method according to any one of claims 1 to 7, characterized in that After the processed image features are input into the component analysis model for analysis, the size and distribution of kidney stones in the B-ultrasound image are obtained. After the kidney stone components, the size and distribution of kidney stones in the B-ultrasound image are obtained, the method further includes: Generate a visual analysis report based on the analysis results.

9. The method according to claim 8, characterized in that Based on the analysis results, a visual analysis report is generated, including: Mapping the analysis result back to the B-ultrasound image and accurately locating the position and range of the kidney stones; Using different colors and shapes to mark the stone areas of different stone compositions in the B-ultrasound image; The visual analysis report is generated based on the annotated B-ultrasound image, the kidney stone components obtained in the B-ultrasound image, and the size and distribution of the kidney stones.

10. A kidney stone component analysis system based on B-ultrasound images, characterized in that: include: A data acquisition module, used for acquiring at least a B-ultrasound image; A preprocessing module, used for at least preprocessing the B-ultrasound image; A deep learning model construction module, used to construct a deep learning model; the deep learning model includes an adaptive convolutional neural network, and the deep learning model is used to extract features from the preprocessed B-ultrasound image to obtain image features; A feature processing module, used for performing feature processing on the image features to obtain processed image features; The feature processing includes: performing weighted processing on the image features based on an attention mechanism and performing feature fusion processing on the image features and the patient's clinical information based on a dynamic feature fusion mechanism, wherein the processed image features include weighted image features with attention weights and fused image features; The component analysis model construction module is used to construct a component analysis model; the component analysis model includes a multi-level classifier, and the component analysis model is used to analyze the processed image features to obtain the kidney stone components of the B-ultrasound image.

Images