Method and system for analyzing kidney stone composition based on b-ultrasound images

Abstract

The application relates to a kidney stone composition analysis method and system based on a B-ultrasound image. The kidney stone composition analysis system based on the B-ultrasound image comprises the following steps: constructing a deep learning model; acquiring a B-ultrasound image and preprocessing the B-ultrasound image; performing feature extraction on the preprocessed B-ultrasound image based on the deep learning model to obtain image features; performing feature processing on the image features to obtain processed image features; the feature processing comprises: performing weighted processing on the image features based on an attention mechanism and performing feature fusion processing on the image features and clinical information of a patient based on a dynamic feature fusion mechanism; the processed image features comprise weighted image features with attention weights and fusion image features; constructing a composition analysis model; inputting the processed image features into the composition analysis model for analysis to obtain kidney stone composition of the B-ultrasound image. The application can accurately and efficiently analyze kidney stone composition in a B-ultrasound image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image processing, in particular to a kidney stone composition analysis method and system based on B-ultrasound images. BACKGROUND

[0002] Kidney stones are a common disease, and determining the composition of kidney stones is crucial for medical research. However, in the prior art, the analysis of the composition of kidney stones mainly relies on laboratory chemical analysis, which is not only time-consuming but also costly. SUMMARY

[0003] Therefore, it is necessary to provide a kidney stone composition analysis method and system based on B-ultrasound images to solve the problems in the related art.

[0004] To achieve the above-mentioned purpose, in a first aspect, the present application provides a kidney stone composition analysis method based on B-ultrasound images, which comprises:

[0005] constructing a deep learning model, wherein the deep learning model comprises an adaptive convolutional neural network;

[0006] obtaining a B-ultrasound image and pre-processing the B-ultrasound image;

[0007] performing feature extraction on the pre-processed B-ultrasound image based on the deep learning model to obtain image features;

[0008] performing feature processing on the image features to obtain processed image features, wherein the feature processing comprises: performing weighted processing on the image features based on an attention mechanism and performing feature fusion processing on the image features and clinical information of a patient based on a dynamic feature fusion mechanism, and the processed image features comprise weighted image features with attention weights and fusion image features;

[0009] constructing a composition analysis model, wherein the composition analysis model comprises a multi-level classifier;

[0010] inputting the processed image features into the composition analysis model for analysis to obtain the composition of kidney stones of the B-ultrasound image.

[0011] In some embodiments, the construction of the deep learning model comprises:

[0012] obtaining a data set comprising historical B-ultrasound images, wherein the data set comprises historical B-ultrasound images covering various sizes, shapes and compositions of kidney stones, and each of the historical B-ultrasound images is labeled with a chemical composition analysis result of kidney stones;

[0013] constructing an initial convolutional neural network comprising a convolutional layer and a pooling layer;

[0014] Using the data set to train the initial convolutional neural network;

[0015] evaluating whether the performance of the trained initial convolutional neural network triggers an adjustment mechanism; if so, adjusting the number of at least one of the convolutional layers and the pooling layers in the initial convolutional neural network based on a predetermined adjustment rule; and training the adjusted initial convolutional neural network using the dataset;

[0016] Repeat the previous step until the performance of the trained initial convolutional neural network does not trigger the adjustment mechanism, thereby obtaining the deep learning model.

[0017] In some embodiments, the data set further includes a synthesized B-ultrasound image; acquiring the data set including the historical B-ultrasound image and the synthesized B-ultrasound image comprises:

[0018] Obtain historical B-ultrasound images;

[0019] Constructing a generative adversarial network, wherein the generative adversarial network includes a generator and a discriminator;

[0020] Iteratively training the generative adversarial network based on the historical B-ultrasound images until convergence, wherein the generator generates a synthetic B-ultrasound image;

[0021] The historical B-ultrasound images and the synthetic B-ultrasound images are collected to obtain the data set.

[0022] In some embodiments, after obtaining the deep learning model, the method further includes: optimizing the deep learning model based on transfer learning.

[0023] In some embodiments, the attention weight is obtained based on the following formula:

[0024] A=softmax(fatt(X))

[0025] Where X is the image feature, fatt(·) is the calculation function of the attention mechanism, and the softmax function is used to normalize the weights;

[0026] The weighted image features are obtained based on the following formula:

[0027] F weighted =A·X

[0028] Among them, F weighted is the weighted image feature, A is the attention weight, and X is the image feature.

[0029] In some embodiments, the multi-stage classifier includes a primary classifier and a detailed classifier.

[0030] In some embodiments, the fused image feature is obtained based on the following formula:

[0031] F fused =α·V+β·C

[0032] wherein F fused is the fused image feature, V is the image feature, C is the clinical information of the patient, and a is the dynamic weight of the image feature, and β is the dynamic weight of the clinical information.

[0033] In some embodiments, after the processed image feature is input into the component analysis model for analysis, the size and distribution of the kidney stones in the B-ultrasound image are also obtained, and after the kidney stone components, the size and distribution of the kidney stones in the B-ultrasound image are obtained, the method further comprises:

[0034] generating a visual analysis report based on the analysis result.

[0035] In some embodiments, the visual analysis report is generated based on the analysis result, comprising:

[0036] mapping the analysis result back to the B-ultrasound image and accurately positioning the location and range of the kidney stones;

[0037] annotating the stone area of the stones of different components in the B-ultrasound image using different colors and shapes;

[0038] generating the visual analysis report based on the annotated B-ultrasound image, the kidney stone components, the size and distribution of the kidney stones in the B-ultrasound image.

[0039] In a second aspect, the present application also provides a kidney stone component analysis system based on a B-ultrasound image; the kidney stone component analysis system based on a B-ultrasound image comprises:

[0040] a data acquisition module configured to acquire at least a B-ultrasound image;

[0041] a preprocessing module configured to preprocess at least the B-ultrasound image;

[0042] a deep learning model construction module configured to construct a deep learning model; the deep learning model comprises an adaptive convolutional neural network, and the deep learning model is configured to extract features from the preprocessed B-ultrasound image to obtain an image feature;

[0043] The feature processing module is configured to perform feature processing on the image features to obtain processed image features; the feature processing comprises: performing weighted processing on the image features based on an attention mechanism and performing feature fusion processing on the image features and clinical information of the patient based on a dynamic feature fusion mechanism; the processed image features comprise weighted image features with attention weights and fusion image features.

[0044] The component analysis model construction module is configured to construct a component analysis model; the component analysis model comprises a multi-level classifier, and the component analysis model is configured to analyze the processed image features to obtain the kidney stone component of the B-ultrasound image.

[0045] In the above kidney stone component analysis method and system based on a B-ultrasound image, a deep learning model is constructed, the deep learning model is used to perform feature extraction on a preprocessed B-ultrasound image to obtain image features, the image features are weighted based on an attention mechanism, and the image features and clinical information of a patient are fused based on a dynamic feature fusion mechanism; the processed image features comprise weighted image features with attention weights and fusion image features; the processed image features are input into a component analysis model comprising a multi-level classifier for analysis, and the kidney stone component of the B-ultrasound image is obtained; the application can accurately and efficiently analyze the kidney stone component in a B-ultrasound image, and provides a non-invasive, rapid and economical diagnostic tool for doctors. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative effort.

[0047] Figure 1 A flowchart of the kidney stone component analysis method based on a B-ultrasound image provided in an embodiment of the application;

[0048] Figure 2 A structural block diagram of the kidney stone component analysis system based on a B-ultrasound image provided in another embodiment of the application;

[0049] Figure 3 An internal structure diagram of a computer device provided in still another embodiment of the application.

[0050] Legend of reference numerals: 10, data acquisition module; 20, preprocessing module; 30, deep learning model construction module; 40, feature processing module; 50, component analysis model construction module. DETAILED DESCRIPTION

[0051] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0052] Kidney stones are a common disease, and determining the composition of kidney stones is crucial for medical research. However, one method of analyzing the composition of kidney stones relies on laboratory chemical analysis, which is not only time-consuming but also costly. Another method of analyzing the composition of kidney stones is for doctors to analyze the composition of kidney stones through B-ultrasound images, but kidney stones of different compositions may have some different performance characteristics on B-ultrasound images, and due to the complexity and noise interference of B-ultrasound images, there are still great challenges in accuracy and efficiency based on artificial analysis of kidney stone composition based on B-ultrasound images.

[0053] In one embodiment, referring to Figure 1 The present application provides a B-ultrasound image-based kidney stone composition analysis method, which comprises the following steps: S10-S60.

[0054] S10: Constructing a deep learning model, the deep learning model comprising an adaptive convolutional neural network.

[0055] S20: Obtaining a B-ultrasound image and pre-processing the B-ultrasound image.

[0056] S30: Feature extraction of the pre-processed B-ultrasound image based on the deep learning model to obtain image features.

[0057] S40: Feature processing of the image features to obtain processed image features; the feature processing comprising: weighting processing of the image features based on an attention mechanism and feature fusion processing of the image features and clinical information of a patient based on a dynamic feature fusion mechanism, the processed image features comprising weighted image features with attention weights and fusion image features.

[0058] S50: Constructing a composition analysis model, the composition analysis model comprising a multi-level classifier.

[0059] S60: Inputting the processed image features into the composition analysis model for analysis to obtain the kidney stone composition of the B-ultrasound image.

[0060] In the kidney stone composition analysis method based on B-ultrasound images of the present application, a deep learning model is constructed, the pre-processed B-ultrasound images are subjected to feature extraction based on the deep learning model to obtain image features, the image features are subjected to weighted processing based on an attention mechanism, and the image features and the clinical information of a patient are subjected to feature fusion processing based on a dynamic feature fusion mechanism, the processed image features include weighted image features with attention weights and fusion image features, the processed image features are input into a composition analysis model including a multi-level classifier for analysis to obtain the kidney stone composition of the B-ultrasound images; the present application can accurately and efficiently analyze the kidney stone composition in the B-ultrasound images, and provide a non-invasive, rapid and economical diagnostic tool for doctors.

[0061] In step S10, please refer to Figure 1 The deep learning model includes an adaptive convolutional neural network.

[0062] As an example, in step S10, constructing the deep learning model can include the following steps: S101-S105.

[0063] S101: Obtain a data set including historical B-ultrasound images, the data set including historical B-ultrasound images covering various kidney stone sizes, shapes and compositions; each of the historical B-ultrasound images is labeled with a chemical composition analysis result of kidney stones.

[0064] S102: Construct an initial convolutional neural network including convolutional layers and pooling layers.

[0065] S103: Train the initial convolutional neural network using the data set.

[0066] S104: Evaluate whether the performance of the trained initial convolutional neural network triggers an adjustment mechanism; if yes, adjust the number of at least one of the convolutional layers and the pooling layers in the initial convolutional neural network based on a predetermined adjustment rule; and train the adjusted initial convolutional neural network using the data set.

[0067] S105: Repeat the previous step until the performance of the trained initial convolutional neural network does not trigger the adjustment mechanism, and then obtain the deep learning model.

[0068] As an example, in step S101, obtaining the data set including historical B-ultrasound images can include the steps of data collection and preprocessing.

[0069] As an example, the data collection step can specifically include the following: collecting a wide range of historical B-ultrasound images through a data collection module (for example, a data acquisition module), which cover a large number of different types of various kidney stones in terms of size, shape and composition; before data collection, each historical B-ultrasound image has professional and accurate annotations to mark the chemical composition analysis results of kidney stones to ensure the accuracy of the composition information of the annotated kidney stones in the historical B-ultrasound images.

[0070] As an example, the preprocessing step can specifically include the following: denoising: filtering techniques can be used to remove random noise in historical B-ultrasound images to preserve the key features of kidney stones in historical B-ultrasound images; contrast enhancement: adjusting the contrast of historical B-ultrasound images to make the contrast between kidney stone regions and surrounding tissues in historical B-ultrasound images more obvious to facilitate subsequent recognition; standardization: converting all the above processed historical B-ultrasound images to the same size and pixel value range to ensure consistent data format input into the subsequent deep learning model. Through the image preprocessing step, the quality of the historical B-ultrasound images can be improved, so that the subsequent deep learning model can more accurately identify and analyze the B-ultrasound images.

[0071] Specifically, in step S101, a B-ultrasound image dataset containing a variety of kidney stone compositions is collected and organized. Each historical B-ultrasound image is carefully annotated, recording the specific composition information of the kidney stones. A series of preprocessing operations are performed on the collected historical B-ultrasound images, including denoising to eliminate irrelevant noise in the historical B-ultrasound images, contrast enhancement to highlight stone features, and standardization to unify data formats. The above preprocessing steps greatly improve the recognizability of the images, laying a solid foundation for subsequent composition analysis.

[0072] As an example, steps S102 to S105 are steps for constructing and training an initial convolutional neural network to obtain a deep learning model.

[0073] As an example, to improve the accuracy of a deep learning model in identifying kidney stone composition, an adaptive convolutional neural network (CNN) can be designed. The adaptive CNN can dynamically adjust its network depth and width based on the complexity of the input historical B-mode ultrasound images to adapt to the learning needs of different image features. Specifically, the adaptive CNN can achieve adaptive adjustment in the following ways: during the training of the initial CNN using the dataset, the number of convolutional and pooling layers can be automatically increased or decreased based on the difficulty of feature extraction to achieve optimal feature extraction. In the adaptive adjustment of the adaptive CNN, the performance indicators of the adaptive CNN at the current training stage, such as the change of the loss function or the accuracy of the validation set, can be monitored to determine whether to adjust the network structure.

[0074] In a specific example, the following steps can be included: initializing an initial CNN with basic convolutional and pooling layers; training the initial CNN using the dataset; evaluating the performance of the trained initial CNN on the validation set after each training period; triggering the adjustment mechanism if the performance reaches the trigger mechanism; increasing or decreasing the number of at least one of the convolutional and pooling layers according to predetermined rules; for example, if the loss function does not decrease significantly for several periods, a convolutional layer is added; if the loss function value decreases too quickly, it may be overfitting, and a convolutional layer is reduced.

[0075] Specifically, an indicator I(t) can be set to measure the performance of the trained initial CNN (e.g., loss function value or validation set error rate), and the adjustment mechanism can be represented as:

[0076] If I(t) > θ and I(t) does not decrease significantly for n consecutive training periods, a convolutional layer is added;

[0077] If I(t) < θ' and I(t) does not decrease significantly for m consecutive training periods, a convolutional layer is added;

[0078] Where t represents the current training period, θ and θ' are preset performance thresholds, and n and m are predetermined training period numbers.

[0079] It should be noted that θ and θ' here need to be pre-set according to the specific task and dataset; in addition, the specific operation of increasing or decreasing the layer can be realized by modifying the weight matrix and bias term of the initial CNN, while keeping the parameters of other layers unchanged.

[0080] Step S10 is one of the core points of the present application, by setting an adaptive convolutional neural network as a deep learning model, the adaptive convolutional neural network can intelligently adjust its internal structure according to the characteristics of the B-ultrasound image, including but not limited to adjusting the number of convolutional layers and pooling layers, to optimize the feature extraction process.

[0081] As an example, in order to improve the performance of the adaptive convolutional neural network, some improvement measures can be adopted.

[0082] In one example, the performance of the adaptive convolutional neural network can be improved by enhanced data synthesis technology. In this example, the data set also includes synthetic B-ultrasound images; obtaining a data set including historical B-ultrasound images and synthetic B-ultrasound images can include the following steps: obtaining historical B-ultrasound images; constructing a generative adversarial network (GAN), the generative adversarial network including a generator and a discriminator; iteratively training the generative adversarial network based on the historical B-ultrasound images until convergence, the generator generating synthetic B-ultrasound images; and integrating the historical B-ultrasound images and the synthetic B-ultrasound images to obtain the data set. Using a generative adversarial network to generate the synthetic B-ultrasound images can increase the diversity and quantity of training data, thereby improving the robustness and generalization ability of the deep learning model.

[0083] Specifically, one embodiment of the process of generating the synthetic B-ultrasound images by the generative adversarial network using enhanced data synthesis technology can include the following steps: initializing a generator G and a discriminator D; training the generator G using historical B-ultrasound images to generate as realistic B-ultrasound images as possible, while training the discriminator D using historical B-ultrasound images to distinguish between real historical B-ultrasound images and generated synthetic B-ultrasound images; during the training process, the generator G tries to generate synthetic B-ultrasound images that the discriminator D judges as real, while the discriminator G tries not to be deceived by the generator G; through iterative training, until the generator G can generate high-quality synthetic B-ultrasound images, and the discriminator D cannot distinguish between the generated synthetic B-ultrasound images and the real historical B-ultrasound images.

[0084] As an example, the training process of the generative adversarial network can be summarized as the following two optimization problems:

[0085] The optimization goal of the generator is to minimize the following loss function:

[0086]

[0087] where z is a random noise drawn from a prior noise distribution p z (z), G(z) is the image generated by the generator, D(.) is the probability that the discriminator outputs a real image, E appears in the expression of the loss function, representing the error or loss of the generator.

[0088] The optimization objective of the discriminator is to minimize the following loss function:

[0089]

[0090] where x is the real historical B-mode ultrasound image, pdata(x) is the distribution of the real historical B-mode ultrasound image, z is random noise drawn from the prior noise distribution p z (z), G(z) is the image generated by the generator, D(·) is the probability of the discriminator outputting a real image, E appears in the expression of the loss function, representing the error or loss of the discriminator.

[0091] In one specific example, the following steps can be included: a certain number of real historical B-mode ultrasound images are included as training data; a generative adversarial network GAN of the generator G and the discriminator D is designed, the generator G usually contains multiple deconvolution layers, and the discriminator D usually contains multiple convolution layers; the generative adversarial network GAN is trained until convergence, and the generator G can generate realistic synthetic B-mode ultrasound images; the generated synthetic B-mode ultrasound images and the real historical B-mode ultrasound images are used together as the data set for training the deep learning model, so as to improve the robustness and generalization ability of the deep learning model.

[0092] Through the enhanced data synthesis technology in the above example, the generative adversarial network helps to synthesize new training data, increases the diversity and quantity of the data set, and thus helps to improve the performance of the deep learning model.

[0093] In another example, the deep learning model can also be optimized based on transfer learning. Through the transfer learning technology, an existing image recognition model (such as ResNet or Inception) can be used to improve the training effect and accuracy of the deep learning model.

[0094] Transfer learning is a technique that leverages pre-trained models on large datasets to improve performance on specific tasks; one specific example of transfer learning in B-mode ultrasound image recognition tasks can include the following: selecting a pre-trained image recognition model, such as ResNet or Inception, which are typically trained on large-scale image datasets; removing the top layers (usually fully connected layers) of the pre-trained image recognition model, which are designed for the original task (such as ImageNet classification); adding a new fully connected layer with an output node number corresponding to the number of class recognition for the current task (for example, if the task is to recognize different lesion types in B-mode ultrasound images, the output node number is equal to the number of lesion types); connecting the new fully connected layer with the rest of the pre-trained image recognition model to form a new model; fine-tuning the new model using a small amount of labeled historical B-mode ultrasound images, i.e., continuing training on the new dataset, usually using a smaller learning rate to avoid destroying the feature extraction ability of the pre-trained model.

[0095] Specifically, the fine-tuning process of transfer learning can be described as an optimization problem with the following steps:

[0096] First, define a loss function L to measure the difference between the model's prediction and the true label, the common loss function is cross-entropy loss:

[0097]

[0098] where n is the number of samples, yi is the true label of the i-th sample, is the predicted probability of the i-th sample.

[0099] The optimization goal in the fine-tuning process is to minimize the loss on the new dataset:

[0100]

[0101] where x is the input historical B-mode ultrasound image, θ is the model parameter, f(x; θ) represents the prediction of the deep learning model for the input x, and y is the true label.

[0102] One specific example of the fine-tuning process can include the following: collect a certain number of labeled historical B-mode ultrasound images as the fine-tuning dataset; remove the top layer from the pre-trained Resnet or Inception model and add a new fully connected layer; use cross-entropy loss as the loss function and choose a suitable optimizer (such as SGD or Adam); fine-tune the model using a small learning rate until the model's performance on the B-mode ultrasound image dataset no longer improves; evaluate the accuracy and efficiency of the fine-tuned model on the B-mode ultrasound image recognition task.

[0103] Through transfer learning, the rich features learned by the pre-trained model can be utilized, thereby improving the training effect and accuracy of the model on a specific task.

[0104] In step S20, please refer to the S20 step in Figure 1 , to obtain the B-ultrasound image, and pre-process the B-ultrasound image.

[0105] As an example, the B-ultrasound image in this step can be obtained in the same way as the historical B-ultrasound image in step S10; however, it should be noted that the B-ultrasound image in this step does not have any annotation information, but is only the original B-ultrasound image obtained by the data acquisition module, for example, the B-ultrasound image obtained by the data acquisition module from the B-ultrasound device. Similarly, the B-ultrasound image in this step can be pre-processed in the same way as the historical B-ultrasound image in step S10; that is, the B-ultrasound image in this step is pre-processed by denoising, enhancing contrast, and standardizing, etc.

[0106] In step S30, please refer to the S30 step in Figure 1 , to perform feature extraction on the pre-processed B-ultrasound image based on the deep learning model to obtain image features.

[0107] As an example, the B-ultrasound image pre-processed in step S20 can be input into the deep learning model for feature extraction to obtain the image features.

[0108] In step S40, please refer to the S40 step in Figure 1 , to perform feature processing on the image features to obtain processed image features; the feature processing includes weighting processing on the image features based on an attention mechanism and feature fusion processing on the image features and the clinical information of the patient based on a dynamic feature fusion mechanism, and the processed image features include weighted image features with attention weights and fusion image features.

[0109] As an example, the attention mechanism uses a weight distribution mechanism to enable the subsequent component analysis model to automatically find and focus on the stone area when processing the image, ignoring irrelevant background information, thereby improving the accuracy of classification.

[0110] As an example, one specific example of weighting the image features based on the attention mechanism can include the following steps: defining an attention module that learns the importance weights of each pixel or feature on the B-ultrasound image; inputting the B-ultrasound image into the attention module to generate an attention weight map, which can represent the importance of each part of the B-ultrasound image to kidney stone identification; using the attention weight map to weight the image features to obtain weighted image features, so that the component analysis model can focus on the stone area.

[0111] As an example, the attention weight can be obtained based on the following formula:

[0112] A = softmax(fatt(X))

[0113] where X is the image feature, fatt(·) is the calculation function of the attention mechanism, and the softmax function is used to normalize the weight.

[0114] As an example, the weighted image feature can be obtained based on the following formula:

[0115] F weighted = A·X

[0116] where Fweightedis the weighted image feature, A is the attention weight, and X is the image feature.

[0117] As an example, dynamic feature fusion is an advanced feature processing technology that allows the model (e.g., the component analysis model) to dynamically learn and combine information from different sources during training. One specific example can include the following: extracting image features from B-ultrasound images while collecting patient clinical information such as age, gender, and medical history; converting the image features and clinical information into numerical representations that the component analysis model can process; designing a feature fusion module that can dynamically adjust the weights between image features and clinical information during the training process of the component analysis model to learn the best fusion method.

[0118] As an example, the clinical information can be obtained by a data acquisition module.

[0119] As an example, the fused image feature can be obtained based on the following formula:

[0120] F fused = a·V + b·C

[0121] where F fused is the fused image feature, V is the image feature, C is the patient's clinical information, a is the dynamic weight of the image feature, and b is the dynamic weight of the clinical information.

[0122] As an example, the learning of dynamic weights can be implemented in the following way:

[0123] a, b = MLP(V, C)

[0124] where MLP is a multi-layer perceptron that learns the weights a and b from the image features V and the clinical information C.

[0125] The dynamic fusion method described above allows the model to learn the strategy of feature fusion according to the importance of the data itself, rather than relying on pre-set rules or parameters, thereby improving the accuracy of the model in analyzing and identifying the composition of kidney stones.

[0126] In step S50, referring to the S50 step of Figure 1 , a composition analysis model is constructed, which includes a multi-level classifier.

[0127] As an example, the constructed composition analysis model can include a primary classifier and a fine classifier; the primary classifier quickly distinguishes whether there is a kidney stone in the B-ultrasound image, and for the B-ultrasound image containing a kidney stone, the fine classifier is used to further identify the specific composition of the kidney stone. Specifically, the role of the primary classifier is to quickly screen the B-ultrasound image, avoiding unnecessary subsequent processing of images without kidney stones. The fine classifier is responsible for accurately classifying the composition of kidney stones, such as distinguishing between calcium stones and urate stones, etc.

[0128] As an example, the purpose of the primary classifier is to quickly identify and exclude B-ultrasound images that do not contain kidney stones, in order to reduce the computational load and time of subsequent processing. The primary classifier can be a simple binary classification model, and its output result can be "contains kidney stones" or "does not contain kidney stones". The formula corresponding to the primary classifier can be:

[0129]

[0130] where x is the input image feature, fprimary(x) is the output score of the primary classifier, and τ is the score threshold.

[0131] As an example, the fine classifier further classifies the B-ultrasound image identified by the primary classifier as "containing kidney stones", for example, to distinguish between calcium stones and urate stones, etc. The fine classifier is usually a multi-classification model, and its output is the specific type of kidney stone. The formula corresponding to the fine classifier is described as:

[0132] C detail (x) = argkmaxf detail (x)k

[0133] where C detail (x) is the output class of the fine classifier, and fdetail (x) k is the output score of the k-th class by the fine classifier.

[0134] In one specific example, the following can be included:

[0135] Training the primary classifier: The primary classifier can be trained using historical B-ultrasound images labeled as "containing kidney stones" and "not containing kidney stones", which should be simple and fast enough for preliminary screening; here the input data for training the primary classifier can be the historical image features obtained after feature extraction of historical B-ultrasound images in the previous step, or can be the historical B-ultrasound images directly.

[0136] Training the fine classifier: The fine classifier is trained using historical B-ultrasound images labeled as different types of kidney stones, and the input data for the fine classifier can be the historical image features obtained after feature extraction of historical B-ultrasound images with kidney stones in the previous step.

[0137] Applying the primary classifier: The primary classifier performs primary classification on the input B-ultrasound images, and only passes B-ultrasound images classified as "containing kidney stones" to the fine classifier; at this time, the input data for the primary classifier can be the image features obtained in the previous step.

[0138] Applying the fine classifier: The B-ultrasound images screened by the primary classifier are classified accurately, and the specific component type of the kidney stones is output.

[0139] Another specific example can include the following: collecting data: obtaining a data set containing historical B-ultrasound images and corresponding patient clinical information; preprocessing and feature extraction: preprocessing the historical B-ultrasound images, and performing feature extraction on the preprocessed historical B-ultrasound images to obtain historical image features; normalizing the clinical information; feature fusion: using a trainable feature fusion module to combine the historical image features and the clinical information; training the model: training the multi-level classifier based on the data obtained previously; evaluating the performance: evaluating the performance of the multi-level classifier on the test set, and the dynamic feature fusion can improve the accuracy of classification.

[0140] In step S60, please refer to Figure 1 S60 step, input the processed image features into the component analysis model for analysis to obtain the kidney stone components of the B-ultrasound image.

[0141] Assuming there is a pre-processed B-ultrasound image, the primary classifier determines whether there is a kidney stone in the B-ultrasound image; then, the deep learning model extracts deep features of the B-ultrasound image to obtain image features; the attention mechanism then focuses on the stone area and dynamically adjusts the importance of the features; finally, combined with the clinical data of the patient, the feature fusion module generates the final image feature representation (fused image features); the fine classifier identifies the specific composition of the kidney stone in the B-ultrasound image based on the final image features. The above process not only improves the accuracy of identification, but also increases the interpretability of the model.

[0142] As an example, in step S60, after the processed image features are input into the component analysis model for analysis, the size and distribution of the kidney stone in the B-ultrasound image are also obtained. After obtaining the composition of the kidney stone in the B-ultrasound image, the size and distribution of the kidney stone, after step S60, the following steps can also be included:

[0143] S70: Based on the analysis result, a visual analysis report is generated.

[0144] As an example, based on the analysis result, a visual analysis report is generated, i.e., step S70 can include the following steps: S701-S703.

[0145] S701: Map the analysis result back to the B-ultrasound image and accurately locate the position and range of the kidney stone.

[0146] S702: Mark the stone area of the stone of different components in the B-ultrasound image with different colors and shapes.

[0147] S703: Based on the labeled B-ultrasound image, the composition of the kidney stone in the B-ultrasound image, the size and distribution of the kidney stone are obtained, and the visual analysis report is generated.

[0148] As an example, in step S701, involving feature mapping and positioning, the following content can be included: first, map the analysis result back to the original B-ultrasound image and accurately locate the position and range of the kidney stone; this step can be realized by calculating the maximum response value of the kidney stone features in the B-ultrasound image.

[0149] Specifically, the following formula can be used to determine the position of the kidney stone in the B-ultrasound image :

[0150]

[0151] wherein, represents the feature score of the kidney stone at position .

[0152] As an example, in step S702, the visualization processing can include the following: the positioning information obtained in step S701 is visualized, and different stone regions of different component stones are marked by different colors and shapes, so that the position and component of the kidney stone can be identified at a glance.

[0153] Specifically, the visualization can be achieved by assigning a color value C(i,j) to each pixel I(i,j) in the stone region R, and the specific formula is as follows:

[0154]

[0155] wherein C stone represents the color of the stone region, and C background represents the color of the background.

[0156] As an example, in step S703, the generation of the visual analysis report can include the following: the B-ultrasound image after the visualization processing and the related analysis data (for example, the component of the kidney stone obtained from the B-ultrasound image) are integrated into a detailed visual analysis report, which not only contains the size, shape and component information of the kidney stone, but also provides a description of the relationship between the kidney stone and the surrounding tissue.

[0157] On the basis that the component of the kidney stone in the B-ultrasound image has been identified and the position of the kidney stone has been located in step S60, in the visualization processing stage, the stone region can be marked as red and the background region can remain gray. In the generated visual analysis report, in addition to the marked B-ultrasound image, a table can also be attached, which lists in detail the size, component and position information of the kidney stone.

[0158] The kidney stone component analysis method based on the B-ultrasound image of the present application can effectively identify and classify the component of the kidney stone through a deep learning model and an image processing method, has the characteristics of non-invasiveness, rapidness and accuracy, and can be widely applied in clinical diagnosis.

[0159] It should be understood that, although Figure 1 the steps in the flowchart are shown in sequence according to the arrows, these steps are not necessarily executed in sequence according to the arrows. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, Figure 1 at least part of the steps in the flowchart can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these sub-steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or sub-steps or stages of other steps.

[0160] In another embodiment, referring to Figure 2 The application also provides a kidney stone composition analysis system based on B-ultrasound images, which can include a data acquisition module 10, a preprocessing module 20, a deep learning model construction module 30, a feature processing module 40, and a composition analysis model construction module 50. The data acquisition module 10 is configured to acquire at least B-ultrasound images. The preprocessing module 20 is configured to preprocess the B-ultrasound images. The deep learning model construction module 30 is configured to construct a deep learning model. The deep learning model 30 includes an adaptive convolutional neural network, and is configured to extract features from the preprocessed B-ultrasound images to obtain image features. The feature processing module 40 is configured to process the image features to obtain processed image features. The processing includes weighting the image features based on an attention mechanism and fusing the image features with clinical information of a patient based on a dynamic feature fusion mechanism. The processed image features include weighted image features with attention weights and fused image features. The composition analysis model construction module 50 is configured to construct a composition analysis model. The composition analysis model 50 includes a multi-level classifier, and is configured to analyze the processed image features to obtain kidney stone compositions of the B-ultrasound images.

[0161] In the kidney stone composition analysis system based on B-ultrasound images, the data acquisition module 10, the preprocessing module 20, the deep learning model construction module 30, the feature processing module 40, and the composition analysis model construction module 50 are provided, so that the kidney stone compositions in the B-ultrasound images can be accurately and efficiently analyzed, and a non-invasive, fast, and economical diagnostic tool is provided for doctors.

[0162] As an example, the kidney stone composition analysis system based on B-ultrasound images can be used to perform the kidney stone composition analysis method based on B-ultrasound images in the embodiments of the application and related embodiments. Figure 1

[0163] In yet another embodiment, the application also provides a computer device, which can be a server, and an internal structure diagram thereof can be as shown in Figure 3 ​As shown in the figure. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The database of the computer device is used to store historical work data, real-time work data and the like. The network interface of the computer device is used to communicate with external terminals through network connection. The computer program is executed by the processor to implement the method for analyzing the composition of kidney stones based on B-ultrasound images as described in the embodiments of the present application. Figure 1 The method for analyzing the composition of kidney stones based on B-ultrasound images and the embodiments thereof.

[0164] In yet another embodiment, the present application also provides a computer device, which can be a terminal, and its internal structure diagram can be as shown in the figure. Figure 3 The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The network interface of the computer device is used to communicate with external terminals through network connection. The computer program is executed by the processor to implement the method for analyzing the composition of kidney stones based on B-ultrasound images as described in the embodiments of the present application. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc. Figure 1 The method for analyzing the composition of kidney stones based on B-ultrasound images and the embodiments thereof.

[0165] Those skilled in the art can understand that, Figure 3 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0166] In yet another embodiment, the present application also provides a computer device including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method for analyzing the composition of kidney stones based on B-ultrasound images as described in the embodiments of the present application. Figure 1 The method for analyzing the composition of kidney stones based on B-ultrasound images and the embodiments thereof.

[0167] Any combination of the technical features in the above embodiments can be made, and for the sake of brevity, not all possible combinations are described, however, any combination of the technical features is deemed to be within the scope of the present disclosure as long as there is no inconsistency.

[0168] The above embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the patent scope of the application. It should be pointed out that for ordinary skilled persons in the art, some modifications and improvements can be made without departing from the concept of the present application, and these are within the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A method for analyzing a kidney stone composition based on a B-mode image, characterized by, Comprise: Building a deep learning model, the deep learning model comprises an adaptive convolutional neural network, comprising: obtaining a data set comprising historical B-ultrasound images, the data set comprising historical B-ultrasound images covering various kidney stone sizes, shapes and compositions; each of the historical B-ultrasound images is labeled with the chemical composition analysis result of kidney stones; constructing an initial convolutional neural network comprising convolutional layers and pooling layers; training the initial convolutional neural network using the data set; evaluating whether the performance of the trained initial convolutional neural network triggers an adjustment mechanism; if so, adjusting the number of at least one of the convolutional layers and the pooling layers in the initial convolutional neural network based on predetermined adjustment rules; and training the adjusted initial convolutional neural network using the data set; repeat the previous step until the performance of the trained initial convolutional neural network does not trigger the adjustment mechanism, then obtain the deep learning model; Obtaining a B-ultrasound image and preprocessing the B-ultrasound image; the preprocessing includes: denoising, enhancing contrast and standardizing the B-ultrasound image; Based on the deep learning model, the features of the preprocessed B-ultrasound image are extracted to obtain image features; characteristics of the image features are processed to obtain processed image features; the feature processing includes: performing weighted processing on the image features based on an attention mechanism and performing feature fusion processing on the image features and clinical information of a patient based on a dynamic feature fusion mechanism, the processed image features including weighted image features with attention weights and fusion image features; the fusion image features are obtained based on the following formula: F fused =α·V+β·C fused wherein, F is the fusion image features, V is the image features, C is the clinical information of the patient, α is a dynamic weight of the image features, and β is a dynamic weight of the clinical information; Building a composition analysis model, the composition analysis model comprises a multi-level classifier, the multi-level classifier comprises a primary classifier and a fine classifier; The processed image features are input into the composition analysis model for analysis to obtain the composition of kidney stones in the B-ultrasound image.

2. The method of claim 1, wherein, The data set comprising historical B-ultrasound images includes the steps of data collection and preprocessing.

3. The method of claim 1, wherein, The data set also includes synthetic B-ultrasound images; obtaining a data set comprising historical B-ultrasound images and synthetic B-ultrasound images, comprising: Obtaining historical B-ultrasound images; Building a generative adversarial network, the generative adversarial network comprises a generator and a discriminator; Based on the historical B-ultrasound images, the generative adversarial network is iteratively trained until convergence, and the generator generates synthetic B-ultrasound images; The historical B-ultrasound images and the synthetic B-ultrasound images are collected to obtain the data set.

4. The method of claim 1, wherein, After obtaining the deep learning model, the deep learning model is also optimized based on transfer learning.

5. The method of claim 1, wherein, The attention weight is obtained based on the following formula A = softmax(fatt(X)) Where X is the image feature, fatt(·) is the calculation function of the attention mechanism, and the softmax function is used to normalize the weight; The weighted image feature is obtained based on the following formula: F weighted =A·X where F weighted is a weighted image feature, A is an attention weight, and X is an image feature.

6. The method according to any one of claims 1 to 5, characterized in that, After inputting the processed image features into the composition analysis model for analysis, the size and distribution of kidney stones in the B-ultrasound image are also obtained, and after obtaining the composition of kidney stones, the size and distribution of kidney stones in the B-ultrasound image, the following steps are further included: Based on the analysis result, a visual analysis report is generated.

7. The method of claim 6, wherein, Based on the analysis result, a visual analysis report is generated, comprising: Map the analysis result back to the B-ultrasound image and accurately locate the position and range of kidney stones; Different colors and shapes are used to mark the stone area of stones of different compositions in the B-ultrasound image; Based on the labeled B-ultrasound image, the kidney stone composition, the size and the distribution of the kidney stone of the B-ultrasound image are obtained, and the visual analysis report is generated.

8. A B-mode image-based kidney stone composition analysis system, characterized by, The application discloses a kidney stone composition analysis method based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound image. The application discloses a kidney stone composition analysis system based on a B-ultrasound

Images