A deep learning texture analysis system for B-ultrasound images

Abstract

The present invention provides a B-ultrasound image deep learning texture analysis system, which relates to the field of medical image analysis technology. It combines the ResNet-50 model and multi-dimensional texture extraction, and realizes the effective fusion of low-level and high-level texture features by introducing Gabor filters and cross-layer information transmission; by capturing and analyzing the multi-dimensional texture of the B-ultrasound image, a comprehensive feature map is generated, and the system uses the Dropout layer to enhance the generalization ability of the ResNet-50 model, and further improves the accuracy of the analysis through feature fusion and weight learning, and the final output and decision support module can generate more accurate analysis results. The system is applicable to different types of B-ultrasound images, and effectively improves the accuracy and robustness of medical diagnosis.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image analysis, and in particular to a B-ultrasound image deep learning texture analysis system. Background Art

[0002] In the field of medical image analysis, B-ultrasound images are widely used in clinical diagnosis due to their advantages such as non-invasiveness, real-time and low cost. However, traditional B-ultrasound image analysis methods usually rely on manual experience and subjective judgment, which easily leads to inconsistent results and low accuracy.

[0003] Existing deep learning models have been proven to perform well in image classification, detection, and segmentation tasks; however, B-ultrasound images have unique texture features, including blurred edges, high noise, and low resolution, making it difficult for traditional image processing technology to effectively capture these features. When processing the complex textures of B-ultrasound images, it is impossible to fully extract low-level and high-level texture features, resulting in limited recognition accuracy of the model, which in turn causes large errors in model analysis. Summary of the invention

[0004] 1. Technical issues to be resolved In view of the shortcomings of the prior art, the present invention provides a B-ultrasound image deep learning texture analysis system, which solves the problem of how to analyze B-ultrasound images by capturing texture features after improving the quality of B-ultrasound images.

[0005] (II) Technical solution To achieve the above objectives, the present invention is implemented through the following technical solutions: a B-ultrasound image deep learning texture analysis system, comprising: B-ultrasound image acquisition module, used for preprocessing input B-ultrasound images and providing high-definition input B-ultrasound images for feature extraction and analysis; Feature extraction module: The ResNet-50 model introduces multi-dimensional texture extraction. By adsorbing textures of different dimensions on the input B-ultrasound image, the texture features of the B-ultrasound image are obtained, where the texture features include low-level texture features and high-level texture features. After the feature extraction of the ResNet-50 model, the system connects the classification head of the newly introduced Dropout layer, and the classification head is used to output the results. The cross-layer information transmission mechanism is connected between the low-level texture features and the high-level texture features of the feature extraction module to fuse the two features, enhance the system's ability to capture multi-dimensional texture features, and generate a multi-dimensional comprehensive feature map; The attention mechanism module works together with the feature extraction module to further enhance the system's attention to high-level texture features based on texture feature extraction; The transfer learning module uses the pre-trained ResNet-50 model to improve the recognition ability of B-ultrasound images through transfer learning, and changes them according to the texture features of B-ultrasound images to enhance the generalization and analysis capabilities of the system; The output and decision support module outputs the analysis results of the B-ultrasound image by fusing the outputs of the feature extraction and attention mechanism modules and applying the classification head of the Dropout layer.

[0006] Preferably, the image acquisition module converts the original B-ultrasound image into a format suitable for deep learning model analysis through preprocessing operations; the random noise introduced in the acquisition process is eliminated by applying Gaussian filtering, median filtering or deep learning algorithms, and the noise comes from sensors, transmission media or environmental interference; in order to simplify subsequent processing steps and reduce data dimensions, the B-ultrasound image is grayed out, and the values ​​of the three RGB color channels are converted into a single grayscale value by a weighted summation method, and the weight selection is based on the sensitivity of the human eye to different colors; the contrast enhancement step is enhanced by histogram equalization, contrast-limited adaptive histogram equalization, and contrast-limited adaptive histogram equalization. The local or global contrast of B-ultrasound images; Image segmentation is a key step to distinguish different areas in B-ultrasound images, such as tissue and organ boundaries, to provide accurate regional information for subsequent feature extraction and analysis; In the preprocessing process, image standardization ensures that all B-ultrasound images have consistent size and proportion, including image scaling, cropping or padding to meet the specification requirements of model input; In order to increase the diversity of the data set and improve the generalization ability of the model, the data enhancement step generates image variants through operations such as rotation, flipping, scaling, and cropping; Improve the quality of B-ultrasound images and enhance the ability of deep learning to recognize and analyze image texture features.

[0007] Preferably, the feature extraction module embeds the Gabor filter into the convolution layer of the ResNet-50 model to improve the ResNet-50 model's ability to capture local texture patterns. The Gabor filter captures the spatial frequency and direction information of the B-ultrasound image. The Gabor kernel function is defined as the product of a Gaussian function and a sine wave, and the expression is: λ, ,Ψ,σ)=exp( )cos(2 )in, and is the coordinate transformation in the rotating coordinate system, λ is the wavelength, θ is the direction, ψ is the phase shift, γ is the spatial aspect ratio, and σ is the Gaussian standard deviation; by adjusting the parameters of the Gabor kernel function, different texture features in the B-ultrasound image are captured; in the feature extraction process, the input B-ultrasound image is convolved with the Gabor filter, and the convolution operation is λ, ,Ψ,σ), where I(i,j) represents the pixel value of the input B-ultrasound image at the coordinate (i,j), and F(x,y) is the pixel value of the feature map at (x,y) after convolution. After the convolution operation, the generated feature map can reflect the texture characteristics of the B-ultrasound image. The Dropout layer is added after the convolution layer. The Dropout layer randomly closes a part of neurons, forcing the ResNet-50 model not to rely too much on a single feature during training, thereby improving the generalization ability of the model. The Dropout layer is expressed as ,in, represents the original output, and p is the probability of retaining the neuron. After the Dropout layer, a classification head is added, which maps the generated feature map to the final analysis result. The classification head learns how to extract the most useful information from the feature map for accurate analysis and classification.

[0008] Preferably, a mathematical model is established to describe the propagation of sound waves in tissues within the physical constraints of B-ultrasound images, taking into account the speed, frequency, attenuation and interaction factors of sound waves with tissues; the model further takes into account the reflection and scattering phenomena of sound waves when encountering different tissue interfaces, which are described by the scattering theory and reflection coefficient of sound waves, wherein the reflection coefficient can quantify the degree of reflection of sound waves at the interface of two media; these physical properties are embedded in the weights and bias parameters of the convolution layer, and the convolution kernel is adjusted to simulate the reflection and scattering patterns of sound waves in different tissues; in the convolution operation, the input B-ultrasound image is convolved with these adjusted convolution kernels to generate a feature map that can reflect the propagation characteristics of sound waves; wherein the convolution kernel contains the physical characteristics of sound wave propagation, so that the generated feature map can more accurately reflect the texture features in the B-ultrasound image.

[0009] Preferably, low-level texture features are extracted from shallower layers of the model. Low-level texture features include edges, corners, and simple shape structures, and the features contain preliminary visual information of the image. High-level texture features are extracted from deeper layers of the model to represent more complex and abstract texture information, such as local texture patterns and high-level semantic content of the image. In the cross-layer information transmission mechanism, channel alignment and size matching are performed. Since features at different levels have different dimensions and resolutions, directly fusing these features will result in information loss or inconsistency. These features need to be processed by increasing the size of the feature map and adjusting the depth of the features. The processed low-level and high-level features are spliced ​​and merged in the spatial dimension to form a larger feature map. For example, convolutional layers or attention layers are used to weight the merged features. After splicing, a comprehensive feature map containing multiple dimensions is formed. The comprehensive feature map contains low-level texture features of the image and also fuses high-level texture features, so that the model can understand and analyze the image content more comprehensively.

[0010] Preferably, the model uses global average pooling to generate a global contextual representation, aggregates the texture features of the entire image into a global feature vector, which contains the statistical information of the entire image, and learns the weights of each texture feature through one or more fully connected layers. These fully connected layers can adjust the weights according to the importance of the features in the training data, so that the model can identify the features that are more important for B-ultrasound image analysis; in the weight learning process, the model uses back propagation and gradient descent algorithms to optimize the weight values. The model can automatically adjust the degree of attention to different features and strengthen those features that make important contributions to the analysis. These learned weights are used to weight the original texture feature map, so that the model can pay more attention to those recognizable areas when performing B-ultrasound image analysis; the model's sensitivity to key features is improved, and the model's ability to analyze the texture features of B-ultrasound images is enhanced, so that higher accuracy and efficiency can be achieved when performing tasks such as image classification, detection or segmentation.

[0011] Preferably, the comprehensive feature map is input into a Dropout layer, which increases the generalization ability of the model and prevents overfitting by randomly closing a part of neurons. After the Dropout layer, the feature map is sent to the classification head, which is composed of a series of fully connected layers and is responsible for mapping the extracted features to the final analysis results. The fully connected layer can learn the complex relationship between the features and convert these features into prediction results; a decision layer is introduced, which is a neural network structure. The decision layer comprehensively considers the texture features in the feature map and the relationship between the features; the decision layer optimizes the output of the classification head by learning the complex relationship between different features to make more accurate analysis decisions; the output and decision support module combines the features processed by the classification head and the decision layer to generate a comprehensive analysis result.

[0012] (III) Beneficial effects The present invention provides a B-ultrasound image deep learning texture analysis system, which has the following beneficial effects: 1. The present invention introduces the ResNet-50 model, multi-dimensional texture extraction, Gabor filter and cross-layer information transmission mechanism, so that the system can effectively integrate low-level and high-level texture features, thereby improving the recognition ability of complex textures of B-ultrasound images.

[0013] 2. The present invention focuses on key texture features through the attention mechanism module and combines the Dropout layer to prevent overfitting, thereby enhancing the robustness and analysis accuracy of the model, so that more accurate classification and analysis results can be obtained in complex scenarios.

[0014] 3. The present invention can perform well on limited training data, adapt to different types of B-ultrasound images and application scenarios, and significantly enhance the versatility of the system in actual clinical environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] Embodiment 1: The embodiment of the present invention provides a deep learning texture analysis system for B-ultrasound images. First, a B-ultrasound image acquisition module is used to perform a repair operation on an input original B-ultrasound image. The repair step of the B-ultrasound image is the basis for the operation of the entire system and is intended to provide high-quality input data for subsequent texture acquisition. The system performs noise removal, grayscale processing, restoration enhancement and image segmentation operations on the original image. In the noise removal step, the system adopts traditional methods of Gaussian excitation and median excitation, combined with a deep learning denoising algorithm, to remove random noise introduced during the acquisition process due to sensor, transmission medium or environmental interference. Through these operations, the gain of the image is improved, the grayscale processing simplifies the color information of the image, and the B-ultrasound image is converted into a single-channel image grayscale, thereby reducing the data dimension and retaining high-level texture features. Through histogram equalization or restoration-restricted adaptive histogram equalization, the system further enhances the loss of the image, making the texture features more obvious. The image segmentation step can distinguish different areas in the B-ultrasound image and provide accurate area information for subsequent functions.

[0018] After the B-ultrasound image processing is completed, the system enters the feature extraction stage. The core of this stage is the feature extraction module based on the ResNet-50 deep learning model. In order to further enhance ResNet-50's ability to capture complex textures of B-ultrasound images, the system captures texture features of different scales, directions and spatial frequencies in B-ultrasound images through Gabor filters. Its working principle is based on the multi-dimensional parameter adjustment of the Gabor kernel function, including parameters such as wavelength, direction, phase offset, etc., and generates a feature map reflecting the local texture features of the B-ultrasound image through shape operations. In this process, the Gabor filter is combined with the ResNet-50 model, so that the system can capture the multi-dimensional texture features in the B-ultrasound image.

[0019] After feature extraction is completed, the system uses the cross-layer information transmission mechanism to further optimize the fusion of low-level and high-level texture features; through channel matching and size matching processing, the low-level texture features and high-level texture features are effectively spliced ​​and fused to generate a. The introduction of this mechanism solves the inconsistency problem of feature maps at different levels in dimension and resolution, allowing the system to capture the global and local aspects of the B-ultrasound image at the same time.

[0020] The system's attention mechanism module uses a weighted strategy to focus on key areas in B-ultrasound images. During the processing, the system generates a global context representation through global average pooling and assigns weights to each texture feature through the overall connection layer. This weighted process ensures that the system can more intelligently identify high-level feature areas in B-ultrasound images and improve its ability to capture important features. Combined with backpropagation and gradient descent algorithms, the system can continuously optimize weight distribution and further improve the accuracy of image analysis.

[0021] In order to enhance the generalization ability of the system, the transfer learning module plays an important role in the model training process; by using the pre-trained ResNet-50 model, the system can adapt to the specific analysis tasks of B-ultrasound through posture on the existing feature-based images; transfer learning not only speeds up the training speed of the ResNet-50 model, but also improves the observation of the ResNet-50 model in different types of B-images.

[0022] After all the feature extraction and optimization steps are completed, the system finally outputs the analysis results generated by the decision support module; the module finally randomly closes some neurons through the Dropout layer to prevent the model from over-generalization, thereby improving the generalization ability; the feature map is input into the classification head, which is composed of a fully connected layer and is responsible for mapping the features to the final analysis results; the system also introduces a decision layer to further optimize the output results of the model by learning the complex relationships between features, thereby achieving more accurate analysis and decision support.

[0023] In summary, this system successfully solves the common problems of noise interference, low resolution and complex texture extraction in B-ultrasound image analysis, significantly improves the system's image analysis capabilities, and provides an efficient and accurate mechanism for medical image processing.

[0024] Embodiment 2: This embodiment introduces the practical application of a deep learning texture analysis system for B-ultrasound images in the diagnosis of metabolic associated fatty liver disease (MASLD) and metabolic associated fatty liver hepatitis (MASH); MASLD is one of the most common diseases in the world, and MASH, as a subtype, can further develop into cirrhosis and liver cancer; liver biopsy is the gold standard for the diagnosis of MASH, but due to the high cost and invasiveness of biopsy, it is difficult to carry out and promote. B-ultrasound images have been widely used as a non-invasive diagnostic method. However, its ability to detect micro-fatty degeneration is insufficient, and it cannot distinguish simple fatty degeneration from MASH. Therefore, the development of an automatic B-ultrasound image analysis system combined with deep learning technology is of great significance to improving the diagnostic accuracy of MASH.

[0025] The core technical solution of this system is based on the combination of B-ultrasound images and deep learning algorithms. First, the system established an ultrasound data image set provided by the hospital, covering 794 B-ultrasound images of 536 patients. These images were standardized and uniformly adjusted to 224x224 pixels to ensure the consistency of the input data. To enhance the generalization ability of the ResNet-50 model, the system used data enhancement technology, including rotation, mirroring, adding noise, enhancement and other transformation methods, to expand the original data to 7940 images, thereby providing richer samples for the training of the ResNet-50 model.

[0026] After the data is completed, the system uses the pre-trained ResNet-50 model to extract the texture features of the B-ultrasound image; through transfer learning technology, the system makes adjustments based on the pre-trained model to further adapt to the analysis task of the B-ultrasound image; in this process, all parameters of the ResNet-50 model are involved in the training, so that the feature extraction network can effectively capture the features related to MASLD and MASH in the B-ultrasound image; the system has specially designed a new classification head for the MASH diagnosis task, replacing the original fully connected layer of ResNet-50. The new classification head not only adapts to the specific head disease classification needs, but also introduces the Dropout layer to reduce over-fitting.

[0027] The cross entropy loss function is used during the training of the system. , which is the loss function used for the difference between the gradient model prediction and the actual label. is a shared feature extractor, It is a task-specific classification head. is the total number of tasks; the total loss function It is the accumulation of loss functions of different tasks, aiming to minimize the comprehensive performance error of the multi-task network for all tasks: by optimizing this loss function, the system can effectively learn multiple tasks at the same time, including distinguishing normal liver, fatty liver and MASH.

[0028] In the actual application of the system, the patient's liver imaging data is first obtained through a B-ultrasound device; after the B-ultrasound image is repaired and extracted, the system uses a deep learning model to automatically identify fat changes in liver tissue, and diagnoses the patient with MASH based on the MASH clinical research network's scoring system; the advantage of this system is that it can accurately capture advanced texture features in B-ultrasound images, which are often difficult to identify with the naked eye, especially for early fatty liver disease. Through automatic analysis of deep learning, the system can provide doctors with auxiliary diagnostic suggestions and improve the accuracy and consistency of diagnosis.

[0029] In clinical applications, this system not only reduces the workload of doctors, but also greatly improves diagnostic efficiency. Through this system, the non-invasive detection of metabolic-related fatty liver disease has been qualitatively improved, especially in the early warning of liver disease, and has a high application prospect.

[0030] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A B-ultrasound image deep learning texture analysis system, characterized in that: include: B-ultrasound image acquisition module, used for preprocessing input B-ultrasound images and providing high-definition input B-ultrasound images for feature extraction and analysis; Feature extraction module: The ResNet-50 model introduces multi-dimensional texture extraction. By adsorbing textures of different dimensions on the input B-ultrasound image, the texture features of the B-ultrasound image are obtained, where the texture features include low-level texture features and high-level texture features. The system connects the classification head of the newly introduced Dropout layer after the feature extraction of the ResNet-50 model, and the classification head is used to output the results; The cross-layer information transmission mechanism is connected between the low-level texture features and the high-level texture features of the feature extraction module to fuse the two features, enhance the system's ability to capture multi-dimensional texture features, and generate a multi-dimensional comprehensive feature map; The attention mechanism module works together with the feature extraction module to further enhance the system's attention to high-level texture features based on texture feature extraction; The transfer learning module uses the pre-trained ResNet-50 model to improve the recognition ability of B-ultrasound images through transfer learning, and changes them according to the texture features of B-ultrasound images to enhance the generalization and analysis capabilities of the system; The output and decision support module outputs the analysis results of the B-ultrasound image by fusing the outputs of the feature extraction and attention mechanism modules and applying the classification head of the Dropout layer.

2. A B-ultrasound image deep learning texture analysis system according to claim 1, characterized in that: The B-ultrasound image acquisition module converts the input B-ultrasound image into a format for deep learning analysis through preprocessing operations, including denoising the B-ultrasound image to eliminate random noise introduced during the acquisition process. The B-ultrasound image is grayed to reduce the data dimension. The texture features of the B-ultrasound image are made more obvious by contrast enhancement, and the visual effect of the B-ultrasound image is improved to enhance the clarity of the B-ultrasound image. During the preprocessing of the B-ultrasound image, the B-ultrasound image acquisition module segments the B-ultrasound image to distinguish different areas in the B-ultrasound image.

3. The B-ultrasound image deep learning texture analysis system according to claim 1, characterized in that: The feature extraction module embeds the Gabor filter into the convolution layer of the ResNet-50 model to enhance the ResNet-50 model's ability to capture local texture patterns. First, a Gabor kernel function is defined, which is composed of the product of a Gaussian function and a sine wave and has directional selectivity and spatial frequency selectivity. The Gabor kernel function captures different texture features in B-ultrasound images by adjusting the parameters of wavelength, direction, phase offset, spatial aspect ratio, and Gaussian standard deviation. By combining the Gabor filter with the convolution layer, it can capture the multi-dimensional texture features in the B-ultrasound image. In the convolution operation, the input B-ultrasound image is convolved with the Gabor filter to generate a feature map that can reflect the texture features of the B-ultrasound image; a Dropout layer is added after the convolution layer to improve the generalization ability of the model and reduce overfitting. The Dropout layer randomly closes a part of neurons to force the model to improve feature extraction; after the Dropout layer, a classification head is added to map the generated feature map to the final analysis result.

4. The B-ultrasound image deep learning texture analysis system according to claim 3, characterized in that: The convolution layer adds physical constraints to the B-ultrasound image and embeds features into the convolution process through the physical model of sound wave propagation, so as to better capture the special feature textures in the B-ultrasound image caused by sound wave reflection and scattering.

5. The B-ultrasound image deep learning texture analysis system according to claim 1, characterized in that: The cross-layer information transmission mechanism extracts low-level texture features and high-level texture features, performs channel alignment and size matching processing, and splices the low-level texture features and high-level texture features to form a comprehensive feature map containing multiple dimensions.

6. The B-ultrasound image deep learning texture analysis system according to claim 1, characterized in that: The attention mechanism module adopts a weighted strategy to assign a weight to each texture feature by learning the importance of different areas in the B-ultrasound image, wherein the model optimizes the weights through back propagation and gradient descent algorithms, so that the model can automatically focus on the recognizable areas in the B-ultrasound image; a self-attention structure is adopted to generate a global context representation by using global average pooling, and then the weight of each texture feature is learned through one or more fully connected layers, and the weight of the high-level texture features in the B-ultrasound image is strengthened.

7. The B-ultrasound image deep learning texture analysis system according to claim 1, characterized in that: The transfer learning module uses a pre-trained ResNet-50 model and adapts it to a specific B-ultrasound image recognition task through fine-tuning; collects and prepares a data set for a specific B-ultrasound image analysis task, replaces or adjusts the output layer of the ResNet-50 model to adapt to the new task, and through fine-tuning, the ResNet-50 model learns to map the pre-trained features to the output layer; through the introduced attention mechanism, the ResNet-50 model can focus on the high-level texture features in the B-ultrasound image.

8. The B-ultrasound image deep learning texture analysis system according to claim 1, characterized in that: The output and decision support module takes the comprehensive feature map processed by feature extraction and attention mechanism as input, and inputs it into a classification head in the Dropout layer. The classification head consists of a fully connected layer and maps the extracted features to the final analysis results. A decision layer is introduced, which considers the global and local information in the feature map, as well as the relationship between features; the decision layer is a neural network that optimizes the final analysis decision by learning the complex relationship between different features; The output and decision support module combines these processed features with the output of the decision layer to generate a comprehensive analysis result.

Images