A clinical ultrasound image assisted screening system based on deep learning

Abstract

The application relates to the technical field of image processing, in particular to a clinical ultrasonic image auxiliary screening system based on deep learning. The system comprises an ultrasonic image preprocessing module, a lesion region segmentation module, an image lesion labeling module and a lesion auxiliary screening module, can acquire clinical ultrasonic images and carries out gray processing and neighborhood gray balance processing, so as to generate clinical ultrasonic balanced images; carries out pixel normalization and lesion region segmentation based on tissue boundary blur on the clinical ultrasonic balanced images, so as to generate clinical ultrasonic image lesion region blocks; carries out image lesion labeling through the clinical ultrasonic image lesion region blocks, so as to generate a clinical ultrasonic lesion labeling image set; constructs an image lesion auxiliary screening model and carries out lesion auxiliary screening, so as to output the lesion position, lesion type and lesion severity corresponding to the lesion existing on the clinical ultrasonic image. The application can realize accurate analysis of clinical ultrasonic images, so as to assist in efficiently completing lesion screening.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a clinical ultrasound image-assisted screening system based on deep learning. Background Technology

[0002] In recent years, deep learning technology has made significant progress in medical image analysis. Deep learning, especially convolutional neural networks (CNNs), has been widely used in image classification, object detection, and image segmentation, demonstrating extremely high performance and potential. For ultrasound images, deep learning can efficiently screen for lesions by automatically learning feature information from the images. Compared with traditional image analysis methods, deep learning-based ultrasound image analysis methods have advantages such as high automation, fast processing speed, and high accuracy, effectively reducing errors from manual intervention and assisting doctors in accurate image screening. However, existing deep learning-based ultrasound image analysis methods often employ conventional enhancement and noise reduction techniques in the image processing stage, making it difficult to fully extract the potential information in the images. Furthermore, they lack sufficient expression of the global features of complex lesions, resulting in poor accuracy and generalization ability of the models in actual clinical screening. Summary of the Invention

[0003] Therefore, it is necessary for the present invention to provide a clinical ultrasound image-assisted screening system based on deep learning to solve at least one of the above-mentioned technical problems.

[0004] To achieve the above objectives, a deep learning-based clinical ultrasound image-assisted screening system includes the following modules:

[0005] The ultrasound image preprocessing module is used to acquire clinical ultrasound images and perform grayscale processing on the clinical ultrasound images to generate clinical grayscale images; and to perform neighborhood grayscale equalization processing on the clinical grayscale images to generate clinical ultrasound equalization images.

[0006] The lesion region segmentation module is used to normalize the pixels of the clinical ultrasound equalization image to generate a clinical ultrasound normalized image. It obtains the corresponding clinical ultrasound tissue texture features and clinical ultrasound tissue edge features from the clinical ultrasound normalized image, and performs lesion region segmentation based on the blurred tissue boundary of the clinical ultrasound normalized image to generate a clinical ultrasound image lesion region block.

[0007] The image lesion annotation module is used to obtain the location, type and severity of the corresponding lesion area from the lesion area block of clinical ultrasound image, and to annotate the corresponding clinical ultrasound image based on the location, type and severity of the lesion area to generate a set of annotated clinical ultrasound images.

[0008] The lesion-assisted screening module is used to construct an image-assisted lesion screening model based on a set of clinical ultrasound lesion-annotated images and combined with convolutional neural network training; it acquires the clinical ultrasound images to be screened and inputs them into the image-assisted lesion screening model to perform lesion-assisted screening, so as to output the location, type and severity of the lesion corresponding to the lesion on the clinical ultrasound image.

[0009] Furthermore, the ultrasound image preprocessing module includes the following functions:

[0010] Acquiring clinical ultrasound images;

[0011] Pixel blur is calculated for each pixel in the clinical ultrasound image to obtain the pixel blur of the clinical ultrasound image.

[0012] Based on the pixel blur of clinical ultrasound images, each pixel in the clinical ultrasound image is blurred and denoised to obtain a blurred and denoised clinical image.

[0013] The blurred and denoised clinical images are converted to grayscale to generate grayscale clinical images.

[0014] Neighborhood grayscale equalization processing is performed on clinical grayscale images to generate clinical ultrasound equalized images.

[0015] Furthermore, the neighborhood grayscale equalization processing based on the clinical grayscale image includes:

[0016] The clinical grayscale image is divided into various clinical grayscale image blocks according to certain pixel division intervals of 3x3 or 9x9.

[0017] Gray-level histograms are calculated for each clinical gray-level image block to obtain the corresponding gray-level histogram for each gray-level image block.

[0018] By searching for pixels with large gray-level deviations from other pixels in the gray-level histograms corresponding to each gray-level image block, and performing histogram equalization on the pixel based on the gray-level values ​​of the neighboring pixels in the corresponding gray-level histogram, each gray-level equalized image block is generated.

[0019] Bilinear interpolation was used to reconstruct clinical images from each grayscale equalized image block to generate clinical ultrasound equalized images.

[0020] Furthermore, the lesion area segmentation module includes the following functions:

[0021] Pixel normalization is performed on equalized clinical ultrasound images to generate normalized clinical ultrasound images.

[0022] The gray-level co-occurrence matrix calculation method was used to analyze the tissue texture features of normalized clinical ultrasound images to obtain clinical ultrasound tissue texture features, including contrast, uniformity, correlation, energy and entropy.

[0023] The Sobel operator is used to separate the tissue boundaries in the normalized clinical ultrasound image to mark the boundary contours of the tissues in the clinical image. The region growing algorithm is then used to further refine the boundary contours of the tissues in the clinical image to generate the tissue boundaries of the clinical ultrasound image.

[0024] Tissue edge feature analysis was performed on the tissue boundaries of clinical ultrasound images to obtain the clinical ultrasound tissue edge features, including shape, size and area;

[0025] Based on the tissue texture features and edge features of clinical ultrasound, lesion regions are segmented from normalized clinical ultrasound images with blurred tissue boundaries to generate lesion region blocks in clinical ultrasound images.

[0026] Furthermore, the segmentation of lesion regions based on blurred tissue boundaries in normalized clinical ultrasound images using clinical ultrasound tissue texture features and clinical ultrasound tissue edge features includes:

[0027] Based on clinical ultrasound tissue texture features and clinical ultrasound tissue edge features, and combined with U-Net, a corresponding clinical imaging lesion segmentation network is constructed. The clinical ultrasound tissue texture features and clinical ultrasound tissue edge features are encapsulated into a 3x3 network layer, and the ReLU activation function is used for downward convolution. At the same time, a 2x2 pooling layer is used for downward pooling to the bottleneck layer. An attention module is added to the bottleneck layer to perform 2x2 upward convolution and 1x1 convolution output, thereby outputting the initial clinical tissue lesion region.

[0028] Normalized clinical ultrasound images are input into a clinical imaging lesion segmentation network to segment lesion regions, thereby generating initial lesion region segmentation blocks in clinical images.

[0029] The initial lesion region segmentation block of clinical imaging is subjected to lesion refinement correction based on blurred tissue boundaries to generate lesion region blocks of clinical ultrasound imaging.

[0030] Furthermore, the lesion refinement correction process based on blurred tissue boundaries for the initial lesion region segmentation block in clinical images includes:

[0031] Tissue edges are extracted from the initial lesion area segmentation blocks in clinical imaging to generate the boundary lines of lesion tissue in clinical imaging.

[0032] The gray values ​​of the corresponding edge regions within the boundary line of the lesion tissue in clinical imaging are blurred to obtain the gray blurred line region of the lesion tissue boundary.

[0033] Based on the gray-scale blurred line area of ​​the lesion tissue boundary, a segmentation region based on tissue boundary blur is extracted to generate blurred segmentation blocks of the lesion area in clinical imaging.

[0034] The morphological operations corresponding to erosion, dilation, or opening and closing operations are used to refine and correct the contour of the corresponding lesion region within the blurred segmentation block of the lesion region in clinical imaging, so as to generate the lesion region block of clinical ultrasound imaging.

[0035] Furthermore, the step of extracting the segmented region based on the blurred gray-scale line area of ​​the lesion tissue boundary includes:

[0036] By setting a 3x3 grayscale blur sliding window, and calculating the mean and standard deviation of the blur C for each pixel in the grayscale blur line area of ​​the lesion tissue boundary based on the grayscale blur sliding window, the mean and standard deviation of the blur C for each pixel in the blur line area of ​​the lesion tissue can be obtained.

[0037] The tissue blur coefficient is quantified by the mean and standard deviation of blur C corresponding to each pixel in the blurred line area of ​​the lesion tissue, and the tissue blur coefficient corresponding to each pixel in the blurred line area of ​​the lesion tissue is obtained.

[0038] Based on the tissue blur coefficient corresponding to each pixel in the blurred line area of ​​the lesion tissue, a blurred membership block expansion segmentation is performed between each pixel in the gray-scale blurred line area of ​​the lesion tissue boundary to generate a blurred segmentation block of the lesion area in clinical imaging.

[0039] Furthermore, the image lesion annotation module includes the following functions:

[0040] The location of the corresponding lesion area in the clinical ultrasound image is obtained by using the lesion area block in the clinical ultrasound image.

[0041] Clustering algorithms were used to classify lesion regions in clinical ultrasound images to obtain lesion types.

[0042] The severity of lesions in clinical ultrasound imaging areas is assessed to determine the extent of the lesion.

[0043] Based on the location, type, and severity of the lesion, corresponding clinical ultrasound images are labeled to generate a set of labeled clinical ultrasound images.

[0044] Furthermore, the assessment of the degree of lesion in the clinical ultrasound imaging lesion area includes:

[0045] Local binary analysis of lesion texture was performed on the lesion area blocks in clinical ultrasound images to obtain the local binary pattern of texture in the clinical lesion area;

[0046] Local texture complexity of clinical ultrasound imaging lesion regions is evaluated based on the local binary pattern of texture in clinical lesion regions, and the local texture complexity of clinical imaging lesion regions is obtained.

[0047] The elasticity of the lesion tissue corresponding to the clinically measured lesion area is obtained by obtaining the elasticity of the lesion tissue in the clinical ultrasound imaging lesion area, and the lesion hardness is estimated based on the elasticity of the lesion tissue in the clinical ultrasound imaging lesion area to obtain the tissue hardness of the lesion area in the clinical imaging.

[0048] The severity of a lesion is assessed by evaluating the complexity of the local texture and the tissue stiffness of the lesion area in clinical imaging.

[0049] Furthermore, the lesion-assisted screening module includes the following functions:

[0050] A deep learning model architecture based on multi-scale feature fusion was designed and constructed by combining convolutional neural networks. This architecture includes an input layer, a feature extraction layer, a feature fusion layer, and an output layer. The input layer receives labeled clinical ultrasound images. The feature extraction layer uses convolutional neural network branches of different scales (3x3, 5x5, and 7x7) to extract features from the input clinical ultrasound images, capturing image features at different scales. An attention mechanism is also introduced to enhance focus on lesion areas. The feature fusion layer fuses image features from different scales, using skip connections and feature concatenation to generate corresponding clinical lesion feature representation maps. The output layer outputs the corresponding clinical image lesion screening results.

[0051] The clinical ultrasound lesion annotated image set was divided into training, validation, and test sets according to a 7:2:1 ratio. The deep learning model architecture based on multi-scale feature fusion was trained using the training set. During training, cross-entropy loss was used as the optimization objective, and stochastic gradient descent algorithm and its variant Adam were used for parameter updates. Data augmentation with random flipping, rotation, and scaling was used to increase the diversity of training data. An early stopping mechanism was introduced to stop training when the loss on the validation set no longer decreased to prevent overfitting. The trained model was evaluated using the test set, and the performance of the model was verified by accuracy, recall, and F1 score, thus constructing an image lesion-assisted screening model.

[0052] Obtain clinical ultrasound images of the patient to be screened;

[0053] The clinical ultrasound images to be screened are input into the imaging lesion-assisted screening model to perform lesion-assisted screening, so as to output the location, type and severity of the lesion corresponding to the lesion on the clinical ultrasound image.

[0054] The beneficial effects of this invention are:

[0055] The clinical ultrasound image-assisted screening system based on deep learning proposed in this invention consists of an ultrasound image preprocessing module, a lesion region segmentation module, an image lesion annotation module, and a lesion-assisted screening module. Compared with the prior art, the beneficial effects of this application are that grayscale processing can convert ultrasound images into a single grayscale image, removing color interference and making the structural information of the image more prominent. The grayscale image can better reflect the texture and structural features of the image, especially the brightness differences between different areas can be presented more clearly. Further neighborhood grayscale equalization processing can effectively enhance the contrast of the image and improve its visibility. Neighborhood grayscale equalization adjusts the grayscale value of each pixel in the image, making the overall brightness distribution of the image more uniform, thereby making details more obvious, improving the image quality, reducing the impact of image quality problems on the analysis results, and improving the distinguishability of lesion areas. Secondly, pixel normalization unifies the range of pixel values ​​in an image, allowing pixel values ​​from different regions to be compared at the same scale. This reduces the impact of differences in image scale and brightness, improving the accuracy of feature recognition. Furthermore, by extracting tissue texture and edge features from the normalized image, it helps analyze subtle structural changes and further identify potential lesion areas. These features include texture roughness, uniformity, and edge sharpness, providing a detailed description of the global characteristics of tissue lesions in the image and aiding in distinguishing between normal and abnormal areas. Based on this, a segmentation method based on blurred tissue boundaries can more accurately segment lesion areas, especially in cases of blurred boundaries or complex tissue structures. This method provides more sensitive segmentation results, effectively defining the boundaries of lesion areas and generating lesion blocks, providing accurate data support for subsequent analysis. Then, through further analysis of the lesion area, the specific location, type, and severity of the lesion area can be obtained. This information is crucial for image analysis because it can provide quantitative labeled data for clinical research and image processing. The location of the lesion area can help analysts quickly locate the lesion area and further classify the lesion characteristics by combining the type and severity. By labeling the image set, high-quality labeled data can be provided for subsequent model training. The labeled image set provides accurate labels for model training. High-quality labeled image sets can promote the training and optimization of the algorithm, thereby improving the universality and accuracy of the model on different image samples, and helping to provide more accurate auxiliary analysis in practical applications.Finally, by constructing a Convolutional Neural Network (CNN) model, deep learning methods can be used to automate image analysis. The CNN model learns features from annotated image sets, gradually improving its predictive ability on unknown images. Trained on large-scale image datasets, the model can extract high-dimensional features from complex image data and automatically identify lesion areas. This process significantly improves the efficiency and accuracy of image analysis, avoids biases from manual annotation, and provides more objective results. Furthermore, through the analysis of images to be screened, the model can output the location, type, and severity of lesions in real time, thereby improving the model's accuracy and generalization ability in actual clinical screening. The application of this system enables rapid screening of potential lesion areas from large amounts of image data, reducing the risk of missed or misdiagnosed diagnoses and providing strong support for further examination or analysis. Attached Figure Description

[0056] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0057] Figure 1 This is a schematic diagram of the modules of the deep learning-based clinical ultrasound image-assisted screening system of the present invention;

[0058] Figure 2 for Figure 1 A functional flowchart of the ultrasound image preprocessing module;

[0059] Figure 3 for Figure 1 A functional flowchart of the lesion area segmentation module. Detailed Implementation

[0060] The technical system of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0061] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor systems and / or microcontroller systems.

[0062] It should be understood that although the terms "first," "second," etc., may be used herein to describe various units, these units should not be limited by these terms. These terms are used merely to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, a first unit may be referred to as a second unit, and similarly, a second unit may be referred to as a first unit. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0063] To achieve the above objectives, please refer to Figures 1 to 3 This invention provides a deep learning-based clinical ultrasound image-assisted screening system, which includes the following modules:

[0064] The ultrasound image preprocessing module is used to acquire clinical ultrasound images and perform grayscale processing on the clinical ultrasound images to generate clinical grayscale images; and to perform neighborhood grayscale equalization processing on the clinical grayscale images to generate clinical ultrasound equalization images.

[0065] The lesion region segmentation module is used to normalize the pixels of the clinical ultrasound equalization image to generate a clinical ultrasound normalized image. It obtains the corresponding clinical ultrasound tissue texture features and clinical ultrasound tissue edge features from the clinical ultrasound normalized image, and performs lesion region segmentation based on the blurred tissue boundary of the clinical ultrasound normalized image to generate a clinical ultrasound image lesion region block.

[0066] The image lesion annotation module is used to obtain the location, type and severity of the corresponding lesion area from the lesion area block of clinical ultrasound image, and to annotate the corresponding clinical ultrasound image based on the location, type and severity of the lesion area to generate a set of annotated clinical ultrasound images.

[0067] The lesion-assisted screening module is used to construct an image-assisted lesion screening model based on a set of clinical ultrasound lesion-annotated images and combined with convolutional neural network training; it acquires the clinical ultrasound images to be screened and inputs them into the image-assisted lesion screening model to perform lesion-assisted screening, so as to output the location, type and severity of the lesion corresponding to the lesion on the clinical ultrasound image.

[0068] In this embodiment of the invention, clinical ultrasound images are acquired from the storage system of a hospital ultrasound examination device. These images are typically stored in Medical Digital Imaging and Communication (DICOM) format. Taking a color breast ultrasound image with a resolution of 512×512 pixels as an example, a weighted average method is used for grayscale processing, with the formula Gray=0.299R+0.587G+0.114B, where R, G, and B are the pixel values ​​of the red, green, and blue channels in the color image, respectively. Each pixel in the image is traversed, and its RGB value is calculated according to the formula to obtain the corresponding grayscale value, thereby generating a clinical grayscale image. Neighborhood grayscale equalization processing is then performed on the clinical grayscale image using the Local Adaptive Histogram Equalization (CLAHE) method. The image is divided into multiple 8×8 sub-blocks. For each sub-block, its grayscale histogram is calculated, and the frequency of each grayscale level is counted. The cumulative distribution function (CDF) is used to map and adjust the pixel grayscale values ​​within the sub-block, with the formula... ,in The original grayscale value. and These are the minimum and maximum values ​​of the cumulative gray-level distribution function of the sub-blocks, respectively. The total number of gray levels (usually 256) is used. To avoid noise amplification, the contrast limit threshold is set to 40. Pixel gray values ​​that exceed the threshold are cropped and renormalized to finally generate a clinical ultrasound equalized image.

[0069] S2: Lesion region segmentation module, used to normalize the pixels of the clinical ultrasound equalization image to generate a clinical ultrasound normalized image; obtain the corresponding clinical ultrasound tissue texture features and clinical ultrasound tissue edge features through the clinical ultrasound normalized image, and perform lesion region segmentation based on the clinical ultrasound tissue texture features and clinical ultrasound tissue edge features to generate lesion region blocks of clinical ultrasound image.

[0070] In this embodiment of the invention, pixel normalization is performed on the clinical ultrasound equalization image using a minimum-maximum normalization method, as shown in the formula: ,in These are the normalized pixel values. These are the original pixel values. It is the minimum pixel value in the image. It is the maximum pixel value in the image. Suppose that in a certain clinical ultrasound equalization image... =20, =200, for pixel values The point with a value of 50, after normalization The value is approximately 0.17. Normalization is performed by traversing all pixels to generate a normalized clinical ultrasound image. The gray-level co-occurrence matrix (GLCM) is used to calculate the tissue texture features of clinical ultrasound. The gray level is set to 16, and the calculation directions are 0°, 45°, 90°, and 135°, with a distance of 1 pixel. Taking the 0° direction as an example, a 16×16 GLCM matrix is ​​constructed. The number of pixel pairs that meet the conditions is counted, and the contrast is calculated using a formula. Uniformity The features are averaged in four directions, and the edge features of clinical ultrasound tissue are obtained using the Canny edge detection algorithm. First, a Gaussian filter with a standard deviation of 1.5 is used for noise reduction. Then, the Sobel operator is used to calculate the gradient magnitude and direction. A low threshold of 30 and a high threshold of 90 are set for edge connection to obtain the edge contour. Based on these features, U-Net combined with fuzzy C-means clustering is used for lesion region segmentation. The features are used as input to U-Net. A 3×3 convolutional layer extracts features, a 2×2 pooling layer downsamples, and an attention module is added to the bottleneck layer. After further convolution and a 1×1 convolution, the output is processed using fuzzy C-means clustering with a cluster size of 3 (background, lesion edge, lesion core). The membership degree is calculated based on the distance between the pixel and the cluster center, using the formula: (m=2), pixels with a membership degree greater than 0.5 are divided into lesion regions, and finally, clinical ultrasound image lesion region blocks are generated.

[0071] S3: Image lesion annotation module, used to obtain the location, type and severity of the corresponding lesion area from the lesion area block of clinical ultrasound image, and to annotate the corresponding clinical ultrasound image based on the location, type and severity of the lesion area to generate a set of annotated clinical ultrasound images.

[0072] In this embodiment of the invention, the location of the lesion region is obtained by recording its coordinate range in the original clinical ultrasound image for the generated lesion region block. Assuming the coordinates of the upper left corner of the lesion region block are (100, 100) and the lower right corner are (300, 300), the lesion is located within this rectangular area in the image. The lesion region block is then classified using the DBSCAN clustering algorithm, and a neighborhood radius is set. =3 pixels, minimum sample size MinPts=8, calculate the Euclidean distance between pixels. Based on density, clusters are formed, and different clusters are mapped to preset lesion types (such as cysts, tumors, etc.). The severity of the lesion is assessed by calculating the texture complexity, shape factor, and elastic modulus of the lesion region. Texture complexity is calculated in conjunction with GLCM features, and the shape factor is formulated as F = 4πA / C. 2(A is the area of ​​the lesion region, C is the perimeter). The elastic modulus was obtained through ultrasound elastography, and the elastic modulus of the lesion tissue was E=15kPa. Furthermore, a lesion severity assessment formula was established: S=α×(0.3×contrast+0.2×entropy+0.1×energy+0.1×correlation)+β×(1 F)+γ×E, where α=0.3, β=0.3, γ=0.4, and thresholds are set to classify mild, moderate, and severe lesions. Based on the above information, the location of the lesion is marked with a rectangle on the clinical ultrasound image, and the lesion type and severity are marked next to the rectangle. The marked images and related information are stored, and finally a set of marked clinical ultrasound images of lesions is generated.

[0073] S4: Lesion Assistance Screening Module, used to construct an image lesion assistance screening model based on a clinical ultrasound lesion annotation image set and combined with convolutional neural network training; acquire the clinical ultrasound images to be screened, and input the clinical ultrasound images to be screened into the image lesion assistance screening model to perform lesion assistance screening, so as to output the lesion location, lesion type and lesion severity corresponding to the lesions on the clinical ultrasound images.

[0074] In this embodiment of the invention, the clinical ultrasound lesion annotation image set is divided into a training set, a validation set, and a test set in a 7:2:1 ratio. Taking 1000 images as an example, the training set has 700 images, the validation set has 200 images, and the test set has 100 images. A convolutional neural network architecture based on multi-scale feature fusion is designed. The input layer receives images with a size adjusted to 224×224 and converts them into a tensor of (224, 224, 1). The feature extraction layer contains three convolutional branches: 3×3, 5×5, and 7×7. The 3×3 branch has 6 layers with channels of 16, 32, 64, 128, 256, and 512 respectively; the 5×5 branch has 4 layers with channels of 64, 128, 256, and 512; and the 7×7 branch has 3 layers with channels of 128, 256, and 512. Each layer is followed by a ReLU activation function, and a channel attention mechanism is introduced at the end of each branch. The weight calculation formula is as follows: The key features are enhanced by multiplying with the feature map. The feature fusion layer uses bilinear interpolation to adjust the branch output sizes to be consistent before concatenation, and then connects them to the input layer in a skip connection. The output layer compresses the number of channels to the number of lesion categories (assuming 6 categories) through a 1×1 convolution. The classification probability is output through the Softmax function, and cross-entropy loss is used during training. To achieve the goal, update the parameters using the Adam optimizer (learning rate 0.001). =0.9, =0.999, The training employs random flipping, rotation (±15°), and scaling (0.8-1.2x) to augment the data, and introduces an early stopping mechanism: training stops if the loss on the validation set does not decrease after 10 consecutive epochs. The test set is used to calculate accuracy (Precision = TP / TP + FP), recall (Recall = TP / TP + FN), and F1 score (F1 = ...). A 2×Precision×Recall / Precision+Recall assessment model was used to construct an image-assisted lesion screening model. Clinical ultrasound images to be screened were obtained from a hospital database, preprocessed into normalized tensors, and then input into the model. The model outputs the lesion probability of each pixel. The lesion location was determined by threshold judgment (threshold 0.5) and connected region labeling. The lesion type was determined by statistical classification. The severity was judged according to preset relationships. Finally, the model outputs the lesion location, lesion type, and lesion severity corresponding to the lesions on the clinical ultrasound images.

[0075] Furthermore, the ultrasound image preprocessing module includes the following functions:

[0076] Acquiring clinical ultrasound images;

[0077] Pixel blur is calculated for each pixel in the clinical ultrasound image to obtain the pixel blur of the clinical ultrasound image.

[0078] Based on the pixel blur of clinical ultrasound images, each pixel in the clinical ultrasound image is blurred and denoised to obtain a blurred and denoised clinical image.

[0079] The blurred and denoised clinical images are converted to grayscale to generate grayscale clinical images.

[0080] Neighborhood grayscale equalization processing is performed on clinical grayscale images to generate clinical ultrasound equalized images.

[0081] As an embodiment of the present invention, reference is made to... Figure 2 As shown, Figure 1 A functional flowchart of the ultrasound image preprocessing module is shown in this embodiment. The ultrasound image preprocessing module includes the following functions:

[0082] S11: Acquire clinical ultrasound images;

[0083] In this embodiment of the invention, clinical ultrasound images are acquired from ultrasound examination equipment in a hospital. These images are typically stored in a digital format, such as DICOM (Digital Imaging and Communications in Medicine). It is assumed that the acquired clinical ultrasound image is a two-dimensional image with a size of 512×512 pixels and a grayscale value range of 0-255 for each pixel.

[0084] S12: Calculate the pixel blur of each pixel in the clinical ultrasound image to obtain the pixel blur of the clinical ultrasound image.

[0085] In this embodiment of the invention, pixel blurring is calculated for each pixel in a clinical ultrasound image using a gradient magnitude method. Its gradient magnitude can be calculated using the following formula: ,in, and These are the pixels in and The gradient in a direction is typically calculated using the Sobel operator. The Sobel operator... and The templates in the directions are as follows: and Furthermore, by performing a convolution operation between the Sobel operator and the neighborhood surrounding the pixel, the value of the pixel in the neighborhood is obtained. and The gradient value in the direction is used to calculate the gradient magnitude. The smaller the gradient magnitude, the higher the blur of the pixel. For example, for a certain pixel, if its gradient magnitude is calculated to be 10, it is taken as the blur of the pixel, and finally the pixel blur of the clinical ultrasound image is obtained.

[0086] S13: Based on the pixel blur of the clinical ultrasound image, perform blur denoising processing on each pixel in the clinical ultrasound image to obtain a blurred and denoised clinical image.

[0087] In this embodiment of the invention, each pixel in the clinical ultrasound image is blurred and denoised based on the pixel blur of the clinical ultrasound image. A Gaussian filtering method is used for denoising. Gaussian filtering is a linear smoothing filter that reduces noise by weighted averaging of the pixel and its neighborhood. The weight matrix of the Gaussian filter can be calculated using the following formula: ,in, It is the center position of the filter. It is the standard deviation of the Gaussian distribution, which determines the smoothness of the filter. For each pixel, the standard deviation of the Gaussian filter is adjusted according to its blurriness. The higher the blurriness, the larger the standard deviation and the stronger the filtering effect. For example, for pixels with high blurriness, the standard deviation is set to 3; for pixels with low blurriness, the standard deviation is set to 1. By performing Gaussian filtering on each pixel, the final clinical blurred and denoised image is obtained.

[0088] S14: Perform grayscale processing on the blurred and denoised clinical images to generate grayscale clinical images;

[0089] In this embodiment of the invention, the clinical blurred and denoised images are converted to grayscale. If the clinical ultrasound image itself is a grayscale image, the next step can be performed directly. If it is a color image, it needs to be converted to grayscale. A commonly used grayscale conversion method is the weighted average method, and its formula is: Gray=0.299R+0.587G+0.114B, where R, G, and B are the pixel values ​​of the red, green, and blue channels in the color image, respectively. By calculating the RGB value of each pixel according to the above formula, the corresponding grayscale value is obtained, thereby generating a clinical grayscale image.

[0090] S15: Perform neighborhood grayscale equalization processing on the clinical grayscale image to generate a clinical ultrasound equalized image.

[0091] In this embodiment of the invention, neighborhood gray-level equalization is performed on clinical grayscale images using a histogram equalization method. Histogram equalization is a method to enhance image contrast by adjusting the grayscale histogram of an image. Specifically, the grayscale histogram of the clinical grayscale image is calculated, the frequency of each grayscale value is counted, the cumulative distribution function (CDF) is calculated based on the grayscale histogram (i.e., the cumulative frequency of each grayscale value and its sub-grayscale values), and the grayscale value of each pixel is mapped according to the CDF to obtain a new grayscale value. The mapping formula is: newGray=round((L-1) CDF(gray) is a function where L is the range of gray values ​​(usually 0-255), gray is the original gray value of a pixel, and newGray is the mapped gray value. The mapped gray values ​​are applied to each pixel of the clinical grayscale image to ultimately generate a clinical ultrasound equalization image.

[0092] Furthermore, the neighborhood grayscale equalization processing based on the clinical grayscale image includes:

[0093] The clinical grayscale image is divided into various clinical grayscale image blocks according to certain pixel division intervals of 3x3 or 9x9.

[0094] In this embodiment of the invention, when processing clinical grayscale images, a fixed pixel division interval (including 3x3 or 9x9) is used to divide the image into various clinical grayscale image blocks. Taking a clinical grayscale liver ultrasound image with a resolution of 512×512 as an example, a 9×9 pixel division interval is selected. Starting from the upper left corner of the image, the image is divided into rectangular areas of 9×9 pixels. Horizontally, it can be divided into 512÷9=56.89 blocks, rounded up to 57 blocks; vertically, it can be divided into 512÷9=56.89 blocks, rounded up to 57 blocks, resulting in a total of 57×57=3249 clinical grayscale image blocks. During the division process, areas with edges less than 9×9 pixels are padded with zeros to ensure that each image block is 9×9 pixels in size, providing a uniform data unit for subsequent targeted processing of each image block.

[0095] Preferably, a grayscale histogram is calculated for each clinical grayscale image block to obtain the grayscale histogram corresponding to each grayscale image block;

[0096] In this embodiment of the invention, grayscale histograms are calculated for each clinical grayscale image block obtained from the division. Taking a 9×9 clinical grayscale image block as an example, this image block contains 81 pixels, and the grayscale value of each pixel ranges from 0 to 255. An array of length 256 is established to count the frequency of each grayscale value. All 81 pixels in the image block are traversed. For each pixel, its grayscale value is used as the index of the array, and the corresponding array element value is incremented by 1. For example, if there are 3 pixels with a grayscale value of 50 in this image block, then the element value of the element with index 50 in the array will be 3 after the traversal. After the statistics are completed, each element value in the array is divided by the total number of pixels, 81, to obtain the frequency of each grayscale value, thereby obtaining the grayscale histogram corresponding to the grayscale image block. By performing this calculation on all 3249 clinical grayscale image blocks, the grayscale histogram of each image block is finally obtained. These histograms reflect the distribution of grayscale values ​​within each image block.

[0097] Preferably, by searching for pixels with large gray-level deviations from other pixels in the gray-level histograms corresponding to each gray-level image block, and performing histogram equalization on the pixel based on the gray-level values ​​of the neighboring pixels in the corresponding gray-level histogram, each gray-level equalized image block is generated.

[0098] In this embodiment of the invention, pixels with large gray-level deviations from other pixels are searched within the gray-level histograms corresponding to each gray-level image block. Taking the gray-level histogram of a certain gray-level image block as an example, the gray-level deviation threshold is set to 1.5 times the gray-level mean of the image block. First, the gray-level mean of the image block is calculated using the following formula: ,in The grayscale value is Given the number of pixels, assuming the calculated average grayscale value of the image block is 80, the grayscale deviation threshold is 80 × 1.5 = 120. In the grayscale histogram, pixels with a grayscale value greater than 120 and a low frequency of occurrence (set to be less than 5% of the total number of pixels, i.e., 81 × 5% = 4.05, taken as less than or equal to 4) are identified as pixels with large grayscale deviations. For each pixel with a large grayscale deviation, histogram equalization is performed based on the grayscale values ​​of its neighboring pixels in the corresponding grayscale histogram. By employing a local adaptive histogram equalization method, a 3 × 3 neighborhood is selected centered on the pixel, and the average grayscale value and standard deviation of the pixels in the neighborhood are calculated. Let the set of pixels in the neighborhood be N, then the average grayscale value of the neighborhood is... Standard deviation According to the formula new_gray= (in and To adjust the parameters, Take 3. (Take 50) Adjust the gray value of the pixel to obtain a new gray value, and finally generate each gray-level equalized image block to make the gray-level distribution within the image block more even.

[0099] Preferably, bilinear interpolation is used to reconstruct clinical images from each grayscale equalization image block to generate clinical ultrasound equalization images.

[0100] In this embodiment of the invention, bilinear interpolation is used to reconstruct clinical images from each gray-level equalized image block. Taking two adjacent 9×9 gray-level equalized image blocks as an example, during reconstruction, bilinear interpolation is used to calculate pixel values ​​in the transition region between the two image blocks. Assuming that a point within the transition region needs to be calculated... The pixel value of the point is located at the boundary of two adjacent image blocks. Find the pixel value of the point in each of the two image blocks. Four adjacent pixels , , , First of all Perform linear interpolation in the direction to calculate and : , Then Direction and Perform linear interpolation to obtain the point Pixel values: Following this method, bilinear interpolation was performed on the transition regions between all 3249 grayscale equalization image blocks to reassemble the image blocks together, ultimately generating a clinical ultrasound equalization image. This effectively improves the visual effect and detail of the image, providing higher-quality data for subsequent deep learning-based clinical ultrasound image-assisted screening.

[0101] Furthermore, the lesion area segmentation module includes the following functions:

[0102] Pixel normalization is performed on equalized clinical ultrasound images to generate normalized clinical ultrasound images.

[0103] The gray-level co-occurrence matrix calculation method was used to analyze the tissue texture features of normalized clinical ultrasound images to obtain clinical ultrasound tissue texture features, including contrast, uniformity, correlation, energy and entropy.

[0104] The Sobel operator is used to separate the tissue boundaries in the normalized clinical ultrasound image to mark the boundary contours of the tissues in the clinical image. The region growing algorithm is then used to further refine the boundary contours of the tissues in the clinical image to generate the tissue boundaries of the clinical ultrasound image.

[0105] Tissue edge feature analysis was performed on the tissue boundaries of clinical ultrasound images to obtain the clinical ultrasound tissue edge features, including shape, size and area;

[0106] Based on the tissue texture features and edge features of clinical ultrasound, lesion regions are segmented from normalized clinical ultrasound images with blurred tissue boundaries to generate lesion region blocks in clinical ultrasound images.

[0107] As an embodiment of the present invention, reference is made to... Figure 3 As shown, Figure 1 A functional flowchart of the lesion region segmentation module is shown in this embodiment. The lesion region segmentation module includes the following functions:

[0108] S21: Perform pixel normalization on the clinical ultrasound equalization image to generate a clinical ultrasound normalization image.

[0109] In this embodiment of the invention, pixel normalization processing is performed on the clinical ultrasound homogenized image. Taking a 512×512 clinical ultrasound homogenized kidney image as an example, the pixel values ​​in this image range from 0 to 255. The minimum-maximum normalization method is used, and the formula is as follows: ,in These are the normalized pixel values. These are the original pixel values. It is the minimum pixel value in the image. It is the maximum pixel value in the image. Iterate through each pixel in the image; if the image contains... =10, =200, for the original value of a certain pixel. =50, then the normalized value is =50 10 / 200 10≈0.21. By performing this calculation on all 512×512=262144 pixels in the image, the pixel values ​​are uniformly mapped to the range of 0-1, and finally a normalized clinical ultrasound image is generated. This eliminates the problem of inconsistent pixel value scales between different images due to factors such as equipment differences, which facilitates subsequent processing.

[0110] S22: The gray-level co-occurrence matrix calculation method is used to perform tissue texture feature analysis on normalized clinical ultrasound images to obtain clinical ultrasound tissue texture features, including contrast, uniformity, correlation, energy and entropy.

[0111] In this embodiment of the invention, tissue texture feature analysis is performed on normalized clinical ultrasound images using a gray-level co-occurrence matrix calculation method. Taking normalized clinical ultrasound liver images as an example, the gray level is set to 16 levels (the normalized pixel value range of 0-1 is divided into 16 equal parts), and four directions are selected: 0°, 45°, 90°, and 135°. Set to 1 pixel, construct a 16×16 grayscale co-occurrence matrix for each direction and distance. ,in and Indicates grayscale level. Indicates from grayscale level Starting from a given pixel, the grayscale level is reached at a specified direction and distance. The number of pixel pairs, for example, in the 0° direction, starting from a pixel of gray level 3, there are 10 pixel pairs that reach gray level 5 at a distance of 1 pixel. The value at 0° is 10. Calculate the contrast: Contrast reflects the degree of drastic change in grayscale in an image; the higher the value, the coarser the texture, while uniformity... Uniformity measures the evenness of grayscale distribution; the higher the value, the more uniform the texture. Correlation: ,in , These are the mean values ​​of GLCM in the row and column directions, respectively. , These are the standard deviations of GLCM in the row and column directions, respectively. Correlation reflects the linear correlation of gray-level distributions in the image, while energy: Energy represents the concentration of elements in the gray-level co-occurrence matrix; the larger the value, the smoother the image. Entropy: Entropy reflects the randomness of gray-level distribution in an image. The larger the value, the more complex the texture. By calculating these feature values ​​in four directions and taking the average, the clinical ultrasound tissue texture features are finally obtained, providing texture basis for lesion analysis.

[0112] S23: The Sobel operator is used to separate the boundary contours of tissues in the normalized clinical ultrasound image to mark the boundary contours of tissues in the clinical image. The region growing algorithm is then used to further refine the boundary contours of tissues in the clinical image to generate the tissue boundaries of the clinical ultrasound image.

[0113] In this embodiment of the invention, the Sobel operator is used to separate the tissue boundaries within normalized clinical ultrasound images. Taking normalized clinical ultrasound images of the thyroid gland as an example, the Sobel operator... and The templates in the directions are as follows: and Iterate through every pixel in the image, and for each pixel's 3×3 neighborhood, compare it with... and The template is convolved to obtain the value of the pixel. and gradient value in direction and Then through the formula Calculate the gradient magnitude by The gradient direction is calculated, and a threshold of 0.3 is set (an empirical value that can be adjusted according to the actual image). Pixels with a gradient amplitude greater than 0.3 are marked as boundary points to initially define the boundary contours of tissues within the clinical image. The initially defined boundary contours are further refined using a region growing algorithm. One boundary point is selected as a seed point, and the growth criterion is set as follows: pixels with a gray level difference of less than 0.1 (an empirical value) from the seed point and within an 8-neighborhood are selected as new growth points. Starting from the seed point, pixels that meet the growth criteria are continuously included in the region until no pixels meet the criteria. By performing this growth operation on all seed points, a more accurate clinical ultrasound image tissue boundary is finally generated, clearly outlining the tissue contour.

[0114] S24: Perform tissue edge feature analysis on the tissue boundary of clinical ultrasound images to obtain the tissue edge features of clinical ultrasound, including shape, size and area;

[0115] In this embodiment of the invention, tissue edge feature analysis is performed on the tissue boundary of clinical ultrasound images. Taking the boundary of a liver tumor region in a clinical ultrasound image as an example, the coordinate sequence of the boundary points is calculated, and a polygon approximation algorithm (such as the Ramer-Douglas-Peucker algorithm) is used to simplify the boundary contour, obtaining a rough morphological description of the boundary and determining whether it is circular, elliptical, or irregular in shape. The number of pixels in the area enclosed by the boundary is calculated. Assuming that the boundary of the tumor region contains 1000 pixels, and that each pixel represents an actual area of ​​0.01 mm², this method is used. 2 (Based on the resolution and physical dimensions of the ultrasound image), the area of ​​the tumor region is 1000 × 0.01 = 10 mm. 2 Meanwhile, the size is estimated by the number and shape characteristics of boundary points. In this way, the clinical ultrasound tissue edge characteristics, including morphology, size and area, are finally obtained, providing a quantitative basis for lesion diagnosis.

[0116] S25: Based on the clinical ultrasound tissue texture features and clinical ultrasound tissue edge features, the normalized clinical ultrasound image is segmented into lesion regions based on blurred tissue boundaries to generate lesion region blocks in clinical ultrasound images.

[0117] In this embodiment of the invention, lesion region segmentation based on blurred tissue boundaries is performed on normalized clinical ultrasound images using clinical ultrasound tissue texture features and clinical ultrasound tissue edge features. Taking normalized clinical ultrasound breast images as an example, a fuzzy membership function is established. For each pixel in the image, its fuzzy membership in the lesion region is calculated by combining the tissue texture features (such as contrast, entropy, etc.) and edge features (distance from the edge of a known lesion, etc.). Assuming that the influence weight of contrast on the lesion in the texture features is 0.4, the influence weight of entropy is 0.3, and the influence weight of distance from the edge of a known lesion in the edge features is 0.3, for a certain pixel, its contrast value is 0.6 (after normalization), its entropy value is 0.5, and its distance from the edge of a known lesion is 0.2 after normalization, then its membership degree U = 0.4 × 0.6 + 0.3 × 0.5 + 0.3 × (1 0.2) = 0.24 + 0.15 + 0.24 = 0.63. The membership threshold is set to 0.5. Pixels with a membership degree greater than 0.5 are classified as lesion areas. By performing this calculation and classification on all pixels, a lesion area block in clinical ultrasound imaging is finally generated, which can accurately segment the lesion area and assist doctors in clinical diagnosis.

[0118] Furthermore, the segmentation of lesion regions based on blurred tissue boundaries in normalized clinical ultrasound images using clinical ultrasound tissue texture features and clinical ultrasound tissue edge features includes:

[0119] Based on clinical ultrasound tissue texture features and clinical ultrasound tissue edge features, and combined with U-Net, a corresponding clinical imaging lesion segmentation network is constructed. The clinical ultrasound tissue texture features and clinical ultrasound tissue edge features are encapsulated into a 3x3 network layer, and the ReLU activation function is used for downward convolution. At the same time, a 2x2 pooling layer is used for downward pooling to the bottleneck layer. An attention module is added to the bottleneck layer to perform 2x2 upward convolution and 1x1 convolution output, thereby outputting the initial clinical tissue lesion region.

[0120] In this embodiment of the invention, a corresponding clinical imaging lesion segmentation network is constructed based on clinical ultrasound tissue texture features and clinical ultrasound tissue edge features, combined with U-Net. Taking a clinical ultrasound breast image as an example, the clinical ultrasound tissue texture features (such as contrast, uniformity, correlation, energy, and entropy) and clinical ultrasound tissue edge features (such as shape, size, and area) are encapsulated into a 3×3 network layer. In this process, each feature is treated as a channel. Assuming there are 5 texture features and 3 edge features, an 8-channel 3×3 network layer is formed, and the ReLU activation function is used for downward convolution. The formula for the ReLU activation function is... For each neuron in the network layer, the input value is processed by the ReLU function. If the input value is less than 0, the output is 0; if it is greater than 0, it remains unchanged. For example, if the input value of a neuron is -2, the output after ReLU is 0; if the input value is 3, the output is 3. Feature extraction is performed through multiple such 3×3 convolutional layers. The number of channels doubles after each convolution. For example, after the first convolution, the number of channels becomes 16. At the same time, 2×2 pooling layers are used to pool down to the bottleneck layer. The purpose of pooling is to reduce the amount of data and computation. In the 2×2 pooling layer, the maximum value in the 2×2 neighborhood is taken as the output value. For example, for a 2×2 neighborhood... After pooling, the output value is 7. As the number of pooling layers increases, the spatial dimension of the data gradually decreases, while the number of channels gradually increases, eventually reaching the bottleneck layer. At the bottleneck layer, an attention module is added to perform 2×2 up-convolution followed by a 1×1 convolution output. The role of the attention module is to make the network focus more on important feature information. Simultaneously, through the attention mechanism, the importance weight of each channel is calculated, and then the features are weighted and summed according to the weights. For example, for a 16-channel feature map, the attention module calculates the weight of each channel. Then, the feature maps are weighted and summed to obtain a new feature map. Next, a 2×2 upconvolution is performed. Upconvolution is the inverse operation of pooling. The spatial dimension of the data is restored through deconvolution, while the number of channels is halved. Finally, a 1×1 convolution is performed to output the initial clinical tissue lesion area.

[0121] Preferably, the normalized clinical ultrasound image is input into the clinical imaging lesion segmentation network to segment the lesion region, so as to generate the initial lesion region segmentation block of the clinical image;

[0122] In this embodiment of the invention, a normalized clinical ultrasound image is input into a clinical image lesion segmentation network for lesion region segmentation. Taking a normalized clinical ultrasound image of thyroid gland with a size of 512×512 as an example, the image first passes through the network's input layer, and then sequentially through multiple 3×3 convolutional layers, 2×2 pooling layers, bottleneck layers, 2×2 upper convolutional layers, and 1×1 convolutional layers constructed earlier. During this process, the image features are continuously extracted and fused. For example, in the first 3×3 convolutional layer, the image and the convolution kernel perform convolution operations to extract preliminary features. As the network deepens, the features gradually become more abstract and advanced. In the pooling layer, the spatial dimension of the image continuously decreases, and after adding an attention module in the bottleneck layer, the network can more accurately focus on the features of the lesion region. Finally, after processing by the network, the initial lesion region segmentation block of the clinical image is output. This segmentation block is a binary image with the same size as the input image, in which the lesion region is marked as 1 and the non-lesion region is marked as 0. For example, in the output segmentation block, the pixel value of the thyroid lesion region is 1, while the pixel value of the normal tissue region is 0.

[0123] Preferably, the initial lesion region segmentation block of the clinical image is subjected to lesion refinement correction processing based on the blurring of tissue boundaries to generate a lesion region block of the clinical ultrasound image.

[0124] In this embodiment of the invention, the initial lesion region segmentation block of clinical images is subjected to lesion refinement correction processing based on tissue boundary fuzziness. Taking a liver lesion region in the initial lesion region segmentation block of clinical images as an example, the fuzzy membership degree of each pixel in the segmentation block is first calculated. The fuzzy C-means clustering algorithm is used, assuming that the lesion region is divided into 3 categories (normal tissue, lesion tissue edge, and lesion core), and the initial value of the cluster center is set. The distance from each pixel to the cluster center is calculated iteratively. (in It is the first The first pixel 1 eigenvalue, It is the first The first cluster center (each eigenvalue), and update the membership matrix based on the distance. (in It is a fuzzy index, usually taken as 2). After multiple iterations, when the membership matrix no longer changes, the membership degree of each pixel to different categories is obtained. For lesion areas, pixels with a membership degree greater than 0.5 are retained as lesion areas, and pixels with a membership degree less than 0.5 are adjusted. For example, for a pixel, its membership degree to the edge of lesion tissue is 0.4 and its membership degree to normal tissue is 0.6, then it is adjusted to normal tissue. Through this processing, the initial lesion area segmentation block of clinical image is refined and corrected, and finally a lesion area block of clinical ultrasound image is generated, making the boundary of the lesion area more accurate and the division of the lesion area more refined, providing a more reliable basis for clinical diagnosis.

[0125] Furthermore, the lesion refinement correction process based on blurred tissue boundaries for the initial lesion region segmentation block in clinical images includes:

[0126] Tissue edges are extracted from the initial lesion area segmentation blocks in clinical imaging to generate the boundary lines of lesion tissue in clinical imaging.

[0127] In this embodiment of the invention, tissue edges are extracted from the initial lesion region segmentation block of a clinical image. Taking a 512×512 pixel lung clinical image's initial lesion region segmentation block as an example, the Canny edge detection algorithm is used. This algorithm first performs Gaussian filtering on the segmentation block for noise reduction. The standard deviation of the Gaussian filter is set to 1.5, the filter size is 5×5, and the Gaussian filtering formula is: ,in Center of the filter To determine the standard deviation, convolution operations are used to reduce noise interference in the segmented blocks. Next, the image gradient magnitude and direction are calculated. The Sobel operator is used to calculate the horizontal and vertical gradients. The gradient magnitude formula is: The gradient direction formula is: Then, non-maximum suppression is performed. Each pixel is traversed. If the gradient magnitude of a pixel is not a local maximum in its gradient direction, its magnitude is set to 0. Finally, two thresholds are set (low threshold is 30, high threshold is 90) for edge connection. Pixels with gradient magnitudes greater than the high threshold are identified as edge points, and pixels with gradient magnitudes less than the low threshold are excluded. Pixels between the two thresholds are retained as edge points if they are connected to the identified edge points, otherwise they are excluded. Through the above operations, the boundary line of the lesion tissue in the clinical image is finally generated, clearly outlining the general contour of the lesion area.

[0128] Preferably, the gray values ​​of the corresponding edge regions within the boundary line of the lesion tissue in clinical imaging are blurred to obtain the gray blurred line area of ​​the lesion tissue boundary.

[0129] In this embodiment of the invention, the grayscale values ​​of the corresponding edge regions within the boundary line of clinical imaging lesions are blurred. Taking an edge region with a width of 5 pixels within the boundary line as an example, a Gaussian blurring method is used for each pixel within the edge region. The blurred grayscale value is calculated using the following formula: ,in The original grayscale value. The grayscale value after blurring. Choose 2 (i.e., use a 5×5 Gaussian kernel). The Gaussian kernel function is calculated in the same way as the Gaussian filtering formula in the previous step. For example, if the original gray value of a pixel in the edge region is 150, after convolution with the Gaussian kernel, the blurred gray value is 130. This operation is performed on all pixels in the edge region to finally obtain the gray blurred line area of ​​the lesion boundary, making the transition between the lesion area and normal tissue more natural and simulating the blurring of the real tissue boundary.

[0130] Preferably, based on the gray-scale blurred line area of ​​the lesion tissue boundary, a segmentation region extraction based on tissue boundary blur is performed to generate blurred segmentation blocks of the lesion area in clinical imaging.

[0131] In this embodiment of the invention, a segmentation region based on the blurred gray-scale line area at the edge of the lesion tissue boundary is extracted. By setting a gray-scale threshold range of [100, 150] (determined according to the gray-scale distribution of the blurred line area), the clinical image is traversed. For pixels with gray-scale values ​​within this threshold range, it is determined whether they are located within the connected region of the blurred gray-scale line area at the edge of the lesion tissue boundary. A 4-connected region search algorithm is used to start from a pixel that meets the threshold condition and search for its four adjacent pixels above, below, left, and right. If the adjacent pixels also meet the threshold condition and are not marked, they are added to the current connected region and the search continues until all connected regions that meet the conditions have been searched. These connected regions are then extracted, and finally, a blurred segmentation block of the lesion area in the clinical image is generated. Compared with the initial segmentation block, this segmentation block fits the actual boundary of the lesion area better and reflects the blurred characteristics of the tissue boundary.

[0132] Preferably, the lesion morphology is refined and corrected by morphological operations corresponding to erosion, dilation, or opening and closing operations within the blurred segmentation block of the clinical imaging lesion area, so as to generate a clinical ultrasound imaging lesion area block.

[0133] In this embodiment of the invention, the contour shape of the corresponding lesion region within the blurred segmentation block of the clinical image lesion region is refined and corrected by morphological operations corresponding to erosion, dilation, or opening and closing operations. Taking a liver lesion region in the blurred segmentation block of the clinical image lesion region as an example, the erosion operation is first performed using a 3×3 structuring element. For each pixel in the segmentation block, if all pixels in its 3×3 neighborhood belong to the lesion region, the pixel is retained; otherwise, it is removed. The formula is as follows: ,in The lesion area is divided into blurred blocks. Using a 3×3 structuring element, the erosion operation removes small protrusions at the boundaries of the lesion area. This is followed by a dilation operation, also using a 3×3 structuring element. For each pixel in the segmented block, if any pixel within its 3×3 neighborhood belongs to the lesion area, then that pixel is marked as belonging to the lesion area. The formula is: The expansion operation can fill small depressions at the boundary of the lesion area, or perform an opening operation (erosion followed by expansion) to remove small noise blocks outside the lesion area; then perform a closing operation (expansion followed by erosion) to fill small cavities inside the lesion area. Through this series of morphological operations, the contour of the lesion area is refined and corrected, and finally, a lesion area block in clinical ultrasound imaging is generated, making the segmentation results of the lesion area more accurate and providing a more accurate basis for clinical diagnosis.

[0134] Furthermore, the step of extracting the segmented region based on the blurred gray-scale line area of ​​the lesion tissue boundary includes:

[0135] By setting a 3x3 grayscale blur sliding window, and calculating the mean and standard deviation of the blur C for each pixel in the grayscale blur line area of ​​the lesion tissue boundary based on the grayscale blur sliding window, the mean and standard deviation of the blur C for each pixel in the blur line area of ​​the lesion tissue can be obtained.

[0136] In this embodiment of the invention, a 3×3 grayscale blurring sliding window is set in the processing of the grayscale blurring line area at the boundary of the lesion tissue. Taking the grayscale blurring line area at the boundary of the lesion tissue in a clinical ultrasound liver image as an example, this area contains multiple pixels. For each pixel in the line area, a 3×3 sliding window is constructed with that pixel as the center. When calculating the fuzzy C-mean, it is assumed that the pixels are divided into three categories (normal tissue, lesion transition, and lesion core), denoted as . , , Initialize cluster centers , , For example, based on the grayscale distribution of the line area, =80、 =120、 =160 was used as the initial cluster center for each pixel within the sliding window. Calculate its distance from each cluster center. Calculate fuzzy membership degree based on distance ,in Let be the fuzzy index, and take . =2, and the cluster centers are updated iteratively. The calculation continues until the cluster centers stabilize, and the mean of the fuzzy C-value is used as the mean value for that pixel. When calculating the standard deviation of the fuzzy C-value, for each pixel's cluster, the grayscale value of the pixels within that cluster is used using the formula... ,in It is the mean gray value of the pixels within the cluster. Given the number of pixels within a cluster, obtain the fuzzy C standard deviation corresponding to that pixel. Traverse all pixels within the gray-scale blurred line area at the boundary of the lesion tissue to complete the calculation of the fuzzy C mean and fuzzy C standard deviation for each pixel.

[0137] Preferably, the tissue blur coefficient is quantized based on the mean and standard deviation of the blur C corresponding to each pixel in the blurred line area of ​​the lesion tissue, so as to obtain the tissue blur coefficient corresponding to each pixel in the blurred line area of ​​the lesion tissue.

[0138] In this embodiment of the invention, the tissue blur coefficient is quantified by using the mean and standard deviation of blur C corresponding to each pixel within the blurred line area of ​​the lesion tissue, and a formula is employed. ,in and For the weighting coefficients, take respectively =0.6、 =0.4, the neighborhood mean is the mean of the gray values ​​of the pixels within a 3×3 sliding window centered on the pixel, and the maximum standard deviation is the maximum value of the standard deviation of blur C among all pixels within the blurred line area of ​​the lesion tissue. For example, for a pixel within the blurred line area of ​​the lesion tissue, its blur C mean is 130, the neighborhood mean is 120, the blur C standard deviation is 15, and the maximum standard deviation within the line area is 20. Then, the tissue blur coefficient F corresponding to this pixel is F=0.6×|130-120| / 120+0.4×15 / 20=0.05+0.3=0.35. By performing this calculation on all pixels within the blurred line area of ​​the lesion tissue, the tissue blur coefficient corresponding to each pixel is finally obtained, which quantifies the degree of blur characteristics of the pixel within the blurred line area of ​​the lesion tissue.

[0139] Preferably, based on the tissue blur coefficient corresponding to each pixel in the gray-scale blur line area of ​​the lesion tissue boundary, a fuzzy membership block expansion segmentation is performed between each pixel in the gray-scale blur line area of ​​the lesion tissue boundary to generate a fuzzy segmentation block of the lesion area in the clinical image.

[0140] In this embodiment of the invention, based on the tissue blur coefficient corresponding to each pixel within the gray-scale blur line area of ​​the lesion tissue boundary, a fuzzy membership block expansion segmentation is performed between each pixel within the gray-scale blur line area of ​​the lesion tissue boundary. A fuzzy membership threshold of 0.2 is set. For any pixel P within the gray-scale blur line area of ​​the lesion tissue boundary, expansion is performed from P as the center to its 8-neighboring pixels. If the difference between the tissue blur coefficient of neighboring pixel Q and the tissue blur coefficient of P is less than the threshold, and Q is not assigned to other block areas, then Q and P are classified into the same fuzzy membership block area. For example, if the tissue blur coefficient of pixel P is 0.3, and the tissue blur coefficient of its neighboring pixel Q is 0.28, the difference between the two is 0.3. If 0.28 = 0.02 < 0.2, then Q is included in the fuzzy membership block region where P is located. A new starting point is randomly selected from the undivided pixels within the line area, and the above expansion process is repeated until all pixels within the line area are divided into the corresponding fuzzy membership block regions. These fuzzy membership block regions are extracted, and finally, a fuzzy segmentation block of the lesion region in clinical imaging is generated. This segmentation block reflects the fuzzy boundary between the lesion region and normal tissue in more detail, providing a more accurate result for the precise segmentation of lesion regions in clinical ultrasound imaging.

[0141] Furthermore, the image lesion annotation module includes the following functions:

[0142] The location of the corresponding lesion area in the clinical ultrasound image is obtained by using the lesion area block in the clinical ultrasound image.

[0143] In this embodiment of the invention, for a 512×512 pixel clinical ultrasound liver image and its corresponding lesion region, the location of the lesion region is obtained by recording the coordinate range of the lesion region in the clinical ultrasound image. Assuming the coordinates of the upper left pixel of the lesion region in the clinical ultrasound image are (x1, y1) and the coordinates of the lower right pixel are (x2, y2), where x1=100, y1=150, x2=300, y2=350, the specific location range of the lesion region in the image is determined in pixels, i.e., the area from the horizontal coordinate 100 to 300 and the vertical coordinate 150 to 350. This provides precise location information for further analysis and annotation of the lesion region. Simultaneously, this coordinate information is recorded in the corresponding metadata file for convenient subsequent retrieval and use.

[0144] Preferably, a clustering algorithm is used to classify the lesion regions in clinical ultrasound images to obtain the lesion region type;

[0145] In this embodiment of the invention, the DBSCAN (Density Peak Clustering) algorithm is used to classify lesion regions in clinical ultrasound breast images. First, two key parameters are set: neighborhood radius. =5 pixels (representing a circular neighborhood centered on a given pixel), minimum number of samples MinPts=10 (representing the radius of the neighborhood). (The minimum number of sample points contained within a cluster is required to form a cluster core). For each pixel in the lesion region block, calculate its Euclidean distance to other pixels using the formula: ,in and Each pixel and The coordinates of a pixel, if a pixel is within the radius A pixel is considered a core point if the number of sample points within its neighborhood is greater than or equal to MinPts. If a pixel is within the neighborhood of a core point, it is considered a boundary point. Other pixels are considered noise points. By continuously scanning the pixels in the lesion area, core points and their density-reachable points are grouped into clusters. For example, when processing a lesion area containing 2000 pixels, three different clusters are formed after calculation and judgment. Based on the pre-defined correspondence between the lesion area type and the cluster features, if the pixel grayscale features of cluster 1 match the characteristics of a cyst, it is classified as a cyst type; if cluster 2 matches the characteristics of a tumor, it is classified as a tumor type; the remaining noise points are not classified. Finally, the type of lesion area is obtained, providing a reference for clinical diagnosis.

[0146] Preferably, the severity of the lesion is assessed in the clinical ultrasound imaging lesion area to determine the degree of severity of the lesion area;

[0147] In this embodiment of the invention, the severity of lesions in clinical ultrasound imaging of thyroid lesions is assessed by comprehensively considering the texture, shape, and elasticity features of the lesion area. Texture features are calculated using a gray-level co-occurrence matrix (GLCM) to determine parameters such as contrast, entropy, energy, and correlation. For example, for a lesion area, the calculated contrast is 80, entropy is 3.2, energy is 0.1, and correlation is 0.7. Shape features are determined by calculating the perimeter C and area A of the lesion area, thus yielding a shape factor F = 4πA / C. 2 Assume the perimeter of the lesion area is C = 100 pixels and the area is A = 600 pixels. 2 Then the shape factor F = 4π × 600 / 100 2≈0.75, elastic characteristics were obtained by ultrasound elastography, with the elastic modulus of the lesion tissue being E=15kPa. A lesion severity assessment formula was established: S=α×(0.3×contrast+0.2×entropy+0.1×energy+0.1×correlation)+β×(1 F)+γ×E, where α=0.3, β=0.3, γ=0.4. Substituting the parameters calculated above into the formula, we get S=0.3×(0.3×80+0.2×3.2+0.1×0.1+0.1×0.7)+0.3×(1 0.75)+0.4×15=0.3×(24+0.64+0.01+0.07)+0.3×0.25+6=0.3×24.72+0.075+6=7.416+0.075+6=13.491. The severity level of the lesion is set as follows: when S<3, it is a mild lesion; when 3≤S<6, it is a moderate lesion; and when S≥6, it is a severe lesion. Therefore, the lesion area is determined to be a severe lesion.

[0148] Preferably, the corresponding clinical ultrasound images are labeled with lesions based on the location, type, and severity of the lesion area to generate a set of labeled clinical ultrasound images.

[0149] In this embodiment of the invention, based on the lesion location, lesion type, and lesion severity obtained in the preceding steps, lesion annotation is performed on clinical ultrasound renal images. This is achieved by using rectangular bounding boxes to mark the lesion location. A rectangle is drawn on the image with the upper left corner coordinates (x1, y1) and lower right corner coordinates (x2, y2) of the lesion block. A text label is added above the rectangle to clearly indicate the lesion type, such as "tumor," and the lesion severity is indicated below the rectangle, such as "severe." For a clinical ultrasound image containing multiple lesion blocks, each lesion block is annotated sequentially using the above method. The annotated image is saved as a new image file, and information such as the lesion location, type, and severity is recorded in a corresponding XML file. The file contains fields such as image filename, lesion coordinates, type, and severity. By performing this annotation operation on a series of clinical ultrasound images, a set of annotated clinical ultrasound images is finally generated, providing standardized and structured data for subsequent clinical diagnosis, medical research, and deep learning model training.

[0150] Furthermore, the assessment of the degree of lesion in the clinical ultrasound imaging lesion area includes:

[0151] Local binary analysis of lesion texture was performed on the lesion area blocks in clinical ultrasound images to obtain the local binary pattern of texture in the clinical lesion area;

[0152] In this embodiment of the invention, local binary analysis of lesion texture is performed on the lesion region of a clinical ultrasound image. Taking a lesion region of a 256×256 pixel clinical ultrasound breast image as an example, the Basic Local Binary Pattern (LBP) algorithm is used. For each pixel in the lesion region, a 3×3 neighborhood is taken centered on that pixel. The gray value of the center pixel is used as a threshold. For the eight pixels in the neighborhood, if their gray values ​​are greater than or equal to the gray value of the center pixel, then the pixel is assigned a value of 1; otherwise, it is assigned a value of 0. For example, the gray value of the center pixel... If the value is 120, and the grayscale value of the top-left pixel in its neighborhood is 130, then the pixel is assigned a value of 1; if the grayscale value of the top-right pixel in its neighborhood is 110, then the pixel is assigned a value of 0. These 8 binary values ​​are arranged in a clockwise direction to form an 8-bit binary number, which is then converted into a decimal number and used as the local binary pattern value of the central pixel. This process is repeated for all pixels in the lesion area block to obtain the local binary pattern value corresponding to each pixel, thereby generating the local binary pattern of the clinical lesion area texture. This pattern reflects the local grayscale variation characteristics of the lesion area texture.

[0153] Preferably, the local texture complexity of the lesion area in clinical ultrasound imaging is evaluated based on the local binary pattern of the texture in the clinical lesion area to obtain the local texture complexity of the lesion area in clinical imaging.

[0154] In this embodiment of the invention, the local texture complexity of clinical ultrasound image lesion regions is evaluated based on the local binary pattern of the clinical lesion region texture. The uniformity and contrast of the local binary pattern are used to measure texture complexity. First, the uniformity of the local binary pattern is calculated. For each local binary pattern value, it is converted into a cyclic binary number, and the number of transitions between 0 and 1 and between 1 and 0 in its binary bits is calculated. If the number of transitions is less than or equal to 2, the local binary pattern is a uniform pattern; otherwise, it is a non-uniform pattern. The number of pixels in the lesion region block that are uniform and non-uniform patterns is counted. Let the number of pixels in the uniform pattern be... The total number of pixels is Then the uniformity ratio Calculate the contrast of the local binary mode for each local binary mode value. Its contrast ,in The grayscale value of the neighboring pixels. Given the grayscale value of the center pixel, calculate the average contrast of all pixels in the lesion area. By combining the uniformity ratio and the average contrast, the local texture complexity of the lesion area in clinical imaging is obtained. ,in =0.6, =0.4, The maximum value among all pixel contrasts, for example, if =0.7, =20, =50, then =0.6×(1 0.7)+0.4×20 / 50=0.18+0.16=0.34, which quantifies the local texture complexity of the lesion area.

[0155] Preferably, the elasticity of the corresponding lesion tissue is obtained from the lesion area block of the clinical ultrasound image, and the lesion hardness is estimated based on the elasticity of the lesion tissue to obtain the tissue hardness of the lesion area in the clinical ultrasound image.

[0156] In this embodiment of the invention, the elasticity of the corresponding clinically measured lesion tissue is obtained through a clinical ultrasound imaging lesion area. Taking a thyroid lesion area as an example, ultrasound elastography is used. During the examination, a small pressure is applied to the lesion area, and image information of the lesion tissue before and after the pressure is acquired using an ultrasound probe. The elasticity of the lesion tissue is calculated based on the degree of tissue deformation. Assuming a strain elastography method is used, the strain formula is: ,in This refers to the change in length of the tissue after it is subjected to compression. To determine the original length of the tissue, image processing algorithms are used to analyze strain values ​​at different locations within the lesion area, yielding the distribution of elasticity in the lesion tissue. Based on this elasticity, the stiffness of the lesion area in clinical ultrasound imaging is estimated, establishing a mapping relationship between elasticity and stiffness. For example, through extensive clinical data statistics, an empirical formula H=a× +b, where H is the stiffness of the diseased tissue, a= 10, b=15 (determined based on actual statistical data), calculate the elasticity value corresponding to each pixel within the lesion area block to obtain the tissue hardness distribution of the lesion area in clinical imaging. For example, if the elasticity value of a pixel is 0.1, then its hardness H= 10×0.1+15=14, which quantifies the hardness of the tissue in the lesion area.

[0157] Preferably, the severity of the lesion is assessed based on the complexity of the local texture and the tissue hardness of the lesion area in clinical imaging.

[0158] In this embodiment of the invention, the severity of a lesion is assessed based on the local texture complexity and tissue hardness of the lesion area in clinical imaging, thus establishing a lesion severity assessment model. A weighted summation method is used, and the formula is as follows: ,in The severity of the lesion area, and As the weighting coefficient, it is set by... =0.4, =0.6, for example, if the local texture complexity TC of a lesion area in a clinical ultrasound image is 0.4 and the tissue hardness H is 12, then the severity of the lesion area S = 0.4 × 0.4 + 0.6 × 12 = 0.16 + 7.2 = 7.36. Set the lesion severity level classification standard: when S < 3, it is a mild lesion; when 3 ≤ S < 6, it is a moderate lesion; when S ≥ 6, it is a severe lesion. Based on the calculated severity value of the lesion area, the severity of the lesion area is judged to be a severe lesion.

[0159] Furthermore, the lesion-assisted screening module includes the following functions:

[0160] A deep learning model architecture based on multi-scale feature fusion was designed and constructed by combining convolutional neural networks. This architecture includes an input layer, a feature extraction layer, a feature fusion layer, and an output layer. The input layer receives labeled clinical ultrasound images. The feature extraction layer uses convolutional neural network branches of different scales (3x3, 5x5, and 7x7) to extract features from the input clinical ultrasound images, capturing image features at different scales. An attention mechanism is also introduced to enhance focus on lesion areas. The feature fusion layer fuses image features from different scales, using skip connections and feature concatenation to generate corresponding clinical lesion feature representation maps. The output layer outputs the corresponding clinical image lesion screening results.

[0161] In this embodiment of the invention, a deep learning model architecture based on multi-scale feature fusion is constructed by combining convolutional neural networks. Taking a 512×512 pixel labeled clinical ultrasound liver image as an example, the input layer receives the image data and converts it into a tensor form suitable for model processing, i.e., the dimension is (512, 512, 1) (assuming it is a grayscale image; if it is a color image, the number of channels is 3). The feature extraction layer uses three convolutional neural network branches of different scales. The 3×3 convolutional neural network branch contains 5 convolutional layers, with the number of channels in each convolutional layer being 32, 64, 128, 256, and 512 respectively, with a stride of 1 and a padding method of "same" to ensure that the output feature map size remains unchanged; the 5×5 convolutional neural network branch contains 4 convolutional layers, with the number of channels being 64, 128, 256, and 512 respectively. The 7×7 convolutional neural network branch contains three convolutional layers with 128, 256, and 512 channels respectively. Each convolutional layer is followed by a ReLU activation function, with the formula f(x) = max(0, x). Additionally, a spatial attention mechanism is introduced after the last convolutional layer of each branch, calculating the attention weight matrix M, with the formula... ,in For feature maps, and These are average pooling and max pooling operations, respectively. The Sigmoid function enhances attention to lesion regions by multiplying the attention weight matrix with the feature map. The feature fusion layer employs skip connections and feature concatenation, taking the output feature maps from the 3×3, 5×5, and 7×7 branches, adjusting their sizes (using bilinear interpolation) to ensure uniformity, and then concatenating them to obtain a feature map with dimensions (H, W, 512+512+512)=(H, W, 1536). This map is then added to the input layer via skip connections to finally generate a clinical lesion feature representation map. The output layer uses 1×1 convolutions to compress the number of channels to the number of categories (assuming 5 lesion types plus a normal category, totaling 6 categories), and then uses the Softmax function for classification. The Softmax formula is... Output the corresponding clinical imaging lesion screening results, i.e., the first The pixel belongs to the first The probability distribution of each category is used to output the corresponding clinical imaging lesion screening results.

[0162] Preferably, the clinical ultrasound lesion annotated image set is divided into a training set, a validation set, and a test set according to a 7:2:1 ratio. The deep learning model architecture based on multi-scale feature fusion is trained using the training set. During training, cross-entropy loss is used as the optimization objective, and stochastic gradient descent algorithm and its variant Adam are used for parameter updates. Data augmentation with random flipping, rotation, and scaling is used to increase the diversity of training data. An early stopping mechanism is introduced to stop training when the loss on the validation set no longer decreases to prevent overfitting. The trained model is evaluated using the test set, and the performance of the model is verified by accuracy, recall, and F1 score, thereby constructing an image lesion auxiliary screening model.

[0163] In this embodiment of the invention, assuming a set of 1000 clinical ultrasound lesion-annotated images, they are divided into a training set (700 images), a validation set (200 images), and a test set (100 images) according to a 7:2:1 ratio. The deep learning model architecture based on multi-scale feature fusion is trained using the training set. During training, cross-entropy loss is used as the optimization objective. For multi-classification problems, the cross-entropy loss function formula is: ,in For the sample size, For the number of categories, For the sample Category The true label (0 or 1). For the sample Predicted as category The probability is calculated using the Adam algorithm, a variant of the stochastic gradient descent algorithm, for parameter updates. In the Adam algorithm, the learning rate is initialized to 0.001. =0.9, =0.999, In each iteration, the model parameters are updated based on the calculated gradient, as shown in the formula: ,in For the current parameter, For learning rate, and These are first-order moment estimation and second-order moment estimation, respectively. Data augmentation methods corresponding to random flipping, rotation, and scaling are used to increase the diversity of the training data. For example, the probability of randomly flipping the image horizontally is set to 0.5; the rotation angle range is set to... The scaling factor is between 15° and 15°; the scaling ratio is between 0.8 and 1.2. An early stopping mechanism is introduced: training stops when the loss on the validation set no longer decreases after 10 consecutive epochs to prevent overfitting. The trained model is evaluated using a test set, and its performance is verified using accuracy (Precision = TP / TP + FP), recall (Recall = TP / TP + FN), and F1 score (F1 = (2 × Precision × Recall) / Precision + Recall), where TP represents true positives, FP represents false positives, and FN represents false negatives, thus constructing an image-assisted lesion screening model.

[0164] Preferably, clinical ultrasound images of the subjects to be screened are acquired;

[0165] In this embodiment of the invention, clinical ultrasound images to be screened are obtained from the ultrasound examination department database of a hospital. These images are stored in DICOM format and contain metadata such as the patient's basic information, examination time, and equipment parameters. For example, a thyroid ultrasound image of a patient is obtained, with an image size of 384×384 pixels, a bit depth of 12 bits, and a storage format of DICOM 3.0. The image data is converted from DICOM format to a format that the model can process, that is, normalized to a tensor form in the [0,1] interval, while retaining the original resolution and spatial information of the image, in preparation for subsequent lesion-assisted screening.

[0166] Preferably, the clinical ultrasound images to be screened are input into the imaging lesion-assisted screening model for lesion-assisted screening, so as to output the location, type and severity of the lesion corresponding to the lesion on the clinical ultrasound image.

[0167] In this embodiment of the invention, the clinical ultrasound image to be screened is input into a trained image-assisted lesion screening model. Taking the input thyroid ultrasound image as an example, the image first enters the input layer of the model, and then passes through the feature extraction layer. Convolutional neural network branches of different scales extract detailed, medium-scale, and large-scale features of the image, respectively. An attention mechanism is used to enhance the focus on possible lesion areas. In the feature fusion layer, features of different scales are spliced ​​and fused to generate a clinical lesion feature representation map. The output layer outputs the probability distribution of each pixel belonging to different lesion types and normal categories based on the feature representation map. A probability threshold of 0.5 is set. If the probability of a pixel belonging to a certain category is greater than 0.5, then the pixel is determined to belong to that category. By analyzing the classification results of all pixels, a connected component labeling algorithm is used to determine the location of the lesion area. The coordinates of the bounding rectangle of the lesion area are calculated to obtain the specific location of the lesion on the image. The categories of pixels within the lesion area are statistically analyzed to determine the lesion type. Based on the pre-set relationship between lesion severity and category, the severity of the lesion is judged. Finally, the location, type, and severity of the lesion corresponding to the lesion on the clinical ultrasound image are output.

[0168] Therefore, the embodiments should be considered as exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of the equivalents of the application are intended to be included within the invention.

[0169] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features of the invention herein.

Claims

1. A deep learning-based clinical ultrasound image assisted screening system, characterized in that, The method comprises the following modules: An ultrasound image preprocessing module is configured to acquire a clinical ultrasound image, and perform grayscale processing on the clinical ultrasound image to generate a clinical grayscale image; and perform neighborhood grayscale equalization processing on the clinical grayscale image to generate a clinical ultrasound equalization image; A lesion region segmentation module is configured to perform pixel normalization on the clinical ultrasound equalization image to generate a clinical ultrasound normalization image; perform texture feature analysis on the clinical ultrasound normalization image by using a gray level co-occurrence matrix calculation method to obtain clinical ultrasound texture features, including contrast, uniformity, correlation, energy and entropy; perform boundary contour separation on the texture boundary in the clinical ultrasound normalization image by using a Sobel operator to demarcate the boundary contour corresponding to the texture in the clinical image, and further refine the boundary contour corresponding to the texture in the clinical image by using a region growing algorithm to generate a clinical ultrasound image texture boundary; perform texture edge feature analysis on the clinical ultrasound image texture boundary to obtain clinical ultrasound texture edge features, including shape, size and area; and perform lesion region segmentation on the clinical ultrasound normalization image based on the clinical ultrasound texture features and the clinical ultrasound texture edge features based on the fuzzy texture boundary to generate a clinical ultrasound image lesion region block; The lesion region segmentation on the clinical ultrasound normalization image based on the clinical ultrasound texture features and the clinical ultrasound texture edge features based on the fuzzy texture boundary comprises: A clinical image lesion segmentation network is constructed based on the clinical ultrasound texture features and the clinical ultrasound texture edge features in combination with a U-Net to encapsulate the clinical ultrasound texture features and the clinical ultrasound texture edge features into a 3x3 network layer, and perform down convolution by using a ReLU activation function, and perform down pooling by using a 2x2 pooling layer to a bottleneck layer, and perform 2x2 up convolution by adding an attention module to the bottleneck layer and perform 1x1 convolution output to output an initial clinical tissue lesion region; The clinical ultrasound normalization image is input into the clinical image lesion segmentation network for lesion region segmentation to generate a clinical image initial lesion region segmentation block; The clinical image initial lesion region segmentation block is subjected to lesion refinement correction processing based on the fuzzy texture boundary to generate a clinical ultrasound image lesion region block; An image lesion annotation module is configured to acquire a corresponding lesion region position, lesion region type and lesion region severity by using the clinical ultrasound image lesion region block, perform lesion region classification processing on the clinical ultrasound image lesion region block by using a clustering algorithm to obtain the lesion region type, perform lesion degree evaluation on the clinical ultrasound image lesion region block to obtain the lesion region severity, and perform image lesion annotation on the corresponding clinical ultrasound image based on the lesion region position, the lesion region type and the lesion region severity to generate a clinical ultrasound lesion annotation image set. The lesion auxiliary screening module is used for constructing an image lesion auxiliary screening model based on a clinical ultrasound lesion annotation image set and combining a convolutional neural network training; acquiring a clinical ultrasound image to be screened; and inputting the clinical ultrasound image to be screened into the image lesion auxiliary screening model for lesion auxiliary screening to output a lesion position, a lesion type, and a lesion severity corresponding to a lesion existing on the clinical ultrasound image.

2. The deep learning-based clinical ultrasound image assisted screening system of claim 1, wherein, The ultrasound image preprocessing module includes the following functions: Acquiring a clinical ultrasound image; Calculating pixel blur of each pixel point in the clinical ultrasound image to obtain clinical ultrasound image pixel blur; Performing blur denoising processing on each pixel point in the clinical ultrasound image based on the clinical ultrasound image pixel blur to obtain a clinical blur denoising image; Performing grayscale processing on the clinical blur denoising image to generate a clinical grayscale image; Performing neighborhood grayscale equalization processing on the clinical grayscale image to generate a clinical ultrasound equalization image.

3. The deep learning-based clinical ultrasound image assisted screening system of claim 2, wherein, The neighborhood grayscale equalization processing on the clinical grayscale image includes: Dividing the clinical grayscale image into various clinical grayscale image blocks by dividing the clinical grayscale image into 3x3 or 9x9 pixel intervals; Calculating a grayscale histogram of each clinical grayscale image block to obtain a grayscale histogram corresponding to each grayscale image block; Searching for a pixel point with a large grayscale deviation from other pixel points in the grayscale histogram corresponding to each grayscale image block, and performing histogram equalization processing on the pixel point based on grayscale values of neighborhood pixel points in the grayscale histogram corresponding to the pixel point to generate each grayscale equalization image block; Performing clinical image recombination on each grayscale equalization image block by using bilinear interpolation to generate a clinical ultrasound equalization image.

4. The deep learning-based clinical ultrasound image assisted screening system of claim 1, wherein, The lesion morphology refinement correction processing based on tissue boundary blur on the clinical image initial lesion region segmentation block includes: Extracting a tissue boundary edge line of the clinical image initial lesion region segmentation block to generate a clinical image lesion tissue boundary edge line; Performing boundary blurring processing on grayscale values of a corresponding edge region in the clinical image lesion tissue boundary edge line to obtain a lesion tissue boundary edge grayscale fuzzy line region; Performing segmentation region extraction based on tissue boundary blur on the lesion tissue boundary edge grayscale fuzzy line region to generate a clinical image lesion region fuzzy segmentation block; Performing lesion morphology refinement correction on a corresponding lesion region contour morphology in the clinical image lesion region fuzzy segmentation block by using morphological operations corresponding to erosion, dilation, or opening and closing operations to generate a clinical ultrasound image lesion region block.

5. The deep learning-based clinical ultrasound image assisted screening system of claim 4, wherein, The segmentation region extraction based on tissue boundary blur on the lesion tissue boundary edge grayscale fuzzy line region includes: Setting a 3x3 grayscale fuzzy sliding window, and performing fuzzy C-means and standard deviation calculation on each pixel point in the lesion tissue boundary edge grayscale fuzzy line region based on the grayscale fuzzy sliding window to obtain a fuzzy C-means value and a fuzzy C-standard deviation value corresponding to each pixel point in the lesion tissue fuzzy line region; According to the fuzzy C mean and the fuzzy C standard deviation corresponding to each pixel point in the lesion tissue fuzzy line region, tissue fuzzy coefficient corresponding to each pixel point in the lesion tissue fuzzy line region is quantified; Based on the tissue fuzzy coefficient corresponding to each pixel point in the lesion tissue fuzzy line region, fuzzy membership block expansion segmentation is performed between each pixel point in the gray fuzzy line region of the lesion tissue boundary edge, to generate a fuzzy segmentation block of the clinical image lesion region.

6. The deep learning-based clinical ultrasound image assisted screening system of claim 1, wherein, The image lesion annotation module includes the following functions: Obtain the corresponding lesion region position of the clinical ultrasound image in the clinical ultrasound image through the clinical ultrasound image lesion region block; Classify the clinical ultrasound image lesion region block by using a clustering algorithm to obtain the lesion region type; Evaluate the lesion degree of the clinical ultrasound image lesion region block to obtain the lesion region severity; Based on the lesion region position, the lesion region type and the lesion region severity, the corresponding clinical ultrasound image is annotated to generate a clinical ultrasound lesion annotation image set.

7. The deep learning-based clinical ultrasound image assisted screening system of claim 6, wherein, The lesion degree evaluation of the clinical ultrasound image lesion region block includes: Perform lesion texture local binary analysis on the clinical ultrasound image lesion region block to obtain a clinical lesion region texture local binary pattern; Based on the clinical lesion region texture local binary pattern, evaluate the local texture complexity of the clinical ultrasound image lesion region block to obtain the local texture complexity of the clinical image lesion region; Obtain the corresponding lesion tissue elasticity quantity of the clinical ultrasound image lesion region block by measuring the clinical ultrasound image lesion region block, and estimate the lesion hardness of the clinical ultrasound image lesion region block based on the lesion tissue elasticity quantity to obtain the tissue hardness of the clinical image lesion region; According to the local texture complexity of the clinical image lesion region and the tissue hardness of the clinical image lesion region, the lesion degree is evaluated to obtain the lesion region severity.

8. The deep learning-based clinical ultrasound image assisted screening system of claim 1, wherein, The lesion auxiliary screening module includes the following functions: A deep learning model architecture based on multi-scale feature fusion is constructed by combining a convolutional neural network design, including an input layer, a feature extraction layer, a feature fusion layer and an output layer, wherein the input layer is used to receive the annotated clinical ultrasound image; the feature extraction layer adopts a convolutional neural network branch including 3x3, 5x5 and 7x7 different scales to extract features of the input clinical ultrasound image, to capture image features at different scales, and to enhance attention to the lesion region by introducing an attention mechanism; the feature fusion layer fuses image features at different scales to generate a corresponding clinical lesion feature representation graph by using a skip connection and feature splicing; and the output layer outputs the corresponding clinical image lesion screening result; The clinical ultrasound lesion annotation image set is divided into a training set, a verification set and a test set according to a division ratio of 7:2:1, and a deep learning model architecture based on multi-scale feature fusion is trained by using the training set; in the training process, cross-entropy loss is used as the optimization target, and the random gradient descent algorithm and its variant Adam are used for parameter updating; random flipping, rotation and scaling are used for data enhancement to increase the diversity of training data; an early stopping mechanism is introduced to stop training when the loss on the verification set no longer decreases to prevent overfitting; the trained model is evaluated by using the test set; the accuracy, recall rate and F1 value are used to verify the performance of the model, so as to build an image lesion auxiliary screening model; Obtain a clinical ultrasound image to be screened; The clinical ultrasound image to be screened is input into the image lesion auxiliary screening model for lesion auxiliary screening, so as to output the lesion position, lesion type and lesion severity corresponding to the lesion on the clinical ultrasound image.

Images