Clinical ultrasonic image auxiliary screening system based on deep learning

Abstract

The invention relates to the technical field of image processing, in particular to a clinical ultrasonic image auxiliary screening system based on deep learning. The system comprises an ultrasonic image preprocessing module, a lesion area segmentation module, an image lesion marking module and a lesion auxiliary screening module, and can obtain a clinical ultrasonic image and perform graying processing and neighborhood gray level equalization processing to generate a clinical ultrasonic equalization image; performing pixel normalization and tissue boundary fuzzy-based lesion region segmentation on the clinical ultrasonic equalization image to generate clinical ultrasonic image lesion region blocks; carrying out image lesion marking through the clinical ultrasonic image lesion area block to generate a clinical ultrasonic lesion marking image set; and constructing an image lesion auxiliary screening model and carrying out lesion auxiliary screening so as to output lesion positions, lesion types and lesion severity corresponding to lesions on the clinical ultrasonic image. According to the invention, accurate analysis of clinical ultrasonic images can be realized, so that lesion screening can be efficiently completed in an assisted manner.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to a clinical ultrasound image auxiliary screening system based on deep learning. BACKGROUND

[0002] In recent years, the application of deep learning technology in medical image analysis has made significant progress. Deep learning, especially convolutional neural networks (CNN), has been widely used in image classification, object detection, image segmentation and other fields, showing extremely high performance and potential. For ultrasound images, deep learning can automatically learn the feature information in the images and perform efficient lesion screening. Compared with traditional image analysis methods, deep learning-based ultrasound image analysis methods have the advantages of high automation, fast processing speed, high accuracy, etc., which can effectively reduce the error of manual intervention and assist doctors in accurate image screening. However, the existing deep learning-based ultrasound image analysis methods mostly use conventional enhancement and noise reduction techniques in the image processing link, which is difficult to fully exploit the potential information of the images, and the global feature expression of complex lesions is insufficient, resulting in poor accuracy and generalization ability of the model in actual clinical screening. SUMMARY

[0003] Therefore, it is necessary to provide a clinical ultrasound image auxiliary screening system based on deep learning to solve at least one of the above technical problems.

[0004] To achieve the above-mentioned purpose, a clinical ultrasound image auxiliary screening system based on deep learning comprises the following modules: An ultrasound image preprocessing module is configured to acquire a clinical ultrasound image, and perform grayscale processing on the clinical ultrasound image to generate a clinical grayscale image; and perform neighborhood gray balance processing on the clinical grayscale image to generate a clinical ultrasound equalization image. A lesion region segmentation module is configured to perform pixel normalization on the clinical ultrasound equalization image to generate a clinical ultrasound normalized image; acquire corresponding clinical ultrasound tissue texture features and clinical ultrasound tissue edge features through the clinical ultrasound normalized image; and perform lesion region segmentation based on tissue boundary blur on the clinical ultrasound normalized image based on the clinical ultrasound tissue texture features and the clinical ultrasound tissue edge features to generate a clinical ultrasound image lesion region block. An image lesion annotation module is configured to acquire corresponding lesion region positions, lesion region types and lesion region severities through the clinical ultrasound image lesion region block; and perform image lesion annotation on the corresponding clinical ultrasound image based on the lesion region positions, the lesion region types and the lesion region severities to generate a clinical ultrasound lesion annotation image set. The lesion auxiliary screening module is used for constructing an image lesion auxiliary screening model based on a clinical ultrasound lesion annotation image set and combining a convolutional neural network training; acquiring a clinical ultrasound image to be screened, and inputting the clinical ultrasound image to be screened into the image lesion auxiliary screening model for lesion auxiliary screening to output a lesion position, a lesion type and a lesion severity corresponding to a lesion existing on the clinical ultrasound image.

[0005] Further, the ultrasound image preprocessing module comprises the following functions: Acquiring a clinical ultrasound image; Performing pixel blur degree calculation on each pixel point in the clinical ultrasound image to obtain a clinical ultrasound image pixel blur degree; Performing blur denoising processing on each pixel point in the clinical ultrasound image based on the clinical ultrasound image pixel blur degree to obtain a clinical blur denoising image; Performing gray scale processing on the clinical blur denoising image to generate a clinical gray scale image; Performing neighborhood gray scale equalization processing on the clinical gray scale image to generate a clinical ultrasound equalization image.

[0006] Further, the neighborhood gray scale equalization processing according to the clinical gray scale image comprises: Dividing the clinical gray scale image into various clinical gray scale image blocks by dividing the clinical gray scale image into 3x3 or 9x9 pixel intervals; Performing gray scale histogram calculation on each clinical gray scale image block to obtain a gray scale histogram corresponding to each gray scale image block; Searching for a pixel point having a large gray scale deviation from other pixel points in the gray scale histogram corresponding to each gray scale image block, and performing histogram equalization processing on the pixel point based on the gray scale values of the neighborhood pixel points in the gray scale histogram corresponding to the pixel point to generate each gray scale equalization image block; Performing clinical image recombination on each gray scale equalization image block by using bilinear interpolation to generate a clinical ultrasound equalization image.

[0007] Further, the lesion region segmentation module comprises the following functions: Performing pixel normalization on the clinical ultrasound equalization image to generate a clinical ultrasound normalized image; Performing tissue texture feature analysis on the clinical ultrasound normalized image by using a gray scale co-occurrence matrix calculation method to obtain clinical ultrasound tissue texture features, including contrast, uniformity, correlation, energy and entropy; The tissue boundary in the clinical ultrasound normalized image is separated by a Sobel operator to demarcate the corresponding boundary contour of the tissue in the clinical image, and a region growing algorithm is used to further refine the corresponding boundary contour of the tissue in the clinical image to generate the tissue boundary of the clinical ultrasound image. The tissue edge feature analysis is performed on the tissue boundary of the clinical ultrasound image to obtain the clinical ultrasound tissue edge feature, including shape, size and area. Based on the clinical ultrasound tissue texture feature and the clinical ultrasound tissue edge feature, the lesion region segmentation based on the fuzzy tissue boundary is performed on the clinical ultrasound normalized image to generate the clinical ultrasound image lesion region block.

[0008] Further, the lesion region segmentation based on the fuzzy tissue boundary based on the clinical ultrasound tissue texture feature and the clinical ultrasound tissue edge feature includes: Based on the clinical ultrasound tissue texture feature and the clinical ultrasound tissue edge feature, a corresponding clinical image lesion segmentation network is constructed to encapsulate the clinical ultrasound tissue texture feature and the clinical ultrasound tissue edge feature into a 3x3 network layer, and a ReLU activation function is used for down-convolution, while a 2x2 pooling layer is used for down-pooling to the bottleneck layer. The 2x2 up-convolution is performed by adding an attention module to the bottleneck layer, and the 1x1 convolution output is performed, so as to output the initial clinical tissue lesion region. The clinical ultrasound normalized image is input into the clinical image lesion segmentation network for lesion region segmentation to generate the clinical image initial lesion region segmentation block. The clinical image initial lesion region segmentation block is subjected to lesion refinement correction processing based on the fuzzy tissue boundary to generate the clinical ultrasound image lesion region block.

[0009] Further, the lesion refinement correction processing based on the fuzzy tissue boundary of the clinical image initial lesion region segmentation block includes: The clinical image initial lesion region segmentation block is subjected to tissue edge extraction to generate the clinical image lesion tissue boundary edge line. The gray value of the corresponding edge region in the clinical image lesion tissue boundary edge line is subjected to boundary fuzzing processing to obtain the lesion tissue boundary edge gray fuzzy line region. Based on the lesion tissue boundary edge gray fuzzy line region, the segmentation region extraction based on the fuzzy tissue boundary is performed to generate the clinical image lesion region fuzzy segmentation block. The corresponding lesion region contour morphology in the clinical image lesion region fuzzy segmentation block is subjected to lesion morphology refinement correction by morphological operations corresponding to erosion, dilation or opening operation and closing operation to generate the clinical ultrasound image lesion region block.

[0010] Further, the segmentation region extraction based on the fuzzy boundary of the lesion tissue boundary edge gray fuzzy line region comprises: By setting a 3x3 corresponding gray fuzzy sliding window, and based on the gray fuzzy sliding window, fuzzy C mean and standard deviation calculation is performed on each pixel point in the lesion tissue boundary edge gray fuzzy line region, to obtain the fuzzy C mean and fuzzy C standard deviation corresponding to each pixel point in the lesion tissue fuzzy line region; According to the fuzzy C mean and fuzzy C standard deviation corresponding to each pixel point in the lesion tissue fuzzy line region, tissue fuzzy coefficient quantization is performed, to obtain the tissue fuzzy coefficient corresponding to each pixel point in the lesion tissue fuzzy line region; Based on the tissue fuzzy coefficient corresponding to each pixel point in the lesion tissue fuzzy line region, fuzzy membership block expansion segmentation is performed between each pixel point in the lesion tissue boundary edge gray fuzzy line region, to generate a fuzzy segmentation block of the clinical image lesion region.

[0011] Further, the image lesion annotation module comprises the following functions: The corresponding lesion region position in the clinical ultrasound image is obtained by the clinical ultrasound image lesion region block; The lesion region classification processing is performed on the clinical ultrasound image lesion region block by using a clustering algorithm, to obtain the lesion region type; The lesion degree evaluation is performed on the clinical ultrasound image lesion region block, to obtain the lesion region severity; Based on the lesion region position, the lesion region type and the lesion region severity, the image lesion annotation is performed on the corresponding clinical ultrasound image, to generate a clinical ultrasound lesion annotation image set.

[0012] Further, the lesion degree evaluation on the clinical ultrasound image lesion region block comprises: The lesion texture local binary analysis is performed on the clinical ultrasound image lesion region block, to obtain the clinical lesion region texture local binary pattern; Based on the clinical lesion region texture local binary pattern, the local texture complexity evaluation is performed on the clinical ultrasound image lesion region block, to obtain the clinical image lesion region local texture complexity; The corresponding lesion tissue elasticity quantity of the clinical ultrasound image lesion region block is obtained by the clinical ultrasound image lesion region block, and based on the lesion tissue elasticity quantity, the lesion hardness estimation is performed on the clinical ultrasound image lesion region block, to obtain the clinical image lesion region tissue hardness; According to the clinical image lesion region local texture complexity and the clinical image lesion region tissue hardness, the lesion degree evaluation is performed, to obtain the lesion region severity.

[0013] Further, the lesion auxiliary screening module comprises the following functions: The deep learning model architecture based on multi-scale feature fusion is designed and constructed by combining a convolutional neural network, including an input layer, a feature extraction layer, a feature fusion layer and an output layer, wherein the input layer is used to receive the labeled clinical ultrasound image; the feature extraction layer adopts a convolutional neural network branch including different scales of 3x3, 5x5 and 7x7 to extract features of the input clinical ultrasound image, so as to capture image features at different scales, and the attention mechanism is introduced to enhance the attention to the lesion area; the feature fusion layer fuses the image features at different scales to generate a corresponding clinical lesion feature representation graph by using the jump connection and feature splicing; and the output layer outputs the corresponding clinical image lesion screening result. The clinical ultrasound lesion annotation image set is divided into a training set, a validation set and a test set according to a division ratio of 7:2:1, and the deep learning model architecture based on multi-scale feature fusion is trained by using the training set; in the training process, the cross-entropy loss is used as the optimization target, the random gradient descent algorithm and its variant Adam are used for parameter updating, the corresponding data enhancement of random flipping, rotation and scaling is used to increase the diversity of the training data, the early stopping mechanism is introduced to stop training when the loss on the validation set no longer decreases to prevent overfitting, the trained model is evaluated by using the test set, the accuracy, recall rate and F1 value are used to verify the performance of the model, so as to construct an image lesion auxiliary screening model. Obtaining a clinical ultrasound image to be screened; Inputting the clinical ultrasound image to be screened into the image lesion auxiliary screening model for lesion auxiliary screening to output the lesion position, lesion type and lesion severity corresponding to the lesion existing on the clinical ultrasound image.

[0014] The beneficial effects of the present application are: The application has the beneficial effects that the ultrasonic image can be converted into a single gray image through the gray processing, the interference of color is removed, the structural information of the image is more prominent, the image after the gray processing can better reflect the texture and structural features of the image, especially the brightness difference of different regions can be more clearly presented, the contrast of the image can be effectively enhanced through the neighborhood gray equalization processing, the visibility of the image is improved, the neighborhood gray equalization adjusts the gray value of each pixel point in the image, so that the brightness distribution of the whole image is more uniform, so that the details are more obvious, the quality of the image is improved, the influence of the image quality problem on the analysis result can be reduced, and the distinguishability of the lesion area is improved. Secondly, the pixel normalization unifies the pixel value range of the image, so that the pixel values of each region can be compared under the same scale, which can reduce the influence caused by different image scales, brightness differences and the like, improve the accuracy of feature recognition, and through the extraction of the organization texture features and the organization edge features of the normalized image, the subtle structural changes in the image can be helped to be analyzed, and the potential lesion area is further identified. These features include the roughness, uniformity of texture, and the sharpness of the edge, which can provide a detailed description of the global feature expression of the tissue lesion in the image, and help to distinguish the normal and abnormal regions. On this basis, through the segmentation method based on the fuzzy organization boundary, the lesion area can be more accurately segmented, especially in the case of fuzzy boundary or complex organization structure, this method can provide more sensitive segmentation effect, so as to effectively demarcate the boundary of the lesion area, and generate the lesion area block, and provide accurate data support for subsequent analysis. Then, through further analysis of the lesion area block, the specific position, type and severity of the lesion area can be obtained. These information is crucial for image analysis, because they can provide quantitative labeling data for clinical research and image processing. The position of the lesion area can help the analyst to quickly locate the lesion area, and the type and severity can further classify the lesion features. Through the labeled image set, high-quality labeled data can be provided for subsequent model training. The labeled image set provides accurate labels for model training, and high-quality labeled image set can promote the training and optimization of the algorithm, so as to improve the universality and accuracy of the model on different image samples, and help to provide more accurate auxiliary analysis in practical application.Finally, by constructing a Convolutional Neural Network (CNN) model, deep learning methods can be used to automate image analysis. The CNN model learns features from annotated image sets, gradually improving its predictive ability on unknown images. Trained on large-scale image datasets, the model can extract high-dimensional features from complex image data and automatically identify lesion areas. This process significantly improves the efficiency and accuracy of image analysis, avoids biases from manual annotation, and provides more objective results. Furthermore, through the analysis of images to be screened, the model can output the location, type, and severity of lesions in real time, thereby improving the model's accuracy and generalization ability in actual clinical screening. The application of this system enables rapid screening of potential lesion areas from large amounts of image data, reducing the risk of missed or misdiagnosed diagnoses and providing strong support for further examination or analysis. Attached Figure Description

[0015] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a schematic diagram of the modules of the deep learning-based clinical ultrasound image-assisted screening system of the present invention; Figure 2 for Figure 1 A functional flowchart of the ultrasound image preprocessing module; Figure 3 for Figure 1 A functional flowchart of the lesion area segmentation module. Detailed Implementation

[0016] The technical system of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0017] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor systems and / or microcontroller systems.

[0018] It should be understood that, although the terms "first", "second" or the like can be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first element could be termed a second element, and, similarly, a second element could be termed a first element without departing from the scope of the example embodiments. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0019] To achieve the above object, please refer to Figures 1 to 3 The application provides a clinical ultrasound image auxiliary screening system based on deep learning, which comprises the following modules: An ultrasound image preprocessing module is configured to acquire a clinical ultrasound image, perform grayscale processing on the clinical ultrasound image to generate a clinical grayscale image, and perform neighborhood grayscale equalization processing on the clinical grayscale image to generate a clinical ultrasound equalization image. A lesion region segmentation module is configured to perform pixel normalization on the clinical ultrasound equalization image to generate a clinical ultrasound normalization image, acquire corresponding clinical ultrasound tissue texture features and clinical ultrasound tissue edge features through the clinical ultrasound normalization image, and perform lesion region segmentation based on tissue boundary blur on the clinical ultrasound normalization image based on the clinical ultrasound tissue texture features and the clinical ultrasound tissue edge features to generate a clinical ultrasound image lesion region block. An image lesion annotation module is configured to acquire corresponding lesion region positions, lesion region types, and lesion region severities through the clinical ultrasound image lesion region block, and perform image lesion annotation on the corresponding clinical ultrasound image based on the lesion region positions, the lesion region types, and the lesion region severities to generate a clinical ultrasound lesion annotation image set. A lesion auxiliary screening module is configured to construct an image lesion auxiliary screening model based on the clinical ultrasound lesion annotation image set and in combination with convolutional neural network training, acquire a clinical ultrasound image to be screened, input the clinical ultrasound image to be screened into the image lesion auxiliary screening model for lesion auxiliary screening, and output lesion positions, lesion types, and lesion severities corresponding to lesions existing on the clinical ultrasound image.

[0020] In the embodiment of the present application, by acquiring clinical ultrasound images from the storage system of the hospital ultrasonic examination equipment, the images are usually stored in the Digital Imaging and Communications in Medicine (DICOM) format, taking a color breast ultrasound image with a resolution of 512*512 pixels as an example, a weighted average method is used for gray scale processing, the formula is Gray=0.299R+0.587G+0.114B, wherein R, G and B are respectively the pixel values of the red, green and blue three channels in the color image, each pixel point in the image is traversed, the RGB value is calculated according to the formula, the corresponding gray value is obtained, and the clinical gray scale image is generated, and the neighborhood gray balance processing is performed on the clinical gray scale image, the local adaptive histogram equalization (CLAHE) method is used, the image is divided into a plurality of 8*8 sub-blocks, for each sub-block, the gray histogram is calculated, the frequency of each gray level is counted, the pixel gray value in the sub-block is adjusted by the cumulative distribution function (CDF), the formula is , wherein is the original gray value, and are respectively the minimum value and the maximum value of the sub-block gray cumulative distribution function, is the total number of gray levels (usually 256), in order to avoid noise amplification, the contrast limit threshold is set to 40, the pixel gray value exceeding the threshold is clipped and normalized again, and the clinical ultrasound equalization image is finally generated.

[0021] S2: a lesion region segmentation module, configured to perform pixel normalization on the clinical ultrasound equalization image to generate a clinical ultrasound normalized image; obtain corresponding clinical ultrasound tissue texture features and clinical ultrasound tissue edge features through the clinical ultrasound normalized image, and perform lesion region segmentation based on the tissue boundary blur based on the clinical ultrasound tissue texture features and the clinical ultrasound tissue edge features to generate a clinical ultrasound image lesion region block; In the embodiment of the present application, the pixel normalization is performed on the clinical ultrasound equalization image, and the minimum-maximum normalization method is used, the formula is , wherein is the normalized pixel value, is the original pixel value, is the minimum value of the pixel value in the image, is the maximum value of the pixel value in the image, assuming that the clinical ultrasound equalization image =20, =200, for the point with the pixel value =50, after normalization, ≈0.17, all pixel points are normalized to generate a clinical ultrasound normalized image. The gray level co-occurrence matrix (GLCM) is used to calculate the clinical ultrasound tissue texture features, the gray level is set to 16, the calculation direction is 0°, 45°, 90° and 135°, the distance is 1 pixel, taking 0° direction as an example, a 16*16 GLCM matrix is constructed, the number of pixel pairs meeting the conditions is counted, and the contrast , uniformity and other characteristic values are calculated by formula, and the average value of the four directions is taken, and the clinical ultrasound tissue edge features are obtained by using the Canny edge detection algorithm, a Gaussian filter with a standard deviation of 1.5 is used for denoising first, and then a Sobel operator is used to calculate the gradient amplitude and direction, and the edge connection is performed by setting the low threshold value to 30 and the high threshold value to 90 to obtain the edge contour. Based on the above features, U-Net combined with fuzzy C-means clustering is used for lesion region segmentation, the features are used as the input of U-Net, the 3*3 convolution layer in the network extracts the features, the 2*2 pooling layer down-samples, the attention module is added to the bottleneck layer, and then the output is obtained through the upper convolution and 1*1 convolution. The output result is clustered by fuzzy C-means clustering, and the clustering number is set to 3 (background, lesion edge and lesion core). The membership degree is calculated according to the distance between the pixel point and the cluster center, and the formula is (m=2), the pixel points with a membership degree greater than 0.5 are divided into lesion regions, and finally the clinical ultrasound image lesion region block is generated.

[0022] S3: image lesion labeling module, used for acquiring the corresponding lesion region position, lesion region type and lesion region severity through the clinical ultrasound image lesion region block, and labeling the corresponding clinical ultrasound image based on the lesion region position, lesion region type and lesion region severity, to generate a clinical ultrasound lesion labeling image set; In the embodiment of the present application, the generated clinical ultrasound image lesion region block is used to record its coordinate range in the original clinical ultrasound image to obtain the lesion region position. Assuming that the upper left corner coordinate of the lesion region block is (100, 100) and the lower right corner coordinate is (300, 300), the lesion is located in the rectangular range in the image, and the DBSCAN clustering algorithm is used to classify the lesion region block, the neighborhood radius =3 pixels, and the minimum sample number MinPts=8. The Euclidean distance between the pixel points is calculated , the clusters are divided according to the density, the different clusters are corresponded to the preset lesion types (such as cyst, tumor, etc.), and the texture complexity, shape factor and elastic modulus of the lesion region are calculated to evaluate the severity of the lesion. The texture complexity is calculated combined with the GLCM features, and the shape factor formula is F=4πA / C 2(A is the area of the lesion region, C is the perimeter), the elastic modulus is obtained by ultrasonic elastography, and the elastic modulus E of the lesion tissue is obtained by ultrasonic elastography E=15kPa (kiloPascal), and the lesion degree evaluation formula S=α×(0.3×contrast+0.2×entropy+0.1×energy+0.1×correlation)+β×(1 F)+γ×E is also established, wherein α=0.3, β=0.3, γ=0.4, the threshold is set to divide the mild, moderate and severe lesions, based on the above information, the lesion position is marked on the clinical ultrasound image by a rectangular frame, the type and severity of the lesion are marked beside the frame, and the labeled image and related information are stored, and finally the clinical ultrasound lesion labeled image set is generated.

[0023] S4: a lesion auxiliary screening module, configured to construct an image lesion auxiliary screening model based on the clinical ultrasound lesion labeled image set and in combination with a convolutional neural network training; acquire a clinical ultrasound image to be screened, and input the clinical ultrasound image to be screened into the image lesion auxiliary screening model for lesion auxiliary screening, to output a lesion position, a lesion type and a lesion severity corresponding to a lesion existing on the clinical ultrasound image.

[0024] In the embodiment of the present application, the clinical ultrasound lesion labeled image set is divided into a training set, a verification set and a test set according to 7:2:1, for example, 1000 images, 700 images in the training set, 200 images in the verification set and 100 images in the test set, a convolutional neural network architecture based on multi-scale feature fusion is designed, the input layer receives an image with a size of 224*224, which is converted into a tensor with a size of (224, 224, 1), the feature extraction layer includes three convolution branches with sizes of 3*3, 5*5 and 7*7, the 3*3 branch has 6 layers, and the channel numbers are 16, 32, 64, 128, 256 and 512 in sequence; the 5*5 branch has 4 layers, and the channel numbers are 64, 128, 256 and 512; the 7*7 branch has 3 layers, and the channel numbers are 128, 256 and 512, each layer is connected with a ReLU activation function, and a channel attention mechanism is introduced at the end of the branch, and the weight formula is , which is multiplied with the feature map to enhance the key features, the feature fusion layer is used to adjust the output size of the branch to be consistent, then spliced, and then connected with the input layer, the output layer is compressed to the number of lesion categories (assuming 6 categories) through 1*1 convolution, and the classification probability is output through the Softmax function, and the parameters are updated by the Adam optimizer (learning rate 0.001, =0.9, =0.999, =0.9, ), random flip, rotation (±15°), zoom (0.8-1.2 times) enhanced data, early stop mechanism, 10 consecutive epoch validation set loss does not drop, then stop training. With the test set to evaluate the model with accuracy Precision = TP / TP+FP, recall Recall = TP / TP+FN, F1 value F1 = 2xPrecisionxRecall) / Precision+Recall, complete the image lesion auxiliary screening model construction, and obtain the clinical ultrasound image to be screened from the hospital database, pre-process into a normalized tensor, input the model, the model outputs the lesion probability of each pixel point, judges the lesion position through the threshold value (threshold value 0.5), determines the lesion type by marking the connected region, determines the severity according to the preset relationship, and finally outputs the lesion position, lesion type and lesion severity corresponding to the lesion on the clinical ultrasound image. 2xPrecisionxRecall) / Precision+Recall, complete the image lesion auxiliary screening model construction, and obtain the clinical ultrasound image to be screened from the hospital database, pre-process into a normalized tensor, input the model, the model outputs the lesion probability of each pixel point, judges the lesion position through the threshold value (threshold value 0.5), determines the lesion type by marking the connected region, determines the severity according to the preset relationship, and finally outputs the lesion position, lesion type and lesion severity corresponding to the lesion on the clinical ultrasound image.

[0025] Further, the ultrasound image preprocessing module comprises the following functions: Obtain the clinical ultrasound image; Calculate the pixel blur degree of each pixel point in the clinical ultrasound image to obtain the clinical ultrasound image pixel blur degree; Based on the clinical ultrasound image pixel blur degree, perform blur denoising processing on each pixel point in the clinical ultrasound image to obtain the clinical blur denoising image; Perform grayscale processing on the clinical blur denoising image to generate a clinical grayscale image; Perform neighborhood gray balance processing on the clinical grayscale image to generate a clinical ultrasound equalization image.

[0026] As an embodiment of the present application, refer to Figure 2 , which is Figure 1 a functional flow diagram of the ultrasound image preprocessing module in the embodiment, the ultrasound image preprocessing module comprises the following functions: S11: Obtain the clinical ultrasound image; In the embodiment of the present application, the clinical ultrasound image is obtained from the ultrasound examination device of the hospital, and these images are usually stored in digital format, such as DICOM (Digital Imaging and Communications in Medicine) format. It is assumed that the obtained clinical ultrasound image is a two-dimensional image with a size of 512x512 pixels, and the gray value of each pixel point ranges from 0 to 255.

[0027] S12: Calculate the pixel blur degree of each pixel point in the clinical ultrasound image to obtain the clinical ultrasound image pixel blur degree; In the embodiment of the present application, the gradient amplitude method is used to calculate the pixel blurring degree by calculating the pixel blurring degree of each pixel point in the clinical ultrasound image. For each pixel point in the image The gradient amplitude can be calculated by the following formula: Wherein, and are the gradients of the pixel point in the and directions, the gradient is usually calculated using the Sobel operator, and the templates of the Sobel operator in the and directions are: and The gradient values of the pixel point in the and directions are obtained by convolving the Sobel operator with the neighborhood around the pixel point, and the gradient amplitude is calculated, the smaller the gradient amplitude, the higher the blurring degree of the pixel point, for example, for a certain pixel point, the gradient amplitude is 10 after calculation, which is taken as the blurring degree of the pixel point, and the clinical ultrasound image pixel blurring degree is finally obtained.

[0028] S13: Based on the clinical ultrasound image pixel blurring degree, each pixel point in the clinical ultrasound image is subjected to blurring denoising processing to obtain a clinical blurring denoised image; In the embodiment of the present application, each pixel point in the clinical ultrasound image is subjected to blurring denoising processing based on the clinical ultrasound image pixel blurring degree, and the Gaussian filtering method is used for denoising. Gaussian filtering is a linear smoothing filter, which reduces noise by weighted average of the pixel point and its neighborhood. The weight matrix of the Gaussian filter can be calculated by the following formula: Wherein, is the center position of the filter, is the standard deviation of the Gaussian distribution, which determines the smoothing degree of the filter. For each pixel point, the standard deviation of the Gaussian filter is adjusted according to its blurring degree. The higher the blurring degree, the larger the standard deviation, and the stronger the filtering effect. For example, for a pixel point with high blurring degree, the standard deviation is set to 3; for a pixel point with low blurring degree, the standard deviation is set to 1. By performing Gaussian filtering on each pixel point, a clinical blurring denoised image is finally obtained.

[0029] S14: The clinical blurring denoised image is subjected to grayscale processing to generate a clinical grayscale image; In the embodiment of the present application, the clinical fuzzy denoising image is subjected to gray processing. If the clinical ultrasound image itself is a gray image, the next step can be directly performed. If it is a color image, it needs to be converted into a gray image. The commonly used gray processing method is the weighted average method, and the formula is: Gray=0.299R+0.587G+0.114B, wherein R, G and B are respectively the pixel values of the red, green and blue three channels in the color image. The corresponding gray value is obtained by calculating the RGB value of each pixel point according to the above formula, so as to generate the clinical gray image.

[0030] S15: According to the clinical gray image, the neighborhood gray balance processing is performed to generate the clinical ultrasound balanced image.

[0031] In the embodiment of the present application, the neighborhood gray balance processing is performed according to the clinical gray image, and the histogram equalization method is used to realize the neighborhood gray balance. The histogram equalization is a method for adjusting the gray histogram of an image to enhance the contrast of the image. Specifically, the gray histogram of the clinical gray image is calculated, the frequency of each gray value is counted, the cumulative distribution function (CDF) is calculated according to the gray histogram, that is, the cumulative frequency of each gray value and the gray value below it, and the gray value of each pixel point is mapped according to the cumulative distribution function to obtain a new gray value. The mapping formula is: newGray=round((L-1) CDF(gray)), wherein L is the range of the gray value (usually 0-255), gray is the gray value of the original pixel point, and newGray is the mapped gray value. The mapped gray value is applied to each pixel point of the clinical gray image, and finally the clinical ultrasound balanced image is generated.

[0032] Further, the neighborhood gray balance processing according to the clinical gray image comprises: The clinical gray image is divided into various clinical gray image blocks by dividing a certain pixel interval 3x3 or 9x9; In the embodiment of the present application, when processing the clinical grayscale image, the clinical grayscale image is divided into clinical grayscale image blocks by using a fixed pixel division interval (including 3x3 or 9x9). Taking a clinical grayscale liver ultrasound image with a resolution of 512x512 as an example, a 9x9 pixel division interval is selected. Starting from the upper left corner of the image, the image is divided into blocks with a size of 9x9 pixels. The image can be divided into 57 blocks horizontally and 57 blocks vertically, i.e., a total of 57x57=3249 clinical grayscale image blocks. In the division process, the areas with less than 9x9 pixels are filled with zeros to ensure that each image block has a size of 9x9 pixels, providing a uniform data unit for subsequent processing of each image block.

[0033] Preferably, a gray level histogram of each clinical grayscale image block is calculated to obtain a corresponding gray level histogram of each clinical grayscale image block. In the embodiment of the present application, the gray level histogram of each clinical grayscale image block is calculated. Taking a 9x9 clinical grayscale image block as an example, the image block contains 81 pixels, and the gray level of each pixel ranges from 0 to 255. An array with a length of 256 is established to count the number of occurrences of each gray level. All 81 pixels in the image block are traversed. For each pixel, the gray level of the pixel is used as the index of the array, and the corresponding array element value is incremented by 1. For example, if there are three pixels with a gray level of 50 in the image block, the element value at index 50 in the array is 3 after traversal. After the counting is completed, each element value in the array is divided by the total number of pixels 81 to obtain the frequency of each gray level. Thus, the gray level histogram corresponding to the gray level image block is obtained. By calculating all 3249 clinical grayscale image blocks in this way, the gray level histogram of each image block is finally obtained. These histograms reflect the distribution of gray levels in each image block.

[0034] Preferably, the pixel points with large gray level deviation from the remaining pixel points are searched in the gray level histogram corresponding to each gray level image block, and the histogram equalization processing is performed on the pixel points based on the gray level of the neighboring pixel points in the gray level histogram corresponding to the pixel points to generate gray level equalization image blocks. In the embodiment of the present application, the pixel points with large gray level deviation from the remaining pixel points are searched in the gray level histogram corresponding to each gray level image block. Taking the gray level histogram of a certain gray level image block as an example, the gray level deviation threshold is set to 1.5 times the average gray level of the image block. First, the average gray level of the image block is calculated, and the formula is wherein is the gray level of the pixel point. Given the number of pixels, assuming the calculated average grayscale value of the image block is 80, the grayscale deviation threshold is 80 × 1.5 = 120. In the grayscale histogram, pixels with a grayscale value greater than 120 and a low frequency of occurrence (set to be less than 5% of the total number of pixels, i.e., 81 × 5% = 4.05, taken as less than or equal to 4) are identified as pixels with large grayscale deviations. For each pixel with a large grayscale deviation, histogram equalization is performed based on the grayscale values ​​of its neighboring pixels in the corresponding grayscale histogram. By employing a local adaptive histogram equalization method, a 3 × 3 neighborhood is selected centered on the pixel, and the average grayscale value and standard deviation of the pixels in the neighborhood are calculated. Let the set of pixels in the neighborhood be N, then the average grayscale value of the neighborhood is... Standard deviation According to the formula new_gray= (in and To adjust the parameters, Take 3. (Take 50) Adjust the gray value of the pixel to obtain a new gray value, and finally generate each gray-level equalized image block to make the gray-level distribution within the image block more even.

[0035] Preferably, bilinear interpolation is used to reconstruct clinical images from each grayscale equalization image block to generate clinical ultrasound equalization images.

[0036] In this embodiment of the invention, bilinear interpolation is used to reconstruct clinical images from each gray-level equalized image block. Taking two adjacent 9×9 gray-level equalized image blocks as an example, during reconstruction, bilinear interpolation is used to calculate pixel values ​​in the transition region between the two image blocks. Assuming that a point within the transition region needs to be calculated... The pixel value of the point is located at the boundary of two adjacent image blocks. Find the pixel value of the point in each of the two image blocks. Four adjacent pixels , , , First of all Perform linear interpolation in the direction to calculate and : , Then Direction and Perform linear interpolation to obtain the points Pixel values: According to the method, the transition areas between all 3249 gray equalization image blocks are calculated by bilinear interpolation, the image blocks are recombined together, and finally the clinical ultrasound equalization image is generated, so that the visual effect and detail performance of the image are effectively improved, and higher quality data is provided for subsequent clinical ultrasound image auxiliary screening based on deep learning.

[0037] Further, the lesion area segmentation module comprises the following functions: pixel normalization is performed on the clinical ultrasound equalization image to generate a clinical ultrasound normalized image; texture features of the clinical ultrasound normalized image are analyzed by using a gray level co-occurrence matrix calculation method to obtain clinical ultrasound texture features, including contrast, uniformity, correlation, energy and entropy; the boundary contour of the tissue in the clinical ultrasound normalized image is separated by using a Sobel operator to mark the boundary contour of the tissue in the clinical image, and a region growing algorithm is used to further refine the boundary contour of the tissue in the clinical image to generate a clinical ultrasound image tissue boundary; the tissue edge features of the clinical ultrasound image tissue boundary are analyzed to obtain clinical ultrasound tissue edge features, including shape, size and area; the clinical ultrasound normalized image is segmented based on the fuzzy tissue boundary based on the clinical ultrasound texture features and the clinical ultrasound tissue edge features to generate a clinical ultrasound image lesion area block.

[0038] As an embodiment of the present application, referring to Figure 3 , it is a function flow diagram of the lesion area segmentation module in the Figure 1 , in the embodiment, the lesion area segmentation module comprises the following functions: S21: pixel normalization is performed on the clinical ultrasound equalization image to generate a clinical ultrasound normalized image; In the embodiment of the present application, the pixel normalization is performed on the clinical ultrasound equalization image, and a clinical ultrasound equalization kidney image with a size of 512*512 is taken as an example, the pixel value range in the image is between 0-255, the minimum-maximum normalization method is used, and the formula is , wherein is the normalized pixel value, is the original pixel value, is the minimum value of the pixel value in the image, is the maximum value of the pixel value in the image. Each pixel point in the image is traversed, if the image =10, =200, for a pixel point with an original value =50, the normalized value is = 50 10 / 200 10≈0.21, by calculating all 512x512=262144 pixel points in the image in this way, the pixel value is uniformly mapped to the interval of 0-1, finally the clinical ultrasound normalized image is generated, which eliminates the problem of inconsistent pixel value scale caused by factors such as device difference between different images, and is convenient for subsequent processing.

[0039] S22: the tissue texture feature analysis is performed on the clinical ultrasound normalized image by using a gray level co-occurrence matrix calculation method, so as to obtain the clinical ultrasound tissue texture features, including contrast, uniformity, correlation, energy and entropy; In the embodiment of the application, the tissue texture feature analysis is performed on the clinical ultrasound normalized image by using the gray level co-occurrence matrix calculation method, taking the clinical ultrasound normalized liver image as an example, the gray level is set to 16 levels (the normalized pixel value interval of 0-1 is equally divided into 16 parts), the direction is selected as 0°, 45°, 90° and 135°, the distance is set as 1 pixel, for each direction and distance, a 16x16 gray level co-occurrence matrix is constructed , wherein , and represent the gray level, represents the number of pixel pairs starting from the pixel point of the gray level and reaching the pixel point of the gray level in the specified direction and distance, for example, in the 0° direction, there are 10 pixel pairs starting from the pixel point of the gray level 3 and reaching the pixel point of the gray level 5 with a distance of 1 pixel, then the value in the 0° direction is 10, and the contrast is calculated as: The contrast reflects the intensity of the gray level change in the image, the greater the value, the rougher the texture, and the uniformity: The uniformity measures the uniformity of the gray level distribution, the greater the value, the more uniform the texture, and the correlation: , wherein , are the mean values of the GLCM in the row and column directions, , are the standard deviations of the GLCM in the row and column directions, the correlation reflects the linear correlation of the gray level distribution in the image, and the energy: The energy represents the concentration of the elements in the gray level co-occurrence matrix, the greater the value, the smoother the image, and the entropy: The entropy reflects the randomness of the gray level distribution in the image, the greater the value, the more complex the texture, by calculating the characteristic values in the four directions and taking the average value, the clinical ultrasound tissue texture features are finally obtained, which provides the texture basis for lesion analysis.

[0040] S23: The Sobel operator is used to separate the boundary contours of tissues in the normalized clinical ultrasound image to mark the boundary contours of tissues in the clinical image. The region growing algorithm is then used to further refine the boundary contours of tissues in the clinical image to generate the tissue boundaries of the clinical ultrasound image. In this embodiment of the invention, the Sobel operator is used to separate the tissue boundaries within normalized clinical ultrasound images. Taking normalized clinical ultrasound images of the thyroid gland as an example, the Sobel operator... and The templates in the directions are as follows: and Iterate through every pixel in the image, and for each pixel's 3×3 neighborhood, compare it with... and The template is convolved to obtain the value of the pixel. and gradient value in direction and Then through the formula Calculate the gradient magnitude by The gradient direction is calculated, and a threshold of 0.3 is set (an empirical value that can be adjusted according to the actual image). Pixels with a gradient amplitude greater than 0.3 are marked as boundary points to initially define the boundary contours of tissues within the clinical image. The initially defined boundary contours are further refined using a region growing algorithm. One boundary point is selected as a seed point, and the growth criterion is set as follows: pixels with a gray level difference of less than 0.1 (an empirical value) from the seed point and within an 8-neighborhood are selected as new growth points. Starting from the seed point, pixels that meet the growth criteria are continuously included in the region until no pixels meet the criteria. By performing this growth operation on all seed points, a more accurate clinical ultrasound image tissue boundary is finally generated, clearly outlining the tissue contour.

[0041] S24: Perform tissue edge feature analysis on the tissue boundary of clinical ultrasound images to obtain the tissue edge features of clinical ultrasound, including shape, size and area; In this embodiment of the invention, tissue edge feature analysis is performed on the tissue boundary of clinical ultrasound images. Taking the boundary of a liver tumor region in a clinical ultrasound image as an example, the coordinate sequence of the boundary points is calculated, and a polygon approximation algorithm (such as the Ramer-Douglas-Peucker algorithm) is used to simplify the boundary contour, obtaining a rough morphological description of the boundary and determining whether it is circular, elliptical, or irregular in shape. The number of pixels in the area enclosed by the boundary is calculated. Assuming that the boundary of the tumor region contains 1000 pixels, and that each pixel represents an actual area of ​​0.01 mm², this method is used. 2According to the resolution and physical size calibration of the ultrasound image, the area of the tumor region is 1000*0.01=10mm 2 At the same time, the size of the tumor is estimated by the number and shape features of the boundary points, so that the clinical ultrasound tissue edge features including the shape, size and area are obtained, and a quantitative basis is provided for the lesion diagnosis.

[0042] S25: Based on the clinical ultrasound tissue texture features and the clinical ultrasound tissue edge features, the lesion region segmentation of the clinical ultrasound normalized image based on the fuzzy tissue boundary is performed to generate the clinical ultrasound image lesion region block.

[0043] In the embodiment of the present application, based on the clinical ultrasound tissue texture features and the clinical ultrasound tissue edge features, the lesion region segmentation of the clinical ultrasound normalized image based on the fuzzy tissue boundary is performed, and the fuzzy membership function is established based on the clinical ultrasound normalized breast image. For each pixel point in the image, the fuzzy membership of the pixel point belonging to the lesion region is calculated by combining the tissue texture features (such as contrast, entropy, etc.) and the edge features (such as the distance from the known lesion edge) of the region where the pixel point is located. Assuming that the influence weight of the contrast in the texture features on the lesion is 0.4, the influence weight of the entropy is 0.3, and the influence weight of the distance from the known lesion edge in the edge features is 0.3, for a certain pixel point, the contrast value is 0.6 (after normalization), the entropy value is 0.5, and the distance from the known lesion edge is normalized to 0.2, then the membership U=0.4*0.6+0.3*0.5+0.3*(1 0.2)=0.24+0.15+0.24=0.63, and the membership threshold is set to 0.5. The pixel points with a membership greater than 0.5 are divided into the lesion region. Through the calculation and division of all pixel points, the clinical ultrasound image lesion region block is finally generated, the accurate segmentation of the lesion region is realized, and the clinical diagnosis is assisted.

[0044] Further, the lesion region segmentation of the clinical ultrasound normalized image based on the fuzzy tissue boundary based on the clinical ultrasound tissue texture features and the clinical ultrasound tissue edge features comprises: Based on the clinical ultrasound tissue texture features and the clinical ultrasound tissue edge features, a corresponding clinical image lesion segmentation network is constructed by combining U-Net, so as to encapsulate the clinical ultrasound tissue texture features and the clinical ultrasound tissue edge features into a 3x3 network layer, and utilize a ReLU activation function to down-convolve, and utilize a 2x2 pooling layer to down-pool to a bottleneck layer, so as to perform 2x2 up-convolution by adding an attention module to the bottleneck layer and perform 1x1 convolution output, thereby outputting the initial clinical tissue lesion region. In the embodiment of the present application, by constructing a corresponding clinical image lesion segmentation network based on clinical ultrasound tissue texture features and clinical ultrasound tissue edge features and combining U-Net, taking a clinical ultrasound breast image as an example, the clinical ultrasound tissue texture features (such as contrast, uniformity, correlation, energy and entropy) and the clinical ultrasound tissue edge features (such as shape, size and area) are packaged into a 3x3 network layer. In this process, each feature is regarded as a channel. Assuming that there are 5 texture features and 3 edge features, an 8-channel 3x3 network layer is formed, and ReLU activation function is used for down convolution. The formula of ReLU activation function is For each neuron in the network layer, the input value is processed by the ReLU function. If the input value is less than 0, the output is 0, and if the input value is greater than 0, the output remains unchanged. For example, if the input value of a neuron is -2, the output after ReLU function is 0; if the input value is 3, the output is 3. Feature extraction is performed through multiple 3x3 convolution layers. After each convolution, the number of channels is doubled. For example, after the first convolution, the number of channels becomes 16. At the same time, 2x2 pooling layer is used for down-pooling to the bottleneck layer. The purpose of the pooling operation is to reduce the amount of data and calculation. In the 2x2 pooling layer, the maximum value in the 2x2 neighborhood is taken as the output value. For example, for a 2x2 neighborhood , the output value after pooling is 7. With the increase of the number of pooling layers, the spatial dimension of the data gradually decreases, while the number of channels gradually increases. Finally, the bottleneck layer is reached. The attention module is added to the bottleneck layer for 2x2 up-convolution and 1x1 convolution output. The role of the attention module is to make the network pay more attention to important feature information. At the same time, through the attention mechanism, the importance weight of each channel is calculated, and then the features are weighted and summed according to the weight. For example, for a 16-channel feature map, the attention module calculates the weight of each channel , and then the feature map is weighted and summed to obtain a new feature map. Then 2x2 up-convolution is performed. Up-convolution is the inverse operation of pooling. Through inverse convolution, the spatial dimension of the data is restored, and the number of channels is halved. Finally, 1x1 convolution is performed for output. The role of 1x1 convolution is to adjust the number of channels. Finally, the initial clinical tissue lesion area is output.

[0045] Preferably, the clinical ultrasound normalized image is input into the clinical image lesion segmentation network for lesion area segmentation to generate a clinical image initial lesion area segmentation block. In this embodiment of the invention, a normalized clinical ultrasound image is input into a clinical image lesion segmentation network for lesion region segmentation. Taking a normalized clinical ultrasound image of thyroid gland with a size of 512×512 as an example, the image first passes through the network's input layer, and then sequentially through multiple 3×3 convolutional layers, 2×2 pooling layers, bottleneck layers, 2×2 upper convolutional layers, and 1×1 convolutional layers constructed earlier. During this process, the image features are continuously extracted and fused. For example, in the first 3×3 convolutional layer, the image and the convolution kernel perform convolution operations to extract preliminary features. As the network deepens, the features gradually become more abstract and advanced. In the pooling layer, the spatial dimension of the image continuously decreases, and after adding an attention module in the bottleneck layer, the network can more accurately focus on the features of the lesion region. Finally, after processing by the network, the initial lesion region segmentation block of the clinical image is output. This segmentation block is a binary image with the same size as the input image, in which the lesion region is marked as 1 and the non-lesion region is marked as 0. For example, in the output segmentation block, the pixel value of the thyroid lesion region is 1, while the pixel value of the normal tissue region is 0.

[0046] Preferably, the initial lesion region segmentation block of the clinical image is subjected to lesion refinement correction processing based on the blurring of tissue boundaries to generate a lesion region block of the clinical ultrasound image.

[0047] In this embodiment of the invention, the initial lesion region segmentation block of clinical images is subjected to lesion refinement correction processing based on tissue boundary fuzziness. Taking a liver lesion region in the initial lesion region segmentation block of clinical images as an example, the fuzzy membership degree of each pixel in the segmentation block is first calculated. The fuzzy C-means clustering algorithm is used, assuming that the lesion region is divided into 3 categories (normal tissue, lesion tissue edge, and lesion core), and the initial value of the cluster center is set. The distance from each pixel to the cluster center is calculated iteratively. (in It is the first The first pixel 1 eigenvalue, It is the first The first cluster center (each eigenvalue), and update the membership matrix based on the distance. (in is a fuzzy index, usually taking 2), after multiple iterations, when the membership matrix no longer changes, the membership of each pixel point belonging to different categories is obtained, for the lesion area, the pixel points with the membership greater than 0.5 are retained as the lesion area, and the pixel points with the membership less than 0.5 are adjusted, for example, for a pixel point, the membership of the pixel point belonging to the lesion tissue edge is 0.4, and the membership of the pixel point belonging to the normal tissue is 0.6, then the pixel point is adjusted to be the normal tissue, through the processing, the initial lesion area segmentation block of the clinical image is refined and corrected, and finally the clinical ultrasound image lesion area block is generated, so that the boundary of the lesion area is more accurate, the division of the lesion area is more fine, and a more reliable basis is provided for clinical diagnosis.

[0048] Further, the lesion refinement correction processing based on tissue boundary fuzziness on the initial lesion area segmentation block of the clinical image comprises: extracting a tissue edge from the initial lesion area segmentation block of the clinical image to generate a clinical image lesion tissue boundary edge line; In the embodiment of the application, the initial lesion area segmentation block of the clinical image is extracted from the initial lesion area segmentation block of the clinical image, and a 512*512 pixel lung clinical image initial lesion area segmentation block is taken as an example, a Canny edge detection algorithm is used, the algorithm first performs Gaussian filtering denoising on the segmentation block, sets the standard deviation of the Gaussian filter to 1.5, the filter size to 5*5, and the Gaussian filtering formula as: , wherein is the center of the filter, is the standard deviation, the noise interference in the segmentation block is reduced through convolution operation, then the image gradient amplitude and direction are calculated, the Sobel operator is used to calculate the gradient in the horizontal and vertical directions, the gradient amplitude formula is: , and the gradient direction formula is: Then non-maximum suppression is performed, each pixel point is traversed, if the gradient amplitude of the pixel point is not the local maximum value in its gradient direction, the amplitude is set to 0, finally the double threshold (the low threshold is 30 and the high threshold is 90) is set to connect the edges, the pixel points with the gradient amplitude greater than the high threshold are determined as edge points, the pixel points with the gradient amplitude less than the low threshold are excluded, and the pixel points with the gradient amplitude between the two thresholds are retained as edge points if they are connected with the determined edge points, otherwise they are excluded, through the above operations, the clinical image lesion tissue boundary edge line is finally generated, and the approximate outline of the lesion area is clearly outlined.

[0049] Preferably, the gray value of the corresponding edge region in the clinical image lesion tissue boundary edge line is subjected to boundary fuzzification processing to obtain a lesion tissue boundary edge gray fuzzy line region. In the embodiment of the present application, the gray value of the corresponding edge region in the edge line of the clinical image lesion tissue boundary is processed by boundary blurring, taking the edge region with a width of 5 pixels in the edge line as an example, a Gaussian blurring method is adopted, for each pixel point in the edge region The blurred gray value is calculated by the following formula: , wherein is the original gray value, is the blurred gray value, is 2 (i.e., a 5x5 Gaussian kernel is used), is a Gaussian kernel function, and the calculation method is the same as the Gaussian filtering formula in the previous step, for example, the original gray value of a certain pixel point in the edge region is 150, and the blurred gray value is 130 after convolution calculation with the Gaussian kernel, and this operation is performed on all pixel points in the edge region, and finally the lesion tissue boundary edge gray blur line area is obtained, so that the transition between the lesion area and the normal tissue is more natural, and the real tissue boundary blur condition is simulated.

[0050] Preferably, the lesion region fuzzy segmentation block of the clinical image is generated by performing tissue boundary blur-based segmentation region extraction according to the lesion tissue boundary edge gray blur line area; In the embodiment of the present application, the lesion region fuzzy segmentation block of the clinical image is generated by performing tissue boundary blur-based segmentation region extraction according to the lesion tissue boundary edge gray blur line area, by setting the gray threshold range as [100, 150] (determined according to the gray distribution of the blur line area), traversing the clinical image, for the pixel points with a gray value in the threshold range, judging whether they are located in the connected region of the lesion tissue boundary edge gray blur line area, using a 4-connected region search algorithm, starting from a pixel point satisfying the threshold condition, searching the four adjacent pixel points above, below, left and right, if the adjacent pixel points also satisfy the threshold condition and have not been marked, adding them to the current connected region and continuing to search, until all the connected regions satisfying the condition are searched, and the connected regions are extracted, finally generating the lesion region fuzzy segmentation block of the clinical image, which is more consistent with the actual boundary of the lesion region than the initial segmentation block, and reflects the blur characteristics of the tissue boundary.

[0051] Preferably, the lesion morphology refinement correction is performed on the corresponding lesion region contour morphology in the lesion region fuzzy segmentation block of the clinical image by morphological operations corresponding to erosion, dilation or opening and closing operations, to generate the clinical ultrasound image lesion region block.

[0052] In the embodiment of the present application, the morphological operation corresponding to erosion and dilation is used to modify the lesion region contour shape in the fuzzy segmentation block of the clinical image lesion region. Taking a liver lesion region in the fuzzy segmentation block of the clinical image lesion region as an example, first, the erosion operation is performed, and a 3*3 structure element is used. For each pixel point in the segmentation block, if all the pixel points in the 3*3 neighborhood of the pixel point belong to the lesion region, the pixel point is retained, otherwise, the pixel point is removed, and the formula is: wherein is the fuzzy segmentation block of the lesion region, is the structure element, and the erosion operation can remove small protrusions on the boundary of the lesion region. Then, the dilation operation is performed, and a 3*3 structure element is also used. For each pixel point in the segmentation block, if any pixel point in the 3*3 neighborhood of the pixel point belongs to the lesion region, the pixel point is marked as the lesion region, and the formula is: The dilation operation can fill small depressions on the boundary of the lesion region, or the opening operation (erosion followed by dilation) is performed to remove small noise blocks outside the lesion region. Then, the closing operation (dilation followed by erosion) is performed to fill small cavities inside the lesion region. After the series of morphological operations, the lesion region contour shape is modified, and finally, the clinical ultrasound image lesion region block is generated, so that the segmentation result of the lesion region is more accurate, and a more accurate basis is provided for clinical diagnosis.

[0053] Further, the fuzzy segmentation region extraction based on the tissue boundary based on the fuzzy line region of the lesion tissue boundary edge gray scale includes: A 3*3 corresponding gray scale fuzzy sliding window is set, and fuzzy C-means and standard deviation calculation is performed on each pixel point in the fuzzy line region of the lesion tissue boundary edge gray scale based on the gray scale fuzzy sliding window, so as to obtain the fuzzy C-means and fuzzy C-standard deviation corresponding to each pixel point in the fuzzy line region of the lesion tissue. In the embodiment of the present application, a 3*3 corresponding gray scale fuzzy sliding window is set in the processing of the fuzzy line region of the lesion tissue boundary edge gray scale. Taking the fuzzy line region of the lesion tissue boundary edge gray scale of a clinical ultrasound liver image as an example, the region contains a plurality of pixel points. For each pixel point in the line region, a 3*3 sliding window is constructed with the pixel point as the center. When the fuzzy C-means calculation is performed, the pixel points are divided into three categories (normal tissue category, lesion transition category, and lesion core category), denoted as , , The cluster centers are initialized as , , For example, according to the gray scale distribution of the line region, =80, =120, =160 As the initial clustering center, the distance of each pixel point in the sliding window is calculated with each clustering center is the fuzzy index, and =2, and the clustering center is updated by iteration until the clustering center is stable, as the fuzzy C-means corresponding to the pixel point, when calculating the fuzzy C standard deviation, for the cluster to which each pixel point belongs, according to the gray value of the pixel point in the cluster, the formula is the mean of the gray value of the pixel point in the cluster, is the number of pixel points in the cluster, and the fuzzy C standard deviation corresponding to the pixel point is obtained, and the calculation of the fuzzy C mean and the fuzzy C standard deviation corresponding to each pixel point is completed by traversing all the pixel points in the gray fuzzy line area of the lesion tissue boundary edge.

[0054] Preferably, the tissue fuzzy coefficient corresponding to each pixel point in the lesion tissue fuzzy line area is obtained by quantifying the tissue fuzzy coefficient according to the fuzzy C mean and the fuzzy C standard deviation corresponding to each pixel point in the lesion tissue fuzzy line area. In the embodiment of the present application, by quantifying the tissue fuzzy coefficient according to the fuzzy C mean and the fuzzy C standard deviation corresponding to each pixel point in the lesion tissue fuzzy line area, the formula and are weight coefficients, respectively =0.6, =0.4, the neighborhood mean is the mean of the gray value of the pixel point in the 3*3 sliding window centered on the pixel point, and the maximum standard deviation is the maximum value of the fuzzy C standard deviation of all pixel points in the lesion tissue fuzzy line area. For example, for a certain pixel point in the lesion tissue fuzzy line area, the fuzzy C mean is 130, the neighborhood mean is 120, the fuzzy C standard deviation is 15, and the maximum standard deviation in the line area is 20, then the tissue fuzzy coefficient F corresponding to the pixel point is 0.6*|130-120| / 120+0.4*15 / 20=0.05+0.3=0.35, by calculating all the pixel points in the lesion tissue fuzzy line area, the tissue fuzzy coefficient corresponding to each pixel point is finally obtained, which quantifies the fuzzy characteristic degree of the pixel point in the lesion tissue fuzzy line area.

[0055] Preferably, the fuzzy membership block expansion segmentation is performed between each pixel point in the gray fuzzy line area of the lesion tissue boundary edge based on the tissue fuzzy coefficient corresponding to each pixel point in the lesion tissue fuzzy line area, to generate a fuzzy segmentation block of the lesion area in the clinical image.

[0056] ​​​​​In the embodiment of the present application, by means of the tissue blur coefficient corresponding to each pixel point in the blur line area of the lesion tissue, the fuzzy membership block expansion segmentation is carried out between each pixel point in the gray blur line area of the lesion tissue boundary edge, the fuzzy membership threshold is set to 0.2, for any one pixel point P in the gray blur line area of the lesion tissue boundary edge, the expansion is carried out to the 8-neighborhood pixel point of P, if the difference between the tissue blur coefficient of the neighborhood pixel point Q and the tissue blur coefficient of P is less than the threshold, and Q is not divided into other block areas, then Q and P are classified into the same fuzzy membership block area, for example, the tissue blur coefficient of pixel point P is 0.3, the tissue blur coefficient of the neighborhood pixel point Q of P is 0.28, the difference between the two is 0.3 0.28=0.02<0.2, then Q is included in the fuzzy membership block area where P is located, a new starting point is randomly selected from the pixel points in the line area which are not divided, the above expansion process is repeated until all the pixel points in the line area are divided into corresponding fuzzy membership block areas, the fuzzy membership block areas are extracted, and finally the fuzzy segmentation block of the clinical image lesion area is generated, which more accurately reflects the fuzzy boundary between the lesion area and the normal tissue, and provides more accurate results for the accurate segmentation of the clinical ultrasound image lesion area.

[0057] Further, the image lesion annotation module comprises the following functions: The position of the lesion area block in the clinical ultrasound image is obtained; In the embodiment of the present application, by means of a clinical ultrasound liver image with a size of 512*512 pixels and its corresponding lesion area block, the position of the lesion area block is obtained by recording the coordinate range of the lesion area block in the clinical ultrasound image, assuming that the coordinates of the left upper corner pixel point of the lesion area block in the clinical ultrasound image are (x1, y1), and the coordinates of the right lower corner pixel point are (x2, y2), wherein x1=100, y1=150, x2=300, and y2=350, in units of pixels, the specific position range of the lesion area block in the image is determined, that is, the area from the horizontal coordinate 100 to 300 and the vertical coordinate 150 to 350, which provides accurate position information for the further analysis and annotation of the lesion area. At the same time, the coordinate information is recorded in the corresponding metadata file, which is convenient for subsequent retrieval and use.

[0058] Preferably, the clustering algorithm is used to classify the lesion area of the clinical ultrasound image lesion area block to obtain the lesion area type. In the embodiment of the present application, the DBSCAN (density peak clustering) algorithm is used to classify the lesion area block of the clinical ultrasound breast image, first, two key parameters are set: neighborhood radius = 5 pixels (representing a circular neighborhood centered at a certain pixel), and the minimum number of samples MinPts = 10 (representing the number of sample points contained in the neighborhood radius = 5 pixels (representing a circular neighborhood centered at a certain pixel), and the minimum number of samples MinPts = 10 (representing the number of sample points contained in the neighborhood radius , wherein and are the coordinates of the pixel points and , respectively, if the number of sample points contained in the neighborhood of a certain pixel within a radius is greater than or equal to MinPts, the pixel point is a core point; if a certain pixel point is in the neighborhood of a core point, the pixel point is a boundary point; other pixel points are noise points, and by continuously scanning the pixel points in the lesion area block, the core points and the density-reachable points are divided into a cluster. For example, in processing a lesion area block containing 2000 pixel points, after calculation and judgment, 3 different clusters are formed, according to the pre-set corresponding relationship between the lesion area type and the cluster characteristics, such as the pixel gray scale characteristics of cluster 1 conforming to the characteristics of cyst, which is classified as a cyst type; cluster 2 conforms to the characteristics of tumor, which is classified as a tumor type; the remaining noise points are not classified, and finally the type of the lesion area is obtained, providing a reference basis for clinical diagnosis.

[0059] Preferably, the lesion degree of the clinical ultrasound image lesion area block is evaluated to obtain the severity of the lesion area. In the embodiment of the present application, the lesion degree of the clinical ultrasound thyroid image lesion area block is evaluated to comprehensively consider the texture features, shape features and elasticity features of the lesion area, the texture features are calculated by using a gray level co-occurrence matrix (GLCM) to calculate parameters such as contrast, entropy, energy and correlation, for example, for a lesion area block, the contrast is 80, the entropy is 3.2, the energy is 0.1, and the correlation is 0.7, the shape features are calculated by calculating the perimeter C and the area A of the lesion area, and then the shape factor F = 4πA / C 2 is obtained, assuming that the perimeter C of the lesion area block is 100 pixels and the area A is 600 pixels 2 , then the shape factor F = 4π×600 / 100 2 ≈0.75, the elasticity features are obtained by ultrasonic elastography to obtain the elastic modulus E = 15 kPa (kilopascal) of the lesion tissue, and by establishing a lesion degree evaluation formula S = α × (0.3 × contrast + 0.2 × entropy + 0.1 × energy + 0.1 × correlation) + β × (1 F) + γ x E, where a = 0.3, β = 0.3, γ = 0.4, the parameters obtained by the above calculation are substituted into the formula, S = 0.3 x (0.3 x 80 + 0.2 x 3.2 + 0.1 x 0.1 + 0.1 x 0.7) + 0.3 x (1 0.75) + 0.4 x 15 = 0.3 x (24 + 0.64 + 0.01 + 0.07) + 0.3 x 0.25 + 6 = 0.3 x 24.72 + 0.075 + 6 = 7.416 + 0.075 + 6 = 13.491, the lesion degree is set to be graded, when S < 3, it is mild lesion, when 3 < S < 6, it is moderate lesion, and when S < 6, it is severe lesion, so the lesion area is judged to be severe lesion.

[0060] Preferably, the corresponding clinical ultrasound image is image lesion labeled based on the lesion area position, the lesion area type and the lesion area severity, to generate a clinical ultrasound lesion labeled image set.

[0061] In the embodiment of the present application, the clinical ultrasound kidney image is image lesion labeled based on the lesion area position, the lesion area type and the lesion area severity obtained in the previous steps, to label the lesion area position by using a rectangular frame, draw a rectangular frame on the image with the upper left corner coordinates (x1, y1) and the lower right corner coordinates (x2, y2) of the lesion area block, add a text label above the rectangular frame to clearly label the lesion area type, such as "tumor", and label the lesion area severity below the rectangular frame, such as "severe", for a clinical ultrasound image containing multiple lesion area blocks, each lesion area is labeled in turn according to the above method, and the labeled image is saved as a new image file, and the lesion area position, type and severity information are recorded in the corresponding XML file, which contains image file name, lesion area coordinates, type, severity and other fields, by performing such labeling operation on a series of clinical ultrasound images, a clinical ultrasound lesion labeled image set is finally generated, which provides standardized and structured data for subsequent clinical diagnosis, medical research and deep learning model training.

[0062] Further, the lesion degree evaluation of the clinical ultrasound image lesion area block comprises: performing lesion texture local binary analysis on the clinical ultrasound image lesion area block to obtain a clinical lesion area texture local binary pattern; In the embodiment of the present application, the local binary texture analysis is performed on the clinical ultrasound image lesion region block, taking a clinical ultrasound breast image lesion region block with a size of 256*256 pixels as an example. The basic local binary pattern (LBP) algorithm is adopted. For each pixel point in the lesion region block, a 3*3 neighborhood is taken with the pixel point as the center. The gray value of the center pixel point is taken as the threshold. For the 8 pixel points in the neighborhood, if the gray value is greater than or equal to the gray value of the center pixel point, the pixel point is assigned a value of 1, otherwise, the pixel point is assigned a value of 0. For example, the gray value of the center pixel point is 120, the gray value of the upper left pixel point in the neighborhood is 130, and the pixel point is assigned a value of 1. The gray value of the upper right pixel point in the neighborhood is 110, and the pixel point is assigned a value of 0. In the clockwise direction, the 8 binary values are sequentially arranged to form an 8-bit binary number, which is converted into a decimal number as the local binary pattern value of the center pixel point. All the pixel points in the lesion region block are traversed to obtain the local binary pattern value corresponding to each pixel point, thereby generating the local binary texture pattern of the clinical lesion region, which reflects the local gray level change characteristics of the lesion region texture.

[0063] Preferably, the local texture complexity of the clinical image lesion region is evaluated based on the local binary texture pattern of the clinical lesion region. In the embodiment of the present application, the local texture complexity of the clinical ultrasound image lesion region block is evaluated based on the local binary texture pattern of the clinical lesion region. The uniformity and contrast of the local binary pattern are used to measure the texture complexity. First, the uniformity of the local binary pattern is calculated. For each local binary pattern value, it is converted into a cyclic binary number, and the conversion times of 0 to 1 and 1 to 0 in the binary bit are calculated. If the conversion times are less than or equal to 2, the local binary pattern is a uniform pattern. Otherwise, it is a non-uniform pattern. The number of pixel points of the uniform pattern and the non-uniform pattern in the lesion region block is counted. Let the number of pixel points of the uniform pattern be , and the total number of pixel points be , then the uniformity ratio is . The contrast of the local binary pattern is calculated. For each local binary pattern value , the contrast is , where is the gray value of the neighborhood pixel point, and is the gray value of the center pixel point. The average value of the contrast of all the pixel points in the lesion region block is calculated as . The uniformity ratio and the average value of the contrast are combined to obtain the local texture complexity of the clinical image lesion region , where =0.6, =0.4, The maximum value in the contrast of all pixel points is taken, for example, if =0.7, =20, =50, then =0.6×(1 0.7)+0.4×20 / 50=0.18+0.16=0.34, which quantifies the local texture complexity of the lesion area.

[0064] Preferably, the corresponding lesion tissue elasticity of the clinical ultrasound image lesion area block is obtained by the clinical ultrasound image lesion area block, and the lesion hardness of the clinical ultrasound image lesion area block is estimated based on the lesion tissue elasticity, to obtain the tissue hardness of the clinical image lesion area; In the embodiment of the present application, the corresponding lesion tissue elasticity of the clinical ultrasound image lesion area block is obtained by the clinical ultrasound image lesion area block, and the lesion hardness of the clinical ultrasound image lesion area block is estimated based on the lesion tissue elasticity, to obtain the tissue hardness of the clinical image lesion area; In the embodiment of the present application, the corresponding lesion tissue elasticity of the clinical ultrasound image lesion area block is obtained by the clinical ultrasound image lesion area block, and the lesion hardness of the clinical ultrasound image lesion area block is estimated based on the lesion tissue elasticity, to obtain the tissue hardness of the clinical image lesion area; wherein is the length change of the tissue after being pressed, is the original length of the tissue, the strain value of the lesion area at different positions is analyzed by an image processing algorithm, the elasticity distribution of the lesion tissue is obtained, and the lesion hardness of the clinical ultrasound image lesion area block is estimated based on the lesion tissue elasticity, to establish a mapping relationship between the elasticity and the hardness, for example, an empirical formula H=a× +b is obtained through a large amount of clinical data statistics, wherein H is the hardness of the lesion tissue, a= 10, and b=15 (determined according to actual statistical data), the elasticity of each pixel point in the lesion area block is calculated, so that the hardness distribution of the clinical image lesion area is obtained, for example, the elasticity of a certain pixel point is 0.1, and its hardness H= 10×0.1+15=14, which quantifies the hardness of the lesion area tissue.

[0065] Preferably, the lesion severity is evaluated according to the local texture complexity of the clinical image lesion area and the hardness of the clinical image lesion area, to obtain the severity of the lesion area.

[0066] In the embodiment of the present application, the lesion severity is evaluated according to the local texture complexity of the clinical image lesion area and the hardness of the clinical image lesion area, to establish a lesion severity evaluation model, and a weighted summation method is adopted, and the formula is wherein is the severity of the lesion area, and is a weight coefficient, is set by = 0.4, = 0.6, for example, the local texture complexity of a certain clinical ultrasound image lesion region block is TC = 0.4, the tissue hardness is H = 12, then the lesion region severity S = 0.4 * 0.4 + 0.6 * 12 = 0.16 + 7.2 = 7.36, and the lesion degree grading standard is set, when S < 3, it is a mild lesion, when 3 ≤ S < 6, it is a moderate lesion, and when S ≥ 6, it is a severe lesion, and according to the calculated lesion region severity value, the severity of the lesion region is judged to be a severe lesion.

[0067] Further, the lesion auxiliary screening module comprises the following functions: The input layer is used to receive the labeled clinical ultrasound image; the feature extraction layer adopts a convolutional neural network branch with different scales of 3x3, 5x5 and 7x7 to extract features of the input clinical ultrasound image, so as to capture image features at different scales, and the attention mechanism is introduced to enhance the attention to the lesion region; the feature fusion layer fuses the image features at different scales to generate a corresponding clinical lesion feature representation graph by using the way of skip connection and feature splicing; and the output layer outputs the corresponding clinical image lesion screening result. In the embodiment of the present application, a deep learning model architecture based on multi-scale feature fusion is constructed by combining a convolutional neural network design, taking a 512x512 pixel labeled clinical ultrasound liver image as an example, the input layer receives the image data and converts it into a tensor form suitable for model processing, i.e. the dimension is (512, 512, 1) (assuming it is a grayscale image, if it is a color image, the number of channels is 3), the feature extraction layer adopts three convolutional neural network branches with different scales, the 3x3 convolutional neural network branch contains 5 convolutional layers, the number of channels of each convolutional layer is 32, 64, 128, 256, 512 in turn, the step length is 1, and the padding mode is "same", so as to ensure that the output feature map size is unchanged; the 5x5 convolutional neural network branch contains 4 convolutional layers, and the number of channels is 64, 128, 256, 512 respectively; the 7x7 convolutional neural network branch contains 3 convolutional layers, and the number of channels is 128, 256, 512, and a ReLU activation function is connected after each convolutional layer, the formula is f(x) = max(0, x), and a spatial attention mechanism is introduced after the last convolutional layer of each branch to calculate the attention weight matrix M, the formula is wherein is a feature map, and are average pooling and maximum pooling operations respectively, is a Sigmoid function, by multiplying the attention weight matrix with the feature map, the attention to the lesion area is enhanced, the feature fusion layer adopts the way of jump connection and feature splicing, the output feature maps of the three branches of 3x3, 5x5 and 7x7 are spliced after adjusting the size (using bilinear interpolation) to make them consistent in size, a feature map with a dimension of (H, W, 512+512+512)=(H, W, 1536) is obtained, then added to the input layer through jump connection, and finally the clinical lesion feature representation map is generated, and the output layer adopts 1x1 convolution to compress the channel number to the class number (assuming that there are 5 types of lesions, plus the normal class, a total of 6 classes), and then classified through the Softmax function, the Softmax formula is , the output corresponding clinical image lesion screening result, that is, the probability distribution of the first pixel belonging to the first class, so as to output the corresponding clinical image lesion screening result.

[0068] Preferably, the clinical ultrasound lesion annotation image set is divided into a training set, a validation set and a test set according to a division ratio of 7:2:1, and the deep learning model architecture based on multi-scale feature fusion is trained by using the training set, in the training process, the cross-entropy loss is used as the optimization target, the random gradient descent algorithm and its variant Adam are used for parameter updating, random flipping, rotation and scaling are used for data enhancement to increase the diversity of training data, the early stopping mechanism is introduced to stop training when the loss on the validation set no longer decreases to prevent overfitting, the trained model is evaluated by using the test set, the accuracy, recall rate and F1 value are used to verify the performance of the model, so as to construct an image lesion auxiliary screening model. In the embodiment of the application, by assuming that there are 1000 clinical ultrasound lesion annotation images, the training set (700), the validation set (200) and the test set (100) are divided according to the division ratio of 7:2:1, and the deep learning model architecture based on multi-scale feature fusion is trained by using the training set, in the training process, the cross-entropy loss is used as the optimization target, for the multi-classification problem, the cross-entropy loss function formula is , wherein is the number of samples, is the number of classes, is the true label (0 or 1) of the sample belonging to the class , is the probability that the sample is predicted to belong to the class , the parameter is updated by using the variant Adam of the random gradient descent algorithm, in the Adam algorithm, the learning rate is initialized to 0.001, =0.9, =0.999, In each iteration, the model parameters are updated based on the calculated gradient, as shown in the formula: ,in For the current parameter, For learning rate, and These are first-order moment estimation and second-order moment estimation, respectively. Data augmentation methods corresponding to random flipping, rotation, and scaling are used to increase the diversity of the training data. For example, the probability of randomly flipping the image horizontally is set to 0.5; the rotation angle range is set to... The scaling factor is between 15° and 15°; the scaling ratio is between 0.8 and 1.2. An early stopping mechanism is introduced: training stops when the loss on the validation set no longer decreases after 10 consecutive epochs to prevent overfitting. The trained model is evaluated using a test set, and its performance is verified using accuracy (Precision = TP / TP + FP), recall (Recall = TP / TP + FN), and F1 score (F1 = (2 × Precision × Recall) / Precision + Recall), where TP represents true positives, FP represents false positives, and FN represents false negatives, thus constructing an image-assisted lesion screening model.

[0069] Preferably, clinical ultrasound images of the subjects to be screened are acquired; In this embodiment of the invention, clinical ultrasound images to be screened are obtained from the ultrasound examination department database of a hospital. These images are stored in DICOM format and contain metadata such as the patient's basic information, examination time, and equipment parameters. For example, a thyroid ultrasound image of a patient is obtained, with an image size of 384×384 pixels, a bit depth of 12 bits, and a storage format of DICOM 3.0. The image data is converted from DICOM format to a format that the model can process, that is, normalized to a tensor form in the [0,1] interval, while retaining the original resolution and spatial information of the image, in preparation for subsequent lesion-assisted screening.

[0070] Preferably, the clinical ultrasound images to be screened are input into the imaging lesion-assisted screening model for lesion-assisted screening, so as to output the location, type and severity of the lesion corresponding to the lesion on the clinical ultrasound image.

[0071] In the embodiment of the present application, by inputting the clinical ultrasound image to be screened into the trained image lesion auxiliary screening model, taking the input thyroid ultrasound image as an example, the image first enters the input layer of the model, passes through the feature extraction layer, different scale convolutional neural network branches extract the details, medium scale and large scale features of the image respectively, and the attention mechanism is used to enhance the attention to the possible lesion area, in the feature fusion layer, the features of different scales are spliced and fused to generate a clinical lesion feature representation map, the output layer outputs the probability distribution of each pixel point belonging to different lesion types and normal categories according to the feature representation map, and sets the probability threshold to 0.5, if the probability of a pixel point belonging to a certain category is greater than 0.5, it is determined that the pixel point belongs to the category, by analyzing the classification results of all pixel points, using the connected region marking algorithm, the position of the lesion area is determined, the coordinates of the bounding rectangle of the lesion area are calculated, and the specific position of the lesion on the image is obtained; the categories of the pixel points in the lesion area are counted, and the lesion type is determined; according to the pre-set relationship between the lesion severity and the category, the lesion severity is judged, and finally the lesion position, lesion type and lesion severity corresponding to the lesion existing on the clinical ultrasound image are output.

[0072] Therefore, from any viewpoint, the embodiments should be considered as being exemplary and non-limiting, the scope of the present application being defined by the appended claims and not by the above description, and it is intended to encompass all variations falling within the meaning and the scope of the equivalent elements of the claims.

[0073] The above description is merely one specific implementation of the application, which enables a person skilled in the art to understand or implement the application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the application. Therefore, the present application will not be limited to these embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A clinical ultrasound image-assisted screening system based on deep learning, characterized in that, Includes the following modules: The ultrasound image preprocessing module is used to acquire clinical ultrasound images and perform grayscale processing on the clinical ultrasound images to generate clinical grayscale images; and to perform neighborhood grayscale equalization processing on the clinical grayscale images to generate clinical ultrasound equalization images. The lesion region segmentation module is used to normalize the pixels of the clinical ultrasound equalization image to generate a clinical ultrasound normalized image. It obtains the corresponding clinical ultrasound tissue texture features and clinical ultrasound tissue edge features from the clinical ultrasound normalized image, and performs lesion region segmentation based on the blurred tissue boundary of the clinical ultrasound normalized image to generate a clinical ultrasound image lesion region block. The image lesion annotation module is used to obtain the location, type and severity of the corresponding lesion area from the lesion area block of clinical ultrasound image, and to annotate the corresponding clinical ultrasound image based on the location, type and severity of the lesion area to generate a set of annotated clinical ultrasound images. The lesion-assisted screening module is used to construct an image-assisted lesion screening model based on a set of clinical ultrasound lesion-annotated images and combined with convolutional neural network training; it acquires the clinical ultrasound images to be screened and inputs them into the image-assisted lesion screening model to perform lesion-assisted screening, so as to output the location, type and severity of the lesion corresponding to the lesion on the clinical ultrasound image.

2. The deep learning-based clinical ultrasound image-assisted screening system according to claim 1, characterized in that, The ultrasound image preprocessing module includes the following functions: Acquiring clinical ultrasound images; Pixel blur is calculated for each pixel in the clinical ultrasound image to obtain the pixel blur of the clinical ultrasound image. Based on the pixel blur of clinical ultrasound images, each pixel in the clinical ultrasound image is blurred and denoised to obtain a blurred and denoised clinical image. The blurred and denoised clinical images are converted to grayscale to generate grayscale clinical images. Neighborhood grayscale equalization processing is performed on clinical grayscale images to generate clinical ultrasound equalized images.

3. The deep learning-based clinical ultrasound image-assisted screening system according to claim 2, characterized in that, The neighborhood grayscale equalization processing based on clinical grayscale images includes: The clinical grayscale image is divided into various clinical grayscale image blocks according to certain pixel division intervals of 3x3 or 9x9. Gray-level histograms are calculated for each clinical gray-level image block to obtain the corresponding gray-level histogram for each gray-level image block. By searching for pixels with large gray-level deviations from other pixels in the gray-level histograms corresponding to each gray-level image block, and performing histogram equalization on the pixel based on the gray-level values ​​of the neighboring pixels in the corresponding gray-level histogram, each gray-level equalized image block is generated. Bilinear interpolation was used to reconstruct clinical images from each grayscale equalized image block to generate clinical ultrasound equalized images.

4. The deep learning-based clinical ultrasound image-assisted screening system according to claim 1, characterized in that, The lesion area segmentation module includes the following functions: Pixel normalization is performed on equalized clinical ultrasound images to generate normalized clinical ultrasound images. The gray-level co-occurrence matrix calculation method was used to analyze the tissue texture features of normalized clinical ultrasound images to obtain clinical ultrasound tissue texture features, including contrast, uniformity, correlation, energy and entropy. The Sobel operator is used to separate the tissue boundaries in the normalized clinical ultrasound image to mark the boundary contours of the tissues in the clinical image. The region growing algorithm is then used to further refine the boundary contours of the tissues in the clinical image to generate the tissue boundaries of the clinical ultrasound image. Tissue edge feature analysis was performed on the tissue boundaries of clinical ultrasound images to obtain the clinical ultrasound tissue edge features, including shape, size and area; Based on the tissue texture features and edge features of clinical ultrasound, lesion regions are segmented from normalized clinical ultrasound images with blurred tissue boundaries to generate lesion region blocks in clinical ultrasound images.

5. The deep learning-based clinical ultrasound image-assisted screening system according to claim 4, characterized in that, The segmentation of lesion regions based on blurred tissue boundaries in normalized clinical ultrasound images, based on clinical ultrasound tissue texture features and clinical ultrasound tissue edge features, includes: Based on clinical ultrasound tissue texture features and clinical ultrasound tissue edge features, and combined with U-Net, a corresponding clinical imaging lesion segmentation network is constructed. The clinical ultrasound tissue texture features and clinical ultrasound tissue edge features are encapsulated into a 3x3 network layer, and the ReLU activation function is used for downward convolution. At the same time, a 2x2 pooling layer is used for downward pooling to the bottleneck layer. An attention module is added to the bottleneck layer to perform 2x2 upward convolution and 1x1 convolution output, thereby outputting the initial clinical tissue lesion region. Normalized clinical ultrasound images are input into a clinical imaging lesion segmentation network to segment lesion regions, thereby generating initial lesion region segmentation blocks in clinical images. The initial lesion region segmentation block of clinical imaging is subjected to lesion refinement correction based on blurred tissue boundaries to generate lesion region blocks of clinical ultrasound imaging.

6. The deep learning-based clinical ultrasound image-assisted screening system according to claim 5, characterized in that, The process of refining and correcting the initial lesion region segmentation blocks in clinical images based on blurred tissue boundaries includes: Tissue edges are extracted from the initial lesion area segmentation blocks in clinical imaging to generate the boundary lines of lesions in clinical imaging tissue. The gray values ​​of the corresponding edge regions within the boundary line of the lesion tissue in clinical imaging are blurred to obtain the gray blurred line region of the lesion tissue boundary. Based on the gray-scale blurred line area of ​​the lesion tissue boundary, a segmentation region based on tissue boundary blur is extracted to generate blurred segmentation blocks of the lesion area in clinical imaging. The morphological operations corresponding to erosion, dilation, or opening and closing operations are used to refine and correct the contour of the corresponding lesion region within the blurred segmentation block of the lesion region in clinical imaging, so as to generate the lesion region block of clinical ultrasound imaging.

7. The deep learning-based clinical ultrasound image-assisted screening system according to claim 6, characterized in that, The segmentation region extraction based on the blurred gray-scale line area at the boundary of the lesion tissue includes: By setting a 3x3 grayscale blur sliding window, and calculating the mean and standard deviation of the blur C for each pixel in the grayscale blur line area of ​​the lesion tissue boundary based on the grayscale blur sliding window, the mean and standard deviation of the blur C for each pixel in the blur line area of ​​the lesion tissue can be obtained. The tissue blur coefficient is quantified by the mean and standard deviation of blur C corresponding to each pixel in the blurred line area of ​​the lesion tissue, and the tissue blur coefficient corresponding to each pixel in the blurred line area of ​​the lesion tissue is obtained. Based on the tissue blur coefficient corresponding to each pixel in the blurred line area of ​​the lesion tissue, a blurred membership block expansion segmentation is performed between each pixel in the gray-scale blurred line area of ​​the lesion tissue boundary to generate a blurred segmentation block of the lesion area in clinical imaging.

8. The deep learning-based clinical ultrasound image-assisted screening system according to claim 1, characterized in that, The image lesion annotation module includes the following functions: The location of the corresponding lesion area in the clinical ultrasound image is obtained by using the lesion area block in the clinical ultrasound image. Clustering algorithms were used to classify lesion regions in clinical ultrasound images to obtain lesion types. The severity of lesions in clinical ultrasound imaging areas is assessed to determine the extent of the lesion. Based on the location, type, and severity of the lesion, corresponding clinical ultrasound images are labeled to generate a set of labeled clinical ultrasound images.

9. The deep learning-based clinical ultrasound image-assisted screening system according to claim 8, characterized in that, The assessment of the severity of lesions in clinical ultrasound imaging regions includes: Local binary analysis of lesion texture was performed on the lesion area blocks in clinical ultrasound images to obtain the local binary pattern of texture in the clinical lesion area; Local texture complexity of clinical ultrasound imaging lesion regions is evaluated based on the local binary pattern of texture in clinical lesion regions, and the local texture complexity of clinical imaging lesion regions is obtained. The elasticity of the lesion tissue corresponding to the clinically measured lesion area is obtained by obtaining the elasticity of the lesion tissue in the clinical ultrasound imaging lesion area, and the lesion hardness is estimated based on the elasticity of the lesion tissue in the clinical ultrasound imaging lesion area to obtain the tissue hardness of the lesion area in the clinical imaging. The severity of a lesion is assessed by evaluating the complexity of the local texture and the tissue stiffness of the lesion area in clinical imaging.

10. The deep learning-based clinical ultrasound image-assisted screening system according to claim 1, characterized in that, The lesion-assisted screening module includes the following functions: A deep learning model architecture based on multi-scale feature fusion was designed and constructed by combining convolutional neural networks. This architecture includes an input layer, a feature extraction layer, a feature fusion layer, and an output layer. The input layer receives labeled clinical ultrasound images. The feature extraction layer uses convolutional neural network branches of different scales (3x3, 5x5, and 7x7) to extract features from the input clinical ultrasound images, capturing image features at different scales. An attention mechanism is also introduced to enhance focus on lesion areas. The feature fusion layer fuses image features from different scales, using skip connections and feature concatenation to generate corresponding clinical lesion feature representation maps. The output layer outputs the corresponding clinical image lesion screening results. The clinical ultrasound lesion annotated image set was divided into training, validation, and test sets according to a 7:2:1 ratio. The deep learning model architecture based on multi-scale feature fusion was trained using the training set. During training, cross-entropy loss was used as the optimization objective, and stochastic gradient descent and its variant Adam were used for parameter updates. Data augmentation with random flipping, rotation, and scaling was used to increase the diversity of training data. An early stopping mechanism was introduced to stop training when the loss on the validation set no longer decreased to prevent overfitting. The trained model was evaluated using the test set, and the performance of the model was verified by accuracy, recall, and F1 score, thus constructing an image lesion auxiliary screening model. Obtain clinical ultrasound images of the patient to be screened; The clinical ultrasound images to be screened are input into the imaging lesion-assisted screening model to perform lesion-assisted screening, so as to output the location, type and severity of the lesion corresponding to the lesion on the clinical ultrasound image.

Images