An intelligent classification and detection method, electronic device and storage medium for hepatic echinococcosis ultrasound images based on deep learning

Abstract

The present invention provides an intelligent classification and detection method, an electronic device and a storage medium for ultrasonic images of hepatic echinococcosis based on deep learning, comprising the following steps: collecting data; performing denoising processing by an adaptive overlapping block denoising model; establishing a classification data set and training an adaptive multi-scale attention residual classification model; target detection annotation; and training a detection model Fractal-YOLO. The model of the present invention can more comprehensively understand and utilize the morphological features of the lesions, thereby improving the detection performance, ensuring the computational efficiency, greatly increasing the accuracy rate of echinococcosis and reducing the misdiagnosis rate.

Description

Technical Field

[0001] The present invention belongs to the field of ultrasonic image classification and detection, and in particular relates to an intelligent classification and detection method, an electronic device and a storage medium for ultrasonic images of hepatic echinococcosis based on deep learning. Background Art

[0002] Hepatic echinococcosis is a zoonotic parasitic disease caused by the larval echinococcus parasitizing in humans (or animals). Once the disease occurs, it will endanger the physical health of patients and cause great economic losses. The preferred site of hepatic echinococcosis is the liver, accounting for 80% of the total number of cases. Followed by other organs such as the lungs, and finally the whole body is involved. Hepatic echinococcosis can be pathologically divided into hepatic cystic echinococcosis and hepatic alveolar echinococcosis. Among them, hepatic cystic echinococcosis is more common, accounting for more than 98% of the cases of hepatic echinococcosis. The World Health Organization classifies hepatic echinococcosis into cystic echinococcosis and alveolar echinococcosis according to the ultrasonic image characteristics of hepatic echinococcosis. Among them, cystic echinococcosis is divided into 5 types, namely, unilocular type (CE-1), multilocular type (CE-2), collapsed internal capsule type (CE-3), necrotic solid type (CE-4) and calcified type (CE-5). Alveolar echinococcosis is divided into 3 types, namely, infiltrative type, calcified type and liquefied cavity type. The incidence of hepatic alveolar echinococcosis is lower than that of hepatic cystic echinococcosis, accounting for about 3% - 5% of hepatic echinococcosis.

[0003] Since the echinococcus grows slowly in the initial stage of liver infection, patients usually do not show obvious clinical symptoms. Only some will show allergic symptoms or occasional discomfort in the liver area. As the hepatic echinococcosis lesions gradually grow, they may compress the surrounding blood vessels or damage the bile ducts. At this time, patients will show obvious clinical symptoms such as abdominal discomfort, pain or jaundice. At present, the prevention and treatment of hepatic echinococcosis still face many difficulties. Clinically, clinical diagnosis, imaging diagnosis, immunology and serological examinations are generally used to diagnose hepatic echinococcosis. Among them, ultrasonic examination is the preferred method for the diagnosis and screening of hepatic echinococcosis. As a non-invasive examination method, it uses the physical imaging characteristics to realize the judgment of the physical characteristics, morphological structure and functional state of human soft tissues. However, conventional ultrasonic images need to be quantitatively analyzed, and it takes a lot of time and energy for ultrasonic doctors and oncology experts to initially complete the diagnosis of the disease. There are problems such as strong human subjectivity, poor accuracy of measurement results and poor reproducibility, often resulting in missed diagnosis or misdiagnosis of the disease. Clinically, doctors face a large number of ultrasonic images generated by physical examinations, and it is difficult to ensure that they have enough energy to find all hepatic echinococcosis lesions, especially when facing diseases with similar structures, it is difficult to effectively identify all cystic echinococcosis lesions. Summary of the Invention

[0004] In view of this, the present invention aims to propose an intelligent classification and detection method, an electronic device, and a storage medium for hepatic echinococcosis ultrasound images based on deep learning, so as to solve the challenges of low accuracy and misdiagnosis of traditional ultrasound image processing for echinococcosis.

[0005] To achieve the above object, the technical solution of the present invention is realized as follows:

[0006] An intelligent classification and detection method for hepatic echinococcosis ultrasound images based on deep learning, comprising the following steps:

[0007] S1. Collect hepatic ultrasound image data;

[0008] S2. Preprocess the hepatic ultrasound image data, and perform denoising processing on the preprocessed hepatic ultrasound image data based on an information entropy-guided adaptive overlapping block denoising model;

[0009] S3. Perform classification annotation, divide the pathological classification criteria, establish a classification data set, and label the images as cystic echinococcosis, alveolar echinococcosis, and other liver lesions;

[0010] S4. Based on the classification data set, train an adaptive multi-scale attention residual classification model;

[0011] S5. Perform target detection annotation on the cystic echinococcosis data, and establish a cystic echinococcosis detection data set;

[0012] S6. Perform target detection annotation on the alveolar echinococcosis data, and establish an alveolar echinococcosis detection data set;

[0013] S7. Train the detection model Fractal-YOLO based on the cystic echinococcosis data set and the alveolar echinococcosis detection data set respectively.

[0014] Further, in step S2, the denoising processing of the preprocessed hepatic ultrasound image data based on the information entropy-guided adaptive overlapping block denoising model includes:

[0015] S21. Load the input hepatic ultrasound image data;

[0016] S22. Calculate the local entropy value distribution;

[0017] S23. Divide the region types according to the threshold;

[0018] S24. Determine the region type and parameter settings for each reference block position, perform block matching denoising, and accumulate the results into the output image;

[0019] S25. Generate the final denoised result image.

[0020] Further, in step S22, calculating the local entropy value distribution includes:

[0021] Given the input image , first calculate the information entropy distribution of its local regions; for each position in the image, define a sliding window of size 7×7, and at each window position , calculate the information entropy within the window:

[0022] ;

[0023] ;

[0024] where represents the occurrence probability of the gray value within the window, obtained by calculating the gray histogram within the window, is the number of pixels with the gray value within the sliding window, is the total number of pixels in the window.

[0025] Further, in step S23, classifying the region types according to the threshold includes:

[0026] Based on the local entropy values calculated in step S22, establish an entropy value matrix with the same size as the original image , and by setting two thresholds, a high threshold and a low threshold , divide the image regions into three categories:

[0027] When > , it is a high-entropy region, and the high-entropy region contains important structural edges or texture details;

[0028] When < , it is a low-entropy region: the low-entropy region is a uniform tissue region or a noise interference region;

[0029] Medium-entropy region: a transitional region between the two;

[0030] Based on the statistical analysis of 8-bit grayscale images, the high-entropy threshold is selected to be 75% higher than the theoretical maximum entropy value, and the low-entropy threshold is set to 45% of the theoretical maximum entropy value;

[0031] For different types of regions, set different denoising parameters:

[0032] For high-entropy regions:

[0033] Use a search radius lower than the set value;

[0034] Adopt a similarity threshold with a coefficient value ranging from 0.01 to 0.1 ;

[0035] Select a block size smaller than the set value ;

[0036] For low-entropy regions:

[0037] Use a search radius larger than the set value ;

[0038] Adopt a similarity threshold with a coefficient value ranging from 0.1 to 0.3 ;

[0039] Select a block size larger than the set value .

[0040] Furthermore, in step S24, for each reference block position, determine the region type and parameter settings, perform block matching denoising, and accumulate the results into the output image, including:

[0041] Next, perform matching and denoising for each reference block position in the image :

[0042] Obtain the entropy value type of the reference block position , determine the corresponding parameter set , within the search window centered at with a radius , find the image block similar to the reference block; calculate the similarity using the normalized mean square error:

[0043] ;

[0044] wherein, is the target block to be denoised, is the comparison block within the search window, is the relative coordinate of the pixels within the block, is selected according to the region type or ;

[0045] When the similarity is less than the threshold , add the image block to the set of similar blocks , and perform weighted average calculation for all similar blocks:

[0046] ;

[0047] is the weight of the similar block; is the weight decay coefficient, which controls the rate of weight decrease with the similarity distance; thus, the module for removing noise is:

[0048] ;

[0049] Among them, represents a similar set.

[0050] Furthermore, in step S25, a final denoised result image is generated, including:

[0051] For each pixel position , collect the denoising block results covering the pixel, and calculate the weighted average to obtain the final pixel value:

[0052] ;

[0053] Among them, represents the set of all denoising blocks covering the pixel ; is the number of denoising blocks covering the pixel ; represents the th denoising block at the position pixel value.

[0054] Furthermore, in step S4, based on the classification dataset, an adaptive multi-scale attention residual classification model is trained, including:

[0055] Perform data augmentation on the image. For the augmented input image , the network first enters the improved Stage0 for feature preprocessing: Let After passing through the 7×7 convolution, batch normalization, and activation function of the main branch, the main branch feature is output. This process is denoted as:

[0056] ;

[0057] Among them, represents the learnable parameters of the 7×7 convolution kernel, the number of convolution kernels is 64, and the stride is 2; the auxiliary branch uses two consecutive 3×3 convolutions to replace the single large convolution. After each convolution, batch normalization and activation function are also performed, and the auxiliary branch feature is output, which is expressed as follows:

[0058] ;

[0059] ;

[0060] Among them, is the intermediate feature map in the auxiliary branch, and is the output feature map of the auxiliary branch; among them, and They are the parameters of two 3×3 convolution operations respectively, with the number of channels being 32 and the stride being 1; subsequently, and are concatenated in the channel dimension to obtain a fused feature:

[0061] ;

[0062] Then, a 3×3 max pooling is applied to with a pooling stride of 2, and the output feature is denoted as ; On an SE channel attention mechanism is introduced. Channel statistics information is extracted through global average pooling, and a two-layer fully connected layer plus a non-linear activation is used to screen the channel importance, and the generated channel weight vector is multiplied with channel by channel, so as to obtain an output containing attention-weighted information; It is expressed in mathematical form as:

[0063] ;

[0064] ;

[0065] ;

[0066] Among them, represents max pooling, represents global average pooling, represents element-wise multiplication, , are the parameters of the fully connected layers for dimension reduction and dimension increase respectively, is the Sigmoid activation function;

[0067] After the processing of Stage0, the feature map carries preliminary channel attention information;

[0068] The subsequent Stage1 contains three Bottleneck structures. Among them, the first Bottleneck is denoted as BTNK1-1. If the input feature is denoted as , then the dynamic convolution operation is represented by the following expression;

[0069] ;

[0070] Among them, are several groups of kernels learned in advance, calculates variable attention weights according to the input feature;

[0071] The latter two Bottlenecks are denoted as BTNK1-2, and the ChannelShuffle operation is introduced before and after the feature transformation respectively;

[0072] After the three bottlenecks in Stage 1 are completed, the output spatial resolution is the same as that of the previous stage, but the number of channels gradually increases;

[0073] ;

[0074] Denote the first bottleneck in Stage 2 to Stage 4 as BTNK2-1, BTNK3-1, and BTNK4-1 respectively. In BTNK2-1, BTNK3-1, and BTNK4-1, replace the downsampling operation of their 3×3 strided convolution with deformable convolution. For the output position , the sampling position of the deformable convolution kernel is jointly determined by the preset grid position and the learnable offset , so that the convolution calculation satisfies:

[0075] ;

[0076] Among them, represents the coordinate mapping relationship of downsampling;

[0077] Add a feature pyramid structure inside BTNK2-1, BTNK3-1, and BTNK4-1. During forward inference, dynamically crop some channels or branches for different inputs through a conditional calculation mechanism. The adaptive calculation is denoted as:

[0078] ;

[0079] Among them, is a computational volume metric estimated based on input features or network states, is the control strategy threshold. For BTNK2-2 in stage 2, it is further upgraded to a structure of 1×1 dimensionality reduction - 3×3 depthwise separable convolution - 1×1 dimensionality increase, and a channel attention mechanism is embedded in it;

[0080] Add a residual adaptive mechanism in BTNK4-1. Balance the fusion weight of the identity mapping and the residual transformation through a gating control parameter. The mathematical form of the fusion weight is expressed as:

[0081] ;

[0082] Among them, λ ∈ [0,1] is learned or dynamically determined by the input data, is the input feature, is the residual transformation function;

[0083] After each main branch convolution in Stage4, DropBlock is introduced as a regularization method, and the regularization method is denoted as:

[0084] ;

[0085] Among them, is element-wise multiplication, represents a square mask randomly generated on the feature map, represents the input feature map.

[0086] Furthermore, in step S7, the detection model Fractal-YOLO is trained based on the alveolar echinococcosis dataset and the cystic echinococcosis detection dataset respectively, including:

[0087] Introduce fractal dimension analysis to enhance the model's perception ability of lesion morphology and texture:

[0088] In the C2F-Backbone module of YOLOV8, for each feature map, in addition to performing conventional convolution operations, fractal dimension calculation is introduced; for the feature map F, calculate its local fractal dimension:

[0089] ;

[0090] Among them, represents the feature map position, is the local window size, uses a grid with a side length of to cover the minimum number of windows required for non-zero eigenvalues;

[0091] Adopt approximate calculation of discrete scales:

[0092] ;

[0093] Among them, is the number of sampling scales, is the th sampling scale, is the minimum number of windows required for a grid with a side length of to cover non-zero eigenvalues;

[0094] In the C2F-Backbone module, fractal feature enhancement is performed on the main branch and the short circuit branch respectively, where the main branch feature enhancement:

[0095] ;

[0096] Short circuit branch feature enhancement:

[0097] ;

[0098] Among them, is a non-linear mapping function, and are learnable parameters;

[0099] The final output of the C2F-Backbone module is:

[0100] ;

[0101] Among them, represents the final output feature of the C2F-Backbone module, represents the enhancement of the main branch feature, represents the enhancement of the short-circuit branch feature;

[0102] During the feature aggregation process of PANet, the fractal feature participates in the upsampling and downsampling processes:

[0103] Bottom-up path:

[0104] ;

[0105] Top-down path:

[0106] ;

[0107] Among them, represents the feature level index, represents the th layer of the original feature, represents the th layer of the original feature, represents the fractal feature of the th layer, represents the upsampling / downsampling operation, represents the convolutional layer;

[0108] Boundary fractal dimension calculation is introduced. By measuring the boundary perimeter at different scales , the boundary fractal dimension reflecting the irregularity of the boundary is obtained:

[0109] ;

[0110] Among them, is the measurement scale, is the boundary perimeter measured using the scale , is the boundary information reflecting the irregularity of the boundary. This boundary information is used to optimize the prediction of the bounding box. Through the learnable weight coefficient and the mapping function , the original predicted box is fine-tuned:

[0111] ;

[0112] Generate weight coefficients for different feature channels through global average pooling and fully connected layers to achieve adaptive fusion of features;

[0113] Finally, introduce the fractal feature loss, and the loss function is expressed as:

[0114] ;

[0115] ;

[0116] Among them, is the fractal loss weight, , is the predicted and target fractal dimensions, is the KL divergence, which measures the difference between the predicted and target fractal feature distributions, is the KL loss weight.

[0117] An electronic device includes a processor and a memory communicatively connected to the processor and used to store instructions executable by the processor. The memory stores instructions executable by the processor, and the instructions are executed by the processor. The processor is used to execute the above-mentioned intelligent classification and detection method for liver echinococcosis ultrasound images based on deep learning.

[0118] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the above-mentioned intelligent classification and detection method for liver echinococcosis ultrasound images based on deep learning.

[0119] Compared with the prior art, the above-mentioned intelligent classification and detection method for liver echinococcosis ultrasound images based on deep learning, electronic device and storage medium of the present invention have the following beneficial effects:

[0120] For the intelligent classification and detection method for liver echinococcosis ultrasound images based on deep learning, electronic device and storage medium of the present invention, the present invention can use an adaptive overlapping block denoising model to denoise the preprocessed liver ultrasound image data, then establish a classification data set, and based on the classification data set, train an adaptive multi-scale attention residual classification model; then through target detection annotation, establish a cystic echinococcosis detection data set and an alveolar echinococcosis detection data set respectively; finally, train the detection model Fractal-YOLO based on the cystic echinococcosis data set and the alveolar echinococcosis detection data set respectively, so that the model can more comprehensively understand and utilize the morphological features of the lesions, thereby improving the detection performance, ensuring the calculation efficiency, and at the same time greatly improving the accuracy of echinococcosis and reducing the misdiagnosis rate. Description of the Drawings

[0121] The accompanying drawings that form a part of the present invention are used to provide a further understanding of the present invention. The schematic embodiments and descriptions thereof of the present invention are used to explain the present invention and do not constitute an improper limitation to the present invention. In the drawings:

[0122] Figure 1 It is a schematic diagram of the overall method flow described in the embodiment of the present invention;

[0123] Figure 2 It is a schematic diagram of the denoising process of the preprocessed liver ultrasound image data by the adaptive overlapping block denoising model guided by information entropy described in the embodiment of the present invention;

[0124] Figure 3 It is a schematic diagram of the architecture of the adaptive multi-scale attention residual classification model described in the embodiment of the present invention;

[0125] Figure 4 It is a schematic diagram of the process of training the detection model Fractal-YOLO based on 2 detection data sets respectively described in the embodiment of the present invention. Detailed implementation manners

[0126] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0127] In the description of the present invention, it should be understood that the orientation or positional relationship indicated by the terms "center", "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. is the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus cannot be understood as a limitation to the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first", "second", etc. may explicitly or implicitly include one or more of such features. In the description of the present invention, unless otherwise specified, the meaning of "plurality" is two or more.

[0128] In the description of the present invention, it should be noted that unless otherwise clearly specified and defined, the terms "installation", "connection", and "coupling" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0129] The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0130] As Figures 1 to 4 shown, an intelligent classification and detection method for liver echinococcosis ultrasound images based on deep learning includes the following steps:

[0131] S1. Data collection: Collect high-quality liver ultrasound image data to ensure data representativeness and diversity;

[0132] S2. Preprocess the liver ultrasound image data, and perform denoising processing on the preprocessed liver ultrasound image data based on an information entropy-guided adaptive overlapping block denoising model;

[0133] S3. Perform classification annotation, divide clear pathological classification criteria, establish a classification data set, and label the images as cystic echinococcosis, alveolar echinococcosis, and other liver lesions;

[0134] S4. Based on the classification data set, train an adaptive multi-scale attention residual classification model;

[0135] S5. Perform object detection annotation on cystic echinococcosis data to establish a cystic echinococcosis detection data set;

[0136] S6. Perform object detection annotation on alveolar echinococcosis data to establish an alveolar echinococcosis detection data set;

[0137] S7. Train the detection model Fractal-YOLO based on the two detection data sets (cystic echinococcosis data set, alveolar echinococcosis detection data set) respectively.

[0138] Finally, the experimental results of this method:

[0139] Cystic echinococcosis: P = 97.6%, R = 98.3%, mAP50 = 94.6%, mAP50-95 = 92.7%; Alveolar echinococcosis: P = 98.7%, R = 99.3%, mAP50 = 95.6%, mAP50-95 = 94.6%, where P is precision, R is recall rate, mAP50 is the average precision calculated based on the IoU (Intersection over Union) threshold of 0.5, and mAP50-95 is the average value of the average precisions calculated when the IoU threshold ranges from 0.5 to 0.95 (usually with a step of 0.05).

[0140] In a preferred embodiment of the present invention, in step S2, the preprocessed liver ultrasound image data is denoised by an information entropy-guided adaptive overlapping block denoising model, including:

[0141] In liver ultrasound image processing, there are two key challenges: one is to retain important structural features required for diagnosis (such as the edge of the echinococcal cyst), and the other is to effectively suppress the speckle noise that affects the image quality. To solve this problem, the present invention proposes an information entropy-guided adaptive overlapping block denoising method. This method first identifies the key regions and noise regions in the image through information entropy analysis, and then performs differential denoising processing according to the entropy value distribution.

[0142] Algorithm flow:

[0143] 1. Load the input image;

[0144] 2. Calculate the local entropy value distribution;

[0145] 3. Divide the region types according to the threshold;

[0146] 4. For each reference block position:

[0147] Determine the region type and parameter settings;

[0148] Perform block matching denoising;

[0149] Accumulate the results to the output image;

[0150] 5. Generate the final denoised result image.

[0151] Specifically, given the input image , first calculate the information entropy distribution of its local region. For each position in the image, define a sliding window with a size of 7×7. At each window position , calculate the information entropy within this window:

[0152] ;

[0153] ;

[0154] Among them, represents the occurrence probability of the gray value within the window, which is obtained by calculating the gray histogram within the window. is the number of pixels with the gray value of within the sliding window, and is the total number of pixels in the window. The information entropy

[0155] reflects the complexity of the image information in this local area: the higher the entropy value, the more dispersed the gray distribution in this area, and it may contain more structural details; the lower the entropy value, the more concentrated the gray distribution in this area, which may be a relatively uniform tissue area or an area affected by noise. Based on the calculated local entropy values, an entropy value matrix with the same size as the original image is established . By setting two thresholds, a high threshold and , the image area is divided into three categories:

[0156] High-entropy area ( > ): It may contain important structural edges or texture details;

[0157] Low-entropy area ( < ): It may be a uniform tissue area or a noise interference area;

[0158] Medium-entropy area: The transitional area between the two.

[0159] Based on the statistical analysis of 8-bit grayscale images (256 levels), the high-entropy threshold is selected to be slightly higher than 75% of the theoretical maximum entropy value, and the low-entropy threshold is set to be about 45% of the theoretical maximum entropy value.

[0160] For different types of areas, different denoising parameters are set:

[0161] For high-entropy areas:

[0162] Use a smaller search radius ;

[0163] Adopt a stricter similarity threshold (between the coefficient values of 0.01 and 0.1, applicable to high-quality images, especially in areas with rich details and low noise, which can ensure the selection of very similar blocks) ;

[0164] Select a smaller block size .

[0165] For low-entropy areas:

[0166] Use a larger search radius ;

[0167] Adopt a looser similarity threshold (the coefficient value is usually between 0.1 and 0.3, used in situations with strong noise or relatively uniform regional backgrounds, to find more similar blocks by accepting larger errors and improve the denoising effect) ;

[0168] Select a larger block size .

[0169] Next, perform matching and denoising for each reference block position in the image :

[0170] Obtain the entropy value type at this position , determine the corresponding parameter set , within the search window centered at with a radius , search for image blocks similar to the reference block. And to prevent the similarity of blocks of different sizes from being directly comparable, use the normalized mean square error to calculate the similarity (Euclidean distance):

[0171] ;

[0172] is the target block to be denoised, is the comparison block within the search window, is the relative coordinate of the pixels within the block, is selected according to the region type or .

[0173] When the similarity is less than the threshold , add this block to the set of similar blocks , and perform weighted average calculation on all similar blocks:

[0174] ;

[0175] is the weight of the similar block; is the weight decay coefficient, controlling the rate of decline of the weight with the similarity distance; so the module for removing noise is:

[0176] ;

[0177] where represents the similar set. Due to the adoption of the overlapping block strategy, the same pixel position may be covered by multiple denoised blocks. For each pixel position , collect the denoising block results covering the pixel, calculate the weighted average to obtain the final pixel value:

[0178] ;

[0179] denotes the set of all denoising blocks covering the pixel ; is the number of denoising blocks covering the pixel ; denotes the pixel value of the th denoising block at the position ;

[0180] In a preferred embodiment of the present invention, in step S3, classification annotation is performed, a clear pathological classification standard is divided, a classification data set is established, and the image is labeled as cystic echinococcosis, alveolar echinococcosis, and other liver lesions.

[0181] In a preferred embodiment of the present invention, in step S4, an adaptive multi-scale attention residual classification model:

[0182] The input of the model is an RGB image with a shape of (3, 224, 224). To further enhance the robustness of the model, data augmentation is performed on the image, including random flipping, random cropping, and color jitter.

[0183] For the enhanced input image , the network first enters the improved Stage0 for feature preprocessing. This stage adopts a dual-branch parallel structure and obtains a feature representation with reduced spatial resolution and relatively high number of channels through feature fusion and max pooling in subsequent stages. Specifically, let pass through the 7×7 convolution, batch normalization, and activation function of the main branch and output the main branch feature , and this process is denoted as:

[0184] ;

[0185] where denotes the learnable parameters of the 7×7 convolution kernel, the number of convolution kernels is 64, and the stride is 2. The auxiliary branch uses two consecutive 3×3 convolutions to replace the single large convolution, and after each convolution, batch normalization and activation function are also connected, and the auxiliary branch feature is output, and the specific expression is as follows:

[0186] ;

[0187] ;

[0188] It is the intermediate feature map in the auxiliary branch, while is the output feature map of the auxiliary branch. Among them, and are the parameters of two 3×3 convolution operations respectively, with the number of channels being 32 and the stride being 1. Subsequently, and are concatenated in the channel dimension to obtain the fused feature:

[0189] ;

[0190] Then, a 3×3 max pooling is applied to for further downsampling, with the pooling stride being 2, and the output feature is denoted as . At this time, in order to improve the diversity and robustness of feature representation, an SE channel attention mechanism is also introduced on . The channel statistical information is extracted through global average pooling, and a two-layer fully connected layer plus a non-linear activation is used to screen the channel importance. The generated channel weight vector is multiplied with channel by channel, so as to obtain the output containing attention-weighted information. It is expressed in mathematical form as:

[0191] ;

[0192] ;

[0193] Among them, represents global average pooling, and ( , ) are the parameters of the fully connected layers for dimension reduction and dimension increase respectively, is the Sigmoid activation function. After the processing of Stage0, the shape of the feature map becomes (64, 56, 56), and it carries preliminary channel attention information.

[0194] The subsequent Stage1 contains three improved Bottleneck structures. The first Bottleneck (denoted as BTNK1-1) uses dynamic convolution to replace the standard convolution to adapt to the feature distribution of different positions or input samples. In the dynamic convolution branch, the convolution kernel is no longer a fixed single parameter, but different weight combinations are calculated according to the input, so as to dynamically generate effective convolution kernels. If the input feature is denoted as , then the dynamic convolution operation is represented by the following expression;

[0195] ;

[0196] Among them, are several groups of pre-learned kernels, Variable attention weights are calculated based on the input features. BTNK1-1 maintains the settings of stride S = 1 and output channel number C = C in the original ResNet50, but adds an additional spatial attention module in the main branch to perform attention weighting on the input feature map based on the spatial dimension, thereby highlighting key information. The subsequent two Bottlenecks (denoted as BTNK1-2) introduce the ChannelShuffle operation for channel rearrangement before and after feature transformation respectively, so as to enable effective information interaction between different channels in the case of grouped convolution or parallel branches. After the three Bottlenecks in Stage1 are completed, the output spatial resolution is the same as that of the previous stage, but the number of channels gradually increases to construct a more expressive feature map. 1 = 64, but an additional spatial attention module is added in the main branch to perform attention weighting on the input feature map based on the spatial dimension, thereby highlighting key information. The subsequent two Bottlenecks (denoted as BTNK1-2) introduce the ChannelShuffle operation for channel rearrangement before and after feature transformation respectively, so as to enable effective information interaction between different channels in the case of grouped convolution or parallel branches. After the three Bottlenecks in Stage1 are completed, the output spatial resolution is the same as that of the previous stage, but the number of channels gradually increases to construct a more expressive feature map.

[0197] ;

[0198] Denote the first Bottleneck in Stage2 to Stage4 as BTNK2-1, BTNK3-1, and BTNK4-1 respectively. In BTNK2-1, BTNK3-1, and BTNK4-1, replace the downsampling operation of their 3×3 stride convolution with deformable convolution. For the output position , the sampling position of the deformable convolution kernel is jointly determined by the preset grid position and the learnable offset , so that the convolution calculation satisfies:

[0199] ;

[0200] where corresponds to the downsampling coordinate mapping relationship with stride = 2, and stride = 2 is the 3×3 stride convolution.

[0201] Thus, higher flexibility is maintained when dealing with deformed objects or complex backgrounds. At the same time, a feature pyramid structure is added inside BTNK2-1, BTNK3-1, and BTNK4-1 to retain multi-scale information, and a conditional calculation mechanism is used to dynamically crop some channels or branches for different inputs during forward inference. Such adaptive calculation can be denoted as:

[0202] ;

[0203] where is a computational amount metric based on the input features or network state estimation, It is a control strategy threshold to dynamically reduce redundant calculations while ensuring accuracy. For BTNK2-2 in stage2, it is further upgraded to a structure of "1×1 dimensionality reduction - 3×3 depthwise separable convolution - 1×1 dimensionality increase", and a channel attention mechanism is embedded in it to highlight important channels or suppress redundant information during the process of scaling the channel dimension.

[0204] Inside the 4 Bottlenecks in Stage2, the output features of each layer are concatenated with the input of the subsequent layer in the channel dimension to improve the feature reuse rate, thereby effectively strengthening the gradient flow and information retention. For the 6 Bottlenecks after entering Stage3, in addition to retaining the idea of multi-scale feature fusion, group convolution is also widely used to reduce the computational amount. At this time, in order to ensure smooth information flow between different groups, an operation of channel rearrangement is inserted between the groups, so that group convolution will not overly cause channel isolation. For the 3 Bottlenecks in Stage4, information aggregation is further strengthened on a global scale. By modeling the similarity between any two positions through a non-local attention module, the output features can capture long-range dependencies. A residual adaptation mechanism is additionally added in BTNK4-1 in Stage4. By a gating control parameter, the fusion weight between the identity mapping and the residual transformation is balanced. The mathematical form of the fusion weight is expressed as:

[0205] ;

[0206] Among them, λ ∈ [0,1] is obtained by learning or dynamically determined by the input data, and can also be dynamically determined by the input data, so as to perform fine-grained control on the residual contribution degrees of different layers while maintaining the network stability. is the input feature, is the residual transformation function;

[0207] After each main branch convolution, DropBlock is introduced as a regularization method, and the regularization method is denoted as:

[0208] ;

[0209] Among them, is element-wise multiplication, represents a square mask randomly generated on the feature map, and its size and survival probability change with the training process, represents the input feature map.

[0210] At the overall structural level, the network also adopts a cross-layer feature aggregation mechanism. Through the skip connections or aggregation modules built at the end of each stage, the key information of the historical stages is passed backward step by step, reducing information loss.

[0211] At the output layer, usually, global average pooling is first performed on the final feature map to aggregate the spatial dimension into a single value. Subsequently, a linear classification layer is connected to map the output to the number of classes to achieve the final classification prediction.

[0212] In a preferred embodiment of the present invention, in step S5, target detection annotation is performed on the cystic echinococcosis data to establish a cystic echinococcosis detection dataset, and the categories are single-cyst type (CE-1), multi-daughter cyst type (CE-2), internal capsule collapse type (CE-3), necrosis solidification type (CE-4), and calcification type (CE-5).

[0213] In a preferred embodiment of the present invention, in step S6, target detection annotation is performed on the alveolar echinococcosis data to establish an alveolar echinococcosis detection dataset, and the categories are infiltration type, calcification type, and liquefaction cavity type.

[0214] In a preferred embodiment of the present invention, in step S7, the detection model Fractal-YOLO is trained based on 2 detection datasets respectively. The target detection model developed this time is based on the SOTA model yolov8. In order to be more adaptable to the hydatid liver dataset, fractal dimension analysis is introduced to enhance the model's perception ability of the lesion morphology and texture:

[0215] In the C2F-Backbone module of YOLOV8, in addition to performing conventional convolution operations on each feature map, fractal dimension calculation is introduced. For the feature map F, its local fractal dimension is calculated:

[0216] ;

[0217] where represents the position of the feature map, is the local window size, uses the minimum number of windows required to cover the non-zero eigenvalues with a grid of side length . To improve the calculation efficiency, approximate calculation with discrete scales is adopted:

[0218] ;

[0219] where is the number of sampling scales, is the th sampling scale. The fractal dimension map obtained in this way reflects the local complexity of each position of the feature map and provides a basis for subsequent feature enhancement.

[0220] In the C2F-Backbone module, fractal feature enhancement is performed on the main branch and the shortcut branch respectively. For the main branch feature enhancement:

[0221] ;

[0222] For the shortcut branch feature enhancement:

[0223] ;

[0224] Among them, is a non-linear mapping function, and are learnable parameters; the final output of the C2F-Backbone module is:

[0225] ;

[0226] Among them, represents the final output feature of the C2F-Backbone module, represents the main branch feature enhancement, represents the shortcut branch feature enhancement;

[0227] During the feature aggregation process of PANet, the fractal features participate in the upsampling and downsampling processes:

[0228] Bottom-up path:

[0229] ;

[0230] Top-down path:

[0231] ;

[0232] Among them, represents the feature level index, represents the th layer of the original feature, represents the th layer of the fractal feature, represents the upsampling / downsampling operation, represents the convolutional layer. This multi-scale feature fusion ensures the transmission and integration of morphological information at different scale levels, enabling the model to simultaneously perceive local details and global structures.

[0233] To improve the perception ability of the lesion boundary, a dedicated boundary fractal dimension calculation is introduced. By measuring the boundary perimeter at different scales , the boundary fractal dimension reflecting the irregularity degree of the boundary can be obtained:

[0234] ;

[0235] Among them, is the measurement scale, is the scale used The boundary perimeter obtained by measurement, is the boundary information reflecting the irregularity of the boundary. This boundary information is used to optimize the prediction of the bounding box through the learnable weight coefficient and the mapping function , and fine-tune the original predicted box:

[0236] ;

[0237] Through global average pooling (GAP) and fully connected layers (FC), weight coefficients of different feature channels are generated to achieve adaptive fusion of features. This mechanism enables the model to dynamically adjust the importance of different types of features according to the characteristics of the input image;

[0238] Finally, in the design of the loss function, in addition to the conventional bounding box regression loss , classification loss and object existence loss In addition, a fractal feature loss is introduced. This loss term includes the L2 distance between the prediction and the target fractal dimension, as well as the KL divergence between the prediction and the target fractal feature distribution, which helps the model better learn morphological features. The loss function is expressed as:

[0239] ;

[0240] ;

[0241] Among them, is the fractal loss weight, , is the prediction and the target fractal dimension, is the KL divergence, which measures the difference between the prediction and the target fractal feature distribution, is the KL loss weight.

[0242] Through this multi-level fractal feature enhancement mechanism, the model obtains the ability to perceive the morphology of the lesion at different levels: the bottom layer can capture local texture features, the middle layer extracts morphological structure information, and the high layer integrates global morphological features. This hierarchical design enables the model to more comprehensively understand and utilize the morphological features of the lesion, thereby improving the detection performance. At the same time, since the calculation of the fractal feature can be carried out simultaneously with the forward propagation of the backbone network and multiple optimization strategies are adopted, the entire scheme ensures the computational efficiency while maintaining the performance improvement.

[0243] The present invention can perform denoising processing on the preprocessed liver ultrasound image data by using an adaptive overlapping block denoising model, then establish a classification data set, and based on the classification data set, train an adaptive multi-scale attention residual classification model; and then through target detection annotation, respectively establish a cystic echinococcosis detection data set and an alveolar echinococcosis detection data set; finally, train the detection model Fractal-YOLO based on the cystic echinococcosis data set and the alveolar echinococcosis detection data set respectively, so that the model can more comprehensively understand and utilize the morphological characteristics of the lesions, thereby improving the detection performance, ensuring the calculation efficiency, and at the same time greatly improving the accuracy of echinococcosis and reducing the misdiagnosis rate.

[0244] The present invention also provides an electronic device, including a processor and a memory communicatively connected to the processor and used for storing instructions executable by the processor. The memory stores instructions executable by the processor, and the instructions are executed by the processor, and the processor is used to execute the above-mentioned intelligent classification and detection method for liver echinococcosis ultrasound images based on deep learning.

[0245] The present invention also provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, it implements the above-mentioned intelligent classification and detection method for liver echinococcosis ultrasound images based on deep learning.

[0246] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for intelligent classification and detection of hepatic echinococcosis ultrasound images based on deep learning, characterized in that: The following steps are involved: S1. Collect liver ultrasound imaging data; S2, preprocessing the liver ultrasound image data, and denoising the preprocessed liver ultrasound image data based on an adaptive overlapping block denoising model guided by information entropy; S3. Classify and annotate, divide the pathological classification standards, establish a classification data set, and annotate the images as cystic echinococcosis, alveolar echinococcosis, and other liver lesions; S4. Based on the classification dataset, train an adaptive multi-scale attention residual classification model; S5. Perform target detection and annotation on the cystic echinococcosis data to establish a cystic echinococcosis detection data set; S6. Perform target detection and annotation on alveolar echinococcosis data to establish an alveolar echinococcosis detection data set; S7, training the detection model Fractal-YOLO based on the cystic echinococcosis dataset and the alveolar echinococcosis detection dataset respectively; In step S2, the preprocessed liver ultrasound image data is denoised based on an adaptive overlapping block denoising model guided by information entropy, including: S21, loading and inputting liver ultrasound image data; S22, calculating the local entropy value distribution; S23, dividing the area types according to the threshold value; S24, determining the region type and parameter settings for each reference block position, performing block matching denoising, and accumulating the results into an output image; S25, generating a final denoising result image; In step S22, the local entropy value distribution is calculated, including: Given an input image I(x,y), first calculate the information entropy distribution of its local area; for each position in the image, define a sliding window of size 7×7, and at each window position (i,j), calculate the information entropy within the window: Among them, P k Represents the probability of occurrence of gray value k in the window, which is obtained by calculating the gray histogram in the window, N k is the number of pixels with grayscale value k in the sliding window, N total is the total number of pixels in the window; In step S23, the region types are divided according to the threshold, including: Based on the local entropy value calculated in step S22, an entropy matrix E(i, j) with the same size as the original image is established, and two high and low thresholds Th are set. high and Th low , the image regions are divided into three categories: When H(i,j)>Th high It is a high entropy area, which contains important structural edges or texture details; When H(i,j) <Th low Low entropy area: Low entropy area is a uniform tissue area or a noise interference area; Medium entropy region: the transitional region between the two; Based on the statistical analysis of 8-bit grayscale images, the high entropy threshold is selected to be higher than 75% of the theoretical maximum entropy value, and the low entropy threshold is set to 45% of the theoretical maximum entropy value; For different types of areas, set different denoising parameters: For high entropy regions: Use a search radius r below the set value small ; The similarity threshold δ is set to be between 0.01 and 0.

1. strict ; Choose a block size K that is smaller than the set value small ; For low entropy regions: Use a search radius r larger than the set value large ; The similarity threshold δ is set to be between 0.1 and 0.

3. loose ; Choose a block size K larger than the set value large ; In step S24, the region type and parameter settings are determined for each reference block position, block matching denoising is performed, and the results are accumulated into the output image, including: Next, matching and denoising are performed for each reference block position (x, y) in the image: Get the entropy value type E(x,y) of the reference block position, determine the corresponding parameter set (r,δ,K), and find the image block similar to the reference block in the search window with radius r and centered at (x,y); calculate the similarity using normalized mean square error: Among them, B x,y is the target block to be denoised, B u,v is the comparison block in the search window, m, n are the relative coordinates of the pixels in the block, and K is selected according to the region type. large or K small ; When the similarity is less than the threshold δ, the image block is added to the similar block set S, and the weighted average calculation is performed on all similar blocks: w u,v =exp(-d(B x,y ,B u,v ) / σ s 2 ); w u,v is the weight of similar blocks; σ s is the weight decay coefficient, which controls the speed at which the weight decreases with the similarity distance; so the noise removal module is: Among them, S represents a similar set; In step S25, a final denoising result image is generated, including: For each pixel position (x, y), collect the denoising block results covering the pixel and calculate the weighted average to get the final pixel value: Where C(x,y) represents the set of all denoising blocks covering pixel (x,y); N(x,y) is the number of denoising blocks covering pixel (x,y); B denoised,k (x,y) represents the pixel value of the kth denoising block at position (x,y).

2. The method for intelligent classification and detection of hepatic echinococcosis ultrasound images based on deep learning according to claim 1, characterized in that: In step S4, based on the classification data set, an adaptive multi-scale attention residual classification model is trained, including: Perform data enhancement on the image. For the enhanced input image X', the network first enters Stage 0 for feature preprocessing: let X' pass through the main branch's 7×7 convolution, batch normalization and activation function, and then output the main branch feature F m , and record this process as: F m =ReLU(BN(Conv 7×7 (X;W m ))); Among them, W m represents the learnable parameters of the 7×7 convolution kernel, the number of convolution kernels is 64, and the step size is 2; the auxiliary branch uses two consecutive 3×3 convolutions instead of a single large convolution, and the batch normalization and activation function are also performed after each convolution, and the auxiliary branch feature F is output a , expressed as follows: F a1 =ReLU(BN(Conv 3×3 (X′;W a1 ))); F a =ReLU(BN(Conv 3×3 (F a1 ;W a2 ))); Among them, F a1 is the intermediate feature map in the auxiliary branch, and F a is the output feature map of the auxiliary branch; where W a1 With W a2 are the parameters of two 3×3 convolution operations, with 32 channels and a stride of 1. m With F a The fusion features are obtained by splicing in the channel dimension: F0=Concat(F m ,F a ); Then apply 3×3 maximum pooling to F0 with a pooling step of 2, and the output feature is recorded as F'0; introduce the SE channel attention mechanism on F'0, extract channel statistical information through global average pooling, use a two-layer fully connected layer plus nonlinear activation to filter channel importance, and multiply the generated channel weight vector with F'0 channel by channel to obtain the output containing attention weighted information In mathematical form: F′0=MaxPool(F0); F se =σ(W2×ReLU(W1×GAP(F′0))); Among them, MaxPool represents maximum pooling, GAP represents global average pooling, ⊙ represents element-by-element multiplication, W1 and W2 are the fully connected layer parameters for reducing and increasing dimensions respectively, and σ is the Sigmoid activation function; after being processed by Stage0, the feature map carries preliminary channel attention information; The subsequent Stage 1 contains three Bottleneck structures, among which the first Bottleneck is denoted as BTNK1-1. If the input feature is denoted as F in , then the dynamic convolution operation is represented by the following expression; Among them, K j are several sets of kernels learned in advance, π j The variable attention weights are calculated based on the input features. The last two Bottlenecks are denoted as BTNK1-2, which introduce channel reorganization operations before and after feature transformation. After the three bottlenecks of Stage 1 are completed, the spatial resolution of the output is the same as that of the previous stage, but the number of channels is gradually increased; The first Bottleneck in Stage 2 to Stage 4 is denoted as BTNK2-1, BTNK3-1, and BTNK4-1 respectively. In BTNK2-1, BTNK3-1, and BTNK4-1, the downsampling operation of the 3×3 strided convolution is replaced by a deformable convolution. For the output position p, the sampling position of the deformable convolution kernel is changed from the preset grid position p to the output position p. k1 and the learnable offset Δp k1 Together, the convolution calculation satisfies: Among them, 2p represents the coordinate mapping relationship of downsampling; A feature pyramid structure is added inside BTNK2-1, BTNK3-1, and BTNK4-1. The conditional calculation mechanism is used to dynamically cut some channels or branches for different inputs during forward reasoning. The adaptive calculation is recorded as: Among them, cost is a computational indicator based on input features or network state estimation, τ is the control strategy threshold, and BTNK2-2 in stage2 is upgraded to a 1×1 dimension reduction-3×3 depthwise separable convolution-1×1 dimension increase structure, in which a channel attention mechanism is embedded; A residual adaptive mechanism is added in BTNK4-1, and a gate control parameter is used to balance the fusion weight of the identity mapping and the residual transformation. The mathematical form of the fusion weight is expressed as: F out =λF in +(1-λ)G(F in ); Among them, λ∈[0,1] is learned or dynamically determined by the input data, F in is the input feature, G is the residual transformation function; After each main branch convolution in Stage 4, DropBlock is introduced as a regularization method. The regularization method is recorded as: Y=X⊙M; Among them, ⊙ is the element-by-element multiplication, M represents a square mask randomly generated on the feature map, and X represents the input feature map.

3. The method for intelligent classification and detection of hepatic echinococcosis ultrasound images based on deep learning according to claim 1, characterized in that: In step S7, the detection model Fractal-YOLO is trained based on the cystic echinococcosis dataset and the alveolar echinococcosis detection dataset, respectively, including: Introducing fractal dimension analysis to enhance the model’s perception of lesion morphology and texture: In the C2F-Backbone module of YOLOV8, in addition to the conventional convolution operation, each feature map introduces fractal dimension calculation; for the feature map F, its local fractal dimension is calculated: Where (x, y) represents the feature map location, r is the local window size, and N(r, x, y) is the minimum number of windows required to cover non-zero eigenvalues ​​with a grid of side length r. Approximate calculation using discrete scale: Among them, k is the number of sampling scales, r i is the i-th sampling scale, N(r i ,x,y) is a side with length r i The minimum number of windows required for the grid to cover the non-zero eigenvalues; In the C2F-Backbone module, the fractal features of the main branch and the short-circuit branch are enhanced respectively, where the main branch features are enhanced: F main =F conv ·(1+α·φ(D main )); Short-circuit branch feature enhancements: F sh ortcut =F bypass ·(1+β·φ(D sh ortcut )); Among them, φ is a nonlinear mapping function, α and β are learnable parameters; The final C2F-Backbone module output is: F CSP =Concat[F main ,F sh ortcut ]; Among them, F CSP represents the final output feature of the C2F-Backbone module, F main Indicates that the main branch features are enhanced, F sh ortcut It indicates that the short-circuit branch characteristics are enhanced; In the feature aggregation process of PANet, fractal features participate in the up- and down-sampling process: Bottom-up path: Top-down path: Among them, l represents the feature level index, F l represents the original features of the lth layer, F l+1 represents the original features of the l+1th layer, represents the fractal features of the lth layer, Up / Down represents upsampling / downsampling operations, and Conv represents the convolutional layer; The calculation of boundary fractal dimension is introduced. By measuring the boundary perimeter P(s) at different scales s, the boundary fractal dimension reflecting the degree of boundary irregularity is obtained: Where s is the measurement scale, P(s) is the perimeter of the boundary measured using scale s, and D edge It reflects the degree of irregularity of the boundary. This boundary information is used to optimize the prediction of the bounding box. The original prediction box is fine-tuned through the learnable weight coefficient λ and mapping function g: B pred =B original +λ·g(D edge ); Through global average pooling and fully connected layers, weight coefficients of different feature channels are generated to achieve adaptive fusion of features; Finally, the fractal feature loss is introduced, and the loss function is expressed as: L fractal =|D pred -D tar get |2+γ·KL(P fractal ,Q fractal ); L total =L box +L cls +L df +λ fractal ·L fractal ; Among them, λ fractal is the fractal loss weight, D pred , D tar get is the predicted and target fractal dimension, KL is the KL divergence, which measures the difference between the predicted and target fractal feature distributions, and γ is the KL loss weight.

4. An electronic device, comprising a processor and a memory connected to the processor for storing instructions executable by the processor, characterized in that: The memory stores instructions that can be executed by the processor, and the instructions are executed by the processor. The processor is used to execute the deep learning-based intelligent classification and detection method for hepatic echinococcosis ultrasound images as described in any one of claims 1 to 3.

5. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for intelligent classification and detection of hepatic echinococcosis ultrasound images based on deep learning as described in any one of claims 1 to 3 is implemented.

Images