Target detection and classification method for ultrasonic image
By introducing multi-scale asymmetric convolution module and parallel depth separable convolution module in the YOLOv8 detection network, the problem of insufficient target detection accuracy and speed in ultrasonic images is solved, and higher detection accuracy and computing efficiency are achieved.
Patent Information
- Application Number
- CN202411876315.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-19
- Publication Date
- 2025-04-15
AI Technical Summary
The existing object detection methods have problems that detection accuracy and speed are difficult to meet actual needs when processing ultrasound images, especially in the case of complex background noise and blurred boundaries.
YOLOv8 is used as the basic detection network, and a multi-scale asymmetric convolution module is used in its bottleneck structure module to replace the standard convolution module, while parallel depth separable convolution and multi-pooling fusion module are introduced in the downsampling layer.
The multi-scale features of the target are captured by the multi-scale asymmetric convolution module, which improves the accuracy and robustness of detection; the parallel depth separable convolution and multi-pooling fusion module further improves the computing efficiency and feature extraction capabilities.
Smart Images
Figure CN119832301A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer vision, and in particular to an ultrasonic image-oriented target detection and classification method. Background Art
[0002] With the development of computer vision technology, object detection and classification have become one of the core technologies of many applications. Traditional object detection methods usually rely on manual feature extraction and classifier training, but the detection accuracy and speed in complex environments often cannot meet actual needs. In recent years, the development of deep learning technology has greatly promoted the progress of object detection and classification technology, especially the detection framework based on convolutional neural network (CNN) (such as YOLO series, Faster R-CNN, etc.) has become mainstream.
[0003] However, existing target detection methods still have certain limitations when processing certain types of images (such as ultrasound images). Due to their special imaging mechanism, ultrasound images may contain complex background noise and fuzzy boundaries, which puts higher requirements on target detection algorithms. In addition, existing methods also need to be further optimized in terms of computational efficiency and robustness. Summary of the invention
[0004] In order to solve the problems of the prior art, the present invention provides a target detection and classification method for ultrasound images, which improves the accuracy and speed of target detection and classification and reduces the demand for computing resources.
[0005] The technical solution of the present invention mainly includes the following steps:
[0006] Step 1: Normalize the input image to ensure that each pixel value is within a certain range, thereby improving the stability and consistency of subsequent processing.
[0007] Step 2: Use data augmentation methods such as rotating, scaling, and flipping the input image to increase the diversity and robustness of the training data.
[0008] Step 3: YOLOv8 is used as the basic detection network, which has the characteristics of strong real-time performance and fast detection speed.
[0009] Step 4: Replace the original standard convolution module with a multi-scale asymmetric convolution module in the bottleneck structure module of YOLOv8.
[0010] Step 5: Introduce parallel depthwise separable convolution and multi-pooling fusion modules into the downsampling layer of YOLOv8.
[0011] Step 6: Crop the detected target area and input it into the classification network ResNet for classification
[0012] Furthermore, the step 4 comprises:
[0013] In step 4-1, the multi-scale asymmetric convolution module uses multiple asymmetric convolution kernels of different sizes, each of which is composed of convolution kernels of different sizes in the vertical and horizontal directions. This design can more effectively capture the multi-scale features of the target.
[0014] In step 4-2, the multi-scale asymmetric convolution module uses standard convolution, and uses grouped convolution in the standard convolution layer, where the number of groups is determined by the greatest common divisor of the number of input channels and the number of output channels. This design can reduce the amount of calculation and improve calculation efficiency.
[0015] In step 4-3, a channel rearrangement layer is set after the standard convolutional layer to rearrange the channel order to enhance the interaction between features.
[0016] In step 4-4, all convolution results (including asymmetric convolution and standard convolution) are concatenated together, and batch normalization (BN) and activation function (SiLU) are applied to the concatenated feature maps to enhance feature expression capabilities.
[0017] In step 4-5, the feature map is compressed to the required number of channels through point-by-point convolution to further reduce the amount of calculation and improve the calculation efficiency.
[0018] Furthermore, the step 5 comprises:
[0019] Step 5-1, use depth-wise separable convolution to process the input feature map, where depth-wise separable convolution includes depth-wise convolution and point-wise convolution. Batch normalization (BN) and activation function (SiLU) are applied after depth-wise convolution to enhance feature expression capability.
[0020] In step 5-2, the input feature map is processed using maximum pooling and average pooling respectively.
[0021] Step 5-3, concatenate the results of depth-wise separable convolution, maximum pooling, and average pooling together.
[0022] In step 5-4, the number of channels of the concatenated feature map is adjusted through 1x1 convolution to match the input requirements of the subsequent layers. Batch normalization (BN) and activation function (SiLU) are also applied after 1x1 convolution to further enhance the feature expression ability and model stability.
[0023] The beneficial effects of the present invention are:
[0024] 1) By using a multi-scale asymmetric convolution module, the multi-scale features of the target are effectively captured, the accuracy and robustness of target detection are improved, and the computational efficiency is reduced;
[0025] 2) By using parallel depth-wise separable convolution and multi-pooling fusion modules, the computational efficiency and feature extraction capability of the model are further improved;
[0026] 3) By adopting a modular design concept, the multi-scale asymmetric convolution module can be easily integrated into the existing YOLOv8 detection framework, with good flexibility and scalability;
[0027] 4) By cascading an efficient classification network ResNet and classifying the detected target areas, the classification accuracy is further improved;
[0028] 5) By normalizing the input image, each pixel value is ensured to be within a certain range, thereby improving the stability and consistency of subsequent processing;
[0029] 6) By using image enhancement methods such as rotating, scaling, and flipping the input image, the diversity and robustness of the training data are increased. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0031] Figure 1 It is a schematic diagram of a flow chart provided in the embodiment of the present invention;
[0032] Figure 2 is an image normalization flow chart provided in the embodiment of the present invention;
[0033] Figure 3 It is a diagram of the improved YOLOv8 detection network structure provided in the embodiment of the present invention;
[0034] Figure 4 It is a schematic diagram of the structure of a multi-scale asymmetric convolution module provided in the embodiments of the present invention;
[0035] Figure 5 It is a schematic diagram of the structure of the asymmetric convolution provided in the embodiments of the present invention;
[0036] Figure 6 It is a structural diagram of the parallel depth-wise separable convolution and multi-pooling modules provided in the embodiments of the present invention. DETAILED DESCRIPTION
[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0038] The present invention provides a method for detecting and classifying targets in ultrasound images. A specific implementation is as follows:
[0039] Example 1: Ultrasound breast nodule detection and grading, the overall process is as follows Figure 1 shown.
[0040] Step 1: normalize the input ultrasound breast image, such as Figure 2 As shown, each pixel value is ensured to be between 0 and 255, thereby improving the stability and consistency of subsequent processing.
[0041] Step 2: Increase the diversity and robustness of the training data by using data augmentation methods such as rotation, scaling, and flipping on the input image. Specific data augmentation operations include but are not limited to random rotation (angle range [-10°, 10°]), random scaling (scaling ratio range [0.9, 1.1]), random horizontal flipping, etc.
[0042] Step 3: Use YOLOv8 as the basic detection network, such as Figure 3 As shown, it has the characteristics of strong real-time performance and fast detection speed.
[0043] Step 4: Replace the original standard convolution module with a multi-scale asymmetric convolution module in the bottleneck structure module of YOLOv8, such as Figure 4 and Figure 5 shown.
[0044] Step 5, introduce parallel depth-wise separable convolution and multi-pooling fusion modules into the downsampling layer of YOLOv8, as shown in Figure 6 shown.
[0045] Step 6: Based on the detection results, the detected target area is cropped and input into the classification network ResNet to classify breast nodules into grades 0-6 according to the BI-RADS standard.
[0046] In the target detection and classification method for ultrasound images described in this embodiment, step 4 includes:
[0047] Step 4-1, use multiple asymmetric convolution kernels of different sizes, each of which is composed of convolution kernels of different sizes in the vertical and horizontal directions. The specific implementation includes using convolution kernels of (1,3), (3,1), (1,5), (5,1), (1,7) and (7,1) shapes.
[0048] In step 4-2, group convolution is used in the standard convolution layer to reduce the amount of calculation and improve the calculation efficiency. The number of groups is determined by the greatest common divisor of the number of input channels and the number of output channels.
[0049] In step 4-3, a channel rearrangement layer is set after the standard convolutional layer to rearrange the channel order to enhance the interaction between features.
[0050] In step 4-4, all convolution results (including asymmetric convolution and standard convolution) are concatenated together, and batch normalization (BN) and activation function (SiLU) are applied to the concatenated feature maps to enhance feature expression capabilities.
[0051] In step 4-5, the feature map is compressed to the required number of channels through point-by-point convolution to further reduce the amount of calculation and improve the calculation efficiency.
[0052] Further, in one implementation, the specific process of step 5 is as follows: Figure 6 As shown, including:
[0053] Step 5-1, use depth-wise separable convolution to process the input feature map, where depth-wise separable convolution includes depth-wise convolution and point-wise convolution. Batch normalization (BN) and activation function (SiLU) are applied after depth-wise convolution to enhance feature expression capability.
[0054] In step 5-2, the input feature map is processed using maximum pooling and average pooling respectively.
[0055] Step 5-3, concatenate the results of depth-wise separable convolution, maximum pooling, and average pooling together.
[0056] In step 5-4, the number of channels of the concatenated feature map is adjusted through 1x1 convolution to match the input requirements of the subsequent layers. Batch normalization (BN) and activation function (SiLU) are also applied after 1x1 convolution to further enhance the feature expression ability and model stability.
[0057] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, the above is only a preferred embodiment of the present invention. Since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited to this. Any technician familiar with the technical field is within the technical scope disclosed by the present invention. For ordinary technicians in the technical field, changes or replacements that can be easily thought of should be covered within the protection scope of the present invention without departing from the principle of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims
1. A method for object detection and classification for ultrasound images, characterized in that: The following steps are involved: Step 1, normalize the input image; Step 2: Use image enhancement methods to increase the diversity and robustness of training data; Step 3, use YOLOv8 as the basic detection network; Step 4: Replace the original standard convolution module with a multi-scale asymmetric convolution module in the bottleneck structure module of YOLOv8. The specific steps are as follows: Step 4-1, using multiple asymmetric convolution kernels of different sizes, each asymmetric convolution kernel is composed of convolution kernels of different sizes in the vertical direction and the horizontal direction; Step 4-2, use grouped convolution in the standard convolution layer; Step 4-3, a channel rearrangement layer is set after the standard convolutional layer to rearrange the channel order to enhance the interaction between features; Step 4-4, concatenate all convolution results, including asymmetric convolution and standard convolution, and apply batch normalization and activation function on the concatenated feature map; Step 4-5, compress the feature map to the required number of channels through point-by-point convolution; Step 5: Introduce a parallel depthwise separable convolution and multi-pooling fusion module in the downsampling layer of YOLOv8. This module combines depthwise separable convolution, maximum pooling, and average pooling, and uses 1x1 convolution to adjust the number of output channels. Step 6: crop the detected target area and input it into the classification network ResNet for classification.
2. The method for object detection and classification for ultrasonic images according to claim 1, characterized in that: The normalization process in step 1) includes but is not limited to normalizing the pixel values of the image to the range of [0, 255].
3. The method for object detection and classification for ultrasonic images according to claim 1, characterized in that: The data enhancement in step 2) includes but is not limited to transformations of random rotation, random scaling, and random horizontal flipping.
4. The method for object detection and classification for ultrasonic images according to claim 1, characterized in that: The multiple asymmetric convolution kernel sizes of different sizes in step 4-1) include but are not limited to 3, 5 and 7.
5. The method for object detection and classification for ultrasonic images according to claim 1, characterized in that: In step 4-2), the number of groups of the grouped convolution is determined by the greatest common divisor of the number of input channels and the number of output channels.
6. The method for object detection and classification for ultrasonic images according to claim 1, characterized in that: The specific process of step 5) is as follows: Step 5-1, use depth-wise separable convolution to process the input feature map, and apply batch normalization and activation function after depth convolution to enhance feature expression ability; Step 5-2, use maximum pooling and average pooling to process the input feature map respectively; Step 5-3, concatenate the results of depthwise separable convolution, maximum pooling, and average pooling together; In step 5-4, the number of channels of the concatenated feature map is adjusted through 1x1 convolution to match the input requirements of the subsequent layers.
7. The method for ultrasonic image-oriented object detection and classification according to claim 6, characterized in that: Step 5-1) The depth-wise separable convolution includes depth-wise convolution and point-wise convolution.
8. The method for ultrasonic image-oriented target detection and classification according to claim 6 or 7, characterized in that: In step 5-4), batch normalization and activation function are applied after the 1x1 convolution to enhance the feature expression ability and the stability of the model.