Medical image target detection method and device, electronic equipment and readable storage medium

Abstract

The invention relates to a medical image target detection method and device, electronic equipment and a readable storage medium, and the method comprises the steps: obtaining a to-be-detected medical image and a medical image target detection model, according to the medical image target detection model, an expandable residual module is added in a backbone network, a multi-branch attention mechanism module is added to a head in a neck network, and at least part of conv modules in the backbone network and the neck network are replaced by YOLOv8 models of deep down-sampling modules. The medical image target detection model is obtained through joint regression loss function training; and inputting the to-be-detected medical image into the medical image target detection model to obtain a medical image target detection result. According to the medical image target detection method provided by the invention, the accuracy of medical image target detection is improved.

Description

Technical Field

[0001] The present application relates to the field of medical image processing technology, and in particular to a medical image target detection method, device, electronic device and readable storage medium. Background Art

[0002] With the rapid development of computer technology, it has brought far-reaching impacts to the medical field, greatly improving the quality and efficiency of medical services. Computer technology has been widely used in many fields such as medical imaging, robotic surgery, medical big data analysis, remote diagnosis, and preventive medicine. Computer technology plays a particularly critical role in target detection, which aims to detect, accurately locate, and classify various targets of interest, such as tumors, nodules, and lesions, from complex and diverse medical images.

[0003] In recent years, with the rapid development of deep learning technology, target detection technology has gradually shifted from traditional manual feature extraction methods to automated technology based on deep learning. Target detection algorithms based on deep learning have significantly improved detection accuracy, processing speed, generalization ability, and robustness compared to traditional methods. According to the needs of application scenarios, deep learning target detection algorithms can be divided into two categories: two-stage detection algorithms and single-stage detection algorithms. Two-stage detection algorithms such as R-CNN, Fast R-CNN, and Faster R-CNN not only have slow processing speeds but also large computational workloads. These characteristics make them face many challenges in actual deployment and real-time detection. In contrast, single-stage detection algorithms such as SSD, RetinaNet, CenterNet, and the YOLO series have the advantages of high detection accuracy, good real-time application, fast detection speed, and easy deployment.

[0004] In addition, the existing technology still has some shortcomings in practical applications, mainly including the following aspects:

[0005] 1) Diversity of target size and shape: Lesions or diseased areas are often very small and account for a very low proportion of the entire image, which poses a great challenge to detection. Small targets are prone to losing important information during downsampling, resulting in missed detection; targets in medical images have different shapes, such as irregular tumors or diseased areas, which increases the difficulty of detection.

[0006] 2) Image quality issues: Medical images (such as X-ray, CT, MRI) may have low contrast or blur issues, especially when the boundaries between tissues are blurred, making it difficult to clearly distinguish the lesion area.

[0007] 3) Noise and interference: Medical imaging equipment may introduce noise or artifacts such as scattering and motion artifacts, which can interfere with the performance of the object detection model.

[0008] In order to solve the above problems and achieve efficient and high-performance target detection, it is necessary to develop algorithms with better detection effects to meet the needs in the medical field. Summary of the invention

[0009] In this embodiment, a medical image target detection method, device, electronic device and storage medium are provided to solve the problem existing in the related art that it is difficult to ensure detection accuracy while making the model lightweight.

[0010] In a first aspect, a method for detecting a medical image object is provided in the present application, comprising:

[0011] Obtain a medical image to be detected and a medical image target detection model, wherein the medical image target detection model is a YOLOv8 model in which an expandable residual module is added to the backbone network, a multi-branch attention mechanism module is added to the head of the neck network, and at least part of the conv modules in the backbone network and the neck network are replaced with deep downsampling modules, and the medical image target detection model is obtained by training with a joint regression loss function;

[0012] The medical image to be detected is input into the medical image target detection model to obtain a medical image target detection result.

[0013] In some of the embodiments, the deep downsampling module performs average pooling on the input features, and then divides them into the first, second and third branch channels through a split layer, wherein the first and second branch channels perform average pooling and maximum pooling respectively, the third branch channel performs a convolution, and the first and second branch channels perform a convolution again after pooling, respectively, and finally the output results obtained from the first, second and third branch channels are spliced ​​and output.

[0014] In some of the embodiments, the multi-branch channel attention mechanism module changes the dimensional order of the input tensor by performing a transpose operation on the input tensor of each of the three branches, and then passes the tensor to the Z pool, reduces the dimension from 3 dimensions to 2 dimensions through the Z pool, and connects the average pooled features and the maximum pooled features on each dimension, and further generates corresponding attention weights through convolutional layers and sigmoid operations; the first and second branches of the three branches further perform a transpose operation on the corresponding generated attention weights to restore the original input shape, and then perform a sigmoid operation on the outputs of the first and second branches to obtain a first output result and a second output result, and the attention weight generated by the third branch of the three branches is added to the first output result and the second output result to obtain an output feature map.

[0015] In some of the embodiments, an expandable residual module is introduced into the BottleNeck block of the C2f module, and the expandable residual module compresses the input features through convolution and enters the multi-branch convolution channel to fuse the multi-scale receptive field to generate a feature map, and then convolves and max-pools the generated feature maps and splices them, and then fuses them through convolution layers, and finally obtains the multi-scale feature map through residual operation.

[0016] In some embodiments, the joint regression loss function is obtained by weighting the NMD loss function and the M-IoU loss function:

[0017] The calculation steps of the NMD loss function are as follows:

[0018] 1) Distribution modeling of bounding boxes;

[0019]

[0020] in, (c x , c y ) is the center coordinate; ω is the width of the bounding box, h is the height; x is the coordinate (x, y) of the Gaussian distribution, μ is the mean vector of the Gaussian distribution, T is the transpose of the matrix; n represents the dimension, ∑ is the covariance matrix of the Gaussian distribution, σ 2 is the scaling factor;

[0021] 2) Normalized distance;

[0022]

[0023] xx a 、cy a and cx b 、cy b The target N a and N b The center coordinates of a 、h b The target N a and N b The width and height of the vector; the "2" below is the sum of the width and height of the denominator, which is used to normalize the difference; the "2" above represents the square of the L2 norm, which is used to calculate the sum of the squares of the vector; and These two parts calculate the difference in the center points of the two boxes in the x and y directions; These two parts measure the target N a and N b Differences in width and height;

[0024]

[0025] Where c is a constant that can be adjusted according to actual conditions; NWD(N a , N b ) is the normal distribution horizontal distance between variables, which is used to represent the NMD loss function; w2 2 (N a , N b ) is the difference measure between the two targets calculated previously; The average value of target width is introduced as a normalization factor;

[0026] The calculation steps of the M-IoU loss function are as follows:

[0027]

[0028]

[0029] Among them, Aiou is used to represent the M-IoU loss function, w i H i represents the intersection area, s u represents the union area, α represents the introduced factor, which is used to characterize the adjustment item related to the target box size; s g Represents the area of ​​the target box, k is a constant and is adjusted according to actual conditions.

[0030] In a second aspect, the present application provides a medical image object detection device, comprising:

[0031] An acquisition unit, used for acquiring a medical image to be detected and a medical image target detection model, wherein the medical image target detection model is a YOLOv8 model in which an expandable residual module is added to the backbone network, a multi-branch attention mechanism module is added to the head of the neck network, and at least part of the conv modules in the backbone network and the neck network are replaced with deep downsampling modules, and the medical image target detection model is obtained by training with a joint regression loss function;

[0032] The detection unit is used to input the medical image to be detected into the medical image target detection model to obtain a medical image target detection result.

[0033] In some embodiments, the expandable residual module compresses the input features through convolution and then enters the multi-branch convolution channel to fuse the multi-scale receptive field to generate a feature map, and then convolution and maximum pooling are performed on the generated feature maps, and then they are concatenated, then fused through the convolution layer, and finally the residual operation is performed and output;

[0034] The multi-branch channel attention mechanism module changes the dimensional order of the input tensor by performing a transpose operation on the input tensor of each of the three branches, and then passes the tensor to the Z pool, reduces the dimension from 3 dimensions to 2 dimensions through the Z pool, and connects the average pooled features and the maximum pooled features on each dimension, and further generates corresponding attention weights through a convolution layer and a sigmoid operation; the first and second branches of the three branches further perform a transpose operation on the corresponding generated attention weights to restore the original input shape, and then perform a sigmoid operation on the outputs of the first and second branches to obtain a first output result and a second output result, and the attention weight correspondingly generated by the third branch of the three branches is added to the first output result and the second output result and then output;

[0035] The deep downsampling module performs average pooling on the input features, and then divides them into the first, second and third branch channels through the split layer, wherein the first and second branch channels perform average pooling and maximum pooling respectively, the third branch channel performs a convolution, and the first and second branch channels perform a convolution again after pooling respectively, and finally the output results obtained by the first, second and third branch channels are spliced ​​and output.

[0036] In a third aspect, the present application provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to implement the medical image target detection method described in the first aspect above.

[0037] In a fourth aspect, the present application provides a readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the medical image target detection method described in the first aspect above.

[0038] Compared with the related art, the present application provides a medical image target detection method, including: obtaining a medical image to be detected and a medical image target detection model, wherein the medical image target detection model is a YOLOv8 model in which an expandable residual module is added to the backbone network, a multi-branch attention mechanism module is added to the head of the neck network, and at least part of the conv modules in the backbone network and the neck network are replaced with a deep downsampling module, and the medical image target detection model is obtained by training the joint regression loss function; the medical image to be detected is input into the medical image target detection model to obtain a medical image target detection result. The medical image target detection method proposed in the present invention simplifies the learning process through the expandable residual module and ensures a more orderly and more effective extraction of relevant features. The multi-branch channel attention mechanism module is added to the head of the neck network, and the attention speeds up the convergence speed and accuracy with only minimal increase in parameters. The deep downsampling module can enhance the feature extraction capability of the model while maintaining computational efficiency, and effectively capture the contextual information in the image by introducing a deeper network layer and a more complex connection method, which greatly improves the recognition and positioning accuracy of the target. The medical image target detection model is obtained through training with a joint regression loss function, and can dynamically adjust the regression loss during the training process, thereby improving the reliability of the medical image target detection model and further improving the accuracy of the medical image target detection in this application.

[0039] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0041] Figure 1 is a flow chart of the medical image target detection method of the present invention;

[0042] Figure 2 It is a network architecture diagram of the improved Yolov8 model of the present invention;

[0043] Figure 3 is a network architecture diagram of the C2f-MWR module of the present invention;

[0044] Figure 4 This is a network architecture diagram of the depth downsampling module D-down of the present invention;

[0045] Figure 5 It is a network architecture diagram of the multi-branch attention mechanism module MA of the present invention;

[0046] Figure 6 is a structural block diagram of the medical image object detection device of the present invention;

[0047] Figure 7 It is a diagram of the electronic device architecture of the present invention. DETAILED DESCRIPTION

[0048] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0049] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the", "these" and the like in this application do not represent quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. Usually, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0050] The present application can be used in many general or special computing device environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multi-processor devices, distributed computing environments including any of the above devices or equipment, etc.

[0051] Next, combine Figure 1 The medical image target detection method of the present application is introduced in detail, including the following steps:

[0052] Step S11, obtaining a medical image to be detected and a medical image target detection model.

[0053] The medical image object detection model can be a YOLOv8 model in which an expandable residual module is added to the backbone network, a multi-branch attention mechanism module is added to the head of the neck network, and at least part of the conv modules in the backbone network and the neck network are replaced with deep downsampling modules.

[0054] The expandable residual module compresses the input features through convolution and enters the multi-branch convolution channel to fuse the multi-scale receptive field to generate a feature map. The generated feature map is then convolved and max-pooled and concatenated, followed by fusion through the convolution layer, and finally the residual operation is performed on the output.

[0055] Specifically, Figure 2 As shown in FIG. 1 , the expandable residual module is added to the penultimate C2f module of the Backbone layer, and the module is represented by C2f-MWR in the figure. Among them, the expandable residual module is introduced into the BottleNeck block of the C2f module. For example, Figure 3 As shown in the figure, the expandable residual module compresses the input features through a 3×3 convolution kernel and then enters the three-branch convolution channel. The three-branch convolution channel uses convolution kernel channels with hole numbers of 1, 3, and 5, and 3×3 sizes to fuse multi-scale receptive fields to generate feature maps. The generated feature maps are then convolved and max-pooled and then concatenated, then fused through a 1×1 convolution layer, and finally output by residual operation.

[0056] Through the expandable residual module, a two-step method is used inside the residual to efficiently draw multi-scale contextual information, and the feature maps generated by the multi-scale receptive field are fused. The generated feature maps are convolved and max-pooled respectively, and then spliced. Further convolution fusion can maintain the complete feature expression, which not only makes feature fusion efficient, but also improves the feature expression ability, helps to improve the generalization ability of the model, avoids overfitting, effectively captures multi-scale contextual information, and ensures the orderly and efficient extraction of related features.

[0057] The deep downsampling module performs average pooling on the input features, and then divides them into the first, second and third branch channels through the split layer, where the first and second branch channels perform average pooling and maximum pooling respectively, and the third branch channel performs a convolution. After pooling, the first and second branch channels perform a convolution again respectively, and finally the output results obtained from the first, second and third branch channels are spliced ​​and output.

[0058] Specifically, Figure 2 As shown in FIG. 1 , the penultimate Conv module of the Backbone layer and the penultimate Conv module of the Neck layer are replaced by a deep downsampling module, which is represented by D-down in the figure. For example, Figure 4As shown in the figure, the deep downsampling module performs average pooling on the input features, and then divides them into left, right and middle channels through the split layer, where the left and right branch channels perform average pooling and maximum pooling respectively, the middle channel performs a convolution, and the left and right branch channels perform a convolution again after pooling, and finally the output results of the left and right branch channels and the middle channel are spliced ​​and output.

[0059] The deep downsampling module first averages the input of the previous layer to reduce the number of parameters and calculations of the model, and then divides the input channel into three through the split layer. The first and second branch channels perform average pooling and maximum pooling respectively, and the third branch channel performs a convolution. After the first and second branch channels are pooled, they are convolved again. This can extract features at a higher level, help capture more global features, and improve the model's expressiveness. Finally, the output results of the three channels are spliced ​​and output, which helps the model further learn multi-scale information and improve its ability to capture complex patterns.

[0060] The multi-branch channel attention mechanism module changes the dimensional order of the input tensor by performing a transpose operation on the input tensor of each of the three branches, and then passes the tensor to the Z pool, reduces the dimension from 3 dimensions to 2 dimensions through the Z pool, and connects the average pooled features and the maximum pooled features on each dimension, and further generates corresponding attention weights through convolutional layers and sigmoid operations; the first and second branches of the three branches further perform a transpose operation on the corresponding generated attention weights to restore the original input shape, and then perform a sigmoid operation on the outputs of the first and second branches to obtain the first output result and the second output result, and the attention weight generated by the third branch of the three branches is added to the first output result and the second output result and then output.

[0061] Specifically, Figure 2 As shown in FIG, a multi-branch attention mechanism module is added to the head of the neck network, and the multi-branch attention mechanism module is represented by MA in the figure. Figure 5As shown, the multi-branch channel attention mechanism module changes the dimensional order of the input tensor by performing a transpose operation on the input tensor of each of the three branches, and then passes the tensor to the Z pool, reduces the dimension from 3 dimensions to 2 dimensions through the Z pool, and connects the average pooled features and the maximum pooled features on each dimension, and further generates corresponding attention weights through convolutional layers and sigmoid operations; the first and second branches of the three branches further perform a transpose operation on the corresponding generated attention weights to restore the original input shape, and then perform a sigmoid operation on the outputs of the first and second branches to obtain the first output result and the second output result, and the attention weight generated by the third branch of the three branches is added to the first output result and the second output result and then output.

[0062] The multi-branch channel attention mechanism module captures the interaction of cross dimensions and uses a three-branch structure. Each branch is responsible for capturing the cross dimension between the spatial dimension and the channel dimension of the input. The first two branches and the third branch extract features of different dimensions respectively. By combining these features, richer and more diverse feature representations can be generated, thereby improving the feature extraction capability of the model.

[0063] The medical image object detection model is trained using a joint regression loss function.

[0064] Step S12, inputting the medical image to be detected into the medical image target detection model to obtain a medical image target detection result.

[0065] It can be seen from the above technical solutions that the medical image target detection method provided by the present application can use the medical image target detection model to perform target detection on medical images, wherein the medical image target detection model is a YOLOv8 model in which an expandable residual module is added to the backbone network, a multi-branch attention mechanism module is added to the head of the neck network, and at least part of the conv modules in the backbone network and the neck network are replaced with deep downsampling modules. The medical image target detection method proposed by the present method simplifies the learning process through the expandable residual module and ensures a more orderly and more effective extraction of relevant features. The multi-branch channel attention mechanism module is added to the head of the neck network, and the attention speeds up the convergence speed and accuracy with only minimal increase in parameters. The deep downsampling module can enhance the feature extraction capability of the model while maintaining computational efficiency, and effectively capture the contextual information in the image by introducing deeper network layers and more complex connection methods, which greatly improves the recognition and positioning accuracy of the target.

[0066] Specifically, the joint regression loss function is weighted according to the NMD loss function and the M-IoU loss function:

[0067] The calculation steps of the NMD loss function are as follows:

[0068] 1) Distribution modeling of bounding boxes;

[0069]

[0070] in, (c x , c y ) is the center coordinate; ω is the bounding box width, h is the height; x is the coordinate (x, y) of the Gaussian distribution; μ is the mean vector of the Gaussian distribution; T is the transpose of the matrix; n represents the dimension, ∑ is the covariance matrix of the Gaussian distribution; σ 2 is the scaling factor;

[0071] 2) Normalized distance;

[0072]

[0073] xx a 、cy a and cx b 、cy b The target N a and N b The center coordinates of a 、w b and h a 、h b The target N a and N b The width and height of the vector; the "2" below is the sum of the width and height of the denominator, which is used to normalize the difference; the "2" above represents the square of the L2 norm, which is used to calculate the sum of the squares of the vector; and These two parts calculate the difference in the center points of the two boxes in the x and y directions; These two parts measure the target N a and N b Differences in width and height;

[0074]

[0075] Where c is a constant that can be adjusted according to actual conditions; NWD(N a , N b ) is the normal distribution horizontal distance between variables, which is used to represent the NMD loss function; w2 2 (N a , N b ) is the difference measure between the two targets calculated previously; The average value of target width is introduced as a normalization factor;

[0076] The calculation steps of the M-IoU loss function are as follows:

[0077]

[0078]

[0079] Among them, Aiou is used to represent the M-IoU loss function, w i H i represents the intersection area, s u represents the union area, α represents the introduced factor, which is used to characterize the adjustment item related to the target box size; s g Represents the area of ​​the target box, k is a constant and is adjusted according to actual conditions.

[0080] When there is no overlap between borders, Make A Iou Since the gradient of back propagation disappears, the bounding box regression loss is changed to the following paradigm:

[0081] A i =A iou +R i

[0082] R i : Scale adaptive term R i It is a term related to the size of the target box, which can be used to compensate for the problem that IoU is insensitive to small objects. Small objects have low IoU, and even a small error will lead to a large IoU difference. i Can reduce penalties for small targets.

[0083] Define the dynamic focusing mechanism as f(β), where By assigning small gradient gains to high-quality anchor boxes with small β and small gradients to low-quality anchors, the harm of low-quality samples to bounding box regression is reduced.

[0084] The parameters of the medical image target detection model are adjusted by the above-mentioned joint regression loss function, which can improve the reliability of the medical image target detection model and further improve the accuracy of medical image target detection in this application.

[0085] In order to verify the superiority of the improved YOLOv8 model proposed in this application, the following experiment uses the Tumor dataset obtained from the roboflow platform. The Tumor dataset includes 9,900 brain tumor images and 3 types of targets. The images are trained on the network and randomly divided into training set, validation set and test set in a ratio of 8:1:1, including 7,920 training set data, 990 validation set data and 990 test set data. In order to improve the diversity of the data set, each image is annotated multiple times, and each method includes different angles and scale transformations to simulate visual scenes in the real world. The experiment uses the operating system windows10, the CPU is Intel Xeon Silver 4214R, the GPU used is NVDIA GeForce RTX3080Ti, 12G video memory; the deep learning framework is pytorch 1.13.1, CUDA 11.1. For each algorithm, epochs is uniformly set to 200.

[0086] Evaluation indicators include recall (R), precision (P), mean average precision (mAP), frames per second (FPS), and model parameters (params). The mAP value is calculated by precision (P) and recall (R). Precision refers to the proportion of correctly detected targets, and recall refers to the proportion of targets actually detected in the data set.

[0087] This application uses 8 models in the target detection comparison experiment, including Tood, Atss, Ddq, Dino, YOLOv5n, YOLOv6n, YOLOv8n, YOLOv9c and the method proposed in this application. Its performance is evaluated by 6 indicators including mAP0.5, mAP0.5:0.95, P, R, FPS, and Params.

[0088] In order to verify the superiority and generalization of the method of this application, the results of the comparative experiments on the Tumor dataset are shown in Table 1. The results show that compared with YOLOv5n, the algorithm proposed in this application has increased P and R by 4.1% and 4.9% respectively, mAP0.5 and mAP0.5:0.95 by 4.5% and 5.7% respectively, and the parameter amount Params has also decreased by 2%, indicating that the proposed model is more lightweight; compared with YOLOv6n, P and R increased by 5.3% and 4.4% respectively, mAP0.5 and mAP0.5:0.95 increased by 4.6% and 4.8% respectively, and the parameter amount Params decreased by 40.9% compared with it, which shows that the model of this application still has advantages in lightweight; compared with YOLOv9c, P and R increased by 2.3% and 1.6% respectively, mAP0.5 and mAP0.5:0.95 increased by 1.4% and 0.3% respectively, and the parameter amount Params decreased by 80.6% compared with it. Although other performance indicators of the model have improved, the FPS has slightly decreased. This shows that the optimization strategy effectively enhances the overall performance of the model, but also introduces a certain amount of computational overhead, resulting in a slowdown in the detection speed. However, this trade-off is still acceptable in applications with high precision requirements such as medical images. Therefore, the model of this application not only achieves a significant improvement in comprehensive performance, but also achieves a lightweight model.

[0089]

[0090] Table 1

[0091] Compared with the original YOLOv8n model, the improved algorithm in this application has improved the Precision, Recall and mAP@0.5 indicators by 2.9%, 5.5% and 1.4% respectively. At the same time, the FLOPs indicator of the model has been reduced by 5.7%, and the Params indicator has been reduced by 5.0%. These results show that this application has achieved significant performance improvements while taking into account both model scale and detection accuracy.

[0092] In a second aspect, the present invention provides a medical image object detection device 600, Figure 6 is a block diagram of the medical image target detection device of the present invention, such as Figure 6 As shown, the device comprises:

[0093] An acquisition unit 601 is used to acquire a medical image to be detected and a medical image target detection model, wherein the medical image target detection model is a YOLOv8 model in which an expandable residual module is added to the backbone network, a multi-branch attention mechanism module is added to the head of the neck network, and at least part of the conv modules in the backbone network and the neck network are replaced with deep downsampling modules, and the medical image target detection model is obtained by training with a joint regression loss function;

[0094] The detection unit 602 is used to input the medical image to be detected into the medical image target detection model to obtain a medical image target detection result.

[0095] In this device, the learning process is simplified by the expandable residual module, and a more orderly and more efficient extraction of relevant features is ensured. A multi-branch channel attention mechanism module is added to the head in the neck network, and this attention speeds up the convergence speed and accuracy with only minimal increase in parameters. The deep downsampling module can enable the model to enhance the feature extraction capability while maintaining computational efficiency. By introducing deeper network layers and more complex connection methods to effectively capture contextual information in the image, the recognition and positioning accuracy of the target is greatly improved. The medical image target detection model is trained with a joint regression loss function, and can dynamically adjust the regression loss during the training process, thereby improving the reliability of the medical image target detection model and further improving the accuracy of the medical image target detection in this application.

[0096] In some embodiments, the expandable residual module compresses the input features through convolution and then enters the multi-branch convolution channel to fuse the multi-scale receptive field to generate a feature map, and then convolves and max-pools the generated feature maps respectively, and then fuses them through the convolution layer, and finally performs a residual operation on the output; the multi-branch channel attention mechanism module changes the dimensional order of the input tensor by performing a transpose operation on the input tensor of each of the three branches, and then passes the tensor to the Z pool, reduces the dimension from 3 dimensions to 2 dimensions through the Z pool, and connects the average pooled features and the maximum pooled features on each dimension, and further generates corresponding attention weights through the convolution layer and the sigmoid operation; the first and second branches of the three branches will generate the corresponding attention weights The weights are further transposed to restore the original input shape, and then the outputs of the first and second branches are sigmoided to obtain the first output result and the second output result. The attention weight corresponding to the third branch of the three branches is added to the first output result and the second output result and then output; the deep downsampling module average pools the input features, and then divides them into the first, second and third branch channels through the split layer, wherein the first and second branch channels are average pooled and maximum pooled respectively, and the third branch channel is convolved once, and the first and second branch channels are convolved once after pooling respectively, and finally the output results obtained from the first, second and third branch channels are spliced ​​and output.

[0097] Through the expandable residual module, a two-step method is used inside the residual to efficiently draw multi-scale context information, and the feature maps generated by the multi-scale receptive field are fused. The generated feature maps are convolved and max-pooled respectively, and then spliced. Further convolution fusion can maintain the complete feature expression, which not only makes feature fusion efficient, but also improves the feature expression ability, helps to improve the generalization ability of the model, avoids overfitting, effectively captures multi-scale context information, and ensures the orderly and efficient extraction of related features. The multi-branch channel attention mechanism module captures the interaction of cross-dimensionality and uses a three-branch structure. Each branch is responsible for capturing the cross-dimensionality between the input spatial dimension and the channel dimension. The first two branches and the third branch extract features of different dimensions respectively; by combining these features, richer and more diverse feature representations can be generated, thereby improving the feature extraction ability of the model. The deep downsampling module first averages the input of the previous layer to reduce the number of parameters and calculations of the model, and then divides the input channel into three through the split layer. The first and second branch channels perform average pooling and maximum pooling respectively, and the third branch channel performs a convolution. After the first and second branch channels are pooled, they are convolved again. This can extract features at a higher level, help capture more global features, and improve the model's expressiveness. Finally, the output results of the three channels are spliced ​​and output, which helps the model further learn multi-scale information and improve its ability to capture complex patterns.

[0098] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0099] The following embodiments are described by taking the above method applied to a computer device as an example. It can be understood that the computer device can be any device with computing and processing functions, and can be but not limited to a server or a personal laptop computer, etc. In one embodiment, the computer device can be an application server, and the application server can be a server for running an application to be tested.

[0100] For the third aspect, see Figure 7, which shows a hardware block diagram of an electronic device, the electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0101] like Figure 7 As shown, the electronic device includes: at least one processor 1, at least one communication interface 2, at least one memory 3 and at least one communication bus 4;

[0102] In the embodiment of the present application, the number of the processor 1, the communication interface 2, the memory 3, and the communication bus 4 is at least one, and the processor 1, the communication interface 2, and the memory 3 communicate with each other through the communication bus 4;

[0103] The processor 1 may be a central processing unit CPU, or an application-specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention, etc.;

[0104] The memory 3 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory;

[0105] Among them, the memory stores a program, and the processor can call the program stored in the memory, and the program is used to: implement each processing flow of the aforementioned helmet wearing detection method, or implement each processing flow of the aforementioned model training method.

[0106] In a fourth aspect, an embodiment of the present invention further provides a readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned medical image target detection method.

[0107] In a fifth aspect, an embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the aforementioned medical image target detection method.

[0108] The above-mentioned embodiments have described the present invention in particular detail with respect to possible scenarios, and those skilled in the art will recognize that the present invention can be practiced through other embodiments. The specific naming of components, the capitalization of terms, attributes, data structures, or any other programming or structural aspects are not mandatory or important, and the mechanisms or features of the present invention may have different names, forms, or procedures. The system may be implemented by a combination of hardware and software (as described), entirely by hardware elements, or entirely by software elements. The specific division of functions between the various system components described herein is exemplary only and not mandatory; on the contrary, the functions performed by a single system component may be performed by multiple components, or the functions performed by multiple components may be performed by a single component.

[0109] Those skilled in the art should understand that the various steps of the above disclosed method can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented with program codes executable by the computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the embodiments of the present invention are not limited to any specific combination of hardware and software.

[0110] These computing device executable programs (also referred to as programs, software, software applications, or code) include machine instructions for programmable processors, and these computing programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0111] Certain aspects of the present invention include process steps and instructions described herein in the form of algorithms. It should be noted that the process steps and instructions of the present invention can be implemented in software, firmware and / or hardware, and when implemented by software, it can be downloaded, stored on different platforms used by various operating systems and operated from the platforms.

[0112] Those skilled in the art will understand that the structures shown in the accompanying drawings are merely block diagrams of partial structures related to the scheme of the present application, and do not constitute a limitation on the terminal device to which the scheme of the present application is applied. The specific terminal device may include more or fewer components than shown in the figures, or combine certain components, or have a different arrangement of components.

[0113] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "possible design" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0114] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0115] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A medical image target detection method, characterized in that: include: Obtain a medical image to be detected and a medical image target detection model, wherein the medical image target detection model is a YOLOv8 model in which an expandable residual module is added to the backbone network, a multi-branch attention mechanism module is added to the head of the neck network, and at least part of the conv modules in the backbone network and the neck network are replaced with deep downsampling modules, and the medical image target detection model is obtained by training with a joint regression loss function; The medical image to be detected is input into the medical image target detection model to obtain a medical image target detection result.

2. The detection method according to claim 1, characterized in that: The deep downsampling module performs average pooling on the input features, and then divides them into the first, second and third branch channels through the split layer, wherein the first and second branch channels perform average pooling and maximum pooling respectively, the third branch channel performs a convolution, and the first and second branch channels perform a convolution again after pooling respectively, and finally the output results obtained by the first, second and third branch channels are spliced ​​and output.

3. The detection method according to claim 2, characterized in that: The multi-branch channel attention mechanism module changes the dimensional order of the input tensor by performing a transpose operation on the input tensor of each of the three branches, and then passes the tensor to the Z pool, reduces the dimension from 3 dimensions to 2 dimensions through the Z pool, and connects the average pooled features and the maximum pooled features on each dimension, and further generates corresponding attention weights through convolutional layers and sigmoid operations; the first and second branches of the three branches further perform a transpose operation on the corresponding generated attention weights to restore the original input shape, and then perform a sigmoid operation on the outputs of the first and second branches to obtain a first output result and a second output result, and the attention weight correspondingly generated by the third branch of the three branches is added to the first output result and the second output result and then output.

4. The detection method according to claim 3, characterized in that: An expandable residual module is introduced into the BottleNeck block of the C2f module. The expandable residual module compresses the input features through convolution and then enters the multi-branch convolution channel to fuse the multi-scale receptive field to generate a feature map. The generated feature map is then convolved and max-pooled and then spliced. Then, it is fused through the convolution layer and finally the residual operation is performed on the output.

5. The detection method according to claim 4, characterized in that: The joint regression loss function is weighted according to the NMD loss function and the M-IoU loss function: The calculation steps of the NMD loss function are as follows: 1) Distribution modeling of bounding boxes; in, (c x ,c y ) is the center coordinate; ω is the bounding box width, h is the height; x is the coordinate (x, y) of the Gaussian distribution; μ is the mean vector of the Gaussian distribution, Τ is the transpose of the matrix; n represents the dimension, ∑ is the covariance matrix of the Gaussian distribution, σ 2 is the scaling factor; 2) Normalized distance; xx a 、cy a and cx b 、cy b The target N a and N b The center coordinates of a 、h b The target N a and N b The width and height of the vector; the "2" below is the sum of the width and height of the denominator, which is used to normalize the difference; the "2" above represents the square of the L2 norm, which is used to calculate the sum of the squares of the vector; and These two parts calculate the difference in the center points of the two boxes in the x and y directions; These two parts measure the target N a and N b Differences in width and height; Where c is a constant that can be adjusted according to actual conditions; NWD(N a ,N b ) is the normal distribution horizontal distance between variables, used to represent the NMD loss function; w2 2 (N a ,N b ) is the difference measure between the two targets calculated previously; The average value of target width is introduced as a normalization factor; The calculation steps of the M-IoU loss function are as follows: Among them, Aiou is used to represent the M-IoU loss function, w i H i represents the intersection area, s u represents the union area, α represents the introduced factor, which is used to characterize the adjustment item related to the target box size; s g Represents the area of ​​the target box, k is a constant and is adjusted according to actual conditions.

6. A medical image object detection device, characterized in that: include: An acquisition unit, used for acquiring a medical image to be detected and a medical image target detection model, wherein the medical image target detection model is a YOLOv8 model in which an expandable residual module is added to the backbone network, a multi-branch attention mechanism module is added to the head of the neck network, and at least part of the conv modules in the backbone network and the neck network are replaced with deep downsampling modules, and the medical image target detection model is obtained by training with a joint regression loss function; The detection unit is used to input the medical image to be detected into the medical image target detection model to obtain a medical image target detection result.

7. The detection device according to claim 6, characterized in that: The expandable residual module compresses the input features through convolution and then enters the multi-branch convolution channel to fuse the multi-scale receptive field to generate a feature map, and then convolution and maximum pooling are performed on the generated feature maps, and then they are concatenated, and then fused through the convolution layer, and finally the residual operation is performed and output; The multi-branch channel attention mechanism module changes the dimensional order of the input tensor by performing a transpose operation on the input tensor of each of the three branches, and then passes the tensor to the Z pool, reduces the dimension from 3 dimensions to 2 dimensions through the Z pool, and connects the average pooled features and the maximum pooled features on each dimension, and further generates corresponding attention weights through a convolution layer and a sigmoid operation; the first and second branches of the three branches further perform a transpose operation on the corresponding generated attention weights to restore the original input shape, and then perform a sigmoid operation on the outputs of the first and second branches to obtain a first output result and a second output result, and the attention weight correspondingly generated by the third branch of the three branches is added to the first output result and the second output result and then output; The deep downsampling module performs average pooling on the input features, and then divides them into the first, second and third branch channels through the split layer, wherein the first and second branch channels perform average pooling and maximum pooling respectively, the third branch channel performs a convolution, and the first and second branch channels perform a convolution again after pooling respectively, and finally the output results obtained by the first, second and third branch channels are spliced ​​and output.

8. An electronic device, comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to implement the medical image object detection method according to any one of claims 1 to 5.

9. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the medical image object detection method according to any one of claims 1 to 5 is implemented.

Images