A method, system and device for testing surgical instruments

Abstract

This invention belongs to the field of image data processing technology, specifically relating to a method, system, and device for detecting surgical instruments. It involves acquiring a dataset of surgical instrument images, classifying, locating, and annotating the images using directional bounding boxes, and then performing data augmentation on the annotated images to obtain sample images. The sample images are then subjected to convolution, feature extraction, and attention enhancement, with the importance of pixels in the deep attention feature map evaluated based on an energy function. The features are then fused at multiple scales to obtain a multi-scale feature map. This multi-scale feature map is simultaneously input into a directional bounding box detection head, and classification, localization, and angle regression detection are performed through a three-branch parallel branch. This method reduces computational complexity and the number of parameters while maintaining detection accuracy, and can be efficiently deployed on mobile devices.

Description

Technical Field

[0001] This invention belongs to the field of image data processing technology, and specifically relates to a method, system and device for detecting surgical instruments. Background Technology

[0002] Accurate detection and management of surgical instruments are crucial for ensuring surgical safety and mitigating medical risks such as instrument loss. Traditional instrument counting relies on manual collaboration, which is susceptible to interference from the operating room environment, unexpected intraoperative situations, and communication errors, posing potential threats to patient health. To improve this situation, radio frequency identification (RFID) and barcode technologies have been introduced. However, RFID is prone to electromagnetic interference with other medical devices, and barcode technology is susceptible to identification failure due to bloodstains and instrument wear, neither of which completely solves the core pain points of surgical instrument management.

[0003] With the development of deep learning technology, computer vision-based detection methods are gradually becoming alternatives. Among existing deep learning methods, single-stage detection algorithms are favored for balancing accuracy and real-time performance. However, most methods use horizontal bounding boxes, making them difficult to adapt to scenarios where surgical instruments are placed at multiple angles and densely arranged, and they are also prone to introducing a large amount of background interference. While attention-based architectures can improve feature capture capabilities, they suffer from high computational overhead and demanding hardware requirements, limiting their clinical application. Furthermore, the complex lighting conditions in surgical scenarios, the mutual occlusion of instruments, and the high similarity of visual features among some instruments lead to problems such as feature ambiguity, classification difficulties, and false negatives / missed detections in existing algorithms.

[0004] Therefore, achieving model lightweighting while ensuring detection accuracy is a pressing technical problem that needs to be solved. Summary of the Invention

[0005] The purpose of this invention is to provide a method, system, and device for detecting surgical instruments, so as to solve the problems of difficulty in classifying surgical instruments, missed detection and false detection, and high computational complexity.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows.

[0007] A method for testing surgical instruments includes the following steps: A dataset of surgical instrument images was obtained, and the images were classified, located, and annotated with angles using directional bounding boxes. Data augmentation was then performed on the annotated images to obtain sample images.

[0008] The input sample image is subjected to convolution and feature extraction operations in sequence, and an enhanced feature map is obtained by downsampling. The enhanced feature map is then downsampled and attention-enhanced to obtain a shallow attention-enhanced feature map. The shallow attention-enhanced feature map is downsampled and attention-enhanced again to obtain a deep attention-enhanced feature map. The importance of each pixel in the deep attention-enhanced feature map is evaluated based on the energy function to obtain the energy value of each pixel, and an energy feature map is obtained based on the energy value.

[0009] The energy feature map is upsampled, concatenated with the shallow attention-enhanced feature map, and then attention enhancement is performed to obtain a refined feature map. The refined feature map is upsampled, concatenated with the enhanced feature map, and then attention enhancement is performed again to obtain a large-size feature map. The large-size feature map is downsampled, concatenated with the refined feature map, and then attention enhancement is performed to obtain a medium-size feature map. The medium-size feature map is downsampled, concatenated with the energy feature map, and then feature enhancement is performed to obtain a small-size feature map.

[0010] Large, medium, and small feature maps are simultaneously input into the oriented bounding box detection head, and classification, localization, and angle regression detection are performed through three parallel branches.

[0011] Preferably, the downsampling process includes the following operations: The sample image is pooled and divided into two parallel sub-branches evenly according to the number of channels. One sub-branch uses convolution to downsample and extract detailed features, while the other sub-branch uses max pooling to extract salient features. The two sub-branches are then concatenated to complete the downsampling operation.

[0012] Preferably, the importance of pixels in the deep attention feature map is evaluated based on an energy function: Each pixel is treated as a neuron. The matching deviation between the current neuron and the target feature is calculated, and the activation deviation with the surrounding neurons is measured. The matching deviation and activation deviation are fused to obtain an energy value. The importance of the pixel is evaluated based on the energy value.

[0013] Preferably, the energy feature map includes the following operations: The reciprocal of the energy value is normalized by an activation function to generate feature weights; the feature weights are then multiplied element-wise with the sample image to obtain the energy feature map.

[0014] Preferably, the three-branch parallel branch includes: The classification branch uses a dual 1×1 convolutional structure to perform semantic classification of the target; the localization branch uses three layers of depthwise separable convolutions and a single layer of 1×1 convolutions to obtain the target's precise location and size information; the angle regression branch uses a single layer of depthwise separable convolutions and a single layer of 1×1 convolutions to predict the target's rotation angle.

[0015] A surgical instrument testing system includes the following modules: Sample construction module: Acquire images of surgical instrument dataset, classify, locate and annotate the images using directional bounding boxes; perform data augmentation processing on the annotated images to obtain sample images.

[0016] Feature enhancement module: Input sample image, perform convolution and feature extraction operations in sequence, and obtain enhanced feature map through downsampling; perform downsampling and attention enhancement processing on the enhanced feature map to obtain shallow attention enhanced feature map; perform downsampling and attention enhancement again on the shallow attention enhanced feature map to obtain deep attention enhanced feature map; evaluate the importance of pixels in the deep attention enhanced feature map based on the energy function to obtain the energy value of each pixel, and obtain energy feature map based on the energy value.

[0017] Feature fusion module: The energy feature map is upsampled, concatenated with the shallow attention-enhanced feature map, and then attention enhancement is performed to obtain a refined feature map; the refined feature map is upsampled, concatenated with the enhanced feature map, and then attention enhancement is performed again to obtain a large-size feature map; the large-size feature map is downsampled, concatenated with the refined feature map, and then attention enhancement is performed to obtain a medium-size feature map; the medium-size feature map is downsampled, concatenated with the energy feature map, and then feature enhancement is performed to obtain a small-size feature map.

[0018] The detection output module simultaneously inputs large-size, medium-size, and small-size feature maps into the oriented bounding box detection head, and performs classification, localization, and angle regression detection through three parallel branches.

[0019] Preferably, the feature enhancement module includes: The downsampling module performs pooling on the sample image and divides it into two parallel sub-branches evenly according to the number of channels. One sub-branch uses convolution to downsample and extract detailed features, while the other sub-branch performs max pooling to extract salient features. The two sub-branches are then concatenated to complete the downsampling operation.

[0020] Preferably, the feature enhancement module includes: The feature map pixel importance evaluation module treats each pixel as a neuron, calculates the matching deviation between the current neuron and the target feature, and then measures the activation deviation with the surrounding neurons. The matching deviation and activation deviation are fused to obtain an energy value, and the importance of the pixel is evaluated based on the energy value.

[0021] Preferably, the feature enhancement module includes: The energy feature map construction module normalizes the inverse of the energy value using an activation function to generate feature weights; the feature weights are then multiplied element-wise with the sample image to obtain the energy feature map.

[0022] A surgical instrument testing device includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement a surgical instrument testing method.

[0023] Compared with the prior art, the technical solution provided by this invention has the following advantages: A dual-branch decoupled downsampling method is used to extract and fuse detailed features and salient features in parallel, which reduces the number of parameters and computational cost while maintaining detection accuracy and model lightweightness.

[0024] By assessing feature importance through neuron energy functions, complex background noise is effectively suppressed, and effective representation is enhanced, significantly improving target focusing ability without increasing computational overhead.

[0025] By configuring differentiated convolutions for different branches, the accuracy of classification, localization, and angle regression detection is guaranteed while reducing the number of parameters and computational complexity. Attached Figure Description

[0026] Figure 1 The invention demonstrates the visualized detection results in a general scenario; Figure 2 The invention demonstrates the visualized detection results in complex scenarios. Figure 3 The invention demonstrates the visualized detection results in challenging scenarios; Figure 4 A predefined one-to-one mapping relationship between each category and the bounding box color. Detailed Implementation

[0027] To further understand the content of this invention, the invention will be described in detail with reference to the embodiments.

[0028] A method for testing surgical instruments includes the following steps: S1. Obtain a dataset of surgical instrument images, and use directional bounding boxes to classify, locate, and annotate the images at angles; perform data augmentation on the annotated images to obtain sample images.

[0029] First, images of surgical instruments were acquired. During the operation, 17 representative instruments were selected from basic surgical instruments in different departments such as surgery, otolaryngology, rhinology, ophthalmology, and dentistry. Images were acquired using a 500w pixel Hikvision industrial camera against a background of green, blue, and black non-woven fabric, resulting in 840 original images, which served as the initial dataset.

[0030] The initial dataset for this operation included the following 17 surgical instruments: hemostatic forceps, bile duct forceps, stone retrieval forceps, towel forceps, tongue forceps, gastric forceps, intestinal forceps, surgical scissors (ZC288), surgical scissors (ZC290), surgical scissors (ZC343), surgical scissors (ZC414), nasal forceps, ophthalmic forceps, medical forceps, gallstone spoon, mouth mirror, and probe.

[0031] Secondly, the open-source X-anylabeling tool was used to annotate the images with oriented bounding boxes. Compared to horizontal bounding boxes, oriented bounding boxes showed significant advantages in target detection tasks that require precise capture of target orientation and shape. This is because surgical instruments are often placed at arbitrary angles, and traditional horizontal bounding boxes often include a large amount of irrelevant background areas, thus diluting the purity of target features. The oriented bounding box format can tightly surround the target instrument, minimizing background information interference within the bounding box, thereby providing more accurate target localization and shape description.

[0032] Furthermore, when multiple instruments are densely arranged, horizontal bounding boxes tend to overlap significantly, making it easy for non-maximum suppression algorithms to miss detections or suppress errors. In contrast, the compact representation of oriented bounding boxes effectively reduces the overlap of bounding boxes in dense scenes, improving the performance of subsequent processing.

[0033] To ensure the quality of image annotation, a multi-person annotation and cross-validation method was adopted to ensure that each bounding box is closely attached to the instrument and that the center point of each bounding box falls on the instrument.

[0034] To further enhance the model's generalization ability and data diversity, data augmentation strategies were employed to expand the dataset, balancing the number of samples across different classes and preventing excessive bias towards the majority class during the learning process. Furthermore, images were subjected to horizontal and vertical flipping, salt-and-pepper noise, and motion blur to simulate complex scenes in real-world environments.

[0035] The final dataset expanded from 840 images to 1675, and the number of samples expanded from 3674 to 7472.

[0036] S2. Input sample image, perform convolution and feature extraction operations in sequence, and obtain enhanced feature map by downsampling; perform downsampling and attention enhancement processing on the enhanced feature map to obtain shallow attention enhanced feature map; perform downsampling and attention enhancement on the shallow attention enhanced feature map again to obtain deep attention enhanced feature map; evaluate the importance of pixels in the deep attention enhanced feature map based on the energy function to obtain the energy value of each pixel, and obtain energy feature map based on the energy value.

[0037] Using a surgical instrument image of size 640×640×3 as input, the image is first processed by a 3×3 convolution to extract initial features, and then converted into a 320×320×64 feature map to quickly capture low-level visual features such as edges and corners.

[0038] Next, 3×3 convolution is used to further reduce the resolution to 160×160×128, laying the foundation for subsequent deep feature extraction.

[0039] A two-layer feature extraction operation is performed on the 160×160×128 feature map output from the upper layer. The feature extraction module uses residual connections and depthwise separable grouped convolutions to enhance the representation capability of mid-level features without increasing the computational cost, while mitigating the gradient vanishing problem in deep networks through residual paths.

[0040] Next, downsampling and feature enhancement processing are performed to obtain an enhanced feature map of size 80×80×256.

[0041] Downsampling reduces the spatial resolution of the feature map while preserving important feature information, thereby abstracting more global semantic information from the local texture of the instrument. Traditional convolutional downsampling structures consist of stacked 2D convolutional layers, batch normalization layers, and activation function layers, involving a large amount of computation, which limits detection speed. When dealing with slender, multi-angle rotating metal instruments with complex surface textures, traditional downsampling often struggles to achieve the optimal balance between preserving high-frequency details and extracting low-frequency semantics.

[0042] Therefore, this invention adopts a decoupled downsampling structure, which decomposes the feature extraction task into two parallel branches: a detail information extraction branch and a salient feature extraction branch. This improves detection accuracy while simplifying the design of structural parameters.

[0043] Preferably, downsampling in this invention includes the following operations: The sample image is pooled and divided into two parallel sub-branches evenly according to the number of channels. One sub-branch uses convolution to downsample and extract detailed features, while the other sub-branch uses max pooling to extract salient features. The two sub-branches are then concatenated to complete the downsampling operation.

[0044] Assuming the input feature map ,in, For the number of channels, and The height and width of the feature map.

[0045] First, an initial aggregation is performed using an average pooling layer, which smooths noise while preserving information between channels. The output space size of this operation is: (1), in, Representation of feature map The height after initial aggregation; Representation of feature map Width after initial aggregation; Indicates the kernel size; Indicates the fill size; Indicates the step size.

[0046] At this time, the feature map Number of channels The size remains unchanged, but the dimensions change. .

[0047] Subsequently, through segmentation operations, the feature map It is divided into two parallel sub-branches, with the number of channels being halved uniformly, and the corresponding feature maps are as follows: and .

[0048] Specifically, within a sub-branch, for Feature downsampling is performed using a 3×3 convolution operation to extract detailed features of surgical instruments. Size transformation is then performed according to formula (1) to obtain the sub-branch output. .

[0049] In another branch, regarding First, the salient features of the surgical instruments are obtained through a max pooling layer. The size is then transformed according to formula (1), and the number of channels is adjusted through a 1×1 convolutional layer to obtain another sub-branch output. .

[0050] Max pooling layers traverse feature maps through a fixed-size sliding window, retaining only the feature value with the highest response intensity within each local region while discarding information from all other pixels within the window. This results in the irreversible loss of fine information such as precise local spatial location, subtle texture gradients, small-scale features, and boundary contours.

[0051] Therefore, the present invention will and The fusion is achieved through a concatenation operation. One sub-branch is responsible for extracting detailed information and maintaining the integrity of the structure, while the other sub-branch is responsible for extracting salient features. This, combined with a 1×1 convolution to integrate channel information, can compensate for the loss of detail during max pooling to some extent.

[0052] Ultimately, the number of output channels is the sum of the number of channels in both branches. The size is and .

[0053] The downsampling process designed in this invention uses parameterless pooling layers to replace some computationally expensive convolutional layers and employs 1×1 convolutions as channel adjustment units, significantly reducing the number of parameters and thus greatly lowering the computational burden of the algorithm. Furthermore, a dual-branch structure is used to achieve differentiated extraction of detailed and salient features, and a complementary mechanism is constructed through fusion, greatly enhancing the algorithm's ability to represent the heterogeneous features of surgical instruments. In image detection in clinical surgical scenarios, this approach improves feature representation capabilities while significantly reducing the number of parameters and computational load, making it adaptable to the computational limitations of clinical equipment.

[0054] After downsampling the enhanced feature map, attention enhancement processing is performed to obtain a shallow attention enhanced feature map with a size of 40×40×512. The shallow attention enhanced feature map is then downsampled and enhanced sequentially to obtain a deep attention enhanced feature map. The importance of each pixel in the deep attention enhanced feature map is evaluated based on the energy function to obtain the energy value of each pixel. An energy feature map is then obtained based on the energy value.

[0055] In clinical surgical settings, there is often a complex interplay of overlapping instruments and other distracting objects. To accurately locate surgical instruments from this noisy background, this invention introduces an attention enhancement mechanism. However, most existing attention enhancement mechanisms generate one-dimensional or two-dimensional weights based on feature inputs and then extend them into channel attention or spatial attention. Conventional channel or spatial attention often relies on additional fully connected layers or large-size convolutions, which increases the risk of overfitting and computational complexity. Therefore, this design abandons the weight injection fusion approach that relies on learnable parameters. Instead, it defines an energy function to evaluate the importance of each neuron, rather than learning additional weights.

[0056] For a deep attention feature map that includes global features of surgical instruments and background information, an energy function is defined to evaluate the importance of each neuron in the feature map, and the minimum energy function is found for each neuron.

[0057] Preferably, the importance evaluation of pixels in the deep attention feature map based on the energy function includes the following operations: Each pixel is treated as a neuron. The matching deviation between the current neuron and the target feature is calculated, and the activation deviation with the surrounding neurons is measured. The matching deviation and activation deviation are fused to obtain an energy value. The importance of the pixel is evaluated based on the energy value.

[0058] Calculate the first channel Energy value of each neuron The formula is expressed as: , in, Indicates the first in the channel One element; This represents the mean of all elements in the channel; This represents the variance of all elements in the channel; The energy balance constant is usually taken as... .

[0059] By solving the above energy function, the energy value of each neuron is obtained. The lower the value, the greater the difference between this neuron and its surrounding neurons, making it easier to distinguish from other neurons, and thus the more important the feature information it carries. The inverse of the energy value is then normalized using the sigmoid activation function to generate feature enhancement weights.

[0060] The normalized feature enhancement weights are multiplied element-wise with the input feature map to enhance key features of the instrument and suppress background noise. The output is an energy feature map with a size of 20×20×1024. : , in, Input feature map; For new feature maps; This represents element-wise multiplication.

[0061] In the complex context of clinical surgery, feature enhancement using energy functions can accurately filter out background interference beyond the instruments, rapidly locate the target instrument, and improve detection accuracy. Since no additional parameters are required, it does not add any computational burden. Therefore, it allows for lightweight deployment and is better suited to the operational needs of portable medical devices in clinical settings.

[0062] S3. Upsample the energy feature map, concatenate it with the shallow attention-enhanced feature map, and then perform attention enhancement processing to obtain a refined feature map; upsample the refined feature map, concatenate it with the enhanced feature map, and then perform attention enhancement again to obtain a large-size feature map; downsample the large-size feature map, concatenate it with the refined feature map, and then perform attention enhancement processing to obtain a medium-size feature map; downsample the medium-size feature map, concatenate it with the energy feature map, and then perform feature enhancement to obtain a small-size feature map.

[0063] The energy feature map is upsampled to a size of 40×40×1024 and concatenated along the channel dimension with a shallow attention-enhanced feature map of size 40×40×512 to obtain a first fused feature map of size 40×40×1536. Attention enhancement processing is then applied to the first fused feature map to eliminate redundant information introduced by the concatenation, generating a refined feature map of size 40×40×512.

[0064] The refined feature map of size 40×40×512 is upsampled to 80×80×512 and concatenated with the enhanced feature map of size 80×80×256 to obtain a second fused feature map of size 80×80×768. Attention enhancement processing is then applied to the second fused feature map to generate a large feature map of size 80×80×256. This large feature map can capture the detailed features of small surgical instruments and is responsible for the detection of small target surgical instruments, thus solving the problem of high false negative rates for small targets in traditional networks.

[0065] The large 80×80×256 feature map is downsampled to 40×40×512 and then stitched together with a refined 40×40×512 feature map to obtain a third fusion feature map of 40×40×1024. Attention enhancement is then applied to this third fusion feature map to generate a medium-sized 40×40×512 feature map. This is responsible for detecting surgical instruments in the medium-sized target area.

[0066] The 40×40×512 mid-sized feature map is downsampled to 20×20×1024 and concatenated with the 20×20×1024 energy feature map to obtain a 20×20×2048 fourth fusion feature map. Attention enhancement processing is then applied to this fourth fusion feature map to generate a 20×20×1024 small-sized feature map, which is responsible for detecting large-target surgical instruments.

[0067] By employing bidirectional feature fusion, high-level semantic information is fully combined with low-level detailed information, while downsampling ensures the lightweight nature of the entire network. This lightweight network is suitable for complex scenarios in surgical instrument detection where there are large differences in size and mutual occlusion, significantly improving the feature representation ability of small and slender targets.

[0068] S4. Input the large-size feature map, medium-size feature map and small-size feature map simultaneously into the oriented bounding box detection head, and perform classification, localization and angle regression detection through three parallel branches.

[0069] Traditional detection heads use a large number of 3×3 convolutional layers, performing calculations simultaneously in both spatial and channel dimensions when processing input feature maps. This results in significant parameter redundancy and resource allocation imbalance. (Parameter count) The formula is expressed as: , in, Input the number of channels; Number of output channels; This represents the kernel size.

[0070] To address the shortcomings of traditional detection heads, such as excessive non-core branches and wasted resources, this invention employs a directional bounding box detection head. Through three parallel branches, it corresponds to the three core tasks of surgical instrument classification, localization, and angle regression, and the convolutional layers are configured differently based on the complexity and importance of each task.

[0071] Preferably, the three-branch parallel branch includes: The classification branch uses a dual 1×1 convolutional structure to perform semantic classification of the target; the localization branch uses three layers of depthwise separable convolutions and a single layer of 1×1 convolutions to obtain the target's precise location and size information; the angle regression branch uses a single layer of depthwise separable convolutions and a single layer of 1×1 convolutions to predict the target's rotation angle.

[0072] Specifically, the classification branch receives feature maps of various sizes, performs semantic classification on the target, and outputs the probability distribution of belonging to a certain type of surgical instrument. Given the relatively low complexity of surgical instrument classification tasks, this branch employs a dual 1×1 convolutional layer structure. Through two rounds of channel dimension integration and feature optimization, it achieves rapid instrument category discrimination, minimizing parameters while ensuring classification accuracy.

[0073] The localization branch receives feature maps of various sizes and uses three layers of depthwise separable convolutions and a single 1×1 convolution to predict the center point coordinates and size parameters of the oriented bounding box, accurately defining the spatial range of the surgical instrument and providing precise location and size information of the target surgical instrument in the image. Specifically, fine-grained features related to the instrument's position are first extracted through depthwise separable convolutions, and then the number of channels is adjusted through a 1×1 convolutional layer to output the bounding box coordinate parameters, achieving accurate localization.

[0074] The angle regression branch receives feature maps of various sizes and predicts the rotation angle of the surgical instrument around the center point through one depthwise separable convolution and one 1×1 convolution. In particular, angle-related features are extracted through depthwise separable convolution, and the predicted rotation angle is output through a 1×1 convolutional layer, balancing accuracy and lightweight design.

[0075] By configuring depthwise separable convolutions and 1×1 convolutions differently for different task branches, it is possible to adapt to the characteristics of each branch and meet its detection accuracy, effectively suppressing feature interference and loss coupling between tasks.

[0076] Compared to standard 3×3 convolution, depthwise separable convolution has fewer parameters. Greatly reduced, satisfying: , Therefore, the present invention uses an oriented bounding box detection head to replace the original detection head, which can effectively reduce the number of parameters and computational overhead, while not significantly affecting the detection accuracy, thus balancing real-time requirements and detection performance.

[0077] To verify the feasibility of this invention, simulation experiments were conducted. The simulation used a server running Ubuntu 22.04 Linux, equipped with an Intel(R) Xeon(R) Platinum 8481C CPU, an NVIDIA GeForce RTX 4090 GPU, and 90GB of RAM. The experiments employed Python 3.12.3 programming language, PyTorch 2.5.1 deep learning framework, and the CUDA 12.4 parallel computing platform.

[0078] All experiments used the same hyperparameters, including: 300 training epochs, input images uniformly scaled to 640×640 pixels, batch size of 8; the optimizer used stochastic gradient descent with momentum, momentum coefficient set to 0.937, initial learning rate of 0.01, and weight decay coefficient of 0.0005.

[0079] Figure 1 The image illustrates the visualized detection results of this invention in a typical scenario. The visualized image contains several oriented bounding boxes, each corresponding to a surgical instrument. A predefined one-to-one mapping relationship is established between each category and the bounding box color, such as... Figure 4 As shown. Figure 1 This demonstrates that the present invention can accurately detect all instruments in general scenarios without any false detections or missed detections, fully reflecting the high robustness of the present invention in routine clinical environments.

[0080] Figure 2The invention demonstrates the visualized detection results in complex scenarios. Even with instruments occluding each other and densely packed together, the invention successfully detected both occluded and overly dense instruments.

[0081] Figure 3 The invention exhibits a small number of missed detections in challenging scenarios, specifically missing 2 out of 22 surgical instruments. Overall, the invention demonstrates good detection performance in various clinical settings, but there is still room for improvement in extremely complex situations.

[0082] To verify the superiority of this invention in surgical instrument counting tasks, it was rigorously compared with other mainstream algorithms.

[0083] Compared to Algorithm 1, based on the classic YOLOv8 architecture, the rotating box detection function is achieved by adding an independent angle prediction branch at the end of the detection head.

[0084] Compared to Algorithm 2, the YOLOv10 object detection framework is used, and spatial-channel decoupled downsampling and a lightweight directional bounding box classification head are employed to further improve the overall running efficiency of the model.

[0085] Compared to Algorithm 3, this algorithm uses the YOLOv11 object detection framework and a directional bounding box detection head, and enhances the ability to extract orientation-sensitive features with a spatial attention module.

[0086] Compared with Algorithm 3, it adopts the YOLOv12 object detection framework and directional bounding box detection head. By introducing a region attention mechanism, the computational complexity of attention is greatly reduced. At the same time, a residual high-efficiency layer aggregation network is used to optimize the multi-scale feature aggregation process.

[0087] Algorithm 4 is a multi-scale enhancement improvement model based on the YOLO architecture. By introducing a super-resolution feature fusion module and a multi-scale context awareness module, it significantly improves the network's ability to represent features of targets of different sizes, and is particularly good at detecting mixed target scenes with large size differences.

[0088] Compared to Algorithm 5, it introduces a linear complexity state-space model into the YOLO architecture and replaces the traditional convolutional backbone network with the ODMamba backbone network. This enables efficient modeling of long-distance dependencies in images and shows significant advantages in scenarios with dense target occlusion and complex background interference.

[0089] This invention was compared with the comparison algorithm in terms of detection precision, recall, and mean precision (mAP). The mean precision was expressed as mAP@0.5, using an intersection-union ratio (IU) threshold of 0.5.

[0090] Table 1 compares the precision, recall, and mAP@0.5 of different algorithms.

[0091] As shown in Table 1, the present invention demonstrates superior performance in terms of detection accuracy, outperforming the comparative methods overall. Specifically, the mAP@0.5 of this algorithm reaches 95.3%, a 2.1% improvement over comparative algorithm 2. In contrast, comparative algorithms 5 and 6 attempted to improve accuracy by increasing the number of parameters, achieving mAP@0.5 of 94.8% and 95.0% respectively, but neither surpassed the accuracy of the present invention.

[0092] Recall rate is a crucial indicator for surgical counts. Compared to algorithms 1-4, whose recall rates hovered between 86.2% and 87.5%, and algorithms 5-6, which did not exceed 88.5%, the recall rate of this invention reached 89.5%. This means that when faced with targets with high similarity and high occlusion rates, such as surgical scissors and forceps, this invention has the lowest risk of missed detection.

[0093] Although the accuracy of this invention is slightly lower than that of other methods, this slight decrease is a controllable technological trade-off made to prioritize ensuring the core requirement of zero missed detections in clinical surgery.

[0094] Table 1: Comparison of Precision, Recall, and mAP@0.5 of Different Algorithms Table 2 compares the number of parameters, computational cost, and model size of different algorithms. Compared to Algorithm 1, the 8.3G computational power requirement is excessive for edge medical devices. Although Algorithm 2 reduces the number of parameters to 2.66M, its computational cost is still as high as 8.1G, indicating that its architecture has a large amount of computational redundancy. This invention, by introducing lightweight downsampling and attention enhancement mechanisms, compresses the number of parameters to 1.77M, making it the only algorithm in the table below 2.0M. At the same time, this invention achieves an extremely low computational cost of 4.1G, approximately 2 / 5 of that of Algorithm 5 and 1 / 3 of that of Algorithm 6. In terms of model size, this invention occupies only 4.04MB of storage space, exhibiting excellent lightweight characteristics and can be efficiently deployed on mobile devices.

[0095] Table 2: Comparison of parameter count, computational cost, and model size for different algorithms Furthermore, to analyze the contribution of different functions in this invention to the detection performance of surgical instruments, ablation experiments were conducted, as shown in Table 3. In Table 3, A represents the downsampling processing in this invention, L represents the directional bounding box detection in this invention, S represents the attention feature enhancement processing based on the energy function in this invention, √ indicates the corresponding processing method is used, and × indicates the ordinary processing method is used.

[0096] Table 3 Ablation Experiment As can be seen from Table 3, Method 2, by applying the improved downsampling operation of the present invention, reduces the number of parameters by 17.48% and the G by 0.7 while improving mAP@0.5 by 0.9% compared to Method 1.

[0097] In the experiment of Method 3, after replacing the traditional detection head with the lightweight detection head of the present invention, the mAP@0.5 was sacrificed by 0.3% compared with Method 1, but in return, the parameters were reduced by 12.86% and the computation was reduced by 1.0G.

[0098] Method 4 uses an energy function to enhance attention features. This attention function has no parameters, so it does not increase the model's parameters or computational cost.

[0099] Method 5 incorporates the improved downsampling and directional bounding box detection processing of this invention, achieving an mAP@0.5 that is 0.7% higher than Method 1, with the lowest parameters and computational cost observed in the ablation experiments. Compared to Method 1, the parameters are reduced by 31.41%, and the computational cost is reduced by 32.8%. Simultaneously, the model size is compressed from 5.55MB to 4.04MB, effectively lowering the deployment threshold and hardware cost.

[0100] Method 6, building upon Method 5, incorporates energy function-based attention feature enhancement, resulting in a mAP@0.5 that reaches the highest value of 95.3% in this experiment. While introducing energy function-based attention feature enhancement alone improves precision by 1.2% compared to Method 1, Method 6 proactively adjusts the classification decision boundary to prioritize reducing false negatives. Therefore, the precision in Method 6 decreases by 2.1% compared to Method 5, but this precision is still higher than Method 1's 91.2%, indicating no performance degradation. Compared to Method 5, the decrease in precision in Method 6 represents a controllable technical trade-off to meet the core requirement of low false negative rates in clinical surgery. This trade-off results in a 1.7% increase in recall and a 0.4% improvement in overall detection performance (mAP@0.5), while maintaining globally optimal levels of parameter and computational complexity, demonstrating a more significant overall advantage.

[0101] A surgical instrument testing system includes the following modules: Sample construction module: Acquire images of surgical instrument dataset, classify, locate and annotate the images using directional bounding boxes; perform data augmentation processing on the annotated images to obtain sample images.

[0102] Feature enhancement module: Input sample image, perform convolution and feature extraction operations in sequence, and obtain enhanced feature map through downsampling; perform downsampling and attention enhancement processing on the enhanced feature map to obtain shallow attention enhanced feature map; perform downsampling and attention enhancement again on the shallow attention enhanced feature map to obtain deep attention enhanced feature map; evaluate the importance of pixels in the deep attention enhanced feature map based on the energy function to obtain the energy value of each pixel, and obtain energy feature map based on the energy value.

[0103] Feature fusion module: The energy feature map is upsampled, concatenated with the shallow attention-enhanced feature map, and then attention enhancement is performed to obtain a refined feature map; the refined feature map is upsampled, concatenated with the enhanced feature map, and then attention enhancement is performed again to obtain a large-size feature map; the large-size feature map is downsampled, concatenated with the refined feature map, and then attention enhancement is performed to obtain a medium-size feature map; the medium-size feature map is downsampled, concatenated with the energy feature map, and then feature enhancement is performed to obtain a small-size feature map.

[0104] The detection output module simultaneously inputs large-size, medium-size, and small-size feature maps into the oriented bounding box detection head, and performs classification, localization, and angle regression detection through three parallel branches.

[0105] Preferably, the feature enhancement module includes: The downsampling module performs pooling on the sample image and divides it into two parallel sub-branches evenly according to the number of channels. One sub-branch uses convolution to downsample and extract detailed features, while the other sub-branch performs max pooling to extract salient features. The two sub-branches are then concatenated to complete the downsampling operation.

[0106] Preferably, the feature enhancement module further includes: The feature map pixel importance evaluation module treats each pixel as a neuron, calculates the matching deviation between the current neuron and the target feature, and then measures the activation deviation with the surrounding neurons. The matching deviation and activation deviation are fused to obtain an energy value, and the importance of the pixel is evaluated based on the energy value.

[0107] Preferably, the feature enhancement module includes: The energy feature map construction module normalizes the inverse of the energy value using an activation function to generate feature weights; the feature weights are then multiplied element-wise with the sample image to obtain the energy feature map.

[0108] A surgical instrument testing device includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement a surgical instrument testing method of the present invention.

Claims

1. A surgical instrument detection method characterized by comprising: Includes the following steps: S1. Obtain a dataset of surgical instrument images, classify, locate, and annotate the images using directional bounding boxes; perform data augmentation on the annotated images to obtain sample images; S2. Input sample image, perform convolution and feature extraction operations in sequence, and obtain enhanced feature map by downsampling; The enhanced feature map is downsampled and attention-enhanced to obtain a shallow attention-enhanced feature map; the shallow attention-enhanced feature map is downsampled and attention-enhanced again to obtain a deep attention-enhanced feature map; the importance of pixels in the deep attention-enhanced feature map is evaluated based on the energy function to obtain the energy value of each pixel, and the energy feature map is obtained based on the energy value. S3. Upsample the energy feature map, concatenate it with the shallow attention-enhanced feature map, and then perform attention enhancement processing to obtain a refined feature map; upsample the refined feature map, concatenate it with the enhanced feature map, and then perform attention enhancement again to obtain a large-size feature map; downsample the large-size feature map, concatenate it with the refined feature map, and then perform attention enhancement processing to obtain a medium-size feature map; After downsampling the medium-sized feature map and concatenating it with the energy feature map, feature enhancement is performed to obtain a small-sized feature map; S4. Input the large-size feature map, medium-size feature map and small-size feature map simultaneously into the oriented bounding box detection head, and perform classification, localization and angle regression detection through three parallel branches.

2. The method of claim 1, wherein Downsampling in S2 includes the following operations: The sample image is pooled and then evenly divided into two parallel sub-branches according to the number of channels. One sub-branch uses convolution to downsample and extract detailed features, while the other sub-branch uses max pooling to extract salient features. The two sub-branches are spliced ​​together to complete the downsampling operation.

3. The method of claim 1, wherein In S2, the importance of pixels in the deep attention feature map is evaluated based on an energy function: Each pixel is treated as a neuron. The matching deviation between the current neuron and the target feature is calculated, and the activation deviation with the surrounding neurons is measured. The matching deviation and activation deviation are fused to obtain an energy value. The importance of the pixel is evaluated based on the energy value.

4. The method of claim 1, wherein The energy feature map in S2 includes the following operations: The reciprocal of the energy value is normalized using an activation function to generate feature weights; The feature weights are multiplied element-wise with the sample image to obtain the energy feature map.

5. The method of claim 1, wherein The three parallel branches in S4 include: The classification branch uses a dual 1×1 convolutional structure to perform semantic classification of the target; the localization branch uses three layers of depthwise separable convolutions and a single layer of 1×1 convolutions to obtain the target's precise location and size information; the angle regression branch uses a single layer of depthwise separable convolutions and a single layer of 1×1 convolutions to predict the target's rotation angle.

6. A surgical instrument detection system characterized by, Includes the following modules: Sample construction module: Acquires images of surgical instruments dataset, classifies, locates, and annotates the images using directional bounding boxes; performs data augmentation processing on the annotated images to obtain sample images; Feature enhancement module: Input sample image, perform convolution and feature extraction operations in sequence, and obtain enhanced feature map through downsampling; The enhanced feature map is downsampled and attention-enhanced to obtain a shallow attention-enhanced feature map. The shallow attention enhancement feature map is downsampled and enhanced again to obtain the deep attention enhancement feature map; the importance of each pixel in the deep attention feature map is evaluated based on the energy function to obtain the energy value of each pixel, and the energy feature map is obtained based on the energy value. Feature fusion module: Upsample the energy feature map, concatenate it with the shallow attention-enhanced feature map, and then perform attention enhancement processing to obtain a refined feature map; Upsample the refined feature map, concatenate it with the enhanced feature map, and then perform attention enhancement again to obtain a large-size feature map; Downsample the large-size feature map, concatenate it with the refined feature map, and then perform attention enhancement processing to obtain a medium-size feature map; After downsampling the medium-sized feature map and concatenating it with the energy feature map, feature enhancement is performed to obtain a small-sized feature map; The detection output module simultaneously inputs large-size, medium-size, and small-size feature maps into the oriented bounding box detection head, and performs classification, localization, and angle regression detection through three parallel branches.

7. A surgical instrument detection system according to claim 6 characterised in that The enhancement modules include: The downsampling module performs pooling on the sample image and divides it into two parallel sub-branches evenly according to the number of channels. One sub-branch uses convolution to downsample and extract detailed features, while the other sub-branch performs max pooling to extract salient features. The two sub-branches are then concatenated to complete the downsampling operation.

8. A surgical instrument detection system according to claim 6 characterised in that The enhancement modules include: The feature map pixel importance evaluation module treats each pixel as a neuron, calculates the matching deviation between the current neuron and the target feature, and then measures the activation deviation with the surrounding neurons. The matching deviation and activation deviation are fused to obtain an energy value, and the importance of the pixel is evaluated based on the energy value.

9. The surgical instrument detection system of claim 6, wherein the feature is a feature of the surgical instrument. The enhancement modules include: ​ The energy feature map construction module normalizes the inverse of the energy value using an activation function to generate feature weights; the feature weights are then multiplied element-wise with the sample image to obtain the energy feature map.

10. A surgical instrument detection apparatus, characterized by, It includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement a surgical instrument testing method as described in any one of claims 1-5.

Images