Surgical instrument classification method, device and equipment and storage medium

Abstract

The invention discloses a surgical instrument classification method and device, equipment and a storage medium, and relates to the technical field of image recognition, and the method comprises the steps: obtaining each surgical instrument image, and constructing a surgical instrument image data set based on the surgical instrument images and labels corresponding to the surgical instrument images; determining a feature extraction mechanism of the initial classification model based on a preset convolutional neural network, determining a feature fusion mechanism of the initial classification model based on a preset feature pyramid network structure, and constructing a target detection head of the initial classification model based on a preset shared convolution technology; and constructing an initial classification model according to the feature extraction mechanism, the feature fusion mechanism, the target detection head and a preset target detection network, and determining a target classification model based on the surgical instrument image data set and the initial classification model. In this way, the target classification model can be constructed through the feature extraction mechanism, the feature fusion mechanism and the target detection head, and the accuracy of surgical instrument classification is improved.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a surgical instrument classification method, device, equipment and storage medium. Background Art

[0002] During medical procedures, accurate and timely classification of surgical instruments is crucial to ensuring smooth operations and patient safety. Traditional methods for surgical instrument classification often rely on manual identification. Medical staff rely on experience and memory to classify surgical instruments, an inefficient, labor-intensive, and prone to misjudgment. Similar instruments may be confused, potentially leading to serious consequences and increased risk as the procedure progresses. In recent years, with the rapid development of deep learning and computer vision technologies, deep learning-based surgical instrument classification methods have demonstrated significant advantages and are gradually replacing traditional methods. Surgical instrument classification involves two key stages: feature extraction and classification. Current deep learning object detection algorithms fall into two categories: two-stage object detection and single-stage object detection. Two-stage detectors offer high accuracy but require significant computational overhead to process a large number of candidate regions. Single-stage methods, on the other hand, enable real-time detection but may result in higher false positives. When current deep learning-based object detection algorithms are applied to surgical instrument classification, the extracted semantic features of surgical instrument images are incomplete and lack modeling of the global environment due to the local receptive field limitations of convolutional neural networks. In addition, the detection network focuses on local feature learning and fails to fully utilize the global context information of surgical instrument images.

[0003] In summary, how to improve the accuracy of surgical instrument classification is a technical problem that needs to be solved urgently. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a surgical instrument classification method, device, equipment and storage medium, which can improve the accuracy of surgical instrument classification. The specific scheme is as follows:

[0005] In a first aspect, the present application provides a method for classifying surgical instruments, comprising:

[0006] Acquire images of various surgical instruments, and construct a surgical instrument image dataset based on the surgical instrument images and labels corresponding to the surgical instrument images;

[0007] Determining a feature extraction mechanism of an initial classification model based on a preset convolutional neural network, determining a feature fusion mechanism of the initial classification model based on a preset feature pyramid network structure, and constructing an object detection head of the initial classification model based on a preset shared convolution technique;

[0008] Construct the initial classification model according to the feature extraction mechanism, the feature fusion mechanism, the target detection head and a preset target detection network, and determine the target classification model based on the surgical instrument image dataset and the initial classification model, so as to classify surgical instruments based on the target classification model.

[0009] Optionally, the feature extraction mechanism for determining the initial classification model based on a preset convolutional neural network includes:

[0010] Determine a target squeeze-and-excitation attention mechanism based on a preset number condition of fully connected layers, and determine the feature extraction mechanism of the initial classification model based on the target squeeze-and-excitation attention mechanism, a preset residual connection technique, a preset depthwise separable convolution technique, a preset convolution operation, and a first preset weight, so that the target classification model extracts features of the surgical instrument image based on the feature extraction mechanism.

[0011] Optionally, before determining the feature fusion mechanism of the initial classification model based on a preset feature pyramid network structure, it further includes:

[0012] Respectively determine a first feature in a preset horizontal direction and a second feature in a preset vertical direction of the surgical instrument image based on a preset coordinate information embedding method;

[0013] Perform splicing processing on the first feature and the second feature based on a preset coordinate attention generation method to obtain an initial feature of the surgical instrument image, and generate a target coordinate attention map of the surgical instrument image based on a preset convolution operation, a preset activation function, and the initial feature;

[0014] Determine a target feature map of the surgical instrument image based on the target coordinate attention map, the surgical instrument image, and a second preset weight.

[0015] Optionally, the feature fusion mechanism for determining the initial classification model based on a preset feature pyramid network structure includes:

[0016] Determine a target convolution kernel based on a preset convolution kernel size condition and a preset convolution kernel number condition, and perform convolution processing on the target feature map with the target convolution kernel to obtain features to be fused;

[0017] Perform feature fusion processing on each of the features to be fused according to a preset splicing method and a third preset weight to obtain features to be adjusted;

[0018] Perform adjustment processing on the features to be adjusted based on a preset spatial attention mechanism to obtain the target features of the surgical instrument image, so as to determine the feature fusion mechanism of the initial classification model.

[0019] Optionally, the object detection head for constructing the initial classification model based on the preset shared convolution technology includes:

[0020] Determine the target convolution operation based on the preset group normalization technology, and construct the object detection head of the initial classification model based on the target convolution operation, the preset depthwise separable convolution technology, the preset regression convolution technology, the preset classification convolution technology, and the preset shared convolution technology.

[0021] Optionally, the constructing of the initial classification model according to the feature extraction mechanism, the feature fusion mechanism, the object detection head, and the preset object detection network includes:

[0022] Determine the target loss function corresponding to the preset object detection network based on the target gradient gain allocation strategy, and construct the initial classification model according to the target loss function, the feature extraction mechanism, the feature fusion mechanism, the object detection head, and the preset object detection network.

[0023] Optionally, the determining of the target classification model based on the surgical instrument image dataset and the initial classification model includes:

[0024] Adjust the surgical instrument images in the surgical instrument image dataset based on the preset image size to obtain the target surgical instrument image dataset, and divide the target surgical instrument image dataset based on the preset ratio condition to obtain a training set, a validation set, and a test set;

[0025] Using the training set, the validation set, and the test set, determine the target classification model based on the preset iteration control parameters, the preset stochastic gradient descent method, and the initial classification model;

[0026] Wherein, the preset iteration control parameters include any one or a combination of several of batch size, number of iterations, and learning rate.

[0027] In a second aspect, the present application provides a surgical instrument classification device, including:

[0028] A surgical instrument image dataset construction module, configured to obtain each surgical instrument image, and construct a surgical instrument image dataset based on the surgical instrument image and the label corresponding to the surgical instrument image;

[0029] A feature extraction mechanism determination module, configured to determine the feature extraction mechanism of the initial classification model based on a preset convolutional neural network, determine the feature fusion mechanism of the initial classification model based on a preset feature pyramid network structure, and construct the object detection head of the initial classification model based on a preset shared convolution technology;

[0030] The surgical instrument classification module is used to construct the initial classification model according to the feature extraction mechanism, the feature fusion mechanism, the target detection head and a preset target detection network, and determine the target classification model based on the surgical instrument image dataset and the initial classification model, so as to classify surgical instruments based on the target classification model.

[0031] In a third aspect, the present application provides an electronic device, including:

[0032] A memory for storing a computer program;

[0033] A processor for executing the computer program to implement the foregoing surgical instrument classification method.

[0034] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the foregoing surgical instrument classification method is implemented.

[0035] In the present application, first, each surgical instrument image is obtained, and a surgical instrument image dataset is constructed based on the surgical instrument image and the label corresponding to the surgical instrument image; then, the feature extraction mechanism of the initial classification model is determined based on a preset convolutional neural network, and the feature fusion mechanism of the initial classification model is determined based on a preset feature pyramid network structure, and the target detection head of the initial classification model is constructed based on a preset shared convolution technology; finally, the initial classification model is constructed according to the feature extraction mechanism, the feature fusion mechanism, the target detection head and a preset target detection network, and the target classification model is determined based on the surgical instrument image dataset and the initial classification model, so as to classify surgical instruments based on the target classification model. As can be seen from the above, in the present application, the feature extraction mechanism of the model is determined based on a preset convolutional neural network; the feature fusion mechanism of the model is determined based on a preset feature pyramid network structure; the target detection head of the model is constructed based on a preset shared convolution technology, then the initial classification model is constructed according to the feature extraction mechanism, the feature fusion mechanism, the target detection head and a preset target detection network, and then the initial classification model is trained using the pre-obtained surgical instrument image dataset to obtain the target classification model, so as to classify surgical instruments based on the target classification model. In this way, the present invention effectively fuses the feature expressions under different extraction capabilities through the feature extraction mechanism, and strongly promotes the detailed fusion of multi-level features; promotes feature aggregation through the feature fusion mechanism, and improves accuracy and robustness; the target detection head can reduce the number of model parameters and complexity, and improve the classification and localization effects. In this way, the target classification model constructed by the feature extraction mechanism, the feature fusion mechanism and the target detection head in the present application can improve the accuracy of surgical instrument classification. Description of the Drawings

[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on the provided drawings.

[0037] Figure 1 It is a system architecture diagram of a surgical instrument classification scheme provided by this application;

[0038] Figure 2 It is a flowchart of a surgical instrument classification method provided by this application;

[0039] Figure 3 It is a schematic diagram of a specific C2f-RMBC module provided by this application;

[0040] Figure 4 It is a schematic diagram of a specific CA-HSFPN module provided by this application;

[0041] Figure 5 It is a schematic diagram of a specific CA attention module provided by this application;

[0042] Figure 6 It is a schematic diagram of a specific SFF module provided by this application;

[0043] Figure 7 It is a schematic diagram of a specific LGN-Detect module provided by this application;

[0044] Figure 8 It is a flowchart of a specific surgical instrument classification method provided by this application;

[0045] Figure 9 It is a schematic diagram of the structure of a surgical instrument classification device provided by this application;

[0046] Figure 10 It is a structural diagram of an electronic device provided by this application. Detailed implementation manners

[0047] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0048] In recent years, with the rapid development of deep learning and computer vision technologies, the classification method of surgical instruments based on deep learning has shown obvious advantages and gradually replaced traditional methods. The classification of surgical instruments includes two key stages: feature extraction and classification. Currently, deep learning object detection algorithms are divided into two categories: two-stage object detection and one-stage object detection. Two-stage detectors have high accuracy, but processing a large number of candidate regions requires a huge computational load. While one-stage methods can achieve real-time detection, there may be more false positives. When current deep learning-based object detection algorithms are applied to the classification of surgical instruments, due to the local receptive field limitation of the convolutional neural network, the semantic feature expression of the extracted surgical instrument images is incomplete, lacking the modeling of the global environment; and the detection network focuses on local feature learning and fails to fully utilize the global context information of surgical instrument images. Therefore, this application provides a surgical instrument classification scheme that can improve the accuracy of surgical instrument classification.

[0049] In the surgical instrument classification scheme of this application, the specific system framework adopted can be referred to Figure 1 as shown. Specifically, this application constructs a surgical instrument classification model based on the C2f-RMBC module, CA-HSFPN module, LGN-Detect module, and YOLOv8 network (YOLO, that is, You Only Look Once). Among them, the SE attention mechanism (i.e., Squeeze-and-Excitation, squeeze excitation attention mechanism) is added to the C2f-RMBC module to extract the features of surgical instrument images; the CA attention mechanism (i.e., Coordinate Attention, coordinate attention mechanism) is added to the HSFPN pyramid feature fusion module (i.e., High spatial Frequency Pyramid network) to select and fuse the features of surgical instrument images; the dynamic detection head LGN-Detect combines grouped convolution and depthwise separable convolution (DWConv, that is, Depthwise Convolution) to perform object detection based on the features output by the surgical instrument feature extraction module and the feature fusion module, and outputs the recognition result.

[0050] Refer to Figure 2 as shown, an embodiment of the present invention discloses a surgical instrument classification method, which may include:

[0051] Step S11, obtain each surgical instrument image, and construct a surgical instrument image data set based on the surgical instrument image and the label corresponding to the surgical instrument image.

[0052] In this embodiment, in order to improve the generalization ability of the dataset, surgical instrument images including various surgical instrument types, different angles, lighting conditions, and backgrounds can be obtained. After that, the surgical instrument images need to be data-annotated to determine the labels corresponding to the surgical instrument images. Finally, a corresponding surgical instrument image dataset is constructed using the surgical instrument images and the labels corresponding to the surgical instrument images.

[0053] Step S12: Determine the feature extraction mechanism of the initial classification model based on a preset convolutional neural network, determine the feature fusion mechanism of the initial classification model based on a preset feature pyramid network structure, and construct the object detection head of the initial classification model based on a preset shared convolution technique.

[0054] It can be understood that in YOLOv8, the C3 module of YOLOv5 (C3, that is, CSP Bottleneck with 3 convolutions) is replaced by the C2f module (C2f, that is, Channel-to-Pixel). The C2f module integrates the concepts of the C3 module and ELAN (that is, Efficient Lightweight Attention Network), and adopts a supplementary branch structure to strengthen the auxiliary flow during gradient backpropagation. However, the C2f module will increase the computational complexity and the number of parameters. To comprehensively balance the model performance and resource utilization, the C2f-RMBC module is introduced in this embodiment. Specifically, to reduce the number of parameters of the model and minimize the impact on the model performance, in this embodiment, the MBConv module (MBConv, that is, mobile inverted bottleneck convolution) is improved according to the residual structure and weight fusion strategy, and the improved MBConv module is named the RMBConv module. The RMBConv module contains an effective SE attention mechanism and a split-weight depthwise separable convolution path, which are used to replace the bottleneck part in the improved C2f module. Among them, to reduce the computational cost associated with the C2f module, in this embodiment, an efficient squeeze-and-excitation attention mechanism is integrated into the MBConv module. The SE attention mechanism, as a prominent channel attention technology, is integrated into the CNN architecture (CNN, that is, Convolutional Neural Network), which can explicitly capture the interdependencies between the feature map channels, thereby enhancing the feature representation ability. The traditional SE attention mechanism will lose channel information when the dimension is reduced. To avoid the problems caused by dimension reduction while retaining the channel information and thus improve the performance, in this embodiment, an effective-SE of the FC layer (Fully Connected Layer) with only one input feature channel, that is, the C channel, is selected to replace the SE.

[0055] See Figure 3As shown, in this embodiment, the feature extraction mechanism for determining the initial classification model based on a preset convolutional neural network may include: determining a target squeeze-and-excitation attention mechanism based on a preset number of fully connected layer conditions, and determining the feature extraction mechanism of the initial classification model based on the target squeeze-and-excitation attention mechanism, a preset residual connection technique, a preset depthwise separable convolution technique, a preset convolution operation, and a first preset weight, so that the target classification model extracts the features of the surgical instrument image based on the feature extraction mechanism. It can be understood that the SE attention module uses global average pooling to compress the spatial dependence relationship, derives a unique channel descriptor, and then scales the input feature map through two FC layers and a sigmoid function, giving priority to and only emphasizing relevant channels. However, the SE attention module has a limitation that dimensionality reduction will cause loss of channel information. To avoid unnecessary increase in model complexity, the two FC layers in the SE attention module must reduce the channel dimension. Specifically, the primary FC layer reduces the C channels to C / r by applying a dimensionality reduction ratio r, and the subsequent FC layer restores the reduced C channels to the original size of the C channels. Information loss of channels will occur during the process of reducing the dimensionality of channels. Therefore, in this embodiment, only one FC layer with C channels is selected, and effective-SE is used as a substitute for SE. Specifically, the operation of the C2f-RMBC module can be expressed as:

[0056] ;

[0057] ;

[0058] ;

[0059] ;

[0060] ;

[0061] ;

[0062] ;

[0063] where represents the feature information passed into the C2f-RMBC module and serves as the input to the C2f-RMBC module, represents the image features extracted after being processed by the C2f-RMBC module, represents the output after the input passes through a 1×1 convolution, , represent the outputs after evenly dividing the number of F1 channels, , , respectively represent the outputs passing through the RMBConv module in sequence. Specifically, the operations of the RMBConv module can be expressed as:

[0064] ;

[0065] ;

[0066] ;

[0067] ;

[0068] Among them, represents the feature information passed into the Backbone part of the C2f module, serving as the input of the RMBConv module, represents the image features extracted after the RMBConv module processes, represents the output after the input passes through 1×1 convolution, 3×3 depthwise separable convolution, SE attention mechanism, and 1×1 convolution, represents the output after passing through 1×1 convolution, 3×3 depthwise separable convolution, 1×1 convolution and with a weight sharing of 0.2, represents the output of the main path of the residual structure with a weight sharing of 0.8. The depthwise separable convolution can reduce the computational amount and accelerate training, and the residual structure skip connection can ensure the smooth propagation of gradients and reduce risks. Moreover, through weight sharing, the RMBConv module can better perform feature fusion, thereby obtaining the features of the surgical instrument image.

[0069] See Figure 4 As shown, in this embodiment, CA-HSFPN is used to replace the neck structure in the YOLOv8 network to extract features of different scales and perform context aggregation to improve the accuracy and robustness of object detection. CA-HSFPN introduces two key components in YOLOV8: the CA module and the HSFPN module. The CA module is used to perform context aggregation on feature maps of different scales. Specifically, the CA module captures context information of different scales by introducing multiple convolution kernels of different sizes and performing multi-scale convolution operations on the feature maps. In this way, it can help the model better understand the context semantics of the object and improve the accuracy of object detection. The HSFPN module is used to extract high-spatial-frequency features. Specifically, the HSFPN module also captures the detailed information of the object by introducing multiple convolution kernels of different sizes and performing multi-scale convolution operations on the feature maps, which helps the model better detect small-size objects or the detailed parts of the object, thereby improving the robustness of object detection.

[0070] See Figure 5As shown, before determining the feature fusion mechanism of the initial classification model based on the preset feature pyramid network structure, it may further include: respectively determining the first feature of the surgical instrument image in the preset horizontal direction and the second feature in the preset vertical direction based on the preset coordinate information embedding method; performing splicing processing on the first feature and the second feature based on the preset coordinate attention generation method to obtain the initial feature of the surgical instrument image, and generating the target coordinate attention map of the surgical instrument image based on the preset convolution operation, preset activation function, and the initial feature; determining the target feature map of the surgical instrument image based on the target coordinate attention map, the surgical instrument image, and the second preset weight. Specifically, the CA attention module mainly consists of three parts: coordinate information embedding, coordinate attention generation, and feature fusion. In the coordinate information embedding part, global average pooling operations are respectively performed on the input surgical instrument image in the horizontal and vertical directions to embed the position information into the channel attention to obtain the feature descriptions of the surgical instrument image in the horizontal and vertical directions, that is, the first feature and the second feature. Then, in the coordinate attention generation part, the first feature in the horizontal direction and the second feature in the vertical direction are first spliced, and then a series of convolution operations and activation functions are used to generate the target coordinate attention map, which can dynamically adjust the importance of different positions and channels. Finally, in the feature fusion part, the generated target coordinate attention map is fused with the input surgical instrument image based on the second preset weight to achieve weighted fusion of features to obtain the target feature map, so that the model can focus on important regions and features and improve the performance of the model. It can be seen that the working principle of the CA attention module is: first, the position information is incorporated into the channel attention through coordinate information embedding, enabling the model to better perceive the spatial position relationship in the image; then, the coordinate attention generation part uses operations such as splicing, convolution, and activation functions to generate a coordinate attention map with direction perception ability; finally, through feature fusion, the coordinate attention map is weighted and fused with the input feature map to enhance the model's attention to important regions and features.

[0071] See Figure 6As shown, the feature fusion mechanism for determining the initial classification model based on the preset feature pyramid network structure may include: determining a target convolution kernel based on the preset convolution kernel size condition and the preset convolution kernel number condition, and performing convolution processing on the target feature map with the target convolution kernel to obtain features to be fused; performing feature fusion processing on each of the features to be fused according to the preset splicing method and the third preset weight to obtain features to be adjusted; adjusting the features to be adjusted based on the preset spatial attention mechanism to obtain the target features of the surgical instrument image, so as to determine the feature fusion mechanism of the initial classification model. Specifically, the SFF module (i.e., Scale-Fusion Feature) consists of multi-scale feature extraction, feature fusion, spatial attention mechanism, and result output. Among them, the multi-scale feature extraction part performs multi-scale operations on the input target feature map, and can obtain the feature information of the target feature map at different levels through convolution kernels or pooling operations of different sizes; the feature fusion part fuses features of different scales through methods such as splicing and weighted summation to comprehensively utilize information at different levels; the spatial attention mechanism will perform spatial weighted adjustment on the fused features, and can dynamically adjust the weights of different positions according to the importance of the spatial position of the features, highlighting important regions and suppressing irrelevant regions. The target features obtained after adjustment are used as the output results for subsequent network processing. The SFF module has functional characteristics such as multi-scale fusion, spatial attention enhancement, and high efficiency. Multi-scale fusion can effectively fuse features of different scales and enrich the feature expression ability; by focusing on important regions through the spatial attention mechanism, the feature pertinence and accuracy are improved, which helps to improve the performance of the network for target detection and target recognition.

[0072] In this embodiment, the target detection head for constructing the initial classification model based on the preset shared convolution technology may include: determining a target convolution operation based on the preset group normalization technology, and constructing the target detection head of the initial classification model based on the target convolution operation, the preset depthwise separable convolution technology, the preset regression convolution technology, the preset classification convolution technology, and the preset shared convolution technology. Specifically, the Head part of the original YOLOv8 model network has a relatively large change compared with YOLOv5. The structure contains 3 detection heads for detecting different sizes. At the same time, the concept of decoupled head is introduced, directly changing the coupled head to a decoupled head structure similar to YOLOX, and dividing the detection head into two branches of Cls classification and Box regression, and removing the calculation branch of Obj_loss, which also makes the parameters of the detection head increase significantly. See Figure 7As shown in (a), the input of each detection head is processed by 3 convolutions in two branches and then concatenated for output. Compared with batch normalization (BN), group normalization (GN) shows superior performance in terms of accuracy improvement. Therefore, in this embodiment, a new convolutional structure combining a convolutional layer and group normalization, namely Conv_GN, is proposed. Refer to Figure 7 As shown in (b), in this embodiment, a structure with three detection heads is constructed, and the results are aggregated using a shared convolutional path. Specifically, each dimension of the detection head route includes a 1×1 group convolution, a 3×3 group convolution, a 1×1 depthwise separable convolution, a 3×3 group convolution, and finally a parallel regression convolution, namely Conv_reg, and a classification convolution, namely Conv_cls. Introducing group convolution and adding depthwise separable convolution in the LGN-Detect module can reduce network parameters, improve the operation rate, and achieve high real-time performance.

[0073] Step S13: Construct the initial classification model according to the feature extraction mechanism, the feature fusion mechanism, the target detection head, and a preset target detection network, and determine the target classification model based on the surgical instrument image dataset and the initial classification model, so as to classify surgical instruments based on the target classification model.

[0074] In this embodiment, the initial classification model can be constructed based on the C2f-RMBC module, the CA-HSFPN module, the LGN-Detect module, and the YOLOv8 network. Then, the initial classification model can be trained and verified based on the surgical instrument image dataset to obtain the target classification model, so as to apply the trained target classification model to surgical instrument classification.

[0075] As can be seen from the above, in this embodiment, first, images of various surgical instruments are obtained, and a surgical instrument image dataset is constructed based on the surgical instrument images and the labels corresponding to the surgical instrument images. Then, based on a preset convolutional neural network, the feature extraction mechanism of the initial classification model is determined, and based on a preset feature pyramid network structure, the feature fusion mechanism of the initial classification model is determined, and the object detection head of the initial classification model is constructed based on a preset shared convolution technique. Finally, the initial classification model is constructed according to the feature extraction mechanism, the feature fusion mechanism, the object detection head, and a preset object detection network, and a target classification model is determined based on the surgical instrument image dataset and the initial classification model, so as to classify surgical instruments based on the target classification model. As can be seen from the above, in this embodiment, the feature extraction mechanism of the model is determined based on a preset convolutional neural network; the feature fusion mechanism of the model is determined based on a preset feature pyramid network structure; the object detection head of the model is constructed based on a preset shared convolution technique. Then, the initial classification model is constructed according to the feature extraction mechanism, the feature fusion mechanism, the object detection head, and a preset object detection network. Then, the initial classification model is trained using the pre-obtained surgical instrument image dataset to obtain the target classification model, so as to classify surgical instruments based on the target classification model. In this way, in this embodiment, through the feature extraction mechanism, the feature expressions under different extraction capabilities are effectively fused, which strongly promotes the detailed fusion of multi-level features; through the feature fusion mechanism, feature aggregation is promoted, and the accuracy and robustness are improved; through the object detection head, the number of model parameters and complexity can be reduced, and the classification and positioning effects can be improved. In this way, the target classification model constructed by the feature extraction mechanism, the feature fusion mechanism, and the object detection head in this embodiment can improve the accuracy of surgical instrument classification.

[0076] See Figure 8 As shown, in order to improve the accuracy of surgical instrument classification using the target classification model, an embodiment of the present invention further discloses a surgical instrument classification method, which may include:

[0077] Step S21, obtain images of various surgical instruments, and construct a surgical instrument image dataset based on the surgical instrument images and the labels corresponding to the surgical instrument images.

[0078] Step S22, determine the feature extraction mechanism of the initial classification model based on a preset convolutional neural network, determine the feature fusion mechanism of the initial classification model based on a preset feature pyramid network structure, and construct the object detection head of the initial classification model based on a preset shared convolution technique.

[0079] Step S23: Construct the initial classification model according to the feature extraction mechanism, the feature fusion mechanism, the target detection head, and the preset target detection network, and determine the target classification model based on the surgical instrument image dataset and the initial classification model, so as to classify surgical instruments based on the target classification model.

[0080] It should be noted that in YOLOv8, two loss functions, DIoU (i.e., Distance-Intersection over Union) and CIoU (Complete Intersection over Union), are built-in. The DIoU loss function introduces the normalized distance between the center points as a penalty term and increases the negative gradient of the minimum bounding box size. However, this mechanism sometimes causes the size of the predicted box to increase, thereby reducing the overlap between the predicted box and the target box. The CIoU loss function further considers the aspect ratio consistency between the predicted box and the target box based on DIoU, but it does not solve the problem of low-quality and small-target samples in the training data well. Therefore, in this embodiment, the bounding box loss function WIoUv3 (WIoU, i.e., Weighted Intersection over Union) based on dynamically changing gradient gain is adopted to optimize the processing effect of different-sized targets in the target detection task, especially improving the detection performance of small targets, and it is more adaptable than the CIoU loss function and the DIoU loss function.

[0081] In this embodiment, the constructing the initial classification model according to the feature extraction mechanism, the feature fusion mechanism, the target detection head, and the preset target detection network may include: determining the target loss function corresponding to the preset target detection network based on the target gradient gain allocation strategy, and constructing the initial classification model according to the target loss function, the feature extraction mechanism, the feature fusion mechanism, the target detection head, and the preset target detection network. Specifically, there are three versions of the WIoU loss function. Wise-IoU v1 constructs distance attention based on distance metrics and applies two-layer attention mechanisms to itself, making the model have better generalization ability. The formula is as follows:

[0082] ;

[0083] where represents the high-quality anchor box loss, represents the position of the real-time anchor box corresponding to the position of the target box , , are the height and width of the minimum bounding box respectively, It is represented as a separation operation. Afterwards, it is constructed in Wise-IoU v2 Monotonic focusing coefficient , is the weight parameter used to control the position penalty term. In the case of Multiplying them together gives us the Wise-IoU v2 loss function, which is as follows:

[0084] ;

[0085] in, express However, during model training, as the IoU (Intersection over Union) decreases, the gradient gain It also decreases, resulting in slower convergence in the later stages of training. Therefore, the mean IoU can be introduced As a normalization factor, the formula is as follows:

[0086] ;

[0087] In Wise-IoU v3, the concept of outlier is defined to describe the quality of the anchor box. The formula is as follows:

[0088] ;

[0089] Therefore, the Wise-IoU v3 loss function can be specifically:

[0090] ;

[0091] Among them, when hour, Make . is a constant, is a dynamic adjustment factor. When IoU is small, for example, the predicted box does not overlap with the real box or the overlap is small, Increase, forcing the model to prioritize optimizing aspect ratio differences; when IoU is large, Reduce, reduce the weight of shape penalty, focus on position matching. At the same time, when the degree of outlier of the anchor box meets (C is a constant), the anchor box will obtain the highest gradient gain. It is dynamic, and the quality division criteria of the anchor boxes are also dynamically adjusted. Therefore, WIoU-v3 can adopt the most suitable gradient gain allocation strategy at each moment. In this embodiment, an initial classification model is constructed based on the WIoU-v3 loss function, the feature extraction mechanism, the feature fusion mechanism, the object detection head, and the YOLOv8 network.

[0092] It should be noted that determining the target classification model based on the surgical instrument image dataset and the initial classification model may include: adjusting the surgical instrument images in the surgical instrument image dataset based on a preset image size to obtain a target surgical instrument image dataset, and dividing the target surgical instrument image dataset based on preset ratio conditions to obtain a training set, a validation set, and a test set; using the training set, the validation set, and the test set, and determining the target classification model based on preset iteration control parameters, a preset stochastic gradient descent method, and the initial classification model; where the preset iteration control parameters include any one or several combinations of batch size, number of iterations, and learning rate. Specifically, the size of the surgical instrument images is adjusted to 640px×640px to obtain a target surgical instrument image dataset; then, the target surgical instrument image dataset is divided based on preset ratio conditions to obtain a training set, a validation set, and a test set. In a specific implementation manner, the number of input images per batch in the iteration control parameters, that is, the batch size, can be set to 16, the number of iterations is 500 rounds, and the learning rate is 0.01. The stochastic gradient descent method is selected as the optimization algorithm, and the initial classification model is trained based on the iteration control parameters, the stochastic gradient descent method, and the training set, and is verified using the validation set, and the performance of the model is evaluated using the test set. Finally, the model with the best result on the test set is obtained as the target classification model.

[0093] Among them, for the more specific processing procedures of the above steps S21 and S22, reference can be made to the corresponding content disclosed in the foregoing embodiments, and details will not be repeated here.

[0094] As can be seen from the above, in this embodiment, an initial classification model is constructed based on the WIoU-v3 loss function, the feature extraction mechanism, the feature fusion mechanism, the object detection head, and the YOLOv8 network. Then, the target surgical instrument image dataset is determined, and the target surgical instrument image dataset is divided to obtain a training set, a validation set, and a test set; then, the target classification model is determined based on the training set, the validation set, the test set, the preset iteration control parameters, the preset stochastic gradient descent method, and the initial classification model. In this way, the accuracy and recall rate of the target classification model can be improved in this embodiment.

[0095] Correspondingly, as shown in Figure 9 this application embodiment also provides a surgical instrument classification device, which may include:

[0096] The surgical instrument image dataset construction module 11 is used to obtain images of each surgical instrument and construct a surgical instrument image dataset based on the surgical instrument images and the labels corresponding to the surgical instrument images;

[0097] The feature extraction mechanism determination module 12 is used to determine the feature extraction mechanism of the initial classification model based on a preset convolutional neural network, determine the feature fusion mechanism of the initial classification model based on a preset feature pyramid network structure, and construct the object detection head of the initial classification model based on a preset shared convolution technique;

[0098] The surgical instrument classification module 13 is used to construct the initial classification model according to the feature extraction mechanism, the feature fusion mechanism, the object detection head and a preset object detection network, and determine the target classification model based on the surgical instrument image dataset and the initial classification model, so as to classify surgical instruments based on the target classification model.

[0099] As can be seen from the above, in this application, first, images of each surgical instrument are obtained, and a surgical instrument image dataset is constructed based on the surgical instrument images and the labels corresponding to the surgical instrument images; then, the feature extraction mechanism of the initial classification model is determined based on a preset convolutional neural network, the feature fusion mechanism of the initial classification model is determined based on a preset feature pyramid network structure, and the object detection head of the initial classification model is constructed based on a preset shared convolution technique; finally, the initial classification model is constructed according to the feature extraction mechanism, the feature fusion mechanism, the object detection head and a preset object detection network, and the target classification model is determined based on the surgical instrument image dataset and the initial classification model, so as to classify surgical instruments based on the target classification model. As can be seen from the above, in this application, the feature extraction mechanism of the model is determined based on a preset convolutional neural network; the feature fusion mechanism of the model is determined based on a preset feature pyramid network structure; the object detection head of the model is constructed based on a preset shared convolution technique, then the initial classification model is constructed according to the feature extraction mechanism, the feature fusion mechanism, the object detection head and a preset object detection network, and then the initial classification model is trained using the pre-obtained surgical instrument image dataset to obtain the target classification model, so as to classify surgical instruments based on the target classification model. In this way, the present invention effectively fuses the feature expressions under different extraction capabilities through the feature extraction mechanism, and strongly promotes the detailed fusion of multi-level features; promotes feature aggregation through the feature fusion mechanism, and improves the accuracy and robustness; the object detection head can reduce the number of model parameters and complexity, and improve the classification and positioning effects. In this way, the target classification model constructed by the feature extraction mechanism, the feature fusion mechanism and the object detection head in this application can improve the accuracy of surgical instrument classification.

[0100] In some specific embodiments, the feature extraction mechanism determination module 12 may include:

[0101] A feature extraction unit, configured to determine a target squeeze-and-excitation attention mechanism based on a preset number condition of fully connected layers, and determine the feature extraction mechanism of the initial classification model based on the target squeeze-and-excitation attention mechanism, a preset residual connection technique, a preset depthwise separable convolution technique, a preset convolution operation, and a first preset weight, so that the target classification model extracts features of the surgical instrument image based on the feature extraction mechanism.

[0102] In some specific embodiments, the surgical instrument classification device may further include:

[0103] A feature determination module, configured to respectively determine a first feature in a preset horizontal direction and a second feature in a preset vertical direction of the surgical instrument image based on a preset coordinate information embedding method;

[0104] A target coordinate attention map generation module, configured to perform splicing processing on the first feature and the second feature based on a preset coordinate attention generation method to obtain an initial feature of the surgical instrument image, and generate a target coordinate attention map of the surgical instrument image based on a preset convolution operation, a preset activation function, and the initial feature;

[0105] A target feature map determination module, configured to determine a target feature map of the surgical instrument image based on the target coordinate attention map, the surgical instrument image, and a second preset weight.

[0106] In some specific embodiments, the feature extraction mechanism determination module 12 may include:

[0107] A to-be-fused feature determination unit, configured to determine a target convolution kernel based on a preset convolution kernel size condition and a preset convolution kernel number condition, and perform convolution processing on the target feature map based on the target convolution kernel to obtain to-be-fused features;

[0108] A to-be-adjusted feature determination unit, configured to perform feature fusion processing on each of the to-be-fused features according to a preset splicing method and a third preset weight to obtain to-be-adjusted features;

[0109] A feature fusion mechanism determination unit, configured to perform adjustment processing on the to-be-adjusted features based on a preset spatial attention mechanism to obtain the target features of the surgical instrument image, so as to determine the feature fusion mechanism of the initial classification model.

[0110] In some specific embodiments, the feature extraction mechanism determination module 12 may include:

[0111] A target detection head construction unit is configured to determine a target convolution operation based on a preset group normalization technique, and construct the target detection head of the initial classification model based on the target convolution operation, a preset depthwise separable convolution technique, a preset regression convolution technique, a preset classification convolution technique, and the preset shared convolution technique.

[0112] In some specific embodiments, the surgical instrument classification module 13 may include:

[0113] An initial classification model construction unit is configured to determine a target loss function corresponding to the preset target detection network based on a target gradient gain allocation strategy, and construct the initial classification model according to the target loss function, the feature extraction mechanism, the feature fusion mechanism, the target detection head, and the preset target detection network.

[0114] In some specific embodiments, the surgical instrument classification module 13 may include:

[0115] A target surgical instrument image dataset partitioning unit is configured to adjust the surgical instrument images in the surgical instrument image dataset based on a preset image size to obtain a target surgical instrument image dataset, and partition the target surgical instrument image dataset based on a preset ratio condition to obtain a training set, a validation set, and a test set;

[0116] A target classification model determination unit is configured to use the training set, the validation set, and the test set to determine the target classification model based on a preset iteration control parameter, a preset stochastic gradient descent method, and the initial classification model; wherein, the preset iteration control parameter includes any one or a combination of several of batch size, number of iterations, and learning rate.

[0117] Furthermore, an embodiment of the present application also discloses an electronic device, Figure 10 It is a structural diagram of an electronic device 20 shown according to an exemplary embodiment, and the content in the figure cannot be considered as any limitation to the scope of use of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. Among them, the memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the surgical instrument classification method disclosed in any of the foregoing embodiments. Additionally, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0118] In this embodiment, the power supply 23 is used to provide operating voltages for the various hardware devices on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and specific limitations thereof are not provided herein; the input / output interface 25 is used to obtain external input data or output data to the outside, and the specific interface type thereof can be selected according to specific application requirements, and specific limitations are not provided herein.

[0119] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, a random access memory, a magnetic disk, an optical disc, etc., and the resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be transient storage or permanent storage.

[0120] Among them, the operating system 221 is used to manage and control the various hardware devices and the computer program 222 on the electronic device 20, and it can be Windows Server, Netware, Unix, Linux, etc. The computer program 222 can further include a computer program capable of performing other specific tasks in addition to the computer program capable of implementing the surgical instrument classification method executed by the electronic device 20 disclosed in any of the foregoing embodiments.

[0121] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the surgical instrument classification method disclosed above is implemented. For the specific steps of this method, reference can be made to the corresponding content disclosed in the foregoing embodiments, and details are not described herein again.

[0122] In this specification, the various embodiments are described in a progressive manner. Each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0123] Those skilled in the art can further realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of the examples have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0124] The steps of the methods or algorithms described in connection with the embodiments disclosed herein may be implemented directly in hardware, in software modules executed by a processor, or in a combination thereof. The software modules may be located in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium well-known in the art.

[0125] Finally, it should also be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0126] The technical solutions provided in this application have been introduced in detail above. Specific examples are used herein to illustrate the principles and implementation manners of this application. The description of the above embodiments is only used to help understand the method and its core idea of this application; at the same time, for those of ordinary skill in the art, according to the idea of this application, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to this application.

Claims

1. A method for classifying surgical instruments, characterized in that, Including: Obtain images of each surgical instrument, and construct a surgical instrument image dataset based on the surgical instrument images and the labels corresponding to the surgical instrument images; Determine the feature extraction mechanism of the initial classification model based on a preset convolutional neural network, determine the feature fusion mechanism of the initial classification model based on a preset feature pyramid network structure, and construct the object detection head of the initial classification model based on a preset shared convolution technique; Construct the initial classification model according to the feature extraction mechanism, the feature fusion mechanism, the object detection head and a preset object detection network, and determine the target classification model based on the surgical instrument image dataset and the initial classification model, so as to classify surgical instruments based on the target classification model.

2. The surgical instrument classification method according to claim 1, wherein The determining the feature extraction mechanism of the initial classification model based on a preset convolutional neural network includes: Determine a target squeeze-and-excitation attention mechanism based on a preset number of fully connected layer conditions, and determine the feature extraction mechanism of the initial classification model based on the target squeeze-and-excitation attention mechanism, a preset residual connection technique, a preset depthwise separable convolution technique, a preset convolution operation and a first preset weight, so that the target classification model extracts features of the surgical instrument image based on the feature extraction mechanism.

3. The surgical instrument classification method according to claim 1, characterized in that Before determining the feature fusion mechanism of the initial classification model based on a preset feature pyramid network structure, it further includes: Respectively determine a first feature of the surgical instrument image in a preset horizontal direction and a second feature of the surgical instrument image in a preset vertical direction based on a preset coordinate information embedding method; Perform splicing processing on the first feature and the second feature based on a preset coordinate attention generation method to obtain an initial feature of the surgical instrument image, and generate a target coordinate attention map of the surgical instrument image based on a preset convolution operation, a preset activation function and the initial feature; Determine a target feature map of the surgical instrument image based on the target coordinate attention map, the surgical instrument image and a second preset weight.

4. The surgical instrument classification method according to claim 3, characterized in that, The determining the feature fusion mechanism of the initial classification model based on a preset feature pyramid network structure includes: Determine a target convolution kernel based on a preset convolution kernel size condition and a preset convolution kernel number condition, and perform convolution processing on the target feature map with the target convolution kernel to obtain features to be fused; Perform feature fusion processing on each of the features to be fused according to a preset splicing method and a third preset weight to obtain features to be adjusted; Perform adjustment processing on the features to be adjusted based on a preset spatial attention mechanism to obtain the target features of the surgical instrument image, so as to determine the feature fusion mechanism of the initial classification model.

5. The surgical instrument classification method according to claim 1, characterized in that, The constructing the object detection head of the initial classification model based on a preset shared convolution technique includes: Determine a target convolution operation based on a preset group normalization technique, and construct the object detection head of the initial classification model based on the target convolution operation, a preset depthwise separable convolution technique, a preset regression convolution technique, a preset classification convolution technique and the preset shared convolution technique.

6. The surgical instrument classification method according to claim 1, wherein Constructing the initial classification model according to the feature extraction mechanism, the feature fusion mechanism, the target detection head, and a preset target detection network includes: Determining a target loss function corresponding to the preset target detection network based on a target gradient gain allocation strategy, and constructing the initial classification model according to the target loss function, the feature extraction mechanism, the feature fusion mechanism, the target detection head, and the preset target detection network.

7. The surgical instrument classification method according to any one of claims 1 to 6, characterized in that, Determining a target classification model based on the surgical instrument image dataset and the initial classification model includes: Adjusting the surgical instrument images in the surgical instrument image dataset to a preset image size to obtain a target surgical instrument image dataset, and dividing the target surgical instrument image dataset based on preset ratio conditions to obtain a training set, a validation set, and a test set; Using the training set, the validation set, and the test set, determining the target classification model based on preset iteration control parameters, a preset stochastic gradient descent method, and the initial classification model; wherein the preset iteration control parameters include any one or a combination of several of batch size, number of iterations, and learning rate.

8. A surgical instrument classification device, characterized in that, Includes: A surgical instrument image dataset construction module, configured to obtain each surgical instrument image, and construct a surgical instrument image dataset based on the surgical instrument image and the label corresponding to the surgical instrument image; A feature extraction mechanism determination module, configured to determine a feature extraction mechanism of the initial classification model based on a preset convolutional neural network, determine a feature fusion mechanism of the initial classification model based on a preset feature pyramid network structure, and construct a target detection head of the initial classification model based on a preset shared convolution technique; A surgical instrument classification module, configured to construct the initial classification model according to the feature extraction mechanism, the feature fusion mechanism, the target detection head, and a preset target detection network, and determine a target classification model based on the surgical instrument image dataset and the initial classification model, so as to perform surgical instrument classification based on the target classification model.

9. An electronic device, characterized in that, The electronic device includes a processor and a memory; wherein, the memory is used to store a computer program, and the computer program is loaded and executed by the processor to implement the surgical instrument classification method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, For storing a computer program, the computer program, when executed by a processor, implements the surgical instrument classification method according to any one of claims 1 to 7.

Images