Laparoscopic instrument real-time detection method

Abstract

The invention discloses a laparoscopic instrument real-time detection method. The laparoscopic instrument real-time detection method specifically comprises the following steps: S1, acquiring an image data set; S2, constructing an SH-YOLO network model; and S3, training and detecting. Wherein the SH-YOLO network model comprises three main parts: a StarNet feature extraction network, an HAM-PAFPN feature fusion layer and a target detection head. Compared with YOLOv5, the model training method SH-YOLO has the advantages that the model training method SH-YOLO is obviously improved, and the model training method SH-YOLO is obviously superior to YOLOv5 in the aspect of real-time performance especially in the aspect of processing non-central objects, unobvious features, false alarms, unconventional postures and motion blur. The SH-YOLO better meets the working requirements of high precision and real-time performance of a laparoscopic surgical instrument identification task, and provides a reliable basis for subsequent surgical step identification, surgical quality control and surgical operation evaluation.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing technology, and in particular to the field of designing a technology for device recognition based on a neural network model. Background Art

[0002] The maturity and clinical promotion of laparoscopic surgical systems mark the arrival of the era of minimally invasive surgery. With the maturity of laparoscopic surgical technology and the advancement of imaging equipment, laparoscopic surgery has been widely used in simple gallbladder and appendectomy, gastrointestinal tumor resection and complex pancreatic surgery, and has generated a large number of surgical video images and image data. By observing these videos to evaluate surgical skills, effective feedback can be provided, the quality of surgery can be improved, and the incidence of complications can be reduced. At present, the surgical evaluation process requires experts to observe with the naked eye, which is extremely time-consuming and labor-intensive. Therefore, it is very necessary to use artificial intelligence technologies such as computer vision for automatic surgical video analysis and evaluation.

[0003] Surgical instrument recognition is one of the key foundations for realizing automatic surgical video analysis and has become a research hotspot in the future smart medical field. On the one hand, real-time detection of surgical instruments can accurately provide doctors with instrument location information, helping doctors avoid misoperation caused by misjudgment of instrument location, such as accidental damage to surrounding tissues. On the other hand, surgical instrument recognition is one of the key foundations for identifying surgical steps and realizing automatic surgical video analysis, which has an important impact on controlling surgical quality and evaluating surgical operations.

[0004] With the development of modern medicine, surgical instrument detection methods can be roughly divided into two stages: traditional methods and deep learning-based methods. Traditional methods mainly include hardware-based methods, surgical tool marking combined with image processing algorithms, and surgical tool non-marking methods. This method is faster than the marking method, but the stability is poor. With the rise of neural networks and the development of computer hardware, surgical tool detection methods based on deep learning have gradually emerged and occupied a mainstream position, such as Faster R-CNN, Mask R-CNN, YOLO series and other methods.

[0005] In the surgical environment, the real-time detection of laparoscopic surgical instruments still faces many challenges: 1) The number of surgical instrument dataset samples is limited: the existing surgical instrument detection methods require a large number of datasets for training. The current public laparoscopic surgical instrument datasets are relatively small, and it is difficult to obtain real surgical scene datasets, resulting in poor training results for neural networks; 2) Poor real-time performance: It is difficult to strike a balance between computational economy and detection accuracy, and most of the surgical instrument detections are static, and surgical instruments cannot be tracked in real time, resulting in slow processing speed; 3) High false detection rate: complex surgical operation scenes cause occlusion and smoke in the acquired images. In addition, the recognition task becomes more difficult due to the high distortion or artifacts in the images in many cases. Therefore, the current development trend is mainly to continuously improve the speed and accuracy of surgical instrument recognition by improving traditional deep learning neural networks, and on this basis, to achieve lightweight networks.

[0006] In the existing technology, Jiang et al. proposed an improved YOLOv5 algorithm for surgical instrument recognition in the paper "Surgical Instrument Recognition Based on Improved YOLOv5" to reduce computational complexity and memory usage. However, due to the limitations of the network structure, insufficient feature extraction and representation capabilities, this method still has the problem of not being able to balance real-time performance and accuracy in some complex and special image recognition tasks, such as non-central objects, unclear features, false positives, unconventional postures, and motion blur. Summary of the invention

[0007] In view of the problems existing in the prior art, the present invention proposes a real-time detection method for laparoscopic instruments, which has high real-time performance and high accuracy when processing complex and special image recognition tasks.

[0008] In order to achieve the above-mentioned object of the invention, the present invention adopts the following technical solutions:

[0009] A real-time detection method for laparoscopic instruments, characterized by comprising the following steps:

[0010] S1. Obtain an image dataset, where the image dataset includes a training set, a validation set, and a test set;

[0011] S2. Constructing an SH-YOLO network model, wherein the SH-YOLO network model comprises a feature extraction network, a feature fusion network and a detection head connected in sequence;

[0012] The feature extraction network receives the image data set and outputs initial feature data to the feature fusion network, wherein the feature extraction network is composed of multiple layers of StarBlocks, and the StarBlocks are stacked based on element multiplication;

[0013] The feature fusion network fuses the initial feature data to obtain a fused feature map and outputs the fused feature map to the detection head;

[0014] The detection head performs target detection and classification on the fused feature map, and outputs the target detection result;

[0015] S3, training and testing, inputting the training set and the validation set into the SH-YOLO network model for training and verification, and inputting the test set into the pre-trained SH-YOLO network model for testing, and outputting the target detection result.

[0016] YOLO (You Only Look Once) is a popular computer vision recognition algorithm, which mainly includes a feature extraction network, a feature fusion network and a detection head. The model is trained by a training set and a verification machine, and then the test set is input into the trained model, which can better achieve the object detection task. The SH-YOLO network model in this technical solution is an improvement based on the original YOLO network model, as follows: In the field of computer vision and deep learning, element multiplication (Star Operation) usually refers to the element-wise multiplication operation between two tensors of the same size. This technical solution is based on the element multiplication stacked to form the StarBlock, and a star feature extraction network (StarNet) is constructed by multi-layer StarBlock. The ability to map the input to a high-dimensional nonlinear feature space without widening the network. Unlike traditional neural networks that increase the network width (also known as the number of channels), Star Operation is similar to a kernel function that performs paired feature multiplication on different channels, especially a polynomial kernel function, which can improve the expression ability of the feature space by nonlinearly combining the input features. When it is applied to a neural network and stacked in multiple layers, each layer will bring about an exponential increase in the complexity of the implicit dimension, which means that through multi-layer stacking, the network can significantly improve the feature representation capability without significantly increasing the computational complexity. Therefore, through multiple layers of StarBlock, StarNet can recursively increase the implicit feature dimension, and then approach infinite dimensions, thereby significantly enhancing the feature expression capability, and achieving efficient feature extraction while maintaining low computational complexity. The existing YOLO series networks are difficult to balance between real-time and detection accuracy, while StarNet in this technical solution can achieve efficient feature extraction and improve detection accuracy without increasing the computational burden, solving the trade-off between computational complexity and performance in the real-time detection task of laparoscopic surgical instruments, and has high real-time and high accuracy when processing complex and special image recognition tasks.

[0017] As a preferred embodiment of the present invention, the step S1, obtaining an image data set, specifically comprises the following steps:

[0018] S11, acquiring video data, the video data being a laparoscopic surgery video, and performing frame extraction conversion on the video data to obtain image frame data;

[0019] S12, preprocessing the image frame data to obtain a preprocessed image;

[0020] S13, annotating the preprocessed image with surgical instrument information to obtain the image data set.

[0021] Step S11 takes appendicitis surgery as an example and collects laparoscopic appendicitis surgery videos of about 60 cases in the medical imaging database, with each video lasting about 20 minutes to 60 minutes. Using the commercial video processing software Premiere, each surgical video is framed at intervals of 10 seconds and converted into image frames with an image resolution of 1920×1080PPI. Step S12 preprocessing includes but is not limited to image, image scaling, image normalization, data enhancement, image format conversion, image denoising, etc. to ensure the quality of the data set. Step S13 uses the labelme software method to manually mark the preprocessed image, annotate the location and name of all instruments that appear in the image during the operation, and divide it into training set, validation set and test set in a ratio of 8:1:1 for subsequent training.

[0022] As a preferred embodiment of the present invention, the real-time detection method for laparoscopic instruments further comprises a format conversion step located after S1, acquiring the image data set: converting the image data set into a COCO data set;

[0023] The COCO (Common Objects in Context) dataset is a dataset sponsored by Microsoft Research and widely used in large-scale image recognition, segmentation and annotation. The COCO dataset includes all training image folders, all validation image folders and corresponding annotation folders. The corresponding annotation folders include the training set, validation set and test set json annotation files for the corresponding object detection and segmentation tasks.

[0024] As a preferred embodiment of the present invention, the feature fusion network is a HAM-PAFPN feature fusion network, and the HAM-PAFPN feature fusion network is formed by integrating the HAM hybrid attention mechanism into the PAFPN feature fusion network, and the HAM hybrid attention mechanism is constructed by the CBAM attention mechanism and SEAttention.

[0025] Deep learning algorithms use deep neural networks to extract features, but as the number of deep neural network layers increases, some shallow information is easily lost, and interference in surgical scenes such as blood and organs can cause the model to miss targets in complex environments. To address this problem, a multi-scale feature fusion method is used to fuse deep and shallow features during feature extraction, which can enhance information transfer between different network layers. Therefore, optimizing the feature fusion network can allow the algorithm to pay more attention to key areas and important features in the image, enhance the model's ability to extract key information, and help more accurately identify surgical instruments in complex backgrounds, thereby reducing the false detection rate.

[0026] In this technical solution, the initial feature data is first input into the reduction layer to reduce the dimensionality of the high-dimensional data to reduce the computational complexity, and then input into the CBAM attention mechanism to enhance the expression ability of the initial feature data, and then input into the SEAttention to further enhance the expression ability of the features by adaptively recalibrating the channel weights of the initial feature data, and then upsampling and downsampling operations are performed through the upsampling layer and the downsampling layer to achieve the fusion of features of different scales, and then through the top-to-bottom layer and the bottom-to-top layer for top-to-bottom and bottom-to-top feature transfer, to enhance the information exchange between features at different levels, and finally through the splicing layer to splice and fuse the features at different levels to enrich the feature expression.

[0027] In addition, different attention mechanisms can play different roles in different positions. The CBAM module can enhance the features of channel and spatial dimensions, and is suitable for use after the initial feature data is reduced in dimension to enhance the expressiveness of the initially extracted features. The SEAttention module can adaptively recalibrate the channel weights of the feature map, and is suitable for use after feature fusion. It can further enhance the expressiveness of the features after feature fusion and capture multi-scale information; using the CBAM module again before the output layer can further enhance the expressiveness of the final output feature map. The two are mixed and integrated into the PAFPN feature fusion network, and the HAM-PAFPN feature fusion network is designed.

[0028] By combining the CBAM and the SEAttention, the proposed hybrid attention mechanism can better enhance the expressiveness of the feature map in the channel and spatial dimensions, and improve the ability of the HAM-PAFPN feature fusion network to capture small features in the laparoscopic surgical instrument recognition task.

[0029] As a preferred embodiment of the present invention, the detection head adopts YOLOv5Head, and the target detection result includes the position of the device boundary box in the image, the confidence and the category probability.

[0030] The YOLOv5Head in the prior art is used for the final target detection and classification. The fused feature map output by the feature fusion network is passed through a series of convolutional layers and activation functions to finally generate a target detection result. The target detection result specifically includes the position of the device bounding box in the image, the confidence level, and the category probability. The design of using the YOLOv5Head for target detection and classification can effectively combine the advantages of deep learning to achieve fast and accurate target detection, which is suitable for application scenarios that require a balance between real-time and accuracy.

[0031] As a preferred embodiment of the present invention, the S3, training and testing, specifically comprises the following steps:

[0032] S31, constructing a loss function, wherein the loss function includes a coordinate loss function, a confidence loss function and a classification loss function, and calculating corresponding coordinate loss values, confidence loss values ​​and classification loss values;

[0033] S32, calculating the gradient of the loss function to the model parameters by a back propagation algorithm;

[0034] S33. According to the calculated gradient, the model parameters are updated using an optimization algorithm to reduce the loss value; the process of forward propagation, loss calculation, back propagation and parameter update is repeated until a stopping condition is reached.

[0035] By constructing the coordinate loss function model, the bounding box position of the target can be learned more accurately, thereby improving the accuracy of target detection. By constructing the confidence loss function, the model is helped to learn how to evaluate the confidence of its predictions, which helps to screen out reliable prediction results and reduce false positives during the detection process. By constructing the classification loss function, it is ensured that the model can correctly identify the category of the target, which is crucial for multi-category target detection tasks.

[0036] By integrating multiple losses, an end-to-end training process is achieved, the training process is simplified, and the model can simultaneously optimize positioning and classification in a single training cycle. The gradient of the loss function to the model parameters is calculated using the back-propagation algorithm, and the model parameters are updated through the optimization algorithm. This process can effectively guide the model parameters to adjust in the direction of the optimal solution. The optimization algorithm can use the Adam optimizer in the prior art for parameter tuning. This training and detection method not only improves the accuracy and efficiency of detection by comprehensively optimizing multiple aspects of the model, but also enhances the overall performance and applicability of the model, providing a solid foundation for practical applications.

[0037] As a preferred embodiment of the present invention, the S31, constructing the loss function is specifically: calculating the coordinate loss value by the mean square error, calculating the confidence loss value by the binary cross entropy, and calculating the classification loss value by the cross entropy loss function.

[0038] The coordinate loss is calculated by the mean squared error (MSE), which can accurately measure the position difference between the predicted bounding box and the true bounding box. MSE processes the square of the error, which amplifies the impact of large errors and forces the model to focus more on accurate bounding box positioning. The confidence loss uses binary cross entropy (BCE) to help the model learn how to accurately predict whether each bounding box contains an object. BCE penalizes the confidence of the prediction. When the predicted confidence differs greatly from the true label (the presence or absence of the object), the loss value will increase, thereby guiding the model to improve the accuracy of the confidence prediction. The classification loss uses cross entropy, which can effectively measure the model's prediction accuracy for the target category. The cross entropy loss quantifies the difference between the predicted probability distribution and the distribution of the true label, prompting the model to learn more robust and accurate classification features. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 is the overall method flow chart;

[0041] Figure 2 It is an example of a laparoscopic surgical instrument;

[0042] Figure 3 This is an example of the dataset annotation results;

[0043] Figure 4 This is the SH-YOLO network structure diagram;

[0044] Figure 5 This is the StarNet feature extraction network structure diagram;

[0045] Figure 6 This is the HAM-FPFPN feature fusion network structure diagram. DETAILED DESCRIPTION

[0046] The present invention is further described below by specific examples and drawings. The examples of the present invention are intended to enable those skilled in the art to better understand the present invention, and do not limit the present invention in any way.

[0047] This embodiment provides a real-time detection method for laparoscopic surgical instruments based on SH-YOLO, and the flow chart is as follows: Figure 1 As shown, the following specific steps are included:

[0048] S1. Obtain image dataset

[0049] The video data is obtained, which is a laparoscopic surgery video, and the video data is extracted and converted into image frame data. Specifically, taking appendicitis surgery as an example, about 60 cases of laparoscopic appendicitis surgery videos were collected from the medical imaging database, and the length of each video ranged from about 20 minutes to 60 minutes. Using the video processing commercial software Premiere, each surgical video was extracted at intervals of 10 seconds and converted into image frame data. The image resolution was 1920×1080PPI. The surgical instruments used in the operation, such as Figure 2 shown.

[0050] The frame data is preprocessed to obtain the preprocessed image. Preprocessing includes but is not limited to image, image scaling, image normalization, data enhancement, image format conversion, image denoising, etc. to ensure the quality of the data set. The preprocessed image is manually labeled with surgical instrument information using labelme software, and the location and name of all instruments appearing in the surgical process are marked in the image, and divided into training set, validation set and test set in a ratio of 8:1:1 for subsequent training.

[0051] In an example of this embodiment, the annotation includes 7 types of instruments: atraumatic graspers, aspirators, separating forceps, electrocoagulation hooks, clip appliers, needle holders, and scissors, which account for 40.68%, 28.90%, 12.50%, 5.49%, 1.27%, 9.79%, and 1.35% of the total number of instruments in the dataset, respectively. The annotation results are as follows: Figure 3 As shown in the figure, the constructed image dataset format is the COCO dataset format. The COCO dataset includes all training image folders, all verification image folders, and corresponding annotation folders. The corresponding annotation folders include the training set, verification set, and test set json annotation files for the corresponding object detection and segmentation tasks.

[0052] S2. Build SH-YOLO network model.

[0053] like Figure 4 As shown in Figure 1, the SH-YOLO network model includes a feature extraction network, a feature fusion network, and a detection head connected in sequence.

[0054] Backbone: StarNet is used as the feature extraction network. StarNet maps the input to a high-dimensional nonlinear feature space through Star Operation (element multiplication) and has efficient feature extraction capabilities. The main structure of StarNet includes multiple StarBlocks, which enhance the feature expression capability by recursively increasing the implicit feature dimension.

[0055] Stem Layer: The initial convolutional layer, used to perform basic feature extraction on the input image.

[0056] Stage Layer 1 to Stage Layer 4: Consists of multiple StarBlocks, extracting higher-level features layer by layer. The output feature map of each Stage Layer is downsampled and then enters the Neck part for further processing.

[0057] Neck: Use HAM-PAFPN for feature fusion, combining different attention mechanisms (CBAM and SEAttention) to enhance the expressiveness of feature maps in channel and spatial dimensions.

[0058] Reduce Layer: Reduce the dimensionality of high-dimensional feature maps to reduce computational complexity.

[0059] CBAM (Convolutional BlockAttention Module): Used after feature map dimensionality reduction to enhance the expressiveness of the initial extracted features.

[0060] SEAttention (Squeeze-and-ExcitationAttention): Used after feature fusion, it further improves the expressiveness of features by adaptively recalibrating the channel weights of feature maps.

[0061] UpSample and DownSample Layer: Through upsampling and downsampling operations, the fusion of features of different scales is achieved.

[0062] TopDown and BottomUp Layer: used for top-down and bottom-up feature transfer to enhance information exchange between features at different levels.

[0063] Concat: Concatenate and fuse features at different levels to enrich feature expression.

[0064] YOLOv5Head: Adopts the detection head of YOLOv5, which is responsible for the final target detection and classification.

[0065] Head: Performs target detection and classification based on the fused feature map output by Neck, and outputs the target detection result;

[0066] Loss: Calculates the loss between the predicted result and the true label to guide the training and optimization of the model.

[0067] The feature extraction network receives the image data set and outputs the initial feature data to the feature fusion network. The feature extraction network consists of multiple layers of StarBlock, which are stacked based on element multiplication. The feature fusion network fuses the initial feature data to obtain the fused feature map and outputs it to the detection head. The detection head performs target detection and classification on the fused feature map and outputs the target detection result. The details are as follows:

[0068] (I) Feature Extraction Network (Backbone)

[0069] A StarBlock is designed based on element multiplication, and a feature extraction network StarNet is constructed through multiple layers of Star Block to recursively increase the implicit feature dimension and approach infinite dimension, thereby significantly enhancing the expressiveness of the feature and achieving efficient feature extraction while maintaining low computational complexity. Star Block operates on weight factors and feature vectors based on Star Operation.

[0070]

[0071]

[0072] Where w1 and w2 represent weight factors, x represents the feature vector, α represents the coefficient of each item, d represents the number of channels, and (i, j) represents the channel index.

[0073] like Figure 5 The feature extraction network StarNet shown in the figure. StarNet maps the input to a high-dimensional nonlinear feature space through Star Operation (element multiplication) and has efficient feature extraction capabilities. The main structure of StarNet includes multiple Star Blocks, which enhance the feature expression capability by recursively increasing the implicit feature dimension. StarBlock is the basic building block in the network and contains two convolution modules, namely 1×1 and 3×3 convolutions, which are used to reduce the number of channels and spatial feature extraction, respectively. During forward propagation, the input first undergoes a 1×1 convolution, then interacts with its own features through element multiplication, and finally undergoes a 3×3 convolution process and is skip-connected and added to the input tensor to output the final result.

[0074] Optionally, each StarBlock also includes an optional depth-wise separable convolution and a product operation based on ReLU6 activation.

[0075] Optionally, after StarBlock, an optional channel attention mechanism is also included, which can perform channel weighting on the concatenated feature maps through the ChannelAttention module.

[0076] (II) Feature Fusion Network

[0077] A hybrid attention mechanism (HAM) is constructed based on the attention mechanism CBAM and SEAttention and integrated into the PAFPN feature fusion network. The HAM-PAFPN feature fusion network is designed to better enhance the expressive ability of feature maps in channel and spatial dimensions, and improve the ability of HAM-PAFPN feature fusion network to capture small features in the task of laparoscopic surgical instrument recognition.

[0078] like Figure 6 The feature fusion network HAM-PAFPN shown in the figure. The initial feature data extracted by StarNet is input into the feature fusion network HAM-PAFPN. After the initial feature data is reduced in dimension, CBAM is used to enhance the expressiveness of the initial extracted features. After feature fusion, SEAttention is used to adaptively recalibrate the channel weights of the feature map to further enhance the expressiveness of the features and capture multi-scale information. The CBAM module is used again before the output layer to further enhance the expressiveness of the final output feature map, so that the model can effectively use features of different scales for target detection.

[0079] Specifically, the feature map extracted by StarNet is input into the reduction layer to reduce the dimension of the high-dimensional feature map to reduce the computational complexity, and then the CBAM attention module is input to enhance the expressiveness of the initially extracted features, followed by SEAttention to further improve the expressiveness of the features by adaptively recalibrating the channel weights of the feature map, and then upsampling and downsampling operations are performed through the upsampling layer and the downsampling layer to achieve the fusion of features of different scales, and then through the top-to-bottom layer and the bottom-to-up layer, the top-to-bottom and bottom-to-up feature transfer is used to enhance the information exchange between features at different levels, and finally the features at different levels are spliced ​​and fused through the splicing layer to enrich the feature expression.

[0080] (III) Detection head

[0081] YOLOv5Head is used to detect and classify the target. The fused feature map output by the feature fusion network HAM-PAFPN is passed through a series of convolutional layers and activation functions to generate the target detection results, including the position of the device bounding box in the image, confidence, and category probability.

[0082] S3. Training and testing

[0083] Input the training set and validation set into the SH-YOLO network model for training and validation, and input the test set into the pre-trained SH-YOLO network model for detection, and output the target detection results. Specifically: Construct a loss function, which includes a coordinate loss function, a confidence loss function, and a classification loss function, and calculate the corresponding coordinate loss value, confidence loss value, and classification loss value. The specific calculation method is as follows: Calculate the coordinate loss value through the mean square error, calculate the confidence loss value through the binary cross entropy, and calculate the classification loss value through the cross entropy loss function.

[0084] The coordinate loss is calculated by the mean squared error (MSE), which can accurately measure the position difference between the predicted bounding box and the true bounding box. MSE processes the square of the error, which amplifies the impact of large errors and forces the model to focus more on accurate bounding box positioning. The confidence loss uses binary cross entropy (BCE) to help the model learn how to accurately predict whether each bounding box contains an object. BCE penalizes the confidence of the prediction. When the predicted confidence differs greatly from the true label (the presence or absence of the object), the loss value will increase, thereby guiding the model to improve the accuracy of the confidence prediction. The classification loss uses cross entropy, which can effectively measure the model's prediction accuracy for the target category. The cross entropy loss quantifies the difference between the predicted probability distribution and the distribution of the true label, prompting the model to learn more robust and accurate classification features.

[0085] The gradient of the loss function to the model parameters is calculated through the back-propagation algorithm. Based on the calculated gradient, the optimization algorithm is used to update the model parameters to reduce the loss value; the process of forward propagation, loss calculation, back-propagation and parameter update is repeated until the stopping condition is reached.

[0086] The test set is input into the pre-trained SH-YOLO network model for detection, and the target detection result is output. The present invention constructs a laparoscopic surgical instrument dataset using appendicitis surgery as an example. The present invention constructs a laparoscopic surgical instrument dataset using appendicitis surgery as an example. SH-YOLO has made significant improvements over YOLOv5 in identifying laparoscopic instruments, especially in dealing with non-central objects, unclear features, false alarms, unconventional postures and motion blur, and is significantly better than YOLOv5 in real-time performance. SH-YOLO better meets the high-precision and real-time requirements of laparoscopic surgical instrument recognition tasks, and provides reliable data for subsequent identification of surgical steps, control of surgical quality, and evaluation of surgical operations.

[0087] It should be noted that the above embodiments are only detailed descriptions of the preferred embodiments and principles of the present invention. For ordinary technicians in this field, there will be changes in the specific implementation methods based on the ideas provided by the present invention, and these changes are also considered to be within the scope of protection of the present invention.

Claims

1. A real-time detection method for laparoscopic instruments, characterized in that: The following steps are involved: S1. Obtain an image dataset, where the image dataset includes a training set, a validation set, and a test set; S2. Constructing an SH-YOLO network model, wherein the SH-YOLO network model comprises a feature extraction network, a feature fusion network and a detection head connected in sequence; the feature extraction network receives the image data set and outputs initial feature data to the feature fusion network, wherein the feature extraction network is composed of multiple layers of Star Block, and the Star Block is stacked based on element multiplication; The feature fusion network fuses the initial feature data to obtain a fused feature map and outputs it to the detection head; the detection head performs target detection and classification on the fused feature map and outputs a target detection result; S3, training and testing, inputting the training set and the validation set into the SH-YOLO network model for training and verification, and inputting the test set into the pre-trained SH-YOLO network model for testing, and outputting the target detection result.

2. A laparoscopic instrument real-time detection method according to claim 1, characterized in that: The step S1, obtaining an image data set, specifically comprises the following steps: S11, acquiring video data, the video data being a laparoscopic surgery video, and performing frame extraction conversion on the video data to obtain image frame data; S12, preprocessing the image frame data to obtain a preprocessed image; S13, annotating the preprocessed image with surgical instrument information to obtain the image data set.

3. A laparoscopic instrument real-time detection method according to claim 1, characterized in that: The real-time detection method for laparoscopic instruments further comprises a format conversion step after S1, acquiring the image data set: converting the image data set into a COCO data set.

4. A laparoscopic instrument real-time detection method according to claim 1, characterized in that: The feature fusion network is a HAM-PAFPN feature fusion network, which is formed by integrating the HAM hybrid attention mechanism into the PAFPN feature fusion network. The HAM hybrid attention mechanism is constructed through the CBAM attention mechanism and SEAttention.

5. A real-time detection method for laparoscopic instruments according to claim 1, characterized in that: The detection head adopts YOLOv5Head, and the target detection result includes the position of the device boundary box in the image, the confidence and the category probability.

6. A real-time detection method for laparoscopic instruments according to any one of claims 1 to 5, characterized in that: The S3, training and testing, specifically includes the following steps: S31, constructing a loss function, wherein the loss function includes a coordinate loss function, a confidence loss function and a classification loss function, and calculating corresponding coordinate loss values, confidence loss values ​​and classification loss values; S32, calculating the gradient of the loss function to the model parameters by a back propagation algorithm; S33. According to the calculated gradient, the model parameters are updated using an optimization algorithm to reduce the loss value; the process of forward propagation, loss calculation, back propagation and parameter update is repeated until a stopping condition is reached.

7. A real-time detection method for laparoscopic instruments according to claim 6, characterized in that: The S31, constructing the loss function specifically comprises: calculating the coordinate loss value by the mean square error, calculating the confidence loss value by the binary cross entropy, and calculating the classification loss value by the cross entropy loss function.

Images