Laparoscopic surgical instrument detection method and device based on improved YOLOv8

Abstract

The application discloses a laparoscopic surgical instrument detection method and device based on improved YOLOv8, and relates to the field of image processing.The method comprises the following steps: constructing a laparoscopic surgical instrument detection model based on improved YOLOv8 and training the model to obtain a trained laparoscopic surgical instrument detection model; the laparoscopic surgical instrument detection model replaces the C2F module in the backbone network with a multi-scale direction perception gating module and replaces the SPPF module with a frequency domain enhancement pyramid pooling module on the basis of the standard YOLOv8; the neck network is replaced with a cross-scale progressive feature fusion neck network, the double-domain cross-coupling rescaling module is adopted in the cross-scale progressive feature fusion neck network, and the boundary box regression result output by the detection head is constrained by adopting the direction decoupling gradient modulation loss in the training process; the laparoscopic surgical image to be detected is acquired and input into the trained laparoscopic surgical instrument detection model to obtain the detection result of each surgical instrument, and the problems of insufficient feature expression and poor target detection effect are solved.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more specifically to a method, apparatus, and readable medium for detecting laparoscopic surgical instruments based on an improved YOLOv8. Background Technology

[0002] Laparoscopic minimally invasive surgery, due to its advantages of minimal trauma and rapid postoperative recovery, has been widely used in various surgical procedures, including general surgery, obstetrics and gynecology, and urology. During laparoscopic surgery, surgical instruments are essential tools for surgeons to perform operations such as traction, shearing, clamping, dissection, and electrocoagulation. Accurate identification and positioning of surgical instruments in laparoscopic images is crucial for understanding the surgical process, analyzing instrument trajectories, evaluating surgical skills, monitoring surgical safety, and developing intelligent surgical assistance systems. Therefore, developing accurate and efficient methods for detecting laparoscopic surgical instruments is key to improving surgical scene perception capabilities and assisting in intelligent surgical analysis.

[0003] Currently, deep learning-based object detection methods have been applied to the detection of laparoscopic surgical instruments. However, laparoscopic surgical images are characterized by complex backgrounds, uneven lighting, slender instrument shapes, significant directional changes, and large differences in target scale. Furthermore, interference factors such as mirror reflections, smoke, and blood occlusion during surgery can easily weaken the edge and structural details of instruments, making it difficult for detection models to extract stable and effective features. Therefore, existing methods still suffer from insufficient feature representation, inadequate directional information modeling, and poor performance in multi-scale object detection under complex surgical scenarios. Summary of the Invention

[0004] The purpose of this application is to propose a method and device for testing laparoscopic surgical instruments based on an improved YOLOv8 to address the aforementioned technical problems.

[0005] In a first aspect, the present invention provides a method for detecting laparoscopic surgical instruments based on an improved YOLOv8, comprising the following steps:

[0006] A laparoscopic surgical instrument detection model based on an improved YOLOv8 was constructed and trained to obtain the trained laparoscopic surgical instrument detection model. Based on the standard YOLOv8 model, the C2F module in the backbone network was replaced with a multi-scale orientation-aware gating module, and the SPPF module was replaced with a frequency-domain enhanced pyramid pooling module. The neck network was replaced with a cross-scale progressive feature fusion neck network, which employs a dual-domain cross-coupling recalibration module. During training, orientation decoupling gradient modulation loss was used to constrain the bounding box regression results output by the detection head.

[0007] The laparoscopic surgical image to be detected is acquired and input into a trained laparoscopic surgical instrument detection model. First, the improved backbone network is used for feature extraction to obtain multi-scale image features. The multi-scale image features are then input into a cross-scale progressive feature fusion neck network for feature fusion to obtain cross-scale fused features. The cross-scale fused features are then input into the detection head to obtain the detection results of each surgical instrument in the laparoscopic surgical image to be detected.

[0008] Preferably, the frequency domain enhanced pyramid pooling module includes a first convolutional module connected in sequence, a frequency domain enhancement branch and a multi-scale pooling branch arranged in parallel, a first stitching layer and a second convolutional module; the frequency domain enhancement branch includes one discrete wavelet transform unit, one low-frequency gating unit, three high-frequency gating units and one discrete wavelet inverse transform unit; the multi-scale pooling branch includes a second stitching layer and three max-pooling layers arranged in series; the low-frequency gating unit includes a first convolutional layer and a first sigmoid activation function layer connected in sequence, and each high-frequency gating unit... Each control unit includes a first depthwise separable convolutional layer and a second sigmoid activation function layer connected in sequence. The input features of the frequency domain enhancement pyramid pooling module are processed by the first convolutional module to obtain initial enhanced features. The initial enhanced features are input into the frequency domain enhancement branch and the multi-scale pooling branch, respectively. In the frequency domain enhancement branch, the initial enhanced features undergo a two-dimensional discrete wavelet transform by the discrete wavelet transform unit to obtain low-frequency subbands, horizontal high-frequency subbands, vertical high-frequency subbands, and diagonal high-frequency subbands. The low-frequency subbands are processed by the low-frequency gating unit to generate a low-frequency gating weight map. The low-frequency subband is weighted using a low-frequency gating weight map to obtain the suppressed low-frequency subband. The horizontal, vertical, and diagonal high-frequency subbands are each processed by a corresponding high-frequency gating unit to generate corresponding high-frequency gating weight maps. These high-frequency gating weight maps are then used to weight the horizontal, vertical, and diagonal high-frequency subbands respectively, resulting in enhanced horizontal, vertical, and diagonal high-frequency subbands. The suppressed low-frequency subband, enhanced horizontal high-frequency subband, and enhanced vertical high-frequency subband... The enhanced diagonal high-frequency subbands are reconstructed by a discrete wavelet inverse transform unit to obtain frequency domain enhanced features. In the multi-scale pooling branch, the initial enhanced features are sequentially passed through three max pooling layers to obtain pooling features at each level. The initial enhanced features and pooling features at each level are concatenated along the channel dimension by the second concatenation layer to obtain multi-scale pooling features. After the frequency domain enhanced features and multi-scale pooling features are concatenated along the channel dimension by the first concatenation layer, they are input into the second convolution module for channel compression and feature fusion to obtain the output features of the frequency domain enhanced pyramid pooling module.

[0009] Preferably, the multi-scale orientation perception gating module includes a third convolutional module, a first channel separation unit, a fourth convolutional module, parallel multi-scale local feature branches and orthogonal strip orientation branches, a gating weighting unit, a third stitching layer, and a fifth convolutional module connected in sequence; the multi-scale local feature branches include a second depthwise separable convolutional layer, a third depthwise separable convolutional layer, and a fourth depthwise separable convolutional layer connected in parallel, as well as a fourth stitching layer, a second convolutional layer, a first batch of normalization layers, and a sixth convolutional module connected in sequence; the orthogonal strip orientation branches include a first strip... The system consists of a convolutional layer, a second strip convolutional layer, and a fifth concatenation layer, a third convolutional layer, a second batch normalization layer, and a third sigmoid activation function layer, connected sequentially. The input features from the multi-scale orientation sensing gating module are input into the third convolutional module for convolution processing to obtain the first initial features. These initial features are then input into the first channel separation unit, where they are evenly divided along the channel dimension into first identity branch features and first processing branch features. The first processing branch features are input into the fourth convolutional module to obtain the first basic features. Finally, the first basic features are input into the multi-scale local feature branch and the orthogonal strip area, respectively. In the multi-scale local feature branch, the first basic feature is processed by the second, third, and fourth depthwise separable convolutional layers. The outputs of these three layers are concatenated along the channel dimension by the fourth concatenation layer, and then sequentially passed through the second convolutional layer, the first batch of normalization layers, and the sixth convolutional module for channel fusion and nonlinear transformation to obtain local multi-scale features. In the orthogonal strip directional branch, the first basic feature is processed by the first and second strip convolutional layers for directional context extraction. The outputs of these two layers are processed along the channel dimension by the fifth concatenation layer. After dimensional concatenation, the features are processed sequentially through the third convolutional layer, the second batch normalization layer, and the third sigmoid activation function layer to obtain the orientation gating weight map. The local multi-scale features and the orientation gating weight map are then input into the gating weighting unit, where the local multi-scale features are weighted element-wise by multiplication using the orientation gating weight map to obtain the orientation enhancement features. The first identity branch features, the first basic features, and the orientation enhancement features are concatenated along the channel dimension through the third concatenation layer and then input into the fifth convolutional module for channel compression and feature fusion to obtain the output features of the multi-scale orientation perception gating module.

[0010] Preferably, the cross-scale progressive feature fusion neck network includes two upsampling modules, five dual-domain cross-coupling recalibration modules, five concatenation layers, and six convolutional layers. The improved backbone network outputs first, second, and third backbone features with sequentially reduced resolutions, which are then input into the cross-scale progressive feature fusion neck network. The third backbone feature is the output feature of the frequency domain enhanced pyramid pooling module. In the cross-scale progressive feature fusion neck network, the third backbone feature is input into the first convolutional layer to obtain the first feature; the first feature is input into the first upsampling module to obtain the second feature; the second backbone feature is input into the second convolutional layer to obtain the third feature; the second and third features are input into the first concatenation layer to obtain the fourth feature; and the fourth feature is input into the first dual-domain cross-coupling recalibration module to obtain the fifth feature.

[0011] The fifth feature is input into the second upsampling module to obtain the sixth feature; the first backbone feature is input into the third convolutional layer to obtain the seventh feature; the sixth and seventh features are input into the second concatenation layer to obtain the eighth feature; the eighth feature is input into the second dual-domain cross-coupling recalibration module to obtain the ninth feature; the ninth feature is input into the fourth convolutional layer to obtain the tenth feature; the tenth and seventh features are input into the third concatenation layer to obtain the eleventh feature; the eleventh feature is input into the third dual-domain cross-coupling recalibration module to obtain the first detection feature.

[0012] The first detection feature is input into the fifth convolutional layer for downsampling to obtain the twelfth feature; the twelfth and fifth features are input into the fourth concatenation layer to obtain the thirteenth feature; the thirteenth feature is input into the fourth dual-domain cross-coupling recalibration module to obtain the second detection feature;

[0013] The second detection feature is input into the sixth convolutional layer for downsampling to obtain the fourteenth feature; the fourteenth feature and the first feature are input into the fifth concatenation layer to obtain the fifteenth feature; the fifteenth feature is input into the fifth dual-domain cross-coupling recalibration module to obtain the third detection feature; the first detection feature, the second detection feature and the third detection feature constitute a multi-scale fusion feature.

[0014] Preferably, the dual-domain cross-coupling recalibration module includes a seventh convolutional module, a second channel separation unit, an eighth convolutional module, two dual-domain cross-coupling recalibration units connected in series, a sixth splicing layer, and a ninth convolutional module connected in sequence.

[0015] The input features of the dual-domain cross-coupling recalibration module are first processed by the seventh convolutional module for channel adjustment and local feature mapping to obtain the second initial features. Subsequently, the second initial features are divided into the second identity branch features and the second processing branch features along the channel dimension by the second channel separation unit. The second processing branch features are input into the eighth convolutional module to obtain the second basic features. The second basic features are then processed by two dual-domain cross-coupling recalibration units to obtain the cross-coupling features and the recalibration enhancement features, respectively. The second identity branch features, the second processing branch features, the cross-coupling features, and the recalibration enhancement features are then input into the sixth concatenation layer for concatenation along the channel dimension, and then processed by the ninth convolutional module for channel compression and feature fusion to obtain the output features of the dual-domain cross-coupling recalibration module.

[0016] The dual-domain cross-coupled recalibration unit includes a first branch and a second branch set in parallel, a dual-domain interactive weight mapping module and a fourth convolutional layer. The first branch includes a tenth convolutional module and a channel enhancement unit connected in sequence. The second branch includes an eleventh convolutional module and a spatial enhancement unit connected in sequence.

[0017] The input features of the dual-domain cross-coupled recalibration unit are input into the first branch and the second branch, respectively, and processed by the tenth and eleventh convolutional modules to obtain the first branch features and the second branch features. The first branch features are input into the channel enhancement unit to obtain the channel weights. The channel weights are multiplied element-wise with the first branch features to obtain the first recalibration features. In the channel enhancement unit, the first branch features pass through the first global average pooling layer, the first fully connected layer, the ReLU activation function layer, the second fully connected layer, and the fourth Sigmoid activation function layer in sequence to obtain the channel weights.

[0018] The second branch feature is input into the spatial augmentation unit to obtain spatial weights. The spatial weights are then multiplied element-wise with the second branch feature to obtain the second recalibrated feature. In the spatial augmentation unit, the second branch feature is input into the second global average pooling layer and the global max pooling layer to obtain the average spatial description map and the maximum spatial description map. The average spatial description map and the maximum spatial description map are concatenated along the channel dimension and then passed through the fifth convolutional layer and the fifth sigmoid activation function layer to generate spatial weights.

[0019] The first and second calibration features are input into the dual-domain interactive weight mapping module for cross-coupling. The first and second calibration features are added element-wise to obtain a fused feature. Then, the element-wise mean of the fused feature is calculated to obtain a local mean feature. The first calibration feature is subtracted element-wise from the local mean feature, and then passed through a sixth sigmoid activation function layer to obtain a second feature sub-map. The second feature sub-map is multiplied element-wise with the second calibration feature, and then passed through a seventh sigmoid activation function layer to obtain the second branch weights. The second branch features are weighted using the second branch weights to obtain the second cross-recalibration feature. The second calibration feature is subtracted element-wise from the local mean feature, and then passed through a sixth sigmoid activation function layer to obtain a first feature sub-map. The first feature sub-map is multiplied element-wise with the first calibration feature, and then passed through a seventh sigmoid activation function layer to obtain the first branch weights. The first branch features are weighted using the first branch weights to obtain the first cross-recalibration feature.

[0020] The first and second cross-recalibration features are concatenated and then residually connected with the basic features. After being fused through the fourth convolutional layer, the output features of the dual-domain cross-coupled recalibration unit are obtained.

[0021] Preferably, the directional decoupling gradient modulation loss includes a complete intersection-union term, a rigid aspect ratio prior term, and a directional decoupling gradient modulation mechanism. The directional decoupling gradient modulation loss affects the predicted bounding box output by the detection head. With the corresponding truth box The forward computation of satisfies the following formula:

[0022] ;

[0023] ;

[0024] ;

[0025] ;

[0026] ;

[0027] ;

[0028] ;

[0029] Directional decoupling gradient modulation loss affects the predicted bounding box of the detector head output. With the corresponding truth box The auxiliary quantity for inverse gradient modulation is calculated according to the following formula:

[0030] ;

[0031] ;

[0032] ;

[0033] Directional decoupling gradient modulation loss affects the predicted bounding box of the detector head output. With the corresponding truth box The actual backpropagation gradient calculation is shown below:

[0034] ;

[0035] in, Predicted bounding boxes for the head regression output The coordinates; For truth boxes The coordinates; The width and height of the predicted bounding box; The width and height of the truth box; This is the intersection-union ratio (IUU) of the predicted bounding box and the ground truth bounding box. The square of the distance between the center point of the prediction box and the center point of the truth box; It is the square of the diagonal length of the smallest bounding rectangle that simultaneously contains both the predicted and ground truth boxes; As a measure of aspect ratio consistency, The weights are used for the standard full intersection-union loss method, which does not backpropagate gradients. It is the arctangent function; Indicates that it belongs to the first in the training set. The set of all truth boxes for each instrument category; i represents the index of the endpoint of the predicted box coordinates. i=1 corresponds to the coordinates of the upper left endpoint of the prediction box. i=2 corresponds to the coordinates of the lower right endpoint of the prediction box. j represents the sample index of the truth box of the c-th type of device in the training set; For the first Rigid a priori datum for aspect ratio of each instrument category; This indicates taking the median; It is a logarithmic function; The aspect ratio is a rigid prior weight; For a rigid aspect ratio prior, its gradient is propagated back in the conventional manner, without passing through... modulation; This represents the forward loss value for bounding box regression; The thin length is calculated from the width and height of the truth box; This refers to the gradient modulation sensitivity coefficient; It is the hyperbolic tangent function; These are the gradient modulation coefficients; , These represent taking the maximum and minimum values, respectively. These are the transverse gradient modulation weights and the longitudinal gradient modulation weights, respectively. , The conventional gradient is obtained by taking the partial derivatives of the partial term with respect to the x and y coordinates of the predicted bounding box for complete intersection and merging. , The general gradient of the aspect ratio rigid prior with respect to the x and y coordinates of the prediction box; The total gradient is the actual gradient back to the horizontal and vertical coordinates of the prediction box. The total gradient is propagated back layer by layer through the detection head and the improved backbone network to update the network parameters of the laparoscopic surgical instrument detection model.

[0036] Secondly, the present invention provides a laparoscopic surgical instrument detection device based on an improved YOLOv8, comprising:

[0037] The model building module is configured to build and train a laparoscopic surgical instrument detection model based on an improved YOLOv8, resulting in a trained laparoscopic surgical instrument detection model. The laparoscopic surgical instrument detection model replaces the C2F module in the backbone network with a multi-scale orientation-aware gating module and the SPPF module with a frequency-domain enhanced pyramid pooling module, based on the standard YOLOv8. The neck network is replaced with a cross-scale progressive feature fusion neck network, which employs a dual-domain cross-coupling recalibration module. During training, orientation-decoupled gradient modulation loss is used to constrain the bounding box regression results output by the detection head.

[0038] The prediction module is configured to acquire the laparoscopic surgical image to be detected and input it into the trained laparoscopic surgical instrument detection model. The improved backbone network first performs feature extraction to obtain multi-scale image features. The multi-scale image features are then input into the cross-scale progressive feature fusion neck network for feature fusion to obtain cross-scale fused features. The cross-scale fused features are then input into the detection head to obtain the detection results of each surgical instrument in the laparoscopic surgical image to be detected.

[0039] Thirdly, the present invention provides an electronic device including one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any implementation of the first aspect.

[0040] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any of the implementations of the first aspect.

[0041] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method as described in any of the implementations in the first aspect.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] (1) The laparoscopic surgical instrument detection method based on improved YOLOv8 mentioned in this invention replaces the SPPF module in the existing YOLOv8 network with the frequency domain enhanced pyramid pooling module. The features are decomposed into low-frequency sub-bands and high-frequency sub-bands through discrete wavelet transform units. The gating mechanism is used to suppress low-frequency redundant information and enhance high-frequency edge details, which effectively alleviates the problem of high-frequency feature loss caused by reflection and smoke in laparoscopic scenes and enhances the model's ability to extract instrument contour features.

[0044] (2) The laparoscopic surgical instrument detection method based on the improved YOLOv8 mentioned in this invention replaces the C2F module in the existing YOLOv8 network with a multi-scale orientation perception gating module. It extracts local multi-scale features through depth-separable convolution operations of various receptive fields and generates a single-channel orientation gating weight map by combining orthogonal strip convolution operations. After the single-channel orientation gating weight map is broadcast through the channel dimension, the local multi-scale features are weighted element by element, thereby enhancing the model's ability to perceive the slender structure and variable orientation features of surgical instruments.

[0045] (3) The laparoscopic surgical instrument detection method based on the improved YOLOv8 mentioned in this invention adopts a cross-scale progressive feature fusion neck network in the neck network. This cross-scale progressive feature fusion neck network achieves deep integration of multi-resolution features through a progressive bidirectional structure; and utilizes a dual-domain cross-coupled recalibration module to dynamically recalibrate features from the spatial domain and channel domain, highlighting important instrument features and suppressing background clutter. This significantly improves the feature extraction accuracy of the model under overlapping, occlusion and complex lighting environments, and effectively solves the problems of missed detection and false detection caused by large scale differences of surgical instruments and target deformation.

[0046] (4) The laparoscopic surgical instrument detection method based on the improved YOLOv8 mentioned in this invention uses orientation-decoupled gradient modulation loss to constrain the bounding box regression results output by the detection head during the training process. This loss function decouples the position error of the bounding box from the orientation change and independently modulates and dynamically balances different gradient components, effectively avoiding gradient penalty distortion caused by slender instruments with variable orientations and unbalanced aspect ratios. This method significantly accelerates the model's training convergence process and greatly improves the model's bounding box regression localization accuracy for surgical instruments with complex postures. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a schematic flowchart of a laparoscopic surgical instrument testing method based on an improved YOLOv8, as an embodiment of this application.

[0049] Figure 2 This is a schematic diagram of the overall structure of the laparoscopic surgical instrument detection model, which is based on the improved YOLOv8 laparoscopic surgical instrument detection method according to an embodiment of this application.

[0050] Figure 3 This is a schematic diagram of the frequency domain enhancement pyramid pooling module of the laparoscopic surgical instrument detection method based on the improved YOLOv8, as an embodiment of this application.

[0051] Figure 4 This is a schematic diagram of the structure of the multi-scale orientation sensing gating module of the laparoscopic surgical instrument detection method based on the improved YOLOv8, which is an embodiment of this application.

[0052] Figure 5 This is a schematic diagram of the dual-domain cross-coupled recalibration module of the laparoscopic surgical instrument detection method based on the improved YOLOv8, as an embodiment of this application.

[0053] Figure 6 This is a schematic diagram of the channel enhancement unit in the laparoscopic surgical instrument detection method based on the improved YOLOv8, as an embodiment of this application.

[0054] Figure 7 This is a schematic diagram of the spatial enhancement unit of the laparoscopic surgical instrument detection method based on the improved YOLOv8, which is an embodiment of this application.

[0055] Figure 8 This is a comparative schematic diagram of the detection results of the ablation experiment of the laparoscopic surgical instrument detection method based on the improved YOLOv8 according to an embodiment of this application;

[0056] Figure 9 This is a schematic diagram of a laparoscopic surgical instrument detection device based on an improved YOLOv8, according to an embodiment of this application.

[0057] Figure 10 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0059] Figure 1 This application illustrates an embodiment of a laparoscopic surgical instrument detection method based on an improved YOLOv8, comprising the following steps:

[0060] S1. Construct and train a laparoscopic surgical instrument detection model based on an improved YOLOv8 to obtain the trained laparoscopic surgical instrument detection model. Based on the standard YOLOv8, the laparoscopic surgical instrument detection model replaces the C2F module in the backbone network with a multi-scale orientation-aware gating module, and replaces the SPPF module with a frequency domain-enhanced pyramid pooling module. The neck network is replaced with a cross-scale progressive feature fusion neck network. The cross-scale progressive feature fusion neck network adopts a dual-domain cross-coupling recalibration module. During training, orientation decoupling gradient modulation loss is used to constrain the bounding box regression results output by the detection head.

[0061] For details, please refer to Figure 2The laparoscopic surgical instrument detection model mentioned in the embodiments of this application is an improvement on the existing standard YOLOv8 network. The improvements mainly include the backbone network and the neck network, while the detection head network adopts the existing standard YOLOv8 network's head network. In the backbone network, the laparoscopic surgical instrument detection model in the embodiments of this application replaces the C2F module in the existing YOLOv8 network's backbone network with a Multi-scale Direction-aware Gated Attention (MDGA) module, and replaces the SPPF module in the existing YOLOv8 network's backbone network with a Wavelet-based Spectral Pyramid PoolingFast (WSPPF) module, resulting in the improved backbone network. In the neck network portion, the laparoscopic surgical instrument detection model in this application replaces the optimized PAN-FPN (Path Aggregation Network-Feature Pyramid Network) used in the neck network of the existing YOLOv8 network with a Multi-scale Cross-Progressive Fusion Neck (MCPF-Neck). This Multi-scale Cross-Progressive Fusion Neck network employs a Dual-Domain Cross-Coupling Recalibration (DCCR) module, and during training, it uses directional decoupling gradient modulation loss to constrain the bounding box regression results output by the detection head.

[0062] The following details the structure of the frequency domain enhanced pyramid pooling module, the multi-scale orientation-aware gating module, the cross-scale progressive feature fusion neck network, the dual-domain cross-coupling recalibration module, and the orientation-decoupled gradient modulation loss. It should be noted that the first, second, third, fourth, fifth, sixth, seventh, eighth, ninth, tenth, and eleventh convolutional modules mentioned in the embodiments of this application all adopt a CBS structure. The CBS structure consists of a sixth convolutional layer with a kernel size of 1×1, a third batch normalization layer, and a SiLU activation function layer connected sequentially.

[0063] In a specific embodiment, the frequency domain enhanced pyramid pooling module includes a first convolutional module connected in sequence, a frequency domain enhancement branch and a multi-scale pooling branch arranged in parallel, a first stitching layer and a second convolutional module; the frequency domain enhancement branch includes one discrete wavelet transform unit, one low-frequency gating unit, three high-frequency gating units and one discrete wavelet inverse transform unit; the multi-scale pooling branch includes a second stitching layer and three max-pooling layers arranged in series; the low-frequency gating unit includes a first convolutional layer and a first sigmoid activation function layer connected in sequence, each... Each high-frequency gating unit includes a first depthwise separable convolutional layer and a second sigmoid activation function layer connected in sequence. The input features of the frequency domain enhancement pyramid pooling module are processed by the first convolutional module to obtain initial enhanced features. The initial enhanced features are input into the frequency domain enhancement branch and the multi-scale pooling branch, respectively. In the frequency domain enhancement branch, the initial enhanced features undergo a two-dimensional discrete wavelet transform by the discrete wavelet transform unit to obtain low-frequency subbands, horizontal high-frequency subbands, vertical high-frequency subbands, and diagonal high-frequency subbands. The low-frequency subbands are processed by the low-frequency gating unit to generate low-frequency gating weights. The figure shows that the low-frequency subband is weighted using a low-frequency gating weight map to obtain the suppressed low-frequency subband. The horizontal high-frequency subband, vertical high-frequency subband, and diagonal high-frequency subband are each processed by their respective high-frequency gating units to generate corresponding high-frequency gating weight maps. These high-frequency gating weight maps are then used to weight the horizontal, vertical, and diagonal high-frequency subbands respectively, resulting in the enhanced horizontal, vertical, and diagonal high-frequency subbands. The suppressed low-frequency subband, enhanced horizontal high-frequency subband, and enhanced vertical high-frequency subband are then weighted. The subband and the enhanced diagonal high-frequency subband are reconstructed by a discrete wavelet inverse transform unit to obtain frequency domain enhanced features. In the multi-scale pooling branch, the initial enhanced features are sequentially passed through three max pooling layers to obtain pooling features at each level. The initial enhanced features and pooling features at each level are concatenated along the channel dimension by the second concatenation layer to obtain multi-scale pooling features. After the frequency domain enhanced features and multi-scale pooling features are concatenated along the channel dimension by the first concatenation layer, they are input into the second convolution module for channel compression and feature fusion to obtain the output features of the frequency domain enhanced pyramid pooling module.

[0064] For details, please refer to Figure 3 The processing procedure of the frequency domain enhancement pyramid pooling module is shown in the following formula:

[0065] ;

[0066] ;

[0067] ;

[0068] ;

[0069] ;

[0070]

[0071] ;

[0072] ;

[0073] Where X represents the input feature of the frequency domain enhanced pyramid pooling module. This represents a two-dimensional discrete wavelet transform operation, which corresponds to a discrete wavelet transform unit. These are the low-frequency sub-band, horizontal high-frequency sub-band, vertical high-frequency sub-band, and diagonal high-frequency sub-band obtained from the decomposition, respectively. The kernel size is The depthwise separable convolution operation corresponds to the first depthwise separable convolutional layer; Indicates the kernel size as The convolution operation, which corresponds to the first convolutional layer. for Activation function These correspond to the horizontal high-frequency sub-band, the vertical high-frequency sub-band, and the diagonal high-frequency sub-band, respectively. These are the low-frequency gating weight map and the high-frequency gating weight map corresponding to the b-th high-frequency sub-band, respectively. This indicates element-wise multiplication. For the suppressed low-frequency subband, For the b-th enhanced high-frequency sub-band, i.e. These correspond to the enhanced horizontal high-frequency sub-band, the enhanced vertical high-frequency sub-band, and the enhanced diagonal high-frequency sub-band, respectively. This represents the two-dimensional discrete wavelet inverse transform operation, which corresponds to the discrete wavelet inverse transform unit. For the reconstructed frequency domain enhancement features, The kernel size is A max pooling layer with a step size of 1 and a fill size of 2. This represents the output feature of the (l-1)th max pooling layer. This represents the output feature of the l-th max pooling layer. ; This is the initial enhancement feature (i.e., X0). This represents the maximum pooling results at each level. This indicates splicing along the channel dimension. This is a multi-scale pooling feature. This represents the CBS structure. Y represents the output characteristic of the frequency-domain enhanced pyramid pooling module.

[0074] The frequency domain enhanced pyramid pooling module maps its input features from the spatial domain to the frequency domain by introducing a two-dimensional discrete wavelet transform operation, resulting in one low-frequency sub-band and three high-frequency sub-bands. The low-frequency sub-band mainly contains the image's contour and overall structural information, while the three high-frequency sub-bands correspond to edge and texture details in the horizontal, vertical, and diagonal directions, respectively. Considering that surgical instruments and tissue backgrounds in laparoscopic surgical images are similar in color and texture, spatial domain pooling alone is insufficient to effectively distinguish foreground from background. This embodiment utilizes a low-frequency gating unit to generate low-frequency gating weight maps corresponding to the low-frequency sub-bands, thereby reducing low-frequency redundant information in the background. Simultaneously, a high-frequency gating unit generates corresponding high-frequency gating weight maps for each high-frequency sub-band, highlighting key discriminative information such as instrument edges. The frequency domain enhanced features obtained after reconstruction using the two-dimensional discrete wavelet inverse transform are concatenated with the multi-scale pooling features output from the multi-scale pooling branch through a first concatenation layer along the channel dimension and then input into the second convolution module for fusion. This preserves contextual information under different receptive fields and enhances the model's ability to discriminate instrument edges.

[0075] In a specific embodiment, the multi-scale orientation perception gating module includes a third convolutional module, a first channel separation unit, a fourth convolutional module, parallel multi-scale local feature branches and orthogonal strip orientation branches, a gating weighting unit, a third stitching layer, and a fifth convolutional module connected in sequence; the multi-scale local feature branches include a second depthwise separable convolutional layer, a third depthwise separable convolutional layer, and a fourth depthwise separable convolutional layer connected in parallel, as well as a fourth stitching layer, a second convolutional layer, a first batch of normalization layers, and a sixth convolutional module connected in sequence; the orthogonal strip orientation branches include a third convolutional module, a second channel separation unit, a fourth ... The system consists of a first convolutional layer, a second convolutional layer, a fifth concatenation layer, a third convolutional layer, a second batch normalization layer, and a third sigmoid activation function layer, connected sequentially. The input features from the multi-scale orientation sensing gating module are input into the third convolutional module for convolution processing to obtain the first initial features. These initial features are then input into the first channel separation unit, where they are evenly divided along the channel dimension into first identity branch features and first processing branch features. The first processing branch features are input into the fourth convolutional module to obtain the first basic features. Finally, the first basic features are input into the multi-scale local feature branches and orthogonal bars, respectively. Directional branches are used in the multi-scale local feature branch. The first basic feature is processed by the second, third, and fourth depthwise separable convolutional layers. The outputs of these three layers are concatenated along the channel dimension by the fourth concatenation layer, and then sequentially passed through the second convolutional layer, the first batch of normalization layers, and the sixth convolutional module for channel fusion and nonlinear transformation to obtain local multi-scale features. In the orthogonal strip directional branch, the first basic feature is processed by the first and second strip convolutional layers for directional context extraction. The outputs of both layers are processed along the channel dimension by the fifth concatenation layer. After concatenation along the channel dimension, the features are processed sequentially through the third convolutional layer, the second batch normalization layer, and the third sigmoid activation function layer to obtain the orientation gating weight map. The local multi-scale features and the orientation gating weight map are then input into the gating weighting unit, where the local multi-scale features are weighted element-wise by multiplication using the orientation gating weight map to obtain the orientation enhancement features. The first identity branch features, the first basic features, and the orientation enhancement features are concatenated along the channel dimension through the third concatenation layer and then input into the fifth convolutional module for channel compression and feature fusion to obtain the output features of the multi-scale orientation perception gating module.

[0076] For details, please refer to Figure 4 The processing procedure of the multi-scale orientation sensing gating module is shown in the following formula:

[0077] ;

[0078] ;

[0079] ;

[0080] ;

[0081] ;

[0082] ;

[0083] ;

[0084] ;

[0085] in, The feature map is used as input to the multi-scale orientation perception gating module. C represents the number of channels, and H and W represent the height and width of the feature map, respectively. Let be the set of real numbers. This is the first initial feature; This indicates that the channel is divided into two parts along the channel dimension; this operation corresponds to the first channel separation unit. This is the first identity branch feature. Features of the first processing branch The first basic feature; , The kernel size is respectively , The depthwise separable convolution operation corresponds to the second and third depthwise separable convolutional layers, respectively. The kernel size is A depthwise separable convolution operation with a hole ratio d of 2, which corresponds to the fourth depthwise separable convolutional layer; For multi-scale local stitching features, by Passing through , and The processed components are then stitched together along the channel dimension. For batch normalization operations, For local multi-scale features, ; , The kernel size is respectively , Strip convolution operation; For the direction-gated weighted graph, For directional enhancement features, This refers to the output characteristics of the multi-scale orientation sensing gating module.

[0086] The multi-scale orientation perception gating module divides the initial features into first identity branch features and first processing branch features through the first channel separation unit to reduce the number of parameters and computational cost. In the processing branch, the multi-scale local feature branch utilizes depthwise separable convolution operations with kernel sizes of 3×3 and 5×5, as well as depthwise separable convolution operations with a dilation rate of 2 and a kernel size of 3×3, to extract local features under different receptive fields. The orthogonal strip orientation branch utilizes strip convolutions with kernel sizes of 7×1 and 1×7 to extract long-range contextual information in the vertical and horizontal directions, generating an orientation gating weight map. Finally, the orientation gating weight map is integrated into the local multi-scale features through the gating weighting unit and concatenated with the first basic features and the first identity branch features, thereby enhancing the model's ability to detect slender, multi-directional surgical instruments.

[0087] The improved backbone network in this embodiment includes: a first layer is a CBS module, which uses a 64-channel convolutional layer with a 3×3 kernel to perform convolution operations on the input laparoscopic surgical image, obtaining a 64-channel feature map, and then reducing it to half its original size; a second layer is a CBS module, which uses a 128-channel convolutional layer with a 3×3 kernel to further convolve the feature map, obtaining a 128-channel feature map, and then reducing it to half its original size again; a third layer is a multi-scale orientation perception gating module with 128 channels; and a fourth layer is a CBS module that uses 256 channels to convolve the feature map. The first layer performs convolution on the feature map using 512 channels, then reduces it to half its original size. The second layer is a multi-scale orientation-aware gating module with 512 channels, outputting the second backbone feature. The third layer is a CBS module with 1024 channels, then reduces it to half its original size. The fourth layer is a multi-scale orientation-aware gating module with 1024 channels. The fifth layer is a multi-scale orientation-aware gating module with 256 channels, outputting the first backbone feature. The sixth layer is a CBS module with 512 channels, then reduces it to half its original size. The seventh layer is a multi-scale orientation-aware gating module with 512 channels, outputting the second backbone feature. The eighth layer is a CBS module with 1024 channels, then reduces it to half its original size. The ninth layer is a multi-scale orientation-aware gating module with 1024 channels. The tenth layer is a frequency domain enhanced pyramid pooling module, outputting the third backbone feature. The first, second, and third backbone features constitute the multi-scale image features output by the improved backbone network.

[0088] In a specific embodiment, the cross-scale progressive feature fusion neck network includes two upsampling modules, five dual-domain cross-coupling recalibration modules, five concatenation layers, and six convolutional layers. The improved backbone network outputs first, second, and third backbone features with sequentially reduced resolutions, which are then input into the cross-scale progressive feature fusion neck network. The third backbone feature is the output feature of the frequency domain enhanced pyramid pooling module. In the cross-scale progressive feature fusion neck network, the third backbone feature is input into the first convolutional layer to obtain the first feature; the first feature is input into the first upsampling module to obtain the second feature; the second backbone feature is input into the second convolutional layer to obtain the third feature; the second and third features are input into the first concatenation layer to obtain the fourth feature; and the fourth feature is input into the first dual-domain cross-coupling recalibration module to obtain the fifth feature.

[0089] The fifth feature is input into the second upsampling module to obtain the sixth feature; the first backbone feature is input into the third convolutional layer to obtain the seventh feature; the sixth and seventh features are input into the second concatenation layer to obtain the eighth feature; the eighth feature is input into the second dual-domain cross-coupling recalibration module to obtain the ninth feature; the ninth feature is input into the fourth convolutional layer to obtain the tenth feature; the tenth and seventh features are input into the third concatenation layer to obtain the eleventh feature; the eleventh feature is input into the third dual-domain cross-coupling recalibration module to obtain the first detection feature.

[0090] The first detection feature is input into the fifth convolutional layer for downsampling to obtain the twelfth feature; the twelfth and fifth features are input into the fourth concatenation layer to obtain the thirteenth feature; the thirteenth feature is input into the fourth dual-domain cross-coupling recalibration module to obtain the second detection feature;

[0091] The second detection feature is input into the sixth convolutional layer for downsampling to obtain the fourteenth feature; the fourteenth feature and the first feature are input into the fifth concatenation layer to obtain the fifteenth feature; the fifteenth feature is input into the fifth dual-domain cross-coupling recalibration module to obtain the third detection feature; the first detection feature, the second detection feature and the third detection feature constitute a multi-scale fusion feature.

[0092] Specifically, the cross-scale progressive feature fusion neck network in the embodiments of this application consists of 2 upsampling modules, 5 dual-domain cross-coupled recalibration modules, 5 concatenation layers, and 6 convolutional layers. The connection relationships of its internal structures are as follows: Figure 2 As shown.

[0093] In a specific embodiment, the dual-domain cross-coupling recalibration module includes a seventh convolutional module, a second channel separation unit, an eighth convolutional module, two dual-domain cross-coupling recalibration units arranged in series, a sixth splicing layer, and a ninth convolutional module connected in sequence.

[0094] The input features of the dual-domain cross-coupling recalibration module are first processed by the seventh convolutional module for channel adjustment and local feature mapping to obtain the second initial features. Subsequently, the second initial features are divided into the second identity branch features and the second processing branch features along the channel dimension by the second channel separation unit. The second processing branch features are input into the eighth convolutional module to obtain the second basic features. The second basic features are then processed by two dual-domain cross-coupling recalibration units to obtain the cross-coupling features and the recalibration enhancement features, respectively. The second identity branch features, the second processing branch features, the cross-coupling features, and the recalibration enhancement features are then input into the sixth concatenation layer for concatenation along the channel dimension, and then processed by the ninth convolutional module for channel compression and feature fusion to obtain the output features of the dual-domain cross-coupling recalibration module.

[0095] The dual-domain cross-coupled recalibration unit includes a first branch and a second branch set in parallel, a dual-domain interactive weight mapping module and a fourth convolutional layer. The first branch includes a tenth convolutional module and a channel enhancement unit connected in sequence. The second branch includes an eleventh convolutional module and a spatial enhancement unit connected in sequence.

[0096] The input features of the dual-domain cross-coupled recalibration unit are input into the first branch and the second branch, respectively, and processed by the tenth and eleventh convolutional modules to obtain the first branch features and the second branch features. The first branch features are input into the channel enhancement unit to obtain the channel weights. The channel weights are multiplied element-wise with the first branch features to obtain the first recalibration features. In the channel enhancement unit, the first branch features pass through the first global average pooling layer, the first fully connected layer, the ReLU activation function layer, the second fully connected layer, and the fourth Sigmoid activation function layer in sequence to obtain the channel weights.

[0097] The second branch feature is input into the spatial augmentation unit to obtain spatial weights. The spatial weights are then multiplied element-wise with the second branch feature to obtain the second recalibrated feature. In the spatial augmentation unit, the second branch feature is input into the second global average pooling layer and the global max pooling layer to obtain the average spatial description map and the maximum spatial description map. The average spatial description map and the maximum spatial description map are concatenated along the channel dimension and then passed through the fifth convolutional layer and the fifth sigmoid activation function layer to generate spatial weights.

[0098] The first and second calibration features are input into the dual-domain interactive weight mapping module for cross-coupling. The first and second calibration features are added element-wise to obtain a fused feature. Then, the element-wise mean of the fused feature is calculated to obtain a local mean feature. The first calibration feature is subtracted element-wise from the local mean feature, and then passed through a sixth sigmoid activation function layer to obtain a second feature sub-map. The second feature sub-map is multiplied element-wise with the second calibration feature, and then passed through a seventh sigmoid activation function layer to obtain the second branch weights. The second branch features are weighted using the second branch weights to obtain the second cross-recalibration feature. The second calibration feature is subtracted element-wise from the local mean feature, and then passed through a sixth sigmoid activation function layer to obtain a first feature sub-map. The first feature sub-map is multiplied element-wise with the first calibration feature, and then passed through a seventh sigmoid activation function layer to obtain the first branch weights. The first branch features are weighted using the first branch weights to obtain the first cross-recalibration feature.

[0099] The first and second cross-recalibration features are concatenated and then residually connected with the basic features. After being fused through the fourth convolutional layer, the output features of the dual-domain cross-coupled recalibration unit are obtained.

[0100] For details, please refer to Figure 5 The dual-domain cross-coupling recalibration module comprises a seventh convolutional module, a second channel separation unit, an eighth convolutional module, two cascaded dual-domain cross-coupling recalibration units, a sixth concatenation layer, and a ninth convolutional module, all connected in sequence. The input features of the dual-domain cross-coupling recalibration module first undergo channel adjustment and local feature mapping in the seventh convolutional module to obtain the second initial features. Subsequently, the second channel separation unit divides the second initial features along the channel dimension into a second identity branch feature and a second processing branch feature. The second identity branch feature is used to preserve the basic semantic and detailed information in the original fused features. The second processing branch feature is input into the eighth convolutional module to obtain the second basic feature. The second basic feature then passes through two dual-domain cross-coupling recalibration units sequentially to perform interactive modeling and adaptive enhancement in both the channel and spatial domains.

[0101] The dual-domain cross-coupled recalibration unit comprises a first branch and a second branch configured in parallel. The first branch performs channel domain recalibration, and the second branch performs spatial domain recalibration. The first branch uses a tenth convolutional module and a channel enhancement unit (SE), while the second branch uses an eleventh convolutional module and a spatial enhancement unit (CE).

[0102] For the first dual-domain cross-coupling recalibration unit, the second basic feature is processed by the tenth and eleventh convolutional modules with a kernel size of 1×1, respectively, to obtain the first branch feature and the second branch feature. For example... Figure 6 As shown, the first branch feature is input to the channel enhancement unit. The channel enhancement unit first performs global average pooling on the first branch feature to obtain a channel description vector characterizing the global response intensity of each channel. Then, it sequentially passes through a first fully connected layer, a ReLU activation function layer, a second fully connected layer, and a fourth Sigmoid activation function layer to generate channel weights corresponding to each channel of the first branch feature. Finally, the channel weights are multiplied element-wise with the first branch feature to obtain the first recalibrated feature after channel recalibration. Through this process, the dual-domain cross-coupling recalibration module can enhance the effective channel responses related to instrument edges, clamping ends, rod-like structures, etc., based on the response differences of surgical instrument features in different channels, while suppressing interfering channel responses such as tissue background, smoke, and reflections.

[0103] like Figure 7 As shown, the second branch feature is input to the spatial enhancement unit. The spatial enhancement unit first performs global average pooling (GAP) and global max pooling (GMP) along the channel dimension to obtain the average spatial description map and the maximum spatial description map. Then, the average and maximum spatial description maps are concatenated along the channel dimension and passed through a fifth convolutional layer with a kernel size of 7×7 and a fifth sigmoid activation function layer to generate spatial weights. Finally, the spatial weights are multiplied element-wise with the second branch feature to obtain the spatially recalibrated second recalibrated feature. Through this process, the dual-domain cross-coupled recalibration module can highlight the area where surgical instruments are located in space, while weakening background tissue, blood occlusion, specular reflection, and irrelevant texture areas in laparoscopic images, thereby improving the response intensity of the neck network to the instrument target area.

[0104] After obtaining the first and second calibration features, they are input into the dual-domain interactive weight mapping module for cross-coupling. Specifically, the dual-domain interactive weight mapping module first adds the first and second calibration features element-wise to obtain the fused features; then, it calculates the element-wise mean of the fused features to obtain the local mean features. The local mean features are used to characterize the common response benchmark between the channel domain recalibration results and the spatial domain recalibration results. Further, the first calibration features are subtracted element-wise from the local mean features, and then passed through the sixth Sigmoid activation function layer to obtain the second feature submap; the second feature submap is multiplied element-wise with the second calibration features, and then passed through the seventh Sigmoid activation function layer to obtain the second branch weights; the second branch features are weighted using the second branch weights to obtain the second cross-recalibration features. Accordingly, the second recalibrated feature is subtracted element-wise from the local mean feature, and then passed through the sixth Sigmoid activation function layer to obtain the first feature sub-map; the first feature sub-map is multiplied element-wise with the first recalibrated feature, and then passed through the seventh Sigmoid activation function layer to obtain the first branch weight; the first branch feature is weighted using the first branch weight to obtain the first cross-recalibration feature.

[0105] Through the aforementioned cross-coupling method, the channel domain recalibration results can guide the generation of spatial domain weights, and the spatial domain recalibration results can also guide the generation of channel domain weights. This ensures that channel attention information and spatial attention information no longer act independently, but interact bidirectionally under the constraint of a common response benchmark. Therefore, the dual-domain cross-coupling recalibration module can simultaneously focus on "which channels are more important" and "which spatial regions are more important" during multi-scale fusion, thereby improving the feature representation capabilities for slender surgical instruments, partially occluded instruments, and instruments with significant scale changes.

[0106] The first and second cross-recalibration features are concatenated along the channel dimension and then residually connected with the basic features. This concatenation is then fused through a fourth convolutional layer with a 1×1 kernel to obtain the cross-coupling features. These cross-coupling features are further input into a second dual-domain cross-coupling recalibration unit for deep recursive enhancement, resulting in recalibrated enhanced features. Finally, the second identity branch feature, the second processing branch feature, the cross-coupling features, and the recalibrated enhanced features are input into a sixth concatenation layer, concatenated along the channel dimension, and then processed by a ninth convolutional module for channel compression and feature fusion, yielding the output features of the dual-domain cross-coupling recalibration module.

[0107] The processing procedure of the dual-domain cross-coupled recalibration module can be represented as follows:

[0108] Let the input features be:

[0109] ;

[0110] in, This is the input feature map for the dual-domain cross-coupling recalibration module. This represents the number of channels in the feature map. Indicates the height of the feature map. Indicates the width of the feature map.

[0111] The second initial feature is obtained after processing by the seventh convolutional module. :

[0112] ;

[0113] The second initial feature Divided into second identity branch features along the channel dimension Second processing branch features As shown in the following formula:

[0114] ;

[0115] in, ;

[0116] Processing branch features The second basic feature is obtained after passing through the eighth convolutional module. As shown in the following formula:

[0117] ;

[0118] Second basic characteristic The first branch features are obtained by passing the features through the tenth and eleventh convolutional modules respectively. Second branch features As shown in the following formula:

[0119] , ;

[0120] Among them, the channel enhancement unit provides features for the first branch. The processing steps are as follows: First, perform a global average pooling operation to obtain the channel description vector. The channel weights are then obtained by passing through a fully connected layer, a ReLU activation function, another fully connected layer, and a Sigmoid activation function. Finally, the channel weights are used to recalibrate the first branch features.

[0121] ;

[0122] in, For the first The global average pooling output scalar for each channel represents the output of the first branch feature map. No. All spatial locations on each channel The response value is taken as the average. For the spatial height direction index, This is an index in the direction of spatial width;

[0123] ;

[0124] For all channels The channel description vector is composed of; This is the weight matrix of the first fully connected layer. This refers to the channel compression ratio. Represents the ReLU activation function, used to introduce nonlinearity; This is the weight matrix of the second fully connected layer, which restores the compressed features to the original number of channels; For the first The channel attention weight scalar corresponding to each channel.

[0125] ;

[0126] The first branch feature map after channel recalibration is in The eigenvector at that location; For the attention weights of the corresponding channels, by adjusting... Each channel is weighted and scaled.

[0127] Spatial enhancement unit for second branch features The processing steps are as follows: First, global average pooling and global max pooling operations are performed in parallel. Then, the results of the two operations are concatenated and passed through a fifth convolutional layer with a kernel size of 7×7 and a fifth sigmoid activation function layer to obtain spatial weights. Finally, the spatial weights are used to multiply the features of the second branch element by element, as shown in the following formula:

[0128] ;

[0129] ;

[0130] in, For the second branch feature map In spatial location The result of global average pooling is obtained by taking the average value along all channels. For channel indexing; A spatial description graph generated for the global average pooling operation; For the second branch feature map In spatial location The result of global max pooling obtained by taking the maximum value along all channels; The maximum space description graph generated for the global max pooling operation.

[0131] ;

[0132] in, This indicates the global average pooling graph. With global max pooling graph It splices along the channel dimension, simultaneously encoding both mean and extreme values, two types of spatial statistical information; Indicates the kernel size as Convolution operations with 1 output channel; For spatial weights.

[0133] ;

[0134] in, For the spatially recalibrated second branch feature map in The feature vector at the location; using right The corresponding spatial positions of each channel are weighted and scaled.

[0135] Then, the first calibration feature Second calibration features The input dual-domain cross-coupled recalibration unit first adds the two features element-wise to obtain the fused feature U. Then, it calculates the element-wise mean of the sum to obtain the local mean feature M. Each branch feature is subtracted from the local mean feature M and passed through the sixth Sigmoid activation function layer before being recalibrated with the first recalibrated feature. Second calibration features After element-wise multiplication, the weights are then processed through the seventh Sigmoid activation function layer to obtain the weights of the first branch. Second branch weight :

[0136] ;

[0137] ;

[0138] ;

[0139] ;

[0140] The cross-coupling feature is obtained as follows:

[0141] ;

[0142] ;

[0143] in, This indicates recalibrated enhancement features. This represents the function corresponding to the two-domain cross-coupled recalibration unit.

[0144] Finally, the second identity branch feature Features of the second processing branch Cross-coupling characteristics And recalibration enhancement features The features are concatenated along the channel dimension and processed by the ninth convolutional module to obtain the output features of the dual-domain cross-coupled recalibration module. As shown in the following formula:

[0145] ;

[0146] The dual-domain cross-coupling recalibration module, through the synergistic effect of the channel enhancement unit, spatial enhancement unit, and dual-domain interactive weight mapping module, enables the neck network to simultaneously utilize semantic channel information and target spatial location information during cross-scale feature fusion, dynamically filtering and enhancing instrument features at different scales. For laparoscopic surgical instrument detection tasks, this module can effectively alleviate problems such as slender instruments, large target scale variations, local occlusion, complex background textures, and specular reflection interference, improving the discriminativeness and robustness of cross-scale fused features.

[0147] In a specific embodiment, the directional decoupling gradient modulation loss includes a complete intersection-union term, a rigid aspect ratio prior term, and a directional decoupling gradient modulation mechanism. The directional decoupling gradient modulation loss affects the predicted bounding box output by the detection head. With the corresponding truth box The forward computation of satisfies the following formula:

[0148] ;

[0149] ;

[0150] ;

[0151] ;

[0152] ;

[0153] ;

[0154] ;

[0155] Directional decoupling gradient modulation loss affects the predicted bounding box of the detector head output. With the corresponding truth box The auxiliary quantity for inverse gradient modulation is calculated according to the following formula:

[0156] ;

[0157] ;

[0158] ;

[0159] Directional decoupling gradient modulation loss affects the predicted bounding box of the detector head output. With the corresponding truth box The actual backpropagation gradient calculation is shown below:

[0160] ;

[0161] in, Predicted bounding boxes for the head regression output The coordinates; For truth boxes The coordinates; The width and height of the predicted bounding box; The width and height of the truth box; This is the intersection-union ratio (IUU) of the predicted bounding box and the ground truth bounding box. The square of the distance between the center point of the prediction box and the center point of the truth box; It is the square of the diagonal length of the smallest bounding rectangle that simultaneously contains both the predicted and ground truth boxes; As a measure of aspect ratio consistency, The weights are used for the standard full intersection-union loss method, which does not backpropagate gradients. It is the arctangent function; Indicates that it belongs to the first in the training set. The set of all truth boxes for each instrument category; i represents the index of the endpoint of the predicted box coordinates. i=1 corresponds to the coordinates of the upper left endpoint of the prediction box. i=2 corresponds to the coordinates of the lower right endpoint of the prediction box. j represents the sample index of the truth box of the c-th type of device in the training set; For the first Rigid a priori datum for aspect ratio of each instrument category; This indicates taking the median; It is a logarithmic function; The aspect ratio is a rigid prior weight; For a rigid aspect ratio prior, its gradient is propagated back in the conventional manner, without passing through... modulation; This represents the forward loss value for bounding box regression; The thin length is calculated from the width and height of the truth box; This refers to the gradient modulation sensitivity coefficient; It is the hyperbolic tangent function; These are the gradient modulation coefficients; , These represent taking the maximum and minimum values, respectively. These are the transverse gradient modulation weights and the longitudinal gradient modulation weights, respectively. , The conventional gradient is obtained by taking the partial derivatives of the partial term with respect to the x and y coordinates of the predicted bounding box for complete intersection and merging. , The general gradient of the aspect ratio rigid prior with respect to the x and y coordinates of the prediction box; The total gradient is the actual gradient back to the horizontal and vertical coordinates of the prediction box. The total gradient is propagated back layer by layer through the detection head and the improved backbone network to update the network parameters of the laparoscopic surgical instrument detection model.

[0162] Specifically, during the training of the laparoscopic surgical instrument detection model, directional decoupling gradient modulation loss is used to constrain the bounding box regression results output by the detection head. The detection head network still adopts the existing standard YOLOv8 detection head structure. The classification branch is trained using classification loss, the distribution regression branch is trained using distribution focal loss, and the bounding box regression branch uses directional decoupling gradient modulation loss to replace or improve the standard full intersection-union loss component. The directional decoupling gradient modulation loss includes a full intersection-union component, a rigid aspect ratio prior, and a directional decoupling gradient modulation mechanism.

[0163] Specifically, during the training of the laparoscopic surgical instrument detection model, the loss is calculated based on the predicted bounding box output by the detection head, the ground truth bounding box matching the predicted bounding box, and the instrument category corresponding to the ground truth bounding box. First, the width and height of the predicted bounding box, the width and height of the ground truth bounding box, and the perfect intersection-union term are calculated from the predicted bounding box and the ground truth bounding box. Then, the median of the logarithm of the width-to-height ratio of the ground truth bounding box for each instrument category in the training set is taken to obtain the rigid aspect ratio prior for the corresponding category. The absolute value of the difference between the logarithm of the width-to-height ratio of the predicted bounding box and the rigid aspect ratio prior is multiplied by the rigid aspect ratio prior weight to obtain the rigid aspect ratio prior term. The rigid aspect ratio prior term is added to the perfect intersection-union term to obtain the forward loss value of the bounding box regression.

[0164] Simultaneously, the thin length is calculated from the width and height of the truth box, and the gradient modulation coefficient is obtained based on the thin length. Then, according to the long axis direction of the truth box, the gradient modulation coefficient and its reciprocal are respectively assigned as the horizontal gradient modulation weight and the vertical gradient modulation weight. The horizontal gradient modulation weight and the vertical gradient modulation weight are determined by the shape of the truth box, are constant for the prediction box, and do not participate in the forward numerical calculation of the complete intersection and union fraction.

[0165] During backpropagation, the gradient of the complete intersection and union term with respect to the x-coordinate of the prediction box is scaled by the lateral gradient modulation weight, and the gradient of the complete intersection and union term with respect to the y-coordinate of the prediction box is scaled by the longitudinal gradient modulation weight. The gradient of the aspect ratio rigid prior term is backpropagated in the conventional manner without being modulated by the lateral and longitudinal gradient modulation weights. Finally, the gradient of the complete intersection and union term after directional modulation is added to the gradient of the aspect ratio rigid prior term to obtain the total gradient back to the prediction box coordinates. This gradient is then backpropagated layer by layer through the detection head and the improved backbone network to update the parameters of the laparoscopic surgical instrument detection model.

[0166] When the slenderness is equal to 1 and the aspect ratio rigid prior weight is zero, the directional decoupling gradient modulation loss degenerates into the standard perfect intersection-union ratio loss.

[0167] Gradient modulation sensitivity coefficient Rigid prior weights with aspect ratio For the pre-defined hyperparameters, in one example, , ;remove and Apart from the input predicted bounding box coordinates, ground truth bounding box coordinates, or statistical adaptation of ground truth bounding boxes in the training set, all other quantities are obtained by adapting to the input predicted bounding box coordinates, ground truth bounding box coordinates, or ground truth bounding boxes in the training set, and vary with the training samples or training set.

[0168] The laparoscopic surgical instrument target detection model mentioned in the embodiments of this application uses the open-source m2cai16-tool-locations dataset, in which all samples are derived from actual images of real clinical laparoscopic surgical scenarios (denoted as GroundTruth). The dataset presents challenging problems such as complex in vivo tissue backgrounds, severe interference from surgical smoke and reflections, large differences in instrument scale, and partial occlusion. Training the model using this dataset can better improve the effectiveness and robustness of the model in detecting laparoscopic surgical instruments in complex surgical environments.

[0169] The following section presents ablation experiments on the various improved modules of the laparoscopic surgical instrument testing model. The results of the surgical instrument testing are analyzed to verify the performance improvement effect of the various improved modules proposed in this application on the model, as shown in Table 1.

[0170] Table 1. Results of module ablation experiments:

[0171] Multi-scale orientation-gated module (MDGA) Frequency Domain Enhanced Pyramid Pooling Module (WSPPF) Cross-scale progressive feature fusion neck network (MCPF-Neck) Directional decoupling gradient modulation loss mAP50 (%) mAP50: 95 (%) Number of parameters (M) × × × × 93.2 57.2 3.0 √ × × × 93.7 57.5 4.2 √ √ × × 94.1 57.9 5.3 √ √ √ × 94.3 58.2 8.6 √ √ √ √ 94.8 58.4 8.6

[0172] Experimental results show that by using the YOLOv8 model as the baseline (denoted as Baseline) and gradually introducing various improved modules, the overall detection performance of the model shows a stable upward trend. Without any improved modules, the mAP50 and mAP50:95 of the YOLOv8 model are 93.2% and 57.2%, respectively, with 3.0M parameters. After introducing only the multi-scale orientation-aware gating module (denoted as Baseline+MDGA), the mAP50 increases to 93.7%, and the mAP50:95 increases to 57.5%, representing improvements of 0.5% and 0.3% respectively compared to the baseline model. This indicates that the multi-scale orientation-aware gating module can effectively capture the multi-scale local features and orientation context information of surgical instruments, enhancing the model's ability to represent slender, oriented instruments.

[0173] Building upon this foundation, the addition of a frequency domain enhancement pyramid pooling module (denoted as Baseline+MDGA+WSPPF) further improved the model's mAP50 and mAP50:95 to 94.1% and 57.9%, respectively, representing increases of 0.4% and 0.4% compared to when only MDGA was added. This indicates that through frequency domain decomposition, low-frequency redundancy suppression, and high-frequency detail enhancement, the model can more fully extract the edges, contours, and texture details of surgical instruments in laparoscopic images, thereby mitigating the feature degradation problem caused by reflections, smoke, and tissue background interference.

[0174] After further introducing a cross-scale progressive feature fusion neck network (denoted as Baseline+MDGA+WSPPF+MCPF-Neck), the model's mAP50 and mAP50:95 were further improved to 94.3% and 58.2%, respectively, representing increases of 0.2% and 0.3% compared to the previous stage. This indicates that cross-scale progressive feature fusion can promote full interaction between shallow detail features and deep semantic features. Simultaneously, combined with a dual-domain cross-coupling recalibration mechanism, effective instrument features in the channel and spatial domains are dynamically filtered and enhanced, thereby improving the model's ability to detect instrument targets of different scales, poses, and partial occlusion.

[0175] Ultimately, by simultaneously introducing a multi-scale orientation-aware gating module, a frequency-domain enhanced pyramid pooling module, a cross-scale progressive feature fusion neck network, and orientation-decoupled gradient modulation loss (denoted as Baseline+MDGA+WSPPF+MCPF-Neck+Orientation-Decoupled Gradient Modulation Loss), the model achieved an mAP50 of 94.8% and an mAP50:95 of 58.4%, representing improvements of 1.6% and 1.2% respectively compared to the baseline model. Furthermore, the final model had 8.6M parameters, which was not increased compared to before the introduction of the orientation-decoupled gradient modulation loss, indicating that this loss function primarily functions during the training phase by modulating the bounding box regression gradient without introducing additional network parameters. Overall, the improved modules exhibit good complementarity and synergistic gains: MDGA enhances orientation awareness and multi-scale local feature extraction, WSPPF strengthens the representation of instrument edges and frequency domain details, MCPF-Neck improves the quality of cross-scale feature fusion, and the orientation-decoupled gradient modulation loss further optimizes the bounding box regression process for slender instruments, thereby effectively improving the accuracy and robustness of surgical instrument detection in complex laparoscopic surgical scenarios.

[0176] Furthermore, Figure 8 This diagram illustrates the detection results obtained after processing a laparoscopic surgical instrument detection model provided in this embodiment of the invention. Even in laparoscopic surgical images with complex tissue backgrounds, instrument occlusion, strong local reflections, and elongated target shapes, the trained laparoscopic surgical instrument detection model can identify surgical instruments such as Grasper, Irrigator, SpecimenBag, and Hook, and output corresponding category labels, confidence scores, and bounding box location information. The bounding boxes in the detection results cover the main area of ​​the corresponding surgical instrument, indicating that this embodiment of the invention, through the synergistic effect of a multi-scale orientation perception gating module, a frequency domain enhanced pyramid pooling module, a cross-scale progressive feature fusion neck network, and orientation decoupling gradient modulation loss, can enhance the model's ability to represent the directional structure, edge contour features, and effective target areas in complex backgrounds of elongated instruments, thereby improving the accuracy and stability of the laparoscopic surgical instrument detection results.

[0177] S2. Acquire the laparoscopic surgical image to be detected and input it into the trained laparoscopic surgical instrument detection model. First, the improved backbone network is used for feature extraction to obtain multi-scale image features. The multi-scale image features are input into the cross-scale progressive feature fusion neck network for feature fusion to obtain cross-scale fused features. The cross-scale fused features are input into the detection head to obtain the detection results of each surgical instrument in the laparoscopic surgical image to be detected.

[0178] Specifically, the trained laparoscopic surgical instrument detection model proposed in the embodiments of this application is deployed. After the laparoscopic surgical image to be detected, obtained during laparoscopic surgery, is input into the trained laparoscopic surgical instrument detection model, the detection results of the surgical instruments in the image can be accurately identified. The detection results include the position coordinates of the predicted bounding box and the instrument category.

[0179] The data mentioned in the embodiments of this application were all obtained in a legal and compliant manner, and their application scenarios are also legal and compliant.

[0180] Further reference Figure 9 As an implementation of the methods shown in the above figures, this application provides an embodiment of a laparoscopic surgical instrument detection device based on an improved YOLOv8. This device embodiment is similar to... Figure 1 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0181] This application provides a laparoscopic surgical instrument detection device based on an improved YOLOv8, comprising:

[0182] Model building module 1 is configured to build and train a laparoscopic surgical instrument detection model based on an improved YOLOv8, resulting in a trained laparoscopic surgical instrument detection model. The laparoscopic surgical instrument detection model, based on the standard YOLOv8, replaces the C2F module in the backbone network with a multi-scale orientation-aware gating module, replaces the SPPF module with a frequency-domain enhanced pyramid pooling module, and replaces the neck network with a cross-scale progressive feature fusion neck network. The cross-scale progressive feature fusion neck network employs a dual-domain cross-coupling recalibration module, and during training, orientation-decoupled gradient modulation loss is used to constrain the bounding box regression results output by the detection head.

[0183] Prediction module 2 is configured to acquire the laparoscopic surgical image to be detected and input it into the trained laparoscopic surgical instrument detection model. First, the improved backbone network is used for feature extraction to obtain multi-scale image features. The multi-scale image features are input into the cross-scale progressive feature fusion neck network for feature fusion to obtain cross-scale fused features. The cross-scale fused features are input into the detection head to obtain the detection results of each surgical instrument in the laparoscopic surgical image to be detected.

[0184] Figure 10 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. For example... Figure 10As shown, the electronic device of this embodiment includes a processor 1001 and a memory 1002; wherein the memory 1002 is used to store computer execution instructions; and the processor 1001 is used to execute the computer execution instructions stored in the memory to implement the various steps performed by the electronic device in the above embodiment. For details, please refer to the relevant descriptions in the foregoing method embodiments.

[0185] Alternatively, the memory 1002 can be either standalone or integrated with the processor 1001.

[0186] When the memory 1002 is set up independently, the electronic device also includes a bus 1003 for connecting the memory 1002 and the processor 1001.

[0187] This invention also provides a computer storage medium storing computer execution instructions, which, when executed by the processor 1001, implement the above method.

[0188] This invention also provides a computer program product, including a computer program, which, when executed by a processor 1001, implements the above-described method.

[0189] In the embodiments provided by this invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0190] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to implement the solution of this embodiment according to actual needs.

[0191] Furthermore, the functional modules in the various embodiments of this invention can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The unit formed by the above modules can be implemented in hardware or in the form of hardware plus software functional units.

[0192] The integrated modules implemented as software functional modules described above can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor 1001 to execute certain steps of the methods of the various embodiments of this application.

[0193] It should be understood that the processor 1001 described above can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor, or the processor 1001 can be any conventional processor 1001. The steps of the method disclosed in this invention can be directly manifested as execution by the hardware processor 1001, or execution by a combination of hardware and software modules within the processor 1001.

[0194] The memory 1002 may include high-speed RAM memory, and may also include non-volatile memory NVM, such as at least one disk storage device, and may also be a USB flash drive, portable hard drive, read-only memory, disk or optical disc, etc.

[0195] Bus 1003 can be an Industry Standard Architecture (ISA), a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Bus 1003 can be divided into address bus, data bus, control bus, etc. For ease of illustration, the bus 1003 in the accompanying drawings of this application is not limited to only one bus 1003 or one type of bus 1003.

[0196] The aforementioned storage medium can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium accessible to general-purpose or special-purpose computers.

[0197] An exemplary storage medium is coupled to a processor 1001, enabling the processor 1001 to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor 1001. The processor 1001 and the storage medium can reside in an application-specific integrated circuit (ASIC). Alternatively, the processor 1001 and the storage medium can exist as discrete components in an electronic device or a main control device.

[0198] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0199] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting laparoscopic surgical instruments based on an improved YOLOv8, characterized in that, Includes the following steps: A laparoscopic surgical instrument detection model based on an improved YOLOv8 was constructed and trained to obtain the trained laparoscopic surgical instrument detection model. The laparoscopic surgical instrument detection model, based on the standard YOLOv8, replaces the C2F module in the backbone network with a multi-scale orientation perception gating module. The multi-scale orientation perception gating module includes a third convolutional module, a first channel separation unit, a fourth convolutional module, parallel multi-scale local feature branches and orthogonal strip direction branches, a gating weighting unit, a third stitching layer, and a fifth convolutional module connected in sequence. The SPPF module is replaced with a frequency domain enhanced pyramid pooling module. The frequency domain enhanced pyramid pooling module includes a first convolutional module, parallel frequency domain enhancement branches and multi-scale pooling branches, a first stitching layer, and a second... The convolutional module replaces the neck network with a cross-scale progressive feature fusion neck network. This cross-scale progressive feature fusion neck network employs a dual-domain cross-coupling recalibration module. The cross-scale progressive feature fusion neck network includes two upsampling modules, five dual-domain cross-coupling recalibration modules, five concatenation layers, and six convolutional layers. The dual-domain cross-coupling recalibration module includes a seventh convolutional module, a second channel separation unit, an eighth convolutional module, two cascaded dual-domain cross-coupling recalibration units, a sixth concatenation layer, and a ninth convolutional module, all connected sequentially. During training, a directional decoupling gradient modulation loss is used to constrain the bounding box regression results output by the detection head. This directional decoupling gradient modulation loss includes a complete intersection-and-merge term, a rigid aspect ratio prior term, and a directional decoupling gradient modulation mechanism. The laparoscopic surgical image to be detected is acquired and input into the trained laparoscopic surgical instrument detection model. First, the improved backbone network is used for feature extraction to obtain multi-scale image features. The multi-scale image features are then input into the cross-scale progressive feature fusion neck network for feature fusion to obtain cross-scale fused features. The cross-scale fused features are then input into the detection head to obtain the detection results of each surgical instrument in the laparoscopic surgical image to be detected.

2. The method for detecting laparoscopic surgical instruments based on improved YOLOv8 according to claim 1, characterized in that... The frequency domain enhancement branch includes one discrete wavelet transform unit, one low-frequency gating unit, three high-frequency gating units, and one discrete wavelet inverse transform unit. The multi-scale pooling branch includes a second splicing layer and three cascaded max pooling layers. The low-frequency gating unit includes a first convolutional layer and a first sigmoid activation function layer connected in sequence. Each high-frequency gating unit includes a first depthwise separable convolutional layer and a second sigmoid activation function layer connected in sequence. The input features of the frequency domain enhancement pyramid pooling module are processed by the first convolutional module to obtain initial enhanced features. The initial enhancement features are input into the frequency domain enhancement branch and the multi-scale pooling branch, respectively. In the frequency domain enhancement branch, the initial enhancement features undergo a two-dimensional discrete wavelet transform by the discrete wavelet transform unit to obtain low-frequency sub-bands, horizontal high-frequency sub-bands, vertical high-frequency sub-bands, and diagonal high-frequency sub-bands. The low-frequency sub-bands are processed by the low-frequency gating unit to generate a low-frequency gating weight map, and the low-frequency sub-bands are weighted using the low-frequency gating weight map to obtain suppressed low-frequency sub-bands. The horizontal high-frequency sub-bands, vertical high-frequency sub-bands, and diagonal high-frequency sub-bands are each processed by their respective high-frequency gating units to generate corresponding high-frequency gating weight maps, and the horizontal high-frequency sub-bands, vertical high-frequency sub-bands, and diagonal high-frequency sub-bands are weighted using each high-frequency gating weight map to obtain enhanced high-frequency sub-bands. The high-frequency subband, enhanced vertical high-frequency subband, and enhanced diagonal high-frequency subband are reconstructed by the discrete wavelet inverse transform unit to obtain frequency domain enhanced features. In the multi-scale pooling branch, the initial enhanced features are sequentially passed through three max pooling layers to obtain pooling features at each level. The initial enhanced features and the pooling features at each level are concatenated along the channel dimension by the second concatenation layer to obtain multi-scale pooling features. After the frequency domain enhanced features and the multi-scale pooling features are concatenated along the channel dimension by the first concatenation layer, they are input into the second convolution module for channel compression and feature fusion to obtain the output features of the frequency domain enhanced pyramid pooling module.

3. The method for detecting laparoscopic surgical instruments based on improved YOLOv8 according to claim 1, characterized in that, The multi-scale local feature branch includes a second, third, and fourth depthwise separable convolutional layer arranged in parallel, and a fourth concatenation layer, a second convolutional layer, a first batch normalization layer, and a sixth convolutional module connected in sequence. The orthogonal strip direction branch includes a first and second strip convolutional layer arranged in parallel, and a fifth concatenation layer, a third convolutional layer, a second batch normalization layer, and a third sigmoid activation function layer connected in sequence. The input features of the multi-scale direction-aware gating module are input into the third convolutional module for convolution processing to obtain a first initial feature. The first initial feature is input into the first channel separation unit and divided into a first identity branch feature and a first processing branch feature along the channel dimension. The first processing branch feature is input into the fourth convolutional module to obtain a first basic feature. The first basic feature is input into the multi-scale local feature branch and the orthogonal strip direction branch respectively. In the multi-scale local feature branch, the first basic feature passes through the second depthwise separable convolutional layer, the third depthwise separable convolutional layer, the second depthwise separable convolutional layer, the third depthwise separable convolutional layer, and the sixth convolutional module respectively. The outputs of the three features are processed by the convolutional layer and the fourth depthwise separable convolutional layer. After being concatenated along the channel dimension by the fourth splicing layer, they are sequentially processed by the second convolutional layer, the first batch of normalization layers, and the sixth convolutional module for channel fusion and nonlinear transformation to obtain local multi-scale features. In the orthogonal strip directional branches, the first basic features are processed by the first strip convolutional layer and the second strip convolutional layer for directional context extraction. The outputs of the two are concatenated along the channel dimension by the fifth splicing layer. After being processed by the third convolutional layer, the second batch of normalization layers, and the third sigmoid activation function layer to obtain a directional gating weight map. The local multi-scale features and the directional gating weight map are input into the gating weighting unit. The local multi-scale features are weighted element-wise by multiplication using the directional gating weight map to obtain directional enhancement features. The first identity branch features, the first basic features, and the directional enhancement features are concatenated along the channel dimension by the third splicing layer and then input into the fifth convolutional module for channel compression and feature fusion to obtain the output features of the multi-scale directional awareness gating module.

4. The method for detecting laparoscopic surgical instruments based on improved YOLOv8 according to claim 1, characterized in that, The improved backbone network outputs first, second, and third backbone features with sequentially decreasing resolution, which are then input into the cross-scale progressive feature fusion neck network. The third backbone feature is the output feature of the frequency domain enhanced pyramid pooling module. In the cross-scale progressive feature fusion neck network, the third backbone feature is input into the first convolutional layer to obtain the first feature; the first feature is input into the first upsampling module to obtain the second feature; the second backbone feature is input into the second convolutional layer to obtain the third feature; the second and third features are input into the first concatenation layer to obtain the fourth feature; and the fourth feature is input into the first dual-domain cross-coupling recalibration module to obtain the fifth feature. The fifth feature is input into the second upsampling module to obtain the sixth feature; the first backbone feature is input into the third convolutional layer to obtain the seventh feature; the sixth and seventh features are input into the second concatenation layer to obtain the eighth feature; the eighth feature is input into the second dual-domain cross-coupling recalibration module to obtain the ninth feature; the ninth feature is input into the fourth convolutional layer to obtain the tenth feature; the tenth and seventh features are input into the third concatenation layer to obtain the eleventh feature; the eleventh feature is input into the third dual-domain cross-coupling recalibration module to obtain the first detection feature. The first detection feature is input into the fifth convolutional layer for downsampling to obtain the twelfth feature; the twelfth and fifth features are input into the fourth concatenation layer to obtain the thirteenth feature; The thirteenth feature is input into the fourth dual-domain cross-coupling recalibration module to obtain the second detection feature; The second detection feature is input into the sixth convolutional layer for downsampling to obtain the fourteenth feature; the fourteenth feature and the first feature are input into the fifth concatenation layer to obtain the fifteenth feature; the fifteenth feature is input into the fifth dual-domain cross-coupling recalibration module to obtain the third detection feature, and the first detection feature, the second detection feature and the third detection feature constitute a multi-scale fusion feature.

5. The method for detecting laparoscopic surgical instruments based on improved YOLOv8 according to claim 1, characterized in that, The input features of the dual-domain cross-coupling recalibration module are first processed by the seventh convolution module for channel adjustment and local feature mapping to obtain the second initial features; then, the second initial features are divided into the second identity branch features and the second processing branch features along the channel dimension by the second channel separation unit. The second processing branch features are input into the eighth convolutional module to obtain the second basic features. The second basic features are then passed through two dual-domain cross-coupling recalibration units to obtain cross-coupling features and recalibration enhancement features, respectively. The second identity branch feature, the second processing branch feature, the cross-coupling feature, and the recalibration enhancement feature are input into the sixth splicing layer and spliced ​​along the channel dimension. The splicing is then performed by the ninth convolution module for channel compression and feature fusion to obtain the output features of the dual-domain cross-coupling recalibration module. The dual-domain cross-coupled recalibration unit includes a first branch and a second branch set in parallel, a dual-domain interactive weight mapping module and a fourth convolutional layer. The first branch includes a tenth convolutional module and a channel enhancement unit connected in sequence, and the second branch includes an eleventh convolutional module and a spatial enhancement unit connected in sequence. The input features of the dual-domain cross-coupled recalibration unit are respectively input to the first branch and the second branch, and processed by the tenth convolution module and the eleventh convolution module to obtain the first branch features and the second branch features; the first branch features are input to the channel enhancement unit to obtain the channel weights; the channel weights are multiplied element-wise with the first branch features to obtain the first recalibration features; In the channel enhancement unit, the first branch feature sequentially passes through a first global average pooling layer, a first fully connected layer, a ReLU activation function layer, a second fully connected layer, and a fourth Sigmoid activation function layer to obtain channel weights; The second branch feature is input into the spatial enhancement unit to obtain spatial weights. The spatial weights are then multiplied element-wise with the second branch feature to obtain the second recalibrated feature. In the spatial enhancement unit, the second branch feature is input into the second global average pooling layer and the global max pooling layer to obtain the average spatial description map and the maximum spatial description map. The average spatial description map and the maximum spatial description map are concatenated along the channel dimension and then passed through the fifth convolutional layer and the fifth sigmoid activation function layer in sequence to generate spatial weights. The first and second recalibrated features are input into a dual-domain interactive weight mapping module for cross-coupling. The first and second recalibrated features are added element-wise to obtain a fused feature. Then, the element-wise mean of the fused feature is calculated to obtain a local mean feature. The first recalibrated feature is subtracted element-wise from the local mean feature, and then passed through a sixth Sigmoid activation function layer to obtain a second feature sub-map. The second feature sub-map is multiplied element-wise with the second recalibrated feature, and then passed through a seventh Sigmoid activation function layer to obtain a second branch weight. The second branch feature is weighted using the second branch weight to obtain a second cross-recalibrated feature. The second recalibrated feature is subtracted element-wise from the local mean feature, and then passed through a sixth Sigmoid activation function layer to obtain a first feature sub-map. The first feature sub-map is multiplied element-wise with the first recalibrated feature, and then passed through a seventh Sigmoid activation function layer to obtain a first branch weight. The first branch feature is weighted using the first branch weight to obtain a first cross-recalibrated feature. The first cross-recalibration feature and the second cross-recalibration feature are concatenated and then residually connected with the basic feature. After being fused by the fourth convolutional layer, the output feature of the dual-domain cross-coupled recalibration unit is obtained.

6. The method for detecting laparoscopic surgical instruments based on improved YOLOv8 according to claim 1, characterized in that, The directional decoupling gradient modulation loss affects the predicted bounding box output by the detector head. With the corresponding truth box The forward computation of satisfies the following formula: ； ； ； ； ； ； ； The directional decoupling gradient modulation loss affects the predicted bounding box output by the detector head. With the corresponding truth box The auxiliary quantity for inverse gradient modulation is calculated according to the following formula: ； ； ； The directional decoupling gradient modulation loss affects the predicted bounding box output by the detector head. With the corresponding truth box The actual backpropagation gradient calculation is shown below: ； in, Predicted bounding boxes for the head regression output The coordinates; For truth boxes The coordinates; The width and height of the predicted bounding box; The width and height of the truth box; This is the intersection-union ratio (IUU) of the predicted bounding box and the ground truth bounding box. The square of the distance between the center point of the prediction box and the center point of the truth box; It is the square of the diagonal length of the smallest bounding rectangle that simultaneously contains both the predicted and ground truth boxes; As a measure of aspect ratio consistency, The weights are used for the standard full intersection-union loss method, which does not backpropagate gradients. It is the arctangent function; Indicates that it belongs to the first in the training set. The set of all truth boxes for each instrument category; i represents the index of the endpoint of the predicted box coordinates. i=1 corresponds to the coordinates of the upper left endpoint of the prediction box. i=2 corresponds to the coordinates of the lower right endpoint of the prediction box. j represents the sample index of the truth box of the c-th type of device in the training set; For the first Rigid a priori datum for aspect ratio of each instrument category; This indicates taking the median; It is a logarithmic function; The aspect ratio is a rigid prior weight; For a rigid aspect ratio prior, its gradient is propagated back in the conventional manner, without passing through... modulation; This represents the forward loss value for bounding box regression; The thin length is calculated from the width and height of the truth box; This refers to the gradient modulation sensitivity coefficient; It is the hyperbolic tangent function; These are the gradient modulation coefficients; , These represent taking the maximum and minimum values, respectively. These are the transverse gradient modulation weights and the longitudinal gradient modulation weights, respectively. , The conventional gradient is obtained by taking the partial derivatives of the partial term with respect to the x and y coordinates of the predicted bounding box for complete intersection and merging. , The general gradient of the aspect ratio rigid prior with respect to the x and y coordinates of the prediction box; The total gradient is the actual gradient back to the horizontal and vertical coordinates of the prediction box. The total gradient is propagated back layer by layer through the detection head and the improved backbone network to update the network parameters of the laparoscopic surgical instrument detection model.

7. A laparoscopic surgical instrument testing device based on an improved YOLOv8, characterized in that, include: The model building module is configured to build and train a laparoscopic surgical instrument detection model based on an improved YOLOv8, resulting in a trained laparoscopic surgical instrument detection model. The laparoscopic surgical instrument detection model replaces the C2F module in the backbone network of the standard YOLOv8 with a multi-scale orientation perception gating module. This multi-scale orientation perception gating module includes a third convolutional module, a first channel separation unit, a fourth convolutional module, parallel multi-scale local feature branches and orthogonal strip direction branches, a gating weighting unit, a third stitching layer, and a fifth convolutional module connected in sequence. The SPPF module is replaced with a frequency domain enhanced pyramid pooling module, which includes a first convolutional module, parallel frequency domain enhancement branches and multi-scale pooling branches, a first stitching layer, and a fifth convolutional module connected in sequence. The neck network is replaced with a cross-scale progressive feature fusion neck network, which employs a dual-domain cross-coupling recalibration module. This cross-scale progressive feature fusion neck network includes two upsampling modules, five dual-domain cross-coupling recalibration modules, five splicing layers, and six convolutional layers. The dual-domain cross-coupling recalibration module comprises a seventh convolutional module, a second channel separation unit, an eighth convolutional module, two cascaded dual-domain cross-coupling recalibration units, a sixth splicing layer, and a ninth convolutional module, connected sequentially. During training, a directional decoupling gradient modulation loss is used to constrain the bounding box regression results output by the detection head. This directional decoupling gradient modulation loss includes a complete intersection-and-merge term, a rigid aspect ratio prior term, and a directional decoupling gradient modulation mechanism. The prediction module is configured to acquire the laparoscopic surgical image to be detected and input it into the trained laparoscopic surgical instrument detection model. First, it performs feature extraction through the improved backbone network to obtain multi-scale image features. The multi-scale image features are then input into the cross-scale progressive feature fusion neck network for feature fusion to obtain cross-scale fused features. The cross-scale fused features are then input into the detection head to obtain the detection results of each surgical instrument in the laparoscopic surgical image to be detected.

8. An electronic device, comprising: One or more processors; A storage device for storing one or more programs, characterized in that, when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-6.

Images